Method and device for placing items

Abstract

The embodiments of the present disclosure disclose an object placement method and device. A specific implementation of the method includes: obtaining a depth map of the object placement area, and obtaining the size and the position of the object to be placed; determining a depth threshold according to the size, and binarizing the depth map using the depth threshold to obtain a binary depth map, wherein the depth threshold is greater than the distance between the position to be placed and the object placement area; determining the overlapping edges included in the projection area of ​​the object to be placed, wherein the projection area is formed by projecting the object to be placed to the object placement area when the object to be placed is located at the position to be placed, and the corresponding edges of the overlapping edges in the binary depth map include target pixels, and the target pixels are pixels with values ​​greater than the depth threshold; determining the placement path of the object to be placed according to the overlapping edges, and placing the object to be placed in the object placement area according to the placement path and the position to be placed. This implementation realizes the convenient determination of the placement path.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a method and device for placing items. Background Art

[0002] In the fields of logistics and warehousing, sorting and palletizing goods is one of the more time-consuming processes. As the scale of logistics and warehousing gradually expands, the sorting and palletizing of large quantities of goods usually requires various automated devices to assist in completing in order to reduce labor costs. The existing automated palletizing process usually uses a preset palletizing algorithm to calculate the placement coordinates and placement path of the goods (such as the rotation direction and entry path relative to the destination pallet, etc.), and then grabs the goods one by one from the source trailer or conveyor belt according to the placement path to the placement position of the destination trailer or conveyor belt.

[0003] A relatively simple placement path is the straight up and straight down type, that is, stacking the goods from a higher place to a designated position along the vertical direction. However, this method requires that there should be no obstructions on the placement path. Another more flexible placement path is the oblique insertion type, which can flexibly choose to complete the placement of goods along various directions. Summary of the invention

[0004] The embodiments of the present disclosure provide a method and device for placing items.

[0005] In a first aspect, an embodiment of the present disclosure provides an object placement method, the method comprising: obtaining a depth map of an object placement area, and obtaining the size and the position of the object to be placed; determining a depth threshold according to the size, and binarizing the depth map using the depth threshold to obtain a binary depth map, wherein the depth threshold is greater than the distance between the position to be placed and the object placement area; determining overlapping edges included in a projection area of ​​the object to be placed, wherein the projection area is formed by projecting the object to be placed onto the object placement area when the object to be placed is at the position to be placed, and the corresponding edges of the overlapping edges in the binary depth map include target pixel points, and the target pixel points are pixel points having a value greater than the depth threshold; determining a placement path of the object to be placed according to the overlapping edges, and placing the object to be placed in the object placement area according to the placement path and the position to be placed.

[0006] In a second aspect, an embodiment of the present disclosure provides an object placement device, the device comprising: an acquisition unit, configured to acquire a depth map of an object placement area, and acquire the size and the position to be placed of an object to be placed; an image processing unit, configured to determine a depth threshold according to the size, and to binarize the depth map using the depth threshold to obtain a binary depth map, wherein the depth threshold is greater than the distance between the position to be placed and the object placement area; an overlapping edge determination unit, configured to determine an overlapping edge included in a projection area of ​​the object to be placed, wherein the projection area is formed by projecting the object to be placed onto the object placement area when the object to be placed is at the position to be placed, and the corresponding edge of the overlapping edge in the binary depth map includes a target pixel point, and the target pixel point is a pixel point having a value greater than the depth threshold; a placement unit, configured to determine a placement path of the object to be placed according to the overlapping edge, and to place the object to be placed in the object placement area according to the placement path and the position to be placed.

[0007] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0008] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0009] The article placement method and device provided by the embodiments of the present disclosure use a depth map of the cargo article placement area, and binarize the depth map based on the size of the article to be placed, then determine that the projection area formed by the projection of the article to be placed onto the article placement area corresponds to the overlapping edges of the pixels with larger values ​​in the binarized depth map, and then determine the placement path of the article to be placed according to the overlapping edges, and then can place the article to be placed at the corresponding position in the article placement area according to the to-be-placed position of the article to be placed and the determined placement path, thereby, on the basis of the known article placement area and the to-be-placed position of the article to be placed, conveniently determine the placement path of the article to be placed, and can be applicable to the placement path determination in various scenarios, with high applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Other features, objects and advantages of the present disclosure will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0011] Figure 1 is an exemplary system architecture diagram in which an embodiment of the present disclosure may be applied;

[0012] Figure 2 is a flow chart of an embodiment of an object placement method according to the present disclosure;

[0013] Figure 3a-3c is a schematic diagram of an embodiment of generating a depth map of an object placement area;

[0014] Figure 4 is a schematic diagram of an embodiment of determining overlapping edges;

[0015] Figure 5 is a schematic diagram of an embodiment of the vertical direction of each edge of the projection area;

[0016] Figure 6 is a structural schematic diagram of an embodiment of an article placement device according to the present disclosure;

[0017] Figure 7 It is a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure. DETAILED DESCRIPTION

[0018] The present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0019] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0020] Figure 1 An exemplary architecture 100 is shown to which an embodiment of an object placement method or an object placement device of the present disclosure can be applied.

[0021] like Figure 1 As shown, the system architecture 100 may include a tool 101 for performing item placement and a server 102. The tool 101 for performing item placement and the server 102 may be connected to each other by various wired or wireless communication links or optical fiber cables.

[0022] The tool 101 for performing item placement can be any tool capable of performing item placement, such as a robot, a mechanical arm, a palletizer, etc. The server 102 can be a server that provides various services. For example, the server 102 can be a server that provides management and control for item placement. The server 102 can determine the placement path of the item to be placed, and then send the placement path to the tool 101 for performing item placement, so as to control the tool 101 for performing item placement to complete the placement of the item to be placed according to the received placement path.

[0023] It should be noted that the object placement method provided in the embodiments of the present disclosure is generally executed by the server 102 , and accordingly, the object placement device is generally disposed in the server 102 .

[0024] It should be noted that the tool 101 for performing item placement may also be provided with a control center such as a processor, which may determine the placement path of the item to be placed, and control the tool 101 for performing item placement to complete the placement of the item to be placed according to the received placement path. In this case, the item placement method provided by the embodiment of the present disclosure may also be performed by the tool 101 for performing item placement, and accordingly, the item placement device may also be provided in the tool 101 for performing item placement. In this case, the exemplary system architecture 100 may not have the server 102.

[0025] It should be noted that the server 102 can be hardware or software. When the server 102 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server 102 is software, it can be implemented as multiple software or software modules (for example, multiple software or software modules for providing distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0026] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0027] Continue to refer Figure 2 , which shows a process 200 of an embodiment of an object placement method according to the present disclosure. The object placement method comprises the following steps:

[0028] Step 201, obtaining a depth map of the object placement area, and obtaining the size and placement position of the object to be placed.

[0029] In this embodiment, the item placement area can be used to place items. The items can be various items, which can be set according to the actual application scenario. As an example, the items can be various goods, such as goods included in an order on an e-commerce platform. The item placement area can be flexibly set according to the actual application scenario. For example, the item placement area can be a pre-designated space, such as the storage space of a designated cargo box.

[0030] The depth map of the item placement area can be used to represent the height of each position in the item placement area. Specifically, the value of a pixel in the depth map can represent the height of the position corresponding to the pixel in the item placement area. It should be noted that the item placement area can usually be a three-dimensional space, so the height can refer to the height in a direction perpendicular to the horizontal plane.

[0031] The item to be placed may be any item to be placed in the item placement area. The size of the item to be placed may be used to describe the volume of the item to be placed. The position to be placed of the item to be placed may refer to the position where the item to be placed is to be placed in the item placement area. Among them, the size and position to be placed of the item to be placed may be represented in various ways according to specific application requirements. For example, the size of the item to be placed may be represented by the length, width and height of the item to be placed. The position to be placed of the item to be placed may be represented by the vertex coordinates (such as the coordinates of the lower left vertex) when the item to be placed is placed in the item placement area.

[0032] The execution entity of the item placement method (such as Figure 1 The server 102 shown in the figure can obtain the depth map of the object placement area, the size of the object to be placed, and the position to be placed from the local and / or other storage devices, etc. The depth map of the object placement area, the size of the object to be placed, and the position to be placed can be determined in advance using various methods.

[0033] For example, the depth map of the object placement area can be obtained by photographing the object placement area using a depth camera device (such as a depth camera, etc.). The size of the object to be placed can be obtained by measuring the object to be placed. The position to be placed of the object to be placed can be pre-specified or calculated by a pre-set algorithm for determining the position to be placed, which can be determined according to the actual application scenario.

[0034] Step 202: Determine a depth threshold according to the size of the object to be placed, and use the depth threshold to perform binarization processing on the depth map of the object placement area to obtain a binarized depth map.

[0035] In this embodiment, the depth threshold may be greater than the distance between the position to be placed of the item to be placed and the item placement area. The distance between the position to be placed of the item to be placed and the item placement area may be flexibly determined by various methods according to actual application scenarios. For example, the distance between the coordinates of a point on the item to be placed (such as the lowest point, etc.) and the coordinates of a point on the item placement area (such as the highest point, etc.) may be determined as the distance between the position to be placed of the item to be placed and the item placement area.

[0036] Specifically, the depth threshold can be determined by various methods according to actual application scenarios. For example, the corresponding relationship between the size and the depth threshold can be preset, and in this case, the depth threshold corresponding to the size of the object to be placed can be directly queried.

[0037] After obtaining the depth threshold, the depth threshold can be used to binarize the depth map of the area where the item is placed, and the binarization result can be determined as a binary depth map. Generally, based on the relationship between the values ​​of each pixel in the depth map and the depth threshold, each pixel in the depth map can be divided into two categories of pixels, so as to achieve binarization of the depth map. For example, if the value of one category of pixels is greater than the depth threshold, the value of this category of pixels can be updated to "1", and if the value of another category of pixels is not greater than the depth threshold, the value of this category of pixels can be updated to "0", as shown in the following formula:

[0038]

[0039] Wherein, binary_img(i, j) represents the value of the pixel (i, j) in the binary depth map, depth_img(i, j) represents the value of the pixel (i, j) in the depth map before binarization. "thresh" represents the depth threshold.

[0040] Step 203: Determine the overlapping edges included in the projection area where the object is to be placed.

[0041] In this embodiment, the projection area of ​​the item to be placed can be formed by projecting the item to be placed onto the item placement area when the item to be placed is located at the position to be placed. The corresponding edge of the overlapping edge in the binary depth map may include a target pixel point. The target pixel point may refer to a pixel point whose value is greater than the above-mentioned depth threshold. Since the projection area is a projection onto the item placement area, the projection area may have a corresponding area in the binary depth map, and the corresponding area and the projection area correspond to the same position in the item placement area.

[0042] Specifically, for each edge included in the projection area where the object is to be placed, the corresponding edge of the edge in the binary depth map may be determined first, and then it may be determined whether the corresponding edge includes the target pixel point. If so, the edge may be determined to be an overlapping edge.

[0043] Step 204: determine a placement path for the object to be placed according to the overlapping edges, and place the object to be placed in the object placement area according to the placement path and the position to be placed.

[0044] In this embodiment, the placement path can be used to complete the placement of the object to be placed. Since the overlapping edge is an edge including the target pixel point, and the height of the object placement area indicated by the target pixel point is greater than the depth threshold, the overlapping edge may hinder the placement of the object to be placed. Based on this, after determining the overlapping edge, various methods can be used to determine the placement path of the object to be placed according to the overlapping edge to avoid the overlapping edge from hindering the placement process of the object to be placed. For example, several placement paths can be pre-set, and in this case, a placement path that does not touch the overlapping edge can be selected from the preset placement paths as the placement path of the object to be placed.

[0045] After the placement path is determined, the items to be placed can be placed at the positions to be placed in the item placement area according to the placement path. The placement of the items to be placed can be completed by the execution subject, or by the execution subject controlling other objects.

[0046] For example, after determining the placement path, the executor subject may send the determined placement path to a tool for executing item placement (such as Figure 1 101 etc. shown in the figure) to control the tool for performing the object placement to complete the placement of the object to be placed according to the placement path and the position to be placed, that is, to place the object to be placed at the position to be placed in the object placement area.

[0047] It should be noted that the placement path determined can be specifically applied in combination with actual application scenarios. For example, in some application scenarios, the size of the items to be placed can be obtained by using a camera or other device, and then the position to be placed of each item to be placed in the item placement area can be calculated using a preset stacking algorithm, so that the placement path of the items to be placed can be determined by the above method provided in the embodiment of the present application. A robotic arm or other device can first grab the items to be placed according to the grabbing position of the items to be placed determined by using a camera or other device, and then place the grabbed items to be placed at their corresponding positions to be placed according to the determined placement path.

[0048] The depth map of the item placement area is binarized using a depth threshold determined based on the size of the item to be placed, and based on the corresponding area of ​​the projection area formed by the projection of the item to be placed onto the item placement area on the binarized depth map, the edges corresponding to the projection area including pixels with larger values ​​are determined as overlapping edges, so that the current item placement situation in the item placement area can be understood based on the determined overlapping edges and non-overlapping edges, and the placement path of the item to be placed can be reasonably planned based on these overlapping edges, so that the placement path of the item to be placed is adapted to the current item placement situation in the item placement area, and the reasonable placement of the item to be placed is achieved. Moreover, this method only involves relatively simple operations such as binary image processing and image projection processing, which is relatively simple and low-cost, and has fewer restrictions on the specific application scenarios of item placement and is highly applicable.

[0049] In some optional implementations of this embodiment, the item placement area may include an item placement tool. The item placement tool may be used to place items. For example, the item placement tool includes a shelf, a cargo box, a tray, a pallet, etc. In this case, the item placement area may be formed by an area provided by the item placement tool where items can be placed.

[0050] In this case, you can obtain the depth map of the area where the object is placed by following the steps below:

[0051] Step 1: Determine the depth map of the object placement tool.

[0052] In this step, the depth map of the item placement tool can be used to represent the height of each position of the item placement tool. Specifically, the value of a pixel point in the depth map can represent the height of the position corresponding to the pixel point in the item placement tool. The depth map of the item placement tool can be determined in advance according to the size of the item placement tool.

[0053] Step 2: In response to determining that no object is placed on the object placement tool, determine the depth map of the object placement tool as the depth map of the object placement area.

[0054] In this step, it can be determined whether an object is currently placed on the object placement tool. If no object is currently placed on the object placement tool, the depth map of the object placement tool can be directly determined as the depth map of the object placement area.

[0055] Step three: in response to determining that an object is placed on the object placement tool, obtaining the size and placement position of the object placed on the object placement tool, and generating a depth map of the object placement area according to the size and placement position of the placed object.

[0056] In this step, if there are items placed on the item placement tool, the size and placement position of each item placed on the item placement tool can be obtained first. The placement position can refer to the position where the placed item is placed on the item placement tool. Specifically, the size and placement position of the item placed on the item placement tool can be obtained from a local or other storage device. The size and placement position of the placed item can be determined in advance using various methods. For example, the size of the placed item can be obtained by measuring the placed item. The placement position of the placed item can be recorded when the item is placed.

[0057] After obtaining the size and placement position of the placed object, the depth map of the object placement tool can be further updated to generate a depth map of the object placement area. Specifically, for the pixel point corresponding to the placement position of the placed object in the depth map, the value of the pixel point can be updated so that the updated pixel point value can represent the height of the object placement tool plus the object placed thereon.

[0058] As an example, see further Figure 3a-3c , which shows a schematic diagram of an embodiment of generating a depth map of an object placement area. Figure 3a , assuming that the length of the item placement area (such as the loading area of ​​a tray) is "l" and the width is "w", the pixels in the depth map of the item placement area can be initialized to "0". Specifically, the length direction of the item placement area can be used as the "x" axis, and the width direction of the item placement area can be used as the "y" axis. The initialized depth map of the item placement area has "l" pixels in the "x" axis direction and "w" pixels in the "y" axis direction, that is, the depth map matches the item placement area.

[0059] Assume that "n" items have been placed on the item placement area. Figure 3b As shown, the height direction of the item placement area is the "z" axis, then for the "i"th item that has been placed, the coordinates of the lower left corner of the item can be (x i ,y i , z i ) indicates the placement of the item, and the length of the item is "l i ", width is "w i ", and the height is "h i ” indicates the size of the item.

[0060] Then, for each placed object, the initialized depth map can be updated in turn using the following method: Figure 3c As shown, the first "y i ” to “y i +w i " and "x i ” to “xi +l i The values ​​of the pixels in the area formed by the column are updated to "z i +h i ”, where “z i " is the current value of the pixel when it is updated.

[0061] By using the size of the item placement area and the size and placement position of the items placed thereon, a depth map of the item placement area is generated. This makes it possible to obtain a depth map of the item placement area without installing devices such as depth cameras, so that the placement path based on the depth map can be determined in scenarios where it is inconvenient to install devices such as depth cameras.

[0062] In some optional implementations of this embodiment, the depth threshold may be determined based on the sum of the distance between the object to be placed and the object placement area and a preset offset. The preset offset may be predetermined based on a tool used to perform object placement (e.g., Figure 1 101, etc.). Specifically, the offset can be preset by a technician based on a tool for performing item placement to assist or facilitate the tool for performing item placement to complete the placement of the item to be placed. It can be seen that the depth thresholds corresponding to different items to be placed can be the same or different. The preset offsets corresponding to different items to be placed can be the same.

[0063] The depth threshold is determined based on the sum of an offset predetermined by a tool for performing item placement and a distance between the item to be placed and the item placement area. Binarization based on the depth threshold can more accurately distinguish whether the tool for performing item placement is convenient for placing the item, so that the ultimately determined placement path can match the tool for performing item placement.

[0064] In some optional implementations of this embodiment, the overlapping edges included in the projection area where the object is to be placed may be determined by the following steps:

[0065] Step 1: For each edge of the projection area, determine the number of target pixels included in the corresponding edge of the edge in the binary depth map.

[0066] In this step, the number of target image points included in each edge included in the projection area and corresponding to the corresponding edge in the binary depth map may be counted first.

[0067] Step 2: In response to determining that the number of target pixels included in the corresponding edge is greater than a preset number threshold, determine that the edge is an overlapping edge.

[0068] In this step, the preset number threshold can be preset by a technician according to an actual application scenario. Specifically, it can be determined whether the number of target pixels included in the corresponding edge of each edge is greater than the preset number threshold. If greater, it can be determined that the edge is an overlapping edge.

[0069] Step three: in response to determining that the number of target pixels included in the corresponding edge is not greater than a preset number threshold, determine that the edge is not an overlapping edge, that is, a non-overlapping edge.

[0070] By setting a preset number threshold and determining whether the edge is an overlapping edge by judging whether the total number of target pixels included in the corresponding edge of each edge of the projection area in the binary depth map is greater than the preset number threshold, the preset number threshold can be flexibly set according to the influence of various factors such as actual application scenarios and possible errors, thereby flexibly adjusting the judgment scale of overlapping edges, thereby further improving the flexibility and applicability of the placement path determination method.

[0071] Optionally, the projection area can be generated by the following steps:

[0072] Step 1: According to the size and placement position of the object to be placed, determine the projection area corresponding to the object to be placed as the initial projection area.

[0073] In this step, the initial projection area is a projection area formed by projecting the object placement area when the object to be placed is located at the position to be placed.

[0074] Step 2: Enlarge the initial projection area to obtain an enlarged projection area, and determine the enlarged projection area as the projection area for the object to be placed.

[0075] In this step, the projection area can be expanded to a certain extent, and the specific expansion range can be flexibly set by the technician according to the actual application scenario. For example, the expansion range can be determined based on historical object placement experience. For another example, the expansion range can be determined based on the size of the object to be placed and / or the size of the object placement area.

[0076] Since certain errors may exist in actual application scenarios, expanding the projection area and then determining the overlapping edges based on the expanded projection area helps to improve the accuracy of the placement path determined based on the overlapping edges and reduce the impact of errors on the determined placement path.

[0077] Optionally, the preset number threshold can be determined by various methods according to the expansion range of the initial projection area. For example, the corresponding relationship between the expansion range and the number threshold can be preset by a technician. In this case, the number threshold corresponding to the expansion range of the initial projection area can be queried as the preset number threshold. For another example, the preset number threshold can be set to be greater than the number of pixels corresponding to the expansion of the projection area.

[0078] By flexibly setting the preset number threshold according to the expansion range of the projection area to facilitate determination of the overlapping edge, the influence of the expansion of the projection area on the determined placement path can be reduced while reducing the influence of the error on the determined placement path.

[0079] As an example, see further Figure 4 , which shows a schematic diagram of an embodiment of determining overlapping edges. Figure 4 As shown, it is assumed that the pixels in the white area of ​​the binary depth map are the target pixels. It is assumed that the quadrilateral area composed of four sides "a", "b", "c" and "d" in the figure is the projection area after expanding by several pixels.

[0080] Then, a Boolean variable can be set for each of the four edges "a", "b", "c" and "d", and the initial value is set to "false" to indicate that the edge is a non-overlapping edge. For edge "a", the total number of target pixels among the pixels covered in the binary depth map can be counted. If the total number counted is greater than the preset number threshold, the Boolean value of edge "a" is updated to "true", indicating that the edge is an overlapping edge, that is, the edge of the original projection area before expansion corresponding to edge "a" is an overlapping edge. Correspondingly, if the total number counted is not greater than the preset number threshold, the Boolean value of edge "a" is kept as "false". The method for determining whether other edges "b", "c" and "d" are overlapping edges is the same as that of edge "a", which will not be repeated here.

[0081] In some optional implementations of this embodiment, the direction of the placement path of the object to be placed may be different from the vertical direction of the overlapping edge included in the projection area. The vertical direction of the overlapping edge may refer to the vertical direction of the direction of the overlapping edge. Specifically, the direction of the placement path of the object to be placed may be flexibly determined. For example, the angle between the direction of the placement path of the object to be placed and the vertical direction of the overlapping edge may be set as a preset angle, so that the direction of the determined placement path can avoid the vertical direction of the overlapping edge.

[0082] By controlling the direction of the placement path of the object to be placed to be different from the perpendicular direction of the overlapping edge, the placement path of the object to be placed can be prevented from being affected by the overlapping edge, which helps to improve the placement efficiency and success rate of the object to be placed.

[0083] Optionally, the placement path of the object to be placed can be determined by the following steps:

[0084] Step 1: Determine the sum of the unit vectors corresponding to each overlapping edge included in the projection area.

[0085] In this step, the unit vector may represent the vertical direction of the corresponding edge. Specifically, the unit vectors corresponding to the overlapping edges included in the projection area may be accumulated to obtain a sum. The sum obtained is generally also a vector.

[0086] As an example, Figure 5 As shown, it shows a schematic diagram of an embodiment of the vertical direction of each side of the projection area. The vertical direction of side "a" can be represented by a unit vector va = (-1, 0), the vertical direction of side "b" can be represented by a unit vector vb = (0, 1), the vertical direction of side "c" can be represented by a unit vector vc = (1, 0), and the vertical direction of side "b" can be represented by a unit vector vd = (0, -1).

[0087] Step 2: In response to determining that the obtained sum is not a zero vector, the obtained sum is normalized to obtain a normalized result, and the direction represented by the normalized result is determined as the direction of the placement path of the object to be placed.

[0088] In this step, if the obtained sum is not a zero vector, the obtained sum can be first normalized to obtain a unit vector corresponding to the sum, and then the direction indicated by the unit vector corresponding to the sum can be determined as the direction of the placement path of the object to be placed.

[0089] Specifically, various situations where the obtained sum is not a zero vector can be further subdivided to further determine the placement path of the items to be placed, which can be flexibly set according to the actual application scenario.

[0090] by Figure 5 As an example, if the projection area has only one overlapping edge, such as edge "a" is an overlapping edge and the other edges are non-overlapping edges, the direction of the placement path can be determined to be the direction represented by va=(-1, 0).

[0091] If the projection area has two overlapping edges, such as edges "a" and "b" are overlapping edges and the other edges are non-overlapping edges, the direction of the placement path can be determined to be a forty-five degree angle direction of va=(-1, 0) and vb=(0, 1).

[0092] If the projection area has three overlapping edges, such as edges "a", "b" and "c" are overlapping edges, and edge "c" is a non-overlapping edge, the direction of the placement path can be determined as the direction represented by vb=(0,1).

[0093] If the obtained sum is a zero vector, the direction of the placement path of the object to be placed can be determined according to the specific distribution of the vertical direction of the overlapping edge to avoid the influence of the overlapping edge on the placement of the object to be placed.

[0094] Still Figure 5 As an example, if "a", "b", "c" and "d" are all overlapping edges, the direction of the placement path can be determined to be straight up or straight down. If edges "a" and "c" are overlapping edges and the other edges are non-overlapping edges, the direction of the placement path can be determined to be parallel to the "y" axis. If edges "b" and "d" are overlapping edges and the other edges are non-overlapping edges, the direction of the placement path can be determined to be parallel to the "x" axis.

[0095] It should be noted that Figure 5 What is shown is only an example that the projection area is a quadrilateral. In actual applications, the shapes of objects can be various, so the shape of the projection area is not limited to a quadrilateral, and can be various shapes, which are determined according to the actual application scenario.

[0096] For each edge of the projection area, the sum is calculated using the unit vector representing the vertical direction of each edge, and based on various situations where the sum is zero or non-zero, the direction of the placement path is flexibly set, providing a relatively simple method for determining the direction of the placement path, avoiding the impact of overly complex calculations on real-time performance and cost.

[0097] Further references Figure 6 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an article placement device, which is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0098] like Figure 6As shown, the object placement device 600 provided in this embodiment includes an acquisition unit 601, an image processing unit 602, an overlapping edge determination unit 603 and a placement unit 604. The acquisition unit 601 is configured to acquire a depth map of the object placement area, and acquire the size and the position of the object to be placed; the image processing unit 602 is configured to determine a depth threshold according to the size, and to perform binarization processing on the depth map using the depth threshold to obtain a binarized depth map, wherein the depth threshold is greater than the distance between the position to be placed and the object placement area; the overlapping edge determination unit 603 is configured to determine the overlapping edges included in the projection area of ​​the object to be placed, wherein the projection area is formed by projecting the object to be placed to the object placement area when the object to be placed is located at the position to be placed, and the corresponding edge of the overlapping edge in the binarized depth map includes a target pixel point, and the target pixel point is a pixel point with a value greater than the depth threshold; the placement unit 604 is configured to determine the placement path of the object to be placed according to the overlapping edge, and place the object to be placed in the object placement area according to the placement path and the position to be placed.

[0099] In this embodiment, in the object placement device 600, the specific processing of the acquisition unit 601, the image processing unit 602, the overlapping edge determination unit 603 and the placement unit 604 and the technical effects thereof can be referred to respectively. Figure 2 The relevant descriptions of step 201, step 202, step 203 and step 204 in the corresponding embodiment are not repeated here.

[0100] In some optional implementations of this embodiment, the item placement area includes an item placement tool; and the acquisition unit 601 is further configured to: determine a depth map of the item placement tool; in response to determining that no item is placed on the item placement tool, determine the depth map of the item placement tool as the depth map of the item placement area; in response to determining that an item is placed on the item placement tool, acquire the size and placement position of the item placed on the item placement tool; and update the depth map of the item placement tool according to the size and placement position to obtain a depth map of the item placement area.

[0101] In some optional implementations of this embodiment, the depth threshold is determined according to the sum of the distance and a preset offset, wherein the preset offset is determined according to a tool used to perform object placement.

[0102] In some optional implementations of the present embodiment, the overlapping edge determination unit 603 is further configured to: for each edge of the projection area, determine the number of target pixel points included in the corresponding edge of the edge in the binary depth map; in response to determining that the number of target pixel points included in the corresponding edge is greater than a preset number threshold, determine that the edge is an overlapping edge; in response to determining that the number of target pixel points included in the corresponding edge is not greater than a preset number threshold, determine that the edge is not an overlapping edge.

[0103] In some optional implementations of the present embodiment, the projection area is generated through the following steps: determining the projection area corresponding to the object to be placed as the initial projection area according to the size and the position of the object to be placed; expanding the initial projection area, and determining the expanded projection area as the projection area of ​​the object to be placed; and the preset number threshold is determined according to the expansion range of the initial projection area.

[0104] In some optional implementations of this embodiment, the direction of the placement path of the object to be placed is different from the vertical direction of the overlapping edges included in the projection area.

[0105] In some optional implementations of the present embodiment, the placement unit 604 is further configured to: determine the sum of unit vectors corresponding to each overlapping edge included in the projection area, wherein the unit vector represents the vertical direction of the corresponding edge; in response to determining that the sum is not a zero vector, normalize the sum, and determine the direction represented by the normalized result as the direction of the placement path of the object to be placed.

[0106] The device provided by the above-mentioned embodiment of the present disclosure obtains the depth map of the object placement area through the acquisition unit, and obtains the size and the position to be placed of the object to be placed; the image processing unit determines the depth threshold according to the size, and uses the depth threshold to perform binarization processing on the depth map to obtain a binarized depth map, wherein the depth threshold is greater than the distance between the position to be placed and the object placement area; the overlapping edge determination unit determines the overlapping edges included in the projection area of ​​the object to be placed, wherein the projection area is formed by projecting the object to be placed to the object placement area when it is located at the position to be placed, and the corresponding edge of the overlapping edge in the binarized depth map includes a target pixel point, and the target pixel point is a pixel point with a value greater than the depth threshold; the placement unit determines the placement path of the object to be placed according to the overlapping edges, and places the object to be placed in the object placement area according to the placement path and the position to be placed. Therefore, the current item placement situation in the item placement area can be understood through the determined overlapping edges and non-overlapping edges, so that the placement path of the object to be placed can be reasonably planned based on these overlapping edges, so that the placement path of the object to be placed is adapted to the current item placement situation in the item placement area, and the reasonable placement of the object to be placed is achieved. Moreover, this method only involves relatively simple operations such as binary image processing and image projection processing, which is relatively simple and low-cost. It has fewer restrictions on the specific application scenarios of object placement and is highly applicable.

[0107] Reference below Figure 7 , which shows an electronic device (eg, Figure 1 A schematic diagram of the structure of the server 700. Figure 7 The server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0108] like Figure 7 As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0109] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The electronic device 700 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 7 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0110] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 709, or installed from a storage device 708, or installed from a ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0111] It should be noted that the computer-readable medium described in the embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0112] The computer-readable medium may be included in the electronic device; or it may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains a depth map of the object placement area, and obtains the size and the position of the object to be placed; determines a depth threshold according to the size, and uses the depth threshold to binarize the depth map to obtain a binarized depth map, wherein the depth threshold is greater than the distance between the position to be placed and the object placement area; determines the overlapping edges included in the projection area of ​​the object to be placed, wherein the projection area is formed by projecting the object to be placed to the object placement area when the object to be placed is at the position to be placed, and the corresponding edges of the overlapping edges in the binarized depth map include target pixels, and the target pixels are pixels with values ​​greater than the depth threshold; determines the placement path of the object to be placed according to the overlapping edges, and places the object to be placed in the object placement area according to the placement path and the position to be placed.

[0113] Computer program code for performing the operations of embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as a separate software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0114] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0115] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The units described may also be provided in a processor, for example, may be described as: a processor including an acquisition unit, an image processing unit, an overlapping edge determination unit, and a placement unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the acquisition unit may also be described as "a unit for acquiring a depth map of the object placement area, and acquiring the size and position of the object to be placed".

[0116] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) to form a technical solution.

Claims

1. A method for placing an object, comprising: Obtaining a depth map of the object placement area, and obtaining the size and placement position of the object to be placed, wherein the depth map is used to indicate the height of each position of the object placement area; Determining a depth threshold according to the size, and binarizing the depth map using the depth threshold to obtain a binarized depth map, wherein the depth threshold is greater than a distance between the position to be placed and the highest point of the object placement area; Determine an overlapping edge included in the projection area of ​​the object to be placed, wherein the projection area is formed by projecting the object to be placed to the object placement area when the object to be placed is located at the to-be-placed position, and a corresponding edge of the overlapping edge in the binary depth map includes a target pixel point, and the target pixel point is a pixel point having a value greater than the depth threshold; A placement path of the object to be placed is determined according to the overlapping edges, and the object to be placed is placed in the object placement area according to the placement path and the position to be placed.

2. The method according to claim 1, wherein: The object placement area includes an object placement tool; and The step of obtaining a depth map of the area where the object is placed includes: determining a depth map of the object placement tool; In response to determining that no object is placed on the object placement tool, determining the depth map of the object placement tool as the depth map of the object placement area; In response to determining that an object is placed on the object placement tool, the size and placement position of the object placed on the object placement tool are obtained; and according to the size and placement position, the depth map of the object placement tool is updated to obtain the depth map of the object placement area.

3. The method according to claim 1, wherein: The determining of the overlapping edges of the projection area of ​​the object to be placed includes: For each edge of the projection area, determine the number of target pixels included in the corresponding edge of the edge in the binary depth map; In response to determining that the number of target pixels included in the corresponding edge is greater than a preset number threshold, determining that the edge is an overlapping edge; In response to determining that the number of target pixels included in the corresponding edge is not greater than a preset number threshold, it is determined that the edge is not an overlapping edge.

4. The method according to claim 3, wherein: The projection area is generated by the following steps: According to the size and placement position of the object to be placed, determining a projection area corresponding to the object to be placed as an initial projection area; Expanding the initial projection area, and determining the expanded projection area as the projection area for the object to be placed; as well as The preset number threshold is determined according to the expansion range of the initial projection area.

5. The method according to any one of claims 1 to 4, wherein: The direction of the placement path of the object to be placed is different from the vertical direction of the overlapping edge in the plane where the projection area is located.

6. The method according to claim 5, wherein: The step of determining a placement path for the object to be placed according to the overlapping edges includes: Determine the sum of unit vectors corresponding to the overlapping edges included in the projection area, wherein the unit vector represents the vertical direction of the corresponding edge in the plane where the projection area is located; In response to determining that the sum is not a zero vector, the sum is normalized, and a direction represented by the normalized result is determined as a direction of a placement path for the to-be-placed object.

7. An article placement device, wherein: The device comprises: an acquisition unit configured to acquire a depth map of the object placement area and acquire the size and placement position of the object to be placed, wherein the depth map is used to indicate the height of each position of the object placement area; an image processing unit, configured to determine a depth threshold according to the size, and to perform binarization processing on the depth map using the depth threshold to obtain a binarized depth map, wherein the depth threshold is greater than a distance between the position to be placed and a highest point of the object placement area; an overlapping edge determining unit, configured to determine an overlapping edge included in the projection area of ​​the to-be-placed object, wherein the projection area is formed by projecting the to-be-placed object to the object placement area when the to-be-placed object is located at the to-be-placed position, and a corresponding edge of the overlapping edge in the binary depth map includes a target pixel point, and the target pixel point is a pixel point having a value greater than the depth threshold; The placement unit is configured to determine a placement path of the object to be placed according to the overlapping edges, and to place the object to be placed in the object placement area according to the placement path and the position to be placed.

8. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images