Method and apparatus for boxing

Abstract

Provided is a method for boxing, comprising: obtaining three-dimensional point cloud data of a boxing container, the three-dimensional point cloud data being obtained based on data obtained by a three-dimensional camera disposed above the boxing container; converting the three-dimensional point cloud data into a coordinate system of the boxing container to obtain three-dimensional point cloud data subjected to coordinate conversion; determining a first height map of the boxing container based on the three-dimensional point cloud data subjected to coordinate conversion, wherein the value of each pixel point in the first height map represents a depth value along the boxing depth direction of the boxing container; and using the first height map to determine a position at which a first item is to be loaded into the boxing container. This makes it possible to place the articles in a more appropriate position in the boxing container, avoiding collisions.

Description

Technical Field

[0001] The present invention relates to the field of logistics, in particular to packaging. Background Art

[0002] With the development of robotics technology, various types of robots (including robotic arms) are applied to various links in the logistics field to improve work efficiency.

[0003] In a warehouse packing environment, items of different sizes are transported on a conveyor belt. The sizes of these items are unknown. A camera is set above a certain position on the conveyor belt. When an item is transported by the conveyor belt to under the camera, the camera obtains an image of the item. Then the robot picks up the item and positions it at a certain position in the packing container based on the image of the item from the camera to complete the packing. This is called online packing. Summary of the invention

[0004] It is desirable to provide a method and apparatus for packing, which makes it possible to place items in a packing container in a more suitable position, avoiding collisions.

[0005] According to one aspect, a method for packing is provided, the method comprising: acquiring three-dimensional point cloud data of a packing container, the three-dimensional point cloud data being acquired based on data obtained by a three-dimensional camera arranged above the packing container; converting the three-dimensional point cloud data into a coordinate system of the packing container to obtain three-dimensional point cloud data after coordinate transformation; determining a first height map of the packing container based on the three-dimensional point cloud data after coordinate transformation, the value of each pixel point in the first height map representing a depth value along a packing depth direction of the packing container; and using the first height map to determine a position where a first item is to be packed into the packing container.

[0006] According to another aspect, a device for packing is provided, comprising: an acquisition unit, which is used to acquire three-dimensional point cloud data of a packing container, the three-dimensional point cloud data being acquired based on data obtained by a three-dimensional camera arranged above the packing container; and a processing unit, which is used to convert the three-dimensional point cloud data into a coordinate system of the packing container to obtain three-dimensional point cloud data after coordinate transformation; determine a first height map of the packing container based on the three-dimensional point cloud data after coordinate transformation, the value of each pixel point in the first height map representing a depth value along a packing depth direction of the packing container; and use the first height map to determine the position of a first object to be packed into the packing container.

[0007] According to another aspect, a system for box packing is provided, comprising: a three-dimensional camera disposed above a box packing container; and the device for box packing according to various embodiments of the present disclosure.

[0008] According to still another aspect, a computer readable medium is provided, storing a computer program, which, when executed by a processor, causes the processor to perform the method according to various embodiments of the present invention.

[0009] According to various embodiments of various aspects of the present invention, imaging detection is performed on the packing container. By acquiring three-dimensional point cloud data of the packing container and converting the three-dimensional point cloud data into the coordinate system of the packing container, and then performing projection along the depth direction of the packing container, a height map of the packing container can be obtained. Such a height map is used for subsequent packing, so that it is possible to determine the packing position based on the height map during the online packing process, and it is particularly possible to identify objects that are not placed at the expected position and determine the placement position of subsequent objects based on the actual position and orientation of the identified objects, thereby avoiding collisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Embodiments are illustrated by way of example only and not by way of limitation in the figures of the accompanying drawings in which like reference numerals refer to similar elements.

[0011] Figure 1 A flow chart of a method for packing boxes according to an embodiment of the present invention is shown;

[0012] Figure 2 shows a coordinate system of a box container according to one embodiment of the present invention;

[0013] Figure 3 shows a first height map according to one embodiment of the present invention;

[0014] Figure 4 A flow chart showing a method for determining a position where a first object is to be loaded into a packing container according to a first height map according to one embodiment of the present invention;

[0015] Figure 5 shows a first height map and a second height map according to one embodiment of the present invention;

[0016] Figure 6 A block diagram of a device for packaging according to an embodiment of the present invention is shown.

[0017] Various aspects and features of various embodiments of the present invention are described with reference to the above drawings. The above drawings are only illustrative and not restrictive. In addition, not all parts of the device of the embodiment of the present invention are marked with reference numerals in the above drawings. Only relevant parts are marked in some drawings, which does not limit the parts to those shown in the drawings of the specification. DETAILED DESCRIPTION

[0018] Although online packing technology has been applied, various types of errors may still occur during the online packing process, such as camera calibration errors, target recognition errors, posture recognition errors, robot picking position errors, and positioning errors of items in the packing container. All these errors may result in the items not being placed at the expected position, but ultimately being placed at the wrong position in the packing container or placed in the packing container with the wrong orientation, making the items in the packing container not compact enough and the space not reasonably utilized. In this case, if subsequent items are still placed based on the expected position, collisions between items or between items and the packing container may occur.

[0019] Based on the above cognition, the inventors of the present application have designed a method and apparatus for packing. This requires setting a three-dimensional camera above a packing container (such as a pallet, a box, etc.), and the three-dimensional camera is configured to image the entire interior of the packing container, thereby obtaining three-dimensional point cloud data of the packing container. By converting the three-dimensional point cloud data into the coordinate system of the packing container and performing a projection along the depth direction of the packing container, a height map of the packing container can be obtained, and the value of each pixel of the height map represents the depth value along the packing depth direction of the packing container. By obtaining the height map in the coordinate system of the packing container, it is possible to pack based on the height map during the online packing process, and it is particularly possible to identify items that are not placed at the expected position and determine the placement position of subsequent items based on the position and orientation where the identified items are actually placed, thereby avoiding collisions.

[0020] Figure 1 FIG. 1 is a flow chart of a method 100 for packing a box according to an embodiment of the present invention. Figure 1 As shown, according to the method 100, at step 110, the three-dimensional point cloud data of the packing container is obtained, and the three-dimensional point cloud data is based on the data obtained by the three-dimensional camera arranged above the packing container. On the one hand, the three-dimensional point cloud data from the three-dimensional camera can be directly obtained, and on the other hand, the three-dimensional point cloud data from the three-dimensional camera can also be pre-processed and used in the method of the present invention. For example, the three-dimensional point cloud data collected by the three-dimensional camera may include other areas besides the packing container. Therefore, the three-dimensional point cloud data from the three-dimensional camera can be pre-processed to set the area of ​​interest, thereby obtaining three-dimensional point cloud data containing only the packing container. In addition, the three-dimensional point cloud data from the three-dimensional camera can also be subjected to operations such as removing noise points (outlier points) and downsampling. The three-dimensional point cloud data obtained at step 110 is the pre-processed three-dimensional point cloud data.

[0021] At step 120, the three-dimensional point cloud data is converted into the coordinate system of the container to obtain the three-dimensional point cloud data after coordinate conversion. The three-dimensional point cloud data obtained by the three-dimensional camera arranged above the container is in the coordinate system of the three-dimensional camera. In order to facilitate subsequent applications, it is necessary to convert it into the coordinate system of the container. Any known coordinate conversion method can be applied to perform the conversion, and no limitation is made to the coordinate conversion method herein.

[0022] In one example, a 3D model of a boxed container can be obtained, such as a 3D CAD model of a boxed container constructed in a CAD system, and the following transformation matrix Tc is determined by aligning the three-dimensional point cloud data obtained by a three-dimensional camera with the 3D model of the boxed container and converting them into the coordinate system of the 3D model.

[0023]

[0024] Among them, R c and t c They are rotation and translation matrices respectively. They can be determined by the 3D model point set and the container point set in the 3D point cloud data Np and Nq are the number of points in the two point sets, respectively, and they may not be equal. Assuming that the container point set P in the 3D point cloud data has Euclidean motion (rotation vector R, translation vector t), the transformed point set can be expressed as

[0025]

[0026] The distance between the container point set P in the 3D point cloud data and the CAD model point set Q can be expressed as:

[0027]

[0028] By minimizing the above distance, we can obtain the corresponding R and t as the rotation matrix and translation matrix, thereby obtaining the transformation matrix Tc. Then, based on the transformation matrix Tc, each point in the three-dimensional point cloud data is transformed into the coordinate system of the container. In the coordinate system of the container, Figure 2 As shown, the Z direction is along the boxing depth of the boxing container, while the X and Y directions are in planes parallel to the boxing plane of the boxing container.

[0029] Specifically, suppose that the coordinates of a point in the 3D point cloud data in the 3D camera coordinate system are (x c ,y c , z c ), then the corresponding point (x, y, z) of the point in the coordinate system of the packing container can be determined by the following coordinate transformation:

[0030]

[0031] Wherein, K is the intrinsic parameter of the 3D camera.

[0032] At step 130 , a first height map of the packing container is determined based on the coordinate-converted three-dimensional point cloud data, wherein the value of each pixel in the height map represents a depth value along a packing depth direction (ie, Z direction) of the packing container.

[0033] Specifically, the first height map can be obtained by gridding the coordinate-converted three-dimensional point cloud data in the OXY plane and making the value of each cell in the grid correspond to the Z coordinate value (depth value) of the corresponding three-dimensional point cloud data. For example, assuming that the X coordinate and Y coordinate discrete resolutions of the grid are dx and dy, the position (x h ,y h ) can be expressed as:

[0034]

[0035] The pixel value at the corresponding pixel position in the height map can be the z value at the corresponding point in the three-dimensional point cloud data after coordinate transformation (i.e., the depth value in the box container). Figure 3 A first height map obtained according to one embodiment of the present invention is shown.

[0036] The vector in the above formula and vector The "1" in the matrix The last row and column of are only added to indicate that this is the processing of 3D point cloud data, and the above-mentioned added content can also be deleted in actual calculation.

[0037] At step 140, the first height map is used to determine the position of the first object to be loaded into the container. The first object is the object to be loaded into the container. If there are already objects loaded into the container before the first object is loaded, these already loaded objects may be referred to as second objects. These second objects may be shown in the first height map, and the heights of the areas where the second objects have been loaded may be different, which may be indicated in the first height map. For example, Figure 3 As shown, the dark area represents a region with a large depth, and the light area represents a region with a small depth because the second object or a higher second object is loaded. This is not restrictive, and the dark area can also be used to represent a region with a small depth, while the light area represents a region with a large depth.

[0038] In one embodiment, the position and / or orientation of one or more second items that have been loaded into the packing container can be determined based on the first height map, and the determined position and / or orientation of the one or more second items are then used to determine the position of the first item to be loaded into the packing container.

[0039] The first height map obtained above can represent the placement (position and / or orientation) of the items in the current packing container. Obtaining this information is helpful in determining how subsequent items are loaded into the packing container, thereby possibly avoiding collisions between items and between items and the packing container.

[0040] Figure 4 FIG. 1 is a flowchart showing a method 140 for determining the position of a first object to be loaded into a container using the first height map obtained above according to an embodiment of the present invention. Figure 1 Step 140 of method 100 for boxing is shown.

[0041] like Figure 4 As shown, at step 142, a first height map of the current container is received, and the orientation of one or more second objects already loaded into the container is determined. The first height map received here may be a first height map after preprocessing (e.g., dilation and erosion), and thus may be different from the first height map determined in step 130 of method 100.

[0042] Different image processing methods can be used to determine the orientation of the second object in the packing container based on the received first height map. In one example, a region growing method can be used to determine the region corresponding to the second object in the first height map, thereby determining the corresponding boundary of the second object, and determining the orientation of the second object based on the determined boundary of the second object.

[0043] In an example, Figure 3 As shown, the second object has a rectangular shape (this includes the shape of a cube), such as a rectangular box, which should be displayed as a rectangle (this includes a square) in the first height map, but due to factors such as noise, collision and reflection, it may not be displayed as a standard rectangle. To this end, a region growing method can be used to determine the regions corresponding to these rectangles to determine their boundaries, thereby adapting them to standard rectangles with the same shape as the second object, and in particular, the orientation of the placement of these objects can be determined. Although the second object in the shape of a rectangular parallelepiped is described as an example, this is not restrictive, and the second object can also be of other shapes, such as a triangular pyramid (correspondingly displayed as a triangle in the first height map) or a sphere (correspondingly displayed as a circle in the first height map).

[0044] At step 144, it is determined whether the determined orientation conforms to a predetermined orientation. Specifically, it can be determined whether the orientation of the second article that has been loaded into the packing container conforms to a predetermined orientation. The predetermined orientation can correspond to a predetermined arrangement direction of the second article in the packing container.

[0045] Generally, it is expected that the items are arranged in a packing container in a predetermined orientation so as to satisfy the requirement of placing as many items as possible in the same packing container (packing problem). By determining the orientation of the second item through the above steps 142 and 144, and comparing the determined orientation with the predetermined orientation, it is possible to find the second items that are not placed in the predetermined orientation. Figure 3 As shown, the second object B is not placed in the predetermined orientation. Figure 3 In the figure, each rectangular second object is placed along the X and Y coordinate axes of the packing container, while the second object B is placed diagonally and not along the X and Y coordinate axes of the packing container.

[0046] If it is determined at step 144 that the orientation of at least one second object does not conform to the predetermined orientation (i.e. Figure 4 ), then, at step 146, the area in the first height map corresponding to at least one second object that is not placed in the predetermined orientation is replaced by a corresponding area of ​​the predetermined shape set in the predetermined orientation to completely cover the area of ​​the corresponding second object, thereby obtaining a second height map.

[0047] Specifically, at step 144, it is determined that Figure 3 After the second object B is not placed in the predetermined orientation, it can be placed in a predetermined shape (such as Figure 3 The case shown is a rectangle) covers the area corresponding to the second object B, thereby obtaining a second height map. In one example, the area covering the second object can be realized by an axis-aligned bounding box (AABB) algorithm based on the area corresponding to the second object in the first height map. It is usually desired to find the minimum area with a predetermined orientation that can cover the area of ​​the second object to cover it. Figure 5 , a first height map and a second height map corresponding to each other are shown, wherein a rectangular area placed according to a predetermined orientation covers an original area that is not placed according to the predetermined orientation.

[0048] After the second height map is obtained at step 146, the second height map can be used to determine the placement position of the subsequent first item in the packing container. Usually, the first item has the same shape as the second item. By using a larger area to completely cover the area corresponding to the items not placed in the predetermined orientation to obtain the second height map, and using the second height map to determine the placement position of the subsequent first item, it is possible to avoid the subsequent loaded items from colliding with the already loaded items. In other words, by determining the packing position of the subsequent first item based on such a second height map, even if the previously loaded items are tilted, the subsequently loaded items will not collide with them.

[0049] If it is determined at step 144 that the orientation of the second object meets the predetermined orientation (i.e. Figure 4 If the first height map is “Y” in the figure, no processing is required and the first height map can be directly used to determine the placement position of the subsequent first object in the packing container at step 150.

[0050] Although the process of deriving the second height map based on the first height map is described with reference to the above steps 142-150, it can be understood that this is not restrictive, and corresponding processing can be added or certain processing can be deleted according to the needs of the application scenario to achieve the corresponding effect.

[0051] The following is a brief description of the processing in a scenario. There may be multiple groups of pixels in the first height map, and each group of pixels has a unique corresponding pixel value (i.e., depth value), because there may be gaps between multiple items loaded in the container. First, one pixel in each group of pixels is added to the seed list as a seed point, and regional growth is performed based on the seed point from the seed list, thereby obtaining a region grown by the region. Then, the boundary size of the region grown by the region is compared with the predetermined size. If it is less than the predetermined size, it means that the region is placed according to the predetermined orientation, and further processing of the region is abandoned; on the contrary, if the boundary size of the region grown by the region is greater than the predetermined size, it means that the region is not placed according to the predetermined orientation, and the boundary of the region can be further detected. If the boundary of the region is not closed, further processing of the region can also be abandoned; if the boundary of the region is closed, the number of its vertices can be further determined. For the case of rectangular items, it can be determined whether the vertices are 4. If so, it can be covered by the AABB algorithm using a predetermined orientation and a predetermined rectangular shape area. On the contrary, if the number of vertices is not 4, a rectangular area can be fitted with lines. After the above process is performed on each seed point in the seed list, a second height map can be obtained.

[0052] In addition, in one example, the first height map can be processed in step 142 to remove noise and / or increase its contrast. For example, the first height map can be processed by a dilation and erosion algorithm, such as Figure 3 The first height map shown is generated after this processing Figure 5 (a) can fill Figure 3 Then, a region growing method is performed on the processed first height map to extract the region corresponding to the second object.

[0053] In addition, although it is described that the method first determines whether the second object is not placed in a predetermined orientation, and then covers / replaces the area when the second object is determined to be not placed in the predetermined orientation, this is not restrictive. It is also possible not to perform the above determination, but to perform covering / replacement on all areas corresponding to the second objects via the AABB algorithm.

[0054] Although referring to the above Figure 1-5 A method for packing according to an embodiment of the present invention has been described. It will be appreciated that this is not restrictive and that it is contemplated that one or more steps in the method may be modified / changed to achieve a corresponding effect.

[0055] In one example, the size requirements for the first height map may be different for different tasks. For example, reinforcement learning has an upper limit on the size requirements for the first height map, which will increase the workload of image processing when the image is large. In particular, when the first height map is used for machine learning tasks such as determining the location to be boxed, a large-size image means a significantly increased computational cost. Currently, the size of the image can be reduced by downsampling the image, but this will amplify the noise and blur the image content.

[0056] In one example, scaling each area in the first height map based on a predetermined scaling ratio includes scaling the area corresponding to one or more second objects in the first height map based on the predetermined scaling ratio. Figure 3 The height map shown is used to determine whether the second object is placed in a predetermined orientation. The size of the scaled first height map is reduced according to a predetermined ratio without amplifying noise or blurring image content. The position of the first object to be loaded into the box container can still be accurately determined based on the scaled first height map.

[0057] In another example, there may be the same requirements for the size of the second height map as the first height map. Figure 4In the illustrated embodiment, when it is determined that the second height map is determined when the second object is not placed in a predetermined orientation, the second height map can also be scaled based on a predetermined scaling ratio, which includes scaling the area corresponding to one or more second objects in the second height map based on the predetermined scaling ratio, and then using the scaled second height map to determine the position of the first object to be loaded into the packing container. The second height map thus scaled is reduced in size, and at the same time, noise is not amplified and image content is not blurred. Based on the second height map thus scaled, it can still be used to accurately determine the position of subsequent objects to be loaded when the objects are not placed in a predetermined orientation (tilted placement), thereby avoiding collisions. Although the first object and the second object are described above with reference to a rectangular shape, it can be understood that this is not restrictive, and first objects and second objects of other shapes can also be expected, and even the shapes of the first object and the second object can be different.

[0058] Figure 6 FIG. 1 shows a block diagram of an apparatus 10 for packaging according to an embodiment of the present invention. Figure 6 As shown, the device 10 includes an acquisition unit 11, which is used to acquire various types of data, especially for acquiring three-dimensional point cloud data of a container, the three-dimensional point cloud data is based on data obtained by a three-dimensional camera arranged above the container. In one example, the acquisition unit can also process the data, for example, it can pre-process the three-dimensional point cloud data from the three-dimensional camera to remove noise (including removing uninteresting areas) or increase contrast.

[0059] The device 10 further comprises a processing unit 12, which is used to perform processing operations according to various embodiments of the present invention, and which may be one or more units with processing capabilities.

[0060] Specifically, the processing unit is used to convert the acquired three-dimensional point cloud data into the coordinate system of the packing container to obtain the three-dimensional point cloud data after coordinate transformation; determine the first height map of the packing container based on the three-dimensional point cloud data after coordinate transformation, and the value of each pixel point in the first height map represents the depth value along the packing depth direction of the packing container; finally, the first height map is used to determine the position where the first object is to be packed into the packing container. It can be understood that the processing unit may not directly perform the operation of determining the position where the first object is to be packed based on the first height map, but only send the first height map to the corresponding unit to perform the above-mentioned operation of determining the position where the first object is to be packed.

[0061] In one embodiment, the processing unit 12 is also used to determine the position and / or orientation of one or more second items that have been loaded into the packing container based on the first height map; and then use the position and / or orientation of the one or more second items to determine the position of subsequent first items to be loaded into the packing container.

[0062] In one embodiment, the processing unit 12 is also used to determine whether the orientation of the second object conforms to a predetermined orientation; if it is determined that the orientation of at least one of the one or more second objects does not conform to the predetermined orientation, the area corresponding to the at least one second object in the first height map is replaced with a corresponding area of ​​a predetermined shape set at a predetermined orientation to completely cover the corresponding second object, thereby obtaining a second height map; and then the second height map is used to determine the position where the first object is to be loaded into the packing container.

[0063] In a further embodiment, the processing unit 12 is further used to extract an area corresponding to at least one second object whose placement orientation does not conform to a predetermined orientation in the first height map; and determine what kind of area covers the area of ​​the second object through an axis-aligned bounding box algorithm based on the area corresponding to the at least one second object.

[0064] In one embodiment, the processing unit 12 is further configured to scale each area in the second height map based on a predetermined scaling ratio, including scaling the area corresponding to the one or more second objects in the second height map based on the predetermined scaling ratio; thereby reducing the size of the second height map without introducing noise to blur image information. The scaled second height map is used to determine the position where the first object is to be loaded into the box container, which will reduce the amount of subsequent calculations, thereby reducing the calculation cost, while ensuring the quality of the image, and thus may improve the accuracy of determining the placement position of the subsequent first object.

[0065] References Figure 1-6 Various embodiments of the method and device of the present invention are described, and these embodiments can be combined with each other to obtain different effects without being limited by the subject type. Furthermore, the various units / steps / processes mentioned above are not restrictive, and the functions of the various units / steps / processes mentioned above can be merged / combined / changed / modified to obtain corresponding effects.

[0066] The functions of each unit of the method and device for packing of the present invention can be implemented by software or corresponding hardware, or by means of a processor, for example, the processor can read the computer program in the memory and run the computer program to implement the functions of the above-mentioned units. In one embodiment, the above-mentioned device of the present invention can also be implemented by a memory and a processor.

[0067] It can be understood that the methods of various embodiments of the present disclosure can be implemented by computer programs / software. These software can be loaded into the working memory of the processor and used to execute the methods according to various embodiments of the present disclosure when running.

[0068] It is understood that the method and apparatus according to the present invention can be performed by any device with processing capabilities, and can also be performed at a remote location. The exemplary embodiments of the present disclosure cover both: creating / using the computer program / software of the present disclosure from the beginning, and converting an existing program / software to use the computer program / software of the present disclosure by means of an update.

[0069] According to another embodiment of the present disclosure, a computer program product is provided, such as a machine (e.g., computer) readable medium, such as a CD-ROM, including a computer program code, which, when executed, causes a computer or a processor to perform a method according to each embodiment of the present disclosure. The machine readable medium is, for example, an optical storage medium or a solid-state medium supplied with or as part of other hardware.

[0070] According to another embodiment of the present disclosure, a system for boxing is provided, the system comprising a three-dimensional camera disposed above a boxing container and the device according to various embodiments of the present disclosure.

[0071] The above describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0072] The present disclosure is described above with reference to specific embodiments. It should be understood by those skilled in the art that the technical solution of the present disclosure can be implemented in various ways without departing from the spirit and basic features of the present disclosure. The specific embodiments are merely illustrative and not restrictive. In addition, these embodiments can be arbitrarily combined to achieve the purpose of the present disclosure. The scope of protection of the present disclosure is defined by the attached claims.

[0073] The word "comprising" in the specification and claims does not exclude the existence of other elements or steps. The expressions "first", "second", "third" and the order of the steps shown in the figures do not limit their order or quantity. The functions of the various elements described in the specification or recorded in the claims can also be split or combined and implemented by corresponding multiple elements or a single element.

Claims

1. A method for packing, comprising: Acquire three-dimensional point cloud data of the packing container, wherein the three-dimensional point cloud data is acquired based on data obtained by a three-dimensional camera disposed above the packing container; Converting the three-dimensional point cloud data into the coordinate system of the packing container to obtain three-dimensional point cloud data after coordinate conversion; Determine a first height map of the packing container based on the three-dimensional point cloud data after coordinate transformation, wherein the value of each pixel point in the first height map represents a depth value along a packing depth direction of the packing container; and The first height map is used to determine a position at which a first item is to be loaded into the box container.

2. The method according to claim 1, further comprising: determining an orientation of one or more second items that have been loaded into the box container based on the first height map; and The orientation of the one or more second articles is used to determine a position in which the first article is to be loaded into the box container.

3. The method according to claim 2, further comprising: determining whether the orientation meets a predetermined orientation; If it is determined that the orientation of at least one of the one or more second objects does not conform to the predetermined orientation, replacing the area corresponding to the at least one second object in the first height map with a corresponding area having a predetermined shape arranged in the predetermined orientation to completely cover the corresponding second object, thereby obtaining a second height map; The second height map is used to determine the position of the first object to be loaded into the box container.

4. The method according to claim 3, wherein: The first article and the one or more second articles have a rectangular parallelepiped shape, the predetermined orientation represents a predetermined arrangement direction of the one or more second articles in the packing container, and the predetermined shape is a rectangle.

5. The method according to claim 3, further comprising: Extracting a region corresponding to the at least one second object in the first height map; A corresponding area of ​​the predetermined shape arranged in the predetermined orientation is determined by an axis-aligned bounding box algorithm based on the area corresponding to the at least one second item.

6. The method according to any one of claims 1 to 2, further comprising: Scaling each area in the first height map based on a predetermined scaling ratio, including scaling an area in the first height map corresponding to the one or more second objects based on the predetermined scaling ratio; and The scaled first height map is used to determine a position where a first item is to be loaded into the box container.

7. The method according to any one of claims 3 to 5, further comprising: Scaling each area in the second height map based on a predetermined scaling ratio, including scaling an area in the second height map corresponding to the one or more second objects based on the predetermined scaling ratio; and The scaled second height map is used to determine a position where a first object is to be loaded into the box container.

8. The method according to any one of claims 1 to 5, further comprising: Obtaining a three-dimensional model of the packing container; Registering the three-dimensional model with the three-dimensional point cloud data to obtain a transformation matrix; and The three-dimensional point cloud data is transformed into a coordinate system of the packing container based on the transformation matrix to obtain three-dimensional point cloud data after coordinate transformation.

9. A device for packing, comprising: An acquisition unit, which is used to acquire three-dimensional point cloud data of the packing container, wherein the three-dimensional point cloud data is acquired based on data obtained by a three-dimensional camera arranged above the packing container; and A processing unit, which is used to convert the three-dimensional point cloud data into the coordinate system of the packing container to obtain three-dimensional point cloud data after coordinate transformation; determine a first height map of the packing container based on the three-dimensional point cloud data after coordinate transformation, the value of each pixel point in the first height map represents the depth value along the packing depth direction of the packing container; and use the first height map to determine the position of the first object to be packed into the packing container.

10. The apparatus of claim 9, wherein: The processing unit is also used to determine the orientation of one or more second items that have been loaded into the packing container based on the first height map; and use the orientation of the one or more second items to determine the position where the first item is to be loaded into the packing container.

11. The apparatus of claim 10, wherein: The processing unit is further configured to determine whether the orientation conforms to a predetermined orientation; if it is determined that the orientation of at least one of the one or more second objects does not conform to the predetermined orientation, replace the area corresponding to the at least one second object in the first height map with a corresponding area having a predetermined shape and arranged in the predetermined orientation to completely cover the corresponding second object, thereby obtaining a second height map; and using the second height map to determine a position at which the first object is to be loaded into the box container.

12. The apparatus of claim 11, wherein: The processing unit is also used to extract the area corresponding to the at least one second object in the first height map; and determine the corresponding area of ​​the predetermined shape set in the predetermined orientation through an axis-aligned bounding box algorithm based on the area corresponding to the at least one second object.

13. The apparatus according to any one of claims 11-12, wherein: The processing unit is further configured to scale each region in the second height map based on a predetermined scaling ratio, including scaling a region in the second height map corresponding to the one or more second objects based on the predetermined scaling ratio; and The scaled second height map is used to determine a position where a first object is to be loaded into the box container.

14. A system for packing, comprising: A three-dimensional camera is arranged above the packing container; and Apparatus for packing according to any one of claims 9 to 13.

15. A computer-readable medium storing a computer program, which, when executed by a processor, causes the processor to perform the method for boxing according to any one of claims 1 to 8.

Images