Robot for tidily placing objects in box body

Abstract

The invention provides a robot for tidily placing objects in a box body, and the robot comprises a first camera unit which is used for collecting a point cloud picture of the box body and a target object stored in the inner side of the box body in a scene; the processor module is used for acquiring point cloud charts of the box body and target objects stored on the inner side of the box body, generating height charts according to the point cloud charts, clustering and layering the height charts to form a plurality of layer spaces which are sequentially arranged from low to high, constructing a free placement map in each layer space, searching for idle positions in the free placement map located at the bottom layer, and storing the idle positions in the free placement map; when an idle position exists, the target object is placed at the idle position, and when the idle position does not exist, the free placement map located on the upper layer is searched for the idle position until the free placement map located on the topmost layer is full of the target object; and the robot unit comprises an end effector and is used for acquiring the idle position and placing the target object at the idle position. According to the invention, a target object can be placed in the box body in order.

Description

Technical Field

[0001] The present invention relates to a picking robot, in particular to a robot used for neatly placing objects in a box. Background Art

[0002] The picking robot is an intelligent device equipped with sensors, objective lenses and electronic optical systems that can quickly sort and move goods.

[0003] More and more visual sensors and force sensors will be used in picking robots, and picking robots will become more and more intelligent. With the advancement of sensing and recognition systems, artificial intelligence and other technologies, robots are moving from being controlled in one direction to storing and applying data by themselves, and are gradually becoming information-based.

[0004] When goods are sorted and transported by a picking robot, it is necessary to capture images of the goods through a visual sensor, identify the images of the goods to determine the posture of the goods, and then the picking robot picks or transports the goods through a loaded clamp or suction cup.

[0005] When packing incoming materials, it is necessary to store the incoming materials in multiple layers in the box. The packing method in the existing technology has a poor accuracy. Improper placement of items will reduce the number of items loaded in each layer, or affect the number of layers of packing. Too much gap between adjacent items will also affect the packing rate. Therefore, a new solution is needed. Summary of the invention

[0006] In view of the defects in the prior art, an object of the present invention is to provide a robot for neatly placing objects in a box.

[0007] The robot for neatly placing objects in a box provided by the present invention comprises:

[0008] The first camera unit is used to collect a point cloud image of the box and the target object stored inside the box in the scene;

[0009] A processor module is used to obtain a point cloud map of the box and the target object stored inside it, generate a height map according to the point cloud map, cluster and layer the height map to form a plurality of layer spaces arranged in sequence from low to high, construct a free placement map in each layer space, search for an idle position in the free placement map at the bottom layer, and when there is an idle position, control the robot to place the target object at the idle position; when there is no idle position, search for an idle position in the free placement map at the upper layer, until the target object fills the free placement map at the top layer;

[0010] The robot unit includes an end effector, which is used to obtain the idle position and place the target object at the idle position.

[0011] Preferably, the first camera unit comprises:

[0012] A projector, used to project structured light onto the side of the box in the scene;

[0013] A light receiving sensor, used for receiving the structured light reflected by the box and the target object stored inside the box to generate a structured light pattern;

[0014] A processing circuit is used to reconstruct and generate the box point cloud image according to the structured light pattern.

[0015] Preferably, the processor module comprises:

[0016] A height map generation module is used to obtain a point cloud map of the box and the target object stored inside it, and generate a height map according to the point cloud map;

[0017] A placement map generation module is used to cluster and layer the height map to form a plurality of layer spaces arranged in sequence from low to high, and to construct a free placement map in each layer space;

[0018] The placement position calculation module is used to search for free positions in the free placement map from the bottom layer to the top layer. When there is a free position, the placement position is output. The placement position is used by the robot to place the target object:

[0019] The placement control module is used to control the height map generation module, the placement map generation module and the placement position calculation module to run repeatedly until the objects fill the box or the picking of the task quantity is completed.

[0020] Preferably, further comprising a second camera unit;

[0021] The second camera unit has a visual scanning area covering at least a target object stored in the box, and is used to perform visual scanning on the target object, collect a depth image of the target object, and generate position information of the target object according to the depth image;

[0022] The robot unit is connected to the depth camera for receiving the position information and controlling the end effector to pick out the target object in a corresponding grasping posture according to the position information.

[0023] Preferably, the height map generation module includes the following units:

[0024] A point cloud acquisition unit is used to acquire a point cloud image of the box and the target object stored inside the box, project the point cloud image in the camera coordinate system to the image coordinate system, calculate the coordinates of each point in the point cloud image in the image coordinate system, and obtain a two-dimensional point cloud in the image coordinate system;

[0025] A coordinate system conversion unit is used to convert the two-dimensional point cloud in the image coordinate system to the pixel coordinate system, calculate the coordinates of the two-dimensional point cloud in the image coordinate system in the image pixel coordinate system, and obtain the two-dimensional point cloud in the pixel coordinate system;

[0026] The height map generating unit is used to project the two-dimensional point cloud in the pixel coordinate system and the height value corresponding to each point into a preset height map model to generate the target height map.

[0027] Preferably, the point cloud acquisition unit includes the following parts:

[0028] A projector control part, used to control the projector in the depth camera to project structured light onto the box;

[0029] A sensor control part, used for controlling the light receiving sensor in the depth camera to receive the structured light reflected by the object to generate a plurality of structured light images;

[0030] The depth reconstruction part is used to perform depth reconstruction based on the multiple structured light images to generate a point cloud image of the box and the target object stored inside it.

[0031] Preferably, each pixel point in the free placement map indicates whether the point can support the target object in the layer space, and a plurality of pixel points that can stably support the target object form the idle position.

[0032] Preferably, the placement position calculation module includes the following units:

[0033] A location search unit, used to search for an idle location along a preset oblique direction in a free-placement map located at the bottom layer;

[0034] A placement control unit, used to control the robot to place the target object at the idle position when searching for an idle position, thereby triggering the height map generation module, and triggering the search control unit when there is no idle position;

[0035] The search control unit is used to search for free positions on the free placement maps on the upper layer in sequence until the free placement map on the top layer.

[0036] Preferably, the placement position calculation module further includes the following units:

[0037] The top layer construction unit is used to construct a free placement map on a higher layer after increasing a preset target height according to the free placement map on the top layer, so as to place the target object in the free placement map on the higher layer.

[0038] Preferably, when the robot is controlled to place the target object at an idle position, the following units are used:

[0039] A size determination unit, used to obtain a point cloud image of a target object, and determine a three-dimensional size of the target object according to the point cloud image of the target object;

[0040] A scale adjustment unit, used for acquiring a preset scale adjustment factor, and adjusting the three-dimensional size of the target object according to the scale adjustment factor to generate a target three-dimensional size;

[0041] A placement control unit is used to control the position search unit to search for an idle position according to the target three-dimensional size, and then control the robot unit to place the target object at the idle position.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] In the present invention, a height map is first generated according to a point cloud map of the box and the target object stored inside it, and then the height map is clustered and layered, and a layer space is constructed layer by layer from low to high. A free placement map is constructed in each layer space, and free positions are searched in the free placement map from the bottom layer to the upper layer. When there is a free position, the placement position is output. The placement position is used for the robot to place the target object, and the objects are repeatedly placed in the box to fill the box or complete the task quantity of picking, thereby achieving neat placement of the target objects in the box. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings in the following descriptions are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work. By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, purposes and advantages of the present invention will become more obvious:

[0045] Figure 1 This is a schematic diagram of the structure of a robot used for neatly placing objects in a box in the first embodiment of the present invention;

[0046] Figure 2 This is a schematic diagram of a module of a robot for neatly placing objects in a box according to the first embodiment of the present invention;

[0047] Figure 3 is a schematic diagram of a module of a camera unit in a first embodiment of the present invention;

[0048] Figure 4 This is a schematic diagram of a module of a robot for neatly placing objects in a box according to a second embodiment of the present invention;

[0049] Figure 5 This is a schematic diagram of a module of a robot for neatly placing objects in a box according to a third embodiment of the present invention;

[0050] Figure 6 This is a schematic diagram of a module of a robot for neatly placing objects in a box according to a fourth embodiment of the present invention;

[0051] Figure 7 This is a schematic diagram of a module of a robot for neatly placing objects in a box according to a fifth embodiment of the present invention;

[0052] Figure 8 A flowchart of the steps of finding an idle position in the prior art; and

[0053] Fig. 9 The following is a flowchart of the steps of searching for free locations in various embodiments of the present invention. DETAILED DESCRIPTION

[0054] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0055] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein, for example, can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0056] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0057] The following specific embodiments are used to describe in detail the technical solutions of the present invention and how the technical solutions of the present application solve the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0058] Figure 1 FIG. 1 is a schematic diagram of a module of a robot for neatly placing objects in a box according to a first embodiment of the present invention. Figure 1 As shown, the robot provided by the present invention for neatly placing objects in a box comprises:

[0059] The first unit and the second unit are used for storing and / or transporting the target object, and the first unit can be configured as a box 106;

[0060] The first camera unit 102 is used to collect a point cloud image of the box and the target object stored inside it in the scene;

[0061] A second camera unit 103, whose visual scanning area at least covers the first unit where the target object is stored or transported, is used to visually scan the target object, collect a depth image of the target object, and generate position information and storage location of the target object according to the depth image;

[0062] A processor module is used to obtain a point cloud map of the box and the target object stored inside it, generate a height map according to the point cloud map, cluster and layer the height map to form a plurality of layer spaces arranged in sequence from low to high, construct a free placement map in each layer space, search for an idle position in the free placement map at the bottom layer, and when there is an idle position, control the robot to place the target object at the idle position; when there is no idle position, search for an idle position in the free placement map at the upper layer, until the target object fills the free placement map at the top layer;

[0063] The robot unit 101 is in communication with the first camera unit 102 and the second camera unit 103 , and includes an end effector 105 for acquiring the position of the box 106 and picking the target object into the box.

[0064] Box 106, used for storing neatly arranged target objects;

[0065] The robot unit 101 is communicatively connected to the second camera unit 103, and is used for receiving the posture information and the storage position, judging the placement state of the target object according to the posture information and the storage position, picking up the target object according to the placement state and transferring it to the first unit.

[0066] The first unit may be configured to transport or store the picked target objects, such as a box that is convenient for arranging the objects neatly.

[0067] The second unit may also be provided with a transport unit, so that the robot unit 101 can move the target object on the transport unit into the box.

[0068] The second camera unit 103 is arranged on a camera bracket.

[0069] Figure 2 FIG. 1 is a schematic diagram of a module of a robot for neatly placing objects in a box according to a first embodiment of the present invention. Figure 2 As shown, the robot provided by the present invention for neatly placing objects in a box comprises

[0070] A first camera unit, used for collecting a box point cloud image in the scene;

[0071] A processor module is used to obtain a point cloud map of the box and the target object stored inside it, generate a height map according to the point cloud map, cluster and layer the height map to form a plurality of layer spaces arranged in sequence from low to high, construct a free placement map in each layer space, search for an idle position in the free placement map at the bottom layer, and when there is an idle position, control the robot to place the target object at the idle position; when there is no idle position, search for an idle position in the free placement map at the upper layer, until the target object fills the free placement map at the top layer;

[0072] The robot unit includes an end effector, which is used to obtain the idle position and place the target object at the idle position.

[0073] In an embodiment of the present invention, a height map is first generated based on a point cloud map of the box and the target object stored inside it, and then the height map is clustered and layered to construct a layer space from low to high. A free placement map is constructed in each layer space, and free positions are searched in the free placement map from the bottom layer to the top layer. When there is a free position, the placement position is output. The placement position is used for the robot to place the target object, and the process is repeated many times to fill the box with objects or complete the picking of the task quantity, thereby achieving neat placement of the target objects in the box.

[0074] Figure 3 FIG. 1 is a schematic diagram of a module of a camera unit in a first embodiment of the present invention. Figure 3 As shown, the first camera unit includes:

[0075] A projector 1031 is used to project structured light onto the box in the scene;

[0076] A light receiving sensor 1032 is used to receive the structured light reflected by the box and the target object stored inside to generate a structured light pattern;

[0077] The processing circuit 1033 is used to reconstruct and generate a point cloud image of the box and the target object stored inside the box according to the structured light pattern.

[0078] The processing circuit is also used to control the projector and the light receiving sensor to be turned on or off at the same time; the processing circuit can be an independent dedicated chip, such as a dedicated SOC chip, FPGA chip, ASIC chip, etc., or it can include a general-purpose processor. For example, when the depth camera unit is integrated into a smart terminal such as a mobile phone, a computer, or a robot controller, the processor in the terminal can serve as at least a part of the processing circuit.

[0079] Figure 4 FIG. 1 is a schematic diagram of a module of a robot for neatly placing objects in a box according to a second embodiment of the present invention. Figure 4 As shown, the robot for neatly placing objects in a box provided by the present invention, the processor module includes:

[0080] A height map generation module is used to obtain a point cloud map of the box and the target object stored inside it, and generate a height map according to the point cloud map;

[0081] A placement map generation module is used to cluster and layer the height map to form a plurality of layer spaces arranged in sequence from low to high, and to construct a free placement map in each layer space;

[0082] The placement position calculation module is used to search for free positions in the free placement map from the bottom layer to the top layer. When there is a free position, the placement position is output. The placement position is used by the robot to place the target object:

[0083] The placement control module is used to control the height map generation module, the placement map generation module and the placement position calculation module to run repeatedly until the objects fill the box or the picking of the task quantity is completed.

[0084] In an embodiment of the present invention, the point cloud is a data set, each point in the data set represents a set of X, Y, Z geometric coordinates and an intensity value, which records the intensity of the return signal according to the reflectivity of the object surface. When these points are combined together, a point cloud is formed, that is, a collection of data points representing a 3D shape or object in space. The point cloud can also be automatically colored to achieve more realistic visualization.

[0085] Figure 5 FIG. 1 is a schematic diagram of a module of a robot for neatly placing objects in a box according to a third embodiment of the present invention. Figure 5As shown, the height map generation module includes the following units:

[0086] A point cloud acquisition unit is used to acquire a point cloud image of the box and the target object stored inside the box, project the point cloud image in the camera coordinate system to the image coordinate system, calculate the coordinates of each point in the point cloud image in the image coordinate system, and obtain a two-dimensional point cloud in the image coordinate system;

[0087] A coordinate system conversion unit is used to convert the two-dimensional point cloud in the image coordinate system to the pixel coordinate system, calculate the coordinates of the two-dimensional point cloud in the image coordinate system in the image pixel coordinate system, and obtain the two-dimensional point cloud in the pixel coordinate system;

[0088] The height map generating unit is used to project the two-dimensional point cloud in the pixel coordinate system and the height value corresponding to each point into a preset height map model to generate the target height map.

[0089] In the embodiment of the present invention, the pixel coordinate system takes the upper left corner of the image as the origin. P , a two-dimensional rectangular coordinate system is established with the row and column directions of the pixels being the U and V coordinate axis directions respectively.

[0090] The image coordinate system is a two-dimensional rectangular coordinate system established with the intersection of the lens optical axis and the image plane as the origin O1 and the row and column directions parallel to the pixels as the X and Y coordinate axis directions.

[0091] The camera coordinate system takes the optical center of the lens as the origin. C , the row and column directions parallel to the pixels are X C Axis and Y C Axis direction, determine Z according to the right-hand coordinate system rule C A three-dimensional rectangular coordinate system established along the axis direction.

[0092] In an embodiment of the present invention, the height map model is divided into squares to obtain a plurality of grids, and the grids are numbered to obtain grid numbers. Similarly, the XOY plane in the camera coordinate system and the image coordinate system is also divided into squares of the same size to obtain a plurality of grids, and the grids are numbered. According to the grid number and the size of each grid, the x-axis and y-axis coordinates corresponding to the grid can be obtained. According to the conversion relationship between the camera coordinate system, the image coordinate system and the pixel coordinate system, the x-axis and y-axis coordinates are converted to the camera coordinate system to obtain the x-axis and y-axis coordinates in the camera coordinate system, and then according to the x-axis and y-axis coordinates in the camera coordinate system, the grid corresponding to a certain grid in the pixel coordinate system in the camera coordinate system can be obtained. Then, the height value in the height map model corresponding to the multi-frame two-dimensional point cloud in the pixel coordinate system is calculated by the conversion relationship.

[0093] In an embodiment of the present invention, the point cloud acquisition unit includes the following parts:

[0094] A projector control part, used to control the projector in the depth camera to project structured light onto the box;

[0095] A sensor control part, used for controlling the light receiving sensor in the depth camera to receive the structured light reflected by the object to generate a plurality of structured light images;

[0096] The depth reconstruction part is used to perform depth reconstruction based on the multiple structured light images to generate a point cloud image of the box and the target object stored inside it.

[0097] In an embodiment of the present invention, the projector is a DLP projector, the light receiving sensor is a camera unit, which is used to collect the structured light pattern formed by the structured light on the box, and the processor module is used to control the synchronous switching of the projector and the camera unit, and generate a point cloud image or depth image of the box and the target object stored inside it according to the structured light pattern.

[0098] Figure 6 FIG. 4 is a schematic diagram of a module of a robot for neatly placing objects in a box according to a fourth embodiment of the present invention. Figure 6 As shown, the layout map generation module includes the following units:

[0099] A clustering and stratification unit, used for clustering and stratifying the height map to form a plurality of layer spaces arranged in sequence from low to high;

[0100] A map construction unit, used to construct a free-placement map in each layer space and determine the layer height of the free-placement map, wherein the layer height of the free-placement map is greater than or equal to the height of the target object;

[0101] The height determination unit is used to determine the height of the freely placed map according to the layer height of the freely placed map, and the height of the freely placed map is the lowest point of the freely placed maps on this layer.

[0102] In an embodiment of the present invention, a kernel density estimation (KDE) method is used, and Gaussian is selected as the kernel function to cluster and stratify the height map, including the following steps:

[0103] dilating and corroding the height map;

[0104] Evenly separate the minimum height value and the maximum height value of the height map and store them into a data array. For example, if the minimum height value is 10 and the maximum height value is 50, separated by 10, the data array is [10, 20, 30, 40, 50].

[0105] Generate a KDE value for each value in the data array, and then calculate a local maxima based on the KDE value;

[0106] Each pixel value in the height map is replaced by the corresponding and closest local maximum value in the layer space.

[0107] Figure 7 FIG. 4 is a schematic diagram of a module of a robot for neatly placing objects in a box according to a fifth embodiment of the present invention. Figure 7 As shown, the placement position calculation module includes the following units:

[0108] A location search unit, used to search for an idle location along a preset oblique direction in a free-placement map located at the bottom layer;

[0109] A placement control unit, used to control the robot to place the target object at the idle position when searching for an idle position, thereby triggering the height map generation module, and triggering the search control unit when there is no idle position;

[0110] The search control unit is used to search for free positions on the free placement maps on the upper layer in sequence until the free placement map on the top layer.

[0111] In the embodiment of the present invention, an idle position is searched along a 45° oblique angle direction, such as Fig. 9 As shown, it is possible to avoid Figure 8 In the case of missing free positions when searching for free positions row by row or column by column.

[0112] Set the length of the target object to l, the width to w, and the height to h. When searching for free positions, check whether the area is free by placing l along the long side of the material box and w along the short side of the material box; then check whether the area is free by placing l along the short side of the material box and w along the long side of the material box.

[0113] In a variation of the present invention, the placement position calculation module further includes the following units:

[0114] The top layer construction unit is used to construct a free placement map on a higher layer after increasing a preset target height according to the free placement map on the top layer, so as to place the target object in the free placement map on the higher layer.

[0115] The preset target height is determined according to the height tolerance error of the objects placed in the box, such as 1 cm and 0.5 cm.

[0116] When controlling the robot to place the target object in an idle position, it is achieved through the following units:

[0117] A size determination unit, used to obtain a point cloud image of a target object, and determine a three-dimensional size of the target object according to the point cloud image of the target object;

[0118] A scale adjustment unit, used for acquiring a preset scale adjustment factor, and adjusting the three-dimensional size of the target object according to the scale adjustment factor to generate a target three-dimensional size;

[0119] A placement control unit is used to control the position search unit to search for an idle position according to the target three-dimensional size, and then control the robot unit to place the target object at the idle position.

[0120] In a variation of the present invention, when the gap between adjacent target objects is greater than a preset gap threshold, the proportional adjustment factor is less than 1. For example, when the gap threshold along the length direction of the target object is 10% of the length of the target object, the proportional adjustment factor can be set to between 0.92 and 0.98.

[0121] When adjacent target objects collide and overlap, the proportional adjustment factor is greater than 1, and the proportional adjustment factor can be set between 1.02 and 1.1.

[0122] In an embodiment of the present invention, a height map is first generated based on a point cloud map of the box and the target object stored inside it, and then the height map is clustered and layered to construct a layer space from low to high. A free placement map is constructed in each layer space, and free positions are searched in the free placement map from the bottom layer to the top layer. When there is a free position, the placement position is output. The placement position is used for the robot to place the target object, and the process is repeated many times to fill the box with objects or complete the picking of the task quantity, thereby achieving neat placement of the target objects in the box.

[0123] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown in this article, but will comply with the widest range consistent with the principles and novel features disclosed herein.

[0124] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A robot for neatly placing objects in a box, characterized in that: include: The first camera unit is used to collect a point cloud image of the box and the target object stored inside the box in the scene; A processor module is used to obtain a point cloud map of the box and the target object stored inside it, generate a height map according to the point cloud map, cluster and layer the height map to form a plurality of layer spaces arranged in sequence from low to high, construct a free placement map in each layer space, search for an idle position in the free placement map at the bottom layer, and when there is an idle position, control the robot to place the target object at the idle position; when there is no idle position, search for an idle position in the free placement map at the upper layer, until the target object fills the free placement map at the top layer; The robot unit includes an end effector, which is used to obtain the idle position and place the target object at the idle position.

2. The robot for neatly placing objects in a box according to claim 1, characterized in that: The first camera unit comprises: A projector, used to project structured light onto the side of the box in the scene; A light receiving sensor, used for receiving the structured light reflected by the box and the target object stored inside the box to generate a structured light pattern; A processing circuit is used to reconstruct and generate the box point cloud image according to the structured light pattern.

3. The robot for neatly placing objects in a box according to claim 1, characterized in that: The processor module comprises: A height map generation module is used to obtain a point cloud map of the box and the target object stored inside it, and generate a height map according to the point cloud map; A placement map generation module is used to cluster and layer the height map to form a plurality of layer spaces arranged in sequence from low to high, and to construct a free placement map in each layer space; The placement position calculation module is used to search for free positions in the free placement map from the bottom layer to the top layer. When there is a free position, the placement position is output. The placement position is used by the robot to place the target object: The placement control module is used to control the height map generation module, the placement map generation module and the placement position calculation module to run repeatedly until the objects fill the box or the picking of the task quantity is completed.

4. The robot for neatly placing objects in a box according to claim 1, characterized in that: Also included is a second camera unit; The second camera unit has a visual scanning area covering at least a target object stored in the box, and is used to perform visual scanning on the target object, collect a depth image of the target object, and generate position information of the target object according to the depth image; The robot unit is connected to the depth camera for receiving the position information and controlling the end effector to pick out the target object in a corresponding grasping posture according to the position information.

5. The robot for neatly placing objects in a box according to claim 1, characterized in that: The height map generation module includes the following units: A point cloud acquisition unit is used to acquire a point cloud image of the box and the target object stored inside the box, project the point cloud image in the camera coordinate system to the image coordinate system, calculate the coordinates of each point in the point cloud image in the image coordinate system, and obtain a two-dimensional point cloud in the image coordinate system; A coordinate system conversion unit is used to convert the two-dimensional point cloud in the image coordinate system to the pixel coordinate system, calculate the coordinates of the two-dimensional point cloud in the image coordinate system in the image pixel coordinate system, and obtain the two-dimensional point cloud in the pixel coordinate system; The height map generating unit is used to project the two-dimensional point cloud in the pixel coordinate system and the height value corresponding to each point into a preset height map model to generate the target height map.

6. The robot for neatly placing objects in a box according to claim 2, characterized in that: The point cloud acquisition unit includes the following parts: A projector control part, used to control the projector in the depth camera to project structured light onto the box; A sensor control part, used for controlling the light receiving sensor in the depth camera to receive the structured light reflected by the object to generate a plurality of structured light images; The depth reconstruction part is used to generate a point cloud image of the box and the target object stored inside it by performing depth reconstruction based on the multiple structured light images.

7. The robot for neatly placing objects in a box according to claim 1, characterized in that: Each pixel point in the free placement map indicates whether the point can support the target object in the layer space, and a plurality of pixel points that can stably support the target object form the idle position.

8. The robot for neatly placing objects in a box according to claim 7, characterized in that: The placement position calculation module includes the following units: A location search unit, used to search for an idle location along a preset oblique direction in a free-placement map located at the bottom layer; A placement control unit, used to control the robot to place the target object at the idle position when searching for an idle position, thereby triggering the height map generation module, and triggering the search control unit when there is no idle position; The search control unit is used to search for free positions on the free placement maps on the upper layer in sequence until the free placement map on the top layer.

9. The robot for neatly placing objects in a box according to claim 8, characterized in that: The placement position calculation module also includes the following units: The top layer construction unit is used to construct a free placement map on a higher layer after increasing a preset target height according to the free placement map on the top layer, so as to place the target object in the free placement map on the higher layer.

10. The robot for neatly placing objects in a box according to claim 1, characterized in that: When the robot is controlled to place the target object in an idle position, it is achieved through the following units: A size determination unit, used to obtain a point cloud image of a target object, and determine a three-dimensional size of the target object according to the point cloud image of the target object; A scale adjustment unit, used for acquiring a preset scale adjustment factor, and adjusting the three-dimensional size of the target object according to the scale adjustment factor to generate a target three-dimensional size; A placement control unit is used to control the position search unit to search for an idle position according to the target three-dimensional size, and then control the robot unit to place the target object at the idle position.

Images