Alignment state confirmation method, control system and carrying equipment

Abstract

The embodiment of the invention discloses an alignment state confirmation method, a control system and carrying equipment. According to the main technical scheme, the method comprises the steps that a controller obtains target point clouds of a first stacking object and a second stacking object through a sensor; the controller extracts a first point cloud of a first target area of the first stacking object and a second point cloud of a second target area of the second stacking object from the target point cloud; the controller matches the first point cloud with the first template point cloud to obtain the pose of the first stacking object, and matches the second point cloud with the second template point cloud to obtain the pose of the second stacking object; the controller determines a difference value between the pose of the first stacked object and the pose of the second stacked object, and compares the difference value with a threshold value to confirm the alignment state of the first stacked object and the second stacked object. The embodiment of the invention is used for realizing alignment of the first stacking object and the second stacking object during stacking.

Description

Technical Field

[0001] The present application relates to the field of warehousing logistics and machine vision technology, and in particular to an alignment status confirmation method, a control system and a handling device. Background Art

[0002] Systems that use handling equipment such as AGV (automated guided vehicle) have the advantages of being highly unmanned, automated, and intelligent, which improves production efficiency and operational levels for industries such as warehousing, manufacturing, and logistics. As one of the more typical scenarios, handling equipment is often responsible for the handling of various goods. In the process of handling, it is inevitable to stack goods. Goods are usually packaged in cartons, etc., or stored in cages, wooden boxes, plastic boxes, etc.

[0003] Considering the space utilization, the handling equipment will involve stacking objects such as cartons, cages, wooden boxes, etc. in the process of handling goods. Considering the stability, the handling equipment needs to accurately stack one stacking object on top of another stacking object. In this process, if the two stacking objects cannot be aligned, it may affect the safety of the operation. Summary of the invention

[0004] The present application provides an alignment state confirmation method, a control system and a handling device, which are used to control the alignment of a first stacking object and a second stacking object during stacking.

[0005] This application provides the following solutions:

[0006] According to a first aspect, a method for confirming an alignment status is provided, the method comprising: a controller acquires a target point cloud of a first stacking object and a second stacking object through a sensor; the controller extracts a first point cloud of a first target area of ​​the first stacking object and a second point cloud of a second target area of ​​the second stacking object from the target point cloud; the controller matches the first point cloud with the first template point cloud to obtain a posture of the first stacking object, and matches the second point cloud with the second template point cloud to obtain a posture of the second stacking object; the controller determines a difference between the posture of the first stacking object and the posture of the second stacking object, and compares the difference with a threshold value to confirm the alignment status of the first stacking object and the second stacking object.

[0007] Optionally, the method further includes: before confirming the alignment state, the controller controls the transport device to transport the first stacking object to a stacking preparation position to complete a pre-alignment action relative to the second stacking object.

[0008] Optionally, comparing the difference with a threshold value to confirm the alignment state of the first stacked object and the second stacked object includes:

[0009] If the difference is greater than or equal to the threshold, the alignment state is confirmed to be misaligned;

[0010] If the difference is less than the threshold, the alignment status is confirmed to be aligned.

[0011] Optionally, the method further comprises:

[0012] When the alignment state is misaligned, the controller controls the handling device to adjust the posture;

[0013] The controller reacquires the target point cloud of the first stacked object and the second stacked object through the sensor;

[0014] The controller re-determines the difference between the posture of the first stacked object and the posture of the second stacked object;

[0015] The controller reconfirms the alignment status based on the difference until the difference is smaller than the threshold.

[0016] Optionally, the method further comprises:

[0017] When the alignment state is aligned, the controller controls the transport device to place the first stacking object on the second stacking object to complete the stacking.

[0018] Optionally, controlling the transport equipment to adjust the posture includes: controlling the transport equipment to adjust the posture of the chassis or the posture of the fork.

[0019] Optionally, the method further comprises:

[0020] The controller counts the number of misalignments;

[0021] When the number of misalignments is greater than the preset number of misalignments, an alarm prompt is output.

[0022] Optionally, the first template point cloud and the second template point cloud are obtained in the following manner:

[0023] The controller collects original point clouds of a third stacking object and a fourth stacking object that meet the alignment posture standard through a sensor, the third stacking object has the same structure and size as the first stacking object, and the fourth stacking object has the same structure and size as the second stacking object;

[0024] The controller pre-processes the original point cloud to obtain a standard point cloud of the third stacking object and a standard point cloud of the fourth stacking object;

[0025] The controller extracts a first template point cloud from a standard point cloud of a third stacked object;

[0026] The controller extracts a second template point cloud from the standard point cloud of the fourth stacked object.

[0027] Optionally, the first template point cloud includes a point cloud of a first corner structure region of the third stacked object extracted from a standard point cloud of the third stacked object.

[0028] Optionally, the second template point cloud includes: a point cloud of a second corner structure region of the fourth stacked object extracted from a standard point cloud of the fourth stacked object.

[0029] Optionally, the first corner structure region has a first side line and a third side line, and the first side line intersects with the third side line.

[0030] Optionally, the second corner structure region has a second side line and a fourth side line, and the second side line intersects with the fourth side line.

[0031] Optionally, the first template point cloud further includes: a point cloud of a third corner structure region of the third stacked object extracted from a standard point cloud of the third stacked object;

[0032] The second template point cloud also includes: a point cloud of a fourth corner structure region of the fourth stacking object extracted from the standard point cloud of the fourth stacking object.

[0033] Optionally, the first template point cloud further includes: a point cloud of a fifth corner structure region and a point cloud of a seventh corner structure region of the third stacked object extracted from a standard point cloud of the third stacked object;

[0034] The second template point cloud also includes: a point cloud of a sixth corner structure region and a point cloud of an eighth corner structure region of the fourth stacking object extracted from the standard point cloud of the fourth stacking object.

[0035] Optionally, the first stacking object is a first material cage, and the second stacking object is a second material cage;

[0036] The first target area is located at the bottom of the first cage, and the first target area includes: a first corner structure area of ​​the first cage, a third corner structure area of ​​the first cage, a fifth corner structure area of ​​the first cage, and a seventh corner structure area of ​​the first cage;

[0037] The second target area is located at the top of the second material cage, and the second target area includes: the second corner structure area of ​​the second material cage, the fourth corner structure area of ​​the second material cage, the sixth corner structure area of ​​the second material cage and the eighth corner structure area of ​​the second material cage.

[0038] Optionally, the first template point cloud includes: a point cloud of a partial area of ​​the first foot cup of the third stacked object extracted from a standard point cloud of the third stacked object;

[0039] The second template point cloud includes: a point cloud of a partial area of ​​the second column of the fourth stacking object extracted from the standard point cloud of the fourth stacking object.

[0040] Optionally, the first template point cloud further includes: a point cloud of a partial area of ​​a third foot cup of the third stacked object extracted from a standard point cloud of the third stacked object;

[0041] The second template point cloud also includes: a point cloud of a partial area of ​​a fourth column of the fourth stacking object extracted from the standard point cloud of the fourth stacking object.

[0042] Optionally, the first template point cloud further includes: a point cloud of a partial area of ​​the fifth foot cup and a point cloud of a partial area of ​​the seventh foot cup of the third stacked object extracted from the standard point cloud of the third stacked object;

[0043] The second template point cloud also includes: a point cloud of a partial area of ​​the sixth column and a point cloud of a partial area of ​​the eighth column of the fourth stacking object extracted from the standard point cloud of the fourth stacking object.

[0044] Optionally, the first stacking object is a first material cage, and the second stacking object is a second material cage;

[0045] The first target area includes: the first foot cup of the first material basket, the third foot cup of the first material basket, the fifth foot cup of the first material basket, and the seventh foot cup of the first material basket;

[0046] The second target area includes: the second column of the second material basket, the fourth column of the second material basket, the sixth column of the second material basket, and the eighth column of the second material basket.

[0047] According to a second aspect, a control system is provided, including a memory and a controller, wherein the memory is used to store program instructions, and the controller is used to execute the program instructions to implement any one of the methods in the first aspect.

[0048] According to a third aspect, a handling device is provided, comprising a memory and a controller, wherein the memory is used to store program instructions, and the controller is used to execute the program instructions to implement any one of the methods described in the first aspect.

[0049] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0050] 1) In the present application, the controller extracts the first point cloud of the first target area of ​​the first stacking object from the target point cloud of the first stacking object obtained by the sensor, and extracts the second point cloud of the second target area of ​​the second stacking object from the target point cloud of the second stacking object obtained by the sensor, and matches the first point cloud with the first template point cloud to obtain the posture of the first stacking object, and matches the second point cloud with the second template point cloud to obtain the posture of the second stacking object, and then compares the difference between the posture of the first stacking object and the posture of the second stacking object with the threshold value to accurately confirm the alignment state of the first stacking object and the second stacking object. In this way, the difference between the posture of the first stacking object and the posture of the second stacking object can be determined during the stacking process, and the alignment state between the first stacking object and the second stacking object can be confirmed based on the difference and the threshold value. This solution can avoid the influence of the external environment (such as uneven ground) and the error of the handling equipment itself, and can accurately calculate the posture data of the first stacking object and the second stacking object, and then determine the alignment state of the two.

[0051] 2) The present application dynamically controls the posture adjustment of the handling equipment based on the difference between the posture of the first stacked object and the posture of the second stacked object and a threshold; when the difference is greater than or equal to the threshold, the alignment state is confirmed to be misaligned, and the handling equipment is controlled to adjust the posture; the sensor reacquires the target point cloud of the first stacked object and the second stacked object, so that the controller re-determines the difference between the posture of the first stacked object and the posture of the second stacked object to reconfirm the alignment state, until the difference is less than the threshold, forming a closed-loop servo detection process, and the operation of the handling equipment does not need to be stopped during the servo detection process.

[0052] 3) This application collects point clouds of the third stacked object and the fourth stacked object that meet the alignment posture standard, and can extract the first template point cloud and the second template point cloud from the standard point cloud of the third stacked object and the standard point cloud of the fourth stacked object, respectively, without frequent size measurement, simplifying the data collection process. Before extracting the first template point cloud and the second template point cloud from the standard point cloud of the third stacked object and the standard point cloud of the fourth stacked object, respectively, the original point clouds of the third stacked object and the fourth stacked object that meet the alignment posture standard are preprocessed to obtain the standard point cloud of the third stacked object and the standard point cloud of the fourth stacked object, which can further improve the accuracy and reliability of the point cloud.

[0053] 4) The present application uses the point cloud of at least one corner structure area of ​​the third stacking object in the standard point cloud of the third stacking object as the first template point cloud; and / or uses the point cloud of at least one corner structure area of ​​the fourth stacking object in the standard point cloud of the fourth stacking object as the second template point cloud, which can ensure the accuracy of the posture and reduce the processing amount during matching, thereby improving the matching speed, and further speeding up the speed of confirming the alignment status of the first stacking object and the second stacking object.

[0054] 5) The first target area of ​​the first basket in the present application includes at least one corner structure area of ​​the first basket or a partial area of ​​at least one foot cup; and / or, the second target area of ​​the second basket includes at least one corner structure area of ​​the second basket or a partial area of ​​at least one column. Both methods can further reduce the processing volume during matching.

[0055] Of course, any invention of the present application does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 is a system schematic diagram of a handling device applicable to an embodiment of the present application;

[0058] Figure 2 An application scenario diagram provided for an embodiment of the present application;

[0059] Figure 3 A flowchart of an alignment status confirmation method provided in an embodiment of the present application;

[0060] Figure 4 A schematic diagram of the foot cups and pillars of the first stacking object and the second stacking object provided in an embodiment of the present application;

[0061] Figure 5 A schematic plan view of the foot cups and pillars of the first stacking object and the second stacking object provided in an embodiment of the present application;

[0062] Figure 6 A schematic diagram of the corner structure area and edge lines of the first stacking object and the second stacking object provided in an embodiment of the present application;

[0063] Figure 7 A flowchart for generating a first template point cloud;

[0064] Figure 8 A flowchart for generating a second template point cloud;

[0065] Fig. 9 A schematic diagram of corner structure areas and side lines of a third stacking object and a fourth stacking object provided in an embodiment of the present application;

[0066] Fig.10 A schematic diagram of the foot cups and pillars of the third stacking object and the fourth stacking object provided in the embodiment of the present application;

[0067] Fig.11 A schematic diagram of confirming the alignment state provided in an embodiment of the present application;

[0068] Fig.12 A schematic diagram of a second point cloud and a second template point cloud before loop closure detection provided in an embodiment of the present application;

[0069] Fig.13 A schematic diagram of a second point cloud and a second template point cloud after loop closure detection provided in an embodiment of the present application;

[0070] Fig.14 A flowchart of an alignment status confirmation method provided in an embodiment of the present application;

[0071] Fig.15 A flowchart of an alignment status confirmation method provided in an embodiment of the present application.

[0072] Among them, 100-handling equipment; 101-handling equipment body; 102-sensor; 103-stacking execution component; 104-controller; 105-memory; A'-first stacking object; B'-second stacking object; A-third stacking object; B-fourth stacking object; A1-first target area; B1-second target area; PC1-first point cloud; TM1-first template point cloud; PC2-second point cloud; TM2-second template point cloud; aj1'-first foot cup of the first stacking object; aj3'-third foot cup of the first stacking object; aj5'-fifth foot cup of the first stacking object; aj7'-seventh foot cup of the first stacking object; bj2'-second column of the second stacking object; bj4' - the fourth column of the second stacking object; bj6'- the sixth column of the second stacking object; bj8'- the eighth column of the second stacking object; a1'- the first corner structural area of ​​the first stacking object; a3'- the third corner structural area of ​​the first stacking object; a5'- the fifth corner structural area of ​​the first stacking object; a7'- the seventh corner structural area of ​​the first stacking object; b2'- the second corner structural area of ​​the second stacking object; b4'- the fourth corner structural area of ​​the second stacking object; b6'- the sixth corner structural area of ​​the second stacking object; b8'- the eighth corner structural area of ​​the second stacking object; a11'- the first sideline of the first stacking object; a13'- the third sideline of the first stacking object; a15'- the first sideline of the first stacking object The fifth sideline; a17'-the seventh sideline of the first stacking object; b22'-the second sideline of the second stacking object; b24'-the fourth sideline of the second stacking object; b26'-the sixth sideline of the second stacking object; b28'-the eighth sideline of the second stacking object; aj1-the first foot cup of the third stacking object; aj3-the third foot cup of the third stacking object; aj5-the fifth foot cup of the third stacking object; aj7-the seventh foot cup of the third stacking object; bj2-the second column of the fourth stacking object; bj4-the fourth column of the fourth stacking object; bj6-the sixth column of the fourth stacking object; bj8-the eighth column of the fourth stacking object; a1-the first corner structure area of ​​the third stacking object; a3-the third stacking object a5-the fifth corner structural area of ​​the third stacked object; a7-the seventh corner structural area of ​​the third stacked object; b2-the second corner structural area of ​​the fourth stacked object; b4-the fourth corner structural area of ​​the fourth stacked object; b6-the sixth corner structural area of ​​the fourth stacked object; b8-the eighth corner structural area of ​​the fourth stacked object; a11-the first side line of the third stacked object; a13-the third side line of the third stacked object; a15-the fifth side line of the third stacked object; a17-the seventh side line of the third stacked object; b22-the second side line of the fourth stacked object; b24-the fourth side line of the fourth stacked object; b26-the sixth side line of the fourth stacked object; b28-the eighth side line of the fourth stacked object. DETAILED DESCRIPTION

[0073] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0074] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0075] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, the term "according to" used in this article is not limited to being based on a certain object. For example, determining B based on A can mean: determining B based only on A, or determining B based partially on A.

[0076] In the related art, the stacking of upper and lower stacking objects is completed based on the posture of the lower stacking object relative to the handling equipment, without paying attention to the influence of various factors such as inaccurate posture of the upper stacking object, uneven ground, cumulative error of the odometer, and error of the handling equipment itself, which causes the upper and lower stacking objects to be misaligned during stacking, affecting operation safety.

[0077] In view of this, the present application provides a new idea. In order to facilitate the understanding of the present application, the system schematic diagram of the handling equipment on which the present application is based is first described. Figure 1 A schematic diagram of an exemplary transport device to which the present invention can be applied is shown in FIG. Figure 1 As shown in FIG. 1 , the transport device 100 includes a transport device body 101 , a sensor 102 , a stacking execution component 103 , a controller 104 , and a memory 105 .

[0078] The handling equipment 100 involved in the embodiment of the present application may be an unmanned forklift, a pallet truck, a crane, an AGV (Automated Guided Vehicle), an AMR (Automatic Mobile Robot), a humanoid robot, etc. The corresponding stacking execution component 103 may be a fork, a robotic arm, etc.

[0079] The controller 104 is used to control the stacking execution component 103 of the transport device 100 to execute the stacking of the first stacking object and the second stacking object. The controller 104 can be a control mainboard, a control box, a control unit, a vehicle-mounted computer, a computing platform, a tablet computer, a computer, etc. on the transport device body 101, or a system or device that performs a computing or controlling function in a local server or a cloud server, or can be a handheld controller, a remote controller, etc., etc. This is not limited in the embodiments of the present application.

[0080] The sensor 102 may be in the form of a sensor module, including at least a radar for collecting point clouds, such as a laser radar.

[0081] First, the concepts of terms involved in the embodiments of the present application are introduced.

[0082] Stacking refers to arranging and stacking several objects up and down according to certain rules.

[0083] Stacking objects refer to the objects involved in stacking, which can be the goods themselves, or goods with simple packaging such as wrapping, or containers that can hold and carry goods, such as cages, wooden boxes, plastic boxes, pallets, etc.

[0084] Stacking process: refers to the handling device lifting the first stacking object, so that the first stacking object moves and approaches the second stacking object, and then aligns the first stacking object with the second stacking object by adjusting the posture of the handling device, and then places the first stacking object on the second stacking object to complete the stacking.

[0085] The first stacking object refers to the stacking object located at the top (such as an upper cage), and the second stacking object refers to the stacking object located at the bottom (such as a lower cage).

[0086] Alignment state: refers to the state in which two or more stacked objects are arranged in a vertical direction, and at least part of the border lines between the stacked objects are parallel or overlapped. The vertical direction refers to the Z axis in the coordinate system of the handling equipment (such as Figure 1 and Figure 2 The Z-axis direction.

[0087] like Figure 2 The geometric center of the transport equipment can be taken as the O point of the coordinate system, the front and rear travel direction of the transport equipment (i.e., the longitudinal direction of the transport equipment body 101) can be taken as the X-axis, wherein the positive direction of the X-axis is the direction away from the transport equipment accessories (such as forks), the height direction of the transport equipment can be taken as the Z-axis, the lateral direction of the transport equipment body 101 can be taken as the Y-axis, and the positive direction of the Y-axis is perpendicular to the paper surface and outward.

[0088] First point cloud: refers to the point cloud of the first target area of ​​the first stacking object acquired by the sensor. The first point cloud is matched with the first template point cloud to obtain the position and posture of the first stacking object.

[0089] Second point cloud: refers to the point cloud of the second target area of ​​the first stacked object acquired by the sensor. The second point cloud is matched with the second template point cloud to obtain the position and posture of the second stacked object.

[0090] A first stack object has a first target area (such as Figure 2 In A1), the first target area refers to the key structural area on the first stacking object that contacts the second stacking object after stacking. The first target area can be the key structural area such as the foot cup, frame, corner structure, etc. of the first stacking object.

[0091] A second stack object having a second target area (such as Figure 2 In B1), the second target area refers to the key structural area on the second stacking object that contacts the first stacking object after stacking. The second target area can be key structural areas such as columns, frames, and corner structural areas.

[0092] The third stacking object has the same structure and size as the first stacking object, and has a key structure that contacts with the fourth stacking object after being stacked, and the structure may be a foot cup, a frame, a corner structure, etc. of the third stacking object.

[0093] The fourth stacking object has the same structure and size as the second stacking object, and has a key structural area that contacts the third stacking object after being stacked. The structure may be a column, a frame, a corner structural area, etc. of the fourth stacking object.

[0094] Target point cloud: refers to the point cloud data of the target area acquired by the sensor.

[0095] Standard point cloud: The point cloud obtained after processing the original point cloud.

[0096] Template point cloud: refers to a predefined point cloud model used for comparison, registration or recognition of other point clouds.

[0097] The first template point cloud refers to a point cloud that at least includes a portion of the key structural area of ​​the third stacking object when the third stacking object and the fourth stacking object are aligned.

[0098] The second template point cloud refers to a point cloud that at least includes a portion of the key structural area of ​​the fourth stacking object when the third stacking object and the fourth stacking object are aligned.

[0099] Figure 3 A flowchart of a method for confirming an alignment state provided in an embodiment of the present application, the method can be performed by Figure 1 The handling equipment in the system shown is executed. Figure 3As shown in , the method may include the following steps:

[0100] Step 301: The controller obtains target point clouds of the first stacking object and the second stacking object through a sensor.

[0101] Step 303: The controller extracts a first point cloud of a first target area of ​​the first stacking object and a second point cloud of a second target area of ​​the second stacking object from the target point cloud.

[0102] Step 305: The controller matches the first point cloud with the first template point cloud to obtain the posture of the first stacked object, and matches the second point cloud with the second template point cloud to obtain the posture of the second stacked object.

[0103] Step 307: The controller determines a difference between the posture of the first stacked object and the posture of the second stacked object, and compares the difference with a threshold value to confirm an alignment state of the first stacked object and the second stacked object.

[0104] It can be seen from the above process that the controller of the present application can determine the difference between the posture of the first stacking object and the posture of the second stacking object during the stacking process through the above method, and confirm the alignment state between the first stacking object and the second stacking object based on the difference and the threshold. This scheme can avoid the influence of the external environment (such as uneven ground) and the errors of the handling equipment itself, and can accurately calculate the posture data of the first stacking object and the second stacking object, and then determine the alignment state of the two.

[0105] The following is a detailed description of each step in the above process and the effects that can be further produced in conjunction with the embodiments. It should be noted that the "first" and "second" involved in the present disclosure do not have restrictions on size, order, and quantity, and are only used to distinguish in name, such as "first stacking object" and "second stacking object" are used to distinguish two objects. For another example, "first point cloud" and "second point cloud" are used to distinguish two point clouds.

[0106] First, the above step 301, namely "the controller obtains the target point cloud of the first stacking object and the second stacking object through the sensor" is described in detail in conjunction with the embodiment.

[0107] First, a brief introduction to the stacking scenarios involved in the embodiments of the present application is given. Figure 2 As shown, when the handling device receives a handling task of stacking the first stacking object A' onto the second stacking object B', the controller (such as Figure 1The controller 104 shown controls the fork of the handling device to pick up the first stacking object A', then controls the handling device to move to the vicinity of the second stacking object B', and then controls the posture of the handling device to align the first target area A1 of the first stacking object A' and the second target area B1 of the second stacking object B' up and down, and then stacks the first stacking object A' on the second stacking object B'. When the controller controls the handling device to stack the first stacking object A' on the second stacking object B', the first stacking object A' and the second stacking object B' are not aligned due to the external environment (such as uneven ground) and the error of the handling device itself. The embodiment of the present application is a solution proposed to solve this problem.

[0108] In the embodiment of the present application, multiple sensors can be installed according to different types of sensors to simultaneously scan the first stacked object and the second stacked object.

[0109] Among them, the odometer estimates the distance moved by measuring the movement of the transport equipment, and usually combines the data collected by the sensor to calculate physical quantities such as the position, speed and posture of the transport equipment.

[0110] The sensor can be arranged at a preset distance below the midpoint between the fork arms of the handling device (for example, on the vehicle body or on the fork arm structure), or at other locations according to actual conditions. The odometer can be arranged near the wheels of the handling device to record the number of rotations of the wheels and estimate the distance moved; the odometer can also be arranged at the center of the chassis of the handling device.

[0111] For example, the above-mentioned sensors and odometers can be controlled by a controller built into the transport equipment (such as Figure 1 The controller 104 in the embodiment is controlled, for example, based on a SoC (System-on-a-Chip), which is not specifically limited in the embodiments of the present application.

[0112] In the embodiment of the present application, when the handling device lifts the first stacked object, and the sensor simultaneously scans the first stacked object and the second stacked object, the controller obtains the target point cloud of the first stacked object and the second stacked object through the sensor, specifically:

[0113] The transport device transports the first stacking object to the stacking operation position, and lifts the first stacking object. When the sensor scans the first stacking object and the second stacking object, the sensor obtains the target point cloud of the first stacking object and the target point cloud of the second stacking object.

[0114] Here, the stacking operation position may be an area where the first stacking object and the second stacking object are to be stacked, and in this area, the fork of the handling device can stack the first stacking object and the second stacking object within the variable posture range of the fork. For example, according to the position of the second stacking object, a position within a preset distance range in front of the second stacking object is used as the stacking operation position.

[0115] The sensor involved in the embodiment of the present application may include a radar module, wherein the radar module may include at least one radar. The field of view of the radar can cover the target area of ​​the first pair of stacked objects and the target area of ​​the second stacked object. Therefore, the radar can obtain a target point cloud covering the first stacked object and the second stacked object. Optionally, the radar is a laser radar (Lidar). The laser radar can be a three-dimensional laser radar, etc.

[0116] It should be noted that the embodiments of the present application can be applied to scenarios with multiple stacked objects.

[0117] The above step 303, namely "the controller extracts a first point cloud of a first target area of ​​a first stacking object and a second point cloud of a second target area of ​​a second stacking object from the target point cloud" is described in detail below in conjunction with the embodiments.

[0118] In the embodiment of the present application, after the controller obtains the target point cloud of the first stacking object and the target point cloud of the second stacking object, it is necessary to extract the first point cloud of the first stacking object from the target point cloud of the first stacking object, and to extract the second point cloud of the second stacking object from the target point cloud of the second stacking object.

[0119] Optionally, the first point cloud may be a point cloud in a first target area of ​​the first stacked object, and the second point cloud may be a point cloud in a second target area of ​​the second stacked object. When the alignment state of the first stacked object and the second stacked object is aligned, the first target area and the second target area are in a vertical direction (such as Figure 2 When the alignment state of the first stacking object and the second stacking object is aligned, the controller controls the transport device to place the first stacking object on the second stacking object to complete the stacking.

[0120] Specifically, the controller can obtain pre-stored specification parameters (such as size, shape, etc.) of the first stacking object and the second stacking object from the memory, and then determine the first point cloud of the first target area and the second point cloud of the second target area from the point cloud based on the specification parameters.

[0121] In one example, the first target area includes at least one corner structure area of ​​the first stacked object or a partial area of ​​at least one foot cup. The second target area includes at least one corner structure area of ​​the second stacked object or a partial area of ​​at least one column. Figure 4 , Figure 5 and Figure 6 Give a description.

[0122] exist Figure 4 In the embodiment, the first target area includes the first foot cup aj1' of the first stacking object A', the third foot cup aj3' of the first stacking object A', the fifth foot cup aj5' of the first stacking object A' and the seventh foot cup aj7' of the first stacking object A'. The second target area includes the second column bj2' of the second stacking object B', the fourth column bj4' of the second stacking object B', the sixth column bj6' of the second stacking object B' and the eighth column bj8' of the second stacking object B'.

[0123] exist Figure 5 In the embodiment, the first target area A1 includes the first cup aj1' of the first stacking object A' and the third cup aj3' of the first stacking object A'. The second target area B1 includes the second column bj2' of the second stacking object B' and the fourth column bj4' of the second stacking object B'.

[0124] exist Figure 6 In the figure, the first corner structure area a1' of the first stacking object A', the third corner structure area a3' of the first stacking object A', the fifth corner structure area a5' of the first stacking object A' and the seventh corner structure area a7' of the first stacking object A' are marked in the form of a rectangle; and the second corner structure area b2' of the second stacking object B', the fourth corner structure area b4' of the second stacking object B', the sixth corner structure area b6' of the second stacking object B' and the eighth corner structure area b8' of the second stacking object B' are marked.

[0125] The first target area includes the first corner structure area a1' of the first stacked object A', the third corner structure area a3' of the first stacked object A', the fifth corner structure area a5' of the first stacked object A' and the seventh corner structure area a7' of the first stacked object A'. The second target area includes the second corner structure area b2' of the second stacked object B', the fourth corner structure area b4' of the second stacked object B', the sixth corner structure area b6' of the second stacked object B' and the eighth corner structure area b8' of the second stacked object B'.

[0126] Among them, the first corner structure area a1' of the first stacking object A' has a first side line a11' and a third side line a13' of the first stacking object A', and the first side line a11' of the first stacking object A' intersects with the third side line a13'; the third corner structure area a3' of the first stacking object A' has a third side line a13' and a fifth side line a15' of the first stacking object A', and the third side line a13' of the first stacking object A' intersects with the fifth side line a15'; the fifth corner structure area a5' of the first stacking object A' has a fifth side line a15' and a seventh side line a17' of the first stacking object A', and the fifth side line a15' of the first stacking object A' intersects with the seventh side line a17'; the seventh corner structure area a7' of the first stacking object A' has a seventh side line a17' of the first stacking object A' and the first side line a11', and the seventh side line a17' of the first stacking object A' intersects with the first side line a11'.

[0127] Among them, the second corner structure area b2' of the second stacking object B' has the second side line b22' and the fourth side line b24' of the second stacking object B', and the second side line b22' of the second stacking object B' intersects with the fourth side line b24'; the fourth corner structure area b4' of the second stacking object B' has the fourth side line b24' and the sixth side line b26' of the second stacking object B', and the fourth side line b24' of the second stacking object B' intersects with the sixth side line b26'; the sixth corner structure area b6' of the second stacking object B' has the sixth side line b26' and the eighth side line b28' of the second stacking object B', and the sixth side line b26' of the second stacking object B' intersects with the eighth side line b28'; the eighth corner structure area b8' of the second stacking object B' has the eighth side line b28' and the second side line b22' of the second stacking object B', and the eighth side line b28' of the second stacking object B' intersects with the second side line b22'.

[0128] For example, the first stacking object is the first material basket; the first target area is located at the bottom of the first material basket, and the first target area includes at least one corner structure area or a partial area of ​​at least one foot cup. For example, the first target area includes at least one of the first corner structure area of ​​the first material basket, the third corner structure area of ​​the first material basket, the fifth corner structure area of ​​the first material basket, and the seventh corner structure area of ​​the first material basket; or, the first target area includes at least one of the partial area of ​​the first foot cup of the first material basket, the partial area of ​​the third foot cup of the first material basket, the partial area of ​​the fifth foot cup of the first material basket, and the partial area of ​​the seventh foot cup of the first material basket.

[0129] For example, the second stacking object is a second material basket; the second target area is located at the top of the second material basket, and the second target area includes at least one corner structure area of ​​the second material basket or a partial area of ​​at least one column. For example, the second target area includes at least one of the second corner structure area of ​​the second material basket, the fourth corner structure area of ​​the second material basket, the sixth corner structure area of ​​the second material basket, and the eighth corner structure area of ​​the second material basket; or, the second target area includes at least one of the partial area of ​​the second column of the second material basket, the partial area of ​​the fourth column of the second material basket, the partial area of ​​the sixth column of the second material basket, and the partial area of ​​the eighth column of the second material basket.

[0130] It should be noted that the partial area of ​​the above-mentioned foot cup may refer to the area that at least includes part or all of the foot cup. The partial area of ​​the above-mentioned column may refer to the area that at least includes part or all of the column. Among them, the foot cup refers to the supporting component installed at the bottom of the cage, which is usually used to stabilize the cage, bear the weight, and protect the cage from direct contact with the ground. In industrial, warehousing and logistics scenarios, cages (also called turnover cages or storage cages) are often equipped with foot cups to achieve better mobility, stacking and durability. The column refers to the vertical support structure around the cage, which is usually used to bear the weight of the cage, fix the cage frame structure and provide stacking function. The column is one of the core components of the cage, and its design has a direct impact on the strength, stability and use function of the cage.

[0131] Continuing from the above, in the process of determining the first point cloud based on the target point cloud of the first stacking object, the target point cloud of the first stacking object can also be preprocessed by filtering, anomaly removal, data cleaning and data transformation, and the point cloud of the first target area can be extracted from the preprocessed point cloud as the first point cloud.

[0132] This application preprocesses the point cloud, such as filtering, anomaly removal, data cleaning and data transformation, etc. This preprocessing operation can remove noise points in the point cloud and retain valid data for subsequent use. This can further improve the accuracy and reliability of the point cloud.

[0133] The above step 305, i.e., "the controller matches the first point cloud with the first template point cloud to obtain the posture of the first stacking object, and matches the second point cloud with the second template point cloud to obtain the posture of the second stacking object" is described in detail below in conjunction with the embodiments.

[0134] In the embodiment of the present application, the first point cloud and the first template point cloud are converted to the same coordinate system (such as Figure 2 The first point cloud after the coordinate system conversion is matched with the first template point cloud in the coordinate system where the handling equipment shown in the figure is located to obtain the posture of the first stacking object. And the second point cloud and the second template point cloud are converted to the same coordinate system, and the second point cloud after the coordinate system conversion is matched with the second template point cloud to obtain the posture of the second stacking object.

[0135] In one example, point cloud matching may be performed using nearest neighbor search, ICP (Iterative Closest Point), or the like.

[0136] The embodiment of the present application can pre-obtain the first template point cloud and the second template point cloud based on the following steps: performing point cloud collection on the third stacked object and the fourth stacked object that meet the alignment posture standard to obtain the standard point cloud of the third stacked object and the standard point cloud of the fourth stacked object; then, extracting point clouds from the standard point cloud of the third stacked object and the standard point cloud of the fourth stacked object respectively to pre-obtain the first template point cloud and the second template point cloud.

[0137] Here, the alignment posture standard is a standard for measuring whether the third stacking object and the fourth stacking object are aligned based on the postures of the third stacking object and the fourth stacking object when they are stacked. The standard point cloud of the third stacking object and the standard point cloud of the fourth stacking object are point clouds collected by the sensor carried by the handling equipment when the postures of the third stacking object and the fourth stacking object when they are stacked meet the alignment posture standard.

[0138] For example, the third stacking object and the fourth stacking object may be pre-stacked and aligned, and then point cloud collection may be performed on the third stacking object and the fourth stacking object in the stacked and aligned state, and the template point cloud (including the corresponding position and posture) of the third stacking object and the template point cloud (including the corresponding position and posture) of the fourth stacking object may be respectively extracted from the collected point clouds. When the first stacking object and the second stacking object of the same structure and size are subsequently used for stacking detection, the template point cloud of the third stacking object and the template point cloud of the fourth stacking object extracted in advance may be used to determine the position and posture of the first stacking object and the position and posture of the second stacking object respectively.

[0139] The embodiment of the present application performs a point cloud collection once for the third stacking object and the fourth stacking object that meet the alignment posture standard, and can respectively extract the first template point cloud and the second template point cloud from the standard point cloud of the third stacking object and the standard point cloud of the fourth stacking object, without the need for frequent dimensional measurements, thereby simplifying the data collection process.

[0140] In order to improve the quality of point clouds, before obtaining the standard point cloud of the third stacking object and the standard point cloud of the fourth stacking object, the original point cloud of the third stacking object and the original point cloud of the fourth stacking object collected by the sensor can be preprocessed, such as filtering, denoising and other preprocessing, to obtain the standard point cloud of the third stacking object and the standard point cloud of the fourth stacking object; and point clouds are respectively extracted from the standard point cloud of the third stacking object and the preprocessed standard point cloud of the fourth stacking object to obtain the first template point cloud and the second template point cloud in advance.

[0141] exist Figure 7 In the example of obtaining the first template point cloud, the method specifically includes:

[0142] Step 701: The controller obtains an original point cloud of a third stacking object through a sensor;

[0143] Step 702: The controller pre-processes the original point cloud to obtain a standard point cloud of the third stacking object;

[0144] Step 703: extracting a point cloud of at least one corner structure region from the standard point cloud of the third stacked object as a first template point cloud.

[0145] exist Figure 8 In the example of obtaining the second template point cloud, the following is specifically included:

[0146] Step 801: The controller obtains an original point cloud of a fourth stacking object through a sensor;

[0147] Step 802: The controller pre-processes the original point cloud to obtain a standard point cloud of the fourth stacking object;

[0148] Step 803: extracting a point cloud of at least one corner structure region from the standard point cloud of the fourth stacked object as a second template point cloud.

[0149] In the embodiment of the present application, when performing point cloud matching between the first point cloud and the first template point cloud, all or part of the standard point cloud of the third stacking object (such as a border point cloud with significant significance, a point cloud of at least one corner structure area, such as a point cloud corresponding to a partial area of ​​the foot cup (or column)) can be used as the first template point cloud.

[0150] In an embodiment of the present application, a point cloud of at least one corner structure area of ​​the third stacked object is extracted from the standard point cloud of the third stacked object as a first template point cloud; and / or, a point cloud of at least one corner structure area of ​​the fourth stacked object is extracted from the standard point cloud of the fourth stacked object as a second template point cloud.

[0151] Among them, the at least one corner structure area of ​​the third stacked object includes at least one of the first corner structure area of ​​the third stacked object, the third corner structure area of ​​the third stacked object, the fifth corner structure area of ​​the third stacked object and the seventh corner structure area of ​​the third stacked object.

[0152] The at least one corner structure region of the fourth stacked object includes at least one of the second corner structure region of the fourth stacked object, the fourth corner structure region of the fourth stacked object, the sixth corner structure region of the fourth stacked object, and the eighth corner structure region of the fourth stacked object.

[0153] In one example, the first template point cloud includes a point cloud of a first corner structure region of the third stacked object extracted from a standard point cloud of the third stacked object.

[0154] The second template point cloud includes: a point cloud of a second corner structure region of the fourth stacked object extracted from the standard point cloud of the fourth stacked object.

[0155] In another example, the first template point cloud includes: point clouds of a first corner structure region and a third corner structure region of the third stacked object extracted from a standard point cloud of the third stacked object;

[0156] The second template point cloud also includes: point clouds of the second corner structure region and the fourth corner structure region of the fourth stacked object extracted from the standard point cloud of the fourth stacked object.

[0157] In another example, the first template point cloud includes: a point cloud of the first corner structure area of ​​the third stacked object, a point cloud of the third corner structure area of ​​the third stacked object, a point cloud of the fifth corner structure area of ​​the third stacked object, and a point cloud of the seventh corner structure area of ​​the third stacked object extracted from the standard point cloud of the third stacked object. The second template point cloud includes: a point cloud of the second corner structure area of ​​the fourth stacked object, a point cloud of the fourth corner structure area of ​​the fourth stacked object, a point cloud of the sixth corner structure area of ​​the fourth stacked object, and a point cloud of the eighth corner structure area of ​​the fourth stacked object extracted from the standard point cloud of the fourth stacked object. Fig. 9 Give a description.

[0158] exist Fig. 9 In the figure, the first corner structure area a1 of the third stacking object A, the third corner structure area a3 of the third stacking object A, the fifth corner structure area a5 of the third stacking object A and the seventh corner structure area a7 of the third stacking object A are marked in the form of rectangles; and the second corner structure area b2 of the fourth stacking object B, the fourth corner structure area b4 of the fourth stacking object B, the sixth corner structure area b6 of the fourth stacking object B and the eighth corner structure area b8 of the fourth stacking object B are marked.

[0159] Among them, the first corner structure area a1 of the third stacking object A has the first side line a11 and the third side line a13 of the third stacking object A, and the first side line a11 of the third stacking object A intersects with the third side line a13; the third corner structure area a3 of the third stacking object A has the third side line a13 and the fifth side line a15 of the third stacking object A, and the third side line a13 of the third stacking object A intersects with the fifth side line a15; the fifth corner structure area a5 of the third stacking object A has the fifth side line a15 and the seventh side line a17 of the third stacking object A, and the fifth side line a15 of the third stacking object A intersects with the seventh side line a17; the seventh corner structure area a7 of the third stacking object A has the seventh side line a17 of the third stacking object A and the first side line a11, and the seventh side line a17 of the third stacking object A intersects with the first side line a11.

[0160] Among them, the second corner structure area b2 of the fourth stacking object B has the second side line b22 and the fourth side line b24 of the fourth stacking object B, and the second side line b22 of the fourth stacking object B intersects with the fourth side line b24; the fourth corner structure area b4 of the fourth stacking object B has the fourth side line b24 and the sixth side line b26 of the fourth stacking object B, and the fourth side line b24 of the fourth stacking object B intersects with the sixth side line b26; the sixth corner structure area b6 of the fourth stacking object B has the sixth side line b26 and the eighth side line b28 of the fourth stacking object B, and the sixth side line b26 of the fourth stacking object B intersects with the eighth side line b28; the eighth corner structure area b8 of the fourth stacking object B has the eighth side line b28 and the second side line b22 of the fourth stacking object B, and the eighth side line b28 of the fourth stacking object B intersects with the second side line b22.

[0161] In an embodiment of the present application, a point cloud of a partial area of ​​at least one foot cup of the third stacked object is extracted from the standard point cloud of the third stacked object as a first template point cloud; and / or a point cloud of a partial area of ​​at least one column of the fourth stacked object is extracted from the standard point cloud of the fourth stacked object as a second template point cloud.

[0162] Among them, the partial area of ​​at least one foot cup of the third stack object includes at least one of the partial area of ​​the first foot cup of the third stack object, the partial area of ​​the third foot cup of the third stack object, the partial area of ​​the fifth foot cup of the third stack object, and the partial area of ​​the seventh foot cup of the third stack object.

[0163] The partial area of ​​at least one column of the fourth stacking object includes at least one of the partial area of ​​the second column of the fourth stacking object, the partial area of ​​the fourth column of the fourth stacking object, the partial area of ​​the sixth column of the fourth stacking object, and the partial area of ​​the eighth column of the fourth stacking object.

[0164] In one example, the first template point cloud includes a point cloud of a partial area of ​​the first cup of the third stacked object extracted from a standard point cloud of the third stacked object.

[0165] The second template point cloud includes: a point cloud of a partial area of ​​a second column of the fourth stacking object extracted from the standard point cloud of the fourth stacking object.

[0166] In another example, the first template point cloud includes: point clouds of a partial area of ​​a first foot cup of the third stacked object and a partial area of ​​a third foot cup of the third stacked object extracted from a standard point cloud of the third stacked object;

[0167] The second template point cloud also includes: point clouds of a partial area of ​​the second column of the fourth stacked object and a partial area of ​​the fourth column of the fourth stacked object extracted from the standard point cloud of the fourth stacked object.

[0168] In another example, the first template point cloud includes: point clouds of partial areas of the first foot cup of the third stacked object, partial areas of the third foot cup of the third stacked object, partial areas of the fifth foot cup of the third stacked object, and partial areas of the seventh foot cup of the third stacked object, extracted from the standard point cloud of the third stacked object; the second template point cloud includes: point clouds of partial areas of the second column of the fourth stacked object, partial areas of the fourth column of the fourth stacked object, partial areas of the sixth column of the fourth stacked object, and partial areas of the eighth column of the fourth stacked object, extracted from the standard point cloud of the fourth stacked object. Fig.10 Give a description.

[0169] exist Fig.10 In the embodiment, at least one foot cup of the third stacking object A includes a first foot cup aj1 of the third stacking object A, a third foot cup aj3 of the third stacking object A, a fifth foot cup aj5 of the third stacking object A and a seventh foot cup aj7 of the third stacking object A.

[0170] exist Fig.10 In the embodiment, at least one column of the fourth stacking object B includes a second column bj2 of the fourth stacking object B, a fourth column bj4 of the fourth stacking object B, a sixth column bj6 of the fourth stacking object B and an eighth column bj8 of the fourth stacking object B.

[0171] The matching process between the first point cloud and the first template point cloud is described in detail below.

[0172] In the embodiment of the present application, point cloud matching of the first point cloud with the first template point cloud may include the following methods: matching the first point cloud of at least one corner structure area of ​​the first stacked object with the first template point cloud; or, matching the first point cloud of a partial area of ​​at least one foot cup of the first stacked object with the first template point cloud; or, matching the first point cloud with the first template point cloud of at least one corner structure area of ​​the third stacked object (i.e., the point cloud of at least one corner structure area of ​​the third stacked object extracted from the standard point cloud of the third stacked object); or, matching the first point cloud with the first template point cloud of a partial area of ​​at least one foot cup of the third stacked object; or, matching the first point cloud with the first template point cloud of a partial area of ​​at least one foot cup of the third stacked object; or, matching the first point cloud of at least one corner structure area of ​​the first stacked object with the first template point cloud of at least one corner structure area of ​​the third stacked object; or, matching the first point cloud of a partial area of ​​at least one foot cup of the first stacked object with the first template point cloud of a partial area of ​​at least one foot cup of the third stacked object. These matching methods can also reduce the amount of processing when matching the first point cloud with the first template point cloud.

[0173] Among them, matching the first point cloud of at least one corner structure area of ​​the first stacked object with the first template point cloud of at least one corner structure area of ​​the third stacked object includes at least one of the following: matching the first point cloud of the first corner structure area of ​​the first stacked object with the first template point cloud of the first corner structure area of ​​the third stacked object; matching the first point cloud of the third corner structure area of ​​the first stacked object with the first template point cloud of the third corner structure area of ​​the third stacked object; matching the first point cloud of the fifth corner structure area of ​​the first stacked object with the first template point cloud of the fifth corner structure area of ​​the third stacked object; matching the first point cloud of the seventh corner structure area of ​​the first stacked object with the first template point cloud of the seventh corner structure area of ​​the third stacked object.

[0174] The embodiment of the present application matches the first point cloud of at least one corner structure area of ​​the first stacking object with the first template point cloud of at least one corner structure area of ​​the third stacking object. While ensuring the matching accuracy, it can further reduce the processing amount during matching, improve the matching speed, and thus speed up the stacking of the first stacking object and the second stacking object.

[0175] Among them, matching the first point cloud of a partial area of ​​at least one foot cup of the first stacked object with the first template point cloud of a partial area of ​​at least one foot cup of the third stacked object includes at least one of the following: matching the first point cloud of a partial area of ​​the first foot cup of the first stacked object with the first template point cloud of a partial area of ​​the first foot cup of the third stacked object; matching the first point cloud of a partial area of ​​the third foot cup of the first stacked object with the first template point cloud of a partial area of ​​the third foot cup of the third stacked object; matching the first point cloud of a partial area of ​​the fifth foot cup of the first stacked object with the first template point cloud of a partial area of ​​the fifth foot cup of the third stacked object; matching the first point cloud of a partial area of ​​the seventh foot cup of the first stacked object with the first template point cloud of a partial area of ​​the seventh foot cup of the third stacked object.

[0176] The embodiment of the present application matches the first point cloud of a partial area of ​​at least one foot cup of the first stacking object with the first template point cloud of a partial area of ​​at least one foot cup of the third stacking object. While ensuring the matching accuracy, it can further reduce the processing amount during matching, improve the matching speed, and thus speed up the stacking of the first stacking object and the second stacking object.

[0177] The matching process between the second point cloud and the second template point cloud is described in detail below.

[0178] In the embodiment of the present application, matching the second point cloud with the second template point cloud includes the following methods: matching the second point cloud of at least one corner structure area of ​​the second stacked object with the second template point cloud; or, matching the second point cloud of a partial area of ​​at least one column of the second stacked object with the second template point cloud; or, matching the second point cloud with the second template point cloud of at least one corner structure area of ​​the fourth stacked object; or, matching the second point cloud with the second template point cloud of a partial area of ​​at least one column of the fourth stacked object; or, matching the second point cloud of at least one corner structure area of ​​the second stacked object with the second template point cloud of at least one corner structure area of ​​the fourth stacked object; or, matching the second point cloud of a partial area of ​​at least one column of the second stacked object with the second template point cloud of a partial area of ​​at least one column of the fourth stacked object. These matching methods can also reduce the amount of processing when matching the second point cloud with the second template point cloud.

[0179] Among them, matching the second point cloud of at least one corner structure area of ​​the second stacked object with the second template point cloud of at least one corner structure area of ​​the fourth stacked object includes at least one of the following: matching the second point cloud of the second corner structure area of ​​the second stacked object with the second template point cloud of the second corner structure area of ​​the fourth stacked object; matching the second point cloud of the fourth corner structure area of ​​the second stacked object with the second template point cloud of the fourth corner structure area of ​​the fourth stacked object; matching the second point cloud of the sixth corner structure area of ​​the second stacked object with the second template point cloud of the sixth corner structure area of ​​the fourth stacked object; matching the second point cloud of the eighth corner structure area of ​​the second stacked object with the second template point cloud of the eighth corner structure area of ​​the fourth stacked object.

[0180] The embodiment of the present application matches the second point cloud of at least one corner structure area of ​​the second stacking object with the second template point cloud of at least one corner structure area of ​​the fourth stacking object. While ensuring the matching accuracy, it can further reduce the processing amount during matching, improve the matching speed, and thus speed up the stacking of the first stacking object and the second stacking object.

[0181] Among them, matching the second point cloud of the partial area of ​​at least one column of the second stacking object with the second template point cloud of the partial area of ​​at least one column of the fourth stacking object includes at least one of the following: matching the second point cloud of the partial area of ​​the second column of the second stacking object with the second template point cloud of the partial area of ​​the second column of the fourth stacking object; matching the second point cloud of the partial area of ​​the fourth column of the second stacking object with the second template point cloud of the partial area of ​​the fourth column of the fourth stacking object; matching the second point cloud of the partial area of ​​the sixth column of the second stacking object with the second template point cloud of the partial area of ​​the sixth column of the fourth stacking object; matching the second point cloud of the partial area of ​​the eighth column of the second stacking object with the second template point cloud of the partial area of ​​the eighth column of the fourth stacking object.

[0182] The embodiment of the present application matches the second point cloud of a partial area of ​​at least one column of the second stacking object with the second template point cloud of a partial area of ​​at least one column of the fourth stacking object. While ensuring the matching accuracy, it can further reduce the processing amount during matching, improve the matching speed, and thus speed up the stacking of the first stacking object and the second stacking object.

[0183] Combine the following Fig.11 and Fig.12 A process of obtaining a position and posture of a first stacked object and a second stacked object in one embodiment is described.

[0184] exist Fig.11In the figure, the first corner structure area a1' of the first stacking object A', the third corner structure area a3' of the first stacking object A', the fifth corner structure area a5' of the first stacking object A' and the seventh corner structure area a7' of the first stacking object A' are marked in the form of a rectangle; and the second corner structure area b2' of the second stacking object B', the fourth corner structure area b4' of the second stacking object B', the sixth corner structure area b6' of the second stacking object B' and the eighth corner structure area b8' of the second stacking object B' are marked.

[0185] exist Fig.11 In the figure, the yellow color in the first stacking object A' represents the first point cloud PC1, and the blue color represents the first template point cloud TM1; the yellow color in the second stacking object B' represents the second point cloud PC2, and the red color represents the second template point cloud TM2.

[0186] Match the first template point cloud TM1 corresponding to the first corner structure area a1 of the third stacking object A with the first point cloud PC1 corresponding to the first corner structure area a1' of the first stacking object A', match the first template point cloud TM1 corresponding to the third corner structure area a3 of the third stacking object A with the first point cloud PC1 corresponding to the third corner structure area a3' of the first stacking object A', match the first template point cloud TM1 corresponding to the fifth corner structure area a5 of the third stacking object A with the first point cloud PC1 corresponding to the fifth corner structure area a5' of the first stacking object A', and match the first template point cloud TM1 corresponding to the seventh corner structure area a7 of the third stacking object A with the first point cloud PC1 corresponding to the seventh corner structure area a7' of the first stacking object A' to obtain the posture of the first stacking object A'.

[0187] exist Fig.12 In the embodiment, matching the second template point cloud TM2 with the second point cloud PC2 means: matching the second template point cloud TM2 corresponding to the second corner structure area b2 of the fourth stacking object B with the second point cloud PC2 corresponding to the second corner structure area b2' of the second stacking object B', matching the second template point cloud TM2 corresponding to the fourth corner structure area b4 of the fourth stacking object B with the second point cloud PC2 corresponding to the fourth corner structure area b4' of the second stacking object B', matching the second template point cloud TM2 corresponding to the sixth corner structure area b6 of the fourth stacking object B with the second point cloud PC2 corresponding to the sixth corner structure area b6' of the second stacking object B', and matching the second template point cloud TM2 corresponding to the eighth corner structure area b8 of the fourth stacking object B with the second point cloud PC2 corresponding to the eighth corner structure area b8' of the second stacking object B' to obtain the posture of the second stacking object B'.

[0188] In the embodiment of the present application, the third stacking object has the same structure and size as the first stacking object, and at least one corner structure area of ​​the third stacking object corresponds to the same position as at least one corner structure area of ​​the first stacking object. For example, the position of the first corner structure area on the third stacking object is the same as the position of the first corner structure area on the first stacking object. Geometric shapes of the same size can be used to mark the first corner structure area of ​​the first stacking object and the first corner structure area of ​​the third stacking object.

[0189] The fourth stacking object has the same structure and size as the second stacking object, and at least one corner structure area of ​​the fourth stacking object corresponds to the same position as at least one corner structure area of ​​the second stacking object. For example, the position of the second corner structure area on the fourth stacking object is the same as the position of the second corner structure area on the second stacking object. The second corner structure area of ​​the second stacking object and the second corner structure area of ​​the fourth stacking object can be marked with geometric shapes of the same size. Fig.12 and Fig.13 Give a description.

[0190] It should be noted that Fig.12 The solid black rectangle in the middle represents the corner structure region of the fourth stacked object, and the dashed red rectangle represents the corner structure region of the second stacked object.

[0191] like Fig.12 , which is a point cloud diagram before the second point cloud is matched with the second template point cloud; at this time, the cuboid (i.e., the red dotted cuboid) marking the second corner structure area b2' of the second stacking object does not overlap with the cuboid (i.e., the black cuboid) marking the second corner structure area b2 of the fourth stacking object; the cuboid (i.e., the red dotted cuboid) marking the fourth corner structure area b4' of the second stacking object does not overlap with the cuboid (i.e., the black cuboid) marking the fourth corner structure area b4 of the fourth stacking object; Note: The cuboid of the sixth corner structure area b6' of the second stacking object (i.e., the red dotted cuboid) does not overlap with the cuboid of the sixth corner structure area b6 of the fourth stacking object (i.e., the black cuboid); the cuboid of the eighth corner structure area b8' of the second stacking object (i.e., the red dotted cuboid) does not overlap with the cuboid of the eighth corner structure area b8 of the fourth stacking object (i.e., the black cuboid), and the yellow second point cloud PC2 and the red second template point cloud TM2 have not completed the match.

[0192] exist Fig.13, which is a point cloud diagram after the second point cloud and the second template point cloud are matched; at this time, the cuboid (i.e., the red dotted cuboid) marking the second corner structure area b2' of the second stacking object coincides with the cuboid (i.e., the black cuboid) marking the second corner structure area b2 of the fourth stacking object; the cuboid (i.e., the red dotted cuboid) marking the fourth corner structure area b4' of the second stacking object coincides with the cuboid (i.e., the black cuboid) marking the fourth corner structure area b4 of the fourth stacking object; the cuboid (i.e., the red dotted cuboid) marking the sixth corner structure area b6' of the second stacking object coincides with the cuboid (i.e., the black cuboid) marking the sixth corner structure area b6 of the fourth stacking object; the cuboid (i.e., the red dotted cuboid) marking the eighth corner structure area b8' of the second stacking object coincides with the cuboid (i.e., the black cuboid) marking the eighth corner structure area b8 of the fourth stacking object; and the yellow second point cloud PC2 matches the red second template point cloud TM2.

[0193] After the matching is completed, the position and posture of the second stacked object is determined according to the position and posture relationship before and after the matching; the position and posture of the first stacked object can be determined in the same way.

[0194] The above step 307, i.e., "the controller determines the difference between the posture of the first stacking object and the posture of the second stacking object, and compares the difference with a threshold value to confirm the alignment state of the first stacking object and the second stacking object" is described in detail below in conjunction with the embodiments.

[0195] In the embodiment of the present application, the controller can confirm the alignment state of the first stacked object and the second stacked object based on the difference between the posture of the first stacked object and the posture of the second stacked object and the threshold. When the difference is greater than or equal to the threshold, the alignment state is confirmed to be misaligned; when the difference is less than the threshold, the alignment state is confirmed to be aligned.

[0196] In one example, when the difference is greater than or equal to the threshold, the method further includes:

[0197] The controller controls the handling equipment to adjust its position;

[0198] The controller reacquires the target point cloud of the first stacked object and the second stacked object through the sensor;

[0199] The controller re-determines the difference between the posture of the first stacked object and the posture of the second stacked object;

[0200] The controller reconfirms the alignment status based on the difference until the difference is smaller than the threshold.

[0201] In one example, when the difference is less than a threshold, the method further includes:

[0202] The controller controls the handling device to place the first stacking object on the second stacking object to complete the stacking.

[0203] Here, the threshold is used to measure whether to control the handling device to perform stacking of the first stacking object and the second stacking object, and the threshold can be set according to the alignment accuracy of the first stacking object and the second stacking object.

[0204] Combine the following Figure 4 , Figure 5 and Figure 6 The alignment status is described as aligned respectively.

[0205] exist Figure 4 In the embodiment, if the difference between the posture of the first stacking object and the posture of the second stacking object is less than the threshold value, the alignment state is confirmed to be aligned; at this time, the partial area aj1' of the first foot cup of the first stacking object A' is aligned with the partial area bj2' of the second column of the second stacking object B', and the partial area aj3' of the third foot cup of the first stacking object A' is aligned with the partial area bj4' of the fourth column of the second stacking object B'; the partial area aj5' of the fifth foot cup of the first stacking object A' is aligned with the partial area bj6' of the sixth column of the second stacking object B', and the partial area aj7' of the seventh foot cup of the first stacking object A' is aligned with the partial area bj8' of the eighth column of the second stacking object B'.

[0206] exist Figure 5 If the difference between the posture of the first stacked object and the posture of the second stacked object is less than the threshold, the alignment state is confirmed to be aligned; at this time, the first target area A1 and the second target area B1 are arranged in the vertical direction, the partial area aj3' of the third foot cup of the first stacked object is aligned with the partial area bj4' of the fourth column of the second stacked object, and the partial area aj1' of the first foot cup of the first stacked object is aligned with the partial area bj2' of the second column of the second stacked object.

[0207] exist Figure 6In the embodiment, if the difference between the posture of the first stacking object and the posture of the second stacking object is less than the threshold value, the alignment state is confirmed to be aligned; at this time, the first target area is located at the bottom of the first stacking object A', the second target area is located at the top of the second stacking object B', the first target area and the second target area are arranged in a vertical direction, the first corner structure area a' of the first stacking object A' is aligned with the second corner structure area b2' of the second stacking object B', the third corner structure area a3' of the first stacking object A' is aligned with the fourth corner structure area b4' of the second stacking object B'; the fifth corner structure area a5' of the first stacking object A' is aligned with the fifth corner structure area a6' of the second stacking object B' The sixth corner structural area b6' of object B' is aligned, the seventh corner structural area a7' of the first stacked object A' is aligned with the eighth corner structural area b8' of the second stacked object B'; and, the first side line a11' of the first stacked object A' is aligned with the second side line b22' of the second stacked object B', and the third side line a13' of the first stacked object A' is aligned with the fourth side line b24' of the second stacked object B'; the fifth side line a15' of the first stacked object A' is aligned with the sixth side line b26' of the second stacked object B', and the seventh side line a17' of the first stacked object A' is aligned with the eighth side line b28' of the second stacked object B'.

[0208] In addition, in order to enhance the reliability of the alignment operation and avoid infinite loops of failed attempts due to abnormal conditions, the embodiment of the present application also proposes an alarm mechanism. Specifically, the controller counts the number of misalignments, and once the number of misalignments exceeds the preset number of misalignments or is misaligned within a preset time, an alarm prompt is output. This mechanism is intended to promptly alert the operator or the automatic control system of potential problems, so that corresponding intervention measures can be taken to ensure the continuity and safety of the alignment operation.

[0209] In the embodiment of the present application, before confirming the alignment state, the method further includes: the controller controls the transport device to transport the first stacking object to the stacking preparation position to complete a pre-alignment action relative to the second stacking object.

[0210] The stacking preparation position refers to the position before reaching the stacking operation position, at which the handling device can obtain the posture of the second stacking object. The stacking operation position refers to the position where the sensor on the handling device can simultaneously obtain the target data (such as the target image) of the first stacking object and the second stacking object.

[0211] Pre-alignment means: adjusting the position of the handling equipment so that the first stacking object and the second stacking object are basically aligned in the Y-axis direction. That is, the coordinate difference △Y of the first stacking object and the second stacking object in the Y-axis and the rotation angle difference △Ψ of the Z-axis are within the preset threshold, which can be flexibly adjusted according to different handling equipment and stacking objects, for example, -5cm<△Y<5cm, -3°<△Ψ<3°.

[0212] In the pre-alignment stage, the handling equipment moves with the first stacking object to the second stacking object, firstly lifts the fork to the optimal detection height, then detects the features such as the column of the second stacking object, calculates the posture of the column, and adjusts the chassis or fork to complete the pre-alignment.

[0213] The following describes an implementation of the method proposed in the embodiment of the present application in combination with an actual application scenario. Fig.14 As shown, the execution subject is a handling device, and the process of matching the corner structure area of ​​the first stacking object with the corner structure area of ​​the second stacking object is described as follows. The method may include the following:

[0214] Step 1401: The controller controls the fork of the handling device to pick up the first stacked object and perform closed-loop detection.

[0215] Step 1402: The controller obtains target point clouds of the first stacking object and the second stacking object through a sensor.

[0216] Step 1403: The controller extracts point clouds of four corner structure regions from the target point cloud of the first stacked object; and extracts point clouds of four corner structure regions from the target point cloud of the second stacked object.

[0217] Among them, the controller extracts point clouds of four corner structure areas from the target point cloud of the first stacking object, including: the controller extracts the point cloud of the first corner structure area of ​​the first stacking object, the point cloud of the third corner structure area, the point cloud of the fifth corner structure area and the point cloud of the seventh corner structure area from the target point cloud of the first stacking object.

[0218] Extracting point clouds of four corner structure areas from the target point cloud of the second stacked object includes: extracting point clouds of the second corner structure area, the fourth corner structure area, the sixth corner structure area and the eighth corner structure area of ​​the second stacked object from the target point cloud of the second stacked object.

[0219] Step 1404: The controller preprocesses the point clouds of the four corner structure areas extracted from the target point cloud of the first stacked object as the first point cloud; and preprocesses the point clouds of the four corner structure areas extracted from the target point cloud of the second stacked object as the second point cloud.

[0220] Step 1405: The controller matches the first point cloud with the first template point cloud to obtain the posture of the first stacked object, and matches the second point cloud with the second template point cloud to obtain the posture of the second stacked object.

[0221] Among them, the first template point cloud and the second template point cloud are determined in the following manner: the controller obtains the original point clouds of the third stacking object and the fourth stacking object that meet the alignment posture standard through the sensor; the controller preprocesses the original point clouds to obtain the standard point cloud of the third stacking object and the standard point cloud of the fourth stacking object; from the standard point cloud of the third stacking object, the point cloud of the first corner structure area of ​​the third stacking object, the point cloud of the third corner structure area of ​​the third stacking object, the point cloud of the fifth corner structure area of ​​the third stacking object, and the point cloud of the seventh corner structure area of ​​the third stacking object are extracted as the first template point cloud; from the standard point cloud of the fourth stacking object, the point cloud of the second corner structure area of ​​the fourth stacking object, the point cloud of the fourth corner structure area of ​​the fourth stacking object, the point cloud of the sixth corner structure area of ​​the fourth stacking object, and the point cloud of the eighth corner structure area of ​​the fourth stacking object are extracted as the second template point cloud.

[0222] Step 1406: The controller determines a difference between the pose of the first stacked object and the pose of the second stacked object.

[0223] Step 1407: Determine whether the difference is less than a threshold.

[0224] Step 1408 : When the difference is greater than or equal to the threshold, the controller determines that the alignment relationship between the first stacking object and the second stacking object is misaligned, and generates a control instruction to execute step 1409 .

[0225] Step 1409: The controller controls the transport device to adjust its posture according to the difference, and continues to execute steps 1401 to 1407 until the difference is less than the threshold.

[0226] Step 1410: When the difference is less than a threshold, the controller confirms that the alignment state of the first stacking object and the second stacking object is aligned.

[0227] Step 1411: The controller controls the transport device to place the first stacking object on the second stacking object to complete the stacking.

[0228] Among them, closed-loop detection (corresponding to steps 1402 to 1411) means that after the handling device forks the first stacking object, the posture detection of the first stacking object and the second stacking object is continuously performed, and the difference between the detected posture of the first stacking object and the posture of the second stacking object is compared with the threshold. When the difference is greater than or equal to the threshold, the alignment state is confirmed to be misaligned, and the posture of the first stacking object on the fork is continuously adjusted until the difference is less than the threshold, and then the handling device is controlled to place the first stacking object on the second stacking object to complete the stacking.

[0229] The following describes an implementation of the method proposed in the embodiment of the present application in combination with an actual application scenario. Fig.15 As shown, the execution subject is a handling device, and the process of matching the foot cup of the first stacking object with the column of the second stacking object is described as follows:

[0230] Step 1501: The controller controls the fork of the handling device to pick up the first stacked object and perform closed-loop detection.

[0231] Step 1502: The controller obtains target point clouds of the first stacking object and the second stacking object through a sensor.

[0232] Step 1503: The controller extracts point clouds of partial areas of four foot cups from the target point cloud of the first stacked object; and extracts point clouds of partial areas of four pillars from the target point cloud of the second stacked object.

[0233] Step 1504: The controller preprocesses the point clouds of partial areas of the four foot cups extracted from the target point cloud of the first stacking object as the first point cloud; and preprocesses the point clouds of partial areas of the four pillars extracted from the target point cloud of the second stacking object as the second point cloud.

[0234] Among them, the point cloud of the partial areas of the four cups includes the point cloud of the partial area of ​​the first cup of the first stacked object, the point cloud of the partial area of ​​the third cup of the first stacked object, the point cloud of the partial area of ​​the fifth cup of the first stacked object, and the point cloud of the partial area of ​​the seventh cup of the first stacked object.

[0235] The point cloud of the partial area of ​​the four pillars includes the point cloud of the partial area of ​​the second pillar of the second stacking object, the point cloud of the partial area of ​​the fourth pillar of the second stacking object, the point cloud of the partial area of ​​the sixth pillar of the second stacking object and the point cloud of the partial area of ​​the eighth pillar of the second stacking object.

[0236] Step 1505: The controller matches the first point cloud with the first template point cloud to obtain the posture of the first stacked object, and matches the second point cloud with the second template point cloud to obtain the posture of the second stacked object.

[0237] Among them, the first template point cloud and the second template point cloud are determined in the following manner: the controller obtains the original point clouds of the third stacking object and the fourth stacking object that meet the alignment posture standard through the sensor; the controller preprocesses the original point clouds to obtain the standard point cloud of the third stacking object and the standard point cloud of the fourth stacking object; from the standard point cloud of the third stacking object, the point cloud of the partial area of ​​the first foot cup of the third stacking object, the point cloud of the partial area of ​​the third foot cup of the third stacking object, the point cloud of the partial area of ​​the fifth foot cup of the third stacking object, and the point cloud of the partial area of ​​the seventh foot cup of the third stacking object are extracted as the first template point cloud; from the standard point cloud of the fourth stacking object, the point cloud of the partial area of ​​the second column of the fourth stacking object, the point cloud of the partial area of ​​the fourth column of the fourth stacking object, the point cloud of the partial area of ​​the sixth column of the fourth stacking object, and the point cloud of the partial area of ​​the eighth column of the fourth stacking object are extracted as the second template point cloud.

[0238] Step 1506: The controller determines a difference between the pose of the first stacked object and the pose of the second stacked object.

[0239] Step 1507: Determine whether the difference is less than a threshold.

[0240] Step 1508 : When the difference is greater than or equal to the threshold, the controller confirms that the alignment relationship between the first stacking object and the second stacking object is misaligned, and generates a control instruction to execute step 1509 .

[0241] Step 1509: The controller controls the transport device to adjust its posture according to the difference, and continues to execute steps 1501 to 1507 until the difference is less than the threshold.

[0242] Step 1510: When the difference is less than a threshold, the controller confirms that the alignment state of the first stacking object and the second stacking object is aligned.

[0243] Step 1511: The controller controls the transport device to place the first stacking object on the second stacking object to complete the stacking.

[0244] Among them, closed-loop detection (corresponding to steps 1502 to 1511) means that after the handling device forks the first stacking object, the posture detection of the first stacking object and the second stacking object is continuously performed, and the difference between the detected posture of the first stacking object and the posture of the second stacking object is compared with the threshold. When the difference is greater than or equal to the threshold, the alignment state is confirmed to be misaligned, and the posture of the first stacking object on the fork is continuously adjusted until the difference is less than the threshold, and then the handling device is controlled to place the first stacking object on the second stacking object to complete the stacking.

[0245] in, Fig.12The second point cloud of the second stacking object B' before closed-loop detection and the second template point cloud of the fourth stacking object B (the second point cloud PC2 and the second template point cloud TM2 are not aligned at this time); Fig.13 The second point cloud of the second stacking object B' is aligned with the second template point cloud of the fourth stacking object B through closed-loop detection (at this time, the second point cloud PC2 is aligned with the second template point cloud TM2).

[0246] In the embodiment of the present application, controlling the transport device to adjust the posture may include controlling the transport device to adjust the posture of the first stacking object on the fork and / or controlling the transport device to adjust the posture of the chassis (ie, adjusting the posture of the transport device relative to the second stacking object).

[0247] The above method provided in the embodiment of the present application can be applied to a variety of application scenarios, including but not limited to: including but not limited to: unmanned warehouse scenarios, unmanned loading and unloading scenarios.

[0248] Among them, unmanned warehouses may include unmanned forklifts, storage shelves, picking platforms, RCS control systems and warehouse management systems (Warehouse Management System, WMS), etc.

[0249] Among them, the unmanned loading and unloading scenarios include unmanned forklifts, RCS control systems, and trucks.

[0250] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0251] According to an embodiment of another aspect, a control system is provided for executing to implement any one of the alignment status confirmation methods disclosed in the embodiments of the present application.

[0252] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The system and device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0253] In addition, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a controller, the steps of any one of the methods in the aforementioned method embodiments are implemented.

[0254] And a handling device, comprising:

[0255] Carriage of handling equipment;

[0256] A sensor mounted on the body of the transport equipment;

[0257] one or more controllers; and

[0258] A memory associated with the one or more controllers, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more controllers, execute the steps of the method described in any one of the aforementioned method embodiments.

[0259] The present application also provides a computer program product, including a computer program, which implements the steps of any one of the methods in the aforementioned method embodiments when executed by a controller.

[0260] It can be seen from the above description of the implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can essentially be embodied in the form of a computer program product, which can be stored in a storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc.

[0261] The technical solution provided by the present application is described in detail above. The principle and implementation method of the present application are described in detail using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. A method for confirming an alignment state, characterized in that: include: The controller acquires target point clouds of the first stacking object and the second stacking object through a sensor; The controller extracts a first point cloud of a first target area of ​​the first stacked object and a second point cloud of a second target area of ​​the second stacked object from the target point cloud; The controller matches the first point cloud with the first template point cloud to obtain the posture of the first stacked object, and matches the second point cloud with the second template point cloud to obtain the posture of the second stacked object; The controller determines a difference between a posture of the first stacked object and a posture of the second stacked object, and compares the difference with a threshold value to confirm an alignment state of the first stacked object and the second stacked object.

2. The method according to claim 1, characterized in that The method further comprises: Before confirming the alignment state, the controller controls the transport device to transport the first stacking object to a stacking preparation position to complete a pre-alignment action relative to the second stacking object.

3. The method according to claim 1, characterized in that The step of comparing the difference with a threshold value to confirm an alignment state between the first stacking object and the second stacking object comprises: If the difference is greater than or equal to a threshold, confirming that the alignment state is misaligned; If the difference is less than a threshold, the alignment state is confirmed to be aligned.

4. The method according to claim 3, characterized in that The method further comprises: When the alignment state is misaligned, the controller controls the handling device to adjust the posture; The controller reacquires the target point cloud of the first stacked object and the second stacked object through the sensor; The controller re-determines a difference between the posture of the first stacked object and the posture of the second stacked object; The controller reconfirms the alignment state according to the difference until the difference is less than a threshold value.

5. The method according to claim 3, characterized in that: The method further comprises: When the alignment state is aligned, the controller controls the transport device to place the first stacking object on the second stacking object to complete the stacking.

6. The method according to claim 5, characterized in that The controlling the transporting equipment to adjust the posture includes: controlling the transporting equipment to adjust the posture of the chassis or the posture of the fork.

7. The method according to claim 3, characterized in that The method further comprises: The controller counts the number of misalignments; When the number of misalignments is greater than a preset number of misalignments, an alarm prompt is output.

8. The method according to claim 1, characterized in that The first template point cloud and the second template point cloud are obtained in the following manner: The controller collects original point clouds of a third stacking object and a fourth stacking object that meet the alignment posture standard through the sensor, the third stacking object has the same structure and size as the first stacking object, and the fourth stacking object has the same structure and size as the second stacking object; The controller pre-processes the original point cloud to obtain a standard point cloud of the third stacking object and a standard point cloud of the fourth stacking object; The controller extracts the first template point cloud from the standard point cloud of the third stacked object; The controller extracts the second template point cloud from a standard point cloud of the fourth stacked object.

9. The method according to claim 8, characterized in that The first template point cloud includes a point cloud of a first corner structure region of the third stacked object extracted from a standard point cloud of the third stacked object.

10. The method according to claim 9, characterized in that The second template point cloud includes: a point cloud of a second corner structure region of the fourth stacking object extracted from a standard point cloud of the fourth stacking object.

11. The method according to claim 9, characterized in that The first corner structure region has a first side line and a third side line, and the first side line intersects the third side line.

12. The method according to claim 10, characterized in that The second corner structure region has a second sideline and a fourth sideline, and the second sideline intersects the fourth sideline.

13. The method according to claim 10, characterized in that The first template point cloud further includes: a point cloud of a third corner structure region of the third stacked object extracted from a standard point cloud of the third stacked object; The second template point cloud also includes: a point cloud of a fourth corner structure region of the fourth stacking object extracted from the standard point cloud of the fourth stacking object.

14. The method according to claim 13, characterized in that The first template point cloud also includes: a point cloud of a fifth corner structure region and a point cloud of a seventh corner structure region of the third stacked object extracted from the standard point cloud of the third stacked object; The second template point cloud also includes: a point cloud of a sixth corner structure region and a point cloud of an eighth corner structure region of the fourth stacking object extracted from the standard point cloud of the fourth stacking object.

15. The method according to claim 14, characterized in that The first stacking object is a first material cage, and the second stacking object is a second material cage; The first target area is located at the bottom of the first cage, and the first target area includes: a first corner structure area of ​​the first cage, a third corner structure area of ​​the first cage, a fifth corner structure area of ​​the first cage, and a seventh corner structure area of ​​the first cage; The second target area is located at the top of the second material basket, and the second target area includes: the second corner structure area of ​​the second material basket, the fourth corner structure area of ​​the second material basket, the sixth corner structure area of ​​the second material basket and the eighth corner structure area of ​​the second material basket.

16. The method according to claim 8, characterized in that The first template point cloud comprises: a point cloud of a partial area of ​​a first foot cup of the third stacked object extracted from a standard point cloud of the third stacked object; The second template point cloud includes: a point cloud of a partial area of ​​the second column of the fourth stacking object extracted from the standard point cloud of the fourth stacking object.

17. The method according to claim 16, characterized in that The first template point cloud further includes: a point cloud of a partial area of ​​a third foot cup of the third stacked object extracted from a standard point cloud of the third stacked object; The second template point cloud also includes: a point cloud of a partial area of ​​a fourth column of the fourth stacking object extracted from the standard point cloud of the fourth stacking object.

18. The method according to claim 17, characterized in that The first template point cloud further includes: a point cloud of a partial area of ​​a fifth foot cup and a point cloud of a partial area of ​​a seventh foot cup of the third stacked object extracted from the standard point cloud of the third stacked object; The second template point cloud also includes: a point cloud of a partial area of ​​the sixth column and a point cloud of a partial area of ​​the eighth column of the fourth stacking object extracted from the standard point cloud of the fourth stacking object.

19. The method according to claim 18, characterized in that The first stacking object is a first material cage, and the second stacking object is a second material cage; The first target area includes: the first foot cup of the first material basket, the third foot cup of the first material basket, the fifth foot cup of the first material basket, and the seventh foot cup of the first material basket; The second target area includes: the second column of the second material basket, the fourth column of the second material basket, the sixth column of the second material basket, and the eighth column of the second material basket.

20. A control system, characterized in that: The invention comprises a controller and a memory, wherein the memory is used to store program instructions, and the controller is used to execute the program instructions to implement the method according to any one of claims 1 to 19.

21. A handling device, characterized in that: The invention comprises a controller and a memory, wherein the memory is used to store program instructions, and the controller is used to execute the program instructions to implement the method according to any one of claims 1 to 19.

Images