De-stacking method, system, controller, and computer-readable storage medium

Abstract

The embodiment of the application relates to the technical field of automatic unstacking robots, and discloses an unstacking method, system, controller and computer readable storage medium, the method comprising: acquiring ROI region RGB image information and point cloud information through the 3D camera, identifying the to-be-unstacked article in the RGB image information and generating a grabbing point; and acquiring the camera coordinates of the grabbing point, the box length-width information of the top to-be-unstacked layer and the height information of the box top surface through the point cloud information; performing a grabbing step, after the current grabbed box is unstacked, the height difference of the original position of the current box is acquired through the 3D camera again, the detection height value of the current box is obtained, and laser radar verification is selectively used according to different situations, and finally high-efficiency unstacking is realized. The technical scheme of the application solves the problem that, in the prior art, detection efficiency and operation precision are difficult to be considered in the unstacking process.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of automated depalletizing robots, and in particular to a depalletizing method, system, controller and computer readable storage medium. BACKGROUND

[0002] In the logistics, warehousing and manufacturing industries, automated depalletizing is a crucial link, and its core task is to efficiently and non-destructively take each stacked box, bagged object or bin off the pallet one by one. The key technology to achieve this process lies in the robot's ability to accurately and quickly identify and locate the spatial pose of each target box in the stack, especially the accurate position in the height direction.

[0003] Currently, the mainstream automated depalletizing solution usually relies on machine vision technology, especially 3D vision sensors such as RGB-D cameras and LiDARs. These two sensors each have their own advantages and inherent limitations.

[0004] RGB-D camera-based solution: By projecting coded structured light and calculating the light spot deformation, the depth information of the scene can be obtained, and rich point cloud data can be obtained at one time, which is convenient for overall stack identification and segmentation. However, such cameras are prone to errors in depth calculation when dealing with long-distance measurement, strong environmental light interference, and highly reflective packaging surfaces such as smooth plastic film and bright paper boxes, resulting in data loss or excessive noise, making the detected box height and flatness unreliable. This fluctuation in accuracy seriously affects the success rate of the depalletizing robot's grasp, and may even cause collision risks.

[0005] LiDAR-based solution: LiDAR calculates distance by emitting a laser beam and measuring the return time, which has high ranging accuracy and strong anti-environmental light interference capability, and performs more stably when dealing with reflective surfaces. Therefore, in some scenarios with extremely high positioning accuracy requirements, a single-point LiDAR is often used to scan the bottom of the identified box point by point to verify its height. However, this single-point scanning method requires the robot to move the LiDAR sensor to a specific point for measurement, introducing additional motion and time overhead, which can become a bottleneck in the overall depalletizing efficiency and may not meet the needs of modern logistics high-speed operation.

[0006] In summary, the existing depalletizing methods mainly have the following technical problems:

[0007] Low verification efficiency: Single-point scanning with a LiDAR to verify the box bottom height requires additional motion and measurement time, which can become a bottleneck in the overall depalletizing efficiency.

[0008] Data is isolated and cannot be synergized: the data of the laser radar and the RGB-D camera cannot be deeply fused and processed in linkage, the system cannot intelligently allocate and utilize the characteristics of different sensors, and the system is in a state of "each for itself".

[0009] Detection accuracy fluctuates: the RGB-D camera has large depth measurement noise and low reliability when measuring at a long distance or facing a reflective surface; although the laser radar has high single-point accuracy, its sparse sampling characteristics cannot quickly provide complete surface information.

[0010] Therefore, there is an urgent need in the art for a new unstacking method that can fuse multi-sensor data and balance detection efficiency and operation accuracy to solve the above technical problems. SUMMARY

[0011] In view of the above problems, the embodiments of the present application provide a unstacking method, system, controller and computer readable storage medium, which are used to solve the problem that the detection efficiency and operation accuracy are difficult to balance in the unstacking process of the prior art.

[0012] According to one aspect of the embodiments of the present application, a unstacking method is provided, which is realized based on a 3D camera and a laser radar, and includes the following steps:

[0013] The 3D camera is used to acquire RGB image information and point cloud information of a ROI region, identify a to-be-unstacked object in the RGB image information and generate a grabbing point, and acquire camera coordinates of the grabbing point, box length and width information of a top to-be-unstacked layer, and height information of a box top surface through the point cloud information;

[0014] The grabbing step is executed, and after the current grabbed box is separated from the stack, the 3D camera is used to acquire a height difference of the original position of the current box, and a detection height value of the current box is obtained;

[0015] It is judged whether the current box is the first box separated from the stack of the top layer, if yes, a first unstacking strategy is executed, the first unstacking strategy is to acquire a box verification height value through laser radar identification scanning of a box bottom, and the verification height value is used as an output height value of the current box; otherwise, it is judged whether a difference between the detection height value and the verification height value is within a preset range, if yes, a second unstacking strategy is executed, otherwise, an output height value is acquired through laser radar identification scanning of the box bottom; the second unstacking strategy is to use a weighted average of the detection height value and the verification height value as the output height value of the current box;

[0016] The moving track of the mechanical hand is determined according to the executed unstacking strategy to realize unstacking.

[0017] In an optional manner, the preset range is determined through the following sub-steps:

[0018] Calculate and record the set of differences between the detected height value and the verification height value of the box away from the stack within a first preset time period;

[0019] Calculate the standard deviation of the set;

[0020] And set 1.5 times of the standard deviation as the maximum value of the preset range.

[0021] In an optional manner, the movement trajectory of the manipulator is determined according to the executed unstacking strategy to achieve unstacking, including the following sub-steps:

[0022] Receive the executed unstacking strategy;

[0023] When the unstacking strategy is the first unstacking strategy, the place where the laser radar is located is set as a passing point, and the shortest route from the stack position to the destination is planned;

[0024] When the unstacking strategy is the second unstacking strategy, the shortest route from the stack position to the destination is directly planned.

[0025] In an optional manner, the execution of the grabbing step, after the currently grabbed box is away from the stack, the height difference of the original position of the current box is obtained again through the 3D camera to obtain the detected height value of the current box, including the following sub-steps:

[0026] Detect the grabbing signal and the grabbing coordinate information of the manipulator;

[0027] According to the grabbing coordinate information, the ROI region is positioned;

[0028] The average height coordinate of the ROI region is obtained through the 3D camera;

[0029] Calculate the difference between the height coordinate and the height coordinate of the top box to obtain the detected height value of the current box.

[0030] In an optional manner, when it is judged that the difference between the detected height value and the verification height value is not within the preset range more than a first preset value, the following sub-steps are executed:

[0031] Get the historical verification height value within a second preset time period;

[0032] Get the box length and width information corresponding to the historical verification height value;

[0033] According to the box length and width information, the verification detected height value is clustered to form a detected height value array.

[0034] In an alternative mode, the difference between the current box height value and the check height value is determined whether it is in the preset range, if yes, the second unstacking strategy is executed, otherwise the output height value is obtained by scanning the bottom of the box through the laser radar.

[0035] The box length and width information of the current box is obtained.

[0036] The mapping relationship is compared according to the box length and width information, and the corresponding check height value is obtained from the detection height value array.

[0037] The difference between the current box height value and the corresponding check height value is determined whether it is in the preset range.

[0038] If yes, the second unstacking strategy is executed, otherwise the output height value is obtained by scanning the bottom of the box through the laser radar.

[0039] According to a second aspect of the embodiment of the present application, a kind of unstacking system is provided, comprising:

[0040] 3D camera, for obtaining the box length and width information and height information of the box in the stack with unstacking box;

[0041] Laser radar, for checking the height information of the box with unstacking box;

[0042] Mechanical hand, for unstacking the stack according to the preset unstacking strategy and movement path;And

[0043] Host computer, with 3D camera, laser radar and mechanical hand communication connection, and using the above unstacking method controls the mechanical hand and unstacks the stack.

[0044] In an alternative mode, the laser radar is arranged at the bottom of the system, and the sensing direction is upward.

[0045] According to a third aspect of the embodiment of the present application, a kind of unstacking system controller is provided, comprising:

[0046] Processor, memory, communication interface and communication bus, the processor, the memory and the communication interface are communicated with each other by the communication bus;

[0047] The memory is used to store at least one executable instruction, and the executable instruction makes the processor execute to realize the step of the unstacking method as described above.

[0048] According to a fourth aspect of the embodiments of the present application, a computer readable storage medium is provided, which stores a computer program, and when a device on which the computer readable storage medium is located executes the computer program, the dechucking method described above is implemented.

[0049] The present application realizes efficient and accurate dechucking by complementary advantages of 3D camera and laser radar. The 3D camera provides real-time RGB image and point cloud data for quickly identifying and positioning the to-be-dechucked articles, and the laser radar provides high-precision measurement of the height of the bottom of the box. By dynamically determining whether the laser radar needs to be scanned, unnecessary measurement operations are reduced, and the dechucking efficiency is improved. The weighted mean strategy fuses the data of the two sensors, which guarantees the accuracy while taking into account the efficiency. This multi-modal data cooperative processing mechanism enables the system to adaptively select the optimal detection mode according to the actual situation, effectively solving the problem that accuracy and efficiency are difficult to balance.

[0050] The above description is only a summary of the technical solutions of the embodiments of the present application, in order to more clearly understand the technical means of the embodiments of the present application, and can be implemented according to the content of the specification, and in order to make the above and other purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the specific embodiments of the present application are described below. BRIEF DESCRIPTION OF DRAWINGS

[0051] The accompanying drawings are included to provide a further understanding of the embodiments, and are incorporated herein and constitute a part of the detailed description. It should be noted that in the accompanying drawings, the same or similar elements are denoted by the same reference numerals. In the drawings:

[0052] Figure 1 A flowchart of a first embodiment of the dechucking method provided by the present application is shown;

[0053] Figure 2 A flowchart of the sub-steps of the dechucking method provided by the present application in step 130 of the first embodiment is shown;

[0054] Figure 3 A flowchart of the sub-steps of the dechucking method provided by the present application in step 140 of the first embodiment is shown;

[0055] Figure 4 A flowchart of the sub-steps of the dechucking method provided by the present application in step 120 of the first embodiment is shown;

[0056] Figure 5 A structural schematic diagram of an embodiment of the dechucking system provided by the present application is shown;

[0057] Figure 6 An implementation flowchart of the dechucking method provided by the present application is shown. DETAILED DESCRIPTION

[0058] Exemplary embodiments of the present application will be described in greater detail below with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it is understood that the present application can be embodied in various forms and should not be limited by the embodiments set forth herein.

[0059] Embodiment 1:

[0060] Figure 1 A flowchart of a first embodiment of the de-stacking method of the present application is shown, which is implemented based on a 3D camera and a laser radar, and is executed by a de-stacking system for automated de-stacking. As shown in the figure, the method comprises the following steps: Figure 1

[0061] Step 110: Obtain the RGB image information and point cloud information of the ROI region through the 3D camera, identify the to-be-unstacked article in the RGB image information and generate a grabbing point; and obtain the camera coordinates of the grabbing point, the length and width information of the top layer to-be-unstacked layer, and the height information of the top surface of the box through the point cloud information.

[0062] Among them, the ROI region refers to the to-be-operated region framed in the RGB image through a target detection algorithm, which can be implemented by a deep learning model such as YOLO, and is used to reduce the data processing range and improve the calculation efficiency. The generation of the grabbing point is based on the three-dimensional coordinate analysis of the point cloud data, and the optimal grabbing pose can be determined by surface normal vector calculation.

[0063] Specifically, during the de-stacking operation, the 3D camera continuously collects the RGB image and point cloud data of the target stack, and after positioning the to-be-grabbed box through the image recognition algorithm, the length and width size information of the box and the height information of the top surface of the box are extracted and calculated.

[0064] Step 120: Perform the grabbing step, and after the current grabbed box is unstacked, the height difference of the original position of the current box is obtained again through the 3D camera to obtain the detection height value of the current box.

[0065] Among them, the detection height value refers to the estimated value of the bottom surface height of the box calculated by measuring the height difference before and after grabbing through the 3D camera, which can be implemented by adjacent frame point cloud registration and height difference difference algorithm, and forms real-time feedback to correct the error caused by the change of the stack type.

[0066] Specifically, the robot can be used to perform the grabbing action to unstack the box, and after unstacking, the 3D camera collects the point cloud data of the position again, and the height difference obtained by comparing the two point clouds is the detection height value.

[0067] ​Step 130: determining whether the current box is the first palletized box of the top layer, if so, executing a first de-palletizing strategy, the first de-palletizing strategy being to obtain a box verification height value by laser radar recognition scanning the bottom of the box, and taking the verification height value as the output height value of the current box; otherwise, determining whether the difference between the detection height value and the verification height value is within a preset range, if so, executing a second de-palletizing strategy, otherwise, obtaining the output height value by laser radar recognition scanning the bottom of the box; the second de-palletizing strategy being to take the weighted average of the detection height value and the verification height value as the output height value of the current box.

[0068] wherein the preset range refers to an allowable deviation threshold value set according to the box height tolerance and the sensor error, for deciding whether to trigger the laser radar secondary verification. The weighted average refers to a calculation method of linear fusion of the detection height value and the verification height value, which can specifically adopt a fixed weight coefficient or dynamically allocate a weight according to the sensor confidence, to balance the real-time requirement and the accuracy requirement.

[0069] Specifically, for the first palletized box of the top layer, the verification height value obtained by laser radar scanning the bottom of the box is taken as the output height value. For subsequent boxes, the difference between the detection height value and the verification height value is compared, if within the preset range, the weighted average is taken as the output height value, otherwise, the output height value is obtained by reusing the laser radar scanning.

[0070] For example, the system determines whether the current box is the first palletized box of the layer. If so, the laser radar scans the bottom of the box to obtain a high-precision verification height value, which is taken as the output height value of the current box. For non-first boxes, the system compares the difference between the detection height value and the previously obtained verification height value. If the difference is within the preset range, the second de-palletizing strategy is adopted, i.e., the detection height value and the verification height value are weighted and averaged to obtain the output height value. If the difference exceeds the preset range, the laser radar is restarted to obtain a new output height value.

[0071] In some embodiments, the specific method of taking the weighted average of the detection height value and the verification height value as the output height value of the current box can be implemented in the following manner. The detection height value H camera measured by the 3D camera and the verification height value H lidar measured by the laser radar are dynamically weighted and fused to obtain a fused height value H fused, the formula being as follows:

[0072] H_fused=(w1*H_camera+w2*H_lidar) / (w1+w2)

[0073] wherein w1 and w2 are the weights of the 3D camera and the laser radar respectively. The weights are dynamically allocated according to the historical measurement standard deviations of the 3D camera and the laser radar, the larger the standard deviation, the smaller the weight.

[0074] Optionally, the weight values ​​are: w1=1 / σ_camera, w2=1 / σ_lidar.

[0075] Finally, the fused height value H_fused is used as the output height value for subsequent calculation of the gripping points of the box, thereby optimizing the gripping accuracy in advance and forming a closed-loop feedback.

[0076] Step 140: Develop the robot's movement trajectory based on the destacking strategy to achieve destacking.

[0077] Based on the final determined output height value, the system plans the movement trajectory of the robotic arm to accurately place the box. The entire process is repeated until the entire layer or stack is disassembled.

[0078] Compared with existing technologies, traditional solutions require laser scanning verification for each box. This method reduces the number of laser measurements by combining mandatory verification of the first box with subsequent dynamic judgment. Compared with solutions that rely solely on 3D cameras, this method reduces the height detection error rate in reflective surface scenarios. The data fusion mechanism enables the system to maintain the real-time performance of the 3D camera while inheriting the measurement stability of the lidar, avoiding the error accumulation problem caused by isolated sensor data in traditional solutions.

[0079] The above method leverages the complementary advantages of 3D cameras and LiDAR to achieve efficient and accurate depalletizing. The 3D camera provides real-time RGB images and point cloud data for rapid identification and location of items to be depalletized, while the LiDAR provides high-precision measurements of the box bottom height. By dynamically determining whether LiDAR scanning is needed, unnecessary measurement operations are reduced, improving depalletizing efficiency. The weighted averaging strategy integrates data from both sensors, ensuring both accuracy and efficiency. This multimodal data collaborative processing mechanism allows the system to adaptively select the optimal detection mode based on actual conditions, effectively solving the problem of balancing accuracy and efficiency.

[0080] Please combine Figure 2 , Figure 2 A flowchart illustrating a sub-step in step 130 of the destacking method of the present invention is shown.

[0081] In step 130, the preset range is determined through the following sub-steps:

[0082] Step 131: Calculate and record the set of differences between the detected height value and the verified height value of the off-stack box within the first preset time period.

[0083] The detection height value refers to the height difference value obtained by re-scanning the original position of the box after grabbing by the 3D camera, and is used to represent the actual height of the box in a single measurement; and the verification height value refers to the height data obtained by scanning the bottom of the box by the laser radar, and is used to provide a high-precision reference height value. Specifically, during the continuous unstacking operation, the system records the difference between the detection height value measured by the 3D camera and the verification height value of the laser radar after each grabbing operation in real time, forming a dynamic data set reflecting the measurement error.

[0084] Step 132: Calculate the standard deviation of the set.

[0085] The standard deviation refers to the degree of deviation of each data point in the data set from the average value, and is used to quantify the fluctuation range of the difference between the detection height and the verification height.

[0086] Specifically, by calculating the standard deviation of the data set, the consistency of the measurement results of the two sensors under the current working condition can be objectively evaluated.

[0087] Step 133: Set 1.5 times the standard deviation as the maximum value of the preset range.

[0088] The maximum value of the preset range refers to the upper limit of the threshold value allowed for the difference between the detection height and the verification height, and can be determined by multiplying the standard deviation by a factor of 1.5, and is used to dynamically adjust the condition for triggering the secondary verification of the laser radar.

[0089] Specifically, multiplying the standard deviation by 1.5 as the upper limit of the preset range avoids the problem of excessive verification caused by improper fixed threshold setting. When the difference between the detection height and the verification height exceeds the dynamic threshold value, the supplementary scanning operation of the laser radar is triggered. This dynamic threshold mechanism based on statistics can adapt to the measurement fluctuations caused by different stack structures and environmental disturbances.

[0090] Please refer to Figure 3 , Figure 3 The flowchart shows the sub-steps of the unstacking method of the application in step 140.

[0091] In step 140, the movement trajectory of the manipulator is determined according to the executed unstacking strategy to achieve unstacking, including the following sub-steps:

[0092] Step 141: Receive the executed unstacking strategy;

[0093] Step 142: When the unstacking strategy is the first unstacking strategy, set the place where the laser radar is located as a passing point, and plan the shortest route from the stack position to the destination;

[0094] Step 143: When the unstacking strategy is the second unstacking strategy, directly plan the shortest route from the stack position to the destination.

[0095] Wherein, the passing point refers to a specific coordinate point that the manipulator must pass through in the moving process, and specifically, the installation position coordinate of the laser radar can be used as the passing point, which can be pre-stored in the control system.

[0096] Wherein, the shortest route refers to the path with the minimum moving distance between the starting position and the target position of the manipulator, and the optimal route is determined by comparing the cost functions of different path nodes.

[0097] Wherein, the path planning corresponding to the first unstacking strategy needs to ensure that the manipulator passes through the position of the laser radar in the moving process, so that the laser radar can scan and verify the bottom of the box; the path planning corresponding to the second unstacking strategy does not need to pass through a specific verification point, and directly uses the shortest straight line between two points.

[0098] In the embodiment, after receiving the unstacking strategy instruction, the upper computer first analyzes the type of the strategy to be executed. When the strategy type is the first unstacking strategy that needs laser radar verification, the path planning module inserts the spatial coordinate of the laser radar as a passing node into the path node sequence, and then calls the path search algorithm to calculate the optimal path in the topology graph containing the passing node, so as to ensure that the manipulator can accurately reach the laser radar scanning position to complete the height verification in the process of moving the box.

[0099] When the strategy type is the second unstacking strategy that does not need secondary verification, the path planning module directly constructs an unconstrained path topology between the stack position and the target position, generates a direct route through the shortest path algorithm, and eliminates the redundant movement of detouring to the laser radar position. The path planning of the two strategies both takes minimizing the idle time of the manipulator as the optimization objective, and realizes efficiency optimization by dynamically adjusting the path constraint conditions.

[0100] Please combine Figure 4 , Figure 4 The flowchart of the sub-steps of the unstacking method in step 120 is shown.

[0101] In step 120, the grabbing step is performed, and after the current grabbed box is separated from the stack, the height difference of the original position of the current box is obtained through the 3D camera again, and the detection height value of the current box is obtained, including the following sub-steps:

[0102] Step 121: detecting the grabbing signal and the grabbing coordinate information of the manipulator;

[0103] Step 122: positioning the ROI region according to the grabbing coordinate information;

[0104] Step 123: obtaining the average height coordinate of the ROI region through the 3D camera;

[0105] Step 124: Calculate the difference between the height coordinate and the height coordinate of the top layer box to obtain the detection height value of the current box.

[0106] Wherein, the grabbing signal refers to the trigger signal generated after the manipulator completes the grabbing action, which is used to synchronously trigger the subsequent visual detection process; the grabbing coordinate information refers to the spatial coordinates of the manipulator end effector when grabbing the box, which is used to accurately locate the original position of the target box in the stack.

[0107] Wherein, the average height coordinate refers to the average value of all valid point cloud data in the ROI region in the vertical direction, which can be specifically calculated by arithmetic average of the remaining point cloud height value after removing outliers, and is used to suppress the influence of single point noise on height measurement.

[0108] In this embodiment, when the manipulator completes the box grabbing and leaves the stack, the grabbing signal triggers the visual detection process; the original position of the target box in the stack is determined based on the grabbing coordinate information, and the ROI region is defined with this as the center; the point cloud data in this region is collected by the 3D camera, and the average height coordinate after removing outliers is calculated; the average height is subtracted from the height of the top layer box recorded before unstacking to obtain the actual height change of the current box after leaving the stack. The height difference of the box after leaving the stack can be accurately calculated, and the 3D camera measurement error problem caused by environmental light interference or surface reflection can be solved.

[0109] In some embodiments, when the number of times that the difference between the detection height value and the verification height value is not within the preset range exceeds the first preset value, the following sub-steps are performed:

[0110] Obtain the historical verification height value in the second preset time period;

[0111] Obtain the box length and width information corresponding to the historical verification height value;

[0112] According to the box length and width information, the verification detection height value is clustered to form a detection height value array.

[0113] Wherein, the historical verification height value refers to the box bottom height data obtained by actual scanning of the laser radar, which can be specifically implemented by storing the verification records with time stamp in the database, and this feature provides accurate basic data for subsequent clustering analysis.

[0114] Wherein, the clustering processing refers to grouping and aggregating the historical verification height values with the same box length and width information, and this feature can identify the height distribution law corresponding to different size boxes.

[0115] The detection height value array refers to a set of historical height data classified by the size of the box. The mapping relationship between the length and width information and the height array can be achieved by establishing a hash table. This feature provides a reference benchmark for subsequent height verification of boxes of the same size.

[0116] In this embodiment, the first preset value can be, but is not limited to, 5 times. When the system detects that the number of times of height deviation exceeding the limit is more than 5 times, the data backtracking mechanism is triggered. First, extract all height data of the work cycle in the recent second preset time period that has passed the laser radar verification from the storage unit, and synchronously call the length and width parameters of the box bottom corresponding to these height values. Group the historical height data by clustering through the length and width parameters as classification keys, and form a height data set classified by size.

[0117] For example, for a box with a length of 600 mm and a width of 400 mm, the corresponding historical height value is aggregated into an independent array. This array can be used as a height reference benchmark for boxes of the same size in subsequent work. When the same size box is encountered again, the system can directly call the median or mean of this array for deviation judgment, reducing the dependence on real-time laser scanning.

[0118] The efficiency decline problem caused by frequent verification is solved. The historical data reuse mechanism reduces the number of laser radar scans, and the size classification model improves the reliability of height verification, so that the system can adapt to the height changes of boxes of different sizes and maintain the continuity and stability of the unstacking operation. At the same time, it can also be compatible with the unstacking scene of a stack composed of boxes of different sizes.

[0119] In some embodiments, it is determined whether the difference between the current box height value and the verification height value is within a preset range. If yes, the second unstacking strategy is executed, otherwise the output height value is obtained by laser radar recognition scanning the bottom of the box, including the following sub-steps:

[0120] Obtain the length and width information of the current box;

[0121] Map the length and width information of the box to obtain the corresponding verification height value from the detection height value array;

[0122] Determine whether the difference between the current box height value and the corresponding verification height value is within a preset range;

[0123] If yes, the second unstacking strategy is executed, otherwise the output height value is obtained by laser radar recognition scanning the bottom of the box.

[0124] The mapping relationship comparison refers to similarity matching the length and width parameters of the current box with the box size in the historical data, and selecting the historical data cluster with the smallest difference as the matching result to achieve. It is used to determine the verification height value category to which the current box belongs.

[0125] It should be noted that the lengths of the first and second preset time periods mentioned above can be adjusted. For example, one day or one hour. These are related to environmental variables, such as light intensity, temperature, and humidity. When environmental variables change significantly, the lengths of the first and / or second preset time periods can be adjusted. This includes situations such as the transition between day and night, or temperature changes exceeding a certain limit. For example, when temperature changes are significant, the duration of the first and / or second preset time periods can be shortened to eliminate errors caused by temperature variations.

[0126] In this embodiment, when the difference between the current height value and the verified height value of the enclosure exceeds a preset range, the system first extracts the length and width dimensions of the enclosure. By calculating the distance between the current dimension and the center points of each cluster in the historical data, it determines the category of the closest historical dimension. Subsequently, it retrieves the verified height value from the array of detected height values ​​corresponding to that category and calculates the difference between it and the current detected height value. If the difference is within the preset range, a weighted average strategy is used to fuse the real-time detected value and the historical verified value; if it exceeds the range, the LiDAR is triggered to perform a precise scan of the bottom of the enclosure, and the scan result is used as the final output height value. This process narrows the verification range through a size matching mechanism, avoiding the calculation delay caused by global search.

[0127] The above technical solution enables rapid location of historical verification values ​​of similar dimensions when height anomalies are detected, reducing the number of LiDAR scans and shortening the waiting time for destacking operations. Simultaneously, a size matching mechanism ensures the physical consistency of the verified height values, avoiding misjudgments caused by differences in box dimensions and improving the positioning accuracy and reliability of the destacking action.

[0128] Example 2:

[0129] like Figure 5 As shown, Figure 5 A schematic diagram of an embodiment of the depalletizing system 50 of the present invention is shown. The depalletizing system 50, used to perform a depalletizing method, includes: a 3D camera 51, a lidar 52, a robotic arm 53, and a host computer 54.

[0130] The 3D camera 51 is used to acquire the length, width, and height information of the boxes with unpacking components in the stack. The lidar 52 is used to verify the height information of the boxes with unpacking components. The robotic arm 53 is used to unpack the stack according to a preset unpacking strategy and movement path. The host computer 54 is communicatively connected to the 3D camera 51, lidar 52, and robotic arm 53, and controls the robotic arm 53 to unpack the stack using the aforementioned unpacking method.

[0131] Specifically, the 3D camera 51 performs a global scan on the stack when the system starts, generating a three-dimensional model containing the length, width and initial height of the box. The host computer 54 calculates the grabbing point coordinates according to the model and sends them to the manipulator 53 to perform the first grabbing. After the first box is removed, the 3D camera 51 monitors the height change of the stack in real time. If the deviation between the detected value and the verification value of the laser radar 52 exceeds the threshold, the laser radar 52 is triggered to perform a local scan on the exposed bottom surface of the box. The host computer 54 fuses the data of the two sensors by weighting, generates a corrected height parameter and updates the path planning. For subsequent boxes, the laser radar 52 is only activated when continuous height abnormalities occur, and in other cases, the fusion result of the visual data and the historical verification value is directly used.

[0132] In some embodiments, the laser radar 52 is arranged at the bottom of the system, and the sensing direction is arranged upward.

[0133] When the de-stacking system 50 starts, the laser radar 52 fixed at the bottom continuously emits a laser beam upward. When the de-stacked box is located above the tray, the laser beam penetrates the gap between the bottom of the box and the tray, and the vertical distance between the bottom surface of the box and the laser radar 52 is calculated by measuring the flight time of the laser beam. Since the position of the laser radar 52 is fixed and the direction is constant, it is not necessary to drive the mechanical arm to carry the sensor to a specific position every time the height is verified. The height data of the bottom surface of the box is directly obtained by static measurement.

[0134] The above system realizes efficient and accurate de-stacking by complementing the advantages of the 3D camera 51 and the laser radar 52. The 3D camera 51 provides real-time RGB images and point cloud data for quickly identifying and positioning the de-stacked items, and the laser radar 52 provides high-precision height measurement of the bottom of the box. By dynamically determining whether the laser radar 52 needs to be scanned, unnecessary measurement operations are reduced, and de-stacking efficiency is improved. The weighted mean strategy fuses the data of the two sensors, ensuring accuracy while considering efficiency. This multi-modal data cooperative processing mechanism enables the system to adaptively select the optimal detection mode according to the actual situation, effectively solving the problem of balancing accuracy and efficiency.

[0135] Embodiment 3:

[0136] Figure 6 The structure schematic diagram of the embodiment of the controller of the de-stacking system 50 is shown, and the specific implementation of the controller of the de-stacking system 50 is not limited in the specific embodiments of the application.

[0137] As shown in Figure 6 A controller of a de-stacking system 50, comprising: a processor 601, a memory 603, a communication interface 602 and a communication bus 604.

[0138] The processor 601, the memory 603 and the communication interface 602 complete mutual communication through the communication bus 604. The communication interface 602 is configured to communicate with network elements such as a controller of the unstacking system 50 or other servers. The processor 601 is configured to execute the program 610, and the execution realizes the steps in the unstacking method as described above. The memory 603 is configured to store at least one executable instruction, and the executable instruction realizes the steps in the unstacking method as described above when the processor 601 executes the executable instruction.

[0139] Specifically, the program 610 can include program 610 code including computer executable instructions.

[0140] Specifically, the processor 601 can be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present application. The one or more processors 601 included in the controller of the unstacking system 50 can be the same type of processor 601, such as one or more CPUs; or can be different types of processors 601, such as one or more CPUs and one or more ASICs.

[0141] The memory 603 is configured to store the program 610. The memory 603 can include a high-speed RAM memory 603, and can also include a non-volatile memory 603, such as at least one disk memory 603.

[0142] The program 610 can be specifically invoked by the processor 601 to enable the controller of the unstacking system 50 to perform the following operations:

[0143] The 3D camera 51 acquires the RGB image information and the point cloud information of the ROI region, identifies the to-be-unstacked article in the RGB image information and generates a grabbing point, and acquires the camera coordinates of the grabbing point, the box length and width information of the top to-be-unstacked layer, and the height information of the top surface of the box through the point cloud information;

[0144] The grabbing step is performed, and after the current grabbed box is unstacked, the 3D camera 51 acquires the height difference of the original position of the current box again to obtain a detection height value of the current box;

[0145] If the current box is the first pallet box of the top layer, a first de-palletizing strategy is executed, the first de-palletizing strategy is to obtain a box calibration height value by identifying and scanning the bottom height of the box through the laser radar 52, and the calibration height value is used as the output height value of the current box; otherwise, it is judged whether the difference between the detection height value and the calibration height value is within a preset range, if yes, a second de-palletizing strategy is executed, otherwise, the output height value is obtained by identifying and scanning the bottom of the box through the laser radar 52; the second de-palletizing strategy is to use the weighted mean value of the detection height value and the calibration height value as the output height value of the current box.

[0146] According to the executed de-palletizing strategy, the moving track of the manipulator 53 is determined to realize de-palletizing.

[0147] The data flow in the above embodiment is consistent with the data flow in Embodiment 1, and specific details can be referred to the description of Embodiment 1, which will not be repeated here.

[0148] In an optional implementation, the program 610 is called by the processor 601 to enable the controller of the de-palletizing system 50 to execute the specific sub-steps of the steps 120, 130 and 140 in Embodiment 1.

[0149] The controller of the de-palletizing system 50 above realizes efficient and accurate de-palletizing by complementary advantages of the 3D camera 51 and the laser radar 52. The 3D camera 51 provides real-time RGB images and point cloud data for quickly identifying and positioning the de-palletizing objects, and the laser radar 52 provides high-precision box bottom height measurement. By dynamically judging whether the laser radar 52 needs to be scanned, unnecessary measurement operations are reduced, and de-palletizing efficiency is improved. The weighted mean strategy integrates the data of the two sensors, which guarantees accuracy while taking into account efficiency. This multi-modal data collaborative processing mechanism enables the system to adaptively select the optimal detection mode according to the actual situation, effectively solving the problem of balancing accuracy and efficiency.

[0150] Embodiment 4:

[0151] The embodiment of the application provides a computer readable storage medium, and the computer readable storage medium stores a computer program 610, wherein when a device where the computer readable storage medium is located executes the computer program 610, a de-palletizing method as described above is realized.

[0152] The executable instructions can be used to enable the controller of the de-palletizing system 50 to perform the following operations:

[0153] The 3D camera 51 obtains the RGB image information and the point cloud information of the ROI region, identifies the de-palletizing objects in the RGB image information and generates a grabbing point; and obtains the camera coordinates of the grabbing point, the box length and width information of the top de-palletizing layer, and the height information of the box top surface through the point cloud information;

[0154] The height difference of the current box from the original position is obtained by the 3D camera 51 after the current box is grabbed and separated from the stack, and the detection height value of the current box is obtained.

[0155] If the current box is the first box separated from the stack on the top layer, a first unstacking strategy is executed, which obtains the verification height value of the box by identifying and scanning the bottom height of the box by the laser radar 52, and takes the verification height value as the output height value of the current box; otherwise, it is determined whether the difference between the detection height value and the verification height value is within a preset range, if yes, a second unstacking strategy is executed, otherwise, the output height value is obtained by identifying and scanning the bottom of the box by the laser radar 52; the second unstacking strategy takes the weighted average of the detection height value and the verification height value as the output height value of the current box.

[0156] According to the unstacking strategy executed, the moving track of the manipulator 53 is determined to realize unstacking.

[0157] The controller realizes efficient and accurate unstacking by complementary advantages of the 3D camera 51 and the laser radar 52. The 3D camera 51 provides real-time RGB images and point cloud data for quickly identifying and positioning the unstacked items, and the laser radar 52 provides high-precision measurement of the bottom height of the box. By dynamically determining whether the laser radar 52 needs to be scanned, unnecessary measurement operations are reduced, and unstacking efficiency is improved. The weighted average strategy combines the data of the two sensors, which guarantees accuracy while taking into account efficiency. This multi-modal data collaborative processing mechanism enables the system to adaptively select the optimal detection mode according to the actual situation, effectively solving the problem of balancing accuracy and efficiency.

[0158] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other apparatus. Furthermore, embodiments of the application are not described with reference to any particular programming language.

[0159] In the specification provided herein, a large number of specific details are described. However, it can be understood that the embodiments of the application can be practiced without these specific details. Similarly, in order to simplify the application and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the application, various features of the embodiments of the application are sometimes grouped together into a single embodiment, figure, or description thereof. Among them, the claims following the detailed description are hereby expressly incorporated into the detailed description, wherein each claim itself is a separate embodiment of the application.

[0160] It will be appreciated by those skilled in the art that modules in the apparatuses in the embodiments can be adapted and placed in one or more apparatuses other than that of the embodiments. Modules or units or components in the embodiments can be combined into one module or unit or component and furthermore can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive.

[0161] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a unit claim, any reference to 'composition' should not be construed as a limitation unless the composition is a product of manufacturing. The use of the word 'about' in relation to a numerical value preferably means ± 10 % of the value. The word 'first','second', 'third', etc. does not imply any order. The use of the words 'first','second', 'third', etc. to introduce a list of named items does not imply that the items are to be used according to the order of such introduction. The above-described embodiments of the present application can be modified in various ways without departing from the scope of the present application.

Claims

1. A de-palletizing method characterized by, The method is realized based on a 3D camera and a laser radar, and comprises the following steps: acquiring, by the 3D camera, RGB image information and point cloud information of a ROI region, identifying a to-be-unstacked article in the RGB image information and generating a grabbing point, and acquiring, by the point cloud information, a camera coordinate of the grabbing point, box length-width information of a top to-be-unstacked layer, and height information of a box top surface; performing a grabbing step, and then acquiring, by the 3D camera, a height difference of an original position of a current box after the current box is unstacked, to obtain a detection height value of the current box; judging whether the current box is a first unstacked box of the top layer, and if yes, performing a first unstacking strategy, wherein the first unstacking strategy comprises obtaining a box verification height value by identifying and scanning a box bottom height through the laser radar, and taking the verification height value as an output height value of the current box; otherwise, judging whether a difference between the detection height value and the verification height value is within a preset range, and if yes, performing a second unstacking strategy, otherwise, obtaining the output height value by identifying and scanning the box bottom through the laser radar; the second unstacking strategy comprises taking a weighted average of the detection height value and the verification height value as the output height value of the current box; formulating a moving track of a mechanical hand according to the executed unstacking strategy to realize unstacking; the step of formulating the moving track of the mechanical hand according to the executed unstacking strategy to realize unstacking comprises the following sub-steps: receiving the executed unstacking strategy; when the unstacking strategy is the first unstacking strategy, setting a place where the laser radar is located as a passing point, and planning a shortest route from a stack position to a destination; when the unstacking strategy is the second unstacking strategy, directly planning the shortest route from the stack position to the destination; when a number of times that the difference between the detection height value and the verification height value is not within the preset range exceeds a first preset value, performing the following sub-steps: acquiring historical verification height values in a second preset time period; acquiring box length-width information corresponding to the historical verification height values; performing clustering processing on the verification detection height values according to the box length-width information, to form a detection height value array; the step of judging whether the difference between the current box height value and the verification height value is within the preset range, and if yes, performing the second unstacking strategy, otherwise, obtaining the output height value by identifying and scanning the box bottom through the laser radar, comprises the following sub-steps: acquiring box length-width information of the current box; performing mapping relationship comparison according to the box length-width information, to obtain corresponding verification height values from the detection height value array; judging whether the difference between the current box height value and the corresponding verification height value is within the preset range; if yes, performing the second unstacking strategy, otherwise, obtaining the output height value by identifying and scanning the box bottom through the laser radar.

2. The de-palletizing method of claim 1, wherein, the preset range is determined by the following sub-steps: calculating and recording a set of differences between detection height values and verification height values of unstacked boxes in a first preset time period; calculating a standard deviation of the set; and setting 1.5 times of the standard deviation as a maximum value of the preset range.

3. The de-palletizing method of claim 1, wherein, the step of performing the grabbing step, and then acquiring, by the 3D camera, a height difference of an original position of a current box after the current box is unstacked, to obtain a detection height value of the current box, comprises the following sub-steps: Detect the grabbing signal and the grabbing coordinate information of the manipulator; Position the ROI region according to the grabbing coordinate information; Obtain the average height coordinate of the ROI region through the 3D camera; Calculate the difference between the height coordinate and the height coordinate of the top layer of the box to obtain the detection height value of the current box.

4. A de-palletizing system characterized by, The system comprises: a 3D camera for obtaining the length, width and height information of the box with the unloading box in the stack; a laser radar for verifying the height information of the box with the unloading box; a manipulator for unloading the stack according to a preset unloading strategy and moving path; and a host computer in communication connection with the 3D camera, the laser radar and the manipulator, and adopting the unloading method of any one of claims 1-3 to control the manipulator to unload the stack.

5. The de-palletizing system of claim 4, wherein, The laser radar is arranged at the bottom of the system and the sensing direction is upward.

6. A controller of a de-palletizing system, characterized in that, The system comprises: a processor, a memory, a communication interface and a communication bus, which complete mutual communication through the communication bus; the memory is used to store at least one executable instruction, and the executable instruction makes the processor execute to realize the steps of the unloading method in any one of claims 1-3.

7. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, wherein the device where the computer readable storage medium is located executes the computer program to realize the unloading method in any one of claims 1-3.

Images