Unstacking method and system, controller and computer readable storage medium

Abstract

The embodiment of the invention relates to the technical field of automatic unstacking robots, and discloses an unstacking method and system, a controller and a computer readable storage medium, and the method comprises the steps: obtaining RGB image information and point cloud information of an ROI region through a 3D camera, recognizing a to-be-unstacked article in the RGB image information, and generating a grabbing point; the camera coordinates of the grabbing points, the length and width information of the box body of the top layer to be unstacked and the height information of the top surface of the box body are obtained through the point cloud information; and the grabbing step is executed, after the current grabbed box leaves the stack, the height difference of the original position of the current box is obtained through the 3D camera again, the detection height value of the current box is obtained, laser radar verification is selectively adopted according to different conditions, and finally efficient unstacking is achieved. By means of the technical scheme, the problem that in the unstacking process in the prior art, the detection efficiency and the operation precision are difficult to consider at the same time is solved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of automated depalletizing robots, and in particular to a depalletizing method, system, controller and computer readable storage medium. BACKGROUND

[0002] In the logistics, warehousing and manufacturing industries, automated depalletizing is a crucial link, and its core task is to efficiently and non-destructively take each stacked box, bagged object or bin off the pallet one by one. The key technology to achieve this process lies in the ability of the robot to accurately and quickly identify and locate the spatial pose of each target box in the stack, especially the accurate position in the height direction.

[0003] Currently, the mainstream automated depalletizing solution usually relies on machine vision technology, especially 3D vision sensors such as RGB-D cameras and LiDARs. These two types of sensors each have their own advantages and inherent limitations.

[0004] RGB-D camera-based solution: By projecting coded structured light and calculating the light spot deformation, the depth information of the scene can be obtained, and rich point cloud data can be obtained at one time, which is convenient for overall stack identification and segmentation. However, such cameras are prone to errors in depth calculation when dealing with long-distance measurement, strong environmental light interference, and highly reflective packaging surfaces such as smooth plastic film and bright paper boxes, resulting in data loss or excessive noise, making the detected box height and flatness unreliable. This fluctuation in accuracy seriously affects the success rate of the depalletizing robot's grasp, and may even cause collision risks.

[0005] LiDAR-based solution: LiDAR calculates distance by emitting a laser beam and measuring the return time, which has high ranging accuracy and strong anti-environmental light interference capability, and performs more stably when dealing with reflective surfaces. Therefore, in some scenarios with extremely high positioning accuracy requirements, a single-point LiDAR is often used to scan the bottom of the identified box point by point to verify its height. However, this single-point scanning method requires the robot to move to a specific point with the LiDAR sensor to measure, introducing additional motion and time overhead, which may become a bottleneck in the overall depalletizing efficiency and is difficult to meet the needs of modern logistics high-speed operation.

[0006] In summary, the existing depalletizing methods mainly have the following technical problems: Low verification efficiency: Single-point scanning with a LiDAR to verify the height of the box bottom requires additional motion and measurement time, which may become a bottleneck in the overall depalletizing efficiency.

[0007] Data is isolated and cannot be synergized: the data of the laser radar and the RGB-D camera cannot be deeply fused and processed in linkage, the system cannot intelligently allocate and utilize the characteristics of different sensors, and is in a state of "each for itself".

[0008] Detection accuracy fluctuates: the RGB-D camera has large depth measurement noise and low reliability when measuring at a long distance or facing a reflective surface; although the laser radar has high single-point accuracy, its sparse sampling characteristics cannot quickly provide complete surface information.

[0009] Therefore, there is an urgent need in the art for a new unstacking method that can fuse multi-sensor data and balance detection efficiency and operation accuracy to solve the above technical problems. SUMMARY

[0010] In view of the above problems, the embodiments of the present application provide a unstacking method, system, controller and computer readable storage medium, which are used to solve the problem that the detection efficiency and operation accuracy are difficult to balance in the unstacking process of the prior art.

[0011] According to one aspect of the embodiments of the present application, a unstacking method is provided, which is realized based on a 3D camera and a laser radar, and includes the following steps: The 3D camera is used to acquire RGB image information and point cloud information of a ROI region, identify a to-be-unstacked item in the RGB image information and generate a grabbing point, and acquire camera coordinates of the grabbing point, box length and width information of a top to-be-unstacked layer, and height information of a box top surface through the point cloud information; A grabbing step is performed, and after the current grabbed box is unstacked, the 3D camera is used to acquire a height difference of the original position of the current box, so as to obtain a detection height value of the current box; It is judged whether the current box is the first unstacked box of the top layer, if yes, a first unstacking strategy is performed, the first unstacking strategy is to acquire a box verification height value through laser radar recognition and scanning of a box bottom, and use the verification height value as an output height value of the current box; otherwise, it is judged whether a difference between the detection height value and the verification height value is within a preset range, if yes, a second unstacking strategy is performed, otherwise, an output height value is acquired through laser radar recognition and scanning of the box bottom; the second unstacking strategy is to use a weighted average of the detection height value and the verification height value as the output height value of the current box; According to the executed unstacking strategy, a moving track of a mechanical hand is formulated to realize unstacking.

[0012] In an optional manner, the preset range is determined through the following sub-steps: A set of differences between the detection height values and the verification height values of the unstacked boxes in a first preset time period is calculated and recorded; The standard deviation of the set is calculated; and 1.5 times of the standard deviation is set as the maximum value of the preset range.

[0013] In an alternative way, the movement trajectory of the robot arm is determined according to the executed de-stacking strategy to achieve de-stacking, including the following sub-steps: receiving the executed de-stacking strategy; when the de-stacking strategy is the first de-stacking strategy, the place where the laser radar is located is set as a passing point, and the shortest route from the stack position to the destination is planned; when the de-stacking strategy is the second de-stacking strategy, the shortest route from the stack position to the destination is directly planned.

[0014] In an alternative way, the step of executing the grabbing, after the current box is grabbed away from the stack, the height difference of the original position of the current box is obtained by the 3D camera again, and the detection height value of the current box is obtained, including the following sub-steps: detecting the grabbing signal and the grabbing coordinate information of the robot arm; positioning the ROI region according to the grabbing coordinate information; obtaining the average height coordinate of the ROI region by the 3D camera; calculating the difference between the height coordinate and the height coordinate of the top box to obtain the detection height value of the current box.

[0015] In an alternative way, when the number of times that the difference between the detection height value and the verification height value is not within the preset range exceeds the first preset value, the following sub-steps are executed: obtaining the historical verification height value in the second preset time period; obtaining the box length and width information corresponding to the historical verification height value; performing clustering processing on the verification detection height value according to the box length and width information to form a detection height value array.

[0016] In an alternative way, the step of judging whether the difference between the current box height value and the verification height value is within the preset range, if yes, the second de-stacking strategy is executed, otherwise the output height value is obtained by scanning the bottom of the box through the laser radar, including the following sub-steps: obtaining the box length and width information of the current box; performing mapping relationship comparison according to the box length and width information to obtain the corresponding verification height value from the detection height value array; judging whether the difference between the current box height value and the corresponding verification height value is within the preset range; if yes, the second de-stacking strategy is executed, otherwise the output height value is obtained by scanning the bottom of the box through the laser radar.

[0017] According to a second aspect of the embodiments of the present application, a de-stacking system is provided, comprising: a 3D camera, configured to acquire the length and width information and height information of the box in the stack; a laser radar, configured to verify the height information of the box in the stack; a manipulator, configured to de-stack the stack according to a preset de-stacking strategy and movement path; and a host computer, in communication connection with the 3D camera, the laser radar and the manipulator, and configured to control the manipulator to de-stack the stack by using the de-stacking method.

[0018] In an optional manner, the laser radar is arranged at the bottom of the system, and the sensing direction is upward.

[0019] According to a third aspect of the embodiments of the present application, a controller of a de-stacking system is provided, comprising: a processor, a memory, a communication interface and a communication bus, the processor, the memory and the communication interface are in communication with each other through the communication bus; the memory is configured to store at least one executable instruction, and the executable instruction enables the processor to perform the steps of the de-stacking method when executed.

[0020] According to a fourth aspect of the embodiments of the present application, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, wherein when the device where the computer readable storage medium is located executes the computer program, the de-stacking method is realized.

[0021] The present application realizes efficient and accurate de-stacking through the complementary advantages of the 3D camera and the laser radar. The 3D camera provides real-time RGB images and point cloud data for quickly identifying and positioning the de-stacking objects, and the laser radar provides high-precision measurement of the height of the box bottom. By dynamically determining whether the laser radar needs to be scanned, unnecessary measurement operations are reduced, and the de-stacking efficiency is improved. The weighted mean strategy fuses the data of the two sensors, which guarantees the accuracy while taking into account the efficiency. This multi-modal data cooperative processing mechanism enables the system to adaptively select the optimal detection mode according to the actual situation, effectively solving the problem of balancing accuracy and efficiency.

[0022] The above description is only a summary of the technical solutions of the embodiments of the present application, in order to more clearly understand the technical means of the embodiments of the present application, the embodiments of the present application can be implemented according to the content of the specification, and in order to make the above and other purposes, characteristics and advantages of the embodiments of the present application more obvious and easy to understand, the specific embodiments of the present application are described as follows. BRIEF DESCRIPTION OF DRAWINGS

[0023] The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification. In the drawings: Figure 1 A flowchart of a first embodiment of the de-stacking method provided by the present application is shown; Figure 2 A flowchart of the sub-steps in step 130 of the first embodiment of the de-stacking method provided by the present application is shown; Figure 3 A flowchart of the sub-steps in step 140 of the first embodiment of the de-stacking method provided by the present application is shown; Figure 4 A flowchart of the sub-steps in step 120 of the first embodiment of the de-stacking method provided by the present application is shown; Figure 5 A structural diagram of an embodiment of the de-stacking system provided by the present application is shown; Figure 6 An implementation flowchart of the de-stacking method provided by the present application is shown. DETAILED DESCRIPTION

[0024] Exemplary embodiments of the present application will be described in greater detail below, with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein.

[0025] Embodiment 1: Figure 1 A flowchart of a first embodiment of the de-stacking method provided by the present application is shown, which is implemented based on a 3D camera and a laser radar, executed by a de-stacking system, for automated de-stacking. As shown in the figure, the method comprises the following steps: Figure 1 Step 110: Obtain the RGB image information and point cloud information of the ROI region through the 3D camera, identify the to-be-unstacked items in the RGB image information and generate a grabbing point, and obtain the camera coordinates of the grabbing point, the box length and width information of the top to-be-unstacked layer, and the height information of the top surface of the box through the point cloud information.

[0026] Among them, the ROI region refers to the to-be-operated region framed in the RGB image through a target detection algorithm, which can be implemented by using a deep learning model such as YOLO, for narrowing the data processing range and improving the calculation efficiency. The generation of the grabbing point is based on the three-dimensional coordinate analysis of the point cloud data, and the optimal grabbing pose can be determined by using surface normal vector calculation.

[0027] ​Specifically, during the unstacking operation, the 3D camera continuously collects the RGB image and point cloud data of the target stack, and after positioning the to-be-grabbed box through the image recognition algorithm, the length, width and height information of the box are calculated by extracting the corresponding point cloud region.

[0028] Step 120: Perform the grabbing step, and after the current grabbed box is separated from the stack, the height difference of the original position of the current box is obtained through the 3D camera again to obtain the detection height value of the current box.

[0029] The detection height value refers to the estimated value of the bottom height of the box calculated by measuring the height difference before and after grabbing through the 3D camera. Specifically, adjacent frame point cloud registration and height difference difference algorithm can be used to realize real-time feedback to correct the error caused by stack changes.

[0030] Specifically, a robot can be used to perform the grabbing action to separate the box from the stack. After the box is separated from the stack, the 3D camera collects the point cloud data of the position again, and the height difference obtained by comparing the two point clouds is the detection height value.

[0031] Step 130: Determine whether the current box is the first box separated from the top layer. If so, execute the first unstacking strategy, which is to obtain the box verification height value by scanning the bottom height of the box through the laser radar, and use the verification height value as the output height value of the current box. Otherwise, determine whether the difference between the detection height value and the verification height value is within a preset range. If so, execute the second unstacking strategy, otherwise obtain the output height value by scanning the bottom of the box through the laser radar. The second unstacking strategy is to use the weighted average of the detection height value and the verification height value as the output height value of the current box.

[0032] The preset range refers to the allowed deviation threshold set according to the box height tolerance and sensor error, which is used to determine whether to trigger the laser radar secondary verification. The weighted average refers to a linear fusion calculation method for the detection height value and the verification height value. Specifically, a fixed weight coefficient or a dynamic weight allocation according to the sensor confidence can be used to balance the real-time and accuracy requirements.

[0033] Specifically, for the first box separated from the top layer, the verification height value obtained by scanning the bottom of the box through the laser radar is used as the output height value. For subsequent boxes, the difference between the detection height value and the verification height value is compared. If it is within the preset range, the weighted average is used as the output height value, otherwise the output height value is obtained by scanning the laser radar again.

[0034] For example, the system determines whether the current box is the first box on the layer. If so, the laser radar scans the bottom of the box to obtain a high-precision calibration height value, which is used as the output height value of the current box. For non-first boxes, the system compares the difference between the detection height value and the previously obtained calibration height value. If the difference is within a predetermined range, the second unstacking strategy is adopted, that is, the detection height value and the calibration height value are weighted and averaged to obtain the output height value. If the difference exceeds the predetermined range, the laser radar is restarted to obtain a new output height value.

[0035] In some embodiments, the specific method of using the weighted average of the detection height value and the calibration height value as the output height value of the current box can be implemented as follows. The detection height value H camera measured by the 3D camera and the calibration height value H lidar measured by the laser radar are dynamically weighted and fused to obtain a fused height value H fused, which is calculated as follows: H_fused=(w1*H_camera+w2*H_lidar) / (w1+w2) where w1 and w2 are the weights of the 3D camera and the laser radar, respectively. The weights are dynamically assigned according to the historical measurement standard deviation of the 3D camera and the laser radar. The larger the standard deviation, the smaller the weight.

[0036] Optionally, the weight values are w1=1 / σ_camera and w2=1 / σ_lidar.

[0037] Finally, the fused height value H_fused is used as the output height value for subsequent calculation of the gripping point of the box, thereby optimizing the gripping accuracy in advance and forming a closed-loop feedback.

[0038] Step 140: According to the executed unstacking strategy, the movement trajectory of the robot arm is determined to achieve unstacking.

[0039] According to the finally determined output height value, the system plans the movement trajectory of the robot arm to complete the accurate placement of the box. The entire process is repeated until the unstacking of the entire layer or stack is completed.

[0040] Compared with the prior art, the traditional scheme requires laser scanning calibration for each box, while the present method combines the first box forced calibration with subsequent dynamic judgment, reducing the number of laser measurements. Compared with the scheme that relies solely on the 3D camera, the present method reduces the height detection error rate in the scene with reflective surfaces. The data fusion mechanism enables the system to maintain the real-time performance of the 3D camera while inheriting the measurement stability of the laser radar, thereby avoiding the error accumulation problem caused by the isolation of sensor data in the traditional scheme.

[0041] The above method realizes efficient and accurate unstacking by complementing the advantages of the 3D camera and the laser radar. The 3D camera provides real-time RGB images and point cloud data for quickly identifying and positioning the unstacked items, and the laser radar provides high-precision box bottom height measurement. By dynamically determining whether the laser radar needs to be scanned, unnecessary measurement operations are reduced, unstacking efficiency is improved, and the weighted mean strategy fuses the data of the two sensors to ensure accuracy while taking into account efficiency. This multi-modal data collaborative processing mechanism enables the system to adaptively select the optimal detection mode according to the actual situation, effectively solving the problem of balancing accuracy and efficiency.

[0042] Please refer to Figure 2 , Figure 2 A flowchart showing the sub-steps of the unstacking method of the present application in step 130 is shown.

[0043] In step 130, the preset range is determined by the following sub-steps: Step 131: Calculate and record the set of differences between the detection height value and the verification height value of the box away from the stack within the first preset time period.

[0044] Wherein, the detection height value refers to the height difference value obtained by re-scanning the original position of the box after grabbing by the 3D camera, which is used to represent the actual height of the box measured at a time; the verification height value refers to the height data obtained by scanning the bottom of the box by the laser radar, which is used to provide a high-precision reference height value. Specifically, during continuous unstacking operation, the system records the difference between the detection height value measured by the 3D camera and the verification height value measured by the laser radar after each grabbing operation in real time, forming a dynamic data set reflecting the measurement error.

[0045] Step 132: Calculate the standard deviation of the set.

[0046] Wherein, the standard deviation refers to the degree of deviation of each data point in the data set from the mean value, which is used to quantify the fluctuation range of the difference between the detection height and the verification height.

[0047] Specifically, by calculating the standard deviation of the data set, the consistency of the measurement results of the two sensors under the current working condition can be objectively evaluated.

[0048] Step 133: Set 1.5 times the standard deviation as the maximum value of the preset range.

[0049] Wherein, the maximum value of the preset range refers to the upper limit of the threshold value of the difference between the detection height and the verification height, which can be determined by multiplying the standard deviation by a factor of 1.5, and is used to dynamically adjust the condition for triggering the secondary verification of the laser radar.

[0050] Specifically, the standard deviation is multiplied by 1.5 as the upper limit of the preset range, avoiding the problem of over-calibration caused by improper fixed threshold settings. Only when the difference between the detection height and the calibration height exceeds this dynamic threshold does the lidar trigger a supplemental scan. This statistically based dynamic threshold mechanism can adapt to measurement fluctuations caused by different stack structures and environmental interference.

[0051] Please combine Figure 3 , Figure 3 A schematic flow chart of the sub-steps in step 140 of the depalletizing method of the present invention is shown.

[0052] In step 140, a movement trajectory of the robot is formulated according to the executed depalletizing strategy to achieve depalletizing, including the following sub-steps: Step 141: receiving the executed destacking strategy; Step 142: When the destacking strategy is the first destacking strategy, the location of the laser radar is set as a waypoint, and the shortest route from the stack position to the destination is planned; Step 143: When the depalletizing strategy is the second depalletizing strategy, directly plan the shortest route from the stack position to the destination.

[0053] The waypoint refers to a specific coordinate point that the manipulator must pass through during its movement. Specifically, the coordinates of the installation position of the laser radar can be used as the necessary point, and the coordinates can be pre-stored in the control system.

[0054] Among them, the shortest route refers to the path with the shortest moving distance between the robot's starting position and the target position. The optimal route is determined by comparing the cost functions of different path nodes.

[0055] Among them, the path planning corresponding to the first destacking strategy must ensure that the robot passes through the location of the laser radar during movement, so that the laser radar can scan and verify the bottom of the box; the path planning corresponding to the second destacking strategy does not need to pass through specific verification points, and directly adopts the shortest straight line path between the two points.

[0056] In this embodiment, upon receiving the depalletizing strategy instruction, the host computer first analyzes the strategy type to be executed. If the strategy type is the first depalletizing strategy requiring LiDAR verification, the path planning module inserts the spatial coordinates of the LiDAR into the path node sequence as a required node. It then calls the path search algorithm to calculate the optimal path within the topological graph containing the required node, ensuring that the robot arm can accurately reach the LiDAR scanning position to complete the height verification while moving the box.

[0057] When the strategy type is the second unstacking strategy without secondary checking, the path planning module directly constructs an unconstrained path topology between the stack position and the target position, generates a direct route through the shortest path algorithm, and eliminates the redundant movement of detouring to the laser radar position. The path planning of the two strategies optimizes the efficiency by dynamically adjusting the path constraint conditions, with the minimum robot idle time as the optimization objective.

[0058] Please combine Figure 4 , Figure 4 A flowchart showing the sub-steps of the unstacking method of the present application in step 120 is shown.

[0059] In step 120, the grabbing step is performed, and after the current grabbed box is removed from the stack, the height difference of the original position of the current box is obtained through the 3D camera again, and the detection height value of the current box is obtained, including the following sub-steps: Step 121: detecting the grabbing signal and the grabbing coordinate information of the robot; Step 122: positioning the ROI region according to the grabbing coordinate information; Step 123: obtaining the average height coordinate of the ROI region through the 3D camera; Step 124: calculating the difference between the height coordinate and the height coordinate of the top box to obtain the detection height value of the current box.

[0060] Among them, the grabbing signal refers to the trigger signal generated after the robot completes the grabbing action, which is used to synchronously trigger the subsequent visual detection process; the grabbing coordinate information refers to the spatial coordinates of the robot end effector when grabbing the box, which is used to accurately locate the original position of the target box in the stack.

[0061] Among them, the average height coordinate refers to the average value of all valid point cloud data in the ROI region in the vertical direction, which can be realized by calculating the arithmetic mean of the remaining point cloud height values after removing outliers, and is used to suppress the influence of single-point noise on height measurement.

[0062] In this embodiment, when the robot completes the box grabbing and leaves the stack, the grabbing signal triggers the visual detection process; the original position of the target box in the stack is determined based on the grabbing coordinate information, and the ROI region is defined with this as the center; the point cloud data in this region is collected through the 3D camera, and the average height coordinate after removing outliers is calculated; the average height is difference operated with the height of the top box recorded before unstacking, and the actual height change of the current box after being removed from the stack is obtained. The height difference after the box is removed from the stack can be accurately calculated, and the 3D camera measurement error problem caused by environmental light interference or surface reflection can be solved.

[0063] In some embodiments, when the number of times that the difference between the detection height value and the checking height value is not within the preset range exceeds the first preset value, the following sub-steps are performed: obtain historical calibration height values in a second preset time period; obtain box length-width information corresponding to the historical calibration height values; perform clustering processing on the calibration detection height values according to the box length-width information to form a detection height value array.

[0064] The historical calibration height values refer to the box bottom height data obtained by actual scanning of the laser radar, and can be specifically implemented by storing calibration records with timestamps in a database. This feature provides accurate basic data for subsequent clustering analysis.

[0065] The clustering processing refers to grouping and aggregating historical calibration height values with the same box length-width information. This feature can identify the height distribution law corresponding to different size boxes.

[0066] The detection height value array refers to a historical height data set stored according to box size. The mapping relationship between length-width information and height array can be specifically implemented by establishing a hash table. This feature provides a reference benchmark for subsequent height calibration of the same size box.

[0067] In this embodiment, the first preset value can be but is not limited to 5 times. When the system detects that the number of height deviation out-of-limit times exceeds 5 times, a data backtracking mechanism is triggered. First, extract all height data of boxes that have passed laser radar calibration in the recent second preset time period from the storage unit. Synchronously call the length-width parameters of the box bottom surface corresponding to these height values. Group and cluster the historical height data by taking the length-width parameters as classification key values to form a height data set classified by size.

[0068] For example, for a box with a length of 600 mm and a width of 400 mm, the corresponding historical height values are aggregated into an independent array. This array can be used as a height reference benchmark for the same size box in subsequent operations. When the same size box is encountered again, the system can directly call the median or mean of this array for deviation judgment, reducing the dependence on real-time laser scanning.

[0069] The problem of efficiency decline caused by frequent calibration is solved. The historical data reuse mechanism reduces the number of laser radar scans, and the size classification model improves the reliability of height calibration, so that the system can adapt to the height changes of boxes of different sizes and maintain the continuity and stability of the unstacking operation. At the same time, it can also be compatible with the unstacking scene of a stack composed of boxes of multiple different sizes.

[0070] In some embodiments, it is determined whether the difference between the current box height value and the calibration height value is within a preset range. If yes, the second unstacking strategy is executed, otherwise, the output height value is obtained by laser radar recognition scanning of the box bottom, including the following sub-steps: Obtain the length and width information of the current box body; According to the mapping relationship comparison of the length and width information of the box body, the corresponding verification height value is obtained from the detection height value array; Determine whether the difference between the current box height value and the corresponding verification height value is within the preset range; If yes, execute the second unstacking strategy, otherwise obtain the output height value by laser radar recognition scanning the bottom of the box.

[0071] Among them, the mapping relationship comparison refers to similarity matching the length and width parameters of the current box body with the box size in the historical data, and selecting the historical data cluster with the smallest difference as the matching result to achieve, which is used to determine the verification height value category to which the current box body belongs.

[0072] It should be noted that the length of the first preset time period and the second preset time period can need to be adjusted. For example, 1 day or 1 hour, etc. It is related to environmental variables, such as light, temperature and humidity, etc. When the environmental variables change greatly, the length of the first preset time period and / or the second preset time period can be adjusted. For example, the switching between day and night, the temperature change exceeding a certain limit, etc. For example, when the temperature changes greatly, the length of the first preset time period and / or the second preset time period can be shortened to eliminate the error caused by temperature change.

[0073] In the present embodiment, when it is detected that the difference between the height value of the current box body and the verification height value exceeds the preset range, the system first extracts the length and width size parameters of the box body, determines the historical size category closest to it by calculating the distance between the current size and the center point of each cluster in the historical data, and then retrieves the verification height value from the detection height value array corresponding to the category, and calculates the difference between the current detection height value. If the difference is within the preset range, the weighted average strategy is used to fuse the real-time detection value and the historical verification value; if it exceeds the range, the laser radar is triggered to accurately scan the bottom of the box, and the scanning result is taken as the final output height value. This process narrows down the verification range through the size matching mechanism, avoiding the calculation delay caused by global search.

[0074] Through the above technical solution, the historical verification value of the same size can be quickly located when the height anomaly is detected, the number of laser radar scans is reduced, and the waiting time of the unstacking operation is shortened. At the same time, the size matching mechanism ensures the physical consistency of the verification height value, avoids the misjudgment problem caused by the size difference of the box body, and improves the positioning accuracy and reliability of the unstacking action.

[0075] Embodiment 2: As shown in Figure 5 , the system comprises a box body height detection unit, a box body height verification unit and a box body unstacking unit. Figure 5A structural diagram of an embodiment of the de-stacking system 50 of the present application is shown. A de-stacking system 50 for performing a de-stacking method includes a 3D camera 51, a laser radar 52, a mechanical arm 53, and a host computer 54.

[0076] The 3D camera 51 is used to obtain the length, width, and height information of the boxes in the stack. The laser radar 52 is used to verify the height information of the boxes. The mechanical arm 53 is used to de-stack the stack according to a preset de-stacking strategy and movement path. The host computer 54 is in communication connection with the 3D camera 51, the laser radar 52, and the mechanical arm 53, and controls the mechanical arm 53 to de-stack the stack using the above de-stacking method.

[0077] Specifically, when the system starts, the 3D camera 51 performs global scanning on the stack to generate a three-dimensional model containing the length, width, and initial height of the boxes. The host computer 54 calculates the grabbing point coordinates according to the model and sends them to the mechanical arm 53 to perform the first grabbing. After the first box is removed, the 3D camera 51 monitors the height change of the stack in real time. If the detected value deviates from the verification value of the laser radar 52 by more than a threshold value, the laser radar 52 is triggered to perform local scanning on the exposed bottom surface of the box. The host computer 54 fuses the data of the two sensors by weighting, generates a corrected height parameter, and updates the path planning. For subsequent boxes, the laser radar 52 is activated only when height anomalies occur continuously, and in other cases, the fusion result of the visual data and the historical verification value is directly used.

[0078] In some embodiments, the laser radar 52 is arranged at the bottom of the system, and the sensing direction is upward.

[0079] When the de-stacking system 50 starts, the laser radar 52 fixed at the bottom continuously emits laser beams upward. When the de-stacking box is located above the tray, the laser beams penetrate the gap between the bottom of the box and the tray, and the vertical distance between the bottom surface of the box and the laser radar 52 is calculated by measuring the flight time of the laser beams. Since the position of the laser radar 52 is fixed and the direction is constant, it is not necessary to drive the mechanical arm to carry the sensor to a specific position every time the height is verified. The height data of the bottom surface of the box is directly obtained by static measurement.

[0080] The above system realizes efficient and accurate de-stacking through the complementary advantages of the 3D camera 51 and the laser radar 52. The 3D camera 51 provides real-time RGB images and point cloud data for quickly identifying and positioning the de-stacking items, and the laser radar 52 provides high-precision height measurement of the bottom of the box. By dynamically determining whether the laser radar 52 needs to be scanned, unnecessary measurement operations are reduced, and de-stacking efficiency is improved. The weighted mean strategy fuses the data of the two sensors, ensuring accuracy while considering efficiency. This multi-modal data collaborative processing mechanism enables the system to adaptively select the optimal detection mode according to the actual situation, effectively solving the problem of balancing accuracy and efficiency.

[0081] Embodiment 3 Figure 6 The structural diagram of an embodiment of the controller of the de-stacking system 50 is shown, and the specific embodiments of the application do not limit the specific implementation of the controller of the de-stacking system 50.

[0082] As shown in Figure 6 A controller of a de-stacking system 50, comprising: a processor 601, a memory 603, a communication interface 602 and a communication bus 604.

[0083] The processor 601, the memory 603 and the communication interface 602 communicate with each other through the communication bus 604. The communication interface 602 is used to communicate with network elements such as the controller of the de-stacking system 50 or other servers. The processor 601 is used to execute the program 610, and when executed, it realizes the steps in the de-stacking method as described above. The memory 603 is used to store at least one executable instruction, and the executable instruction makes the processor 601 execute and realize the steps in the de-stacking method as described above to perform de-stacking.

[0084] Specifically, the program 610 can include program 610 code, which includes computer executable instructions.

[0085] Specifically, the processor 601 can be a central processing unit CPU, or an application specific integrated circuit ASIC, or one or more integrated circuits configured to implement embodiments of the application. The one or more processors 601 included in the controller of the de-stacking system 50 can be the same type of processor 601, such as one or more CPUs; or can be different types of processors 601, such as one or more CPUs and one or more ASICs.

[0086] The memory 603 is used to store the program 610. The memory 603 can include a high-speed RAM memory 603, and can also include a non-volatile memory 603, such as at least one disk memory 603.

[0087] The program 610 can be specifically invoked by the processor 601 to make the controller of the de-stacking system 50 perform the following operations: Obtain the RGB image information and the point cloud information of the ROI region through the 3D camera 51, identify the to-be-unstacked object in the RGB image information and generate a grabbing point; and obtain the camera coordinates of the grabbing point, the box length and width information of the top to-be-unstacked layer, and the height information of the top surface of the box through the point cloud information; The height difference of the current box from the original position of the box is obtained by the 3D camera 51 after the current box is grabbed and separated from the stack, and a detection height value of the current box is obtained. If the current box is the first box separated from the stack on the top layer, a first unstacking strategy is executed, the first unstacking strategy is to obtain a verification height value of the box by identifying and scanning the bottom height of the box by the laser radar 52, and the verification height value is used as the output height value of the current box; otherwise, it is determined whether the difference between the detection height value and the verification height value is within a preset range, if yes, a second unstacking strategy is executed, otherwise, the output height value is obtained by identifying and scanning the bottom of the box by the laser radar 52; the second unstacking strategy is to use the weighted average of the detection height value and the verification height value as the output height value of the current box. According to the unstacking strategy executed, the moving track of the manipulator 53 is determined to realize unstacking.

[0088] The data flow in the embodiment is consistent with the data flow in the embodiment 1, and details can be referred to the description of the embodiment 1, and the embodiment will not be repeated.

[0089] In an optional implementation, the program 610 is called by the processor 601 to enable the controller of the unstacking system 50 to execute the specific sub-steps of the steps 120, 130 and 140 in the embodiment 1.

[0090] The controller of the unstacking system 50 realizes efficient and accurate unstacking by complementary advantages of the 3D camera 51 and the laser radar 52. The 3D camera 51 provides real-time RGB images and point cloud data for quickly identifying and positioning the unstacked items, and the laser radar 52 provides high-precision measurement of the bottom height of the box. By dynamically determining whether the laser radar 52 needs to be scanned, unnecessary measurement operations are reduced, and unstacking efficiency is improved. The weighted average strategy combines the data of the two sensors, which guarantees accuracy while taking into account efficiency. This multi-modal data collaborative processing mechanism enables the system to adaptively select the optimal detection mode according to the actual situation, effectively solving the problem of balancing accuracy and efficiency.

[0091] Embodiment 4: The embodiment of the application provides a computer readable storage medium, and the computer readable storage medium stores a computer program 610, wherein when a device where the computer readable storage medium is located executes the computer program 610, the unstacking method as described above is realized.

[0092] The executable instructions can be used to enable the controller of the unstacking system 50 to execute the following operations: The 3D camera 51 acquires the RGB image information and point cloud information of the ROI region, identifies the to-be-unstacked article in the RGB image information, and generates a grabbing point; and acquires the camera coordinates of the grabbing point, the box length and width information of the top to-be-unstacked layer, and the height information of the top surface of the box through the point cloud information; A grabbing step is performed, and after the current to-be-unstacked box is unstacked, the 3D camera 51 is used to acquire the height difference of the original position of the current box, to obtain a detection height value of the current box; It is judged whether the current box is the first to-be-unstacked box of the top layer, if yes, a first unstacking strategy is performed, the first unstacking strategy is to acquire a calibration height value of the box by identifying and scanning the bottom of the box through the laser radar 52, and the calibration height value is used as the output height value of the current box; otherwise, it is judged whether the difference between the detection height value and the calibration height value is within a preset range, if yes, a second unstacking strategy is performed, otherwise, the output height value is acquired by identifying and scanning the bottom of the box through the laser radar 52; the second unstacking strategy is to use the weighted average of the detection height value and the calibration height value as the output height value of the current box; According to the unstacking strategy executed, a moving track of the manipulator 53 is formulated to realize unstacking.

[0093] The controller realizes efficient and accurate unstacking through the complementary advantages of the 3D camera 51 and the laser radar 52. The 3D camera 51 provides real-time RGB image and point cloud data for quickly identifying and positioning the to-be-unstacked article, and the laser radar 52 provides high-precision measurement of the bottom height of the box. By dynamically judging whether the laser radar 52 needs to be scanned, unnecessary measurement operations are reduced, and unstacking efficiency is improved. The weighted average strategy fuses the data of the two sensors, ensuring accuracy while taking into account efficiency. This multi-modal data collaborative processing mechanism enables the system to adaptively select the optimal detection mode according to the actual situation, effectively solving the problem of balancing accuracy and efficiency.

[0094] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other apparatus. Furthermore, embodiments of the present application are not described in terms of any particular programming language.

[0095] In the description provided herein, a large number of specific details are described. However, it can be understood that the embodiments of the present application can be practiced without these specific details. Similarly, in order to simplify the present application and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, various features of the embodiments of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. Among them, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the present application.

[0096] It will be appreciated by those skilled in the art that modules in the apparatuses in the embodiments can be adapted and placed in one or more apparatuses other than that of the embodiments. Modules or units or components in the embodiments can be combined into one module or unit or component and furthermore can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive.

[0097] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a unit claim, any reference to 'composition' should not be construed as a limitation unless the composition is a product of manufacturing. The use of the word 'about' in relation to a numerical value preferably means ± 10 % of the value. The word 'first','second', 'third', etc. does not imply any order. The use of the terms 'first' and'second' are merely intended to identify the names of the elements and do not and should not imply any order and / or importance to the elements so named. A step by the word 'comprising' does not exclude the presence of additional steps than those stated in the claim.

Claims

1. A destacking method, characterized in that: The method is implemented based on a 3D camera and a laser radar and includes the following steps: The 3D camera is used to obtain RGB image information and point cloud information of the ROI area, identify the items to be depalletized in the RGB image information and generate a grab point; and the camera coordinates of the grab point, the length and width information of the box body of the top layer to be depalletized, and the height information of the top surface of the box are obtained through the point cloud information. Execute the grasping step. After the current grasped box leaves the stack, the height difference of the current box's original position is obtained again through the 3D camera to obtain the detection height value of the current box; Determine whether the current box is the first box to be removed from the stack on the top floor. If so, execute the first destacking strategy, which is to obtain a box verification height value by scanning the bottom height of the box through laser radar recognition, and use the verification height value as the output height value of the current box; otherwise, determine whether the difference between the detected height value and the verification height value is within a preset range. If so, execute the second destacking strategy, otherwise, obtain an output height value by scanning the bottom of the box through laser radar recognition; the second destacking strategy is to use the weighted average of the detected height value and the verification height value as the output height value of the current box; The movement trajectory of the robot is formulated according to the executed destacking strategy to achieve destacking.

2. The destacking method according to claim 1, wherein: The preset range is determined by the following sub-steps: Calculating and recording a set of differences between a detected height value and a verified height value of a box from the stack within a first preset time period; calculating a standard deviation for the set; And 1.5 times of the standard deviation is set as the maximum value of the preset range.

3. The destacking method according to claim 1, wherein: The step of formulating the movement trajectory of the manipulator according to the executed depalletizing strategy to achieve depalletizing includes the following sub-steps: Receive the destacking strategy to be executed; When the destacking strategy is the first destacking strategy, the location of the laser radar is set as a waypoint, and the shortest route from the stack position to the destination is planned; When the destacking strategy is the second destacking strategy, the shortest route from the stack position to the destination is directly planned.

4. The destacking method according to claim 1, wherein: The executing grasping step includes obtaining the height difference of the current box from the original position by the 3D camera again after the current grasped box leaves the stack, and obtaining the detected height value of the current box, including the following sub-steps: Detect the grasping signal and the grasping coordinate information of the manipulator; Locate the ROI area based on the captured coordinate information; Get the average height coordinates of the ROI area through the 3D camera; Calculate the difference between the height coordinate and the height coordinate of the top box to obtain the detection height value of the current box.

5. The destacking method according to claim 1, characterized in that: When it is determined that the number of times that the difference between the detected height value and the verified height value is not within the preset range exceeds a first preset value, the following sub-steps are executed: Obtaining historical calibration altitude values ​​within a second preset time period; Get the box length and width information corresponding to the historical calibration height value; The verification detection height values ​​are clustered according to the box length and width information to form a detection height value array.

6. The destacking method according to claim 5, characterized in that: The determining whether the difference between the current box height value and the calibration height value is within a preset range, and if so, executing the second depalletizing strategy; otherwise, obtaining an output height value by scanning the bottom of the box through the laser radar recognition, includes the following sub-steps: Get the length and width information of the current box; Perform a mapping comparison based on the box length and width information, and obtain a corresponding verification height value from the detection height value array; Determining whether the difference between the current box height value and the corresponding calibration height value is within a preset range; If so, the second depalletizing strategy is executed; otherwise, the bottom of the box is scanned by the laser radar to obtain an output height value.

7. A depalletizing system, characterized in that: include: 3D camera, used to obtain the length, width and height information of the boxes in the stack with the unpacked boxes; Laser radar, used to verify the height information of the unpacked box; A robot arm is used to destacker the stacks according to the preset destacking strategy and movement path; and A host computer is communicatively connected to the 3D camera, the laser radar and the manipulator, and controls the manipulator to destacker the stack using the destacking method according to any one of claims 1 to 6.

8. The depalletizing system according to claim 7, characterized in that: The laser radar is arranged at the bottom of the system, with the sensing direction facing upward.

9. A controller for a depalletizing system, characterized in that: include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to implement the steps in the depalletizing method according to any one of claims 1 to 6 to perform depalletizing when executed.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the device where the computer-readable storage medium is located executes the computer program, the depalletizing method according to any one of claims 1 to 6 is implemented.

Images