Warehouse logistics sorting robot virtual and real synchronous calibration method, computer program product and system

Abstract

The invention provides a warehouse logistics sorting robot virtual-real synchronous calibration method, a computer program product and a system, and the method comprises the following steps: scanning a goods shelf through a visual module, and generating actual point cloud data containing space coordinates of the goods shelf; registering the actual point cloud data with a preset shelf virtual model, and dynamically calculating the coordinate offset of the shelf; and whether the coordinate offset is larger than a preset value or not is judged, if yes, the grabbing coordinates of the sorting robot are corrected according to the coordinate offset, and the corrected grabbing coordinates are fed back to the grabbing control module. The grabbing control module can control the sorting robot to automatically adapt to the position deviation of the goods shelf according to the corrected grabbing coordinates, the goods on the goods shelf with the position deviation are grabbed, and the grabbing coordinates of the sorting robot do not need to be manually calibrated again or the goods shelf does not need to be reset.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent warehousing, and particularly relates to a method for calibrating the virtual-real synchronization of a warehousing logistics picking robot, a computer program product, and a system. Background Art

[0002] With the rapid development of industrial robot technology, picking robots have been widely used in the warehousing logistics field to sort and transport goods to achieve fully automated operation. Most picking robots rely on preset fixed coordinates to grab goods. When the shelf undergoes position offset due to vibration, handling, or other external factors, they cannot adjust the grabbing position in real time and cannot dynamically adapt to the position offset of the shelf, resulting in a significant increase in the grabbing failure rate. Summary of the Invention

[0003] The technical problem to be solved by the present invention is how to enable the picking robot to automatically adapt to the position offset of the shelf.

[0004] To solve the above technical problem, the present invention provides a method for calibrating the virtual-real synchronization of a warehousing logistics picking robot, including the following steps: S1. Scan the shelf through a vision module to generate actual point cloud data containing the spatial coordinates of the shelf; S2. Register the actual point cloud data with a preset virtual model of the shelf, and dynamically calculate the coordinate offset of the shelf; S3. Determine whether the coordinate offset is greater than a preset value. If it is greater, correct the grabbing coordinates of the picking robot according to the coordinate offset, and feedback the corrected grabbing coordinates to the grabbing control module.

[0005] Further, in the step S1, after generating the actual point cloud data, the Gaussian filtering algorithm is used to preprocess the actual point cloud data to remove noise points, and the specific formula is as follows: ; Wherein, is the Gaussian weight coefficient, n ’ is the number of neighborhood points, is the spatial coordinate of the current point, is the spatial coordinate of the current point after removing noise points.

[0006] Further, in the step S2, the relational expression for registering the actual point cloud data with the preset virtual model of the shelf is: ; Wherein, is the actual point cloud data, is the point cloud of the virtual model, is the rotation transformation matrix, is the translation vector.

[0007] Further, in step S2, registering the actual point cloud data with the preset virtual model of the shelf specifically includes the following steps: S21. Obtain the initial positioning data of manual calibration by the user and feature point matching, and align the actual point cloud with the virtual model according to the initial positioning data; S22. Construct an error function based on each point of the virtual model and each point of the actual point cloud data, and then obtain the optimal rotation transformation matrix and translation vector by SVD decomposition to minimize the error function.

[0008] Further, in step S22, the specifically constructed error function is as follows: ; where is the i-th point in the virtual model, is the i-th point in the actual point cloud data, is the transformed point and the actual point the Euclidean distance between them.

[0009] Further, in step S22, obtaining the rotation transformation matrix and translation vector by SVD decomposition specifically includes the following steps: S221. Calculate the centroids of the virtual model and the actual point cloud data, and the formula is as follows: ; ; where is the centroid of the virtual model, is the centroid of the actual point cloud, and n is the number of points in the actual point cloud and the number of points in the virtual model point cloud; S222. Calculate the covariance matrix, and the formula is as follows: ; where is the covariance matrix, is the transpose matrix of; S223. Perform SVD decomposition on the covariance matrix to obtain: ; where U is the left singular matrix, V is the right singular matrix, S is the singular value matrix, is the transpose matrix of V; S224. Calculate the rotation transformation matrix and translation vector to obtain: ; ; where is the transpose matrix of U.

[0010] Further, in the step S2, the coordinate offset is specifically the translation vector The Euclidean distance of, and the calculation formula is: .

[0011] Further, in the step S3, before correcting the grasping coordinates of the picking robot according to the coordinate offset, the Kalman filter is used to perform state estimation on the offsets of 10 consecutive frames, including the following equations: State equation ; Observation equation ; Among them, is the true offset state, is the observed value, is the process noise, is the process noise, A is the state transition matrix, indicating how the system state changes over time, and H is the observation matrix, indicating how to obtain the observed value from the state variables.

[0012] The present invention also provides a computer program product, including a computer program, which when executed by a processor implements the steps in the method described above.

[0013] The present invention also provides a virtual-real synchronous calibration system for a warehousing logistics picking robot, including a vision module, a grasping control module, and a picking robot. The grasping control module is respectively connected to the vision module and the picking robot. The grasping control module includes a memory and a processor connected to each other, and the computer program product described above is stored in the memory.

[0014] The present invention has the following beneficial effects: The present invention registers the actual point cloud data including the shelf space coordinates with the preset virtual shelf model, dynamically calculates the coordinate offset of the shelf. If the coordinate offset is greater than the preset value, it means that the shelf has shifted in position due to vibration, handling, or other external factors. Therefore, the grasping coordinates of the picking robot are corrected according to the coordinate offset, and the corrected grasping coordinates are fed back to the grasping control module. The grasping control module can then control the picking robot to automatically adapt to the position offset of the shelf according to the corrected grasping coordinates, and grasp the goods on the shelf that has shifted in position, without the need for manual re-calibration of the grasping coordinates of the picking robot or resetting the shelf. Brief Description of the Drawings

[0015] Figure 1 is a schematic flowchart of the virtual-real synchronous calibration method for a warehousing logistics picking robot. Detailed Embodiments

[0016] The present invention will be further described in detail below in conjunction with specific embodiments.

[0017] This embodiment provides a virtual-real synchronous calibration system for a warehousing logistics picking robot. The system includes a vision module, a grasping control module, and a picking robot. The grasping control module is respectively connected to the vision module and the picking robot. Among them, the grasping control module includes a PLC controller. The picking robot has a robotic arm and an end effector installed on the robotic arm. The end effector is a suction cup or a gripper. The vision module includes a 3D camera installed at the end of the robotic arm, and point cloud data can be generated by collecting through the 3D camera. The grasping control module includes a memory and a processor connected to each other. A computer program product is stored in the memory, which includes a computer program. When the computer program is executed by the processor, it realizes the virtual-real synchronous calibration method of the warehousing logistics picking robot as shown in Figure 1 and specifically includes the following steps S1, S2, and S3.

[0018] S1. Scan the shelf through the vision module to generate actual point cloud data containing the spatial coordinates of the shelf.

[0019] The grasping control module scans the shelf through the vision module to generate high-precision actual point cloud data. Each point contains the spatial coordinates (x, y, z) of the shelf. Based on the point cloud plane fitting algorithm (such as RANSAC), the plane equation parameters of each layer board of the shelf can be accurately extracted according to the spatial coordinates of the shelf, and then the overall pose data of the shelf can be calculated, including the position (translation vector) and orientation (rotation matrix) of the shelf in space.

[0020] After generating the actual point cloud data, the grasping control module uses the Gaussian filtering algorithm to preprocess the actual point cloud data to remove noise points. The specific formula is as follows: ; where is the spatial coordinate of the current point, is the spatial coordinate of the current point after removing noise points, is the Gaussian weight coefficient, n ’ is the number of neighborhood points. The number of neighborhood points is specifically the number of all points within a set radius centered on the current point.

[0021] S2. Register the actual point cloud data with the preset virtual model of the shelf, and dynamically calculate the coordinate offset of the shelf.

[0022] A virtual model of the shelf is preset in the grasping control module. After generating the actual point cloud data of the shelf and removing noise points, the grasping control module registers the actual point cloud data with the preset virtual model of the shelf. The registration relationship is: ; Among them, is the actual point cloud data, is the virtual model point cloud, is the rotation transformation matrix, is the translation vector.

[0023] The registration specifically includes the following steps S21 and S22: S21. Obtain the initial positioning data of user manual calibration and feature point matching, and align the actual point cloud with the virtual model according to the initial positioning data; S22. Use the improved ICP algorithm for precise positioning. Specifically, construct an error function based on each point of the virtual model and each point of the actual point cloud data, and then obtain the optimal rotation transformation matrix and translation vector through SVD decomposition to minimize the error function.

[0024] In step S22, the constructed error function is specifically as follows: ; Among them, is the i-th point in the virtual model, is the i-th point in the actual point cloud data, is the transformed point and the actual point the Euclidean distance between them.

[0025] Since the actual point cloud and the virtual model have been aligned with the initial positioning data in step S21, that is, the corresponding points of the actual point cloud and the virtual model are known, so in step S22, the optimal rotation transformation matrix R and translation vector can be obtained through SVD decomposition to minimize the error function, specifically including the following steps S221, S222, S223, and S224: S221. Calculate the centroids of the virtual model and the actual point cloud data, and the formula is as follows: ; ; Among them, is the centroid of the virtual model, is the centroid of the actual point cloud, and n is the number of actual point clouds and the number of virtual model point clouds.

[0026] S222. Calculate the covariance matrix, and the formula is as follows: ; Among them, is the covariance matrix, is the transpose matrix of.

[0027] S223. Perform SVD decomposition on the covariance matrix to obtain: ; where U is the left singular matrix, V is the right singular matrix, S is the singular value matrix, is the transpose matrix of V.

[0028] S224. Calculate the rotation transformation matrix R and the translation vector , to obtain: ; ; where, is the transpose matrix of U.

[0029] After obtaining the translation vector in step S2, the grasping control module calculates the Euclidean distance of the translation vector as the coordinate offset of the shelf , and the specific calculation formula is: .

[0030] S3. Determine whether the coordinate offset is greater than the preset value. If it is greater, correct the grasping coordinates of the picking robot according to the coordinate offset and feedback the corrected grasping coordinates to the grasping control module.

[0031] After calculating the coordinate offset of the shelf, the grasping control module determines whether the coordinate offset is greater than the preset value (specifically 5 mm): If the coordinate offset is not greater than the preset value, it means that the shelf has not shifted in position or the offset is very small, and the current grasping coordinates are maintained; if the coordinate offset is greater than the preset value, it means that the shelf has shifted significantly in position due to vibration, handling, or other external factors. Therefore, Kalman filtering is used to perform state estimation on the offsets of 10 consecutive frames, and coordinate smoothing is performed, so as to dynamically optimize the coordinate offset estimation result of the shelf pose by fusing multi-source observation data, improve the registration accuracy and efficiency, and at the same time suppress noise and uncertainty. The state estimation using Kalman filtering includes the following equations: State equation ; Observation equation ; where, is the true offset state, is the observed value, is the process noise, is the process noise, A is the state transition matrix, indicating how the system state changes over time, and H is the observation matrix, indicating how to obtain the observed value from the state variables.

[0032] Then, the grasping control module calculates the coordinate transformation matrix according to the coordinate offset to correct the grasping coordinates of the picking robot, feedbacks the corrected grasping coordinates to the grasping control module, updates the PLC register, and the grasping control module can control the picking robot to automatically adapt to the shelf position offset according to the corrected grasping coordinates, and grasp the goods on the shelf with the position offset through the robotic arm, without manually recalibrating the grasping coordinates of the picking robot or resetting the shelf.

[0033] After controlling the picking robot to grasp the goods on the shelf with the position offset according to the corrected grasping coordinates, the grasping state is detected and verified by the force sensor. If the grasping fails, the exception handling mechanism is triggered and the system is stopped urgently.

[0034] As described above, it is only the implementation mode of the present invention, and does not limit the scope of patent protection. Those skilled in the art make non-substantive changes or substitutions based on the present invention, and still fall within the scope of patent protection.

Claims

1. A method for calibrating the virtual-reality synchronization of a warehousing and logistics picking robot, characterized in that It includes the following steps: S1. Scan the shelf through the vision module to generate actual point cloud data containing the spatial coordinates of the shelf; S2. Register the actual point cloud data with the preset virtual model of the shelf, and dynamically calculate the coordinate offset of the shelf; S3. Determine whether the coordinate offset is greater than the preset value. If it is greater, correct the grasping coordinates of the picking robot according to the coordinate offset, and feed the corrected grasping coordinates back to the grasping control module.

2. The virtual-real synchronous calibration method for the warehousing and logistics picking robot according to claim 1, characterized in that, In step S1, after generating the actual point cloud data, the Gaussian filtering algorithm is used to preprocess the actual point cloud data to remove noise points. The specific formula is as follows: ； Among them, is the Gaussian weight coefficient, and n ’ is the number of neighborhood points, is the spatial coordinate of the current point, is the spatial coordinate of the current point after removing the noise points.

3. The method for calibrating the virtual-reality synchronization of the warehousing and logistics picking robot according to claim 1, characterized in that In step S2, the relational expression for registering the actual point cloud data with the preset virtual model of the shelf is: ； Among them, is the actual point cloud data, is the point cloud of the virtual model, is the rotation transformation matrix, is the translation vector.

4. The virtual-real synchronous calibration method for the warehousing and logistics picking robot according to claim 3, characterized in that In step S2, registering the actual point cloud data with the preset virtual model of the shelf specifically includes the following steps: S21. Obtain the initial positioning data of user manual calibration and feature point matching, and align the actual point cloud with the virtual model according to the initial positioning data; S22. Construct an error function based on each point of the virtual model and each point of the actual point cloud data, and then obtain the optimal rotation transformation matrix and translation vector through SVD decomposition to minimize the error function.

5. The virtual-real synchronous calibration method for the warehousing and logistics picking robot according to claim 4, characterized in that, In step S22, the constructed error function is specifically as follows: ； Among them, is the i-th point in the virtual model, is the i-th point in the actual point cloud data, is the transformed point and the Euclidean distance between the actual point is 6. The virtual-real synchronous calibration method for the warehousing and logistics picking robot according to claim 5, characterized in that, In step S22, obtaining the rotation transformation matrix and translation vector through SVD decomposition specifically includes the following steps: S221. Calculate the centroids of the virtual model and the actual point cloud data. The formula is as follows: ； ； Among them, is the centroid of the virtual model, is the centroid of the actual point cloud, and n is the number of actual point clouds and the number of point clouds of the virtual model; S222. Calculate the covariance matrix. The formula is as follows: ； Among them, is the covariance matrix, is the transposed matrix of; S223. Perform SVD decomposition on the covariance matrix to obtain: ； Among them, U is the left singular matrix, V is the right singular matrix, and S is the singular value matrix, is the transposed matrix of V; S224. Calculate the rotation transformation matrix and translation vector to obtain: ； ； Among them, is the transpose matrix of U.

7. The virtual-real synchronization calibration method of the warehousing and logistics picking robot according to claim 1, characterized in that, In the said step S2, the coordinate offset is specifically the Euclidean distance of the translation vector , and the calculation formula is: .

8. The virtual-real synchronous calibration method of the warehousing and logistics picking robot according to claim 1, characterized in that In step S3, before correcting the grasping coordinates of the picking robot according to the coordinate offset, the Kalman filter is used to perform state estimation on the offsets of 10 consecutive frames, including the following equations: Equation of state ; Observation equation ; wherein, is the true offset state, is the observed value, is the process noise, is the process noise, A is the state transition matrix, indicating how the system state changes over time, and H is the observation matrix, indicating how to obtain the observed value from the state variables.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps in the method described in any one of claims 1 to 8.

10. A virtual-real synchronous calibration system for a warehousing and logistics picking robot, characterized in that, It includes a vision module, a grasping control module, and a picking robot. The grasping control module is respectively connected to the vision module and the picking robot. The grasping control module includes a memory and a processor connected to each other. The computer program product described in claim 9 is stored in the memory.

Images