Container grabbing and releasing alignment method and system and container crane
By integrating lidar and cameras on container cranes to integrate radar point cloud and image data, the problem of container cranes being difficult to accurately identify and locate target containers in traditional technology is solved, and higher grabbing and placement accuracy and detection robustness are achieved.
Patent Information
- Application Number
- CN202311701648.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-11
- Publication Date
- 2025-06-13
AI Technical Summary
In automated yards, traditional container cranes find it difficult to accurately identify and locate the spatial location of the target container, resulting in low grabbing and placement accuracy.
By installing lidar and cameras at the bottom of the trolley platform of the container crane, the radar point cloud data and image data of the target container are obtained, and data fusion is performed to filter out the point cloud fusion data of the target container, so as to accurately calculate its spatial location.
Accurate position identification and positioning of the target container is achieved, the accuracy of grabbing and placement is improved, the background influence is eliminated, and the robustness of detection is improved.
Smart Images
Figure CN120135948A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of container cranes, and particularly to a method and system for grasping, releasing and aligning containers, and a container crane. Background Art
[0002] With the continuous popularization of automated yards, a large number of container cranes have been introduced into the automated yards to grasp or stack containers. At the same time, terminal users have higher and higher requirements for the operation efficiency of container cranes and the accuracy of grasping and releasing containers. In an automated yard, the accuracy of grasping and releasing containers is one of the key parameters to measure its automation level. Traditional detection schemes rely on sensors on the trolley to detect the position of the target container and automatically grasp and release the container. Generally, cameras or radars are mostly used as sensors. When using a camera, it is greatly affected by light and lacks distance information. When using a radar, it is difficult to accurately identify the radar point cloud in the area where the target container is located, resulting in inaccurate radar point cloud data. In summary, how to accurately know the spatial position of the target container is a technical problem that needs to be solved in this field. Summary of the Invention
[0003] In view of this, this application provides a method and system for grasping, releasing and aligning containers, and a container crane, which can accurately screen the radar point cloud of the target container based on the box contour, and can accurately know the spatial position of the target container through the radar point cloud in the point cloud fusion data, so as to accurately know the spatial position of the target container.
[0004] In a first aspect, a method for grasping, releasing and aligning containers provided by this application is applied to a container crane. The container crane includes a trolley platform and a spreader, and the spreader is connected to the trolley platform. A lidar and a camera are arranged at the bottom of the trolley platform. The lidar is used to detect radar point cloud data including containers, and the camera is used to detect image data including containers. The method for grasping, releasing and aligning containers includes: obtaining radar point cloud data and image data including a target container; fusing the radar point cloud data and the image data to screen the point cloud fusion data of the target container; calculating the position and attitude data of the target container according to the point cloud fusion data; and converting the position and attitude data to determine the box position of the target container relative to the center of the spreader.
[0005] When in use, this aspect can fuse radar point cloud data and image data. Through the image coordinates, the radar point cloud of the target container can be accurately screened based on the contour of the container body, so as to obtain the point cloud fusion data of the area where the target container is located. Through the radar point cloud in the point cloud fusion data, the spatial position of the target container can be accurately known, and thus the position of the container body of the target container relative to the center of the spreader can be accurately known. The target container is specifically the container that the spreader is about to grab, or the container where the container already grabbed by the spreader is about to be stacked. Compared with the traditional single radar detection or single image detection, the present invention can better extract the radar point cloud of the area where the target container is located based on the fusion data of the image data and the point cloud data, identify the position and posture of the target container at a long distance, eliminate the background influence, and improve the robustness of the detection.
[0006] Combined with the first aspect, in a possible implementation manner, the fusing the radar point cloud data and the image data to screen the point cloud fusion data of the target container includes: obtaining the external parameter matrix of the lidar and the camera; based on an image recognition algorithm, recognizing the image data to obtain the image coordinates of the target container; according to the external parameter matrix, the internal parameter matrix of the camera, the radar point cloud data, and the image data, projecting the radar point cloud data onto the image data; and according to the image coordinates, screening the target point cloud data of the target container from the radar point cloud data; the point cloud fusion data includes the target point cloud data; wherein, the calculating the position and posture data of the target container according to the point cloud fusion data includes: calculating the position and posture data according to the target point cloud data.
[0007] Combined with the first aspect, in a possible implementation manner, the calculating the position and posture data according to the target point cloud data includes: determining the characteristic coordinates of the target container according to the image coordinates; extracting the characteristic point cloud corresponding to the characteristic coordinates from the target point cloud data; and obtaining the position and posture data according to the characteristic point cloud.
[0008] Combined with the first aspect, in a possible implementation manner, the determining the characteristic coordinates of the target container according to the image coordinates includes: recognizing the top surface edge coordinates and top surface corner coordinates of the target container according to the image coordinates; the extracting the characteristic point cloud corresponding to the characteristic coordinates from the target point cloud data includes: respectively extracting the top surface edge point cloud corresponding to the top surface edge coordinates and the top surface corner point cloud corresponding to the top surface corner coordinates from the target point cloud data; wherein, the obtaining the position and posture data according to the characteristic point cloud includes: determining the position and posture data according to the top surface edge point cloud and the top surface corner point cloud.
[0009] In combination with the first aspect, in a possible implementation manner, the determining the characteristic coordinates of the target container according to the image coordinates further includes: determining the top surface area of the target container according to the top surface edge coordinates and the top surface corner coordinates; the extracting the characteristic point cloud corresponding to the characteristic coordinates from the target point cloud data includes: extracting the top surface point cloud within the top surface area from the target point cloud data; wherein, the obtaining the position and attitude data according to the characteristic point cloud includes: determining the position and attitude data according to the top surface point cloud.
[0010] In combination with the first aspect, in a possible implementation manner, the determining the characteristic coordinates of the target container according to the image coordinates includes: identifying the door coordinates of the target container according to the image coordinates; the extracting the characteristic point cloud corresponding to the characteristic coordinates from the target point cloud data includes: extracting the door point cloud corresponding to the door coordinates from the target point cloud data; wherein, the obtaining the position and attitude data according to the characteristic point cloud includes: determining the position and attitude data according to the door point cloud.
[0011] In combination with the first aspect, in a possible implementation manner, after fusing the radar point cloud data and the image data to screen the point cloud fusion data of the target container, the container grasping and placing alignment method further includes: obtaining the number of layers of the target container; obtaining the corresponding first point cloud density according to the number of layers; and performing point cloud filtering on the point cloud fusion data according to the first point cloud density.
[0012] In combination with the first aspect, in a possible implementation manner, after fusing the radar point cloud data and the image data to screen the point cloud fusion data of the target container, the container grasping and placing alignment method further includes: obtaining the top surface height of the target container; obtaining the corresponding second point cloud density according to the top surface height; and performing point cloud filtering on the point cloud fusion data according to the second point cloud density.
[0013] In combination with the first aspect, in a possible implementation manner, calculating the position and attitude data of the target container based on the point cloud fusion data includes: determining the relative position and attitude of the target container with respect to the lidar according to the point cloud fusion data; and converting the position and attitude data to determine the position of the container body of the target container with respect to the center of the spreader includes: obtaining the installation position of the lidar on the trolley platform; detecting the spreader position of the spreader center with respect to the trolley platform; calculating the position deviation between the lidar and the spreader center according to the installation position and the spreader position; and converting the relative position and attitude of the target container according to the position deviation to determine the position of the container body of the target container with respect to the trolley platform.
[0014] In a second aspect, the present application provides a container grasping and aligning system applied to a container crane. The container crane includes a trolley platform, and a lidar and a camera are disposed at the bottom of the trolley platform. The lidar is used to detect radar point cloud data including a container, and the camera is used to detect image data including a container. The container grasping and aligning system includes: a data acquisition module communicatively connected to the lidar and the camera respectively, and the data acquisition module is configured to: acquire radar point cloud data and image data including a target container; a fusion module communicatively connected to the data acquisition module, and the fusion module is configured to: fuse the radar point cloud data and the image data to screen the point cloud fusion data of the target container; and a position acquisition module communicatively connected to the fusion module, and the position acquisition module is configured to: calculate the position and attitude data of the target container according to the point cloud fusion data; and convert the position and attitude data to determine the position of the container body of the target container with respect to the center of the spreader.
[0015] The second aspect is the corresponding system of the first aspect, and the technical effects of the second aspect will not be elaborated here.
[0016] In a third aspect, the present application provides a container crane, including: a container crane body; a trolley platform disposed on the container crane body, and a lidar and a camera are disposed at the bottom of the trolley platform. The lidar is used to detect radar point cloud data including a container, and the camera is used to detect image data including a container; and the aforementioned container grasping and aligning system.
[0017] The third aspect includes the system of the second aspect, and the technical effects of the third aspect will not be elaborated here. Description of the Drawings
[0018] Figure 1The figure shows a schematic diagram of the method steps of a container grasping and placing alignment method provided by an embodiment of the present application.
[0019] Figure 2 The figure shows a schematic diagram of the method steps of a container grasping and placing alignment method provided by another embodiment of the present application.
[0020] Figure 3 The figure shows a schematic diagram of the method steps of a container grasping and placing alignment method provided by another embodiment of the present application.
[0021] Figure 4 The figure shows a schematic diagram of the method steps of a container grasping and placing alignment method provided by another embodiment of the present application.
[0022] Figure 5 The figure shows a schematic diagram of the method steps of a container grasping and placing alignment method provided by another embodiment of the present application.
[0023] Figure 6 The figure shows a schematic diagram of the method steps of a container grasping and placing alignment method provided by another embodiment of the present application.
[0024] Figure 7 The figure shows a schematic diagram of the method steps of a container grasping and placing alignment method provided by another embodiment of the present application.
[0025] Figure 8 The figure shows a schematic diagram of the method steps of a container grasping and placing alignment method provided by another embodiment of the present application.
[0026] Figure 9 The figure shows a schematic diagram of the method steps of a container grasping and placing alignment method provided by another embodiment of the present application.
[0027] Figure 10 The figure shows a schematic diagram of the system structure of a container grasping and placing alignment system provided by an embodiment of the present application.
[0028] Figure 11 The figure shows a schematic diagram of the system structure of a container crane provided by an embodiment of the present application.
[0029] Figure 12 This is a schematic diagram of the general process of the present application. Detailed implementation manners
[0030] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.
[0031] An exemplary container grasping and placing alignment method is as follows:
[0032] This application provides a container grasping and placing alignment method, which is applied to a container crane. The container crane includes a moving trolley, and the moving trolley includes a trolley platform and a spreader. A lidar and a camera are provided at the bottom of the trolley platform. Specifically, the lidar and the camera can be installed at the corners of the bottom of the trolley platform to increase the detection range and reduce the detection blind area. The lidar is used to detect lidar point cloud data including the container, and the camera is used to detect image data including the container.
[0033] Figure 1 The following is a schematic diagram of the method steps of a container grasping and placing alignment method provided by an embodiment of this application. As Figure 1 shown, the container grasping and placing alignment method includes:
[0034] Step 110: Obtain lidar point cloud data and image data including the target container.
[0035] In this step, the lidar point cloud data is detected by the lidar, and the image data is captured by the camera. This step obtains the detection data of the lidar and the camera.
[0036] Step 120: Fuse the lidar point cloud data and the image data to filter the point cloud fusion data of the target container.
[0037] In this step, since the lidar point cloud data detected by the lidar is likely to contain interfering point clouds. Through data fusion, the lidar point cloud data can be projected onto the image data, and the filtered lidar point cloud data can be obtained according to the image data as the point cloud fusion data to obtain the lidar point cloud data in the area where the target container is located.
[0038] Step 130: Calculate the position and attitude data of the target container according to the point cloud fusion data.
[0039] In this step, since the lidar point cloud data in the point cloud fusion data is data based on the lidar coordinate system, according to the point cloud fusion data in this step, the distance and azimuth of each position point of the target container relative to the lidar can be known, so as to know the position and attitude data of the target container relative to the lidar.
[0040] Step 140: Convert the position and attitude data to determine the box position of the target container relative to the center of the spreader.
[0041] In this step, according to the installation position of the lidar on the trolley platform, the position and attitude data is converted and determined, and the box position of the target container relative to the trolley platform can be obtained.
[0042] When this embodiment is in use, the radar point cloud data and the image data can be fused. Through the image coordinates, the radar point cloud of the target container can be accurately screened based on the contour of the container body, so as to obtain the point cloud fusion data of the area where the target container is located. Through the radar point cloud in the point cloud fusion data, the spatial position of the target container can be accurately known, and thus the position of the container body of the target container relative to the center of the spreader can be accurately known. The target container is specifically the container that the spreader is about to grab, or the container on which the container already grabbed by the spreader is about to be stacked. Compared with the traditional single radar detection or single image detection, this embodiment can better extract the radar point cloud of the area where the target container is located based on the fusion data of the image data and the point cloud data, identify the position and attitude of the target container at a long distance, eliminate the background influence, and improve the robustness of the detection.
[0043] In some embodiments, after obtaining the position of the container body, the spreader can be accurately controlled to grab the target container, or the container already grabbed can be accurately stacked on the target container.
[0044] Figure 2 The figure shows a schematic diagram of the method steps of a container grasping and placing alignment method provided by another embodiment of the present application. In one embodiment, as Figure 2 shown, step 120 includes:
[0045] Step 121, obtain the external parameter matrix of the lidar and the camera.
[0046] Step 122, based on the image recognition algorithm, recognize the image data to obtain the image coordinates of the target container.
[0047] In this step, a deep learning method can be used to preset the characteristics of the container in advance, and the yolov8 model is used to perform feature recognition on the image data, so as to extract the image of the target container from the image data of the camera. After the image of the target container is extracted, the image coordinates of the target container can be obtained, that is, the spatial coordinates of the pixel points of the target container image are known.
[0048] Step 123, project the radar point cloud data onto the image data according to the external parameter matrix, the internal parameter matrix of the camera, the radar point cloud data, and the image data.
[0049] In this step, through the extrinsic parameter matrix and the intrinsic parameter matrix, the radar point cloud data and the image data can be correlated with each other, so that the radar point cloud data can be projected onto the image data, that is, the reference systems of the radar point cloud data and the image data are unified. After the reference system is unified, the radar point cloud data can be accurately filtered according to the coordinates of the image data. For example, when it is necessary to filter the point cloud data of the target container, according to the spatial coordinates of the target container in the image data, the shape contour of the target container can be obtained, and the point cloud data can be filtered from all the radar point cloud data detected by the lidar according to the shape contour. Specifically, the calculation method of projecting the radar point cloud data onto the image data is as follows:
[0050]
[0051] In the above formula, x, y, and z are the coordinates of the radar point cloud data, u and v are the coordinates of the image data, K is the intrinsic parameter matrix of the camera, and the extrinsic parameter matrix includes: the rotation matrix R and the translation matrix T of the lidar relative to the camera.
[0052] Step 124: Filter the target point cloud data of the target container from the radar point cloud data according to the image coordinates.
[0053] The aforementioned point cloud fusion data includes the target point cloud data. In this step, based on the image coordinates of the target container obtained by the image recognition algorithm in step 122 and combined with the corresponding relationship formed by the data projection in step 123, the radar point cloud data within the spatial coordinates of the target container can be extracted and filtered, that is, the radar point cloud corresponding to the target container is obtained, and the radar point cloud outside the target container is excluded, that is, the target point cloud data corresponding to the target container is obtained. The spatial position of the target container can be clearly obtained through the radar point cloud.
[0054] Based on steps 121 to 124, step 130 includes:
[0055] Step 131: Calculate the position and attitude data according to the target point cloud data.
[0056] In this step, the existing radar point cloud calculation method can be used to obtain the target object coordinates and azimuth angle according to the radar point cloud data, that is, the position and attitude data are obtained, so as to know the position and attitude of the current state of the target container.
[0057] Figure 3 The figure shows the schematic diagram of the method steps of a container grasping and placing alignment method provided by another embodiment of the present application. In one embodiment, as Figure 3 shown, step 131 includes:
[0058] Step 1311: Determine the characteristic coordinates of the target container according to the image coordinates.
[0059] In this step, the feature positions are preset, and the feature coordinates are extracted from the image coordinates. Only for the feature coordinates can the data volume and the calculation amount be reduced.
[0060] Step 1312: Extract the feature point cloud corresponding to the feature coordinates from the target point cloud data.
[0061] In this step, the target point cloud data is screened according to the feature coordinates, and the point cloud data at the feature coordinates is screened out as the feature point cloud.
[0062] Step 1313: Obtain the position and attitude data according to the feature point cloud.
[0063] In this step, only calculating the position and attitude data according to the feature point cloud can save and reduce the calculation amount and improve the calculation efficiency.
[0064] Figure 4 The following is a schematic diagram of the method steps of a container grasping and placing alignment method provided by another embodiment of the present application. In one embodiment, as Figure 4 shown, specifically, step 1311 includes:
[0065] Step 1314: Identify the top edge coordinates and top corner coordinates of the target container according to the image coordinates.
[0066] In this step, the top edge and the top corners are set as the feature positions of the target container.
[0067] Step 1312 includes:
[0068] Step 1315: Respectively extract the top edge point cloud corresponding to the top edge coordinates and the top corner point cloud corresponding to the top corner coordinates from the target point cloud data.
[0069] Step 1313 includes:
[0070] Step 1316: Determine the position and attitude data according to the top edge point cloud and the top corner point cloud.
[0071] In this embodiment, since the lidar detects the target container from top to bottom, the top edge and the top corners of the target container can determine the position and attitude of the target container. In this embodiment, the top edge and the top corners of the target container are used as the feature positions, the top edge point cloud and the top corner point cloud are extracted, and the position and attitude data can be determined according to the top edge point cloud and the top corner point cloud. Specifically, according to the top edge point cloud and the top corner point cloud, the first box position of the edge of the target container relative to the trolley platform and the second box position of the top corner of the target container relative to the trolley platform can be obtained, and the position and attitude of the target container can be determined through the first box position and the second box position.
[0072] Figure 5 The figure shows a schematic diagram of the method steps of a container grasping and placing alignment method provided by another embodiment of the present application. In one embodiment, as Figure 5 shown, step 1311 includes:
[0073] Step 1314: Identify the top edge coordinates and top corner coordinates of the target container according to the image coordinates.
[0074] Step 1317: Determine the top surface area of the target container according to the top edge coordinates and top corner coordinates.
[0075] Step 1312 includes:
[0076] Step 1318: Extract the top surface point cloud within the top surface area from the target point cloud data.
[0077] Step 1313 includes:
[0078] Step 1319: Determine the position and attitude data according to the top surface point cloud.
[0079] In this embodiment, after first determining the top edge coordinates and top corner coordinates, the area enclosed by the top edge coordinates and top corner coordinates is the top surface area of the target container. In this embodiment, the radar point cloud located within the top surface area is screened, and the position and attitude data can be determined according to the top surface point cloud.
[0080] Figure 6 The figure shows a schematic diagram of the method steps of a container grasping and placing alignment method provided by another embodiment of the present application. In one embodiment, as Figure 6 shown, step 1311 includes:
[0081] Step 1320: Identify the door coordinates of the target container according to the image coordinates.
[0082] In this step, the door is further set as the characteristic position of the target container.
[0083] Step 1312 includes:
[0084] Step 1321: Extract the door point cloud corresponding to the door coordinates from the target point cloud data.
[0085] Step 1313 includes:
[0086] Step 1322: Determine the position and attitude data according to the door point cloud.
[0087] In this embodiment, by extracting the door point cloud, the orientation of the door of the container can be known, and further the current accurate orientation of the target container can be known, effectively improving the accuracy of the position and attitude data.
[0088] Figure 7 The figure shows a schematic diagram of the method steps of a container grasping and placing alignment method provided by another embodiment of the present application. In one embodiment, as Figure 7 shown, after step 120, the container grasping and placing alignment method further includes:
[0089] Step 150: Obtain the number of layers of the target container.
[0090] In this step, the number of layers of the target container refers to which layer the current target container is located in the stacked containers, and can be input by the staff or detected by the system.
[0091] Step 160: Obtain the corresponding first point cloud density according to the number of layers.
[0092] In this step, due to the detection characteristics of the lidar, the lidar has different point cloud densities for detection results at different height positions. The point cloud densities corresponding to the height positions of different layers are preset in advance. After obtaining the number of layers of the target container, this step determines the point cloud density corresponding to the number of layers as the first point cloud density according to the corresponding relationship.
[0093] Step 170: Perform point cloud filtering on the point cloud fusion data according to the first point cloud density.
[0094] In this step, performing point cloud filtering on the point cloud fusion data can filter out the point cloud regions that do not meet the first point cloud density, that is, filter out the background noise, so as to obtain the point cloud fusion data that meets the first point cloud density. The point cloud fusion data at this layer is the point cloud data of the top surface of the target container, thereby further improving the signal-to-noise ratio of the radar point cloud on the top surface of the target container, and then calculating the position and attitude data of the target container according to the filtered point cloud fusion data.
[0095] Figure 8 The figure shows a schematic diagram of the method steps of a container grasping and placing alignment method provided by another embodiment of the present application. In one embodiment, as Figure 8 shown, after step 120, the container grasping and placing alignment method further includes:
[0096] Step 180: Obtain the top surface height of the target container.
[0097] In this step, the top surface height of the target container refers to the height of the top surface of the current target container relative to the ground, and can be input by the staff or detected by the system.
[0098] Step 190: Obtain the corresponding second point cloud density according to the top surface height.
[0099] In this step, based on the detection characteristics of the lidar, the lidar has different point cloud densities for detection results at different height positions. Different point cloud densities corresponding to different heights are preset in advance. After obtaining the top surface height of the target container, in this step, the point cloud density corresponding to the top surface height of the target container is determined according to the corresponding relationship as the second point cloud density.
[0100] Step 200: Perform point cloud filtering on the point cloud fusion data according to the second point cloud density.
[0101] In this step, by performing point cloud filtering on the point cloud fusion data, the point cloud regions that do not meet the second point cloud density can be filtered out, that is, the background noise is filtered out, so as to obtain the point cloud fusion data that meets the second point cloud density. The point cloud fusion data at this height position is the point cloud data of the top surface of the target container, thereby further improving the signal-to-noise ratio of the radar point cloud on the top surface of the target container, and then calculating the position and attitude data of the target container according to the filtered point cloud fusion data.
[0102] Figure 9 The following is a schematic diagram of the method steps of a container grasping and placing alignment method provided by another embodiment of the present application. Specifically, as Figure 9 shown, step 130 includes:
[0103] Step 132: Determine the relative position and attitude of the target container with respect to the lidar according to the point cloud fusion data.
[0104] In this step, since the point cloud fusion data is essentially obtained by lidar detection, the reference coordinate system of the point cloud fusion data is the reference coordinate system of the lidar, and the relative position and attitude of the target container and the lidar directly calculated according to the point cloud fusion data.
[0105] Step 140 includes:
[0106] Step 141: Obtain the installation position of the lidar on the trolley platform.
[0107] Step 142: Detect the spreader position of the spreader center relative to the trolley platform.
[0108] Step 143: Calculate the position deviation between the lidar and the spreader center according to the installation position and the spreader position.
[0109] Step 144: Convert the relative position and attitude of the target container according to the position deviation to determine the box body position of the target container relative to the spreader center.
[0110] In this embodiment, the relative position and attitude data are subjected to two coordinate conversions according to the position deviation. For the first time, the position of the target container is converted into data with the trolley platform as the reference coordinate according to the installation position of the lidar. For the second time, the position of the target container is converted into data with the center of the spreader as the reference coordinate according to the spreader position, so as to enable the spreader on the mobile trolley to grasp and release the target container. Specifically, the current position of the spreader relative to the trolley platform can be determined through the current status parameters of the crane system, and the spreader position of the center of the spreader relative to the trolley platform can be obtained according to the spreader size parameters.
[0111] Referring to Figure 12 , Figure 12 is a schematic diagram of the general process of this application. First, lidar detection and camera shooting are performed. The image data obtained by camera shooting is subjected to image recognition to obtain the image coordinates of the target container. Then, data fusion of the radar point cloud and the image data is performed, and pose conversion is performed according to the fusion data, so as to determine the position of the target container relative to the trolley platform.
[0112] An exemplary container grasping and releasing alignment system is as follows:
[0113] Figure 10 Shown is a schematic diagram of the system structure of a container grasping and releasing alignment system provided by an embodiment of this application. This application also provides a container grasping and releasing alignment system. As Figure 10 shown, this system is applied to a container crane. The container crane includes a trolley platform. A lidar and a camera are arranged at the bottom of the trolley platform. The lidar is used to detect radar point cloud data including the container, and the camera is used to detect image data including the container; the container grasping and releasing alignment system includes: a data acquisition module 101, a fusion module 102, and a position acquisition module 103.
[0114] The data acquisition module 101 is respectively communicatively connected to the lidar and the camera. The data acquisition module 101 is configured to: acquire radar point cloud data and image data including the target container.
[0115] The fusion module 102 is communicatively connected to the data acquisition module 101. The fusion module 102 is configured to: fuse the radar point cloud data and the image data to screen out the point cloud fusion data of the target container.
[0116] The position acquisition module 103 is communicatively connected to the fusion module 102. The position acquisition module 103 is configured to: calculate the position and attitude data of the target container according to the point cloud fusion data; and perform conversion on the position and attitude data to determine the position of the target container relative to the trolley platform.
[0117] In this embodiment, the radar point cloud data is detected by a lidar, and the image data is captured by a camera. In this step, the detection data of the lidar and the camera are acquired. Since the radar point cloud data detected by the lidar is likely to contain interfering point clouds. Through data fusion, the radar point cloud data can be projected onto the image data, and the filtered radar point cloud data can be obtained based on the image data as the point cloud fusion data to accurately obtain the radar point cloud data corresponding to the target container. Since the radar point cloud data in the point cloud fusion data is data based on the lidar coordinate system, in this step, according to the point cloud fusion data, the distances and azimuths of the respective position points of the target container relative to the lidar can be known, thereby knowing the position and attitude data of the target container relative to the lidar. According to the installation position of the lidar on the trolley platform, the position and attitude data are converted and determined, and then the position of the target container relative to the trolley platform can be obtained.
[0118] This embodiment can fuse the radar point cloud data and the image data to obtain a more accurate radar point cloud, so as to accurately know the position of the target container relative to the trolley platform. The target container is specifically the container that the spreader is about to grab, or the container on which the grabbed container is about to be stacked by the spreader. After obtaining the position of the container body, the spreader can be accurately controlled to grab the target container, or the grabbed container can be accurately stacked on the target container.
[0119] An exemplary container crane is as follows:
[0120] Figure 11 The following shows a schematic structural diagram of a system of a container crane provided by an embodiment of the present application. The present application provides a container crane, which includes a container crane body, a moving trolley, and the aforementioned container grasping, releasing, and alignment system, as Figure 11 shown, the moving trolley includes a trolley platform 10 and a spreader. The spreader is connected to the trolley platform 10. The trolley platform 10 is arranged on the container crane body. A lidar 11 and a camera 12 are arranged at the bottom of the trolley platform 10. The lidar 11 is used to detect radar point cloud data including the container, and the camera 12 is used to detect image data including the container. Specifically, the lidar 11 and the camera 12 can be respectively arranged at two corners of the bottom of the trolley platform 10.
[0121] The basic principle of the present application is described above in conjunction with specific embodiments. However, it should be noted that the advantages, advantages, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, advantages, effects, etc. are essential for each embodiment of the present application. In addition, the above-disclosed specific details are only for the purpose of illustration and easy understanding, rather than limitations. The above details do not limit the present application to necessarily adopt the above specific details to implement.
[0122] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any way. Words such as "including", "comprising", "having", etc. are open-ended terms that mean "including but not limited to" and can be used interchangeably with each other. The word "or" and "and" used herein refer to the phrase "and / or" and can be used interchangeably with it, unless the context clearly indicates otherwise. The phrase "such as" used herein refers to the phrase "such as but not limited to" and can be used interchangeably with it.
[0123] It should also be noted that in the devices, equipment, and methods of this application, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of this application.
[0124] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.
[0125] The above are only the preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent replacements, etc. made within the spirit and principles of this application shall be included within the protection scope of this application.
Claims
1. A method for grasping, releasing and aligning a container, characterized in that, it is applied to a container crane, the container crane includes a trolley platform and a spreader, the spreader is connected to the trolley platform, a lidar and a camera are arranged at the bottom of the trolley platform, the lidar is used to detect radar point cloud data including the container, and the camera is used to detect image data including the container; the container grasping, releasing and aligning method includes: Obtaining radar point cloud data and image data including a target container; Fusing the radar point cloud data and the image data to screen out point cloud fusion data of the target container; Calculating position and attitude data of the target container according to the point cloud fusion data; and Converting the position and attitude data to determine the position of the container body of the target container relative to the center of the spreader.
2. The container grasping, releasing and aligning method according to claim 1, characterized in that, the fusing the radar point cloud data and the image data to screen out point cloud fusion data of the target container includes: Obtaining an external parameter matrix of the lidar and the camera; Based on an image recognition algorithm, recognizing the image data to obtain image coordinates of the target container; Projecting the radar point cloud data onto the image data according to the external parameter matrix, the internal parameter matrix of the camera, the radar point cloud data and the image data; and Screening out target point cloud data of the target container from the radar point cloud data according to the image coordinates; the point cloud fusion data includes the target point cloud data; wherein, the calculating the position and attitude data of the target container according to the point cloud fusion data includes: Calculating the position and attitude data according to the target point cloud data.
3. The container grasping, releasing and aligning method according to claim 2, characterized in that, the calculating the position and attitude data according to the target point cloud data includes: Determining characteristic coordinates of the target container according to the image coordinates; Extracting characteristic point clouds corresponding to the characteristic coordinates from the target point cloud data; and Obtaining the position and attitude data according to the characteristic point clouds.
4. The container grasping, releasing and aligning method according to claim 3, characterized in that, the determining the characteristic coordinates of the target container according to the image coordinates includes: Identifying top surface edge coordinates and top surface corner coordinates of the target container according to the image coordinates; the extracting the characteristic point clouds corresponding to the characteristic coordinates from the target point cloud data includes: Respectively extracting top surface edge point clouds corresponding to the top surface edge coordinates and top surface corner point clouds corresponding to the top surface corner coordinates from the target point cloud data; wherein, the obtaining the position and attitude data according to the characteristic point clouds includes: Determining the position and attitude data according to the top surface edge point clouds and the top surface corner point clouds.
5. The container grasping, releasing and aligning method according to claim 4, characterized in that, the determining the characteristic coordinates of the target container according to the image coordinates further includes: Determine the top surface area of the target container according to the top surface edge coordinates and the top surface corner coordinates; The extracting the feature point cloud corresponding to the feature coordinates from the target point cloud data includes: Extract the top surface point cloud within the top surface area from the target point cloud data; Wherein, the obtaining the position and attitude data according to the feature point cloud includes: Determine the position and attitude data according to the top surface point cloud.
6. The container grasping and placing alignment method according to claim 3, characterized in that The determining the feature coordinates of the target container according to the image coordinates includes: Identify the door coordinates of the target container according to the image coordinates; The extracting the feature point cloud corresponding to the feature coordinates from the target point cloud data includes: Extract the door point cloud corresponding to the door coordinates from the target point cloud data; Wherein, the obtaining the position and attitude data according to the feature point cloud includes: Determine the position and attitude data according to the door point cloud.
7. The container grasping and placing alignment method according to claim 1, characterized in that After fusing the lidar point cloud data and the image data to screen the point cloud fusion data of the target container, the container grasping and placing alignment method further includes: Obtain the number of layers of the target container; Obtain the corresponding first point cloud density according to the number of layers; and Perform point cloud filtering on the point cloud fusion data according to the first point cloud density.
8. The container grasping and placing alignment method according to claim 1, characterized in that After fusing the lidar point cloud data and the image data to screen the point cloud fusion data of the target container, the container grasping and placing alignment method further includes: Obtain the top surface height of the target container; Obtain the corresponding second point cloud density according to the top surface height; and Perform point cloud filtering on the point cloud fusion data according to the second point cloud density.
9. The container grasping and placing alignment method according to claim 1, characterized in that The calculating the position and attitude data of the target container according to the point cloud fusion data includes: Determine the relative position and attitude of the target container relative to the lidar according to the point cloud fusion data; The converting the position and attitude data to determine the position of the container relative to the center of the spreader includes: Obtain the installation position of the lidar on the trolley platform; Detect the spreader position of the spreader center relative to the trolley platform; Calculate the position deviation between the lidar and the spreader center according to the installation position and the spreader position; and Convert the relative position and attitude of the target container according to the position deviation to determine the position of the container relative to the center of the spreader.
10. A container grasping and placing alignment system, characterized in that Applied to a container crane, the container crane includes a trolley platform, and a lidar and a camera are provided at the bottom of the trolley platform. The lidar is used to detect radar point cloud data including a container, and the camera is used to detect image data including a container; the container grasping and placing alignment system includes: A data acquisition module, which is respectively communicatively connected to the lidar and the camera. The data acquisition module is configured to: acquire radar point cloud data and image data including a target container; A fusion module, which is communicatively connected to the data acquisition module. The fusion module is configured to: fuse the radar point cloud data and the image data to screen out the point cloud fusion data of the target container; and A position acquisition module, which is communicatively connected to the fusion module. The position acquisition module is configured to: calculate the position and attitude data of the target container according to the point cloud fusion data; and convert the position and attitude data to determine the position of the container body of the target container relative to the center of the spreader.
11. A container crane, characterized in that, it includes: A container crane body; A trolley platform, which is arranged on the container crane body. A lidar and a camera are provided at the bottom of the trolley platform. The lidar is used to detect radar point cloud data including a container, and the camera is used to detect image data including a container; A spreader, which is connected to the trolley platform; and The container grasping and placing alignment system according to claim 10.