Point cloud processing method and device for automatic operation of ship unloading robot

Abstract

The point cloud processing method for the automatic operation of the ship unloading robot does not depend on a specific ship type or training data, has high universality and adaptability, is high in processing speed, better meets the real-time requirement of the automatic operation of ship unloading, and improves the working efficiency of the ship unloading robot. Comprising the following steps: acquiring laser point cloud data of a laser radar mounted on a ship unloading robot; downsampling the laser point cloud data to obtain downsampled laser point cloud data, and segmenting the downsampled laser point cloud data to obtain ship body point cloud and bulk cargo point cloud; for the bulk cargo point clouds, interference point clouds and noisy points on a vertical arm of the ship unloading robot are removed, and interference-removed bulk cargo point clouds are obtained; based on the obtained ship body point cloud, ship body offset is calculated through a point cloud registration algorithm, and the ship body offset is used for correcting the position of the interference-removed bulk cargo point cloud; and performing point cloud completion on the corrected interference-removed bulk cargo point cloud to obtain a point cloud elevation map which is used for control processing of a ship unloading robot.

Description

Technical Field

[0001] The present invention belongs to the technical field of point cloud processing, and in particular relates to a point cloud processing method for automatic operation of a ship unloading robot. Background Art

[0002] Compared to conventional grab-type unloading robots, ship unloading robots can continuously unload bulk cargo, significantly improving bulk cargo unloading efficiency. Currently, ship unloading robots rely primarily on manual operation or wireless remote control, and their operation is significantly affected by human factors. While a few terminals have implemented automated operations, there is still significant room for improvement.

[0003] The automated operation of the ship unloading robot relies on LiDAR sensor data, using LiDAR point cloud data to perceive the operating area and plan the operation path. Furthermore, due to fluctuations in the amount of material in the ship's hold, the influence of tidal fluctuations, and the movement of the ship's hull as the robot's picker moves through the material, LiDAR point cloud data is needed to detect any deviations in the hull during the automated unloading process.

[0004] Existing point cloud processing methods for automatic operation of ship unloading robots, such as the method disclosed in patent publication number CN117788838A, rely on deep learning algorithms and require point cloud data of a certain quality for training. It is difficult to meet real-time requirements on general deployment equipment, and is difficult to expand to different ship types due to the influence of training data; although the method disclosed in patent publication number CN117963567A can meet real-time requirements, the method of identifying the cabin is too simple and not universal; the method disclosed in patent publication number CN114677424A installs a lidar scanner on a vertical arm, a dock, and a drone, and requires equipment such as a locator to assist in point cloud processing, which is too cumbersome to operate. Summary of the Invention

[0005] In response to the above problems, the present invention provides a point cloud processing method for automatic operation of ship unloading robots. It does not rely on specific ship types or training data, has strong versatility and adaptability, and has a fast processing speed, which better meets the real-time requirements of automatic ship unloading operations.

[0006] The technical solution is as follows: A point cloud processing method for automatic operation of a ship unloading robot, characterized in that it includes the following steps:

[0007] Obtain laser point cloud data from the laser radar installed on the ship unloading robot;

[0008] Downsampling the laser point cloud data to obtain downsampled laser point cloud data, and segmenting the downsampled laser point cloud data to obtain a hull point cloud and a bulk cargo point cloud;

[0009] For the bulk cargo point cloud, the interference point cloud and noise points on the vertical arm of the unloading robot are removed to obtain the de-interferenced bulk cargo point cloud;

[0010] Based on the obtained hull point cloud, the point cloud registration algorithm is used to calculate the hull offset, which is used to correct the position of the de-interfering bulk cargo point cloud;

[0011] The corrected de-interferenced bulk cargo point cloud is completed to obtain a point cloud elevation map, which is used for the control processing of the unloading robot.

[0012] Furthermore, the laser point cloud data of the laser radar installed on the ship unloading robot is obtained as follows:

[0013] Pre-synchronize multiple LiDARs and calibrate the coordinate conversion relationships between them, as well as the conversion relationship between the LiDARs and the dock coordinate system.

[0014] Scanning is performed by multiple laser radars. When all laser radars have acquired a new frame of data, all point cloud data are uniformly converted to the dock coordinate system according to the conversion relationship of the coordinate system to obtain a complete laser point cloud data PC. ori ;

[0015] Furthermore, the laser point cloud data is downsampled as follows:

[0016] According to the preset grid size, the laser point cloud data PC ori Perform 3D grid division to obtain the 3D grid set S vox ;

[0017] Traverse the 3D grid set S vox For each 3D grid in the image, calculate the mean of all point coordinates in the grid and the median of the intensity value of the point cloud, and combine the mean and median of the intensity value in each grid to obtain the downsampled laser point cloud data PC. ds ;

[0018] Furthermore, the downsampled laser point cloud data is segmented to obtain the hull point cloud and the bulk cargo point cloud, which is specifically performed as follows:

[0019] Calculate downsampled laser point cloud data PC ds The point cloud normal vector;

[0020] The downsampled laser point cloud data PC is converted according to the point cloud intensity ds Divided into point cloud PCs whose point cloud intensity is lower than the intensity threshold low and point cloud PC whose point cloud intensity is higher than the intensity threshold high ;

[0021] Traverse Point Cloud PC lowAll points in the cluster are classified into the same category if the distance between the points is less than the distance threshold and the angle between the normal vectors is less than the angle threshold, thus obtaining a clustering result containing multiple different point clouds;

[0022] Traverse the clustering results and put the point cloud with the largest number of point clouds into PC nm The area where it is located is considered to be the area where the bulk cargo point cloud is located;

[0023] Traverse Point Cloud PC nm , get the boundary value X of the point cloud in the plane X, Y axis up and down min 、X max 、Y min 、Y max , point cloud PC ds In the boundary value X min 、X max 、Y min 、Y max The point cloud in the formed area is classified as bulk point cloud PC cargo , the point cloud outside the boundary value is classified as the hull point cloud PC boat .

[0024] Furthermore, for the bulk cargo point cloud, the interference point cloud and noise points on the vertical arm of the unloading robot are removed to obtain the interference-free bulk cargo point cloud, which is specifically performed as follows:

[0025] According to the coordinates P of the unloading robot's material head head and the joint coordinate P above the vertical arm connecting the horizontal arm hoint , the vertical arm is modeled as a cylinder, according to the bulk point cloud PC cargo The intersection of the points in the image and the cylindrical area is used to remove the interference point cloud on the vertical arm, and then the point cloud filtering algorithm is used to remove the noise.

[0026] Further, the said bulk cargo point cloud PC cargo The intersection of the point in the vertical arm and the cylindrical area is removed, and the specific execution is as follows: Calculate the coordinates of the material head P head and the joint coordinate P above the vertical arm connecting the horizontal arm joint Vector V hj :

[0027] V hj =P head -P joint

[0028] Traverse bulk cargo point cloud PC cargo All points P in i , calculate P i With the material head coordinate P head Vector V hi :

[0029] V hi =P head -P i

[0030] According to the vector V hj and vector V hi Calculate P i to P head With P joint The distance D between the lines i :

[0031]

[0032] If the distance D i Greater than the distance threshold D threshold , then point P i Outside the cylinder modeled by the vertical arm, on the contrary, it is considered that point P i On the vertical arm, remove the corresponding point cloud.

[0033] Furthermore, for the bulk cargo point cloud, the interference point cloud and noise points on the vertical arm of the unloading robot are removed to obtain the interference-free bulk cargo point cloud, which is specifically performed as follows:

[0034] Traverse all bulk cargo point cloud PC cargo , calculate bulk cargo point cloud PC cargo The median of the height is used to mark the point cloud with a height lower than the median of the bulk point cloud as bulk, otherwise the point cloud is marked as other;

[0035] According to the X, Y axis coordinates in the plane and the fixed grid size, all bulk cargo point cloud PC cargo The data is divided into a two-dimensional grid G xy middle;

[0036] For a two-dimensional grid G xy For each row or column in the , use the sliding window to traverse the K data G in the corresponding row or column along the positive or negative direction i-k+1,j ,……,G i,j , use the least squares to PC all the point cloud data in K data ki The points marked as bulk are fitted with a quadratic curve L ij ;

[0037] Traversing point cloud data PC ki For the other points marked in the figure, calculate the distance from each point to the quadratic curve L ij When the distance exceeds the threshold, the current point is considered an outlier, the current point is removed, and the two-dimensional grid G ​​is traversed. xy All the data are then de-interfered to obtain the bulk point cloud.

[0038] Furthermore, calculate the distance from each point to the quadratic curve Lij The distance is as follows:

[0039] Select m coordinate points P1, ..., P on the quadratic curve between the current and the next two points according to the set step size. m , calculate the current point and each coordinate point P1, ..., P m The distance between D1, ..., D m , the distance D1, ..., D m The minimum distance between the current point and the quadratic curve L is taken as ij distance.

[0040] Furthermore, based on the obtained hull point cloud, the hull offset is calculated using a point cloud registration algorithm, which is specifically performed as follows:

[0041] The hull point cloud at the initial moment is recorded, and the new hull point cloud is matched with the hull point cloud at the initial moment using the point cloud registration algorithm to obtain the transformation relationship from the hull point cloud at the initial moment to the new hull point cloud, and the hull offset is obtained.

[0042] Furthermore, the point cloud completion for the bulk cargo point cloud is specifically performed as follows:

[0043] Divide the bulk cargo point cloud into a two-dimensional grid according to the X and Y coordinates in the plane and the fixed grid size, and calculate the mean height of the data in the grid;

[0044] The grid position where no data exists is regarded as the grid that needs to be completed. All grids that need to be completed are traversed, and the distance D between the current grid and the nearest grid with data in the positive and negative directions of the X and Y axes is calculated. xp 、D xn 、D yp 、D yn , and find the sum of the reciprocals of the four distance squares D , the sum of the reciprocals of all distance squares Sum D Sort in descending order to get the order of completion;

[0045] Calculate the completed point cloud in the order of completion, and calculate the grid mean height H of the nearest existing data in the positive and negative directions of the X and Y axes. xp 、H xn 、H yp 、H yn and distance D xp 、D xn 、D yp 、D yn , calculate the height value H of the current grid completion i :

[0046]

[0047] Among them, w xp 、w xn 、w yp 、w yn Represent weights,

[0048] According to the preset number of iterations N and the number of grids to be completed N c , get the number of grids completed in sequence each time [N c / N], each time complete [N c / N] grids, the completion order is recalculated and obtained, and then the grids are completed in the newly obtained completion order until all grids are completed to obtain a point cloud elevation map, which can be used for control processing of the ship unloading robot.

[0049] The point cloud processing method for automatic operation of ship unloading robots of the present invention can realize the processing and analysis of point cloud data without adopting deep learning algorithms, so that the requirements of the present invention on computing power are greatly reduced. It does not need to rely on high-performance GPUs or complex training models, and can run on general CPUs; compared with existing traditional methods, the specific implementation in the steps mostly adopts two-dimensional and three-dimensional grid data structures, and parallel technologies such as multi-threading can be used to parallelly process data in the grid, with high computational efficiency. The algorithm runs on a general CPU within 1 second, and can obtain results in real time and quickly, better meeting the real-time requirements of automatic ship unloading operations; using the method of the present invention, the entire process from data acquisition to completion of point cloud processing and generation of point cloud elevation maps can be completed in a short time, and can provide dynamic bulk cargo distribution and hull offset data to the ship unloading robot in real time, ensuring that the robot adjusts its path and action in time during operation to adapt to dynamic scenes such as material changes in the cabin and tide fluctuations. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A flowchart of the method in the embodiment;

[0051] Figure 2 This is a side view of the point cloud scanned by the laser radar installed on the ship unloading robot trolley in the embodiment;

[0052] Figure 3 This is a top view of the point cloud scanned by the laser radar installed on the ship unloading robot vehicle in the embodiment;

[0053] Figure 4 This is a point cloud image after point cloud downsampling in the embodiment;

[0054] Figure 5 The point cloud having an intensity higher than a preset laser point cloud intensity threshold in the embodiment;

[0055] Figure 6The point cloud having an intensity lower than a preset laser point cloud intensity threshold in the embodiment;

[0056] Figure 7 This is an effect diagram of the point cloud segmented into bulk cargo in the embodiment;

[0057] Figure 8 This is a diagram showing the effect of removing the interference point cloud and noise on the vertical arm in the embodiment;

[0058] Figure 9 This is the effect diagram of point cloud completion in the embodiment;

[0059] Figure 10 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION Specific embodiment 1:

[0061] See Figure 1 The point cloud processing method for the automatic operation of a ship unloading robot of the present invention comprises at least the following steps:

[0062] Step S1: Acquire laser point cloud data of the laser radar installed on the ship unloading robot;

[0063] Step S2: downsampling the laser point cloud data to obtain downsampled laser point cloud data, and segmenting the downsampled laser point cloud data to obtain a hull point cloud and a bulk cargo point cloud;

[0064] Step S3: For the bulk cargo point cloud, remove the interference point cloud and noise points on the vertical arm of the unloading robot to obtain the interference-free bulk cargo point cloud;

[0065] Step S4: Based on the obtained hull point cloud, the hull offset is calculated using a point cloud registration algorithm to correct the position of the de-interference bulk cargo point cloud;

[0066] Step S5: Perform point cloud completion on the corrected de-interference bulk cargo point cloud to obtain a point cloud elevation map for control processing of the unloading robot.

[0067] Compared with the method based on deep learning, the method of the present invention has lower requirements on computing power, which greatly reduces the deployment and development costs. It does not need to rely on high-performance computing equipment, which is conducive to promotion and use in resource-limited environments. The method of the present invention is not affected by the quality and quantity of sample data, and can avoid the problem of model performance degradation due to insufficient or low-quality training data, thereby improving the stability and reliability of the system in different environments and conditions. The method of the present invention does not rely on specific ship types or training data, has strong versatility and adaptability, and can better meet the unloading operation needs of different terminals and different ship types. Specific embodiment 2:

[0069] Specifically, in one embodiment of the present invention, two LivoxHAP laser radars are used in a non-repetitive scanning mode. The laser radars are installed on the ship unloading robot. In other embodiments, a larger number of laser radars may be used, or the laser radars may be installed in other locations. Step S1 may be specifically performed as follows:

[0070] Step 101: Pre-synchronize the time of the two laser radars and calibrate the coordinate conversion relationship between the two laser radars and the conversion relationship between the laser radar and the dock coordinate system;

[0071] Step 102: Scan with two laser radars. When all laser radars have acquired a new frame of data, all point cloud data are uniformly converted to the dock coordinate system according to the conversion relationship of the coordinate system to obtain a complete laser point cloud data PC. ori .

[0072] In this embodiment, the data accumulated for 1 second is considered as a new frame. In other embodiments, other time intervals such as 2 seconds or 3 seconds can also be used. The complete point cloud data after conversion to the unified coordinate system is as follows: Figure 2 Through multi-lidar time synchronization and coordinate system calibration, the temporal and spatial consistency of different radar data can be ensured, avoiding point cloud splicing misalignment problems caused by data asynchrony, improving the integrity and accuracy of point cloud data, and forming a globally consistent point cloud view after unified conversion to the dock coordinate system, facilitating unified analysis of the spatial position of the hull and bulk cargo in subsequent processing.

[0073] The present invention only requires the installation of a laser radar on the ship unloading robot cart, and data fusion is achieved through multi-radar time synchronization and coordinate system calibration. The equipment deployment is simple and the cost is low. The method of the present invention does not require additional auxiliary equipment such as locators, which can simplify the operating process and reduce system complexity and failure risks.

[0074] In the embodiment, step S2 is specifically performed as follows:

[0075] Step 201: PC the laser point cloud data according to the preset grid size ori Perform 3D grid division to obtain the 3D grid set S vox ;

[0076] Step 202: Traverse the 3D grid set S vox For each 3D grid in the image, calculate the mean of all point coordinates in the grid and the median of the intensity value of the point cloud, and combine the mean and median of the intensity value in each grid to obtain the downsampled laser point cloud data PC. ds ;

[0077] Step 203: Calculate the downsampled laser point cloud data PC using the plane fitting method ds The point cloud normal vector;

[0078] Step 204: PC the downsampled laser point cloud data according to the point cloud intensity ds It is divided into two categories: one is the point cloud PC whose point cloud intensity is lower than the intensity threshold low , the other type is point cloud PC whose point cloud intensity is higher than the intensity threshold high ;

[0079] Step 205: Point Cloud PC low Perform conditional Euclidean clustering and traverse the point cloud PC low All points in the cluster are classified into the same category if the distance between the points is less than the distance threshold and the angle between the normal vectors is less than the angle threshold, thus obtaining a clustering result containing multiple different point clouds;

[0080] Step 206: Traverse the clustering results and select the point cloud with the largest number of points as PC nm The area where it is located is considered to be the area where the bulk cargo point cloud is located;

[0081] Step 207: Traverse Point Cloud PC nm , get the boundary values ​​X of the point cloud in the X and Y axes in the dock coordinate system min 、X max 、Y min 、Y max , point cloud PC ds In the boundary value X min 、X max 、Y min 、Y max The point cloud in the formed area is classified as bulk point cloud PC cargo , the point cloud outside the boundary value is classified as the hull point cloud PC boat .

[0082] In this embodiment, the preset grid size in step S201 is 0.1 meters, and the point cloud data PC obtained after downsampling is ds like Figure 4 As shown. In this embodiment, the plane fitting method in step S203 adopts least squares for fitting. In other embodiments, the principal component analysis (PCA) method can also be used to calculate the eigenvector corresponding to the minimum eigenvalue of the local point cloud as the normal vector. The point cloud intensity threshold in step S204 is set to 5. In other embodiments, the point cloud intensity threshold can also be set to other values. The laser point cloud data contains laser point cloud intensity value data, which is affected by factors such as object color and material. Bulk cargo point clouds and hull point clouds can be roughly distinguished based on point cloud intensity values. Point clouds with a value higher than the preset point cloud intensity threshold are as follows. Figure 5As shown in the figure, the point clouds below the preset point cloud intensity threshold are as follows: Figure 6 As shown, the bulk point cloud obtained by point cloud segmentation is as follows Figure 7 shown.

[0083] In step S2, by downsampling, the amount of data is reduced while retaining the key features of the point cloud, which can reduce the computational complexity of subsequent processing and improve operational efficiency; by combining the point cloud intensity value with conditional Euclidean clustering, the hull point cloud and the bulk cargo point cloud can be effectively distinguished. This does not rely on prior knowledge of specific ship types, has low computing power requirements and is applicable to different ship types. Regardless of how the hull shape and size change, as long as there is a difference in the point cloud intensity between the bulk cargo and the hull, reliable segmentation can be achieved, and it has strong adaptability to different ship types.

[0084] In one embodiment of the present invention, step S3 is specifically performed as follows:

[0085] Step S311: For the bulk cargo point cloud, remove the interference point cloud and noise points on the vertical arm of the unloading robot to obtain the interference-free bulk cargo point cloud. The specific steps are as follows:

[0086] According to the coordinates P of the unloading robot's material head head and the joint coordinate P above the vertical arm connecting the horizontal arm joint , the vertical arm is modeled as a cylinder, according to the bulk point cloud PC cargo The point cloud on the vertical arm is removed by the intersection of the points in the image and the cylindrical area, and then the noise is removed by using a point cloud filtering algorithm. The point cloud filtering algorithm can be a radius filter, a statistical filter, or other point cloud filtering algorithms.

[0087] In the embodiment, according to the bulk cargo point cloud PC cargo The intersection of the point in the vertical arm and the cylindrical area is removed, and the specific execution is as follows: Calculate the coordinates of the material head P head and the joint coordinate P above the vertical arm connecting the horizontal arm joint Vector V hj :

[0088] V hj =P head -P joint

[0089] Traverse bulk cargo point cloud PC cargo All points P in i , calculate P i With the material head coordinate P head Vector V hi :

[0090] V hi =P head -P i

[0091] According to the vector V hj and vector V hi Calculate P i to P head With P joint The distance D between the lines i :

[0092]

[0093] If the distance D i Greater than the distance threshold D threshold , then point P i Outside the cylinder modeled by the vertical arm, on the contrary, it is considered that point P i On the vertical arm, remove the corresponding point cloud.

[0094] In this embodiment, since the vertical arm of the ship unloading robot is not a perfect cylinder, its upper part can be regarded as consisting of two cylinders, so the distance threshold D threshold According to the material head P head The joint coordinate P connecting the vertical arm and the horizontal arm joint The distance between the point on the connecting line and the material head P head The distance is set to two parts. When the distance to the material head is within 2 meters, the distance threshold D threshold is 0.6 meters. When the distance from the material head is 2 meters away, the distance threshold D threshold 2 meters.

[0095] In one embodiment of the present invention, step S4 is specifically performed as follows:

[0096] The hull point cloud at the initial moment is recorded, and the new hull point cloud is matched with the hull point cloud at the initial moment using the point cloud registration algorithm. The transformation relationship from the hull point cloud at the initial moment to the new hull point cloud is obtained, and the hull offset is obtained, which can be used to correct the coordinate position of the bulk cargo point cloud to accurately reflect the actual distribution of materials in the cabin.

[0097] In an embodiment, the point cloud registration algorithm can adopt a point cloud registration algorithm such as GICP or Teaser, which can accurately calculate the hull offset, provide a more accurate basis for the operation path planning and real-time adjustment of the unloading robot, and effectively reduce the operation errors and safety hazards caused by hull offset.

[0098] In one embodiment of the present invention, step S5 is specifically performed as follows:

[0099] Step 501: Divide the bulk cargo point cloud into a two-dimensional grid according to the X and Y axis coordinates in the plane and a fixed grid size, and calculate the mean height of the data in the grid;

[0100] Step 502: Grid positions where no data exists are considered as grids that need to be completed. For the grids that need to be completed, all grids that need to be completed are traversed and the distance D between the current grid and the nearest grid with data in the positive and negative directions of the X and Y axes is calculated. xp 、D xn 、D yp 、D yn , and find the sum of the reciprocals of the four distance squares D , where Sum D Expressed as:

[0101]

[0102] The sum of the reciprocals of the squares of all distances Sum D Sort in descending order to get the order of completion;

[0103] Step 503: Calculate the completed point cloud in the order of completion, and calculate the grid mean height H of the nearest existing data in the positive and negative directions of the X and Y axes. xp 、H xn 、H yp 、H yn and distance D xp 、D xn 、D yp 、D yn ,

[0104] D xp 、D xn 、D yp 、D yn They represent the distance of the current grid to the nearest existing data grid in the positive direction of the X axis, the distance of the current grid to the nearest existing data grid in the negative direction of the X axis, the distance of the current grid to the nearest existing data grid in the positive direction of the Y axis, and the distance of the current grid to the nearest existing data grid in the negative direction of the Y axis, respectively. Calculate the height value H of the current grid complement i :

[0105]

[0106] Among them, w xp 、w xn 、w yp 、w yn Represent weights,

[0107] Step 504: Based on the preset number of iterations N and the number of grids to be completed N c , get the number of grids completed in sequence each time [N c / N], where [N c / N] means for N c / N round up, each time fill in [N c / N] grids, the completion order is recalculated and obtained, and then the grids are completed in the newly obtained completion order until all grids are completed to obtain a point cloud elevation map, which can be used for control processing of the ship unloading robot.

[0108] In this example, the fixed grid size in step S51 is set to 0.2 meters. The fixed grid size is set according to the needs of subsequent tasks. The number of iterations N in step S54 is set to 20 times. The number of iterations can be adjusted. Figure 9 shown.

[0109] In step S5, the elevation value is calculated based on the inverse square of the distance between the grid to be completed and the surrounding known grids as the weight, making full use of the spatial correlation of the surrounding valid data, and reasonably filling the missing point cloud to generate a continuous and complete point cloud elevation map. Through iterative completion, the grids farther away from the known data are gradually filled to ensure the smoothness and accuracy of the completion result, providing the unloading robot with reliable terrain information of the working area, so that the robot can plan the optimal material collection path and depth. Specific embodiment 3:

[0111] Steps 1, 2, 4, and 5 of Example 3 are the same as those of Example 2. The difference from Example 2 is that in step S3, the interference point cloud and noise points on the vertical arm of the unloading robot are removed from the bulk cargo point cloud to obtain the de-interferenced bulk cargo point cloud. The specific steps are as follows:

[0112] Step 321: Traverse all bulk cargo point cloud PCs cargo , calculate bulk cargo point cloud PC cargo The median of the height is used to mark the point cloud with a height lower than the median of the bulk cargo point cloud as bulk cargo. If the condition is not met, the point cloud is marked as other.

[0113] Step 322: PC all bulk cargo point clouds according to the X and Y axis coordinates in the plane of the dock coordinate system and the fixed grid size cargo The data is divided into a two-dimensional grid G xy middle;

[0114] Step 323: For the two-dimensional grid G xy For each row or column in the , use the sliding window to traverse the K data G in the corresponding row or column along the positive or negative direction i-k+1,j ,……,G i,j , use the least squares to PC all the point cloud data in K data ki The points marked as bulk are fitted with a quadratic curve L ij ;

[0115] Step 324: Traverse the point cloud data PC ki For the other points marked in the figure, calculate the distance from each point to the quadratic curve L ij When the distance exceeds the threshold, the current point is considered an outlier, the current point is removed, and the two-dimensional grid G ​​is traversed. xy All the data are then de-interfered to obtain the bulk point cloud.

[0116] In this embodiment, the quadratic curve L is calculated from each point ij The distance is as follows:

[0117] Select m coordinate points P1, ..., P on the quadratic curve between the current and the next two points according to the set step size. m , calculate the current point and each coordinate point P1, ..., P m The distance between D1, ..., D m , the distance D1, ..., D m The minimum distance between the current point and the quadratic curve L is taken as ij distance.

[0118] In this example, the fixed grid size in step S322 is set to 0.1 meters, the sliding window size K in step S323 is set to 6, and the distance threshold in step S224 is set to 0.3 meters. The fixed grid size, sliding window size K, and distance threshold in the embodiment can all be adjusted. The filtering effect of removing the interference point cloud and noise on the vertical arm of the ship unloading robot is as follows: Figure 8 shown.

[0119] The present invention provides two methods for removing vertical arm interference point clouds in the embodiments, offering flexible options. The method provided in step S311 of embodiment 2 accurately models the ship unloading robot's structural parameters and is applicable to scenarios where the ship unloading robot's structure is known. It specifically removes interference points generated by the robot's own structure, avoiding misidentification of the vertical arm point cloud as bulk cargo or the ship's hull, thereby improving data purity. The method provided in steps S321-S324 of embodiment 3 determines outliers by fitting a quadratic curve, without relying on the robot's specific structure. Outliers are removed based on the spatial distribution characteristics of the point cloud data, making it applicable to various complex operating environments and enhancing the robustness of the solution.

[0120] Step S3 provides two targeted implementations to solve the problem of interference of the robot's own structure on the data, and avoid the problem that the existing solution may misjudge the interference point as the hull or bulk cargo, resulting in subsequent offset calculation or path planning errors.

[0121] In an embodiment of the present invention, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the point cloud processing method for automatic operation of the ship unloading robot as described above is implemented.

[0122] The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 10 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a point cloud processing method for automatic operation of the ship unloading robot is implemented. The display screen of the computer device can be a liquid crystal display or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0123] The memory may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), and electrically erasable programmable read-only memory (EEPROM). The memory is used to store programs, and the processor executes the programs after receiving execution instructions.

[0124] The processor can be an integrated circuit chip with signal processing capabilities. The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. The processor can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The various methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0125] Those skilled in the art will understand that Figure 10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0126] In an embodiment of the present invention, a computer-readable storage medium is further provided, on which a program is stored. When the program is executed by a processor, the point cloud processing method for automatic operation of the ship unloading robot as described above is implemented.

[0127] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, computer devices, or computer program products. Thus, embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0128] The embodiments of the present invention are described with reference to flowcharts and / or block diagrams of methods, computer devices, or computer program products according to the embodiments of the present invention. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, such that execution of the instructions by the processor of the computer or other programmable data processing terminal device produces means for implementing the functions specified in the flowcharts and / or block diagrams.

[0129] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in the flowchart.

[0130] In an embodiment of the present invention, a computer program product is further provided, comprising a computer program / instruction, which implements the steps of the above method when executed by a processor.

[0131] In actual application, the above-mentioned computer program products include but are not limited to: unloading robots, drones, smart phones, desktop computers, laptops, tablet computers, host computers and server platforms, etc., and are not specifically limited here.

[0132] The above is a detailed introduction to the point cloud processing method, computer device, computer-readable storage medium, and computer program product for automatic operation of ship unloading robots provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A point cloud processing method for automatic operation of ship unloading robots, characterized in that: The following steps are involved: Obtain laser point cloud data from the laser radar installed on the ship unloading robot; Downsampling the laser point cloud data to obtain downsampled laser point cloud data, and segmenting the downsampled laser point cloud data to obtain a hull point cloud and a bulk cargo point cloud; For the bulk cargo point cloud, the interference point cloud and noise points on the vertical arm of the unloading robot are removed to obtain the de-interferenced bulk cargo point cloud; Based on the obtained hull point cloud, the point cloud registration algorithm is used to calculate the hull offset, which is used to correct the position of the de-interfering bulk cargo point cloud; The corrected de-interferenced bulk cargo point cloud is completed to obtain a point cloud elevation map, which is used for the control processing of the unloading robot.

2. The point cloud processing method for automatic operation of a ship unloading robot according to claim 1, characterized in that: The laser point cloud data of the laser radar installed on the ship unloading robot is obtained as follows: Pre-synchronize multiple LiDARs and calibrate the coordinate conversion relationships between them, as well as the conversion relationship between the LiDARs and the dock coordinate system. Scanning is performed through multiple laser radars. When all laser radars obtain a new frame of data, all point cloud data are uniformly converted to the terminal coordinate system according to the conversion relationship of the coordinate system to obtain complete laser point cloud data.

3. The point cloud processing method for automatic operation of a ship unloading robot according to claim 1, characterized in that: Downsampling of laser point cloud data is performed as follows: According to the preset grid size, the laser point cloud data PC ori Perform 3D grid division to obtain the 3D grid set S vox ; Traverse the 3D grid set S vox For each 3D grid in the image, calculate the mean of all point coordinates in the grid and the median of the intensity value of the point cloud, and combine the mean and median of the intensity value in each grid to obtain the downsampled laser point cloud data PC. ds .

4. The point cloud processing method for automatic operation of a ship unloading robot according to claim 3, characterized in that: The downsampled laser point cloud data is segmented to obtain the hull point cloud and the bulk cargo point cloud. The specific execution is as follows: Calculate downsampled laser point cloud data PC ds The point cloud normal vector; The downsampled laser point cloud data PC is converted according to the point cloud intensity ds Divided into point cloud PCs whose point cloud intensity is lower than the intensity threshold low and point cloud PC whose point cloud intensity is higher than the intensity threshold high ; Traverse Point Cloud PC low All points in the cluster are classified into the same category if the distance between the points is less than the distance threshold and the angle between the normal vectors is less than the angle threshold, thus obtaining a clustering result containing multiple different point clouds; Traverse the clustering results and put the point cloud with the largest number of point clouds into PC nm The area where it is located is considered to be the area where the bulk cargo point cloud is located; Traverse Point Cloud PC nm , get the boundary value X of the point cloud in the plane X, Y axis up and down min 、X max 、Y min 、Y max , point cloud PC ds In the boundary value X min 、X max 、Y min 、Y max The point cloud in the formed area is classified as bulk point cloud PC cargo , the point cloud outside the boundary value is classified as the hull point cloud PC boat .

5. The point cloud processing method for automatic operation of a ship unloading robot according to claim 4, characterized in that: For the bulk cargo point cloud, the interference point cloud and noise points on the vertical arm of the unloading robot are removed to obtain the interference-free bulk cargo point cloud, which is specifically performed as follows: According to the coordinates of the unloading robot's material picking head and the coordinates of the joint connecting the vertical arm and the horizontal arm, the vertical arm is modeled as a cylinder. The interference point cloud on the vertical arm is removed according to the intersection of the points in the bulk cargo point cloud and the cylindrical area, and then the point cloud filtering algorithm is used to remove noise.

6. The point cloud processing method for automatic operation of a ship unloading robot according to claim 5, characterized in that: The above mentioned method of removing the interference point cloud on the vertical arm according to the intersection of the points in the bulk point cloud and the cylindrical area is specifically performed as follows: Calculate the coordinates P of the material taking head head and the joint coordinate P above the vertical arm connecting the horizontal arm joint Vector V hj : V hj =P head -P joint Traverse bulk cargo point cloud PC cargo All points P in i , calculate P i With the material head coordinate P head Vector V hi : V hi =P head -P i According to the vector V hj and vector V hi Calculate P i to P head With P joint The distance D between the lines i : If the distance D i Greater than the distance threshold D threshold , then point P i Outside the cylinder modeled by the vertical arm, on the contrary, it is considered that point P i On the vertical arm, remove the corresponding point cloud.

7. The point cloud processing method for automatic operation of a ship unloading robot according to claim 4, characterized in that: For the bulk cargo point cloud, the interference point cloud and noise points on the vertical arm of the unloading robot are removed to obtain the interference-free bulk cargo point cloud, which is specifically performed as follows: Traverse all bulk cargo point cloud PC cargo , calculate bulk cargo point cloud PC cargo The median of the height is used to mark the point cloud with a height lower than the median of the bulk point cloud as bulk, otherwise the point cloud is marked as other; According to the X, Y axis coordinates in the plane and the fixed grid size, all bulk cargo point cloud PC cargo The data is divided into a two-dimensional grid G xy middle; For a two-dimensional grid G xy For each row or column in the , use the sliding window to traverse the K data G in the corresponding row or column along the positive or negative direction i-k+1,j ,……,G i,j , use the least squares to PC all the point cloud data in K data ki The points marked as bulk are fitted with a quadratic curve L ij ; Traversing point cloud data PC ki For the other points marked in the figure, calculate the distance from each point to the quadratic curve L ij When the distance exceeds the threshold, the current point is considered an outlier, the current point is removed, and the two-dimensional grid G ​​is traversed. xy All the data are then de-interfered to obtain the bulk point cloud.

8. The point cloud processing method for automatic operation of a ship unloading robot according to claim 7, characterized in that: Calculate the distance from each point to the quadratic curve L ij The distance is as follows: Select m coordinate points P1, ..., P on the quadratic curve between the current and the next two points according to the set step size. m , calculate the current point and each coordinate point P1, ..., P m The distance between D1, ..., D m , the distance D1, ..., D m The minimum distance between the current point and the quadratic curve L is taken as ij distance.

9. The point cloud processing method for automatic operation of a ship unloading robot according to claim 1, characterized in that: Based on the obtained hull point cloud, the hull offset is calculated using the point cloud registration algorithm, which is performed as follows: The hull point cloud at the initial moment is recorded, and the new hull point cloud is matched with the hull point cloud at the initial moment using the point cloud registration algorithm to obtain the transformation relationship from the hull point cloud at the initial moment to the new hull point cloud, and the hull offset is obtained to correct the position of the bulk cargo point cloud.

10. The point cloud processing method for automatic operation of a ship unloading robot according to claim 9, characterized in that: The point cloud completion for bulk cargo point cloud is specifically performed as follows: Divide the corrected interference-free bulk cargo point cloud into a two-dimensional grid according to the X and Y axis coordinates in the plane and the fixed grid size, and calculate the mean height of the data in the grid; The grid position where no data exists is regarded as the grid that needs to be completed. All grids that need to be completed are traversed, and the distance D between the current grid and the nearest grid with data in the positive and negative directions of the X and Y axes is calculated. xp 、D xn 、D yp 、D yn , and find the sum of the reciprocals of the four distance squares D , the sum of the reciprocals of all distance squares Sum D Sort in descending order to get the order of completion; Calculate the completed point cloud in the order of completion, and calculate the grid mean height H of the nearest existing data in the positive and negative directions of the X and Y axes. xp 、H xn 、H yp 、H yn and distance D xp 、D xn 、D yp 、D yn , calculate the height value H of the current grid completion i : Among them, w xp 、w xn 、w yp 、w yn Represent weights, According to the preset number of iterations N and the number of grids to be completed N c , get the number of grids completed in sequence each time [N c / N], each time complete [N c / N] grids, recalculate and obtain the completion order, and then complete the grids in the newly obtained completion order until all grids are completed and the point cloud elevation map is obtained.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the point cloud processing method for automatic operation of the ship unloading robot according to any one of claims 1 to 10 is implemented.

12. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by the processor, the point cloud processing method for automatic operation of the ship unloading robot as described in any one of claims 1 to 10 is implemented.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images