A laser radar calibration method and system for unmanned spiral ship unloader

Abstract

The application discloses a kind of laser radar calibration method and system for unmanned spiral ship unloader, belong to point cloud calibration technical field, its technical solution key points are including, by multiple radars and preset parameter, the reference point cloud coordinates corresponding to each posture of vertical arm are obtained, multiple posture change rates are obtained according to reference point cloud coordinates, whether multiple posture change rates reach preset termination condition is judged, if not, update preset parameter, until multiple posture change rates reach preset termination condition, output current reference point cloud coordinates;According to current reference point cloud coordinates, transformation matrix is obtained, the application is realized by means of the coordinate of the vertical arm of spiral ship unloader, the calibration between laser radar coordinate system and wharf coordinate system, and based on the distribution characteristics of vertical arm end point cloud density, the problem of point cloud distortion caused by high humidity at sea is solved.

Description

Technical Field

[0001] This invention relates to the field of point cloud calibration technology, and more specifically to a lidar calibration method and system for unmanned screw unloaders. Background Technology

[0002] Unmanned screw unloaders rely on laser point cloud data from the ship's hold for automated operation. However, a single lidar unit often cannot acquire complete data from the ship's hold. The application of data from multiple lidar units requires the transformation relationships between the coordinate systems of each lidar unit, as well as the transformation relationship between the lidar unit and the dock coordinate system.

[0003] The coordinate system transformation relationship between multiple lidars can be calibrated through their shared viewing area. However, current technology has not yet been able to realize the transformation relationship between lidar and the dock coordinate system, so the existing technology has shortcomings. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to provide a laser radar calibration method and system for unmanned screw unloaders, which uses the coordinates of the vertical arm of the screw unloader to achieve calibration between the laser radar coordinate system and the dock coordinate system.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] This invention provides a lidar calibration method for unmanned screw unloaders, comprising:

[0007] The first iteration operation includes obtaining the reference point cloud coordinates of the vertical arm in each posture through multiple radars and preset parameters, obtaining multiple posture change rates based on the reference point cloud coordinates, determining whether the multiple posture change rates have reached a preset termination condition, and if not, updating the preset parameters until the multiple posture change rates have reached the preset termination condition, and outputting the current reference point cloud coordinates; the multiple radars are located on the unmanned spiral unloader.

[0008] The transformation matrix is ​​obtained based on the current reference point cloud coordinates.

[0009] As a further improvement of the present invention, the step of obtaining the reference point cloud coordinates of the vertical arm in each posture through multiple radars and preset parameters includes:

[0010] The second iteration operation includes obtaining initial point cloud data corresponding to the current attitude through multiple radars, obtaining a first transformation matrix based on the initial point cloud data, updating the first transformation matrix to obtain a second transformation matrix, obtaining reference point cloud data based on the second transformation matrix, obtaining reference point cloud coordinates corresponding to the current attitude based on the reference point cloud data and the preset parameters, updating the current attitude, until the reference point cloud coordinates corresponding to each attitude are obtained.

[0011] As a further improvement of the present invention, obtaining the first transformation matrix based on the initial point cloud data includes:

[0012] Select one point cloud data from the initial point cloud data as the reference point cloud data;

[0013] For each point cloud data in the remaining point cloud data, determine its overlapping area with the reference point cloud data. The remaining point cloud data is the data after removing the reference point cloud data from the initial point cloud data.

[0014] The first transformation matrix is ​​obtained based on the corresponding point pairs in the overlapping region.

[0015] As a further improvement of the present invention, updating the first transformation matrix to obtain the second transformation matrix includes:

[0016] The third iteration operation includes obtaining temporary point cloud data based on the current first change matrix and the remaining point cloud data, calculating the distance between the temporary point cloud data and the data points in the reference point cloud data, determining whether the distance reaches a preset second termination condition, and if not, updating the current first change matrix until the preset second termination condition is reached, and outputting the second change matrix.

[0017] As a further improvement of the present invention, the step of obtaining the reference point cloud coordinates corresponding to the current attitude based on the reference point cloud data and the preset parameters includes:

[0018] Based on the preset spatial region and the reference point cloud data, the data at the end of the vertical arm is obtained;

[0019] The data at the end of the vertical arm is preprocessed according to the preset parameters to obtain a set of filtered points;

[0020] The reference point cloud coordinates corresponding to the current attitude are obtained based on the filtered point set.

[0021] As a further improvement of the present invention, the preset parameters include a radius and a threshold. Preprocessing the data at the end of the vertical arm according to the preset parameters yields a set of filtered points, including:

[0022] For each data point in the data at the end of the vertical arm, the density corresponding to each data point is obtained based on the radius;

[0023] The filter point set is obtained based on the data at the end of the vertical arm, the density, and the threshold.

[0024] As a further improvement of the present invention, obtaining the reference point cloud coordinates corresponding to the current pose based on the filtered point set includes:

[0025] Based on the set of filtered points, the density weighting center corresponding to each radar is obtained;

[0026] Based on the density-weighted center and the radius, the set of associated points is obtained;

[0027] Based on the associated point set, the reference point cloud coordinates corresponding to the current attitude are obtained.

[0028] As a further improvement of the present invention, obtaining the reference point cloud coordinates corresponding to the current pose based on the associated point set includes:

[0029] Based on the aforementioned set of associated points, the fused coordinates are obtained;

[0030] The reference point cloud coordinates are obtained based on the distance between the data points in the filter point set and the fused coordinates.

[0031] As a further improvement of the present invention, the transformation matrix is ​​obtained based on the current reference point cloud coordinates, including:

[0032] Obtain the end coordinates of the vertical wall in each orientation;

[0033] The objective function is obtained based on the current reference point cloud coordinates and the end coordinates;

[0034] The objective function is solved using a nonlinear least squares algorithm to obtain the transformation matrix.

[0035] This invention provides a lidar calibration system for an unmanned screw unloader, comprising:

[0036] The calculation module is used to perform a first iteration operation, which includes obtaining the reference point cloud coordinates of the vertical arm at each attitude through multiple radars and preset parameters, obtaining multiple attitude change rates based on the reference point cloud coordinates, determining whether the multiple attitude change rates have reached a preset termination condition, and if not, updating the preset parameters until the multiple attitude change rates have reached the preset termination condition, and outputting the current reference point cloud coordinates; the multiple radars are located on the unmanned spiral unloader.

[0037] The calibration module is used to obtain the transformation matrix based on the current reference point cloud coordinates.

[0038] This invention is based on the principle that the point cloud density near the end of the vertical arm is higher than that in the distortion region. It utilizes the characteristics of point cloud density to accurately extract the reference point cloud coordinates, thus solving the problem of point cloud distortion caused by high humidity at sea. Then, by using the end coordinates of the vertical arm in the dock coordinate system and the reference point cloud coordinates, the conversion relationship between the lidar and the dock coordinate system is obtained, thus overcoming the shortcomings of the existing technology. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the method steps of the present invention;

[0040] Figure 2 This is a schematic diagram of the steps for the second iteration operation;

[0041] Figure 3 This is a schematic diagram of the steps for the third iteration operation;

[0042] Figure 4 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0043] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof.

[0044] The term "and / or" in the following text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0045] like Figure 1 As shown in the figure, this application provides a lidar calibration method for an unmanned screw unloader, including:

[0046] The first iteration operation includes obtaining the reference point cloud coordinates of the vertical arm in each attitude through multiple radars and preset parameters, obtaining multiple attitude change rates based on the reference point cloud coordinates, determining whether the multiple attitude change rates have reached the preset termination condition, and if not, updating the preset parameters until the multiple attitude change rates have reached the preset termination condition, and outputting the current reference point cloud coordinates; multiple radars are located on the unmanned spiral unloader.

[0047] The transformation matrix is ​​obtained based on the current reference point cloud coordinates.

[0048] Specifically, when an unmanned screw unloader is operating, it needs to use radar installed on the unloader to determine the location of the cargo and then unload it. When there is a large amount of cargo, one radar cannot scan all of it, so multiple lidars are required. This embodiment does not limit the specific number, as long as it can cover all the cargo. Furthermore, the location of the cargo scanned by the lidar is determined relative to its own coordinate system, not the cargo's location relative to the dock's coordinate system. Unloading is determined based on the cargo's location relative to the dock's coordinate system. Therefore, unloading cannot be completed solely based on the coordinates identified by the lidar. To solve the above problem, this embodiment uses the reference point cloud coordinates of the vertical arm, which are the coordinates of the vertical arm in the radar coordinate system. Sensors installed on the vertical arm obtain its end coordinates in the dock's coordinate system. By converting the end coordinates and the reference point cloud coordinates, the conversion relationship between the lidar and the dock's coordinate system is obtained. Thus, the accurate location of the cargo can be obtained based on the coordinates identified by the radar, and unloading can be completed.

[0049] The radar coordinate system is a local reference system for a single lidar, used to describe the spatial position of the point cloud acquired by the radar within its own field of view. The origin is usually the laser emission center of the lidar, and the X-axis is the horizontal forward direction of the radar, such as the main scanning direction of the laser beam. The dock coordinate system is a global reference system covering the entire unloading area, used to uniformly describe the position of the cargo and the vertical arm. The origin and direction are determined by engineering calibration, and this embodiment does not impose restrictions on them. For example, the center of the unloading machine gantry can be chosen as the origin. Furthermore, this embodiment does not restrict the selection of attitudes, but the selected attitudes should have continuity. For example, this embodiment controls the vertical arm to remain vertical, only changing the yaw angle of the vertical arm, resulting in 6 attitudes with yaw angles of ±10°, ±20°, and ±30°.

[0050] This embodiment is based on the principle that the point cloud density near the end position of the vertical arm is higher than that in the distortion region. It utilizes the characteristics of point cloud density to accurately extract the reference point cloud coordinates, solving the problem of point cloud distortion caused by high humidity at sea. Then, by using the end coordinates of the vertical arm in the dock coordinate system and the reference point cloud coordinates, the transformation relationship between the lidar and the dock coordinate system is obtained. Furthermore, this embodiment selects multiple postures to cover a series of motion ranges during vertical arm operation. Then, by utilizing the constraints of the rigid motion of the vertical arm, the change rate of the end coordinates under adjacent postures can be calculated to filter out abnormal coordinate results caused by differences in radar viewpoints, point cloud distortion, etc. At the same time, based on iterative operations, the preset parameters are continuously optimized to ensure the accuracy of the reference point cloud coordinates.

[0051] Furthermore, such as Figure 2 As shown, this application embodiment provides a step for obtaining the reference point cloud coordinates of a vertical arm in each posture using multiple radars and preset parameters, including:

[0052] The second iteration operation includes obtaining initial point cloud data corresponding to the current attitude through multiple radars, obtaining a first transformation matrix based on the initial point cloud data, updating the first transformation matrix to obtain a second transformation matrix, obtaining reference point cloud data based on the second transformation matrix, obtaining reference point cloud coordinates corresponding to the current attitude based on the reference point cloud data and preset parameters, updating the current attitude, until the reference point cloud coordinates corresponding to each attitude are obtained.

[0053] For each pose, initial point cloud data needs to be acquired through multiple radars. Therefore, the initial point cloud data is a set that includes the point cloud data acquired by each radar. By repeating the above iterative steps for each pose, the reference point cloud coordinates corresponding to each pose can be obtained.

[0054] This embodiment obtains the reference point cloud coordinates corresponding to each attitude through the second iteration operation, forming complete end motion trajectory data, which serves as the basis for subsequent calculation of the rate of change. This continuously corrects the conversion error from multi-radar point clouds to the reference coordinate system, effectively filters distortion interference, and enhances data consistency and robustness.

[0055] Furthermore, embodiments of this application provide a step for obtaining a first transformation matrix based on initial point cloud data, including:

[0056] Select one point cloud data from the initial point cloud data as the reference point cloud data;

[0057] For each point cloud data in the remaining point cloud data, determine its overlapping area with the reference point cloud data. The remaining point cloud data is the data after removing the reference point cloud data from the initial point cloud data.

[0058] The first transformation matrix is ​​obtained based on the corresponding point pairs in the overlapping regions.

[0059] Specifically, for any pose, its corresponding initial point cloud data is denoted as... ,in The number of radars. For the first The radar acquires point cloud data. Then, a point cloud data point is selected from the initial point cloud data as a reference point cloud data point, for example, selecting... Use this as a reference point cloud data. Then, for each point cloud data in the remaining point cloud data... The overlapping area between the radar and the reference point cloud data is determined. The overlapping area refers to the region that both radars can scan. For example, if both radars can scan the same wall of the ship's cabin, then the overlapping area is that wall. The overlapping area can be obtained based on the radar's installation pose, field of view, and scanning range. For each radar, its corresponding scanning range should be a cone with the radar origin as its vertex. Therefore, the overlapping area of ​​two radars should be the intersection of the two cones. After determining the overlapping area, multiple point pairs are manually selected within the overlapping area, for example... and The overlapping area is one wall in the cabin, first in Select multiple points on this wall, for example, select the four corners, then obtain the coordinates corresponding to these four corners, and then... The coordinates of these four angles are determined, resulting in four pairs of points. In each pair, two coordinates correspond to the same angle. For example, one pair of points is... , for The coordinates of one of the angles, For that corner in The corresponding coordinates, since the data coordinates acquired by each radar are based on its own coordinate system, even the same four corners will have different coordinates in the coordinate system. and The coordinates in the coordinates are also different.

[0060] This embodiment found that the different coordinates are due to two main reasons: different radar installation angles (e.g., the first radar is installed horizontally while the second radar is installed at a 30° angle), causing rotational deviations in the coordinates; and different radar installation positions (e.g., the first radar is on the left side of the gantry while the second radar is on the right side), causing translational deviations. Handling rotation and translation separately would increase computational complexity and increase the risk of errors. Therefore, this embodiment uses a homogeneous transformation matrix to describe both operations simultaneously. The homogeneous transformation matrix is ​​in the form of: ,in It is a 3×3 rotation matrix used to describe rotation relationships. It is a 3×1 translation vector used to describe the translation relationship. Since rotation is a linear transformation, it can be implemented by matrix multiplication, therefore, it is designed in the matrix. Translation is a non-linear transformation and cannot be achieved through matrix multiplication. Therefore, we need to add a matrix... Furthermore, in order to describe both linear (rotation) and nonlinear (translation) operations using a unified matrix multiplication method, homogeneous coordinates need to be introduced. This involves adding 1 to the selected three-dimensional coordinates to expand them into four-dimensional coordinates, as shown above. The coordinates are After being expanded to four-dimensional coordinates, it becomes For each selected pair of points, an expansion step is performed to obtain the four-dimensional coordinates corresponding to each coordinate.

[0061] Based on the homogeneous transformation matrix, the transformation formula for each point pair can be obtained, for example, for... The corresponding conversion formula is:

[0062]

[0063] in, for After obtaining the transformation formula for each point pair using the corresponding four-dimensional coordinates, combining these formulas yields a system of equations. Solving this system of equations provides the solution. and ,Will and Substituting the corresponding values ​​into the homogeneous transformation matrix above, we can obtain... and The corresponding transformation matrix is ​​then obtained. The above steps are repeated for each point cloud data in the remaining point cloud data to obtain the transformation matrix between each point cloud data in the remaining point cloud data and the reference point cloud data. The set of all transformation matrices is denoted as the first transformation matrix. ,in for and The corresponding change matrix, for and The corresponding transformation matrix. The system of equations can be solved using the least squares method or singular value decomposition, which are mathematical methods that can be implemented by those skilled in the art. This embodiment will not elaborate on these methods. Furthermore, the reference point cloud data, overlapping regions, and point pairs in this embodiment are merely examples and are not intended to limit the scope of the embodiment. Those skilled in the art can choose according to the actual situation.

[0064] This embodiment introduces a homogeneous transformation matrix, which integrates nonlinear translation operations into a linear transformation framework using homogeneous coordinates. This enables rotation and translation to be completed simultaneously using a single matrix multiplication, greatly simplifying the coordinate transformation process. The final transformation matrix between the radar coordinate system and the dock coordinate system is obtained, which serves as the basis for subsequent calibration between the radar coordinate system and the dock coordinate system.

[0065] Furthermore, such as Figure 3 As shown in the embodiment of this application, a step of updating a first transformation matrix to obtain a second transformation matrix is ​​provided, including:

[0066] The third iteration operation includes obtaining temporary point cloud data based on the current first change matrix and the remaining point cloud data, calculating the distance between the temporary point cloud data and the data points in the reference point cloud data, determining whether the distance has reached the preset second termination condition, and if not, updating the current first change matrix until the preset second termination condition is reached, and outputting the second change matrix.

[0067] This embodiment takes into account that since the point pairs in the first transformation matrix are manually selected, there may be errors. Therefore, in order to further reduce the errors caused by manual selection, this embodiment updates the first transformation matrix through a third iteration operation (i.e., the ICP algorithm).

[0068] For example, in the first iteration of the third iteration operation, for each point cloud data in the remaining point cloud data... According to its relationship with The corresponding transformation matrix is ​​transformed to Temporary point cloud data is obtained in the corresponding radar coordinate system. ,in for Transform to Point cloud data in the corresponding radar coordinate system, for Transform to The corresponding point cloud data in the radar coordinate system. Then, for each point cloud data point in the temporary point cloud data, the distance between it and the data points in the reference point cloud data is calculated. and For example, for Each data point in Find the data point that is closest to the given data point, and use that distance as... The distance corresponding to the data point can then be obtained. The sum of the distances corresponding to each data point in the dataset, and then the values ​​are calculated separately. and The corresponding centroid (mean), and respectively based on the centroid pair and Perform centroid removal, and then... and Calculate the covariance matrix and perform singular value decomposition on it to obtain the updated rotation matrix. and according to The updated translation vector is obtained from the relationship with the centroid. And thus obtain and The corresponding updated change matrix. Repeat the above steps for each point cloud data in the temporary point cloud data to obtain the sum of the distances corresponding to each data point in each point cloud data in the temporary point cloud data. Then, the updated change matrix between each point cloud data and the reference point cloud data can be obtained. The set of each updated change matrix is ​​used as the first updated change matrix.

[0069] Then, a second iteration is performed. This time, the updated first transformation matrix obtained in the first iteration is used as the current first transformation matrix. Then, based on the current first transformation matrix and the remaining point cloud data, the current temporary point cloud data is obtained, denoted as... Then, the process repeats the step of calculating the sum of distances corresponding to each data point in each point cloud data in the current temporary point cloud data, and compares the difference between the two calculated sums of distances, and determines whether the preset second termination condition is met. The preset second termination condition is that the difference is less than a preset threshold. If it is less than the threshold, the current first change matrix is ​​output, which is the updated first change matrix obtained in the first iteration, as the second change matrix. If it is greater than or equal to the threshold, the process of updating the current first change matrix is ​​repeated until the preset second termination condition is met, and the second change matrix is ​​output.

[0070] This embodiment updates the first transformation matrix by distance, which can gradually correct the deviation caused by manually selected point pairs, and finally obtain a more accurate transformation matrix. This achieves high-precision registration of multiple radar point clouds under a unified benchmark, and provides a foundation for obtaining the transformation matrix between the radar coordinate system and the dock coordinate system.

[0071] Furthermore, embodiments of this application provide a method for obtaining the reference point cloud coordinates corresponding to the current attitude based on reference point cloud data and preset parameters, including:

[0072] Based on the preset spatial region and reference point cloud data, the data at the end of the vertical arm is obtained;

[0073] The data at the end of the vertical arm is preprocessed according to preset parameters to obtain a set of filtered points;

[0074] The reference point cloud coordinates corresponding to the current attitude are obtained from the filtered point set.

[0075] In this embodiment, it was found that the working principle of lidar is to calculate the distance by emitting a laser beam and measuring the time and angle at which the beam reflects back to the sensor after encountering an object, thereby obtaining each data point (i.e., coordinates) in the point cloud data. However, in high humidity environments, there are a large number of tiny water droplets (aerosols) in the air. The diameter of these water droplets is usually between a few micrometers and tens of micrometers, which is on the same order of magnitude as the wavelength of the laser. When the laser beam passes through such air, it will undergo Mie scattering with the water droplets, causing some of the laser energy to not propagate in a straight line to the target object, but to be scattered in various directions by the water droplets. Among them, the scattered light from non-target directions may be misinterpreted by the radar receiver as the target reflection signal, thus forming discrete points unrelated to the real object in the point cloud data, i.e., noise points. Moreover, the higher the humidity, the greater the concentration of water droplets in the air, the more intense the scattering phenomenon, and the more noise points there are. Therefore, the high humidity environment at sea is prone to causing distortion in the point cloud data collected by lidar, i.e., the presence of noise, and the area where the noise points are located is the distortion area.

[0076] This embodiment found that the end of the vertical arm is a rigid structure, and the point cloud density near its true position is necessarily higher than that in the distortion region. The specific reason is that when the lidar scans, the laser beam continuously and stably illuminates its surface, thereby forming a large number of continuous reflection points near the true position. Since these reflection points come from the continuous surface of the same rigid structure, the spatial distance between them is relatively close, so the number of points per unit space (i.e., the point cloud density) is relatively high. In contrast, the data points in the distortion region are formed after the scattered light is received by the radar. They are scattered in space and have no fixed relationship with each other, so the number of points per unit space is small and the point cloud density is low. Furthermore, from the perspective of scanning logic, when the lidar scans the rigid end of the vertical arm, since the end is a solid structure, it will block the propagation of the laser, making the reflection of the laser in this region deterministic and continuous, thus forming a dense point cloud. However, the interference region does not have such a rigid entity to continuously reflect the laser, making it difficult to form a dense point cloud. Therefore, the point cloud density is necessarily lower than that near the end of the vertical arm.

[0077] In the structure of a ship unloader, it's not just the end of the vertical arm that is rigid; other components such as the fuselage and boom also possess rigidity. However, this embodiment found that the end of the vertical arm is the operating end of the ship unloader, requiring frequent shifts to different postures during operation. In contrast, components like the fuselage and boom are relatively fixed, with limited scanning angles, making it difficult to provide coordinate correspondences across multiple postures. This hinders the implementation of the transformation matrix derived from the posture change rate, as described in this embodiment. Furthermore, the relatively regular structure of the vertical arm end (e.g., cylindrical or square) facilitates coordinate extraction using density-concentrated features. Other rigid structures, on the other hand, are too large and complex in shape, making the point cloud susceptible to occlusion or interference, increasing the difficulty of coordinate extraction. Therefore, this embodiment uses point cloud density to distinguish the true position and distorted regions of the vertical arm end, thereby achieving accurate coordinate extraction of the vertical arm end.

[0078] Specifically, firstly, based on the second transformation matrix and the remaining point cloud data, the remaining point cloud data is transformed into data in the radar coordinate system corresponding to the reference point cloud data to obtain the reference point cloud data. At this time, the reference point cloud data includes the reference point cloud data and the remaining point cloud data transformed by the second transformation matrix. Furthermore, considering that the scanning range of the lidar is a large range, it is easy for the collected point cloud data to contain data corresponding to other rigid structures, affecting the accuracy of the reference point cloud coordinates. Therefore, this embodiment filters the reference point cloud data according to a preset spatial region, so that the reference point cloud data only includes data points near the end of the vertical arm. Specifically, each radar in this embodiment can scan the end of the vertical arm. Then, based on the kinematic model of the unloader (joint position, maximum swing angle, and extension range of the vertical arm), the preset spatial region corresponding to the end of the vertical arm is obtained. Then, only the data within this spatial region in the reference point cloud data is retained as the end data of the vertical arm. Then, the reference point cloud coordinates are obtained based on the end data of the vertical arm. This step is a technical means that can be implemented by those skilled in the art, and will not be described in detail here.

[0079] This embodiment can accurately define the range of the vertical arm's end by using a preset spatial region, effectively eliminating interference from other high-density structures in the scene. This ensures that the reference point cloud data for subsequent processing is focused on the vertical arm's end. Then, the data at the vertical arm's end is preprocessed according to preset parameters to obtain a filtered point set, which can further remove noise points and improve data purity. Finally, the reference point cloud coordinates corresponding to the current posture are obtained based on the filtered point set, which significantly improves the accuracy of the reference point cloud coordinates and lays a high-quality data foundation for subsequent coordinate fusion and transformation matrix calculation.

[0080] Furthermore, embodiments of this application provide a method for preprocessing data at the end of a vertical arm according to preset parameters to obtain a set of filtered points, including:

[0081] For each data point in the data at the end of the vertical arm, the density corresponding to each data point is obtained based on the radius;

[0082] The filter point set is obtained based on the data, density, and threshold at the end of the vertical arm.

[0083] The preset parameters include radius and threshold.

[0084] Specifically, since the reference point cloud data includes reference point cloud data and the remaining point cloud data transformed by the second transformation matrix, the vertical arm end-effector data selected using the reference point cloud data and the preset spatial region should be a data set corresponding to multiple radars. For example, the remaining point cloud data transformed by the second transformation matrix is ​​denoted as... ,in for The transformed data, for The transformed data is then filtered according to a preset spatial region. The data obtained at the end of the vertical arm includes ,in for The filtered data for The filtered data for The data after filtering.

[0085] Next for For each data point, the number of data points contained within a radius neighborhood centered on that data point is calculated using a KD-tree. The contained data points should be within... Then, this quantity is recorded as the point cloud density corresponding to that data point. Next, for... Repeating the above steps will yield the density corresponding to each data point in the data at the end of the vertical arm. This embodiment does not limit the value of the radius; for example, the radius can be set to 1.5 times the radius of the end of the vertical arm to ensure coverage of the end structure.

[0086] After that The point cloud density corresponding to each data point is compared with a threshold. If it is less than the threshold, the data point is removed; if it is greater than or equal to the threshold, the data point is retained. After performing both removal and retention operations, the filter point set is obtained. ,in for The data obtained after the elimination and retention operations are processed in the same way. In this embodiment, the specific value of the threshold is not limited. For example, the theoretical number of points corresponding to the volume can be estimated based on the volume of the vertical arm and the scanning resolution of the lidar (such as the number of point clouds per square meter, which can be obtained from the lidar parameter table), and the theoretical number of points can be used as the threshold.

[0087] This embodiment quantifies the point cloud density by quantifying the number of data points in the neighborhood, accurately distinguishing between real end structures and distorted noise points. Then, it effectively filters out discrete noise generated in environments such as high humidity by using a threshold, retaining real scan points with higher density due to the continuous end structure. This provides a high-quality, low-noise point cloud data source for the subsequent generation of the transformation matrix, avoiding interference from distorted points with the accuracy of the transformation matrix solution.

[0088] Furthermore, embodiments of this application provide a step for obtaining the reference point cloud coordinates corresponding to the current pose based on a filtered point set, including:

[0089] Based on the set of filtered points, the density weighting center corresponding to each radar is obtained;

[0090] The set of associated points is obtained based on the density-weighted center and radius;

[0091] Based on the associated point set, the reference point cloud coordinates corresponding to the current attitude are obtained.

[0092] Specifically, the point set corresponding to each radar can first be obtained based on the filtered point set, for example... For the point set corresponding to the first radar, This is the point set corresponding to the second radar, and so on.

[0093] For each radar's corresponding point set, the density-weighted center for each radar can be obtained, for example, for Its corresponding density-weighted center for:

[0094]

[0095] in, express Any data point in the dataset, Used as an index for traversal. Each data point in the data, For data points The corresponding point cloud density, For data points exist The corresponding coordinates are given in the diagram. Therefore, the density-weighted center is the coordinate obtained by weighting based on density. Similarly, the coordinates can be calculated. The corresponding density-weighted center is .

[0096] Next, the distance between each data point in each point set and the density-weighted centers of other point sets is calculated to obtain the associated point sets. For example, assume... Then the weighted density center is For For each data point in the dataset, calculate its relationship with... The distance is calculated and compared with the radius. Remove data points whose distance is less than or equal to the radius to obtain the updated data. , recorded as Similarly for For each data point in the dataset, calculate its relationship with... The distance is calculated, and data points with a distance less than or equal to the radius are removed to obtain... Similarly, for For each data point in the dataset, calculate its relationship with... The distance is obtained. ,for For each data point in the dataset, calculate its relationship with... The distance is obtained. ,for For each data point in the dataset, calculate its relationship with... and The distance is obtained. and At this point, the set of associated points is obtained, which is... , , , , and The set of radars can be calculated using the same steps as above for other numbers of radars, which will not be elaborated here.

[0097] Furthermore, embodiments of this application provide a method for obtaining the reference point cloud coordinates corresponding to the current pose based on an associated point set, including:

[0098] Based on the associated point set, the fused coordinates are obtained;

[0099] The reference point cloud coordinates are obtained based on the distance between the data points in the filter point set and the fused coordinates.

[0100] Specifically, based on the set of associated points, the weighted coordinates corresponding to each pair of radars can be calculated. For example, the weighted coordinates corresponding to the first radar and the second radar. for:

[0101]

[0102] in, express Any data point in the dataset, Used as an index for traversal. Each data point in the data, For data points The corresponding point cloud density, For data points exist The corresponding coordinates in the middle, similarly, express Any data point in the dataset, Used as an index for traversal. Each data point in the data, For data points The corresponding point cloud density, For data points exist The corresponding coordinates are then calculated. Similarly, the weighted coordinates corresponding to the first and third radars can be calculated. The weighted coordinates corresponding to the second and third radars. .

[0103] Next, the three weighted coordinates are fused. This embodiment does not limit the fusion method. For example, a weighted average method can be used to take the mean of the three weighted coordinates as the fused coordinates. Alternatively, different weights can be set for each weighted coordinate based on the distance between each radar and the end of the vertical arm. The closer the distance, the higher the weight. Finally, the fused coordinates are obtained by weighting. This embodiment does not limit the specific value of the weights.

[0104] Furthermore, although this embodiment assumes that each radar can scan the end of the vertical arm, it does not specify that each radar can scan the central region of the end of the vertical arm. If each radar can scan the central region of the end of the vertical arm, the fused coordinates are used as the reference point cloud coordinates. If each radar can scan the edge of the end of the vertical arm, the values ​​in the three directions of x, y, and z in the fused coordinates are added to the radius of the end of the vertical arm to obtain the reference point cloud coordinates. The step of determining whether the central region of the end of the vertical arm can be scanned is to calculate the distance between each data point in the data of the end of the vertical arm and the fused coordinates. If the number of data points whose distance is less than the radius of the end of the vertical arm exceeds a threshold, it is considered that each radar can scan the central region of the end of the vertical arm. This embodiment does not limit the threshold; for example, it can be designed to be 90% of the total number of data points. The radius of the end of the vertical arm can be determined according to the radius of the end structure marked on the mechanical drawings.

[0105] This embodiment calculates a density-weighted center to focus the core area positioning on the dense point cloud region at the end. For example, due to the continuous structure at the end of the vertical arm, the point cloud density is high, while the edge noise point density is low. The density-weighted center can weaken edge noise interference and accurately anchor the core position of the end under a single radar view. Then, the density-weighted calculation is used again to calculate the fused coordinates to spatially align and fuse the end point cloud data of the two radars, offsetting the end position deviation caused by the difference in radar installation angles. This makes the fused coordinates closer to the real physical position, solving the problem of inaccurate positioning caused by the difference in multiple radar viewpoints. Furthermore, considering that in actual operation, the radar may only be able to scan the edge of the end due to the limitations of installation position and scanning angle, this embodiment corrects the edge detection results to the actual center position of the end by using the radius of the vertical arm end, making up for the deficiency of the radar's physical scanning range and ensuring that the end coordinates can still accurately reflect the real center under extreme installation or operating postures.

[0106] Specifically, repeat the above steps for each posture to obtain the reference point cloud coordinates corresponding to each posture. Then, arrange the reference point cloud coordinates corresponding to each posture according to the order in which the postures appear during the vertical arm movement. At this time, for any two adjacent reference point cloud coordinates, the rate of change can be calculated as the ratio of the difference between the two adjacent reference point cloud coordinates to the change time of the two postures. The change time of the two postures is the time required to transform from one posture to another. Therefore, based on the reference point cloud coordinates corresponding to each posture, multiple posture change rates can be calculated. Then, it is determined whether the posture change rate conforms to the motion law of the vertical arm end (i.e., whether the preset termination condition is met). For example, if the vertical arm end rotates at a constant speed, the posture change rate should be relatively uniform, that is, each posture change rate should be similar. If a certain posture change rate suddenly exceeds or falls below a preset value, it indicates a sudden change in point cloud quality or a mismatch in preset parameters. For example, sudden rain or direct sunlight hitting the radar can cause a sudden change in point cloud density, or when grabbing materials, the materials may block the end, causing the end point cloud scanned by the radar to be lost, resulting in incorrect positioning of the density concentration area and a sudden change in coordinates. If the radius is too large, too many non-end points will be included, causing the density weighting center to deviate from the true end. If the radius is too small, the effective point cloud will be filtered out, the point cloud density will be insufficient, and the center calculation will be unstable. If the threshold is too high, the effective points at the end edge will be filtered out. If the threshold is too small, a large number of noise points will be retained, and the density weighting center will be disturbed. In this embodiment, the value of the preset value is not limited, and those skilled in the art can determine it according to the actual situation.

[0107] At this point, it is necessary to update the preset parameters and recalculate the reference point cloud coordinates. Specifically, if the attitude change rate suddenly exceeds a preset value, it is usually caused by severe noise interference in the point cloud. In this case, the radius should be reduced, the threshold increased, and noise points filtered out. If the attitude change rate suddenly falls below a preset value, it is usually caused by excessive filtering of the effective point cloud at the end. In this case, the radius should be increased and the threshold decreased. This embodiment does not limit the magnitude of each increase or decrease. For example, the magnitude of each increase or decrease is 5% of the current value.

[0108] To address the issues of point cloud quality fluctuations caused by environmental interference and physical occlusion in complex operational scenarios, as well as insufficient adaptability of initial parameters, this embodiment updates preset parameters when the rate of change of the end coordinates jumps, ensuring the stability and accuracy of the benchmark point cloud coordinate calculation.

[0109] Furthermore, embodiments of this application provide a step for obtaining a transformation matrix based on the current reference point cloud coordinates, including:

[0110] Obtain the end-effector coordinates of the vertical arm in each pose;

[0111] The objective function is obtained based on the current reference point cloud coordinates and the endpoint coordinates;

[0112] The objective function is solved using a nonlinear least squares algorithm to obtain the transformation matrix.

[0113] Specifically, after obtaining the current reference point cloud coordinates, the end coordinates of the vertical arm in each posture of the dock coordinate system are obtained by the sensor installed at the end of the vertical arm. That is, the coordinates of the end of the vertical arm in each posture. The end of the vertical arm is the connection area between the vertical arm and the execution components such as the grab bucket and the lifting device in the mechanical structure. It is the core connection position that undertakes the material grabbing and loading / unloading actions. Since the end of the vertical arm is a large structure, the coordinates of the center point of the structure can be selected as the coordinates of the entire end of the vertical arm.

[0114] Based on the current reference point cloud coordinates and the endpoint coordinates, the objective function is:

[0115]

[0116] in, The total number of poses. For the first The corresponding reference point cloud coordinates For the first Given the corresponding end coordinates, the objective function aims to obtain the transformation matrix. This minimizes the transformation error for all poses. The solution method is a nonlinear least squares algorithm, a commonly used mathematical method, which will not be elaborated upon in this embodiment.

[0117] The objective function provided in this embodiment is guided by minimizing the sum of the squares of the distances between the dock coordinate system coordinates and the reference point cloud coordinates under all attitudes. It systematically quantifies the transformation error, so that the solution focuses on the global optimum, thereby obtaining a transformation matrix that can accurately describe the transformation from the reference point cloud coordinates to the dock coordinate system, ensuring the accuracy of subsequent unloading operations.

[0118] Furthermore, such as Figure 4 As shown in the figure, this application provides a lidar calibration system for an unmanned screw unloader, comprising:

[0119] The calculation module is used to perform the first iteration operation, which includes obtaining the reference point cloud coordinates of the vertical arm at each attitude through multiple radars and preset parameters, obtaining multiple attitude change rates based on the reference point cloud coordinates, determining whether the multiple attitude change rates have reached the preset termination condition, and if not, updating the preset parameters until the multiple attitude change rates have reached the preset termination condition, and outputting the current reference point cloud coordinates; the multiple radars are located on the unmanned spiral unloader.

[0120] The calibration module is used to obtain the transformation matrix based on the current reference point cloud coordinates.

[0121] This application provides a lidar calibration method and system for an unmanned spiral unloader. Based on the principle that the point cloud density near the end of the vertical arm is higher than that in the distortion region, the method utilizes the characteristics of point cloud density to accurately extract the reference point cloud coordinates, thus solving the problem of point cloud distortion caused by high humidity at sea. Then, by using the end coordinates of the vertical arm in the dock coordinate system and the reference point cloud coordinates, the transformation relationship between the lidar and the dock coordinate system is obtained, thus overcoming the shortcomings of the prior art.

[0122] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0124] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0125] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A lidar calibration method for an unmanned screw unloader, characterized in that, include: The first iteration operation includes obtaining the reference point cloud coordinates of the vertical arm in each posture through multiple radars and preset parameters, obtaining multiple posture change rates based on the reference point cloud coordinates, determining whether the multiple posture change rates have reached a preset termination condition, and if not, updating the preset parameters until the multiple posture change rates have reached the preset termination condition, and outputting the current reference point cloud coordinates; the multiple radars are located on the unmanned spiral unloader. The transformation matrix between the radar coordinate system and the dock coordinate system is obtained based on the current reference point cloud coordinates. The step of obtaining the reference point cloud coordinates of the vertical arm in each posture through multiple radars and preset parameters includes: The second iteration operation includes obtaining initial point cloud data corresponding to the current attitude through multiple radars, obtaining a first transformation matrix based on the initial point cloud data, updating the first transformation matrix to obtain a second transformation matrix, obtaining reference point cloud data based on the second transformation matrix, obtaining reference point cloud coordinates corresponding to the current attitude based on the reference point cloud data and the preset parameters, updating the current attitude, until the reference point cloud coordinates corresponding to each attitude are obtained. The step of obtaining the first transformation matrix based on the initial point cloud data includes: Select one point cloud data from the initial point cloud data as the reference point cloud data; For each point cloud data in the remaining point cloud data, determine its overlapping area with the reference point cloud data. The remaining point cloud data is the data after removing the reference point cloud data from the initial point cloud data. The first transformation matrix is ​​obtained based on the corresponding point pairs in the overlapping region; the first transformation matrix is ​​a set of transformation matrices between each point cloud data in the remaining point cloud data and the reference point cloud data; The step of updating the first transformation matrix to obtain the second transformation matrix includes: The third iteration operation includes obtaining temporary point cloud data based on the current first change matrix and the remaining point cloud data, calculating the distance between the temporary point cloud data and the data points in the reference point cloud data, determining whether the distance reaches a preset second termination condition, and if not, updating the current first change matrix until the preset second termination condition is reached, and outputting the second change matrix.

2. The lidar calibration method for an unmanned screw unloader according to claim 1, characterized in that, The step of obtaining the reference point cloud coordinates corresponding to the current pose based on the reference point cloud data and the preset parameters includes: Based on the preset spatial region and the reference point cloud data, the data at the end of the vertical arm is obtained; The data at the end of the vertical arm is preprocessed according to the preset parameters to obtain a set of filtered points; The reference point cloud coordinates corresponding to the current attitude are obtained based on the filtered point set.

3. The lidar calibration method for an unmanned screw unloader according to claim 2, characterized in that, The preset parameters include radius and threshold. The data at the end of the vertical arm is preprocessed according to these preset parameters to obtain a set of filtered points, including: For each data point in the data at the end of the vertical arm, the density corresponding to each data point is obtained based on the radius; The filter point set is obtained based on the data at the end of the vertical arm, the density, and the threshold.

4. A lidar calibration method for an unmanned screw unloader according to claim 3, characterized in that, The step of obtaining the reference point cloud coordinates corresponding to the current pose based on the filtered point set includes: Based on the set of filtered points, the density weighting center corresponding to each radar is obtained; Based on the density-weighted center and the radius, the set of associated points is obtained; Based on the associated point set, the reference point cloud coordinates corresponding to the current attitude are obtained.

5. A lidar calibration method for an unmanned screw unloader according to claim 4, characterized in that, The step of obtaining the reference point cloud coordinates corresponding to the current pose based on the associated point set includes: Based on the aforementioned set of associated points, the fused coordinates are obtained; The reference point cloud coordinates are obtained based on the distance between the data points in the filter point set and the fused coordinates.

6. A lidar calibration method for an unmanned screw unloader according to claim 1, characterized in that, The transformation matrix is ​​obtained based on the current reference point cloud coordinates, including: Obtain the end-effector coordinates of the vertical arm in each pose; The objective function is obtained based on the current reference point cloud coordinates and the end coordinates; The objective function is solved using a nonlinear least squares algorithm to obtain the transformation matrix.

7. A lidar calibration system for an unmanned screw unloader, used to implement the lidar calibration method for an unmanned screw unloader as described in any one of claims 1-6, characterized in that, include: The calculation module is used to perform a first iteration operation, which includes obtaining the reference point cloud coordinates of the vertical arm at each attitude through multiple radars and preset parameters, obtaining multiple attitude change rates based on the reference point cloud coordinates, determining whether the multiple attitude change rates have reached a preset termination condition, and if not, updating the preset parameters until the multiple attitude change rates have reached the preset termination condition, and outputting the current reference point cloud coordinates; the multiple radars are located on the unmanned spiral unloader. The calibration module is used to obtain the transformation matrix based on the current reference point cloud coordinates.

Images