Laser radar calibration method and system for unmanned screw ship unloader

Abstract

The invention discloses a laser radar calibration method and system for an unmanned screw ship unloader, and belongs to the technical field of point cloud calibration, and the method is characterized in that the method comprises the steps: obtaining a reference point cloud coordinate corresponding to each attitude of a vertical arm through a plurality of radars and preset parameters, obtaining a plurality of attitude change rates according to the reference point cloud coordinates, and calculating the attitude change rates according to the attitude change rates; judging whether the plurality of attitude change rates reach a preset termination condition, if not, updating preset parameters until the plurality of attitude change rates reach the preset termination condition, and outputting a current reference point cloud coordinate; according to the method, calibration between the laser radar coordinate system and the wharf coordinate system is achieved by means of the coordinates of the vertical arm of the screw ship unloader, and the problem of point cloud distortion caused by offshore high humidity is solved based on the distribution characteristic of the point cloud density of the tail end of the vertical arm.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of point cloud calibration, in particular to a laser radar calibration method and system for unmanned spiral ship unloaders. BACKGROUND

[0002] Unmanned spiral ship unloaders rely on laser point cloud data of ship cabins for automatic operation, and a single laser radar often cannot obtain complete ship cabin data, and the application of multiple laser radar data requires the conversion relationship between the coordinate systems of each laser radar and the conversion relationship between the coordinate systems of the laser radar and the wharf.

[0003] Among them, the coordinate system conversion relationship between multiple laser radars can be calibrated through the mutual visible area, but the existing technology has not yet realized the conversion relationship between the laser radar and the wharf coordinate system, so the existing technology has defects. SUMMARY

[0004] In view of the defects in the prior art, the purpose of the present application is to provide a laser radar calibration method and system for unmanned spiral ship unloaders, which realizes the calibration between the laser radar coordinate system and the wharf coordinate system with the aid of the coordinates of the vertical arm of the spiral ship unloader.

[0005] To achieve the above purpose, the present application provides the following technical solutions:

[0006] The present application provides a laser radar calibration method for unmanned spiral ship unloaders, comprising:

[0007] A first iteration operation, the first iteration operation comprises obtaining the reference point cloud coordinates of the vertical arm corresponding to each posture by multiple radars and preset parameters, obtaining multiple posture change rates according to the reference point cloud coordinates, judging whether the multiple posture change rates reach a preset termination condition, if not, updating the preset parameters until the multiple posture change rates reach the preset termination condition, and outputting the current reference point cloud coordinates; the multiple radars are located on the unmanned spiral ship unloader;

[0008] According to the current reference point cloud coordinates, a transformation matrix is obtained.

[0009] As a further improvement of the present application, the reference point cloud coordinates of the vertical arm corresponding to each posture obtained by the multiple radars and the preset parameters comprise:

[0010] The second iteration operation comprises: obtaining initial point cloud data corresponding to the current pose by a plurality of radars; obtaining a first change matrix according to the initial point cloud data; updating the first change matrix to obtain a second change matrix; obtaining reference point cloud data according to the second change matrix; obtaining reference point cloud coordinates corresponding to the current pose according to the reference point cloud data and the preset parameter; and updating the current pose until the reference point cloud coordinates corresponding to each pose are obtained.

[0011] As a further improvement of the application, the first change matrix is obtained according to the initial point cloud data, comprising:

[0012] Selecting one point cloud data from the initial point cloud data as reference point cloud data;

[0013] For each point cloud data in the remaining point cloud data, determining the overlapping area with the reference point cloud data, wherein the remaining point cloud data is the data obtained by removing the reference point cloud data from the initial point cloud data;

[0014] According to the corresponding point pairs in the overlapping area, the first change matrix is obtained.

[0015] As a further improvement of the application, the first change matrix is updated to obtain a second change matrix, comprising:

[0016] The third iteration operation comprises: obtaining temporary point cloud data according to the current first change matrix and the remaining point cloud data; calculating the distance between the temporary point cloud data and the data points in the reference point cloud data; judging whether the distance reaches a preset second termination condition; if not, updating the current first change matrix until the preset second termination condition is reached; and outputting the second change matrix.

[0017] As a further improvement of the application, the reference point cloud coordinates corresponding to the current pose are obtained according to the reference point cloud data and the preset parameter, comprising:

[0018] According to the preset spatial region and the reference point cloud data, the vertical arm end data is obtained;

[0019] The vertical arm end data is preprocessed according to the preset parameter to obtain a filtered point set;

[0020] The reference point cloud coordinates corresponding to the current pose are obtained according to the filtered point set.

[0021] As a further improvement of the application, the preset parameter comprises a radius and a threshold value, and the vertical arm end data is preprocessed according to the preset parameter to obtain a filtered point set, comprising:

[0022] For each data point in the vertical arm end data, a density corresponding to each data point is obtained according to the radius;

[0023] According to the vertical arm end data, the density and the threshold, a filtered point set is obtained.

[0024] As a further improvement of the application, the reference point cloud coordinates corresponding to the current pose are obtained according to the filtered point set, comprising:

[0025] According to the filtered point set, a density weighted center corresponding to each radar is obtained;

[0026] According to the density weighted center and the radius, an associated point set is obtained;

[0027] According to the associated point set, the reference point cloud coordinates corresponding to the current pose are obtained.

[0028] As a further improvement of the application, the reference point cloud coordinates corresponding to the current pose are obtained according to the associated point set, comprising:

[0029] According to the associated point set, a fusion coordinate is obtained;

[0030] According to the distance between the data points in the filtered point set and the fusion coordinate, the reference point cloud coordinates are obtained.

[0031] As a further improvement of the application, a transformation matrix is obtained according to the current reference point cloud coordinates, comprising:

[0032] Obtain the end coordinates of the vertical wall corresponding to each pose;

[0033] According to the current reference point cloud coordinates and the end coordinates, a target function is obtained;

[0034] According to the nonlinear least squares algorithm, the target function is solved to obtain the transformation matrix.

[0035] The application provides a laser radar calibration system for unmanned spiral ship unloaders, comprising:

[0036] A calculation module is configured to perform a first iteration operation, which includes obtaining reference point cloud coordinates of a vertical arm corresponding to each pose by a plurality of radars and preset parameters, obtaining a plurality of pose change rates according to the reference point cloud coordinates, determining whether the plurality of pose change rates meet a preset termination condition, updating the preset parameters if the plurality of pose change rates do not meet the preset termination condition, outputting a current reference point cloud coordinate, and the plurality of radars being located on an unmanned spiral ship unloader.

[0037] A calibration module is configured to obtain a transformation matrix according to the current reference point cloud coordinate.

[0038] The application is based on the principle that the point cloud density near the end position of the vertical arm is higher than that of the distortion area, and the reference point cloud coordinates are accurately extracted by using the characteristics of the point cloud density, thereby solving the problem of point cloud distortion caused by high humidity at sea, and then the conversion relationship between the laser radar and the wharf coordinate system is obtained by the end coordinates of the vertical arm in the wharf coordinate system and the reference point cloud coordinates, thereby solving the defects of the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 The figure is a schematic diagram of the method steps of the application;

[0040] Figure 2 The figure is a schematic diagram of the steps of the second iteration operation;

[0041] Figure 3 The figure is a schematic diagram of the steps of the third iteration operation;

[0042] Figure 4 The figure is a schematic diagram of the system structure of the application. DETAILED DESCRIPTION

[0043] The technical solutions of the application will be described in detail below with reference to the drawings and specific embodiments, and it should be understood that the embodiments and specific features in the embodiments are detailed descriptions of the technical solutions of the application, rather than limitations of the technical solutions of the application.

[0044] In the following text, the term "and / or" is only used to describe the association relationship between the associated objects, and it means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the character " / " generally represents that the associated objects before and after it are in an "or" relationship.

[0045] As shown in Figure 1 The embodiment of the application provides a laser radar calibration method for an unmanned spiral ship unloader, which comprises the following steps:

[0046] The first iteration operation comprises the following steps: obtaining the reference point cloud coordinates of the vertical arm corresponding to each posture by a plurality of radars and preset parameters, obtaining a plurality of posture change rates according to the reference point cloud coordinates, judging whether the plurality of posture change rates reach a preset termination condition, if not, updating the preset parameters until the plurality of posture change rates reach the preset termination condition, and outputting the current reference point cloud coordinates; the plurality of radars are located on the unmanned spiral ship unloader;

[0047] The transformation matrix is obtained according to the current reference point cloud coordinates.

[0048] Specifically, when the unmanned spiral ship unloader is working, the position of the cargo needs to be determined by the radar installed on the ship unloader, and the cargo needs to be unloaded. When there is a large amount of cargo, one radar cannot scan all the cargo, so multiple laser radars need to be configured. The present embodiment does not limit the specific number, as long as all the cargo can be covered. Moreover, the position of the cargo scanned by the laser radar is determined in the coordinate system of the laser radar itself, not the position of the cargo in the coordinate system of the wharf. The unloading is determined according to the position of the cargo relative to the coordinate system of the wharf. Therefore, only the coordinates identified by the laser radar cannot complete the unloading operation. In order to solve the above problem, the present embodiment uses the reference point cloud coordinates of the vertical arm, which are the coordinates of the vertical arm in the radar coordinate system, and obtains the end coordinates of the vertical arm in the wharf coordinate system through the sensor installed on the vertical arm. The conversion relationship between the laser radar and the wharf coordinate system is obtained through the conversion relationship between the end coordinates and the reference point cloud coordinates. The accurate position of the cargo can be obtained according to the coordinates identified by the radar, and the unloading can be completed.

[0049] The radar coordinate system is a local reference system of a single laser radar, which is used to describe the spatial position of the point cloud collected by the radar in its own field of view. The origin is usually the laser emission center of the laser radar, and the X-axis is the horizontal forward direction of the radar, such as the main scanning direction of the laser beam. The wharf coordinate system is a global reference system covering the entire unloading area, which is used to uniformly describe the positions of the cargo and the vertical arm. The origin and direction are determined by engineering calibration, and the present embodiment does not limit this. For example, the center of the ship unloader gantry is selected as the origin. Moreover, the present embodiment does not limit the selection of the attitude, but the multiple selected attitudes should have continuity. For example, the present embodiment controls the vertical arm to remain vertical and only changes the yaw angle of the vertical arm, and arranges six attitudes, respectively, with yaw angles of ±10°, ±20°, and ±30°.

[0050] The present embodiment uses the characteristics of point cloud density to achieve accurate extraction of reference point cloud coordinates based on the principle that the point cloud density near the end position of the vertical arm is higher than that of the distortion area, solving the problem of point cloud distortion caused by high humidity at sea. Then, the conversion relationship between the laser radar and the wharf coordinate system is obtained through the end coordinates of the vertical arm in the wharf coordinate system and the reference point cloud coordinates. Moreover, the present embodiment selects multiple attitudes to cover a series of motion ranges of the vertical arm during operation, and then uses the law of rigid motion of the vertical arm to constrain the change rate of the end coordinates in the adjacent attitudes, filter the abnormal coordinates caused by the difference in radar viewing angle, point cloud distortion, etc., and continuously optimize the preset parameters based on the iterative operation to ensure the accuracy of the reference point cloud coordinates.

[0051] Further, as shown in Figure 2 The present embodiment provides a step of obtaining the reference point cloud coordinates of the vertical arm corresponding to each attitude through multiple radars and preset parameters, which includes:

[0052] The second iteration operation includes: obtaining initial point cloud data corresponding to the current pose by the plurality of radars, obtaining a first change matrix according to the initial point cloud data, updating the first change matrix to obtain a second change matrix, obtaining reference point cloud data according to the second change matrix, obtaining reference point cloud coordinates corresponding to the current pose according to the reference point cloud data and a preset parameter, and updating the current pose until reference point cloud coordinates corresponding to each pose are obtained.

[0053] For each pose, initial point cloud data needs to be obtained by the plurality of radars, and therefore the initial point cloud data is a set including point cloud data collected by each radar. The iteration steps described above are repeated for each pose, and reference point cloud coordinates corresponding to each pose are obtained.

[0054] The embodiment obtains reference point cloud coordinates corresponding to each pose by the second iteration operation, forms complete end motion trajectory data, and uses the end motion trajectory data as a basis for subsequent calculation of a change rate, thereby continuously correcting conversion errors of the multi-radar point cloud to the reference coordinate system, effectively filtering distortion interference, and enhancing data consistency and robustness.

[0055] Further, the embodiment of the present application provides a step of obtaining a first change matrix according to initial point cloud data, including:

[0056] selecting one point cloud data from the initial point cloud data as reference point cloud data;

[0057] For each point cloud data in the remaining point cloud data, determining an overlapping region thereof with the reference point cloud data, the remaining point cloud data being data obtained by removing the reference point cloud data from the initial point cloud data;

[0058] obtaining the first change matrix according to corresponding points in the overlapping region.

[0059] Specifically, for any pose, the initial point cloud data corresponding to the pose is denoted as wherein is the number of radars, is point cloud data obtained by the i-th radar. Then, one point cloud data is selected from the initial point cloud data as reference point cloud data, for example, is selected as the reference point cloud data. Then, for each point cloud data in the remaining point cloud data is selected as the reference point cloud data. Then, for each point cloud data in the remaining point cloud data is selected as the reference point cloud data. Then, for each point cloud data in the remaining point cloud data , determine the overlapping area with the reference point cloud data, the overlapping area refers to the area that can be scanned by both radars, for example, both radars can scan the same wall of the cabin, then the overlapping area is the wall, the overlapping area can be obtained according to the installation pose, field of view angle and scanning range of the radar, for each radar, the corresponding scanning range should be a cone with the radar origin as the apex, therefore the overlapping area of the two radars should be the intersection of the two cones. After determining the overlapping area, a plurality of point pairs are selected in the overlapping area by manual selection, for example and , the overlapping area is a wall in the cabin, first select a plurality of points in the wall in , for example, select four corners, then obtain the coordinates corresponding to the four corners, then determine the coordinates corresponding to the four corners in , a total of four point pairs are obtained, the two coordinates in each point pair are the same corner, for example, one of the point pairs is , is the coordinate corresponding to one of the corners in , is the coordinate corresponding to the corner in , since the data coordinates obtained by each radar are based on the coordinate system corresponding to itself, therefore even if the same four corners, the coordinates in and are different.

[0060] The reason for the different coordinates found in this embodiment is that the radar installation angles are different, for example, the first radar is installed horizontally and the second radar is installed at an angle of 30°, resulting in a rotational deviation in the coordinates, and the radar installation positions are different, for example, the first radar is installed on the left side of the gantry and the second radar is installed on the right side of the gantry, resulting in a translational deviation, if the rotation and translation are processed separately, the calculation complexity will increase and errors are easy to occur, therefore this embodiment describes both operations through a homogeneous transformation matrix, the form of the homogeneous transformation matrix is: , wherein is a 3x3 rotation matrix for describing the rotation relationship, is a 3x1 translation vector for describing the translation relationship, wherein, since rotation is a linear transformation, it can be realized through matrix multiplication, therefore is designed in the matrix, while translation is a nonlinear transformation and cannot be realized through matrix multiplication, therefore is added in the matrix, and in order to describe the linear (rotation) and nonlinear (translation) operations simultaneously through uniform matrix multiplication, homogeneous coordinates need to be introduced, that is, the three-dimensional coordinates selected above are added with 1 and expanded to four-dimensional coordinates, for example, the coordinates of the above are , after being expanded to four-dimensional coordinates, they are The extension step is performed on the coordinates of each selected point pair, to obtain the four-dimensional coordinates corresponding to each coordinate pair.

[0061] According to the homogeneous transformation matrix, the transformation formula of each point pair can be obtained, for example, for The corresponding transformation formula is:

[0062]

[0063] wherein, is The corresponding four-dimensional coordinates, after obtaining the transformation formula corresponding to each point pair, each formula is combined, and an equation set can be obtained, and by solving the equation set, the and The numerical values of and are substituted into the above homogeneous transformation matrix, and the and corresponding change matrix is obtained, and then the above steps are repeated for each point cloud data in the remaining point cloud data, and the change matrix between each point cloud data in the remaining point cloud data and the reference point cloud data is obtained. The set of all change matrices is denoted as the first change matrix wherein is The change matrix corresponding to is The change matrix corresponding to The equation set can be solved by using the least square method or singular value decomposition method, which is a mathematical method that can be realized by those skilled in the art. The present embodiment does not repeat it here, and the reference point cloud data, the overlapping area and the point pair in the present embodiment are only examples, and the present embodiment does not limit them. Those skilled in the art can select according to the actual situation.

[0064] The present embodiment introduces the homogeneous transformation matrix, and integrates the nonlinear translation operation into the linear transformation framework by means of the homogeneous coordinates, realizes the completion of rotation and translation by single matrix multiplication, greatly simplifies the coordinate transformation process, and finally obtains the transformation matrix between the radar coordinate systems, as the basis for subsequent calibration between the radar coordinate system and the wharf coordinate system.

[0065] Further, as shown in Figure 3 The present embodiment provides a step of updating the first change matrix to obtain the second change matrix, comprising:

[0066] ​The third iteration operation includes obtaining temporary point cloud data based on the current first change matrix and the remaining point cloud data, calculating the distance between the temporary point cloud data and the data points in the reference point cloud data, determining whether the distance has reached the preset second termination condition, and if not, updating the current first change matrix until the preset second termination condition is reached, and outputting the second change matrix.

[0067] This embodiment takes into account that since the point pairs in the first transformation matrix are manually selected, there may be errors. Therefore, in order to further reduce the errors caused by manual selection, this embodiment updates the first transformation matrix through a third iteration operation (i.e., the ICP algorithm).

[0068] For example, in the first iteration of the third iteration operation, for each point cloud data in the remaining point cloud data... According to its relationship with The corresponding transformation matrix is ​​transformed to Temporary point cloud data is obtained in the corresponding radar coordinate system. ,in for Transform to Point cloud data in the corresponding radar coordinate system, for Transform to The corresponding point cloud data in the radar coordinate system. Then, for each point cloud data point in the temporary point cloud data, the distance between it and the data points in the reference point cloud data is calculated. and For example, for Each data point in Find the data point that is closest to the given data point, and use that distance as... The distance corresponding to the data point can then be obtained. The sum of the distances corresponding to each data point in the dataset, and then the values ​​are calculated separately. and The corresponding centroid (mean), and respectively based on the centroid pair and Perform centroid removal, and then... and Calculate the covariance matrix and perform singular value decomposition on it to obtain the updated rotation matrix. and according to The updated translation vector is obtained from the relationship with the centroid. And thus obtain and The corresponding updated change matrix. Repeat the above steps for each point cloud data in the temporary point cloud data, and obtain the sum of the distances corresponding to each data point in each point cloud data in the temporary point cloud data, and then obtain the updated change matrix between each point cloud data and the reference point cloud data. The set of each updated change matrix is used as the updated first change matrix.

[0069] Then, a second iteration is performed, and the updated first change matrix obtained in the first iteration is used as the current first change matrix. Then, the current temporary point cloud data is obtained according to the current first change matrix and the remaining point cloud data, and is denoted as Then, the step of calculating the sum of the distances corresponding to each data point in each point cloud data in the current temporary point cloud data is repeated, and the difference between the two calculated distance sums is compared, and it is determined whether the second preset termination condition is reached. The second preset termination condition is that the difference is less than a preset threshold. If it is less than the preset threshold, the current first change matrix, that is, the updated first change matrix obtained in the first iteration, is output as the second change matrix. If it is greater than or equal to, the step of updating the current first change matrix is repeated until the second preset termination condition is reached, and the second change matrix is output.

[0070] The first change matrix is updated by distance, which can gradually correct the deviation caused by the manually selected points. A more accurate change matrix is finally obtained, high-precision registration of multi-radar point clouds under a unified reference is achieved, and a basis for obtaining a conversion matrix between a radar coordinate system and a wharf coordinate system is provided.

[0071] Further, the embodiment of the application provides a method for obtaining a reference point cloud coordinate corresponding to a current pose according to reference point cloud data and preset parameters, comprising:

[0072] Obtaining the vertical arm end data according to the preset space region and the reference point cloud data;

[0073] Preprocessing the vertical arm end data according to the preset parameters to obtain a filtered point set;

[0074] Obtaining the reference point cloud coordinate corresponding to the current pose according to the filtered point set.

[0075] Wherein, the embodiment finds that the working principle of the laser radar is to calculate the distance by emitting a laser beam and measuring the time and angle of the reflected light beam back to the sensor after encountering an object, and then obtaining each data point (i.e. coordinates) in the point cloud data. In a high humidity environment, there are a large number of tiny water droplets (aerosols) in the air, and the diameter of these water droplets is usually between a few microns and tens of microns, which is at the same order of magnitude as the wavelength of the laser. When the laser beam passes through such air, Mie scattering occurs between the laser beam and the water droplets, resulting in part of the laser energy not propagating in a straight line to the target object, but being scattered in all directions by the water droplets. The scattered light in the non-target direction may be misjudged by the radar receiver as a target reflection signal, thereby forming discrete points unrelated to the real object in the point cloud data, i.e. noise points. The higher the humidity, the greater the concentration of water droplets in the air, and the more severe the scattering phenomenon, the more noise points there are. Therefore, the high humidity environment on the sea is prone to cause distortion in the point cloud data collected by the laser radar, i.e. noise, and the area where the noise points are located is the distortion area.

[0076] The embodiment finds that the end of the vertical arm is a rigid structure, and the point cloud density near its true position must be higher than that of the distortion area. The specific reason is that when the laser radar scans, the laser beam will continuously and stably irradiate the surface, thereby forming a large number of continuous reflection points near the true position. These reflection points come from the same continuous surface of the rigid structure, and the spatial distance between them is relatively close, so the number of points in a unit space (i.e. point cloud density) is relatively high. The data points in the distortion area come from the points formed by the scattered light received by the radar, which are scattered in space and have no fixed relationship with each other. The number of points in a unit space is small, and the point cloud density is low. From the perspective of scanning logic, when the laser radar scans the rigid end of the vertical arm, the end is an entity structure that blocks the propagation of the laser, making the reflection of the laser in this area have certainty and continuity, thereby forming a dense point cloud. The interference area does not have such a rigid entity to continuously reflect the laser, making it difficult to form a dense point cloud, so the point cloud density must be lower than that near the end of the vertical arm.

[0077] In the structure of the ship unloader, not only the end of the vertical arm is a rigid structure, but also other components such as the body and the large arm have rigid characteristics. However, it is found in the embodiment that the end of the vertical arm is the operation execution end of the ship unloader, which needs to frequently assume different postures during operation, while the body and the large arm are relatively fixed, and the scanning angle is limited, which makes it difficult to provide the coordinate correspondence under multiple postures, and further makes it difficult to implement the step of obtaining the transformation matrix through the posture change rate in the embodiment. In addition, the structure of the end of the vertical arm is relatively regular (such as cylindrical or square), which is convenient for extracting coordinates by using the density concentration feature. However, other rigid structures have a large volume and a complex shape, and the point cloud is easily shielded or interfered, which increases the difficulty of coordinate extraction. In summary, the embodiment distinguishes the real position of the end of the vertical arm from the distortion area through the point cloud density, and further realizes the accurate extraction of the coordinates of the end of the vertical arm.

[0078] Specifically, first, the remaining point cloud data is converted into data in the radar coordinate system corresponding to the reference point cloud data according to the second transformation matrix and the remaining point cloud data, to obtain the reference point cloud data. At this time, the reference point cloud data includes the reference point cloud data and the remaining point cloud data converted through the second transformation matrix. In addition, the embodiment considers that since the scanning range of the laser radar is a relatively large range, it is easy to cause the collected point cloud data to include data corresponding to other rigid structures, which affects the accuracy of the reference point cloud coordinates. Therefore, the embodiment filters the reference point cloud data according to a preset spatial region, so that the reference point cloud data only includes data points near the end of the vertical arm. Specifically, each radar in the embodiment can scan the end of the vertical arm. Then, the kinematic model of the ship unloader (the joint position of the vertical arm, the maximum swing angle, and the telescopic range) is used to obtain a preset spatial region corresponding to the end of the vertical arm. Then, only the data in the reference point cloud data within the spatial region is reserved as the end of the vertical arm data. Then, the reference point cloud coordinates are obtained based on the end of the vertical arm data. This step is a technical means that can be implemented by those skilled in the art, and will not be described here.

[0079] The preset spatial region can accurately frame the range of the end of the vertical arm, effectively exclude the interference of other high-density structures in the scene, ensure that the subsequent processing of the reference point cloud data focuses on the end of the vertical arm, and then preprocess the end of the vertical arm data according to the preset parameters to obtain a filtered point set, which can further remove noise points and improve data purity. Finally, the reference point cloud coordinates corresponding to the current posture are obtained based on the filtered point set, which significantly improves the accuracy of the reference point cloud coordinates, and lays a high-quality data foundation for subsequent coordinate fusion and calculation of the transformation matrix.

[0080] Further, the embodiment of the present application provides a method for preprocessing the end of the vertical arm data according to preset parameters to obtain a filtered point set, comprising:

[0081] For each data point in the vertical arm end data, a density corresponding to each data point is obtained according to the radius;

[0082] According to the vertical arm end data, the density, and the threshold value, a filtered point set is obtained.

[0083] The preset parameters include the radius and the threshold value.

[0084] Specifically, since the reference point cloud data includes the reference point cloud data and the remaining point cloud data converted by the second transformation matrix, the vertical arm end data selected from the reference point cloud data and the preset spatial region should be a data set corresponding to multiple radars. Exemplarily, the remaining point cloud data converted by the second transformation matrix is denoted as , wherein is converted data, is converted data, and then according to the preset spatial region, the data is screened to obtain the vertical arm end data including , wherein is screened data, is screened data, is screened data.

[0085] Then, for each data point in , the number of data points contained in the radius neighborhood centered at each data point is calculated by the KD tree, and the contained data points should be in , and then the number is denoted as the point cloud density corresponding to the data point. Then, for , the above steps are repeated, and the density corresponding to each data point in the vertical arm end data is obtained. The numerical value of the radius in this embodiment is not limited, and exemplarily, the radius can be set to 1.5 times of the radius of the vertical arm end to ensure covering the vertical arm end structure.

[0086] Then, the point cloud density corresponding to each data point in is compared with the threshold value, if less than the threshold value, the data point is removed, and if greater than or equal to the threshold value, the data point is retained, and is subjected to the removal and retention operations, and a filtered point set is obtained, wherein is The data obtained after the elimination and retention operation, and so on. In this embodiment, the specific value of the threshold is not limited, for example, the volume of the vertical arm and the scanning resolution of the laser radar (such as the number of point clouds per square meter, which can be obtained from the radar parameter table) can be used to estimate the theoretical point number corresponding to the volume, and the theoretical point number is used as the threshold.

[0087] In this embodiment, the point cloud density is quantified by the number of data points in the neighborhood, the real end structure is accurately distinguished from the distortion noise points, the discrete noise generated in the high humidity environment is effectively screened out through the threshold, the real scanning points with higher density due to the continuity of the end structure are retained, and high-quality and low-noise point cloud data sources are provided for subsequent generation of the conversion matrix, so that the distortion points are avoided to interfere with the solving accuracy of the conversion matrix.

[0088] Further, the embodiment of the application provides a step of obtaining the reference point cloud coordinates corresponding to the current pose according to the filtered point set, comprising:

[0089] According to the filtered point set, the density-weighted center corresponding to each radar is obtained;

[0090] According to the density-weighted center and the radius, the associated point set is obtained;

[0091] According to the associated point set, the reference point cloud coordinates corresponding to the current pose are obtained.

[0092] Specifically, first, the point set corresponding to each radar can be obtained according to the filtered point set, for example the point set corresponding to the first radar, the point set corresponding to the second radar, and the like.

[0093] For the point set corresponding to each radar, the density-weighted center corresponding to each radar can be obtained, for example, for the density-weighted center corresponding thereto is:

[0094]

[0095] wherein, represents any one data point in is an index for traversing each data point in is the index of the data point corresponding to the point cloud density of the data point is the coordinate corresponding to the data point in , so the density-weighted center is the coordinate obtained by weighting according to the density, and the density-weighted center corresponding to can be calculated in the same way is .

[0096] Then the distance between each data point in each point set and the density weighted center corresponding to other point sets is calculated to obtain the associated point set. Exemplarily, assuming , the weighted density center is , then for each data point in , the distance between the data point and is calculated, and the calculated distance is compared with the radius, and the data point with a distance less than or equal to the radius is removed from to obtain the updated , denoted as , and for each data point in , the distance between the data point and is calculated, and the data point with a distance less than or equal to the radius is removed to obtain , similarly, for each data point in , the distance between the data point and is calculated to obtain , for each data point in , the distance between the data point and is calculated to obtain , for each data point in , the distance between the data point and and is calculated to obtain and . At this time, the associated point set is obtained, and the associated point set is the set of , , , , and . For other numbers of radars, the calculation can also be based on the above steps, which will not be described herein.

[0097] Further, the embodiment of the application provides a method for obtaining a reference point cloud coordinate corresponding to a current pose according to the associated point set, comprising:

[0098] obtaining a fusion coordinate according to the associated point set;

[0099] obtaining a reference point cloud coordinate according to the distance between the data points in the filtered point set and the fusion coordinate.

[0100] Specifically, first, according to the associated point set, the weighted coordinates corresponding to each two radars can be calculated. Exemplarily, the weighted coordinates corresponding to the first radar and the second radar are :

[0101]

[0102] wherein, represents any one of the data points in is an index for traversing each of the data points in is a data point corresponding point cloud density, is a data point corresponding coordinates in , similarly, represents any one of the data points in is an index for traversing each of the data points in is a data point corresponding point cloud density, is a data point corresponding coordinates in . Similarly, the weighted coordinates corresponding to the first radar and the third radar , and the weighted coordinates corresponding to the second radar and the third radar can be calculated.

[0103] Then, the three weighted coordinates are fused. The embodiment does not limit the fusion manner. For example, the mean value of the three weighted coordinates can be used as the fusion coordinate by using the weighted average manner, or different weights are set for each weighted coordinate according to the distance between each radar and the end of the vertical arm, and the closer the distance, the higher the weight, and finally the fusion coordinate is weighted. The embodiment does not limit the specific value of the weight.

[0104] In addition, although the embodiment assumes that each radar can scan the end of the vertical arm, it does not mention that each radar can scan the central region of the end of the vertical arm. If each radar can scan the central region of the end of the vertical arm, the fusion coordinate is taken as the reference point cloud coordinate. If each radar can scan the edge of the end of the vertical arm, the values in the three directions of the x, y, and z axes in the fusion coordinate are added to the radius of the end of the vertical arm to obtain the reference point cloud coordinate. The step of judging whether the central region of the end of the vertical arm can be scanned is to calculate the distance between each data point in the end of the vertical arm data and the fusion coordinate. If the data points whose distance is less than the radius of the end of the vertical arm exceed a threshold value, it is considered that each radar can scan the central region of the end of the vertical arm. The embodiment does not limit the threshold value. For example, it can be designed as 90% of the total number of data points. The radius of the end of the vertical arm can be determined according to the end structure radius marked on the mechanical drawing.

[0105] The embodiment calculates the density-weighted center, so that the core area positioning focuses on the end dense point cloud area. For example, the vertical arm end has continuous structure, high point cloud density, and low edge noise point density. The density-weighted center can weaken the edge noise interference, accurately anchor the end core position under the single radar view, and then perform spatial alignment and fusion on the end point cloud data of the two radars through density-weighted calculation and fusion coordinates, offset the end position deviation caused by the installation angle difference of the radars, make the fused coordinates closer to the real physical position, solve the inaccurate positioning problem caused by the view difference of multiple radars, and consider that in actual operation, the radars may be limited by the installation position and scanning angle and can only scan the end edge. The embodiment corrects the edge detection result to the actual center position of the end through the radius of the vertical arm end, makes up for the defect of insufficient physical scanning range of the radars, and ensures that the end coordinates can accurately reflect the real center in the extreme installation or operation posture.

[0106] Specifically, the above steps are repeated for each posture, so that the reference point cloud coordinates corresponding to each posture are obtained. Then, the reference point cloud coordinates corresponding to each posture are arranged in the order of appearance of the postures during the movement of the vertical arm. At this time, for any two adjacent reference point cloud coordinates, the corresponding change rate can be calculated as the difference between the two adjacent reference point cloud coordinates divided by the change time of the two postures, that is, the time required for the transformation from one posture to another. Therefore, according to the reference point cloud coordinates corresponding to each posture, a plurality of posture change rates can be calculated. Then, it is judged whether the posture change rate conforms to the motion law of the vertical arm end (that is, whether a preset termination condition is reached). For example, if the vertical arm end rotates at a constant speed, the posture change rate should be relatively uniform, that is, each posture change rate should be similar. If a certain posture change rate is suddenly greater than or less than a preset value, it indicates that the point cloud quality suddenly changes or the preset parameters do not match, such as sudden rain or direct sunlight on the radar, which can cause sudden changes in point cloud density, or instantaneous material grabbing, which can cause the material to block the end, resulting in the loss of end point cloud scanned by the radar, positioning error of the dense area, sudden change of the coordinates, excessive radius, and inclusion of too many non-end points, resulting in deviation of the density-weighted center from the real end. The threshold is too low, which can filter out effective points at the end edge, and the threshold is too small, which can retain a large number of noise points, and the density-weighted center is disturbed. The embodiment does not limit the value of the preset value, and a person skilled in the art can determine it according to the actual situation.

[0107] At this time, the preset parameters need to be updated and the reference point cloud coordinates are recalculated. Specifically, the situation that the attitude change rate is suddenly greater than a preset value is usually caused by serious point cloud noise interference. At this time, the radius should be reduced and the threshold should be increased to filter the noise points. The situation that the attitude change rate is suddenly less than a preset value is usually caused by excessive filtering of the end effective point cloud. At this time, the radius should be increased and the threshold should be reduced. The embodiment does not limit the amplitude of each increase or decrease. For example, the amplitude of each increase or decrease is 5% of the current value.

[0108] The embodiment is to cope with the point cloud quality fluctuation caused by environmental interference and physical occlusion and the problem of insufficient adaptability of initial parameters in a complex operation scene. When the end coordinate change rate jumps, the preset parameters are updated to ensure the stability and accuracy of the reference point cloud coordinate calculation.

[0109] Further, the embodiment of the application provides a step of obtaining a transformation matrix according to the current reference point cloud coordinates, comprising:

[0110] Obtaining the end coordinates of the vertical arm corresponding to each attitude in the terminal coordinates;

[0111] Obtaining a target function according to the current reference point cloud coordinates and the end coordinates;

[0112] Solving the target function according to a nonlinear least square algorithm to obtain the transformation matrix.

[0113] Specifically, after obtaining the current reference point cloud coordinates, the end coordinates of the vertical arm corresponding to each attitude in the terminal coordinates are obtained through the sensor installed at the end of the vertical arm, that is, the coordinates of the end of the vertical arm corresponding to each attitude. The end of the vertical arm is the connection area of the vertical arm and the execution components such as the grab bucket and the lifting tool on the mechanical structure, and is the core connection position of the material grabbing and handling action. Since the end of the vertical arm is a larger structure, the coordinates of the center point of the structure can be selected as the coordinates corresponding to the entire end of the vertical arm.

[0114] The target function obtained according to the current reference point cloud coordinates and the end coordinates is:

[0115]

[0116] Wherein, is the total number of attitudes, is the th corresponding reference point cloud coordinate, is the th corresponding end coordinate, and the purpose of the target function is to obtain the transformation matrix that minimizes the conversion error of all attitudes. The solving method is a nonlinear least square algorithm, which is a commonly used mathematical method, and the embodiment will not be repeated here.

[0117] The target function provided by the embodiment minimizes the sum of squares of distances between the coordinates of the wharf coordinate system in all postures and the coordinates of the reference point cloud, systematically quantifies the conversion error, focuses on global optimization, and further obtains a conversion matrix that can accurately describe the conversion of the reference point cloud coordinates to the wharf coordinate system, thereby ensuring the accuracy of subsequent unloading operations.

[0118] Further, as shown in Figure 4 The embodiment of the present application provides a laser radar calibration system for an unmanned spiral ship unloader, which comprises:

[0119] The computing module is configured to perform a first iteration operation, and the first iteration operation comprises: obtaining, by the plurality of radars and the preset parameters, the corresponding reference point cloud coordinates of the vertical arm at each posture, obtaining the plurality of posture change rates according to the reference point cloud coordinates, judging whether the plurality of posture change rates reach a preset termination condition, updating the preset parameters if the plurality of posture change rates do not reach the preset termination condition, and outputting the current reference point cloud coordinates until the plurality of posture change rates reach the preset termination condition; and the plurality of radars are located on the unmanned spiral ship unloader.

[0120] The calibration module is configured to obtain the conversion matrix according to the current reference point cloud coordinates.

[0121] The embodiment of the present application provides a laser radar calibration method and system for an unmanned spiral ship unloader, which is based on the principle that the point cloud density near the end position of the vertical arm is higher than that of the distortion region, uses the characteristics of the point cloud density to accurately extract the reference point cloud coordinates, solves the problem of point cloud distortion caused by high humidity at sea, and then obtains the conversion relationship between the laser radar and the wharf coordinate system through the end coordinates of the vertical arm in the wharf coordinate system and the reference point cloud coordinates, thereby solving the problems of the prior art.

[0122] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0123] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other processing device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other processing device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions specified in the flowchart block or blocks. Figure 1 2. The computer program instructions of clause 1, wherein the computer program instructions are further configured to cause the processor to perform the following operations. Figure 1 3. The computer program instructions of clause 2, wherein the computer program instructions are further configured to cause the processor to perform the following operations. 4. The computer program instructions of any one of clauses 1-3, wherein the computer program instructions are further configured to cause the processor to perform the following operations.

[0124] 5. The computer program instructions of any one of clauses 1-4, wherein the computer program instructions are further configured to cause the processor to perform the following operations.The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other processing device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other processing device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions specified in the flowchart block or blocks. Figure 1 2. The computer program instructions of clause 1, wherein the computer program instructions are further configured to cause the processor to perform the following operations. Figure 1 3. The computer program instructions of clause 2, wherein the computer program instructions are further configured to cause the processor to perform the following operations. 4. The computer program instructions of any one of clauses 1-3, wherein the computer program instructions are further configured to cause the processor to perform the following operations.

[0125] 5. The computer program instructions of any one of clauses 1-4, wherein the computer program instructions are further configured to cause the processor to perform the following operations.The above merely provides the preferred embodiment of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiment. Any technical solution falling within the concept of the present application shall be considered as falling within the protection scope of the present application. It should be noted that, for the ordinary skilled in the art, some improvements and refinements can be made without departing from the principle of the present application, and these improvements and refinements shall be considered as falling within the protection scope of the present application.​​​​

Claims

1. A lidar calibration method for an unmanned screw unloader, characterized in that, include: The first iteration operation includes obtaining the reference point cloud coordinates of the vertical arm in each posture through multiple radars and preset parameters, obtaining multiple posture change rates based on the reference point cloud coordinates, determining whether the multiple posture change rates have reached a preset termination condition, and if not, updating the preset parameters until the multiple posture change rates have reached the preset termination condition, and outputting the current reference point cloud coordinates; the multiple radars are located on the unmanned spiral unloader. The transformation matrix is ​​obtained based on the current reference point cloud coordinates.

2. The lidar calibration method for an unmanned screw unloader according to claim 1, characterized in that, The process of obtaining the reference point cloud coordinates of the vertical arm in each posture using multiple radars and preset parameters includes: The second iteration operation includes obtaining initial point cloud data corresponding to the current attitude through multiple radars, obtaining a first transformation matrix based on the initial point cloud data, updating the first transformation matrix to obtain a second transformation matrix, obtaining reference point cloud data based on the second transformation matrix, obtaining reference point cloud coordinates corresponding to the current attitude based on the reference point cloud data and the preset parameters, updating the current attitude, until the reference point cloud coordinates corresponding to each attitude are obtained.

3. The lidar calibration method for an unmanned screw unloader according to claim 2, characterized in that, The step of obtaining the first transformation matrix based on the initial point cloud data includes: Select one point cloud data from the initial point cloud data as the reference point cloud data; For each point cloud data in the remaining point cloud data, determine its overlapping area with the reference point cloud data. The remaining point cloud data is the data after removing the reference point cloud data from the initial point cloud data. The first transformation matrix is ​​obtained based on the corresponding point pairs in the overlapping region.

4. A lidar calibration method for an unmanned screw unloader according to claim 3, characterized in that, The process of updating the first transformation matrix to obtain the second transformation matrix includes: The third iteration operation includes obtaining temporary point cloud data based on the current first change matrix and the remaining point cloud data, calculating the distance between the temporary point cloud data and the data points in the reference point cloud data, determining whether the distance reaches a preset second termination condition, and if not, updating the current first change matrix until the preset second termination condition is reached, and outputting the second change matrix.

5. A lidar calibration method for an unmanned screw unloader according to claim 2, characterized in that, The step of obtaining the reference point cloud coordinates corresponding to the current pose based on the reference point cloud data and the preset parameters includes: Based on the preset spatial region and the reference point cloud data, the data at the end of the vertical arm is obtained; The data at the end of the vertical arm is preprocessed according to the preset parameters to obtain a set of filtered points; The reference point cloud coordinates corresponding to the current attitude are obtained based on the filtered point set.

6. A lidar calibration method for an unmanned screw unloader according to claim 5, characterized in that, The preset parameters include radius and threshold. The data at the end of the vertical arm is preprocessed according to these preset parameters to obtain a set of filtered points, including: For each data point in the data at the end of the vertical arm, the density corresponding to each data point is obtained based on the radius; The filter point set is obtained based on the data at the end of the vertical arm, the density, and the threshold.

7. A lidar calibration method for an unmanned screw unloader according to claim 6, characterized in that, The step of obtaining the reference point cloud coordinates corresponding to the current pose based on the filtered point set includes: Based on the set of filtered points, the density weighting center corresponding to each radar is obtained; Based on the density-weighted center and the radius, the set of associated points is obtained; Based on the associated point set, the reference point cloud coordinates corresponding to the current attitude are obtained.

8. A lidar calibration method for an unmanned screw unloader according to claim 7, characterized in that, The step of obtaining the reference point cloud coordinates corresponding to the current pose based on the associated point set includes: Based on the aforementioned set of associated points, the fused coordinates are obtained; The reference point cloud coordinates are obtained based on the distance between the data points in the filter point set and the fused coordinates.

9. A lidar calibration method for an unmanned screw unloader according to claim 1, characterized in that, The transformation matrix is ​​obtained based on the current reference point cloud coordinates, including: Obtain the end coordinates of the vertical wall in each orientation; The objective function is obtained based on the current reference point cloud coordinates and the end coordinates; The objective function is solved using a nonlinear least squares algorithm to obtain the transformation matrix.

10. A lidar calibration system for an unmanned screw unloader, used to implement the lidar calibration method for an unmanned screw unloader as described in any one of claims 1-9, characterized in that, include: The calculation module is used to perform a first iteration operation, which includes obtaining the reference point cloud coordinates of the vertical arm at each attitude through multiple radars and preset parameters, obtaining multiple attitude change rates based on the reference point cloud coordinates, determining whether the multiple attitude change rates have reached a preset termination condition, and if not, updating the preset parameters until the multiple attitude change rates have reached the preset termination condition, and outputting the current reference point cloud coordinates; the multiple radars are located on the unmanned spiral unloader. The calibration module is used to obtain the transformation matrix based on the current reference point cloud coordinates.

Images