Hatch positioning method based on multi-modal laser radar point cloud

Abstract

This invention relates to the field of point cloud computing technology, specifically a hatch positioning method based on multimodal lidar point clouds. First, a multi-dimensional point cloud containing three-dimensional coordinates and reflection intensity is acquired. The coordinate system of the multi-source data is unified and fused through radar extrinsic parameter calibration or point cloud fusion algorithms. Then, the deck area point cloud is segmented based on normal vector filtering and DBSCAN clustering, and converted into a two-dimensional image through orthogonal projection. Next, edges are extracted using an improved Canny operator, and the hatch planar position is determined by least-squares rectangle fitting. Finally, the three-dimensional coordinates of the hatch are restored through inverse projection mapping. This invention overcomes the detection limitations of a single radar, effectively improving the accuracy and stability of hatch positioning under complex conditions. It can provide reliable positional information for the automated control of bulk carrier unloading operations, and has significant industrial application value.

Description

Technical Field

[0001] This invention relates to the field of point cloud computing technology, specifically a hatch positioning method based on multimodal lidar point clouds. Background Technology

[0002] In the field of bulk cargo unloading operations in ports, automated operations have become the core development direction for improving efficiency and reducing labor costs and labor intensity. The precise positioning of hatches is a key prerequisite for realizing the automated control of unloaders. Only by obtaining reliable hatch spatial coordinates can the core positional basis be provided for the unloader's grab bucket operation path planning and safe obstacle avoidance, which directly affects the efficiency and safety of operations.

[0003] With the development of lidar technology, although single-mode lidar has been applied to some sensing scenarios, it has significant limitations in complex port conditions: First, the detection range of a single radar is limited, and it is easy to form blind spots due to the obstruction of the ship's structure, resulting in the loss of data in local areas of the hatch; Second, environmental interference such as strong light, dust, rain and snow in port operations will cause a surge in noise in the single radar point cloud, a decrease in feature recognition, and thus a reduction in positioning accuracy, making it difficult to meet the stability requirements of automated operations.

[0004] To compensate for the shortcomings of single radar, the industry has gradually tried multi-sensor solutions, but existing technologies still have obvious deficiencies: some multi-radar solutions only achieve "range coverage" through simple data stitching without coordinate system integration and data fusion, resulting in heterogeneity of data from different radars and easy data redundancy or conflict; other solutions rely on point cloud segmentation strategies with fixed parameters (such as preset height thresholds to filter decks), which cannot adapt to the differences in deck height of different ship types, and do not specifically remove interfering point clouds such as cargo stacks in the hold and unloading machine hulls, making subsequent hatch identification susceptible to interference and difficult to guarantee accuracy.

[0005] Meanwhile, existing hatch positioning technologies have a gap in the two-dimensional detection and three-dimensional reconstruction stages: most solutions can only identify the hatch outline in two-dimensional images, without establishing an accurate projection inverse mapping relationship, and cannot reliably reconstruct the three-dimensional spatial coordinates of the hatch; a few solutions with three-dimensional reconstruction capabilities either ignore the tilt characteristics of the deck plane, resulting in projection deviation, or have insufficient flexibility in fitting algorithms (such as fixed rectangular box search), making it difficult to adapt to square hatches of different sizes, thus limiting the applicable scenarios.

[0006] In summary, how to overcome the limitations of single sensors, achieve efficient fusion and accurate calibration of multi-modal radar data, and improve the stability and adaptability of deck segmentation, hatch fitting and three-dimensional coordinate restoration under complex working conditions has become a key technical problem that urgently needs to be solved in the field of automated operation of bulk cargo unloaders in ports. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the purpose of this invention is to provide a hatch positioning method based on multimodal lidar point clouds that can achieve efficient fusion and accurate calibration of multimodal radar data, while improving the stability and adaptability of deck segmentation, hatch fitting, and three-dimensional coordinate reconstruction under complex working conditions.

[0008] The technical solution adopted by this invention to solve its technical problem is:

[0009] A hatch localization method based on multimodal lidar point clouds, comprising the following steps:

[0010] S1, Multimodal point cloud data acquisition: Deploy at least two different types of lidar at preset positions on the main beam of the unloader, adjust the pitch and horizontal angles of each lidar so that the detection range of all lidars is spliced ​​to cover the entire working area, and collect multi-dimensional point cloud data of the target area.

[0011] S2, Multi-source point cloud integration processing: Point cloud data acquired by multiple lidars are unified into the same coordinate system through radar extrinsic parameter calibration or point cloud fusion algorithm to form fused complete point cloud data;

[0012] S3, Deck Area Point Cloud Segmentation: Identify and segment the deck area point cloud corresponding to the bulk carrier deck area from the fused complete point cloud data;

[0013] S4, 3D point cloud planar projection: Project the point cloud of the deck area along a direction perpendicular to the deck plane, compressing the 3D point cloud data of the deck into a 2D planar projection image of the deck;

[0014] S5, Hatch Edge Fitting and Positioning: In the two-dimensional planar projection image of the deck, the planar position of the hatch edge is determined by image edge detection and geometric shape fitting technology;

[0015] S6, Determining the three-dimensional coordinates of the hatch: Inversely map the planar position coordinates of the hatch edge to the three-dimensional point cloud coordinate system to obtain the actual coordinates of the hatch in three-dimensional space.

[0016] As a preferred embodiment, a further technical solution of the present invention is:

[0017] Preferably, in step S1, the different types of lidar are at least two of 2D / 3D fusion lidar, multi-beam lidar, and high-resolution solid-state lidar, and the overlap ratio of the detection range of any two adjacent lidars is not less than 15%.

[0018] Preferably, in step S2, the radar extrinsic parameter calibration is achieved using a calibration plate method, and the calibration process includes:

[0019] The calibration board is placed in the common detection area of ​​multiple lidars. The calibration board has at least 10 discretely distributed feature points, and the coordinates of each feature point in the coordinate system of each lidar are obtained. ,in, The feature point number, =1,2,...,n, where n≥10 is the total number of feature points on the calibration plate;

[0020] Simultaneously, obtain the standard coordinates of the feature points in the world coordinate system. The extrinsic parameter matrix is ​​solved by minimizing the coordinate transformation error. ,in, It is a 3×3 rotation matrix used to describe the attitude relationship between the lidar coordinate system and the world coordinate system; This is a 3×1 rotational translation vector used to describe the positional offset of the lidar coordinate system relative to the world coordinate system; the coordinate transformation formula is... The error function is ,in The L2 norm, i.e. Euclidean distance, is used to minimize E using the Levenberg-Marquardt algorithm to obtain the optimal extrinsic parameter matrix, thereby unifying the coordinate systems of various lidar systems with the world coordinate system.

[0021] Preferably, in step S2, the point cloud fusion algorithm adopts a weighted average fusion strategy, and the fusion formula is as follows: ,in, The coordinates of the merged point cloud. Let be the number of lidars, and be the point cloud coordinates of the same spatial location collected by the k-th lidar. Let the weight of the k-th lidar be , , Let the standard deviation of the measurement error of the k-th lidar at this spatial location be obtained in advance through calibration experiments at different detection distances. And establish an error standard deviation mapping table for use during fusion.

[0022] Preferably, in step S3, the point cloud segmentation of the deck area is achieved through normal vector filtering and DBSCAN clustering, specifically including:

[0023] S301: Calculate each point in the fused point cloud unit normal vector ;right Select q nearest neighbor points to form a point set q is the number of nearest neighbors; calculate the point set. center of mass Construct the covariance matrix ;in, It is a column vector. Transpose it. It is a 3×3 symmetric positive semi-definite matrix; for Perform eigenvalue decomposition to obtain eigenvalues. ≤ ≤ and corresponding feature vectors , , Take the smallest eigenvalue corresponding feature vector After normalization, we get unit normal vector ;

[0024] S302: Normal vector filtering, retain Angle with z-axis Point clouds with an angle of ≤30° The calculation formula is ,in ;

[0025] S303, DBSCAN clustering, sets the neighborhood radius ε and the minimum number of points MinPts, traverses the filtered point cloud, defines the core point as the point whose number of points in the ε neighborhood is greater than or equal to MinPts; defines the boundary point as the point whose number of points in the ε neighborhood is less than MinPts, but falls within the ε neighborhood of any core point; groups each core point and its connected boundary points into the same cluster, removes noise points not associated with any core point, and determines the largest point cloud cluster obtained by clustering as the deck area point cloud.

[0026] Preferably, in step S4, the direction perpendicular to the deck plane is determined by the global normal vector of the point cloud in the deck area. Sure, ;

[0027] in, The total number of points in the point cloud over the deck area. Let be the unit normal vector of the j-th point, summed and then normalized; the projection transformation uses orthogonal projection, and the projection formula is,

[0028] ;

[0029] in, The point cloud coordinates and projection matrix of the three-dimensional deck area. The projection process satisfies That is, to remove The coordinate components of the direction are retained, retaining those perpendicular to the direction. Two-dimensional plane coordinates.

[0030] Preferably, in step S5, the image edge detection uses an improved Canny operator, specifically including:

[0031] S501: Gaussian filter noise reduction, using a 5×5 Gaussian filter core. ,in =1.2 is the Gaussian kernel standard deviation. , The coordinates of the pixels within the filter kernel relative to the center are given, with values ​​ranging from −2, −1, 0, 1, 2. Convolution operations are performed on the projected image to achieve noise suppression.

[0032] S502: Gradient calculation, performed using the Sobel operator. , Directional gradient , The Sobel operator's convolution kernel in the x-direction is The convolution kernel in the y-direction is gradient strength gradient direction , The range of values ​​is [0, π).

[0033] S503: Non-maximum suppression, which discretizes the gradient direction into 8 directions, compares the gradient intensity of the current pixel with that of the two adjacent pixels along the gradient direction, and retains only the local maxima points to achieve edge refinement;

[0034] S504: Dual threshold filtering, setting a high threshold , The maximum gradient intensity and low threshold of all pixels in the image. Retain strong edge points with gradient strength greater than 0, and those between 0 and 0. and Weak edge points that are connected to strong edge points are identified, and isolated weak edge points are removed to obtain a set of candidate hatch edge points.

[0035] Preferably, in step S5, the geometric shape fitting technique is rectangle fitting, implemented based on the least squares principle, specifically including:

[0036] S505: Let the set of candidate points at the hatch edge be... , Let m be the number of candidate points. Assume the hatch projection is a rectangle, with its four sides satisfying the constraints of opposite sides being parallel and adjacent sides being perpendicular. Let the direction of the major axis of the rectangle be perpendicular to the direction of the hatch projection. The included angle of the axis is Then the equations of the lines on the four sides are respectively,

[0037] Above, ;below, ;left, ;right, ;in, Opposite sides are parallel The adjacent edge is perpendicular to the top and bottom edges;

[0038] in, , is the coordinate variable in the equation of a straight line, representing the coordinates of any point on the plane; These are the specific coordinates of the candidate edge points, which are known points that are substituted into the line equation to calculate the distance;

[0039] S506: Define the fitting error. ;

[0040] Minimize using the Levenberg-Marquardt algorithm Solve for parameters , , , , This yields a rectangular outline, which represents the planar position of the hatch edge.

[0041] Preferably, in step S6, the inverse mapping is achieved through inverse projection transformation, where the two-dimensional projected coordinates are... The corresponding 3D point cloud coordinates are The orthogonal projection matrix is ,in, It is a 3×3 identity matrix. The global normal vector of the deck is given, and the projection process satisfies the following conditions: ;

[0042] During inverse mapping, first calculate the average height of the point cloud in the deck area. , ;

[0043] in, The total number of point clouds on the deck. Let (where z is the z-coordinate of the j-th deck point) be... ,Will Substituting into the projection equation and combining it with the expression for Q, we obtain... Finally, the three-dimensional coordinates of the hatch were obtained. .

[0044] Preferably, in step S503, the neighborhood radius ε and the minimum number of points MinPts for DBSCAN clustering are determined by an adaptive algorithm. Specifically, for the filtered point cloud, the distance from each point to its 5th nearest neighbor is calculated. j=1,2,...,N, where N is the total number of point clouds after filtering, and the neighborhood radius ε is taken from all... The average value of MinPts is set to 10 to ensure that the clustering results are adapted to the deck point cloud distribution density under different scenarios.

[0045] The present invention, which adopts the above technical solution, has the following prominent features compared with the prior art:

[0046] 1. By deploying at least two different types of lidar, combined with a detection range overlap of no less than 15%, the operation area can be covered without blind spots. At the same time, by utilizing the complementary advantages of multi-dimensional point cloud data, interference from complex working conditions such as strong light and dust can be effectively resisted, significantly improving the stability and environmental adaptability of positioning.

[0047] 2. By using external parameter calibration and weighted fusion algorithms to achieve coordinate unification and data optimization of multi-source point clouds, and then using normal vector filtering and DBSCAN adaptive clustering to accurately segment the deck area, interference point clouds such as cargo stacks and fuselage are eliminated, ensuring the accuracy of hatch identification from the data source and solving the problem of difficult target area extraction in complex scenarios.

[0048] 3. The deck 3D point cloud is converted into a 2D image through orthogonal projection. The hatch edges are quickly extracted using the improved Canny operator and least squares rectangle fitting. Then, the 3D coordinates are restored through projection inverse mapping. This reduces the complexity of edge detection in 2D space and ensures 3D positioning accuracy through deck average height calibration, achieving a balance between efficiency and accuracy.

[0049] 4. The neighborhood radius and minimum number of points of DBSCAN clustering are determined by adaptive calculation based on data. Rectangular fitting supports square hatches of different angles and sizes. The multi-radar deployment scheme can flexibly adapt to different specifications of ship unloaders and bulk carriers, solving the defects of poor adaptability and weak generalization ability of traditional schemes. Attached Figure Description

[0050] Figure 1 This is a flowchart of the hatch positioning method in an embodiment of the present invention;

[0051] Figure 2 This is a schematic diagram of the hatch edge rectangle fitting after the point cloud projection of the deck area in an embodiment of the present invention;

[0052] Figure 3 This is a three-dimensional positional diagram of the ship's deck and hatch area in an embodiment of the present invention. Detailed Implementation

[0053] The present invention will be further illustrated below with reference to specific embodiments. The purpose of this illustration is solely to provide a better understanding of the invention. Therefore, the examples given do not limit the scope of protection of the present invention.

[0054] like Figure 1 This embodiment presents a hatch localization method based on multimodal lidar point clouds, the specific steps of which include:

[0055] S1, Multimodal point cloud data acquisition: Deploy at least two different types of lidar at preset positions on the unloader beam, adjust the pitch and horizontal angles of each lidar so that the detection range of all lidars is spliced ​​to cover the entire working area, and collect multi-dimensional point cloud data of the target area.

[0056] Specifically, the different types of lidar are at least two of the following: 2D / 3D fusion lidar, multi-beam lidar, and high-resolution solid-state lidar, and the overlap ratio of the detection range of any two adjacent lidars is not less than 15%.

[0057] S2, Multi-source point cloud integration processing: Point cloud data acquired by multiple lidars are unified into the same coordinate system through radar extrinsic parameter calibration or point cloud fusion algorithm to form fused complete point cloud data.

[0058] Radar extrinsic parameter calibration is achieved using a calibration plate method. The calibration process includes placing a calibration plate in the common detection area of ​​multiple lidars, setting no fewer than 10 discretely distributed feature points on the calibration plate, and obtaining the coordinates of each feature point in the coordinate systems of each lidar. ,in, The feature point number, =1,2,...,n, where n≥10, represents the total number of feature points on the calibration board; simultaneously, the standard coordinates of the feature points in the world coordinate system are obtained. The extrinsic parameter matrix is ​​solved by minimizing the coordinate transformation error. ,in, It is a 3×3 rotation matrix used to describe the attitude relationship between the lidar coordinate system and the world coordinate system; This is a 3×1 rotational translation vector used to describe the positional offset of the lidar coordinate system relative to the world coordinate system; the coordinate transformation formula is... The error function is ,in The L2 norm, i.e. Euclidean distance, is used to minimize E using the Levenberg-Marquardt algorithm to obtain the optimal extrinsic parameter matrix, thereby unifying the coordinate systems of various lidar systems with the world coordinate system.

[0059] The point cloud fusion algorithm employs a weighted average fusion strategy, and the fusion formula is as follows: ,in, The coordinates of the merged point cloud. Let be the number of lidars, and be the point cloud coordinates of the same spatial location collected by the k-th lidar. Let the weight of the k-th lidar be , , Let the standard deviation of the measurement error of the k-th lidar at this spatial location be obtained in advance through calibration experiments at different detection distances. And establish an error standard deviation mapping table for use during fusion.

[0060] S3, Deck Area Point Cloud Segmentation: Identify and segment the deck area point cloud corresponding to the bulk carrier deck area from the fused complete point cloud data.

[0061] Point cloud segmentation of the deck area is achieved through normal vector filtering and DBSCAN clustering, specifically including:

[0062] S301: Calculate each point in the fused point cloud unit normal vector ;right Select q nearest neighbor points to form a point set q is the number of nearest neighbors; calculate the point set. center of mass Construct the covariance matrix ;in, It is a column vector. Transpose it. It is a 3×3 symmetric positive semi-definite matrix; for Perform eigenvalue decomposition to obtain eigenvalues. ≤ ≤ and corresponding feature vectors , , Take the smallest eigenvalue corresponding feature vector After normalization, we get unit normal vector ;

[0063] S302: Normal vector filtering, retain Angle with z-axis Point clouds with an angle of ≤30° The calculation formula is ,in ;

[0064] S303, DBSCAN clustering, sets the neighborhood radius ε and the minimum number of points MinPts, traverses the filtered point cloud, defines the core point as the point whose number of points in the ε neighborhood is greater than or equal to MinPts; defines the boundary point as the point whose number of points in the ε neighborhood is less than MinPts, but falls within the ε neighborhood of any core point; groups each core point and its connected boundary points into the same cluster, removes noise points not associated with any core point, and determines the largest point cloud cluster obtained by clustering as the deck area point cloud.

[0065] The deck point cloud segmentation method in this step prioritizes the segmentation of the deck point cloud area, which can effectively eliminate interference from the point clouds in surrounding areas such as cargo storage, unloading machine hull, and bridge, greatly improving the accuracy of subsequent hatch area detection.

[0066] S4, 3D Point Cloud Plane Projection: The deck area point cloud is projected along a direction perpendicular to the deck plane, compressing the 3D point cloud data into a 2D planar projection image. The deck point cloud segmented in step S3 is mapped to a projection image along a direction perpendicular to the deck plane. The hatch area point cloud in the deck can be mapped to a rectangular (or straight line) shape in the projection image. Subsequently, through image processing using rectangle (straight line) fitting, the edge position of the hatch area is searched in the projection image, and finally, the edge position is converted into spatial coordinate values ​​for output.

[0067] The direction perpendicular to the deck plane is determined by the global normal vector of the point cloud in the deck area. Sure, ;

[0068] in, The total number of points in the point cloud over the deck area. Let be the unit normal vector of the j-th point, summed and then normalized; the projection transformation uses orthogonal projection, and the projection formula is,

[0069] ;

[0070] in, The point cloud coordinates and projection matrix of the three-dimensional deck area. The projection process satisfies That is, to remove The coordinate components of the direction are retained, retaining those perpendicular to the direction. Two-dimensional plane coordinates.

[0071] S5, Hatch Edge Fitting and Positioning: In the two-dimensional planar projection image of the deck, the planar position of the hatch edge is determined by image edge detection and geometric shape fitting techniques, such as... Figure 2 .

[0072] In step S5, the image edge detection uses an improved Canny operator, specifically including:

[0073] S501: Gaussian filter noise reduction, using a 5×5 Gaussian filter core. ,in =1.2 is the Gaussian kernel standard deviation. , The coordinates of the pixels within the filter kernel relative to the center are given, with values ​​ranging from -2, -1, 0, 1, 2. Convolution operations are performed on the projected image to achieve noise suppression.

[0074] S502: Gradient calculation, performed using the Sobel operator. , Directional gradient , The Sobel operator's convolution kernel in the x-direction is The convolution kernel in the y-direction is gradient strength gradient direction , The range of values ​​is [0, π).

[0075] S503: Non-maximum suppression, discretizes the gradient direction into 8 directions, compares the gradient intensity of the current pixel with that of the two adjacent pixels along the gradient direction, and retains only the local maxima points to achieve edge refinement.

[0076] Furthermore, the neighborhood radius ε and minimum number of points MinPts for DBSCAN clustering are determined using an adaptive algorithm. Specifically, for the filtered point cloud, the distance from each point to its 5th nearest neighbor is calculated. j=1,2,...,N, where N is the total number of point clouds after filtering, and the neighborhood radius ε is taken from all... The average value of MinPts is set to 10 to ensure that the clustering results are adapted to the deck point cloud distribution density under different scenarios.

[0077] S504: Dual threshold filtering, setting a high threshold , The maximum gradient intensity and low threshold of all pixels in the image. Retain strong edge points with gradient strength greater than 0, and those between 0 and 0. and Weak edge points that are connected to strong edge points are identified, and isolated weak edge points are removed to obtain a set of candidate hatch edge points.

[0078] The geometric shape fitting technique is rectangle fitting, based on the least squares principle, and specifically includes:

[0079] S505: Let the set of candidate points at the hatch edge be... , Let m be the number of candidate points. Assume the hatch projection is a rectangle, with its four sides satisfying the constraints of opposite sides being parallel and adjacent sides being perpendicular. Let the direction of the major axis of the rectangle be perpendicular to the direction of the hatch projection. The included angle of the axis is Then the equations of the lines on the four sides are respectively,

[0080] Above, ;below, ;left, ;right, ;in, Opposite sides are parallel The adjacent edge is perpendicular to the top and bottom edges;

[0081] in, , is the coordinate variable in the equation of a straight line, representing the coordinates of any point on the plane; The specific coordinates of the candidate edge points are known points that are substituted into the line equation to calculate the distance.

[0082] S506: Define the fitting error. ;

[0083] Minimize using the Levenberg-Marquardt algorithm Solve for parameters , , , , This yields a rectangular outline, which represents the planar position of the hatch edge.

[0084] S6, Hatch three-dimensional coordinates determined: such as Figure 3 The planar position coordinates of the hatch edge are inversely mapped to the three-dimensional point cloud coordinate system to obtain the actual coordinates of the hatch in three-dimensional space.

[0085] Inverse mapping is achieved through inverse projection transformation. Let the two-dimensional projected coordinates be... The corresponding 3D point cloud coordinates are The orthogonal projection matrix is ,in, It is a 3×3 identity matrix. The global normal vector of the deck is given, and the projection process satisfies the following conditions: ;

[0086] During inverse mapping, first calculate the average height of the point cloud in the deck area. , ;

[0087] in, The total number of point clouds on the deck. Let (where z is the z-coordinate of the j-th deck point) be... ,Will Substituting into the projection equation and combining it with the expression for Q, we obtain... Finally, the three-dimensional coordinates of the hatch were obtained. .

[0088] In this method, the deployment based on multi-modal multi-radar can provide a more flexible deployment scheme, effectively covering ship unloader operation scenarios of different sizes, while ensuring high-density coverage and high-precision positioning of the operation target area; the hatch detection method that combines point cloud segmentation and hatch line fitting can not only ensure the accuracy of hatch position detection, but also adapt to different sizes of bulk carriers and square hatches by improving the flexibility of the algorithm.

[0089] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of the invention. All equivalent changes made based on the description and drawings of the present invention are included within the scope of the present invention.

Claims

1. A hatch localization method based on multimodal lidar point clouds, characterized in that, The specific steps include: S1, Multimodal point cloud data acquisition: Deploy at least two different types of lidar at preset positions on the main beam of the unloader, adjust the pitch and horizontal angles of each lidar so that the detection range of all lidars is spliced ​​to cover the entire working area, and collect multi-dimensional point cloud data of the target area. S2, Multi-source point cloud integration processing: Point cloud data acquired by multiple lidars are unified into the same coordinate system through radar extrinsic parameter calibration or point cloud fusion algorithm to form fused complete point cloud data; S3, Deck Area Point Cloud Segmentation: Identify and segment the deck area point cloud corresponding to the bulk carrier deck area from the fused complete point cloud data; S4, 3D point cloud planar projection: Project the point cloud of the deck area along a direction perpendicular to the deck plane, compressing the 3D point cloud data of the deck into a 2D planar projection image of the deck; S5, Hatch Edge Fitting and Positioning: In the two-dimensional planar projection image of the deck, the planar position of the hatch edge is determined by image edge detection and geometric shape fitting technology; S6, Determining the three-dimensional coordinates of the hatch: Inversely map the planar position coordinates of the hatch edge to the three-dimensional point cloud coordinate system to obtain the actual coordinates of the hatch in three-dimensional space.

2. The hatch positioning method based on multimodal lidar point clouds according to claim 1, characterized in that: In step S1, the different types of lidar are at least two of the following: 2D / 3D fusion lidar, multi-beam lidar, and high-resolution solid-state lidar, and the overlap ratio of the detection range of any two adjacent lidars is not less than 15%.

3. The hatch positioning method based on multimodal lidar point clouds according to claim 1, characterized in that: In step S2, the radar extrinsic parameter calibration is achieved using the calibration board method. The calibration process includes: The calibration board is placed in the common detection area of ​​multiple lidars. The calibration board has at least 10 discretely distributed feature points, and the coordinates of each feature point in the coordinate system of each lidar are obtained. ,in, The feature point number, =1,2,...,n, where n≥10 is the total number of feature points on the calibration plate; Simultaneously, obtain the standard coordinates of the feature points in the world coordinate system. The extrinsic parameter matrix is ​​solved by minimizing the coordinate transformation error. ,in, It is a 3×3 rotation matrix used to describe the attitude relationship between the lidar coordinate system and the world coordinate system; This is a 3×1 rotational translation vector used to describe the positional offset of the lidar coordinate system relative to the world coordinate system; the coordinate transformation formula is... The error function is ,in The L2 norm, i.e. Euclidean distance, is used to minimize E using the Levenberg-Marquardt algorithm to obtain the optimal extrinsic parameter matrix, thereby unifying the coordinate systems of various lidar systems with the world coordinate system.

4. The hatch positioning method based on multimodal lidar point clouds according to claim 1, characterized in that: In step S2, the point cloud fusion algorithm adopts a weighted average fusion strategy, and the fusion formula is as follows: ,in, The coordinates of the merged point cloud. Let be the number of lidars, and be the point cloud coordinates of the same spatial location collected by the k-th lidar. Let the weight of the k-th lidar be , , Let the standard deviation of the measurement error of the k-th lidar at this spatial location be obtained in advance through calibration experiments at different detection distances. And establish an error standard deviation mapping table for use during fusion.

5. The hatch positioning method based on multimodal lidar point clouds according to claim 1, characterized in that: In step S3, the point cloud segmentation of the deck area is achieved through normal vector filtering and DBSCAN clustering, specifically including: S301: Calculate each point in the fused point cloud unit normal vector ;right Select q nearest neighbor points to form a point set q is the number of nearest neighbors; calculate the point set. center of mass Construct the covariance matrix ;in, It is a column vector. Transpose it. It is a 3×3 symmetric positive semi-definite matrix; for Perform eigenvalue decomposition to obtain eigenvalues. ≤ ≤ and corresponding feature vectors , , Take the smallest eigenvalue corresponding feature vector After normalization, we get unit normal vector ; S302: Normal vector filtering, retain Angle with z-axis Point clouds with an angle of ≤30° The calculation formula is ,in ; S303, DBSCAN clustering, sets the neighborhood radius ε and the minimum number of points MinPts, traverses the filtered point cloud, defines the core point as the point whose number of points in the ε neighborhood is greater than or equal to MinPts; defines the boundary point as the point whose number of points in the ε neighborhood is less than MinPts, but falls within the ε neighborhood of any core point; groups each core point and its connected boundary points into the same cluster, removes noise points not associated with any core point, and determines the largest point cloud cluster obtained by clustering as the deck area point cloud.

6. The hatch positioning method based on multimodal lidar point clouds according to claim 1, characterized in that: In step S4, the direction perpendicular to the deck plane is determined by the global normal vector of the point cloud in the deck area. Sure, ; in, The total number of points in the point cloud over the deck area. Let be the unit normal vector of the j-th point, summed and then normalized; the projection transformation uses orthogonal projection, and the projection formula is, ； in, The point cloud coordinates and projection matrix of the three-dimensional deck area. The projection process satisfies That is, to remove The coordinate components of the direction are retained, retaining those perpendicular to the direction. Two-dimensional plane coordinates.

7. The hatch positioning method based on multimodal lidar point clouds according to claim 1, characterized in that, In step S5, the image edge detection uses an improved Canny operator, specifically including: S501: Gaussian filter noise reduction, using a 5×5 Gaussian filter core. ,in =1.2 is the Gaussian kernel standard deviation. , The coordinates of the pixels within the filter kernel relative to the center are given, with values ​​ranging from −2, −1, 0, 1, 2. Convolution operations are performed on the projected image to achieve noise suppression. S502: Gradient calculation, performed using the Sobel operator. , Directional gradient , The Sobel operator's convolution kernel in the x-direction is The convolution kernel in the y-direction is gradient strength gradient direction , The range of values ​​is [0, π). S503: Non-maximum suppression, which discretizes the gradient direction into 8 directions, compares the gradient intensity of the current pixel with that of the two adjacent pixels along the gradient direction, and retains only the local maxima points to achieve edge refinement; S504: Dual threshold filtering, setting a high threshold , The maximum gradient intensity and low threshold of all pixels in the image. Retain strong edge points with gradient strength greater than 0, and those between 0 and 0. and Weak edge points that are connected to strong edge points are identified, and isolated weak edge points are removed to obtain a set of candidate hatch edge points.

8. The hatch positioning method based on multimodal lidar point clouds according to claim 1, characterized in that, In step S5, the geometric shape fitting technique is rectangle fitting, which is implemented based on the least squares principle, and specifically includes: S505: Let the set of candidate points at the hatch edge be... , Let m be the number of candidate points. Assume the hatch projection is a rectangle, with its four sides satisfying the constraints of opposite sides being parallel and adjacent sides being perpendicular. Let the direction of the major axis of the rectangle be perpendicular to the direction of the hatch projection. The included angle of the axis is Then the equations of the lines on the four sides are respectively, Above, ;below, ;left, ;right, ;in, Opposite sides are parallel The adjacent edge is perpendicular to the top and bottom edges; in, , is the coordinate variable in the equation of a straight line, representing the coordinates of any point on the plane; These are the specific coordinates of the candidate edge points, which are known points that are substituted into the line equation to calculate the distance; S506: Define the fitting error. ; Minimize using the Levenberg-Marquardt algorithm Solve for parameters , , , , This yields a rectangular outline, which represents the planar position of the hatch edge.

9. The hatch positioning method based on multimodal lidar point clouds according to claim 1, characterized in that: In step S6, the inverse mapping is achieved through the inverse projection transformation. Let the two-dimensional projected coordinates be... The corresponding 3D point cloud coordinates are The orthogonal projection matrix is ,in, It is a 3×3 identity matrix. The global normal vector of the deck is given, and the projection process satisfies the following conditions: ; During inverse mapping, first calculate the average height of the point cloud in the deck area. , ; in, The total number of point clouds on the deck. Let (where z is the z-coordinate of the j-th deck point) be... ,Will Substituting into the projection equation and combining it with the expression for Q, we obtain... Finally, the three-dimensional coordinates of the hatch were obtained. .

10. The hatch positioning method based on multimodal lidar point clouds according to claim 5, characterized in that: In step S503, the neighborhood radius ε and the minimum number of points MinPts for DBSCAN clustering are determined using an adaptive algorithm. Specifically, for the filtered point cloud, the distance from each point to its 5th nearest neighbor is calculated. j=1,2,...,N, where N is the total number of point clouds after filtering, and the neighborhood radius ε is taken from all... The average value of MinPts is 2×k=10 to ensure that the clustering results are adapted to the deck point cloud distribution density under different scenarios.

Images