A multi-sensor-based method for unmanned vessel lock passage identification

Abstract

This invention provides a multi-sensor-based method for unmanned surface vessel (USV) gate passage identification, comprising the following steps: S1, a method for acquiring obstacles in front of the USV based on lidar; S2, a method for acquiring obstacles in front of the USV based on a visual sensor; S3, a method for USV gate passage environment perception based on the fusion of lidar and visual sensors. This method uses two sensors to extract valid obstacles in front of the USV separately, and fuses the obstacle information acquired by the two sensors by constructing a region of interest, thereby improving the USV's environmental perception capability for gate passage.

Description

Technical Field

[0001] This invention belongs to the field of unmanned vessel navigation image processing technology, and in particular relates to a method for sensing and recognizing the gate passage environment based on the fusion of lidar and visual sensors. Background Technology

[0002] With the rapid development of the shipping industry, the efficiency and safety of ship navigation have become research hotspots. The research and application of unmanned surface vessel (USV) technology can effectively reduce labor costs and human error, thus attracting widespread attention. With the rapid advancements in maritime communication, collaborative control, and artificial intelligence technologies, USV environmental recognition, as an important extension of ship intelligent system development, has gradually become a reality in the research and application of USV collaborative navigation. Unlike open waters, locks are very narrow. In these restricted lock passage areas, the distance allowed for USV passage is small. During navigation, the USV adaptively adjusts its lateral distance to the lock walls to ensure safe passage and improve channel capacity.

[0003] The raw data for environmental perception of unmanned vessels comes from shipboard sensing equipment. Existing single environmental perception and detection technologies for unmanned vessels have limitations in meeting the speed and accuracy requirements for environmental perception of locks during high-speed travel. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to propose an environmental perception method based on the fusion of lidar and visual sensors. This method uses two sensors to extract effective obstacles in front of the unmanned vessel, and fuses the obstacle information obtained by the two sensing devices by constructing a region of interest, thereby improving the environmental perception capability of the unmanned vessel when passing through the lock.

[0005] To achieve the above-mentioned technical features, the objective of this invention is as follows: A multi-sensor-based unmanned vessel lock passage identification method, comprising the following steps:

[0006] S1. A method for acquiring obstacles in front of an unmanned surface vessel based on lidar;

[0007] S2. A method for obtaining obstacles in front of an unmanned surface vessel based on visual sensors;

[0008] S3. An environmental perception method for unmanned vessels passing through locks based on the fusion of lidar and visual sensors.

[0009] The specific detection method of the obstacle acquisition method for the unmanned vessel based on lidar in S1 is as follows:

[0010] S1.1: Acquire and detect the distance and angle information between the target lock and the lidar point cloud data, and perform point cloud preprocessing on the obtained information:

[0011] First, the point cloud coordinates are converted into two-dimensional coordinates. Then, filtering is performed to remove noise generated by clutter in the collected point cloud. Statistical filtering and direct-pass filtering are used for processing. Statistical filtering removes sparse outlier noise points. Since the overall distance of the detection lock is relatively far from the unmanned vessel, and the measurement will introduce noise, direct-pass filtering is used to filter out points that are not in the specified dimension direction.

[0012] During data processing, the original point cloud from the lidar must first be converted from spherical coordinates (ρ, α, ω) to Cartesian coordinates (x, y, z) in the unmanned surface vessel coordinate system:

[0013]

[0014] S1.2: Since the target lock is mostly made of low-carbon alloy steel plate, more laser point cloud data is collected than other obstacles. The lock and non-target obstacles are screened by two steps: reflection intensity value screening and point cloud density screening.

[0015] S1.3: After steps S1.1 and S1.2, the amount of point cloud data will be greatly reduced, leaving only the useful point cloud data that needs to be processed, as well as some data that cannot be filtered out by reflection intensity and point cloud density values. The processed data will be used to perform plane extraction by iteratively applying the RANSAC algorithm. The RANSAC algorithm is an iterative method used to estimate the parameters of a mathematical model from a set of observation data containing outliers, in which case the outliers will not affect the estimated values.

[0016] The specific method for selecting the reflection intensity value in S1.2 is as follows:

[0017] Because water waves generate laser echoes, but their reflection intensity is usually low, after multiple data analyses, due to the special reasons of reflection from the water surface or water-borne obstructions, data with excessively low intensity values ​​are not the targets to be detected. Point cloud data with excessively low intensity values ​​reflect water surface points. The intensity values ​​are normalized to a range of 0-100. Therefore, based on the characteristics of water surface reflection values, point cloud data with intensity values ​​of 70-100 are retained as data for further processing.

[0018] The specific method for point cloud density filtering in S1.2 is as follows:

[0019] Because the lidar data acquisition range is large and contains many invalid areas, a pass-through filter is used to subtract the minimum and maximum values ​​of the X and Y coordinates of the data to facilitate subsequent calculations and processing. This ensures that the center point of the planar coordinates is located on the point cloud data, while the elevation Z direction remains unchanged. First, the 3D point cloud is converted into a 2D grid map representation based on the water surface. This represents the point cloud data as multiple grid regions with unconstrained heights. The number of point clouds in each grid region represented by each small square is counted. Since the point cloud reflected by water waves is discrete, and the lock has a high reflectivity, the overall number of point clouds acquired is also large. If the point cloud data in each small square is less than a certain value, it is not the target to be detected.

[0020] The specific implementation process of the RANSAC algorithm in S1.3 is as follows: based on the adaptive formula, the optimal solution for the number of iterations is calculated, and all planes in the dataset that meet the requirements can be extracted. The formula is as follows:

[0021] Let p: the expected probability that the RANSAC algorithm provides at least one useful result after running;

[0022] Let w: points that satisfy the model / all detected points;

[0023] Let n: the number of points that satisfy the current model;

[0024] Then 1-p=(1-w) n ) k ;

[0025] Right now

[0026] Based on the adaptive formula, the optimal solution for the number of iterations can be calculated, and all planes in the dataset that meet the requirements can be extracted.

[0027] The extracted planes are labeled as P. i (i = 1, 2, 3…n), due to the limited performance of lidar in measuring long-range targets, a pass-through filter is used to remove cluster planes with an elevation Z-axis below 2m. Then, in each remaining plane, a line perpendicular to the ground is searched for the left and right endpoints, P and P', respectively. i(left) P i(right) Check the two endpoints of each plane individually. If they do not coincide, continue the search. If P i The line between the two endpoints and P i+1 One of the two endpoints of the straight line coincides. Principal component analysis (PCA) is used to extract P. i(left) P i(right) The normal vectors of the two planes are converted into the plane angle θ. If the plane angle θ fluctuates within 140°±10°, it is identified as a lock.

[0028] The specific method for obtaining obstacles in front of the unmanned vessel based on visual sensors in S2 is as follows:

[0029] Step S2.1: Establish the coordinate system of the vision sensor and transform it with the world coordinate system. To obtain target information in the three-dimensional environment, the position of this two-dimensional plane, i.e., the imaging surface of the camera, must first be found. Then, based on the ranging principle, the three-dimensional coordinates corresponding to the two-dimensional points on the image are calculated, and the camera is calibrated using Zhang Zhengyou's plane calibration method.

[0030] Step S2.2: Due to the complex environment of the unmanned vessel navigating on the water surface to detect the lock, the fluctuation of the water waves causes the unmanned vessel to be unstable, resulting in some interference information during the image acquisition process of the camera. This part of the information will cover the original image data, resulting in image data loss. In order to reduce the interference of the external environment on the acquired image, image preprocessing is required to simplify the data and enhance the detectability of the target image. Image preprocessing methods include image enhancement and image smoothing.

[0031] Step S2.3: In complex water navigation environments, the navigation of large ships and obstacles such as bridge piers can affect the correct detection and identification of locks. Effective image segmentation to independently divide the target area and non-target area of ​​the lock is essential. The K-means segmentation algorithm is used to segment the acquired obstacle images.

[0032] The image preprocessing method in step S2.2 specifically operates as follows:

[0033] First, the image is enhanced through grayscale conversion, grayscale stretching, and histogram equalization. During grayscale conversion, since the human eye perceives red, green, and blue light differently, each color is assigned a different weight to obtain a grayscale value. Grayscale stretching can expand the grayscale range of the acquired image and improve its contrast, making the image clearer and reducing environmental interference during unmanned surface vessel navigation. To reduce the decrease in image resolution caused by lighting interference, histogram equalization is used to improve detection accuracy.

[0034] The formula for converting grayscale values ​​is:

[0035] Y = 0.229 * R + 0.587 * G + 0.114 * B

[0036] During the image acquisition process by the camera, after the image is preprocessed, there are still parts of the image that are affected by noise. This causes the uniform gray values ​​to change abruptly at a certain point. Smoothing the image can eliminate noise in the image while preserving the details of the image information as much as possible, thereby enhancing the image effect. Median filtering, mean filtering and Gaussian filtering are used to achieve this respectively.

[0037] The median filter transformation formula is:

[0038] Q(i,j)=Median{P(im,jn),(m,n)}

[0039] In the formula, P(i,j) is the original image; Q(i,j) is the processed image; and (m,n) is the neighborhood.

[0040] In step S2.3, the K-means segmentation algorithm segments the points in the feature space according to the clustering method, and then maps them back to the original image to obtain the segmentation result. The sum of squared errors is used as the clustering criterion function, and the Euclidean distance is measured to calculate the similarity, which ensures the highest intra-class similarity and the lowest inter-class similarity, thus achieving local optima.

[0041] First, K initial class means need to be selected in the given space. Then, each pixel is assigned to its own class. The process ends when the difference between the obtained class means and the initial class means is less than a set threshold. It can segment the part affected by lighting into the background image and can also effectively preserve the color, texture and shape features of the target image.

[0042] The SURF feature matching algorithm was used to perform stereo matching on the image, and the RANSAC algorithm was used to remove erroneous matching points.

[0043] The specific method of the unmanned vessel lock passage environment perception method based on the fusion of lidar and visual sensors in S3 is as follows:

[0044] Spatial fusion is achieved through coordinate system transformation. The camera calibration adopts Zhang Zhengyou's monocular camera calibration method. Based on the obstacle orientation information detected by the lidar, the corresponding region of interest is constructed and matched with the minimum bounding rectangle of the obstacle contour obtained by the visual algorithm to determine the position and coverage of the effective target.

[0045] The specific operation in S3 is as follows:

[0046] Step S3.1: Obtain the relative position information of the lidar and the vision sensor. Both need to be calibrated. First, based on the relative position relationship between the two and the corresponding coordinate system is established, the radar point cloud data and image information are converted to the actual imaging pixel coordinate system to complete the coordinate transformation.

[0047] Step S3.2: By fusing the corresponding coordinate systems, the LiDAR and visual sensor are unified to the same dimension. Based on this, the region of interest is first drawn with the obstacle position in the environment obtained by the LiDAR as the center and a certain scale as the radius. Then, the minimum bounding rectangle of the lock outline obtained by the visual system is found. The corresponding region of interest is constructed based on the obstacle orientation information detected by the LiDAR. The specific position of the minimum bounding rectangle of the obstacle outline obtained by the visual algorithm can be determined by positioning and coordinate transformation of the unmanned surface vessel. By comparing the position information of the two parts to see if they overlap, it can be determined whether the target is the detection target.

[0048] The present invention has the following beneficial effects:

[0049] 1. This invention employs a multi-sensor fusion method for identifying ship locks, combining 3D LiDAR and visual sensors. The image information provided by the camera and the distance and direction information acquired by the 3D LiDAR can complement each other, thus making the point cloud features of the ship lock more obvious, which is more conducive to detection.

[0050] 2. In this invention, since the three-dimensional lidar detection adopts the filtering of reflection intensity value, the filtering of three-dimensional point cloud density, and the clustering algorithm based on RANSAC algorithm and PCA principal component analysis method to detect plane angle, it also has the advantages of sufficient feature extraction, high detection rate and small computational amount.

[0051] 3. This invention can provide fast and effective lock detection and identification in shipboard electronic chart construction and automatic navigation through locks, realizing intelligent shipping and intelligent industry. Attached Figure Description

[0052] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0053] Figure 1 In the figures, a) and b) are the top view and side view respectively, defined by the coordinate system of the unmanned surface vessel's three-dimensional lidar.

[0054] Figure 2 This shows the top-down coordinate correspondence between the lidar and the camera.

[0055] Figure 3 This is a flowchart of the data fusion process.

[0056] Figure 4 This describes the direct calibration principle for lidar and cameras. Detailed Implementation

[0057] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0058] Example 1:

[0059] See Figure 1-4 A method for identifying unmanned vessels passing through locks based on multiple sensors includes the following steps:

[0060] S1. A method for acquiring obstacles in front of an unmanned surface vessel based on lidar;

[0061] When unmanned vessels are navigating and locating target locks to acquire data, random turbulence and swaying of the water surface can cause the acquired data to be unstable, resulting in the loss of target information or the identification of virtual obstacles.

[0062] S1.1: The lidar point cloud data is used to acquire and detect the distance and angle information between the target lock and the vessel lock. The obtained information is then preprocessed into point cloud data. First, the point cloud coordinates are converted into two-dimensional coordinates. Then, filtering is performed to remove noise generated by clutter in the acquired point cloud. Statistical filtering and direct-pass filtering are used for processing. Statistical filtering removes sparse outlier noise points. Since the overall distance of the detected lock is relatively far from the unmanned vessel, and the measurement will introduce noise, direct-pass filtering is used to filter out points that are not in the specified dimensional direction.

[0063] During data processing, the original point cloud from the lidar must first be converted from spherical coordinates (ρ, α, ω) to Cartesian coordinates (x, y, z) in the unmanned surface vessel coordinate system:

[0064]

[0065] S1.2: Since the target lock is mostly made of low-carbon alloy steel plate, more laser point cloud data is collected than other obstacles. The lock and non-target obstacles are screened by two steps: reflection intensity value screening and point cloud density screening.

[0066] The specific method for selecting the reflection intensity value in S1.2 is as follows:

[0067] Because water waves generate laser echoes, but their reflection intensity is usually low, after multiple data analyses, due to the special reasons of reflection from the water surface or water-borne obstructions, data with excessively low intensity values ​​are not the targets to be detected. Point cloud data with excessively low intensity values ​​reflect water surface points. The intensity values ​​are normalized to a range of 0-100. Therefore, based on the characteristics of water surface reflection values, point cloud data with intensity values ​​of 70-100 are retained as data for further processing.

[0068] The specific method for point cloud density filtering in S1.2 is as follows:

[0069] Because the lidar data acquisition range is large and contains many invalid areas, a pass-through filter is used to subtract the minimum and maximum values ​​of the X and Y coordinates of the data to facilitate subsequent calculations and processing. This ensures that the center point of the planar coordinates is located on the point cloud data, while the elevation Z direction remains unchanged. First, the 3D point cloud is converted into a 2D grid map representation based on the water surface. This represents the point cloud data as multiple grid regions with unconstrained heights. The number of point clouds in each grid region represented by each small square is counted. Since the point cloud reflected by water waves is discrete, and the lock has a high reflectivity, the overall number of point clouds acquired is also large. If the point cloud data in each small square is less than a certain value, it is not the target to be detected.

[0070] S1.3: After steps S1.1 and S1.2, the amount of point cloud data will be greatly reduced, leaving only the useful point cloud data that needs to be processed, as well as some data that cannot be filtered out by reflection intensity and point cloud density values. The processed data will be used to perform plane extraction by iteratively applying the RANSAC algorithm. The RANSAC algorithm is an iterative method used to estimate the parameters of a mathematical model from a set of observation data containing outliers, in which case the outliers will not affect the estimated values.

[0071] The specific implementation process of the RANSAC algorithm in S1.3 is as follows: based on the adaptive formula, the optimal solution for the number of iterations is calculated, and all planes in the dataset that meet the requirements can be extracted. The formula is as follows:

[0072] Let p: the expected probability that the RANSAC algorithm provides at least one useful result after running;

[0073] Let w: points that satisfy the model / all detected points;

[0074] Let n: the number of points that satisfy the current model;

[0075] Then 1-p=(1-w) n ) k ;

[0076] Right now

[0077] Based on the adaptive formula, the optimal solution for the number of iterations can be calculated, and all planes in the dataset that meet the requirements can be extracted.

[0078] The extracted planes are labeled as P. i (i = 1, 2, 3…n), due to the limited performance of lidar in measuring long-range targets, a pass-through filter is used to remove cluster planes with an elevation Z-axis below 2m. Then, in each remaining plane, a line perpendicular to the ground is searched for the left and right endpoints, P and P', respectively. i(left) P i(right)Check the two endpoints of each plane individually. If they do not coincide, continue the search. If P i The line between the two endpoints and P i+1 One of the two endpoints of the straight line coincides. Principal component analysis (PCA) is used to extract P. i(left) P i(right) The normal vectors of the two planes are converted into the plane angle θ. If the plane angle θ fluctuates within 140°±10°, it is identified as a lock.

[0079] Among them, PCA stands for Principal Component Analysis. Its goal is to map high-dimensional data into a low-dimensional space through a linear projection, and to maximize the variance of the data in the projected dimension, thereby using fewer data dimensions while retaining more of the characteristics of the original data points.

[0080] Example 2:

[0081] The specific method for obtaining obstacles in front of the unmanned vessel based on visual sensors in S2 is as follows:

[0082] Step S2.1: Establish the coordinate system of the vision sensor and transform it with the world coordinate system. To obtain target information in the three-dimensional environment, the position of this two-dimensional plane, i.e., the imaging surface of the camera, must first be found. Then, based on the ranging principle, the three-dimensional coordinates corresponding to the two-dimensional points on the image are calculated, and the camera is calibrated using Zhang Zhengyou's plane calibration method.

[0083] Step S2.2: Due to the complex environment of the unmanned vessel navigating on the water surface to detect the lock, the fluctuation of the water waves causes the unmanned vessel to be unstable, resulting in some interference information during the image acquisition process of the camera. This part of the information will cover the original image data, resulting in image data loss. In order to reduce the interference of the external environment on the acquired image, image preprocessing is required to simplify the data and enhance the detectability of the target image. Image preprocessing methods include image enhancement and image smoothing.

[0084] The image preprocessing method in step S2.2 specifically operates as follows:

[0085] First, the image is enhanced through grayscale conversion, grayscale stretching, and histogram equalization. During grayscale conversion, since the human eye perceives red, green, and blue light differently, each color is assigned a different weight to obtain a grayscale value. Grayscale stretching can expand the grayscale range of the acquired image and improve its contrast, making the image clearer and reducing environmental interference during unmanned surface vessel navigation. To reduce the decrease in image resolution caused by lighting interference, histogram equalization is used to improve detection accuracy.

[0086] The formula for converting grayscale values ​​is:

[0087] Y = 0.229 * R + 0.587 * G + 0.114 * B

[0088] During the image acquisition process by the camera, after the image is preprocessed, there are still parts of the image that are affected by noise. This causes the uniform gray values ​​to change abruptly at a certain point. Smoothing the image can eliminate noise in the image while preserving the details of the image information as much as possible, thereby enhancing the image effect. Median filtering, mean filtering and Gaussian filtering are used to achieve this respectively.

[0089] The median filter transformation formula is:

[0090] Q(i,j)=Median{P(im,jn),(m,n)}

[0091] In the formula, P(i,j) is the original image; Q(i,j) is the processed image; and (m,n) is the neighborhood.

[0092] Step S2.3: In complex water navigation environments, the navigation of large ships and obstacles such as bridge piers can affect the correct detection and identification of locks. Effective image segmentation to independently divide the target area and non-target area of ​​the lock is essential. The K-means segmentation algorithm is used to segment the acquired obstacle images.

[0093] In step S2.3, the K-means segmentation algorithm segments the points in the feature space according to the clustering method, and then maps them back to the original image to obtain the segmentation result. The sum of squared errors is used as the clustering criterion function, and the Euclidean distance is measured to calculate the similarity, which ensures the highest intra-class similarity and the lowest inter-class similarity, thus achieving local optima.

[0094] First, K initial class means need to be selected in the given space. Then, each pixel is assigned to its own class. The process ends when the difference between the obtained class means and the initial class means is less than a set threshold. It can segment the part affected by lighting into the background image and can also effectively preserve the color, texture and shape features of the target image.

[0095] The SURF feature matching algorithm was used to perform stereo matching on the image, and the RANSAC algorithm was used to remove erroneous matching points.

[0096] The SURF feature described here is a local feature of an image that remains invariant when the target image undergoes rotation, scaling, or brightness changes, and also maintains a certain degree of stability against viewpoint changes, affine transformations, and noise. The SURF feature extraction algorithm mainly consists of three parts: feature point detection, feature point description, and feature point matching. Feature point detection employs a Hessian matrix-based detector, which outperforms Harris-based detectors in terms of stability and repeatability. Feature point description uses Haar wavelets as feature descriptors, as Haar features are characterized by their speed, reducing computation time and increasing robustness. A box filter approximates the second-order Gaussian filter, and integral images are used to accelerate convolution, reducing computational complexity and improving computational speed.

[0097] Example 3:

[0098] The specific method of the unmanned vessel lock passage environment perception method based on the fusion of lidar and visual sensors in S3 is as follows: spatial fusion is completed through coordinate system transformation. The camera calibration adopts Zhang Zhengyou's monocular camera calibration method. The corresponding region of interest is constructed based on the obstacle orientation information detected by lidar and matched with the minimum bounding rectangle of the obstacle contour obtained by the visual algorithm to determine the position and coverage of the effective target.

[0099] The specific steps are as follows:

[0100] Step S3.1: Obtain the relative position information of the lidar and the vision sensor. Both need to be calibrated. First, based on the relative position relationship between the two and the corresponding coordinate system is established, the radar point cloud data and image information are converted to the actual imaging pixel coordinate system to complete the coordinate transformation.

[0101] Step S3.2: By fusing the corresponding coordinate systems, the LiDAR and visual sensor are unified to the same dimension. Based on this, the region of interest is first drawn with the obstacle position in the environment obtained by the LiDAR as the center and a certain scale as the radius. Then, the minimum bounding rectangle of the lock outline obtained by the visual system is found. The corresponding region of interest is constructed based on the obstacle orientation information detected by the LiDAR. The specific position of the minimum bounding rectangle of the obstacle outline obtained by the visual algorithm can be determined by positioning and coordinate transformation of the unmanned surface vessel. By comparing the position information of the two parts to see if they overlap, it can be determined whether the target is the detection target.

Claims

1. A method for identifying unmanned vessels passing through locks based on multiple sensors, comprising the following steps: S1. A method for acquiring obstacles in front of an unmanned surface vessel based on lidar; S2. A method for obtaining obstacles in front of an unmanned surface vessel based on visual sensors; S3. An environmental perception method for unmanned vessels passing through locks based on the fusion of lidar and visual sensors; The specific detection method of the obstacle acquisition method for the unmanned vessel based on lidar in S1 is as follows: S1.1: Acquire and detect the distance and angle information between the target lock and the lidar point cloud data, and perform point cloud preprocessing on the obtained information: First, the point cloud coordinates are converted into two-dimensional coordinates. Then, filtering is performed to remove noise generated by clutter in the collected point cloud. Statistical filtering and pass-through filtering are used for processing. Statistical filtering removes sparse outlier noise points. During data processing, the original point cloud from the lidar must first be converted from spherical coordinates ( ρ,α,ω Convert to Cartesian coordinates in the unmanned surface vessel coordinate system. x,y,z ): S1.2: Since the target lock is mostly made of low-carbon alloy steel plate, more laser point cloud data is collected than other obstacles. The lock and non-target obstacles are screened by two steps: reflection intensity value screening and point cloud density screening. S1.3: After steps S1.1 and S1.2, the amount of point cloud data will be greatly reduced, leaving only the useful point cloud data that needs to be processed, as well as some data that cannot be filtered out by reflection intensity and point cloud density values; the processed data will be used to perform plane extraction by continuously iterating through the RANSAC algorithm. The RANSAC algorithm is an iterative method used to estimate the parameters of a mathematical model from a set of observation data containing outliers, in which case the outliers will not affect the estimated values; The specific implementation process of the RANSAC algorithm in S1.3 is as follows: based on the adaptive formula, the optimal solution for the number of iterations is calculated, and all planes in the dataset that meet the requirements can be extracted. The formula is as follows: make p The RANSAC algorithm provides the expected probability of at least one useful result after running. make w Points that satisfy the model / all detected points; make n The number of points that satisfy the current model; but ; Right now ; Based on the adaptive formula, the optimal solution for the number of iterations can be calculated, and all planes in the dataset that meet the requirements can be extracted. The extracted planes are marked as follows: P i , i =1,2,3…n, due to the limited performance of lidar in measuring long-range targets, a pass-through filter is used to remove cluster planes with an elevation Z-axis below 2m. Then, in each remaining plane, a search is conducted to find the left and right endpoints perpendicular to the ground. P i（left） , P i（right） Check the two endpoints of each plane individually; if they do not coincide, continue the search. P i The sum of the two endpoints of the line P i+1 One of the two endpoints of the straight line coincides; use PCA principal component analysis to extract... P i（left） , P i（right） The normal vectors of two planes are transformed into the angle between the planes. θ If the angle between the planes θ If it floats within 140°±10°, it is identified as a lock.

2. The method for identifying unmanned vessels passing through locks based on multiple sensors according to claim 1, characterized in that, The specific method for selecting the reflection intensity value in S1.2 is as follows: Because water waves generate laser echoes, but their reflection intensity is usually low, after multiple data analyses, due to the special reasons of reflection from the water surface or water-borne obstructions, data with excessively low intensity values ​​are not the targets to be detected. Point cloud data with excessively low intensity values ​​reflect water surface points. The intensity values ​​are normalized to a range of 0-100. Therefore, based on the characteristics of water surface reflection values, point cloud data with intensity values ​​of 70-100 are retained as data for further processing.

3. The method for identifying unmanned vessels passing through locks based on multiple sensors according to claim 1, characterized in that, The specific method for point cloud density filtering in S1.2 is as follows: Because the lidar data acquisition range is large and contains many invalid areas, a pass-through filter is used to subtract the minimum and maximum values ​​of the X and Y coordinates respectively to facilitate subsequent calculations and processing. This ensures that the center point of the planar coordinates lies on the point cloud data, and the coordinates represent altitude. Z The direction does not need to be changed; first, the 3D point cloud is converted into a 2D raster map representation based on the water surface, thus representing the point cloud data as multiple raster regions with unconstrained heights. The number of point clouds in the raster region represented by each small square is counted. Because the point cloud reflected by the water surface waves is discrete, and the lock has a high reflectivity, the overall number of point clouds obtained is also large. If the point cloud data in each small square is less than a certain value, it is not the target to be detected.

4. The method for identifying unmanned vessels passing through locks based on multiple sensors according to claim 1, characterized in that, The specific method for obtaining obstacles in front of the unmanned vessel based on visual sensors in S2 is as follows: Step S2.1: Establish the coordinate system of the vision sensor and transform it with the world coordinate system. To obtain the target information in the three-dimensional environment, we must first find the position of this two-dimensional plane, that is, the imaging surface of the camera. Then, based on the principle of ranging, the three-dimensional coordinates corresponding to the two-dimensional points on the image are calculated, and the camera is calibrated using Zhang Zhengyou's planar calibration method. Step S2.2: Due to the complex environment of the unmanned vessel navigating on the water surface to detect the lock, the fluctuation of the water waves causes the unmanned vessel to be unstable, resulting in some interference information during the image acquisition process of the camera. This part of the information will cover the original image data, resulting in image data loss. In order to reduce the interference of the external environment on the acquired image, image preprocessing is required to simplify the data and enhance the detectability of the target image. Image preprocessing methods include image enhancement and image smoothing. Step S2.3: In complex water navigation environments, the navigation of large ships and obstacles such as bridge piers can affect the correct detection and identification of locks. Effective image segmentation to independently divide the target area and non-target area of ​​the lock is essential. The K-means segmentation algorithm is used to segment the acquired obstacle images.

5. The method for identifying unmanned vessels passing through locks based on multiple sensors according to claim 4, characterized in that, The image preprocessing method in step S2.2 specifically operates as follows: First, the image is enhanced through grayscale conversion, grayscale stretching, and histogram equalization. During grayscale conversion, since the human eye perceives red, green, and blue light differently, each color is assigned a different weight to obtain a grayscale value. Grayscale stretching can expand the grayscale range of the acquired image and improve its contrast, making the image clearer and reducing environmental interference during unmanned surface vessel navigation. To reduce the decrease in image resolution caused by lighting interference, histogram equalization is used to improve detection accuracy. The formula for converting grayscale values ​​is: Y =0.229* R +0.587* G +0.114* B During the image acquisition process by the camera, after the image is preprocessed, there are still parts of the image that are affected by noise. This causes the uniform gray values ​​to change abruptly at a certain point. Smoothing the image can eliminate noise in the image while preserving the details of the image information as much as possible, thereby enhancing the image effect. Median filtering, mean filtering and Gaussian filtering are used to achieve this respectively. The median filter transformation formula is: Q ( i,j )=Median{ P ( im , jn ),( m , n )} In the formula, P( i,j ) represents the original image; Q( i,j (This is the processed image.) ( m,n () refers to the scope of the field.

6. The method for identifying unmanned vessels passing through locks based on multiple sensors according to claim 4, characterized in that, In step S2.3, the K-means segmentation algorithm segments the points in the feature space according to the clustering method, and then maps them back to the original image to obtain the segmentation result. The sum of squared errors is used as the clustering criterion function, and the Euclidean distance is measured to calculate the similarity, which ensures the highest intra-class similarity and the lowest inter-class similarity, thus achieving local optima. First, K initial class means need to be selected in the given space. Then, each pixel is assigned to its own class. The process ends when the difference between the obtained class means and the initial class means is less than the set threshold. It can segment the part affected by lighting into the background image and can also effectively preserve the color, texture and shape features of the target image. The SURF feature matching algorithm was used to perform stereo matching on the image, and the RANSAC algorithm was used to remove erroneous matching points.

7. The method for identifying unmanned vessels passing through locks based on multiple sensors according to claim 1, characterized in that, The specific method of the unmanned vessel lock passage environment perception method based on the fusion of lidar and visual sensors in S3 is as follows: Spatial fusion is achieved through coordinate system transformation. The camera calibration adopts Zhang Zhengyou's monocular camera calibration method. Based on the obstacle orientation information detected by the lidar, the corresponding region of interest is constructed and matched with the minimum bounding rectangle of the obstacle contour obtained by the visual algorithm to determine the position and coverage of the effective target.

8. The method for identifying unmanned vessels passing through locks based on multiple sensors according to claim 7, characterized in that, The specific operation in S3 is as follows: Step S3.1: Obtain the relative position information of the lidar and the vision sensor. Both need to be calibrated. First, based on the relative position relationship between the two and the corresponding coordinate system is established, the radar point cloud data and image information are converted to the actual imaging pixel coordinate system to complete the coordinate transformation. Step S3.2: By fusing the corresponding coordinate systems, the LiDAR and visual sensor are unified to the same dimension. Based on this, the region of interest is first drawn with the obstacle position in the environment obtained by the LiDAR as the center and a certain scale as the radius. Then, the minimum bounding rectangle of the lock outline obtained by the visual system is found. The corresponding region of interest is constructed based on the obstacle orientation information detected by the LiDAR. The specific position of the minimum bounding rectangle of the obstacle outline obtained by the visual algorithm can be determined by positioning and coordinate transformation of the unmanned surface vessel. By comparing the position information of the two parts to see if they overlap, it can be determined whether the target is the detection target.

Images