Ship type identification method based on laser radar

Abstract

The invention relates to the technical field of ship identification, in particular to a laser radar-based ship type identification method, which comprises the following steps of: scanning a channel scene by using a laser radar, and collecting channel scene point cloud data; after shore side and water surface background points in the navigation channel scene point cloud data are filtered out, an enhanced DBSCAN clustering algorithm is utilized to carry out target detection, an initial point cloud data matrix, an initial reflected light intensity feature matrix and a laser beam ID information matrix of a single ship are extracted, a classification model which is constructed based on a PointNet model and is suitable for ship point cloud in a complex navigation channel scene is input, and a ship point cloud classification model is constructed based on the PointNet model. Identifying the type of a single ship; wherein synchronous image data shot by a camera installed beside a laser radar is combined, ship type information is marked, a model is trained through a marked point cloud training set, and the trained model is used for carrying out type identification on ship types in a navigation channel. The ship type identification precision is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of ship identification, and in particular to a method for identifying ship types based on lidar. Background Art

[0002] In waterway management work, the identification of ship types is an important part of shipping detection. And accurate ship type identification also helps to count the shipping traffic and trade data, providing a reference basis for the management decision-making of the shipping economy.

[0003] In existing inland waterway monitoring projects, manual statistics and video monitoring are mainly relied on. The method of manual statistics has a low degree of automation, high labor costs, and low accuracy and reliability. The method of video monitoring has a high degree of dependence on the environment and is easily interfered by environmental light. Direct sunlight during the day will cause the picture to be overexposed and not clear. At night, it is affected by the street lights of the shore-side buildings and the signal lights and lighting lights on passing ships, resulting in glare, which in turn affects the picture and causes inaccurate detection of the target ship.

[0004] In order to solve the above problems, the patent document with application number 202311444846.X uses laser radar to obtain point cloud data of the target ship, and adopts the Pointnet++ model as a classification model to identify the type of the target ship; the existing Pointnet++ model and Pointnet model are feature extraction networks designed for the characteristics of point cloud data, so that the classification model based on the Pointnet++ model and the Pointnet model can perform type recognition according to the acquired point cloud data. However, the Pointnet model processes the point cloud data through a multi-layer perceptron. Although it can extract global features, it does not pay enough attention to local subtle features and it is difficult to accurately classify based on local slight differences; although the Pointnet++ model improves local feature extraction, when it only relies on the characteristics of the point cloud itself, it may still miss key details in the face of complex local structures, resulting in inaccurate classification. Therefore, if the above two models are applied to the ship type recognition scenario, the above two models only consider the three-dimensional geometric data of the ship point cloud, and do not consider the beam characteristics and reflected light intensity characteristics generated when the lidar is working. As a result, in complex waterway scenarios, the model cannot adapt to point cloud data of different ship types and under different environmental conditions, and has poor generalization ability, so that the ability to handle complex situations is limited, which in turn affects the accuracy of ship type recognition; at the same time, the reflected light intensity characteristics calculated by lidar echo data modeling are not only related to the geometric distance between the ship to be identified and the lidar, but also affected by the incident angle and atmospheric attenuation coefficient. Directly using the reflected light intensity data calculated in this way may misjudge objects with similar reflected light intensities but actually different as the same type, or misjudge the same object as different objects at different angles and different atmospheric conditions, thereby reducing the accuracy of target classification and recognition.

[0005] In addition, the patent document uses laser radars installed on the left and right sides of the channel to collect point cloud data. The equipment cost is high, and when multiple ships are parallel in the channel, the ship close to the laser radar installation side will block the ship far away from the laser radar installation side, which may easily cause the point cloud data of the ship far away from the laser radar installation side to be lost; at the same time, the height of the ship when sailing in the inland waterway depends on whether the cargo hold of the ship is loaded with cargo. When sailing with a full hold, the height of the ship above the water is usually about 1 meter. The use of shore-side laser radar can obtain the deck characteristics of the ship. However, when the ship is unloaded, the height of the ship far exceeds the scanning area of ​​the lidar installed on the shore, resulting in the missing point cloud data of the ship's deck features and cargo hold features. In order to solve the above-mentioned data missing problem, the patent document completes the missing point cloud data of the ship through calibration and leveling, splicing, data enhancement and point cloud completion algorithms, which increases computing power consumption and low processing efficiency. At the same time, the impact of background points on the ship's point cloud data is not considered, resulting in low accuracy of the point cloud data, thereby reducing the accuracy of ship type recognition. Summary of the invention

[0006] To this end, the technical problem to be solved by the present invention is to overcome the problem that the classification model in the prior art does not take into account the characteristics of the line beam and the intensity characteristics of the reflected light, and the efficiency and accuracy of acquiring ship point cloud data are not high, resulting in inaccurate identification of classification results.

[0007] In order to solve the above technical problems, the present invention provides a ship type identification method based on laser radar, comprising:

[0008] Obtaining the initial point cloud data matrix, initial reflected light intensity feature matrix and laser beam ID information matrix of the ship to be identified collected by the laser radar;

[0009] The initial point cloud data matrix of the ship to be identified is input into the spatial transformation network of the classification model, and after the rotation matrix is ​​output, it is multiplied by the initial point cloud data matrix of the ship to be identified to obtain the target point cloud data matrix of the ship to be identified;

[0010] The initial point cloud data matrix, the initial reflected light intensity feature matrix and the laser beam ID information matrix of the ship to be identified are spliced ​​and input into the correction network of the classification model. After the correction matrix is ​​output through a plurality of one-dimensional convolution modules, a maximum pooling module, a single one-dimensional convolution module and a multi-layer perceptron module connected in sequence, the correction matrix is ​​multiplied with the initial reflected light intensity feature matrix of the ship to be identified to obtain the target reflected light intensity feature matrix of the ship to be identified;

[0011] The target point cloud data matrix of the ship to be identified, the target reflected light intensity feature matrix and the laser beam ID information matrix are spliced, input into the classification network of the classification model, and the category result of the ship to be identified is output.

[0012] Preferably, the lidar is horizontally installed at the center of the bridge spanning the waterway, and the installation position of the lidar is higher than the height of all ships sailing in the waterway.

[0013] Preferably, the obtaining of the initial point cloud data matrix, the initial reflected light intensity feature matrix, and the laser beam ID information matrix of the ship to be recognized collected by the lidar includes:

[0014] Using the lidar, scan the waterway scene where the ship to be recognized is located within the range of the lidar, and collect the original point cloud data matrix of the current waterway scene and its corresponding laser beam ID information matrix; according to the mathematical model of the lidar echo data, combined with the echo data matrix of the current waterway scene collected by the lidar, calculate the reflected light intensity feature data matrix corresponding to the original point cloud data matrix in the current waterway scene;

[0015] Using the density clustering algorithm, process the original point cloud data matrix in the current waterway scene to obtain the initial point cloud data matrix of the ship to be recognized, including:

[0016] Extract the data of each point cloud data in the XY plane of the original point cloud data matrix of the current waterway scene to obtain the two-dimensional data matrix of the bird's-eye view of the current waterway scene;

[0017] Add a scaling factor, and perform a stretching transformation on each two-dimensional point cloud data in the Y direction of the two-dimensional data matrix of the bird's-eye view of the current waterway scene to obtain the target point cloud data matrix of the current waterway scene;

[0018] Perform clustering calculation on the target point cloud data matrix of the current waterway scene to detect all ships in the current waterway, and extract the initial point cloud data matrix of the ship to be recognized;

[0019] Based on the initial point cloud data matrix of the ship to be recognized, obtain its corresponding initial reflected light intensity feature data matrix and laser beam ID information matrix.

[0020] Preferably, the expression of the distance function in the density clustering algorithm is:

[0021] ;

[0022] where represents the distance function between the th point cloud and the th point cloud; represents the value of the th point cloud in the X direction; represents the value of the th point cloud in the Y direction; represents the value of the th point cloud in the X direction; represents the value of the nth point cloud in the Y direction; represents the scaling factor.

[0023] Preferably, before extracting the data in the XY plane of each point cloud data from the original point cloud data matrix of the current waterway scene to obtain the two-dimensional data matrix of the bird's-eye view of the current waterway scene, it further includes:

[0024] Filter out invalid points and background points from the original point cloud data matrix of the current waterway scene according to the characteristics of the original point cloud data matrix of the current waterway scene to obtain the processed point cloud data matrix of the current waterway scene;

[0025] Directly delete the invalid points with geometric coordinates and reflected light intensity of 0 from the original point cloud data matrix of the current waterway scene;

[0026] The background points include shore background points, bridge background points and water surface background points;

[0027] Use the method of selecting the range of interest to filter out shore background points and bridge background points from the original point cloud data matrix of the current waterway scene;

[0028] Use the method of diffuse reflection light intensity to filter out water surface background points and ship wake background points from the original point cloud data matrix of the current waterway scene.

[0029] Preferably, multiply the rotation matrix by the initial point cloud data matrix of the ship to be recognized to obtain the target point cloud data matrix of the ship to be recognized, and its expression is:

[0030] ;

[0031] where represents the target point cloud data matrix of the ship to be recognized; represents the initial point cloud data matrix of the ship to be recognized; represents the rotation matrix; represents the number of point cloud data of the ship to be recognized.

[0032] Preferably, multiply the correction matrix by the initial reflected light intensity feature matrix of the ship to be recognized to obtain the target reflected light intensity feature matrix of the ship to be recognized, and its expression is:

[0033] ;

[0034] where represents the target reflected light intensity feature matrix of the ship to be recognized; represents the initial reflected light intensity feature matrix of the ship to be recognized; represents the correction matrix; Represents the number of point cloud data of the ship to be recognized.

[0035] Preferably, the lidar is a mechanical 32-line lidar.

[0036] Preferably, it further includes:

[0037] Obtain the initial point cloud data matrix, initial reflected light intensity feature matrix, and laser beam ID information matrix of different ships in different channel scenes collected by the lidar;

[0038] Through the image data that is spatio-temporally synchronized with the original point cloud data matrix of different channel scenes captured by the camera, label the type information of different ships in different channel scenes to obtain the labels of different ships in different channel scenes; wherein, the camera is installed at the lidar deployment position, and the camera is installed with a downward tilt.

[0039] Based on the initial point cloud data matrix, initial reflected light intensity feature matrix, and laser beam ID information matrix of different ships in different channel scenes, as well as the labels of different ships in different channel scenes, construct a data set.

[0040] Divide the data set into a training set and a test set.

[0041] Use the training set to train the classification model to obtain a trained classification model.

[0042] Preferably, the classification network includes: a first one-dimensional convolutional layer, a transformation network, a second one-dimensional convolutional layer, a third one-dimensional convolutional layer, a max pooling layer, a fourth one-dimensional convolutional layer, and a multi-layer perceptron layer connected in sequence;

[0043] Using the first one-dimensional convolutional layer, the transformation network, the second one-dimensional convolutional layer, and the third one-dimensional convolutional layer, the input matrix is Lifted from the dimension to Dimension; wherein, the input matrix represents the matrix after splicing the target point cloud data matrix, target reflected light intensity feature matrix, and laser beam ID information matrix of the ship to be recognized.

[0044] Using the max pooling layer and the fourth one-dimensional convolutional layer, perform pooling operations and convolutional operations on the Matrix features of the dimension to obtain Point cloud feature vector;

[0045] Using the multi-layer perceptron layer, according to the Point cloud feature vector of, obtain the category result of the ship to be recognized.

[0046] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0047] (1) A method for identifying ship types based on lidar enhances the number of input channels of the Pointnet classification model from three channels that only input point cloud data to five channels that input point cloud data, reflected light intensity features, and beam features. By adding reflected light intensity data and beam data as inputs, the Pointnet classification model can jointly extract scene features from geometric data, reflected light intensity data, and beam data during inference. Among them, the beam feature can enhance the geometric features of the ship to be identified in the dimension of the spherical coordinate system. It includes the beam number emitted by the lidar, and the point cloud data obtained by beams at different positions can reflect information on different parts of the ship. The superstructure and hull of the ship are scanned and obtained by different beams. Through the beam feature, the spatial distribution relationship of these parts can be understood, providing more details for identifying ship types; the reflected light intensity can reflect non-collective features such as the surface material characteristics of different types of ships. Moreover, the surface materials and structures of different types of ships are different, and there are also differences in the reflection characteristics of lasers. Using the reflected light intensity feature as an important basis for identifying ship types helps the classification model better distinguish different types of ships.

[0048] (2) For the ship type recognition method based on lidar according to the present invention, considering that the initial reflected light intensity feature is affected not only by the distance, but also by environmental factors such as the air refractive index and atmospheric attenuation coefficient brought by the weather, and the initial reflected light intensity data may be interfered by factors such as noise and ambient light, an Intensity R-Net correction network is introduced into the classification model; the correction network is successively connected with multiple one-dimensional convolutional modules, a max pooling module, a single one-dimensional convolutional module and a multi-layer perceptron module; the multiple one-dimensional convolutional modules are used to initially extract local features in the point cloud data, reflected light intensity features and beam features; the max pooling module is used to extract the global features of the data, which can highlight the key features while reducing the data dimension; the single one-dimensional convolutional module further refines the features of the pooled data, integrates the previously extracted features, and generates a more representative feature representation; the multi-layer perceptron module is used to learn the complex non-linear relationships between different features, fuse and transform the features after convolution and pooling processing, so that the correction matrix can more comprehensively and accurately reflect the true features of the ship, reduce the influence of noise and interference, and provide an effective basis for accurately correcting the reflected light intensity subsequently; through the above-mentioned modules, the correction network learns the point cloud data features, beam features and original reflected light intensity feature patterns of the ship to be recognized, outputs the correction matrix, and then multiplies it by the initial reflected light intensity data to remove noise and interference, and obtains the target reflected light intensity feature determined by the ship's own material and structure, realizing the enhancement of the reflected light intensity data, and further enabling the model to accurately distinguish ship types based on the enhanced reflected light intensity features, improving the accuracy of ship type recognition; in addition, in the complex waterway scenario, the attitude, position of the ship and the scanning angle of the lidar are constantly changing, resulting in great uncertainty in the point cloud data and reflected light intensity features; the correction network can adapt to the changes in the point cloud data, reflected light intensity and beam features, extract stable features related to the ship type from the complex data, generate an adaptive correction matrix, and adjust the reflected light intensity features to ensure that the corrected reflected light intensity can stably reflect the ship features, enhancing the adaptability and accuracy of the model in the complex waterway.

[0049] (3) For a method for identifying ship types based on lidar according to the present invention, by installing a lidar in the middle of a bridge spanning a waterway, the scanning area can cover the entire area across the waterway, avoiding the problem of ship point cloud occlusion caused by installing a lidar on the side of the waterway. Moreover, the collected point clouds are all actual ship point cloud data, not point clouds completed by algorithms, which can reduce the point cloud data error caused by the completion algorithm, improve the comprehensiveness of ship point cloud data acquisition, and provide a data basis for subsequent ship type identification. By installing a lidar in the middle of a bridge spanning a waterway, the point cloud data inside the ship's cargo hold can be accurately scanned. At the same time, the point cloud data of the deck and cargo hold are incorporated into the classification model, increasing the dimension and richness of the data. Combining information such as the reflected light intensity and laser beam ID obtained by the lidar, the ship characteristics can be described from multiple angles, thereby improving the accuracy of ship type classification.

[0050] (4) For a method for identifying ship types based on lidar according to the present invention, before using the density clustering algorithm to obtain the point cloud data of the ship to be identified, first, each point cloud data in the original point cloud data matrix of the waterway scene is visualized from a bird's-eye view, omitting the height information of the original point cloud data, and then a scaling factor is added to perform a stretching transformation on the bird's-eye view point cloud data in the Y direction to ensure that the point cloud density in the X and Y directions of the ship is approximately the same, reducing the influence of factors such as ship attitude and driving direction changes on data characteristics. This solves the problem that the point cloud of a ship shows a narrow and dense state in the left-right direction and a sparse state in the front-back direction. When there is no point cloud scanning or occlusion in the cargo hold, the point cloud of a single ship will be split into multiple parts. If the method of clustering relying on the spatial Euclidean distance in the density clustering algorithm (DBSCAN) is directly used, a single ship may be identified as two ships, resulting in a decrease in the accuracy of target detection. Furthermore, the stability of data characteristics is enhanced to improve the accuracy of clustering, which is more conducive to the subsequent classification model learning the characteristics of the ship. Brief Description of the Drawings

[0051] To make the content of the present invention easier to understand clearly, the following further details the present invention according to specific embodiments of the present invention and in combination with the accompanying drawings, where:

[0052] Figure 1 is a flowchart of a method for identifying ship types based on lidar provided by the present invention;

[0053] Figure 2 is a bird's-eye view of the original point cloud of the waterway scanned by the lidar;

[0054] Figure 3 is a schematic structural diagram of the classification model;

[0055] Figure 4Schematic diagram of an image and a point cloud sample of a dry bulk barge; among them, figures (a), (c), (e), (d), (g), (i), and (k) represent synchronous images of the dry bulk barge taken by a camera at different times; figures (b), (d), (f), (h), (j), and (l) represent point cloud data of the dry bulk barge collected by a lidar at different times;

[0056] Figure 5 Schematic diagram of an image and a point cloud sample of a liquid transport ship; among them, figures (a), (c), (e), and (g) represent synchronous images of the liquid transport ship taken by a camera at different times; figures (b), (d), (f), and (h) represent point cloud data of the liquid transport ship collected by a lidar at different times;

[0057] Figure 6 Schematic diagram of an image and a point cloud sample of a workboat; among them, figures (a), (c), (e), and (g) represent synchronous images of the workboat taken by a camera at different times; figures (b), (d), (f), and (h) represent point cloud data of the workboat collected by a lidar at different times;

[0058] Figure 7 Schematic diagram of lidar installation and detection range. Detailed implementation manners

[0059] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, so that those skilled in the art can better understand the present invention and be able to implement it, but the embodiments cited are not intended to limit the present invention.

[0060] Currently, in the field of waterway monitoring, traditional methods such as manual statistics, video surveillance, infrared sensing, AIS, etc. have prominent problems such as high cost, low efficiency, great influence by weather, lack of distance information, and large statistical errors. The present invention uses lidar technology to replace traditional methods to achieve monitoring, overcoming the defects of traditional methods; at the same time, for the application scenario of multi-ship monitoring in large-flow waterways in actual work, accurate and fast ship detection, identification, and calculation methods are realized, providing a solid foundation for the analysis of subsequent shipping traffic data.

[0061] Refer to Figure 1 as shown in Figure 1 is a flowchart of a ship type recognition method based on lidar provided by the present invention; specifically including:

[0062] S1: Use a lidar to scan the channel scene where the ship to be recognized is located within the range of the lidar, and collect the original point cloud data matrix of the current channel scene and its corresponding laser beam ID information matrix; according to the mathematical model of the lidar echo data, combined with the echo data matrix of the current channel scene collected by the lidar, calculate the reflected light intensity characteristic data matrix corresponding to the original point cloud data matrix in the current channel scene; where, the lidar is horizontally installed at the center of the bridge spanning the channel, and the installation position of the lidar is higher than the height of all ships traveling in the channel; the lidar is a mechanical 32-line lidar; the bird's-eye view of the lidar scanning the original point cloud of the channel is as Figure 2 shown;

[0063] Among them, the reflected light intensity is the intensity of the echo laser beam received by the lidar after the laser beam is emitted to the target and undergoes diffuse reflection and specular reflection on the target surface; the mathematical model of the lidar echo data is:

[0064] ;

[0065] Among them, represents time; represents the return reception power of the laser beam, that is, the reflected light intensity characteristic data; represents the emission power of the laser beam; represents the aperture diameter of the optical system; represents the system constant; is the surface reflectivity; represents the angle of incidence between the light beam and the surface; represents the distance between the lidar and the th obstacle; is the atmospheric attenuation coefficient;

[0066] Among them, the lidar beam information comes from the ID of the laser transceiver pair installed inside the lidar, which can reflect the geometric information of the target to a certain extent; 32 pairs of laser transceivers are installed inside the lidar, and the laser transceivers are vertically distributed in the lidar, and their distribution spacing is 2° in +15°~+7° and -8°~+10°, 1.5° in +7°~+4°, and 1.33° in +4°~-8°; when the lidar is working, each point in the point cloud data will record the ID of the laser transceiver it belongs to;

[0067] S2: Filter out invalid points and background points from the original point cloud data matrix of the current channel scene according to the characteristics of the original point cloud data matrix of the current channel scene, and obtain the processed point cloud data matrix of the current channel scene;

[0068] Directly delete the invalid points with geometric coordinates and reflected light intensity of 0 from the original point cloud data matrix of the current waterway scene;

[0069] The background points include shore background points, bridge background points and water surface background points;

[0070] Use the method of interest range selection to filter out the shore background points and bridge background points from the original point cloud data matrix of the current waterway scene;

[0071] Use the diffuse reflection light intensity method to filter out the water surface background points and ship wake background points from the original point cloud data matrix of the current waterway scene;

[0072] In summary, the point cloud consists of invalid points, static shore background points, dynamic water surface background points and ship target points; among them, the invalid points are caused by the laser radar scanning the sky and part of the water surface, and the laser beam is absorbed so that no diffuse reflection echo is generated. In the point cloud data, the geometric coordinates and reflected light intensity of the invalid points are both represented as 0, and such invalid points are directly deleted; the static shore background points are composed of the point clouds of the buildings, vegetation and bridges on both sides scanned by the laser radar, and the position of the laser radar is fixed. The static shore background points are fixed on both sides of the waterway and at the bridges. Therefore, the method of interest range selection is used to filter out the shore building plants and bridge background points; the dynamic water surface background points are generated when the laser radar scans the water surface. Due to the wind and waves and the wake generated by the ship's movement, the water surface fluctuates, and then a small amount of reflected echo is generated and received by the laser radar receiver to generate point cloud data. Moreover, there is an obvious difference between the reflected light intensity of the ship point cloud and the reflected light intensity of the water surface background. The reflected light intensity of the water surface background is mainly concentrated in the lower reflected light intensity. Therefore, a filtering threshold is set for the reflected light intensity to achieve the filtering of the dynamic water surface background points;

[0073] S3: Use the density clustering algorithm to process the original point cloud data matrix in the current waterway scene to obtain the initial point cloud data matrix of the ship to be recognized, including:

[0074] Extract the data in the XY plane of each point cloud data in the original point cloud data matrix of the current waterway scene to obtain the two-dimensional data matrix of the bird's-eye view of the current waterway scene;

[0075] Add a scaling factor to perform a stretching transformation on each two-dimensional point cloud data in the Y direction of the two-dimensional data matrix of the bird's-eye view of the current waterway scene to obtain the target point cloud data matrix of the current waterway scene;

[0076] Perform clustering calculation on the target point cloud data matrix of the current waterway scene to detect all ships in the current waterway, and extract the initial point cloud data matrix of the ship to be recognized; among them, if the point cloud of a single ship detected by the target is sparse, the sparse point cloud data is upsampled until enough points of point cloud data are obtained;

[0077] Based on the initial point cloud data matrix of the ship to be recognized, obtain the corresponding initial reflected light intensity feature data matrix and laser beam ID information matrix;

[0078] Among them, the expression of the distance function in the density clustering algorithm:

[0079] ;

[0080] Among them, represents the th point cloud and the th point cloud; represents the value of the th point cloud in the X direction; represents the value of the th point cloud in the Y direction; represents the value of the th point cloud in the X direction; represents the value of the th point cloud in the Y direction; represents the scaling factor;

[0081] In summary, first, the points in the original point cloud are visualized from a bird's-eye view, omitting the height information in the original point cloud ; secondly, apply a stretching transformation in the direction to the bird's-eye view point cloud to ensure that the point cloud density of a single ship target is approximated in the , , direction; finally, use the DBSCAN algorithm to detect the ship target and extract the point cloud information of a single ship;

[0082] S4: Input the initial point cloud data matrix of the ship to be recognized into the spatial transformation network of the classification model. After outputting the rotation matrix, multiply it by the initial point cloud data matrix of the ship to be recognized to obtain the target point cloud data matrix of the ship to be recognized, and its expression is:

[0083] ;

[0084] Among them, represents the target point cloud data matrix of the ship to be recognized; represents the initial point cloud data matrix of the ship to be recognized; represents the rotation matrix; represents the number of point cloud data of the ship to be recognized;

[0085] The initial point cloud data matrix, initial reflected light intensity feature matrix, and laser beam ID information matrix of the ship to be recognized are concatenated and input into the correction network of the classification model. After passing through multiple one-dimensional convolutional modules, max pooling modules, a single one-dimensional convolutional module, and a multi-layer perceptron module connected in sequence, after the corrected matrix is output, it is multiplied by the initial reflected light intensity feature matrix of the ship to be recognized to obtain the target reflected light intensity feature matrix of the ship to be recognized. Its expression is:

[0086] ;

[0087] where, represents the target reflected light intensity feature matrix of the ship to be recognized; represents the initial reflected light intensity feature matrix of the ship to be recognized; represents the corrected matrix; represents the number of point cloud data of the ship to be recognized;

[0088] The target point cloud data matrix, target reflected light intensity feature matrix, and laser beam ID information matrix of the ship to be recognized are concatenated and input into the classification network of the classification model, and the class result of the ship to be recognized is output; wherein, the classification network includes: a first one-dimensional convolutional layer, a transformation network, a second one-dimensional convolutional layer, a third one-dimensional convolutional layer, a max pooling layer, a fourth one-dimensional convolutional layer, and a multi-layer perceptron layer connected in sequence;

[0089] Using the first one-dimensional convolutional layer, transformation network, second one-dimensional convolutional layer, and third one-dimensional convolutional layer, the input matrix is lifted from dimensions to dimensions; wherein, the input matrix represents the matrix after concatenating the target point cloud data matrix, target reflected light intensity feature matrix, and laser beam ID information matrix of the ship to be recognized;

[0090] Using the max pooling layer and the fourth one-dimensional convolutional layer, pooling operation and convolutional operation are performed on the matrix features of dimensions to obtain point cloud feature vectors;

[0091] Using the multi-layer perceptron layer, according to the point cloud feature vectors, the class result of the ship to be recognized is obtained;

[0092] wherein, the structure of the classification model is as shown in Figure 3 The classification model enhances the three channels of the original point cloud input to five channels, and the input features of a single point are expanded from only geometric features , to features composed of geometric features, reflected light intensity features, and laser beam ID information ; wherein, represents the reflected light intensity feature; Indicates the laser beam ID information;

[0093] Among them, the ship types are as Figure 4 , Figure 5 and Figure 6 shown;

[0094] In summary, based on the PointNet deep learning model, the input dimension of the classification model is expanded. The reflected light intensity and the wire harness information are directly input into the model together with the geometric information corrected by the T-Net for feature extraction to construct a classification model. This classification model first uses a T-Net structure to train a rotation matrix according to the initial point cloud data of the ship to be recognized, and applies this rotation matrix to the original point cloud to perform a rotation transformation on the initial point cloud data to obtain the target point cloud data. Secondly, an Intensity R-Net correction network is used to learn the features for correcting the reflected light intensity from the initial point cloud data and correct the reflected light intensity to obtain the target reflected light intensity. For the splicing matrix obtained by splicing the target point cloud data, the target reflected light intensity and the wire harness information, each point in the splicing matrix is then dimensionally elevated through multiple multi-layer perceptron layers, gradually elevating from the original five-dimensional point cloud to a higher data dimension ; Then, max pooling is performed on the data to obtain a point cloud feature vector. Finally, the feature vector of the entire point cloud feature is classified and detected using a multi-layer perceptron layer;

[0095] Among them, the distribution of the lidar wire harnesses is usually distributed at fixed angles, and the specific distribution angles are determined before the production and factory of the lidar. When the target enters the scanning range of the lidar, the corresponding points are generated relative to the spherical coordinate system of the lidar in, the distribution of the axis is derived from the characteristics of the lidar wire harness distribution. The lidar wire harness characteristics are the values in the spherical coordinate system relative to the three-dimensional coordinate system . When using the traditional Pointnet / Pointnet++ model to process point clouds, only the converted three-dimensional coordinate system

[0096] In the mathematical model of the lidar echo data, the aperture diameter of the optical system and the system constant are all derived from the internal parameters of the lidar and will not change due to the target and the environment; The lidar and the first Distance to an obstacle And the incident angle of the light beam with the surface , depending on the geometric position between the lidar and the target ship; The reflected light intensity of the point cloud is only related to the reflectivity of the target surface material . By extracting the features of the reflected light intensity of the point cloud, non-geometric features such as the surface material of the target ship can be shown.

[0097] Training the classification model includes:[[]]

[0098] Obtaining the initial point cloud data matrix, initial reflected light intensity feature matrix, and laser beam ID information matrix of different ships in different channel scenarios collected by the lidar;

[0099] Using the image data captured by the camera that is spatio-temporally synchronized with the original point cloud data matrix of different channel scenarios to label the type information of different ships in different channel scenarios, and obtaining the labels of different ships in different channel scenarios; Among them, the camera is installed at the lidar deployment position, and the camera is installed with a downward tilt;

[0100] Based on the initial point cloud data matrix, initial reflected light intensity feature matrix, and laser beam ID information matrix of different ships in different channel scenarios, as well as the labels of different ships in different channel scenarios, constructing a dataset;

[0101] Dividing the dataset into a training set and a test set;

[0102] Using the training set to train the classification model to obtain a trained classification model for automatically classifying ship point clouds to assist in ship detection work in complex channels.

[0103] Refer to Figure 7 As shown, install a sensor device in the middle of the bridge across the roadway. The sensor device includes a lidar and a camera; The lidar is installed horizontally to reduce the angular error generated with the horizontal plane, and the installation position is higher than the height of the ships traveling in the channel to ensure that the ships at accurate positions can be collected; The camera should be installed with a downward tilt to ensure that the entire lidar scanning and detection area can be covered; The sensor device scans and collects the point cloud information of the channel scenario by the lidar, and at the same time, the camera synchronously takes pictures, and the camera is used as an auxiliary sensing device for data annotation work; The detection target of the sensor device is the ships traveling in the channel on one side of the bridge, and its approximate effective detection area is a semi-circular area from a bird's-eye view.

[0104] In addition, a Linux industrial control computer is used to drive and control the lidar. After edge computing and preprocessing of the collected point cloud data, further transmission and storage are carried out. That is, the processes from S2 to S4 are all carried out in the industrial control machine.

[0105] In summary, the present invention can collect more complete ship point clouds, avoid the occlusion problems generated, can perform feature extraction and classification on the complete point clouds, eliminating the increased computational time consumption and increased errors brought by the point cloud completion algorithm; the present invention uses a multi-channel Pointnet classification model, which has a large improvement in accuracy, and on the premise of ensuring the classification accuracy, the efficiency of model operation and reasoning can be more than 90% faster than using the Pointnet++ model.

[0106] The advantages of the present invention also include the following points:

[0107] A lidar is a radar system that emits laser beams to detect the position, speed, three-dimensional information and other characteristic quantities of a target; among existing data acquisition and acquisition devices, compared with other devices (such as infrared, video, etc.), the lidar has prominent advantages such as a long detection range, strong penetrability (not affected by weather), high accuracy, and can monitor all-weather, and can directly obtain the three-dimensional structure information of the object to be measured.

[0108] The present invention installs a lidar in the middle of a bridge spanning a waterway, and the scanning area can cover the entire area across the waterway, solving the technical problem of ship point cloud occlusion caused by installing the lidar on the side of the waterway, optimizing the computing power consumption brought by the point cloud completion algorithm, and the point clouds collected by the present invention are real point cloud data, not point clouds completed by algorithms, which can reduce the errors brought by the completion algorithm; in addition, by installing a lidar in the middle of a bridge spanning a waterway, the point clouds inside the cargo hold can be accurately scanned, improving the accuracy of ship type classification, and further being able to determine the type of goods in the cargo hold, making up for the lack of point clouds of cargo in the cargo hold caused by the lidar installed on the shore side, thus being unable to scan the point clouds of the cargo hold.

[0109] The enhanced DBSCAN clustering algorithm proposed by the present invention is such that for point cloud traffic targets, they can be approximated as a cuboid, and multiple targets are distributed in the horizontal plane of XY, and there are few cases where multiple targets coincide in height. The height information of the point cloud used in the clustering algorithm is omitted, optimizing the operation time of the algorithm to a certain extent; secondly, a scaling ratio parameter for the length and width of the original point cloud model is introduced. The scaling ratio parameter will scale all the points in the point cloud data in length before the point cloud is clustered, ensuring that the point cloud cluster of the target ship is approximately a circle when performing clustering operations, so as to improve the clustering accuracy.

[0110] The present invention improves the Pointnet model by modifying the number of channels when inputting data, enabling the Pointnet model to jointly extract scene features from geometric data, reflected light intensity data, and wire harness data during inference. Considering that the reflected light intensity feature is affected by distance, an Intensity R-Net part is introduced into the model to enhance the reflected light intensity data.

[0111] Obviously, the above embodiments are merely examples for clear illustration and not limitations on the implementation. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to enumerate all implementation manners here. And the obvious changes or modifications derived therefrom are still within the protection scope of the present invention.

Claims

1. A method for identifying ship types based on lidar, characterized in that, Including: Obtaining an initial point cloud data matrix, an initial reflected light intensity feature matrix, and a laser beam ID information matrix of a ship to be recognized collected by a lidar; Inputting the initial point cloud data matrix of the ship to be recognized into the spatial transformation network of the classification model. After outputting a rotation matrix, multiplying it with the initial point cloud data matrix of the ship to be recognized to obtain a target point cloud data matrix of the ship to be recognized; Concatenating the initial point cloud data matrix, the initial reflected light intensity feature matrix, and the laser beam ID information matrix of the ship to be recognized, inputting them into the correction network of the classification model, passing through a plurality of one-dimensional convolution modules, max-pooling modules, a single one-dimensional convolution module, and a multi-layer perceptron module connected in sequence. After outputting a correction matrix, multiplying it with the initial reflected light intensity feature matrix of the ship to be recognized to obtain a target reflected light intensity feature matrix of the ship to be recognized; Concatenating the target point cloud data matrix, the target reflected light intensity feature matrix, and the laser beam ID information matrix of the ship to be recognized, inputting them into the classification network of the classification model, and outputting the classification result of the ship to be recognized.

2. The method for identifying ship types based on lidar according to claim 1, wherein, The lidar is horizontally installed at the center of a bridge spanning a waterway, and the installation position of the lidar is higher than the height of all ships sailing in the waterway.

3. The method for identifying ship types based on lidar according to claim 2, characterized in that The obtaining of the initial point cloud data matrix, the initial reflected light intensity feature matrix, and the laser beam ID information matrix of the ship to be recognized collected by the lidar includes: Using the lidar to scan the waterway scene where the ship to be recognized is located within the range of the lidar, collecting the original point cloud data matrix of the current waterway scene and its corresponding laser beam ID information matrix; According to the mathematical model of the lidar echo data, combining with the echo data matrix of the current waterway scene collected by the lidar, calculating the reflected light intensity feature data matrix corresponding to the original point cloud data matrix in the current waterway scene; Using the density clustering algorithm to process the original point cloud data matrix in the current waterway scene to obtain the initial point cloud data matrix of the ship to be recognized, including: Extracting the data of each point cloud data in the XY plane of the original point cloud data matrix of the current waterway scene to obtain a two-dimensional data matrix from the bird's-eye view of the current waterway scene; Adding a scaling factor to stretch and transform each two-dimensional point cloud data in the Y direction of the two-dimensional data matrix from the bird's-eye view of the current waterway scene to obtain a target point cloud data matrix of the current waterway scene; Performing clustering calculation on the target point cloud data matrix of the current waterway scene for target detection of all ships in the current waterway, and extracting the initial point cloud data matrix of the ship to be recognized; Based on the initial point cloud data matrix of the ship to be recognized, obtaining its corresponding initial reflected light intensity feature data matrix and laser beam ID information matrix.

4. The method for identifying ship types based on lidar according to claim 3, characterized in that, The expression of the distance function in the density clustering algorithm is: ； Among them, represents the distance function between the th point cloud and the th point cloud; represents the value of the th point cloud in the X direction; represents the value of the th point cloud in the Y direction; represents the value of the th point cloud in the X direction; represents the value of the th point cloud in the Y direction; represents the scaling factor.

5. A method for identifying ship types based on lidar according to claim 3, characterized in that, Before extracting the data of each point cloud data in the XY plane of the original point cloud data matrix of the current waterway scene to obtain a two-dimensional data matrix from the bird's-eye view of the current waterway scene, it further includes: Filtering out invalid points and background points from the original point cloud data matrix of the current waterway scene according to the characteristics of the original point cloud data matrix of the current waterway scene to obtain a processed point cloud data matrix of the current waterway scene; Directly delete the invalid points with geometric coordinates and reflected light intensity of 0 from the original point cloud data matrix of the current channel scene; The background points include shore background points, bridge background points and water surface background points; Using the method of selecting the region of interest, filter out the shore background points and bridge background points from the original point cloud data matrix of the current channel scene; Using the diffuse reflection light intensity method, filter out the water surface background points and ship wake background points from the original point cloud data matrix of the current channel scene.

6. The method for identifying ship types based on lidar according to claim 1, characterized in that, Multiply the rotation matrix by the initial point cloud data matrix of the ship to be recognized to obtain the target point cloud data matrix of the ship to be recognized, and its expression is: ； Among them, represents the target point cloud data matrix of the ship to be recognized; represents the initial point cloud data matrix of the ship to be recognized; represents the rotation matrix; represents the number of point cloud data of the ship to be recognized.

7. The method for identifying ship types based on lidar according to claim 1, wherein, Multiply the correction matrix by the initial reflected light intensity feature matrix of the ship to be recognized to obtain the target reflected light intensity feature matrix of the ship to be recognized, and its expression is: ； Among them, represents the target reflected light intensity feature matrix of the ship to be recognized; represents the initial reflected light intensity feature matrix of the ship to be recognized; represents the correction matrix; represents the number of point cloud data of the ship to be recognized.

8. A method for identifying ship types based on lidar according to claim 1, characterized in that, The lidar is a mechanical 32-line lidar.

9. The method for identifying ship types based on lidar according to claim 1, wherein, It also includes: Obtain the initial point cloud data matrix, initial reflected light intensity feature matrix and laser beam ID information matrix of different ships in different channel scenes collected by the lidar; Label the type information of different ships in different channel scenes through the image data that is spatio-temporally synchronized with the original point cloud data matrix of different channel scenes captured by the camera, and obtain the labels of different ships in different channel scenes; among them, the camera is installed at the lidar deployment position, and the camera is installed with a downward tilt; Based on the initial point cloud data matrix, initial reflected light intensity feature matrix and laser beam ID information matrix of different ships in different channel scenes, as well as the labels of different ships in different channel scenes, construct a data set; Divide the data set into a training set and a test set; Use the training set to train the classification model to obtain a trained classification model.

10. The method for identifying ship types based on lidar according to claim 1, characterized in that, The classification network includes: a first one-dimensional convolutional layer, a transformation network, a second one-dimensional convolutional layer, a third one-dimensional convolutional layer, a max pooling layer, a fourth one-dimensional convolutional layer and a multi-layer perceptron layer connected in sequence; Using the first one-dimensional convolutional layer, transformation network, second one-dimensional convolutional layer, and third one-dimensional convolutional layer, the input matrix is lifted from dimensions to dimensions; where the input matrix represents the matrix obtained by splicing the target point cloud data matrix, target reflected light intensity feature matrix, and laser beam ID information matrix of the ship to be recognized; Using the max pooling layer and the fourth one-dimensional convolutional layer, perform pooling operations and convolutional operations on the matrix features of dimensions to obtain the point cloud feature vector; Using a multi-layer perceptron layer, based on the point cloud feature vector, obtain the classification result of the ship to be recognized.

Images