Ship point cloud multi-scale self-supervised identification method and device

Abstract

The present application relates to the field of ship management and control, and particularly relates to a ship point cloud multi-scale self-supervised identification method and device. A pre-trained ship depth detection model extracts features through an image multi-scale feature extraction module and a point cloud multi-scale feature extraction module, and the image multi-scale feature extraction module is pre-trained based on an existing database. A contrast learning module performs multi-scale contrast identification on the point cloud multi-scale features of a ship target sample and the pre-trained image multi-scale features, thereby achieving accurate self-supervised detection of a ship to be identified. Thus, the present application effectively overcomes the problem that the amount of data in a current ship target point cloud database is small, and a model is difficult to fully learn the feature representation of a target from a small amount of training data, which easily leads to the defect that a ship point cloud detection method based on deep learning tends to overfitting, thereby improving the target detection accuracy.

Description

Technical Field

[0001] This invention relates to the field of ship management and control, and in particular to a method and device for multi-scale self-supervised identification of ship point clouds. Background Technology

[0002] Comprehensive and widely interconnected perception is the foundation for realizing intelligent ship supervision. Deep learning-based video images are widely used in ship supervision, with a large number of detection samples and simple model training. However, it is difficult to obtain the precise position of target ships in three-dimensional space in complex environments by using only two-dimensional images for ship target detection, which has certain limitations in practical applications.

[0003] LiDAR point cloud data possesses rich information such as 3D spatial depth, reflection intensity, and spatial coordinates, enabling 3D target detection. In recent years, with the continuous development of computing power and artificial intelligence algorithms, deep learning-based ship LiDAR point cloud data detection technology has gradually become the mainstream research in this field. However, target detection algorithms based on deep convolutional neural networks typically rely on a large amount of labeled data to obtain powerful feature representations to support the performance of the detection model. Currently, ship target point cloud databases have relatively small amounts of data, making it difficult for models to fully learn the feature representations of targets from a limited amount of training data. This easily leads to overfitting in deep learning-based ship point cloud detection methods.

[0004] The complexity of inland waterway environments, the diversity of vessel scales, and the variety of vessel types introduce more diverse and complex influencing factors for target detection. Furthermore, image and point cloud data, after being processed by feature extraction networks, can generate feature maps with multiple resolutions. Shallow feature maps have higher resolution, smaller receptive fields, and more detailed information, but lack sufficient global semantic information. Deep feature maps have lower resolution but larger receptive fields and more comprehensive global semantic information, but lose more detail.

[0005] Therefore, there is a need for a multi-scale self-supervised recognition method and device that can combine video images with ship point clouds. Summary of the Invention

[0006] The purpose of this invention is to overcome the problem of model overfitting caused by insufficient label data in point cloud depth detection models in the prior art, and to provide a multi-scale self-supervised identification method and device for ship point clouds.

[0007] To achieve the above-mentioned objectives, the present invention provides the following technical solution:

[0008] A multi-scale self-supervised identification method for ship point clouds includes the following steps:

[0009] The detection data of the vessel to be identified is input into a pre-trained vessel depth detection model; the detection data includes image data and point cloud data.

[0010] The ship depth detection model outputs ship information for the ship to be identified; the ship information includes ship length, ship width, and ship type.

[0011] The ship depth detection model includes an image multi-scale feature extraction module, a point cloud multi-scale feature extraction module, a contrastive learning module, and a detection module. The image multi-scale feature extraction module is used to extract image multi-scale features, the point cloud multi-scale feature extraction module is used to extract point cloud multi-scale features, the contrastive learning module is used to compare the image multi-scale features and the point cloud multi-scale features to achieve accurate self-supervised training of the ship to be identified, and the detection module outputs the corresponding ship information based on the accurate self-supervised training results.

[0012] As a preferred embodiment of the present invention, the pre-training of the ship depth detection model includes the following steps:

[0013] S1: Input the detection data of several sets of samples into the ship depth detection model;

[0014] S2: The image multi-scale feature extraction module extracts the image multi-scale features of the detection data, and the point cloud multi-scale feature extraction module extracts the point cloud multi-scale features of the detection data;

[0015] S3: The contrast learning module learns the matching relationship of each classification label in a self-supervised manner based on the multi-scale features of the image and the multi-scale features of the point cloud, and outputs the recognition result of the current ship depth detection model;

[0016] S4: Repeat S1-S3 until the model recognition accuracy reaches the preset value.

[0017] As a preferred embodiment of the present invention, the plurality of samples include positive samples and negative samples in a predetermined ratio;

[0018] The positive samples are those where the image data and point cloud data match; the negative samples are those where the image data and point cloud data do not match.

[0019] As a preferred embodiment of the present invention, the ship depth detection model adopts the infoNCE loss function, the expression of which is:

[0020] ,

[0021] Where L is the total loss of the current batch of samples, N is the total number of samples in the current batch, i and j are the sample indices, and log() is the softmax function. Image data for the i-th sample Point cloud data of the j-th sample The similarity is τ, which is a preset hyperparameter.

[0022] As a preferred embodiment of the present invention, the image multi-scale feature extraction module uses a ResNet-50 network for forward propagation, and its operating expression is as follows:

[0023] ,

[0024] Where FM is the image fusion feature, w1, w2, w3 are trainable weight parameters, and C1, C2, C3 are the return values ​​of the three forward propagations.

[0025] As a preferred embodiment of the present invention, the point cloud multi-scale feature extraction module includes the following operating steps:

[0026] (1) The collected point cloud dataset Voxelization is performed, and its expression is:

[0027] ,

[0028] Where V is a non-empty voxel, ε is the voxel size, M is the number of point clouds, and R... M*3 It is an M*3 dimensional vector;

[0029] (2) Calculate the characteristics of non-empty voxels Its expression is:

[0030] ,

[0031] Where, x i Let V be the i point cloud data contained in a non-empty voxel V, q, k, v be the query, key, and value transformation matrices corresponding to the attention mechanism, S be the softmax normalization process, and δ be the learned relative position encoding.

[0032] (3) Generate multi-scale point cloud voxel features {F V F V↓1 F V↓2},

[0033] Among them, F V↓1 The characteristic F represents the non-empty voxel. V The output after two downsampling operations, F V↓2 The characteristic F represents the non-empty voxel. V Output after 4 downsampling steps;

[0034] (4) Set trainable weight parameters w4, w5, w6, and calculate the point cloud fusion feature F. p Its expression is:

[0035] .

[0036] As a preferred embodiment of the present invention, the pre-training further includes S0:

[0037] S0: Image fusion features F after annotation M The image multi-scale feature extraction module is trained until the model converges.

[0038] As a preferred embodiment of the present invention, step S3 includes the following steps:

[0039] S31: Through the comparative learning module, C1, C2, C3, and F are respectively... V F V↓1 F V↓2 Perform alignment and train F V F V↓1 F V↓2 ;

[0040] S32: Using the comparative learning module, respectively, F... M and F P Perform alignment and train the weight parameters w4, w5, w6;

[0041] S33: Output the recognition results of the current ship depth detection model.

[0042] As a preferred embodiment of the present invention, S3 further includes:

[0043] Based on the recognition results, the image multi-scale feature extraction module and the point cloud multi-scale feature extraction module are updated via backpropagation.

[0044] A ship point cloud multi-scale self-supervised identification device includes at least one processor and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform any of the above-described ship point cloud multi-scale self-supervised identification methods.

[0045] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0046] The pre-trained ship depth detection model of this invention extracts features through an image multi-scale feature extraction module and a point cloud multi-scale feature extraction module. A contrastive learning module compares and identifies the point cloud multi-scale features and image multi-scale features of the ship target sample, achieving accurate detection of the ship to be identified. This effectively overcomes the limitation of video images being greatly affected by environmental factors such as lighting, and improves the accuracy of target detection. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating a multi-scale self-supervised identification method for ship point clouds as described in Embodiment 1 of the present invention.

[0048] Figure 2 This is a schematic diagram of the pre-training process of the ship depth detection model described in Embodiment 2 of the present invention;

[0049] Figure 3 This is a schematic diagram of the operational logic of step S2 in the ship depth detection model described in Embodiment 2 of the present invention;

[0050] Figure 4 This is a schematic diagram of the forward pass of the convolutional network in the ship depth detection model described in Embodiment 2 of the present invention;

[0051] Figure 5 This is a schematic diagram of the CLIP network architecture in the ship depth detection model described in Embodiment 2 of the present invention;

[0052] Figure 6 This is a schematic diagram of the structure of a ship point cloud multi-scale self-supervised identification device that utilizes the ship point cloud multi-scale self-supervised identification method described in the foregoing embodiments, as described in Embodiment 4 of the present invention. Detailed Implementation

[0053] The present invention will be further described in detail below with reference to experimental examples and specific embodiments. However, this should not be construed as limiting the scope of the above-mentioned subject matter of the present invention to the following embodiments; all technologies implemented based on the content of the present invention fall within the scope of the present invention.

[0054] Example 1

[0055] like Figure 1 As shown, a multi-scale self-supervised identification method for ship point clouds includes the following steps:

[0056] The detection data of the vessel to be identified is input into a pre-trained vessel depth detection model; the detection data includes image data and point cloud data.

[0057] The ship depth detection model outputs ship information for the ship to be identified; the ship information includes ship length, ship width, and ship type.

[0058] The ship depth detection model includes an image multi-scale feature extraction module, a point cloud multi-scale feature extraction module, a contrastive learning module, and a detection module. The image multi-scale feature extraction module is used to extract image multi-scale features, the point cloud multi-scale feature extraction module is used to extract point cloud multi-scale features, the contrastive learning module is used to compare the image multi-scale features and the point cloud multi-scale features to achieve accurate self-supervised training of the ship to be identified, and the detection module outputs the corresponding ship information based on the accurate self-supervised training results.

[0059] Example 2

[0060] This embodiment is a specific pre-training method for the ship depth detection model in the multi-scale self-supervised recognition method for ship point clouds described in Embodiment 1, such as... Figure 2 As shown, the pre-training of the ship depth detection model includes the following steps:

[0061] S1: Input the detection data of several sets of samples into the ship depth detection model.

[0062] The sample groups include a preset ratio of positive and negative samples;

[0063] Positive samples are those where the image data and point cloud data match; negative samples are those where the image data and point cloud data do not match. For example, with three image-point cloud pairs, there are three positive samples and six negative samples. As shown in the diagram above, n image vectors and their corresponding cloud vectors will constitute n positive samples (the diagonal portion) and the remaining negative samples. This is called contrastive learning, which defines positive and negative samples based on their similarity or matching, without manually labeling them.

[0064] Furthermore, in this embodiment, the detection data is collected by a multi-line lidar and an 18x black light camera; both are installed at the same height, enabling the capture of 3D point cloud and visible light information of the same target in the field of view.

[0065] S2: The image multi-scale feature extraction module extracts the image multi-scale features of the detection data, and the point cloud multi-scale feature extraction module extracts the point cloud multi-scale features of the detection data. Its operating logic is as follows: Figure 3 As shown, it includes the following steps:

[0066] (1) Perform convolution operation on the acquired image as shown in the figure, such as Figure 4 As shown, the bottom-up process is the forward pass of the convolutional network. The backbone network is ResNet-50, and the return values ​​of the forward pass are C1, C2, and C3, respectively. C2 and C3 are the feature vectors obtained after 2 and 3 pooling operations, respectively.

[0067] (2) Set trainable weight parameters w 1 ,w 2 ,w 3, The expression for image fusion features is:

[0068]

[0069] Where FM is the image fusion feature, w1, w2, w3 are trainable weight parameters, and C1, C2, C3 are the return values ​​of the three forward propagations.

[0070] Furthermore, the point cloud multi-scale feature extraction module uses the POINTNET++ network. It includes the following steps:

[0071] (1) The collected point cloud dataset Voxelization is performed, and its expression is:

[0072] ,

[0073] Where V is a non-empty voxel, ε is the voxel size, M is the number of point clouds, and R... M*3 It is an M*3 dimensional vector;

[0074] (2) Calculate the characteristics of non-empty voxels Its expression is:

[0075] ,

[0076] Where, x i Let V be the i point cloud data contained in a non-empty voxel V, q, k, v be the query, key, and value transformation matrices corresponding to the attention mechanism, S be the softmax normalization process, and δ be the learned relative position encoding.

[0077] (3) Generate multi-scale point cloud voxel features {F V F V↓1 F V↓2},

[0078] Among them, F V↓1 The characteristic F represents the non-empty voxel. V The output after two downsampling operations, F V↓2 The characteristic F represents the non-empty voxel. V Output after 4 downsampling steps;

[0079] (4) Set trainable weight parameters w4, w5, w6, and calculate the point cloud fusion feature F. p Its expression is:

[0080] .

[0081] S3: The contrastive learning module learns the matching relationship of each classification label in a self-supervised manner based on the multi-scale features of the image and the multi-scale features of the point cloud, and outputs the recognition result of the current ship depth detection model.

[0082] Furthermore, this includes the following steps:

[0083] S31: Through the comparative learning module, C1, C2, C3, and F are respectively... V F V↓1 F V↓2 Perform alignment and train F V F V↓1 F V↓2 ;

[0084] S32: Using the comparative learning module, respectively, F... M and F P Perform alignment and train the weight parameters w4, w5, w6;

[0085] S33: Output the recognition result of the current ship depth detection model.

[0086] Furthermore, it also includes backpropagation updates of the image multi-scale feature extraction module and the point cloud multi-scale feature extraction module based on the recognition results.

[0087] S4: Repeat S1-S3 until the model recognition accuracy reaches the preset value.

[0088] During the pre-training phase, the contrastive learning module learns the matching relationship between the feature vector representations of images and point clouds. The model receives a batch of image-point cloud pairs as input and attempts to bring matching image and point cloud vectors closer together in a common semantic space, while pushing mismatched vectors further apart. This involves calculating the cosine similarity between the category label and the prediction; the label with the highest similarity is the predicted classification result. This learning method enables the contrastive learning module to capture the deep semantic connections between images and point clouds, achieving cross-modal understanding.

[0089] The goal of training the model is to maximize the cosine similarity of positive samples and minimize the cosine similarity of negative samples. In this embodiment, the contrastive learning module uses the infoNCE loss function, whose expression is:

[0090] ,

[0091] Where L is the total loss of the current batch of samples, N is the total number of samples in the current batch, i and j are the sample indices, and log() is the softmax function. Image data for the i-th sample Point cloud data of the j-th sample The similarity is denoted by τ, which is a preset hyperparameter. Referring to the negative sign at the beginning of the formula, L represents the loss of a single sample; the denominator is the sum of the similarities between an image and N point cloud datasets in a batch, and the numerator is the similarity between the image and the point cloud datasets it matches. This corresponds to the principle of "making the similarity of positive samples higher and the similarity of negative samples lower," because this results in a smaller loss function value. Furthermore, this embodiment introduces a temperature coefficient τ as a hyperparameter to adjust the model's ability to distinguish between different pairs of positive and negative samples based on their similarity.

[0092] Furthermore, the contrastive learning module is a CLIP (Contrastive Language-Image Pre-Training) network. The operating principle of the CLIP network is as follows: Figure 5 As shown, it is used to compare and learn point cloud feature sequences and image feature sequences.

[0093] Furthermore, this invention considers the ship inspection situation in real-world scenarios, using bridge A and bridge B in a certain location as two experimental scenarios, and diagnoses ship inspection capabilities through specific comparative experiments.

[0094] In this experiment, the sample data specifications are as follows: Video recordings were made using video surveillance with a resolution of 1920*1080 pixels and a frame rate of 30 frames per second; a hybrid solid-state 32-line lidar was used, featuring a 360° horizontal field of view, a measurement accuracy within 2cm for the furthest distance, and a 15° vertical tilt angle. The comparative experiment setup is as follows:

[0095] 1) Comparison Method 1: Using the YOLOv9 model based on image information; 2) Comparison Method 2: Using the Pointnet++ model based on point cloud information; 3) Comparison Method 3: Multimodal information fusion algorithm under incremental graph model.

[0096] Based on the methods of the above comparative experiments, when the detection accuracy of the corresponding method reaches more than 90%, in addition to the publicly available unlabeled samples, the number of labeled training samples required is as follows. The specific test results are shown in Table 1 below.

[0097] Table 1. Comparison of the number of labeled training samples

[0098]

[0099] According to the experimental results shown in Table 1, when the detection accuracy is above 90%, the comparative method 1 requires 10,000 labeled training samples in the embodiment, including 5,000 ship images from the public dataset and 5,000 images from the on-site dataset. The comparative method 2 requires 2,000 frames of on-site point cloud data with labels in the embodiment. The comparative method 3 requires 1,000 labeled on-site training samples in the embodiment, including 500 images and 500 frames of point cloud data. In contrast, the method of this application only requires 5,000 ship images from the public dataset, without the need for labeled on-site data. This demonstrates the effectiveness of comparative learning. It can utilize rich and easily accessible public image datasets to achieve zero-shot self-supervised training of the point cloud deep model, solving the problem that the current ship target point cloud database has a small amount of data, making it difficult for the model to fully learn the feature representation of the target from the limited training data, which easily leads to overfitting in deep learning-based ship point cloud detection methods.

[0100] Furthermore, in this embodiment, 5000 training samples were collected according to the aforementioned sample data specifications to pre-train each comparison method, and the target detection accuracy of each comparison method at a distance of 300m is shown in Table 2 below.

[0101] Table 2. Target detection accuracy results of each comparison method at a distance of 300m.

[0102]

[0103] The experimental results are shown in Table 2. Within a detection range of 300m, the ship detection accuracy rates of comparative method 1 in the examples were 84.2% and 84.6%, respectively; those of comparative method 2 were 88.5% and 89.1%, respectively; and those of comparative method 3 were 93.5% and 93.8%, respectively. In contrast, the ship detection accuracy rates of the method of this application were 97.1% and 97.8%, respectively. The effectiveness of the comparative learning of this invention is clearly evident. It can enhance point cloud learning with image-supervised signals, and the multi-scale feature extraction module effectively fuses deep features with shallow feature maps, constructing a feature pyramid that integrates high-resolution details and strong semantic information, significantly improving the detection capability of targets with scale differences.

[0104] This invention utilizes a contrastive learning model to construct cross-modal positive and negative sample pairs between images and point clouds. Simultaneously, the image multi-scale feature extraction module repeatedly extracts depth image features from the acquired images through multiple forward propagations and pooling. The point cloud multi-scale feature extraction module converts the point cloud into non-empty voxels and performs multiple downsampling before fusion. Finally, the contrastive learning module uses self-supervised pairing of multi-scale features and the infoNCE loss function to accurately align multi-modal data, mining features that are similar between images and point clouds of the same object. This allows the point cloud model to self-supervisedly learn discriminative feature representations from rich image data, achieving knowledge transfer and mutual enhancement between modalities. As shown in Table 1, the experimental results demonstrate that the method of this invention maximizes the use of existing 2D data assets to empower 3D applications, reducing data dependence on point cloud databases and the risk of overfitting. Furthermore, the experimental results in Table 2 show that the method of this invention significantly improves 3D task performance through multi-modal complementarity, significantly enhancing the detection capability of targets with scale differences.

[0105] Example 3

[0106] The difference between this embodiment and Embodiment 2 is that the pre-training further includes S0:

[0107] S0: Image fusion features F after annotation M The image multi-scale feature extraction module is trained until the model converges.

[0108] Example 4

[0109] like Figure 6 As shown, a ship point cloud multi-scale self-supervised identification device includes at least one processor, a memory communicatively connected to the at least one processor, and at least one input / output interface communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed, enables the at least one processor to perform the ship point cloud multi-scale self-supervised identification method described in the foregoing embodiments. The input / output interface may include a display, keyboard, mouse, and USB interface for inputting and outputting data.

[0110] Furthermore, the ship point cloud multi-scale self-supervised recognition device can be a desktop computer, mobile phone, tablet computer, wearable ship point cloud multi-scale self-supervised recognition device, or any other ship point cloud multi-scale self-supervised recognition device capable of performing deep information recognition.

[0111] Furthermore, the processor may include one or more processing cores. The processor connects to various parts of the ship point cloud multi-scale self-supervised identification device using various interfaces and lines. It executes various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory, and by calling data stored in memory. Optionally, the processor may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also be implemented separately as a communication chip, without being integrated into the processor.

[0112] The memory may include random access memory (RAM) or read-only memory (ROM). The memory can be used to store instructions, programs, code, code sets, or instruction sets, such as instructions or code sets used to implement the ship point cloud multi-scale self-supervised identification method provided in this application embodiment. The memory may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the various method embodiments described above, etc. The data storage area may also store data created during the use of the ship point cloud multi-scale self-supervised identification device (such as a modulation sequence-depth mapping table, image data, spectrogram data, etc.).

[0113] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory, magnetic disks, or optical disks.

[0114] When the integrated units of the present invention are implemented as software functional units and sold or used as independent products, they can also be stored in a computer-readable storage medium. The computer-readable storage medium stores program code, which can be called by a processor to execute the methods described in the above method embodiments. Based on this understanding, the technical solution of the embodiments of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes electronic memories such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium includes a non-transitory computer-readable storage medium. The computer-readable storage medium has storage space for program code that executes any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code can be compressed, for example, in an appropriate form.

[0115] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-scale self-supervised identification method for ship point clouds, characterized in that, Includes the following steps: The detection data of the vessel to be identified is input into a pre-trained vessel depth detection model; the detection data includes image data and point cloud data. The ship depth detection model outputs ship information for the ship to be identified; the ship information includes ship length, ship width, and ship type. The ship depth detection model includes an image multi-scale feature extraction module, a point cloud multi-scale feature extraction module, a contrastive learning module, and a detection module. The image multi-scale feature extraction module is used to extract image multi-scale features, the point cloud multi-scale feature extraction module is used to extract point cloud multi-scale features, the contrastive learning module is used to compare the image multi-scale features and the point cloud multi-scale features to achieve accurate self-supervised training of the ship to be identified, and the detection module outputs the corresponding ship information based on the accurate self-supervised training results. The image multi-scale feature extraction module uses a ResNet-50 network for forward propagation, and its operating expression is as follows: ， Where FM is the image fusion feature, w1, w2, w3 are trainable weight parameters, and C1, C2, C3 are the return values ​​of the three forward propagations. The point cloud multi-scale feature extraction module includes the following steps: (1) The collected point cloud dataset Voxelization is performed, and its expression is: ， Where V is a non-empty voxel, ε is the voxel size, M is the number of point clouds, and R... M*3 It is an M*3 dimensional vector; (2) Calculate the characteristics of non-empty voxels Its expression is: ， Where, x i Let V be the i point cloud data contained in a non-empty voxel V, q, k, v be the query, key, and value transformation matrices corresponding to the attention mechanism, S be the softmax normalization process, and δ be the learned relative position encoding. (3) Generate multi-scale point cloud voxel features {F V F V↓1 F V↓2 }, Among them, F V↓1 The characteristic F represents the non-empty voxel. V The output after two downsampling operations, F V↓2 The characteristic F represents the non-empty voxel. V Output after 4 downsampling steps; (4) Set trainable weight parameters w4, w5, w6, and calculate the point cloud fusion feature F. p Its expression is: The pre-training of the contrastive learning module includes the following steps: The comparative learning module was used to analyze C1, C2, C3, and F respectively. V F V↓1 F V↓2 Perform alignment and train F V F V↓1 F V↓2 ; The comparative learning module was used to analyze F respectively. M and F P Perform alignment and train the weight parameters w4, w5, w6; Output the recognition results of the current ship depth detection model.

2. The method for multi-scale self-supervised identification of ship point clouds according to claim 1, characterized in that, The pre-training of the ship depth detection model includes the following steps: S1: Input the detection data of several sets of samples into the ship depth detection model; S2: The image multi-scale feature extraction module extracts the image multi-scale features of the detection data, and the point cloud multi-scale feature extraction module extracts the point cloud multi-scale features of the detection data; S3: The contrastive learning module learns the matching relationship of each classification label in a self-supervised manner based on the multi-scale features of the image and the multi-scale features of the point cloud, and outputs the recognition result of the current ship depth detection model; S4: Repeat S1-S3 until the model recognition accuracy reaches the preset value.

3. The method for multi-scale self-supervised identification of ship point clouds according to claim 2, characterized in that, The sample groups include a preset ratio of positive and negative samples; The positive samples are those where the image data and point cloud data match; the negative samples are those where the image data and point cloud data do not match.

4. The method for multi-scale self-supervised identification of ship point clouds according to claim 2, characterized in that, The ship depth detection model uses the infoNCE loss function, the expression of which is: ， Where L is the total loss of the current batch of samples, N is the total number of samples in the current batch, i and j are the sample indices, and log() is the softmax function. Image data for the i-th sample Point cloud data of the j-th sample The similarity is τ, which is a preset hyperparameter.

5. The method for multi-scale self-supervised identification of ship point clouds according to claim 4, characterized in that, The pre-training also includes S0: S0: Image fusion features F after annotation M The image multi-scale feature extraction module is trained until the model converges.

6. The method for multi-scale self-supervised identification of ship point clouds according to claim 5, characterized in that, S3 further includes: Based on the recognition results, the image multi-scale feature extraction module and the point cloud multi-scale feature extraction module are updated via backpropagation.

7. A multi-scale self-supervised identification device for ship point clouds, characterized in that, The method includes at least one processor and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform a multi-scale self-supervised identification method for ship point clouds according to any one of claims 1 to 6.

Images