A method for detecting underwater targets in unmanned boats

Through technical means such as multi-scale residual attention network and principal component analysis, combined with dynamic detection strategies and multi-task collaborative identification detection network, the problems of weak anti-interference ability and high computational complexity in underwater target detection of unmanned boats are solved. Achieve efficient and accurate underwater target detection.

CN118823562BActive Publication Date: 2024-12-06ZHEJIANG INTELLIGENT SHIP RES INST CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411302927.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-19
Publication Date
2024-12-06
Estimated Expiration
2044-09-19

AI Technical Summary

Technical Problem

The existing unmanned boat underwater target detection method has weak anti-interference ability, low accuracy of recognition of small targets, high computational complexity, and insufficient fusion of multi-source heterogeneous data in complex underwater environments, resulting in unsatisfactory detection performance and affecting actual conditions. Application reliability and real-time.

Method used

Multi-scale residual attention network (MS-RAN) is used to combine principal component analysis and autoencoder for feature extraction and dimensionality reduction, dynamic detection strategies and multi-task collaborative identification detection network, and combined with fluctuation attenuation learning rate scheduler to realize image Multimodal feature fusion and object detection of acoustic data.

Benefits of technology

It improves the target detection capability in complex environments, enhances the recognition accuracy of small targets, reduces the computational complexity, improves detection efficiency and performance stability, provides more accurate detection results and stronger anti-interference ability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118823562B_ABST
    Figure CN118823562B_ABST
Patent Text Reader

Abstract

The present invention provides an underwater target detection method for an unmanned boat, and belongs to the field of underwater target detection for unmanned boats. First, the underwater target historical data is obtained, including underwater shooting image data and underwater acoustic data obtained synchronously; then pre-processing is performed respectively to obtain standard input data for underwater target detection model training; an underwater target detection model is constructed, including a newly designed feature extraction module and a target detection module, and the model is trained and deployed for real-time target detection, and target classification labels and target position labels are output. The overall model structure design of the present invention greatly improves the efficiency and accuracy of underwater target detection. The design of the feature extraction module can more effectively capture features of different scales in complex environments, and improves the model's target detection capability under harsh conditions. The design of the dynamic detection strategy and multi-task collaborative recognition detection improves detection efficiency and accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of underwater target detection for unmanned boats, and in particular relates to an underwater target detection method for unmanned boats. Background Art

[0002] With the rapid progress of ocean development and underwater detection technology, unmanned boats play an increasingly important role in the fields of marine resource exploration, marine environment monitoring, underwater archaeology, etc. However, the complex and changeable underwater environment has brought unprecedented challenges to target detection. Traditional sonar detection and optical imaging technology often find it difficult to achieve fast and accurate target recognition and positioning when faced with problems such as uneven underwater illumination, low visibility, and target occlusion. How to efficiently and accurately detect various targets in a complex underwater environment has become a key issue in the design of unmanned boat systems. Intelligent underwater target detection systems based on computer vision and deep learning technology have emerged, but the existing technology still has many defects such as weak anti-interference ability, difficulty in identifying small targets, and insufficient real-time performance. Therefore, it is of great theoretical significance and practical application value to develop a new intelligent detection algorithm that integrates multi-source sensor information and adaptive learning feature representation to achieve efficient and accurate underwater target detection. This will not only improve the operating efficiency and safety of unmanned boats, but will also provide strong technical support for the development of marine resources, marine ecological protection, and underwater cultural heritage protection. However, the existing technical attempts have the following defects:

[0003] 1. Existing underwater target detection methods have insufficient anti-interference capabilities. When faced with complex underwater environments, such as turbid water, uneven illumination, and water flow interference, it is often difficult to maintain stable detection performance. This leads to false detection or missed detection in practical applications, affecting the reliability of detection results.

[0004] 2. Existing technologies have low accuracy in identifying and locating these small targets, making it difficult to meet the needs of refined detection. Due to the particularity of the underwater environment, many important targets (such as small marine organisms, underwater ruins, etc.) are often small in size or highly similar to the background.

[0005] 3. Existing deep learning-based methods usually have high computational complexity and are difficult to achieve real-time detection under limited hardware resources, which seriously restricts the real-time decision-making and rapid response capabilities of unmanned boats.

[0006] 4. The existing technology still has shortcomings in the effective fusion of multi-source heterogeneous data, and fails to fully utilize the complementary advantages of various sensors, resulting in the detection performance failing to reach the ideal level. Summary of the invention

[0007] In view of the above problems, the present invention provides an underwater target detection method for an unmanned boat, comprising the following steps:

[0008] S1, acquiring underwater target data, including underwater image data and synchronously acquired underwater acoustic data;

[0009] S2, preprocessing the image data and acoustic data respectively to obtain standard input data for underwater target detection model training;

[0010] S3, inputs the standard input data into the trained underwater target detection model, and outputs the target classification label and target location label;

[0011] The underwater target detection model includes a feature extraction module and a target detection module;

[0012] The feature extraction module includes an image feature extraction unit and an acoustic feature extraction unit connected in parallel, and a feature fusion unit and a feature dimension reduction unit connected in series at the back end of the feature extraction unit; the image feature extraction unit captures target features of different scales through multi-scale feature extraction and residual connection, and uses an attention mechanism to emphasize important features; the acoustic feature extraction unit uses a hybrid method based on time-frequency analysis and deep learning to extract acoustic features; the feature fusion unit fuses the extracted image features and acoustic features into multi-modal feature vectors, and performs dimension reduction and denoising through the feature dimension reduction unit to obtain a feature matrix ;

[0013] The target detection module uses a multi-task collaborative recognition detection network framework and a dynamic detection strategy; the multi-task collaborative recognition detection network is based on the input feature matrix , target classification and position regression are performed simultaneously. The dynamic detection strategy adaptively adjusts the detection window and step size according to the size, shape and motion characteristics of the target.

[0014] Preferably, the preprocessing of the image data includes image denoising, image size unification and pixel value normalization;

[0015] The image denoising is performed by using a bilateral filter to perform denoising processing, and weighted average pixel values ​​in a neighborhood are calculated according to the spatial distance between pixels and the difference in pixel values ​​to smooth the image; the image size unification is to adjust all images to a uniform size, and use a bilinear interpolation method to calculate new pixel values ​​by linear interpolation during resampling; the pixel value normalization is to scale the pixel values ​​to the range of [0, 1];

[0016] The preprocessing of the acoustic data includes using a Gaussian filtering algorithm to remove noise in the audio.

[0017] Preferably, the image feature extraction unit includes an input layer, a multi-scale convolution layer, an image feature fusion layer, a residual block and an attention mechanism layer;

[0018] The input layer receives the preprocessed image ;

[0019] The multi-scale convolution layer uses a 3x3 convolution kernel, a 5x5 convolution kernel and a 7x7 convolution kernel to image Extract features in parallel to obtain multi-scale features , and :

[0020]

[0021] in, represents a convolution operation of size 3x3, represents a convolution operation of size 5x5, Represents a convolution operation of size 7x7;

[0022] The image feature fusion layer performs splicing operations on multi-scale features to obtain :

[0023]

[0024] in, Represents a splicing operation;

[0025] The residual block uses multiple residual blocks to further extract features, and the output of the residual block is:

[0026]

[0027] in, represents batch normalization, is the activation function;

[0028] The attention mechanism enhances features by introducing channel attention and spatial attention:

[0029]

[0030]

[0031] in represents the channel attention operation, represents the spatial attention operation, represents the sigmoid activation function, MLP is a multi-layer perceptron, represents the global average pooling operation, represents the average pooling operation, Represents the maximum pooling operation;

[0032] Apply the attention mechanism to the residual features, and the image features enhanced by the attention mechanism .

[0033] Preferably, the image feature extraction unit further includes an underwater feature enhancement mechanism, introducing an underwater image enhancement method based on a physical model;

[0034] First of all, Perform color correction based on depth estimation:

[0035]

[0036] in, represents the color channel, is the attenuation coefficient of each channel, is the estimated depth map, To get the exponential operator with base e, Image features enhanced by the attention mechanism;

[0037] Afterwards, the enhanced features are weightedly fused with the original features to obtain the final image features:

[0038]

[0039] in, is the learnable weight parameter, It is the feature map after color correction.

[0040] Preferably, the acoustic feature extraction unit first undergoes time-frequency analysis:

[0041] Short-time Fourier transform is used to convert the time domain signal into time-frequency domain representation. Next, the acoustic data processed by STFT is extracted through Mel-frequency cepstral coefficients (MFCC) to convert the original acoustic signal into a form that is easy to process by deep learning models while retaining the key time-frequency features of the signal.

[0042] Then perform deep learning feature extraction based on the 1D-CNN and BiLSTM networks:

[0043] First, a one-dimensional convolutional neural network layer 1D-CNN is used to extract The local features of , the operation of 1D-CNN is expressed as:

[0044]

[0045] in, is a one-dimensional convolution operation, is the convolution kernel weight, is the bias term;

[0046] The BiLSTM layer is then used to capture long-term dependencies, and the forward propagation of BiLSTM is simplified as follows:

[0047]

[0048] in, express The BiLSTM hidden layer state at time , express Input at the moment, is the hidden layer state at the previous moment;

[0049] Based on the final hidden layer state representation obtained, it is mapped to a feature space of a specific dimension through a fully connected layer to obtain the final acoustic feature , transforming the temporal information captured by the BiLSTM network into a more compact and effective feature representation.

[0050] Preferably, the specific processing process of the feature dimension reduction unit is:

[0051] First, principal component analysis (PCA) is used to reduce the dimensionality of the fusion features. The data is mapped from high-dimensional space to low-dimensional space through linear transformation, while retaining important information in the data as much as possible to obtain preliminary dimensionality reduction features:

[0052]

[0053] in, is the number of principal components;

[0054] Then, the autoencoder is used to further perform dimensionality reduction and denoising. The autoencoder consists of two parts: the encoder and the decoder. Input to the encoder to get the encoded feature matrix , mapping the high-dimensional data into a lower-dimensional latent space, and then Input to the decoder to generate the reconstructed feature matrix , reconstruct the original feature data from the low-dimensional feature space;

[0055] The training goal of the autoencoder is to minimize the following loss function:

[0056]

[0057] in, Represents the mean square error, which is used to measure the reconstruction features and the original dimension reduction features The goal of minimizing the mean square error is to make the reconstructed data as close to the original data as possible. The regularization term is used to encode the features Imposing sparsity constraints, regularization parameters Control How much the regularization term affects the overall loss.

[0058] Preferably, the multi-task collaborative recognition and detection network framework decomposes the target detection problem into a series of gradually refined subtasks, each of which is processed by a dedicated sub-network, allowing the model to gradually refine features at different scales and complexity levels;

[0059] Among them, the classification head uses the softmax activation function to output the probability distribution of the target category:

[0060]

[0061] represents the target category probability distribution output by the classification head, A function representing a classification head;

[0062] Among them, the regression head uses the softmax activation function to predict the precise position coordinates of the target:

[0063]

[0064] represents the target position coordinates output by the regression head, A function representing a regression head.

[0065] Preferably, the dynamic detection strategy is based on a designed adaptive region proposal network ARPN, which dynamically adjusts its proposal strategy according to the characteristics of the input image; the ARPN uses a lightweight meta-learning module to predict the optimal anchor box size and ratio to adapt to different underwater scenes:

[0066]

[0067] in, represents the generated anchor box set, represents the meta-learning module, Represents a function that generates anchor boxes based on the parameters output by the meta-learning module;

[0068] At the same time, attention-based object localization AGL uses spatial and channel attention to enhance target-related features while suppressing background interference:

[0069]

[0070] in, It represents the features after AGL feature enhancement. represents the spatial attention map, represents the channel attention vector, represents element-wise multiplication, Represents tensor multiplication.

[0071] Preferably, the target detection module further includes non-maximum suppression:

[0072] Use soft NMS to reduce redundant boxes based on the union ratio and only keep the boxes with the highest confidence:

[0073]

[0074] in, Represents a candidate bounding box The initial confidence score of represents the current highest scoring bounding box, represents the bounding box with the highest score With candidate box The intersection ratio of Represents a hyperparameter that controls the degree of decay.

[0075] Preferably, when training the underwater target detection model, the designed adaptive learning rate adjustment mechanism is introduced:

[0076]

[0077] in, Indicated in The learning rate for each epoch, represents the basic learning rate, Represents the total number of training epochs, represents the decay rate;

[0078] Combining cosine annealing and exponential decay, the cosine term makes the learning rate fluctuate periodically during training, while the exponential term ensures that the overall learning rate gradually decreases over time; the specific steps include:

[0079] Initialize the base learning rate 、Total number of epochs and decay rate ;

[0080] At the beginning of each epoch, the learning rate of the current epoch is calculated according to the above formula;

[0081] Apply the calculated learning rate to the optimizer.

[0082] Compared with the prior art, the present invention has the following beneficial effects:

[0083] 1. A method for image feature extraction based on multi-scale residual attention network (MS-RAN) is proposed. The network structure contains multi-scale convolutional layers, residual blocks and attention mechanisms, which can effectively capture target features of different scales and adaptively emphasize important features. Compared with traditional single-scale convolutional networks, MS-RAN can more effectively capture features of different scales in complex environments, especially when dealing with small targets and targets with large shape changes. In addition, the introduction of the attention mechanism makes the network perform better when dealing with uneven lighting, occlusion and other situations, and improves the model's target detection ability under harsh conditions.

[0084] 2. A feature dimension reduction and redundant information filtering mechanism based on the combination of principal component analysis (PCA) and autoencoder is designed. The mechanism first uses PCA for preliminary dimension reduction, and then uses the autoencoder to further compress features and filter redundant information. The encoder part of the autoencoder uses sparse constraints to enhance the expressiveness of features. By combining PCA with the autoencoder, the mechanism reduces the feature dimension while retaining important information in the data. This not only improves the efficiency of subsequent processing, but also enhances the generalization ability of the model, making the model more stable and superior on different data sets.

[0085] 3. An innovative target detection module was developed, which includes two core components: dynamic detection strategy and multi-task collaborative recognition and detection. The dynamic detection strategy adaptively adjusts the detection window and step size according to the size, shape and motion characteristics of the target to improve detection efficiency. Multi-task collaborative recognition and detection simultaneously performs target classification and position regression, and improves overall performance through shared features and mutual promotion between tasks. The dynamic detection strategy adaptively adjusts the detection window and step size according to the target features, greatly reducing the computational complexity while maintaining efficient detection performance. Multi-task collaborative recognition and detection improves the overall performance of target classification, position regression and posture estimation through shared features and information complementarity, providing more accurate detection results.

[0086] 4. An adaptive learning rate adjustment strategy based on a fluctuation decay learning rate scheduler is proposed, which combines the advantages of cosine annealing and exponential decay. Through the dynamic adjustment of periodic fluctuations and gradual decay, the performance of the learning rate in the training process is optimized, the exploration ability of the parameter space is enhanced, and at the same time, it ensures that the model can converge to a better local minimum more stably, significantly improving the effect of learning rate adjustment. The cosine annealing mechanism enhances the exploration ability of the parameter space and reduces the risk of falling into the local minimum, while the exponential decay ensures that the learning rate gradually decreases and enhances the convergence stability of the model. This dynamic adjustment strategy optimizes the training process and improves the overall performance and training efficiency of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 The figure is an overall logic block diagram of the underwater target detection method of the unmanned boat of the present invention.

[0088] Figure 2 This is a comparison chart before and after acoustic data preprocessing in an embodiment of the present invention.

[0089] Figure 3 This is a diagram of the residual block network structure of the present invention.

[0090] Figure 4 This is a network structure diagram of the feature fusion unit of the present invention.

[0091] Figure 5 This is an example diagram of the target detection results of the present invention. DETAILED DESCRIPTION

[0092] The invention will be further described below in conjunction with specific embodiments.

[0093] The present invention aims to solve the problems of existing unmanned boat underwater target detection methods through innovative technical solutions, and provide a new solution for the field of unmanned boat underwater target detection. The overall logic is as follows Figure 1 As shown. First, obtain the underwater target historical data, including underwater image data and synchronously acquired underwater acoustic data; then preprocess the image data and acoustic data respectively to obtain the standard input data for underwater target detection model training; build an underwater target detection model, which includes a feature extraction module and a target detection module, and train and deploy the model for real-time target detection; based on the newly acquired underwater image data and synchronously acquired underwater acoustic data, input the preprocessed standard input data into the trained underwater target detection model, and output the target classification label and target position label.

[0094] 1. Raw data collection

[0095] This phase aims to collect, process and label data for training the underwater target detection model of the unmanned vehicle. High-quality and diverse data is the basis for building an effective underwater target detection model for unmanned vehicles, so this process is crucial to the performance of the model.

[0096] Original underwater target image acquisition:

[0097] Equipment preparation: high-resolution underwater camera (Deep-Sea HDTV Camera System, resolution 4K); sonar system (Multi-beam Echosounder, model ME70); underwater lighting system (DeepSea Power&LightMatrix-3 LED); unmanned underwater vehicle (AUV) (Kongsberg HUGIN AUV).

[0098] Equipment deployment: Install cameras, sonar and lighting systems on the AUV; calibrate all equipment to ensure they are in good working condition.

[0099] Data collection: The AUV navigates underwater along a preset route; the camera continuously collects images at a frame rate of 30fps; the acoustic system synchronously collects acoustic data with a sampling rate of 192kHz to assist in target recognition, and can provide target distance and material information that is difficult to capture in images; the lighting system automatically adjusts the brightness according to the underwater lighting conditions.

[0100] Data transmission and storage: The collected images and acoustic data are transmitted to the AUV internal storage system in real time; the data is saved in the original format, the images are in lossless compressed .tiff format, and the acoustic data are in .wav format.

[0101] Through the above process, the original data set is obtained, which contains high-quality underwater target images and corresponding acoustic data.

[0102] 2. Data Preprocessing

[0103] In the data preprocessing stage, it mainly involves denoising, standardization and normalization of the original image. Indicates that and are the pixel coordinates of the image.

[0104] 1. Image denoising: In order to remove the noise in the image while retaining the edge information of the image, a bilateral filter is used for denoising. The bilateral filter is a nonlinear filter that can weighted average the pixel values ​​in the neighborhood according to the spatial distance between pixels and the difference in pixel values, thereby achieving the effect of smoothing the image.

[0105]

[0106] in, is the pixel position after denoising The pixel value at ; is a Gaussian function in the spatial domain, which is based on the pixel and The spatial distance between pixels is weighted. and For and The pixel value in the neighborhood, Control the spatial range of smoothing; is a Gaussian function with a range of and The pixel value difference between Control the similarity range of pixel values; is the normalization factor.

[0107] 2. Image size uniformity: To facilitate subsequent processing and analysis, all images are resized to a uniform size, e.g. During the adjustment process, the bilinear interpolation method is used. This method calculates new pixel values ​​through linear interpolation during resampling, which can better preserve the details of the image.

[0108]

[0109] in, For the adjusted image, It is a bilinear interpolation method.

[0110] 3. Pixel value normalization: In order to make the image data suitable for the input requirements of the deep learning model, the pixel values ​​are scaled to the range of [0, 1]. This process is achieved by dividing the value of each pixel by 255.

[0111] Through the above denoising, size unification and normalization processing, the preprocessed data set is obtained, which is denoted as , the quality of image data has been significantly improved, laying a solid foundation for subsequent target detection tasks.

[0112] 4. Acoustic Data Denoising

[0113] The original audio data is subjected to a Gaussian filter algorithm to remove the noise in the audio and obtain the denoised audio data. In Gaussian filtering, the filter kernel size is , Gaussian kernel The mathematical expression is:

[0114]

[0115] in, is the row and column index of the filter kernel coordinates, is the relative position of the current pixel relative to the kernel, is the standard deviation of the Gaussian kernel, which controls the smoothness of the filter. Figure 2 shown.

[0116] 5. Data labeling:

[0117] The data labeling process uses a semi-automatic method that combines computer vision algorithms and manual verification. The labels include target category and location information for subsequent classification and regression tasks.

[0118] (1) Target category labeling:

[0119] Definition categories: Based on common underwater targets, targets are divided into 5 categories: 1. Fish; 2. Corals; 3. Shipwrecks; 4. Minerals; 5. Other man-made objects.

[0120] Preliminary automatic classification: Use a pre-trained convolutional neural network to perform preliminary classification of images.

[0121] Manual verification and correction: Professionals review the automatic classification results and make corrections when necessary.

[0122] The target category label is recorded as .

[0123] (2) Target location mark:

[0124] Target detection: Use the YOLO algorithm to perform preliminary detection of targets in the image and obtain the bounding box coordinates.

[0125] Bounding box refinement: Manually review and refine the bounding box to ensure accuracy.

[0126] The target position label is recorded as .

[0127] (3) Label quality control:

[0128] Cross-validation: Different labelers label the same batch of data and compare the results.

[0129] Regular spot checks: 10% of labels are randomly selected for review.

[0130] Annotation Guide Updates: Annotation Guide is updated regularly based on frequently asked questions.

[0131] Through the above labeling process, high-quality labeled data containing target category and location information is obtained, laying the foundation for subsequent classification and regression tasks.

[0132] The obtained complete label is recorded as .

[0133] 6. Dataset construction:

[0134] In order to divide the labeled data into training set, validation set and test set, and prepare for model training, evaluation and testing, it is necessary to construct a dataset: the labeled dataset is recorded as , the training set, validation set, and test set are divided into 8:1:1 ratios.

[0135] In order to divide the labeled data into training set, validation set and test set to prepare for model training, evaluation and testing, the dataset needs to be According to the ratio of 8:1:1, the specific steps are as follows:

[0136] The labeled dataset Divided in the following proportions:

[0137] 80% of the data is used as training set

[0138] 10% of the data is used as a validation set.

[0139] 10% of the data is used as a test set (Test Set)

[0140] Such a division can ensure that there is sufficient data for parameter adjustment and optimization during the model training process, and perform evaluation of the validation set and final performance verification of the test set after model training to ensure the generalization ability and effectiveness of the model.

[0141] 3. Construction of underwater target detection model

[0142] 1. Feature extraction module:

[0143] The feature extraction module aims to extract effective features from the preprocessed image data and acoustic data for subsequent target detection tasks. The feature extraction module uses a feature extraction method based on multi-scale residual attention network (MS-RAN) and a hybrid method based on time-frequency analysis and deep learning to extract image and acoustic features respectively, and finally fuses the two features.

[0144] (1) Image feature extraction unit

[0145] In order to effectively extract the image features of underwater targets, a feature extraction method based on multi-scale residual attention network (MS-RAN) is proposed. The core idea of ​​multi-scale residual attention network (MS-RAN) is to capture target features of different scales through multi-scale feature extraction and residual connection, and use attention mechanism to emphasize important features. It includes input layer, multi-scale convolution layer, image feature fusion layer, residual block and attention mechanism layer.

[0146] Input layer: receives the preprocessed image

[0147] Multi-scale convolution layer: Use convolution kernels of different sizes to extract features in parallel. Use a 3x3 convolution kernel, a 5x5 convolution kernel, and a 7x7 convolution kernel to convolve the image. Extract features in parallel to obtain multi-scale features , and .

[0148]

[0149] in, represents a convolution operation of size 3x3, represents a convolution operation of size 5x5, Represents a convolution operation of size 7x7.

[0150] Image feature fusion layer: concatenate multi-scale features to obtain

[0151]

[0152] in, Represents a concatenation operation.

[0153] Residual block: Use multiple residual blocks to further extract features. The output of the residual block is:

[0154]

[0155] in, represents batch normalization, is the activation function. The residual block network structure is as follows Figure 3 shown.

[0156] Attention mechanism: Enhance features by introducing channel attention and spatial attention.

[0157]

[0158]

[0159] in represents the channel attention operation, represents the spatial attention operation, represents the sigmoid activation function, MLP is a multi-layer perceptron, represents the global average pooling operation, represents the average pooling operation, Represents a max pooling operation.

[0160] The attention mechanism is applied to the residual features to obtain the image features processed by the MS-RAN network and enhanced by the attention mechanism.

[0161]

[0162] Underwater feature enhancement mechanism:

[0163] In order to enhance the unique features of the underwater environment, an underwater image enhancement method based on a physical model is introduced. Perform color correction based on depth estimation:

[0164]

[0165] in, represents the color channel, is the attenuation coefficient of each channel, is the estimated depth map.

[0166] Afterwards, the enhanced features are weightedly fused with the original features to obtain the final image features:

[0167]

[0168] in, are learnable weight parameters.

[0169] (2) Acoustic feature extraction unit

[0170] Acoustic data plays a vital role in underwater target detection. Compared with optical data, sound waves travel farther in water and are less affected by water turbidity, so they can provide more reliable information about long-distance targets. In addition, acoustic data can also provide features such as the material and density of the target that are difficult to capture with optical images. In order to make full use of these advantages of acoustic data, a hybrid method based on time-frequency analysis and deep learning is used for feature extraction.

[0171] Time-frequency analysis:

[0172] First, the short-time Fourier transform (STFT) is used to process the raw acoustic data collected by the device. STFT can convert time-domain signals into time-frequency domain representations, and can simultaneously analyze the frequency content and time changes of the signal.

[0173] Next, the acoustic data processed by STFT is extracted through Mel Frequency Cepstral Coefficients (MFCC). MFCC is a feature extraction method that imitates the auditory characteristics of the human ear and can effectively extract the key features of the acoustic signal. The calculation process of MFCC includes the following steps:

[0174] 1) Perform Mel filter bank processing on the STFT result;

[0175] 2) Take the logarithm;

[0176] 3) Perform discrete cosine transform (DCT).

[0177] The acoustic data after time-frequency analysis is recorded as The purpose of this step is to convert the original acoustic signal into a form that is easier for subsequent deep learning models to process, while retaining the key time-frequency characteristics of the signal.

[0178] Deep Learning Feature Extraction:

[0179] In order to further extract high-level features of acoustic data, a network based on 1D-CNN and BiLSTM is designed. This hybrid network structure can capture both local features and long-term dependencies, which is very suitable for processing time series acoustic data.

[0180] First, a 1D-CNN (one-dimensional convolutional neural network) layer is used to extract The advantage of 1D-CNN is that it can effectively capture the local patterns and short-term dependencies of the signal. The operation of 1D-CNN can be expressed as:

[0181]

[0182] in, is a one-dimensional convolution operation, is the convolution kernel weight, is the bias term.

[0183] The BiLSTM (Bidirectional Long Short-Term Memory) layer is then used to capture long-term dependencies. BiLSTM can consider both past and future contextual information and is particularly suitable for processing sequence data with long-term dependencies. The forward propagation of BiLSTM can be simplified as:

[0184]

[0185] in, express The BiLSTM hidden layer state at time , express Input at the moment, is the hidden layer state at the previous moment.

[0186] Based on the final hidden layer state representation obtained, it is mapped to a feature space of a specific dimension through a fully connected layer to obtain the final acoustic feature This fully connected layer can be seen as a further abstraction and refinement of the hidden layer state, which can transform the time series information captured by the BiLSTM network into a more compact and effective feature representation. In this way, complex time series patterns can be compressed into vectors of fixed dimensions, making them easier to process and analyze later.

[0187] This multi-level feature extraction method can not only fully mine the rich information in acoustic data, but also present this information in a more structured and easy-to-use form, providing strong support for subsequent target detection tasks.

[0188] (3) Feature fusion unit

[0189] Feature fusion is a key step that aims to integrate information from different data to obtain a more comprehensive and robust target representation. A simple but effective feature concatenation method is used to achieve this goal:

[0190]

[0191] The advantage of this fusion method is that it can retain the original information of the two modalities and avoid information loss, while providing an opportunity for the subsequent detection network to automatically learn how to optimally utilize these two complementary information sources. In addition, this fusion method is simple and direct, with high computational efficiency, and is very suitable for implementation in unmanned boat systems with limited computing resources.

[0192] The overall network structure of the image feature extraction unit, acoustic feature extraction unit and feature fusion unit is as follows: Figure 4 shown.

[0193] (4) Feature Dimensionality Reduction Unit

[0194] First, principal component analysis (PCA) is used to reduce the dimensionality of the fusion features. PCA is a commonly used dimensionality reduction technique that aims to map data from a high-dimensional space to a low-dimensional space through a linear transformation while retaining important information in the data as much as possible. The initial dimensionality reduction features are obtained:

[0195]

[0196] in, is the number of principal components.

[0197] After the initial feature dimensionality reduction, the autoencoder is further used for deeper dimensionality reduction and denoising. The autoencoder consists of two parts: the encoder and the decoder. Its structural design is designed to learn the compressed representation and reconstruction of data through nonlinear mapping.

[0198] First, the feature matrix after principal component analysis (PCA) dimensionality reduction is Input to the encoder to get the encoded feature matrix , the role of the encoder is to map high-dimensional data into a lower-dimensional latent space, thereby achieving further compression of features. Input to the decoder to generate the reconstructed feature matrix The task of the decoder is to reconstruct the original feature data from the low-dimensional feature space, thereby ensuring that the information in the compression process can be effectively restored.

[0199] The training goal of the autoencoder is to minimize the following loss function:

[0200]

[0201] in, Represents the mean square error, which is used to measure the reconstruction features and the original dimension reduction features The goal of minimizing the mean square error is to make the reconstructed data as close to the original data as possible. The regularization term is used to encode the features Imposing sparsity constraints makes the feature representation more concise and prevents overfitting. Regularization parameter Control How much the regularization term affects the overall loss.

[0202] Through these steps, we finally get the feature matrix after dimensionality reduction and denoising. ,Right now This feature matrix is ​​not only compressed in dimension, but also retains the key information of the features through the denoising process, so that subsequent analysis and modeling can be carried out on a more concise data representation.

[0203] 2. Object Detection Module

[0204] First, the system uses a dynamic detection strategy to perform preliminary positioning of the target. Specifically, the adaptive region proposal network (ARPN) is used to dynamically adjust the detection proposal strategy. ARPN predicts the optimal anchor frame size and proportion through a lightweight meta-learning module, enabling it to adapt to different underwater scene changes. On this basis, the system also introduces an attention-based target localization (AGL) method, which strengthens the relevant features of the target and weakens background noise interference through spatial and channel attention mechanisms. The optimization of this link ensures that the system has stronger adaptability and accuracy in complex underwater environments.

[0205] After completing the preliminary positioning of the target area, the system enters the multi-task collaborative recognition and detection stage. This stage uses the Dynamic Attention Cascade Network (DACN) framework to decompose the detection problem into multi-level subtasks. Each subtask is processed by a dedicated sub-network, which gradually refines features at different scales and complexities, and performs classification and regression tasks simultaneously. The classification head outputs the probability distribution of the target category through the softmax activation function, and the regression head is responsible for accurately predicting the location coordinates of the target. Through this multi-task collaborative processing method, the model can provide more accurate detection results in complex scenes.

[0206] Finally, the system optimizes the detection results through non-maximum suppression (NMS) technology to reduce redundant detection frames. In order to further improve the processing effect in target-dense scenes, the system uses soft NMS technology to dynamically adjust the confidence of the candidate frame based on the intersection and union ratio of the candidate frame and the highest scoring frame, rather than directly deleting the overlapping frame. This step ensures the accuracy and reliability of the detection results. Subsequently, all detection results are converted into the standard format of the COCO dataset, including image identifiers, target category labels, location information, and detection confidence scores, providing standardized output for subsequent evaluation and application.

[0207] (1) Dynamic detection strategy

[0208] This strategy designs an Adaptive Region Proposal Network (ARPN), which can dynamically adjust its proposal strategy according to the characteristics of the input image. ARPN uses a lightweight meta-learning module to predict the optimal anchor box size and scale to adapt to different underwater scenes.

[0209]

[0210] in, represents the generated anchor box set, represents the meta-learning module, Represents a function that generates anchor boxes based on the parameters output by the meta-learning module.

[0211] This strategy also includes attention-guided Localization (AGL), which uses spatial and channel attention to enhance target-related features while suppressing background interference.

[0212]

[0213] in, It represents the features after AGL feature enhancement. represents the spatial attention map, represents the channel attention vector, represents element-wise multiplication, Represents tensor multiplication.

[0214] (2) Multi-task collaborative recognition and detection network framework

[0215] This paper proposes a novel multi-task collaborative target detection framework, called Dynamic Attention Cascade Network (DACN), which performs classification and regression tasks simultaneously. The core idea of ​​DACN is to decompose the target detection problem into a series of progressively refined subtasks, each of which is handled by a dedicated sub-network. This cascade structure allows the model to gradually refine features at different scales and complexity levels, thereby improving detection accuracy.

[0216] Among them, the classification head uses the softmax activation function to output the probability distribution of the target category:

[0217]

[0218] in, represents the target category probability distribution output by the classification head, A function representing a classification head.

[0219] The regression head uses the softmax activation function to predict the precise position coordinates of the target:

[0220]

[0221] in, represents the target position coordinates output by the regression head, A function representing a regression head.

[0222] (3) Output form and post-processing of target detection

[0223] This part includes non-maximum suppression (NMS) and output format conversion.

[0224] In object detection, the same object may be framed by multiple candidate boxes, especially in dense scenes, where multiple candidate boxes will overlap significantly. In this case, non-maximum suppression (NMS) is needed to reduce redundant boxes and only retain the boxes with the highest confidence. In order to improve detection accuracy in scenes with dense objects, soft NMS does not directly delete candidate boxes with high intersection-to-union ratios like standard NMS, but reduces the confidence scores of these boxes based on the union ratio. The goal of soft NMS is to handle overlapping candidate boxes more gently to avoid mistakenly deleting multiple valid detection boxes.

[0225]

[0226] in, Represents a candidate bounding box The initial confidence score of represents the current highest scoring bounding box, represents the bounding box with the highest score With candidate box The intersection ratio of Represents a hyperparameter that controls the degree of decay.

[0227] In the output of the object detection model, it is usually necessary to convert the detection results into a specific standard format for subsequent evaluation and application. Using the standard format of the COCO (Common Objects in Context) dataset, each detection result contains the following information:

[0228] A unique identifier for an image, indicating which input image this detection belongs to.

[0229] The target category label indicates which category the detected target belongs to.

[0230] The target position label is used to indicate the position of the target in the image.

[0231] The detection confidence score indicates the model's confidence in the detection result.

[0232] This standardized output format facilitates the comparison of model detection results with actual annotated data and is widely used in object detection evaluation tasks, such as calculating precision, recall, and mean average precision.

[0233] IV. Model Training and Deployment

[0234] 1. Model training:

[0235] Based on the design of feature extraction module and underwater target detection module, the labeled training data set is input into the model, and the model is trained end-to-end using autoregression method.

[0236] Data preprocessing: Import the training data set in batches, perform image denoising, image size unification and pixel value normalization on the input image data of each batch, and perform feature extraction and feature fusion on the image data and acoustic data to obtain the final feature representation. .

[0237] Define the loss function: The total loss function consists of classification loss and regression loss:

[0238]

[0239] in, and is the weight coefficient for balancing classification and regression tasks.

[0240] Classification Loss Using cross entropy loss:

[0241]

[0242] in, For the The model prediction output classification encoding result of samples, For the The classification labels of samples.

[0243] Regression Loss Smooth L1 loss is used: .

[0244]

[0245] in, For the The model prediction output of samples is the regression position result, For the The regressed position labels of samples.

[0246] Loss calculation and gradient update: Adopt Adam optimization algorithm, according to loss function Perform gradient updates on encoder / decoder parameters:

[0247]

[0248] in, Represents the parameters to be updated, which represents all the parameters to be updated in the unmanned ship underwater target detection model based on deep learning. is the learning rate, which controls the step size of each parameter update. Represents the loss function Parameters The gradient of , which represents the rate of change of the loss function with respect to the parameters. By multiplying by the learning rate , the parameters can be updated according to the direction and magnitude of the gradient , so that the loss function gradually decreases and the model can better adapt to the training data. The updated parameters It will be used in the next iteration of training process.

[0249] Adaptive learning rate adjustment: A novel adaptive learning rate adjustment mechanism is proposed, called "Oscillating Decay Learning Rate Scheduler" (ODLRS). This mechanism combines the advantages of periodic learning rate adjustment and gradual decay, which can better explore the parameter space during training while ensuring convergence to a better local minimum:

[0250]

[0251] in, Indicated in The learning rate for each epoch, represents the basic learning rate, Represents the total number of training epochs, Represents the decay rate.

[0252] The innovation of this method is that it combines cosine annealing and exponential decay. The cosine term makes the learning rate fluctuate periodically during training, while the exponential term ensures that the overall learning rate gradually decreases over time. The specific steps are:

[0253] 1) Initialize the basic learning rate 、Total number of epochs and decay rate .

[0254] 2) At the beginning of each epoch, the learning rate of the current epoch is calculated according to the above formula.

[0255] 3) Apply the calculated learning rate to the optimizer.

[0256] Training termination condition: According to the loss function value of the validation set, when the loss cannot be reduced for N consecutive epochs, the training process is terminated.

[0257] Model saving: Save the final model parameters at the end of training as the deployment model for unmanned ship underwater target detection.

[0258] 2. Model deployment:

[0259] Hardware platform selection and configuration:

[0260] (1) According to the actual needs of underwater target detection, select appropriate edge computing devices, such as NVIDIA Jetson Xavier, to facilitate real-time processing of image and acoustic data on underwater unmanned vehicles.

[0261] (2) Configure the necessary deep learning frameworks and dependent libraries on the selected hardware platform: CUDA, cuDNN, etc. to support the efficient operation of the model.

[0262] (3) Optimize hardware resource allocation and reasonably set GPU memory usage limits and CPU thread numbers to balance detection performance and system stability.

[0263] Model integration and interface development:

[0264] (1) Design and implement the model inference interface, including image preprocessing, model inference, and result post-processing functional modules.

[0265] (2) Develop communication interfaces with cameras and sonar control systems to achieve real-time transmission and feedback of detection results.

[0266] (3) Build a caching mechanism, optimize data stream processing, and enhance detection speed.

[0267] Real-time data collection and preprocessing:

[0268] (1) Collect images and acoustic data in real time through cameras and sonar to ensure that data quality and acquisition frequency meet detection requirements.

[0269] (2) Implement image preprocessing, including image denoising, standardization and normalization, to improve the quality of the input image.

[0270] Target detection and result output:

[0271] (1) The preprocessed image data and acoustic data are input into the deployed underwater target detection model to perform inference operations.

[0272] (2) Parse the model output and extract the target classification label and target location label. Figure 5 shown.

[0273] (3) Screen and grade the detection results according to preset thresholds to reduce false positives and false negatives.

[0274] Visual storage of test results:

[0275] (1) Develop a real-time visualization interface to intuitively display the detection results, including target classification labels, target location labels, and confidence levels.

[0276] (2) Realize the local storage function of detection results, including original images, detection results and related data.

[0277] Performance evaluation and continuous optimization:

[0278] (1) Establish a regular performance evaluation mechanism, including statistical analysis of key indicators of detection accuracy and processing speed.

[0279] (2) Develop an automated testing process and use standard test sets to evaluate the performance of the model in actual working environments.

[0280] (3) Based on performance evaluation results and actual feedback, continuously optimize model parameters and deployment strategies to continuously improve the overall performance of the detection system.

[0281] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0282] Although the above describes the specific implementation methods of the present invention, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for detecting underwater targets of an unmanned boat, characterized in that: The following steps are involved: S1, acquiring underwater target data, including underwater image data and synchronously acquired underwater acoustic data; S2, preprocessing the image data and acoustic data respectively to obtain standard input data for underwater target detection model training; S3, inputs the standard input data into the trained underwater target detection model, and outputs the target classification label and target location label; The underwater target detection model includes a feature extraction module and a target detection module; The feature extraction module includes an image feature extraction unit and an acoustic feature extraction unit connected in parallel, and a feature fusion unit and a feature dimension reduction unit connected in series at the back end of the feature extraction unit; the image feature extraction unit captures target features of different scales through multi-scale feature extraction and residual connection, and uses an attention mechanism to emphasize important features; the acoustic feature extraction unit uses a hybrid method based on time-frequency analysis and deep learning to extract acoustic features; the feature fusion unit fuses the extracted image features and acoustic features into multi-modal feature vectors, and performs dimension reduction and denoising through the feature dimension reduction unit to obtain a feature matrix ; The target detection module uses a multi-task collaborative recognition detection network framework and a dynamic detection strategy; the multi-task collaborative recognition detection network is based on the input feature matrix , target classification and position regression are performed simultaneously, and the dynamic detection strategy adaptively adjusts the detection window and step size according to the size, shape and motion characteristics of the target; The multi-task collaborative recognition and detection network framework decomposes the object detection problem into a series of gradually refined subtasks, each of which is handled by a dedicated sub-network, allowing the model to gradually refine features at different scales and complexity levels; Among them, the classification head uses the softmax activation function to output the probability distribution of the target category: represents the target category probability distribution output by the classification head, A function representing a classification head; Among them, the regression head uses the softmax activation function to predict the precise position coordinates of the target: represents the target position coordinates output by the regression head, A function representing a regression head; The dynamic detection strategy is based on the designed adaptive region proposal network ARPN, which dynamically adjusts its proposal strategy according to the characteristics of the input image; the ARPN uses a lightweight meta-learning module to predict the optimal anchor box size and scale to adapt to different underwater scenes: in, represents the generated anchor box set, represents the meta-learning module, Represents a function that generates anchor boxes based on the parameters output by the meta-learning module; At the same time, attention-based object localization AGL uses spatial and channel attention to enhance target-related features while suppressing background interference: in, It represents the features after AGL feature enhancement. represents the spatial attention map, represents the channel attention vector, represents element-wise multiplication, Represents tensor multiplication; The target detection module also includes non-maximum suppression: Use soft NMS to reduce redundant boxes based on the union ratio and only keep the boxes with the highest confidence: in, Represents a candidate bounding box The initial confidence score of represents the current highest scoring bounding box, represents the bounding box with the highest score With candidate box The intersection ratio of represents the hyperparameter that controls the degree of attenuation; When training the underwater target detection model, the designed adaptive learning rate adjustment mechanism is introduced: in, Indicated in The learning rate for each epoch, represents the basic learning rate, Represents the total number of training epochs, represents the decay rate; Combining cosine annealing and exponential decay, the cosine term makes the learning rate fluctuate periodically during training, while the exponential term ensures that the overall learning rate decreases gradually over time.

2. The method for detecting underwater targets of an unmanned boat according to claim 1, characterized in that: The preprocessing of the image data includes image denoising, image size unification and pixel value normalization; The image denoising is performed by using a bilateral filter to perform denoising processing, and weighted average pixel values ​​in a neighborhood are calculated according to the spatial distance between pixels and the difference in pixel values ​​to smooth the image; the image size unification is to adjust all images to a uniform size, and use a bilinear interpolation method to calculate new pixel values ​​by linear interpolation during resampling; The pixel value normalization is to scale the pixel value to the range of [0, 1]; The preprocessing of the acoustic data includes using a Gaussian filtering algorithm to remove noise in the audio.

3. The method for detecting underwater targets of an unmanned boat according to claim 1, characterized in that: The image feature extraction unit includes an input layer, a multi-scale convolution layer, an image feature fusion layer, a residual block and an attention mechanism layer; The input layer receives the preprocessed image ; The multi-scale convolution layer uses a 3x3 convolution kernel, a 5x5 convolution kernel and a 7x7 convolution kernel to image Extract features in parallel to obtain multi-scale features , and : in, represents a convolution operation of size 3x3, represents a convolution operation of size 5x5, Represents a convolution operation of size 7x7; The image feature fusion layer performs splicing operations on multi-scale features to obtain : in, Represents a splicing operation; The residual block uses multiple residual blocks to further extract features, and the output of the residual block is: in, represents batch normalization, is the activation function; The attention mechanism enhances features by introducing channel attention and spatial attention: in represents the channel attention operation, represents the spatial attention operation, represents the sigmoid activation function, MLP is a multi-layer perceptron, represents the global average pooling operation, represents the average pooling operation, Represents the maximum pooling operation; Apply the attention mechanism to the residual features, and the image features enhanced by the attention mechanism .

4. The method for detecting underwater targets of an unmanned boat according to claim 3, characterized in that: The image feature extraction unit also includes an underwater feature enhancement mechanism, which introduces an underwater image enhancement method based on a physical model; First of all, Perform color correction based on depth estimation: in, represents the color channel, is the attenuation coefficient of each channel, is the estimated depth map, To get the exponential operator with base e, Image features enhanced by the attention mechanism; Afterwards, the enhanced features are weightedly fused with the original features to obtain the final image features: in, is the learnable weight parameter, It is the feature map after color correction.

5. The method for detecting underwater targets of an unmanned boat according to claim 1, characterized in that: The acoustic feature extraction unit first undergoes time-frequency analysis: Short-time Fourier transform is used to convert the time domain signal into time-frequency domain representation. Next, the acoustic data processed by STFT is extracted through Mel-frequency cepstral coefficients (MFCC) to convert the original acoustic signal into a form that is easy to process by deep learning models while retaining the key time-frequency features of the signal. Then perform deep learning feature extraction based on the 1D-CNN and BiLSTM networks: first One-dimensional convolutional neural network layer 1D-CNN is used to extract The local features of , the operation of 1D-CNN is expressed as: in, is a one-dimensional convolution operation, is the convolution kernel weight, is the bias term; The BiLSTM layer is then used to capture long-term dependencies, and the forward propagation of BiLSTM is simplified as follows: in, express The BiLSTM hidden layer state at time , express Input at the moment, is the hidden layer state at the previous moment; Based on the final hidden layer state representation obtained, it is mapped to a feature space of a specific dimension through a fully connected layer to obtain the final acoustic feature , transforming the temporal information captured by the BiLSTM network into a more compact and effective feature representation.

6. The method for detecting underwater targets of an unmanned boat according to claim 1, characterized in that: The specific processing process of the feature dimension reduction unit is as follows: First, principal component analysis (PCA) is used to reduce the dimensionality of the fusion features. The data is mapped from high-dimensional space to low-dimensional space through linear transformation, while retaining important information in the data as much as possible to obtain preliminary dimensionality reduction features: in, is the number of principal components; Then, the autoencoder is used to further perform dimensionality reduction and denoising. The autoencoder consists of two parts: the encoder and the decoder. Input to the encoder to get the encoded feature matrix , mapping the high-dimensional data into a lower-dimensional latent space, and then Input to the decoder to generate the reconstructed feature matrix , reconstruct the original feature data from the low-dimensional feature space; The training goal of the autoencoder is to minimize the following loss function: in, Represents the mean square error, which is used to measure the reconstruction features and the original dimension reduction features The goal of minimizing the mean square error is to make the reconstructed data as close to the original data as possible. The regularization term is used to encode the features Imposing sparsity constraints, regularization parameters Control How much the regularization term affects the overall loss.

Citation Information

Patent Citations

  • SAR image denoising method based on multi-scale residual attention network

    CN112233026A

  • Hyperspectral remote sensing image classification method

    CN113705526A

  • AUV pose estimation method based on underwater light source visual guidance

    CN115797450A

  • Underwater operation and deep detection method of unmanned ship loaded with soft robot

    CN117875031A

  • Track prediction method and system based on global geographic information coding PlusCode and Transformer

    CN118585961A