A random mask-based forward-looking sonar image target recognition method

Abstract

The present application relates to the technical field of sonar system, and particularly relates to a forward-looking sonar image target recognition method based on random mask. The method comprises the following steps: target image data and sonar image data are respectively divided into training set and test set; the training set and the test set of the target image data are preprocessed, masks are randomly added on the target image to simulate the structural defects caused by the occlusion or signal interference of underwater targets; a target detection model is constructed, a YOLO V9 model is taken as a backbone, and an attention mechanism module is fused; a loss function and an optimization method are selected; pre-training parameter operation is performed on the target image training set after the mask is added, and the pre-training model is tested by using the target image test set; the pre-training model parameters are fine-tuned by using the sonar image training set, and the model is tested by using the sonar image test set. The present application realizes rapid and accurate real-time target detection by combining the attention mechanism with the way of randomly adding masks on the target image.

Description

Technical Field

[0001] This invention relates to the field of sonar system technology, and more specifically to a method for target recognition of forward-looking sonar images based on random masks. Background Technology

[0002] With the continuous growth of the global population, the increasing scarcity of natural resources, and the ongoing environmental degradation, traditional land resources are no longer sufficient to meet humanity's growing needs, urgently requiring the exploration of new frontiers—the ocean. Unmanned underwater vehicles (UUVs), as crucial tools for exploring and developing marine resources, have become a key technology of concern to major maritime powers. Sonar systems, serving as the "eyes and ears" of UUVs, play an indispensable role in achieving long-term, stable, and comprehensive underwater observation and operations. Sonar systems are generally classified into three categories: detection sonar, navigation sonar, and communication sonar. Detection sonar is primarily used to detect underwater targets and to conduct underwater topography, geomorphology, and geological surveys and mapping.

[0003] In underwater operations, unmanned underwater vehicles (UUVs) rely on sensors such as forward-looking sonar to achieve efficient and accurate environmental perception. This is crucial for ensuring the safety of ship navigation, the safe operation of offshore facilities, and the effective development of marine resources. However, underwater channels and ports often contain many unknown and dangerous obstacles, such as reefs, shipwrecks, and offshore development platforms, all of which can threaten the safety of UUVs. Therefore, from a navigation safety perspective, UUVs must be equipped with forward-looking sonar systems. Simultaneously, to enable UUVs to perform tasks such as autonomous guidance, target identification, and obstacle avoidance, building an efficient target identification system is essential.

[0004] In the field of target recognition using forward-looking sonar images, deep learning technology has become the mainstream approach. However, traditional deep learning models face a series of challenges in underwater target recognition. First, obtaining high-quality training datasets is difficult; the unique characteristics of underwater images and the scarcity of data complicate the training process. Second, underwater targets often present incomplete shapes due to occlusion or blurring, increasing the difficulty of target detection. Finally, considering real-time requirements, the model needs to maintain high accuracy while ensuring fast processing speed; achieving this balance is a significant technical challenge.

[0005] Therefore, we need to improve the existing target recognition system to solve the problem of incomplete target shape due to occlusion or blurring, and improve the accuracy of recognition. Summary of the Invention

[0006] The problem to be solved by the present invention is to provide a method for target recognition in forward-looking sonar images to solve the problem of incomplete shape of the target due to occlusion or blurring, thereby improving the accuracy of recognition.

[0007] To address the shortcomings of existing technologies, the technical solution adopted by this invention is: a forward-looking sonar image target recognition method based on random masks, comprising the following steps:

[0008] S1: Collect and label the dataset, and divide the target image data and sonar image data into training set and test set according to a certain ratio;

[0009] S2: Preprocess the training and test sets of the target image data, and randomly add masks to the target image to simulate structural defects caused by underwater target occlusion or signal interference.

[0010] S3: Construct an object detection model, using the YOLO V9 model as the backbone and integrating the attention mechanism module;

[0011] S4: Select the appropriate loss function and optimization method according to the requirements of the solution;

[0012] S5: Perform pre-training parameter operations on the target image training set after adding the mask, and test the pre-trained model using the target image test set;

[0013] S6: Fine-tune the parameters of the pre-trained model using the sonar image training set, and test the model using the sonar image test set.

[0014] Preferably, in S1, bounding boxes are drawn on the target image data using LabelImg annotation software to annotate the target information; bounding boxes are drawn on the targets in the sonar image data using LabelImg annotation software to indicate the target category and other information; the annotated data is divided into training set and test set in a 7:3 ratio.

[0015] Preferably, in S2, a mask is added to the target image. First, several pixels are randomly selected from the target area in the image. These pixel values ​​will serve as the basis for adding the mask. Then, several pixels are randomly selected from all pixels in the target area as the center of the mask. These randomly selected pixels will serve as the core of the mask in subsequent steps, determining the influence range and position of the mask.

[0016] Preferably, in S3, based on the YOLO V9 model, a spatial attention module is fused in the shallow layer of the model to focus on image detail information; in the middle layer of the model, spatial and channel attention are combined to comprehensively improve local details and global information; and in the deep layer of the model, a channel attention module is fused to highlight global features. Through this layered design, YOLO V9 can flexibly use spatial and channel attention at different network layers, helping the network improve the detection accuracy and robustness of targets in various complex scenes.

[0017] Preferably, the loss function selected in S4 is the MPDIoU loss function, which can be expressed by the following formula:

[0018] LMPDIoU=1-IoUmax+λ·Distance(Bp,Bg);

[0019] Where IoUmax is the maximum intersection-union ratio between the predicted box and the ground truth box; Distance(Bp,Bg) is the maximum probabilistic distance metric between the predicted box Bp and the ground truth box Bg; λ is a learnable parameter that controls the impact of the distance metric on the loss function.

[0020] Preferably, in S5, before pre-training the masked target image, all images are normalized and standardized.

[0021] Preferably, S6 includes a fine-tuning stage, which focuses on optimizing the model's adaptability to sonar images by adjusting the learning rate and freezing some layers, while avoiding overfitting.

[0022] The beneficial effects of this invention are as follows:

[0023] (1) The technical solution of the present invention introduces transfer learning. By using a large amount of target image data to pre-train neural network parameters, and then using sonar image data to fine-tune these parameters, the problem of underfitting model training caused by insufficient data is effectively solved.

[0024] (2) In the data preprocessing stage, masks are randomly added to the target image to enhance the occluded parts of the target. This method can increase the diversity and complexity of the data, helping the model learn the complete semantic information of the target. When processing the occluded target image, the model can accurately identify the target category by deeply mining the deep features of the target. This method simulates the situation in real-world applications where the image is not fully displayed due to signal interference or target occlusion, thereby improving the model's ability to learn target features and enhancing the model's robustness and generalization ability. At the same time, by randomly occluding different positions of the target, the dataset can be expanded, further alleviating the problem of insufficient data.

[0025] (3) The method of the present invention uses YOLO V9 as the basic model and combines the attention mechanism for structural optimization to enhance the ability to extract target features, thereby achieving fast and accurate real-time target detection, which is especially suitable for target recognition in forward-looking sonar images. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating the method of the present invention;

[0027] Figure 2 These are example images of the target image before and after masking according to the method of this invention;

[0028] Figure 3 This invention relates to a model training flowchart;

[0029] Figure 4 This is the method of incorporating attention mechanism into the model in this invention. Detailed Implementation

[0030] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. The embodiments of the present invention are given for illustrative and descriptive purposes only, and are not intended to be exhaustive or to limit the invention to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described to better illustrate the principles and practical application of the invention, and to enable those skilled in the art to understand the invention and design various embodiments with various modifications suitable for a particular purpose.

[0031] As described in the background art, traditional deep learning models face a series of challenges in underwater target recognition. Obtaining high-quality training data is difficult, and the unique characteristics and scarcity of underwater images further complicate the training process. Moreover, underwater targets often present incomplete shapes due to occlusion or blurring, increasing the difficulty of target detection. Simultaneously, underwater target recognition demands high real-time performance and accuracy. To address the problems mentioned in the background art, the technical solution adopted in this invention is: a method for efficiently detecting and recognizing sonar images with insufficient data and target occlusion; a forward-looking sonar image target recognition method based on random masks, comprising the following steps:

[0032] S1: Collect and label the dataset. Divide the target image data and sonar image data into training and test sets according to a certain ratio. Collect target image data containing targets such as airplanes, shipwrecks, and rocks. Use LabelImg annotation software to draw bounding boxes on the images and label the target information. Use LabelImg annotation software to draw bounding boxes on the targets in the sonar image data and indicate the target category and other information. Divide the labeled data into training and test sets in a 7:3 ratio.

[0033] In S1, the open-source data annotation tool LabelImg is used for data annotation, and the labels are saved as txt files in YOLO format. If there are multiple targets in the image, it is ensured that each target is correctly labeled and that the same target is not labeled repeatedly. For target image data, data augmentation techniques such as rotation, cropping, and scaling are applied to generate more training samples. Different image augmentation methods are used for target image data and sonar image data to enhance data diversity. For sonar image data, Gaussian noise is added to enhance the dataset. The dataset is divided into training and test sets using k-fold cross-validation. The dataset is divided in a 7:3 ratio and the data is split multiple times to avoid instability in model evaluation results due to imbalanced data partitioning or random factors.

[0034] S2: Preprocess the training and test sets of the target image data by randomly adding masks to the target images to simulate structural defects caused by underwater target occlusion or signal interference. During model pre-training, by randomly adding masks to the target images to simulate structural defects caused by underwater target occlusion or signal interference, the model is prompted to delve deeper into the deep features of the target and accurately identify the target category, thereby improving the model's ability to learn target features and enhancing the model's robustness and generalization ability.

[0035] To add a mask to the target image, first, randomly select several pixels from the target region in the image. Assume the bounding box coordinates of the target are (x_min, y_min) and (x_max, y_max), where (x_min, y_min) are the top-left corner coordinates and (x_max, y_max) are the bottom-right corner coordinates. Extract the pixels of the target region from the image based on the bounding box; these pixel values ​​will serve as the basis for adding the mask: `target_pixels = image[y_min:y_max, x_min:x_max]`. Then, randomly select several pixels from all pixels in the target region as the center of the mask. These randomly selected pixels will serve as the core of the mask in subsequent steps, determining the mask's influence range and position: `numpy.random.choice(target_pixels)`.

[0036] The masked region is typically filled with black or other values ​​(e.g., pure white or 0% opacity) and covers part of the target's pixels, with a quantity of 10 and a size of 50*50 pixels. To further enhance the diversity of the dataset, the above steps can be repeated for each image, with a certain random seed set. Each run can generate multiple different occlusion samples for each target, resulting in multiple images with different occlusion methods, helping the model learn more complex target features. After completing the above operations, the images with added masks are saved, and a new label file is generated for each image. These new images will be used as part of the dataset to train the model, improving the model's adaptability to occlusions in real-world environments.

[0037] like Figure 2 The image shown is an example of the target image before and after masking according to the method of the present invention. Ten pixels are randomly selected from the pixels of the target image as the center, and the image is expanded outward to form a rectangle with a size of 50*50 pixels and filled with black to form the masked image.

[0038] S3: Construct an object detection model, using the YOLO V9 model as the backbone and integrating the attention mechanism module;

[0039] In S3, YOLO V9 introduced Procedural Gradient Information (PGI), providing reliable gradient information to ensure the network fully utilizes the features of each layer during training and effectively updates the model's weights. YOLO V9 employs a novel network architecture—Generalized Efficient Layer Aggregation Network (GELAN)—which optimizes convolutional operations and feature propagation, making the network more efficient, reducing unnecessary computation, and improving training and inference speed. In YOLO V9, PGI enhances training stability and model learning performance by improving gradient information propagation, while GELAN improves computational efficiency and multi-layered feature perception capabilities by optimizing convolutional operations and hierarchical feature aggregation. The combination of these two technologies drives YOLO V9 to achieve higher performance in object detection tasks while ensuring the model's efficiency and robustness in practical applications.

[0040] Building upon the YOLO V9 model, a spatial attention mechanism is inserted after the first few convolutional layers. By learning the importance of spatial features in the image, the model can automatically focus on regions containing the target. In the middle layers of the model, spatial attention is first applied after each convolutional layer to enhance the spatial features of the image region. Then, channel attention is applied to enhance the focus on important channels. The method of fusing spatial and channel attention modules is as follows: Figure 4As shown, this dual attention mechanism (spatial first, then channel) enables finer-grained feature extraction, improving the ability to perceive medium-sized targets. Deeper in the network model, channel attention strengthens the weights of feature map channels. This helps the model focus on more discriminative features at higher levels, improving the detection capability for large objects or long-distance targets in complex scenes.

[0041] S4: Select a suitable loss function and optimization method based on the requirements of this technology;

[0042] In S4, the MPDIoU loss function is selected. The MPDIoU loss function maximizes the relative distance and region overlap between the predicted bounding box and the ground truth bounding box. The relative distance is optimized by calculating the "maximum probability distance" between the predicted bounding box and the ground truth bounding box; the region overlap is measured by maximizing the ratio of the overlapping region (intersection) of the bounding boxes to the overall region (union), i.e., IoU.

[0043] The MPDIoU loss function involved in step 4 can be expressed by the following formula:

[0044] LMPDIoU=1-IoUmax+λ·Distance(Bp,Bg)

[0045] Here, IoUmax is the maximum intersection-union ratio between the predicted and ground truth boxes; Distance(Bp,Bg) is the maximum probabilistic distance metric between the predicted box Bp and the ground truth box Bg; λ is a learnable parameter that controls the impact of the distance metric on the loss function. In the model's loss function, λ is set as a learnable parameter, allowing the model to automatically learn the optimal value of λ during training. This allows for dynamic adjustment of this hyperparameter based on different datasets and task requirements. lambda_param = nn.Parameter(torch.tensor(init_value)), where init_value is the initial value of λ, set to 1.0. During model training, the value of λ is updated through backpropagation, just like other network weights. It automatically learns the balance between IoU loss and position loss based on the training data, eliminating the need for manual adjustment of the λ value and thus optimizing the model training process.

[0046] S5: Perform pre-training parameter operations on the target image training set after adding the mask, and test the pre-trained model using the target image test set.

[0047] In S5, before pre-training the masked target image, the pixel values ​​of all images are normalized to the range [0,1] and standardized according to the dataset statistics (subtracting the mean and dividing by the standard deviation) to ensure the consistency of the model's input data. However, due to the small sample size of this dataset, there is a risk of statistical instability and overfitting when calculating the dataset mean and standard deviation. Therefore, using the standard ImageNet mean (mean = [0.485, 0.456, 0.406]) and standard deviation (std = [0.229, 0.224, 0.225]) can avoid model bias to the dataset and enhance the model's generalization ability.

[0048] The batch size is set to 16; a smaller batch size can improve the model's generalization ability. 50 epochs are selected; if the loss function continuously decreases during training without signs of overfitting, the number of epochs can be increased appropriately. If the model's validation loss stops decreasing, training can be stopped early. The Adam optimizer is chosen to optimize the model; this optimizer is commonly used for object detection tasks, performs well in most cases, and can adaptively adjust the learning rate. The learning rate is dynamically reduced using `steplr`: `torch.optim.lr_scheduler.StepLR(optimizer, step_size=10, gamma=0.1)`.

[0049] In S5, model training is performed cyclically. At the beginning of each epoch, training is conducted using the training set, and the network weights are updated via the optimizer. After each epoch, the model's performance is tested on the validation set, and metrics such as loss and accuracy are recorded. Training is terminated early if the loss on the validation set no longer decreases significantly. Before pre-training on the masked target images, normalization and standardization methods are applied to all images to accelerate model training and make the model more stable to changes in the input images. During the pre-training phase, a larger epoch and a moderate batch size are used, and an appropriate learning rate and optimizer are selected. By monitoring the loss function, accuracy, and other evaluation metrics, the model's performance on the dataset is ensured to reach its optimal level.

[0050] S6: Fine-tune the parameters of the pre-trained model using the sonar image training set, and test the model using the sonar image test set;

[0051] Due to signal interference and the complex underwater environment, targets in sonar images are partially obscured, thus masking is unnecessary. However, when fine-tuning the model parameters, it is necessary to normalize the pixel values ​​of all images to the range [0,1] and standardize them according to the statistics of the dataset (subtract the mean and divide by the standard deviation) to ensure the consistency of the model input data. However, due to the small sample size of this dataset, there is a risk of statistical instability and overfitting when calculating the dataset mean and standard deviation. Therefore, using the standard ImageNet mean (mean = [0.485, 0.456, 0.406]) and standard deviation (std = [0.229, 0.224, 0.225]) can avoid model bias to the dataset and enhance the model's generalization ability.

[0052] In S6, since the target image training phase is already quite good, the first few layers of the model can be frozen, and only the parameters of the last few layers can be trained. This effectively reduces training time while preserving the general features learned in the pre-trained model: `for paramin model.backbone.parameters():param.requires_grad=False`. During the fine-tuning phase, the learning rate is set to 0.0001 because the model is already close to its optimal state, so fine-tuning will not destroy existing knowledge. Epochs are set to 20, and the batch size to 16; in fine-tuning mode, smaller epochs can achieve better performance. The model is trained cyclically, with training performed at each epoch and testing on a sonar image validation set at the end of each epoch to observe performance improvements. Before pre-training on sonar images, normalization and standardization methods are applied to all images, which helps accelerate model training and makes the model more stable to changes in input images. During the fine-tuning phase, by adjusting the learning rate and freezing some layers, the focus is on optimizing the model's adaptability to sonar images while avoiding overfitting. By monitoring the loss function, accuracy, and other evaluation metrics, we ensure that the model performs optimally on both datasets.

[0053] To overcome the aforementioned challenges, this paper proposes an improved target recognition system based on the YOLO V9 algorithm. We employ a transfer learning strategy, using collected image data of aircraft, shipwrecks, and rocks for model pre-training, and then fine-tuning the model parameters using real sonar data to compensate for the limitations of the underwater dataset. Furthermore, to improve the model's generalization ability and simulate situations where underwater targets may be obscured due to complex environments, we introduce a random masking technique. This technique not only enhances the model's adaptability in complex scenes but also significantly improves the model's computational efficiency while maintaining recognition accuracy, meeting the requirements of real-time target detection.

[0054] like Figure 3 The diagram shows the model training flowchart of the present invention. First, in the pre-training parameter stage, the target image data, after masking, is pre-processed and input into the YOLO V9 model incorporating an attention mechanism module, ultimately outputting the target bounding box, category, and confidence score. Next is the parameter fine-tuning interface, where the unmasked sonar image data, after image preprocessing, is input into the pre-trained model, finally outputting the target bounding box, category, and confidence score.

[0055] By randomly adding masks to the target image and enhancing the occluded portions, this method increases data diversity and complexity, helping the model learn the complete semantic information of the target. When processing occluded target images, the model can accurately identify the target category by deeply mining the target's deep features. This method simulates real-world scenarios where incomplete image display is caused by signal interference or target occlusion, thereby improving the model's ability to learn target features and enhancing its robustness and generalization ability. Simultaneously, by randomly occluding different locations of the target, the dataset can be expanded. Based on the YOLO V9 model, structural optimization using an attention mechanism enhances the ability to extract target features, achieving fast and accurate real-time target detection, particularly suitable for target recognition in forward-looking sonar images. This invention effectively solves the problems of data scarcity and incomplete target detection in underwater target detection, providing solid technical support for the autonomous navigation and safe operation of underwater unmanned vehicles.

Claims

1. A method for target recognition in forward-looking sonar images based on random masks, characterized in that: Includes the following steps, S1: Collect and label the datasets, dividing the target image data and sonar image data into training and test sets according to a certain ratio. The target image dataset is used to characterize the geometric and shape features of the target under complete structural conditions. The LabelImg annotation software is used to draw bounding boxes on the images and annotate the target information. The forward-looking sonar image dataset is used to characterize the incomplete echo structure features of the target under sonar imaging conditions due to limited sound wave propagation. The LabelImg annotation software is used to draw bounding boxes on the targets in the sonar image data and indicate the target category information. Finally, the labeled data is divided into training and test sets in a 7:3 ratio. S2: Preprocess the training and test sets of the target image data, and randomly add a mask to the target image to simulate structural defects caused by underwater target occlusion or signal interference. In S2, the mask is added to the target image by first randomly selecting a number of pixels from the target region in the image. These pixel values ​​will serve as the basis for adding the mask. Then, among all the pixels in the target region, a number of pixels are randomly selected as the center of the mask. These randomly selected pixels will serve as the core of the mask in subsequent steps, determining the influence range and position of the mask. S3: Constructing an object detection model with the YOLO V9 model as the backbone and integrating an attention mechanism module; In S3, based on the YOLO V9 model, a spatial attention module is integrated into the shallow layers of the model to focus on image detail information; in the middle layers of the model, spatial and channel attention are combined to comprehensively improve local details and global information; in the deep layers of the model, a channel attention module is integrated to highlight global features; Through this layered design, YOLO V9 flexibly uses spatial and channel attention at different network layers, enabling the network to improve the accuracy and robustness of object detection in various complex scenes; S4: Based on the requirements of the solution, select an appropriate loss function and optimization method; in S4, the MPDIoU loss function is selected, which is expressed by the following formula: LMPDIoU=1-IoUmax+λ•Distance(Bp,Bg); Where IoUmax is the maximum intersection-union ratio between the predicted box and the ground truth box; Distance(Bp,Bg) is the maximum probabilistic distance metric between the predicted box Bp and the ground truth box Bg; λ is a learnable parameter that controls the impact of the distance metric on the loss function. S5: Perform pre-training parameter operations on the training set of target images after masking, and test the pre-trained model using the test set of target images; In S5, before pre-training the masked images, all images are processed using normalization and standardization methods; The mean used for standardization is [0.485, 0.456, 0.406], and the standard deviation is [0.229, 0.224, 0.225]; S6: Fine-tune the pre-trained model parameters using the sonar image training set and test the model using the sonar image test set; S6 includes a fine-tuning phase, which focuses on optimizing the model's adaptability to sonar images by adjusting the learning rate and freezing some layers, while avoiding overfitting.

2. The forward-looking sonar image target recognition method based on random mask as described in claim 1, characterized in that: In S2, a mask block of a predetermined size is generated. Specifically, a rectangular mask area of ​​50×50 pixels is generated centered on a randomly selected pixel and filled with black.

Images