Forward-looking sonar image target identification method based on random mask

Abstract

The invention relates to the technical field of sonar systems, in particular to a forward-looking sonar image target recognition method based on a random mask. Comprising the following steps: respectively dividing target image data and sonar image data into a training set and a test set; preprocessing the training set and the test set of the target image data, randomly adding a mask on a target image, and simulating structural defects caused by shielding or signal interference of an underwater target; constructing a target detection model, taking a YOLO V9 model as a backbone, and fusing an attention mechanism module; selecting a loss function and an optimization method; performing pre-training parameter operation on the target image training set after the mask is added, and testing a pre-training model by using a target image test set; and using the sonar image training set to finely adjust parameters of the pre-training model, and using the sonar image test set to test the model. According to the method, rapid and accurate real-time target detection is realized in combination with an attention mechanism in a mode of randomly adding the masks on the target image.

Description

Technical Field

[0001] The present invention relates to the technical field of sonar systems, and in particular to a forward-looking sonar image target recognition method based on random masks. Background Art

[0002] With the continuous growth of the global population, the increasing scarcity of natural resources and the continuous deterioration of the environment, traditional land resources can no longer meet the growing needs of mankind, and there is an urgent need to open up a new field - the ocean. As an important tool for exploring and developing marine resources, underwater unmanned vehicles (UUVs) have become one of the key technologies that major marine powers are paying attention to. Sonar systems, as the "eyes and ears" of underwater unmanned vehicles, play an indispensable role in achieving long-term, stable and all-round observation and operation underwater. Sonar systems are generally divided into three categories: detection sonar, navigation sonar and communication sonar. Among them, detection sonar is mainly used to detect underwater target information and conduct underwater terrain, landform and geological surveys and mapping.

[0003] In underwater operations, underwater unmanned vehicles rely on sensors such as forward-looking sonar to achieve efficient and accurate environmental perception. This is of great significance for ensuring the safety of ship navigation, the safety of offshore facility operations, and the effective development of marine resources. However, there are often many unknown dangerous obstacles in underwater waterways and ports, such as reefs, sunken ships, marine development platforms, etc., which may pose a threat to the safety of underwater unmanned vehicles. Therefore, from the perspective of navigation safety, underwater unmanned vehicles must be equipped with forward-looking sonar systems. At the same time, in order to achieve tasks such as autonomous guidance, target recognition, and obstacle avoidance of underwater unmanned vehicles, it is crucial to build an efficient target recognition system.

[0004] In the field of target recognition in forward-looking sonar images, deep learning technology has become the mainstream method. However, traditional deep learning models face a series of challenges in underwater target recognition. First, it is difficult to obtain high-quality training data sets. The particularity of underwater images and the scarcity of data make the training process more complicated. Second, underwater targets often present incomplete forms due to occlusion or blur, which increases the difficulty of target detection. Finally, considering the real-time requirements, the model needs to have high accuracy while ensuring fast processing speed. Achieving this balance is a huge technical challenge.

[0005] Therefore, we need to improve the existing target recognition system to solve the problem of incomplete shape of targets due to occlusion or blur in the existing system and improve the accuracy of recognition. Summary of the invention

[0006] The problem to be solved by the present invention is to provide a method for forward-looking sonar image target recognition to solve the problem of incomplete shape of the target due to occlusion or blur, thereby improving the high accuracy of recognition.

[0007] In view of the shortcomings of the prior art, the present invention solves the technical problems by adopting a technical solution: a forward-looking sonar image target recognition method based on a random mask, comprising the following steps:

[0008] S1: Collect and annotate data sets, and divide target image data and sonar image data into training set and test set according to a certain ratio;

[0009] S2: Preprocess the training set and test set of the target image data, randomly add masks to the target image to simulate the structural defects caused by the occlusion or signal interference of the underwater target;

[0010] S3: Build an object detection model, using the YOLO V9 model as the backbone and integrating the attention mechanism module;

[0011] S4: Select appropriate loss function and optimization method according to solution requirements;

[0012] S5: perform pre-training parameter operations on the target image training set after adding the mask, and use the target image test set to test the pre-trained model;

[0013] S6: Fine-tune the pre-trained model parameters using the sonar image training set and test the model using the sonar image test set.

[0014] Preferably, the target image data in S1 uses LabelImg annotation software to draw a bounding box on the image and annotate the target information; the sonar image data uses LabelImg annotation software to draw a bounding box for the target in the sonar image data and indicate the category and other information of the target; the annotated data is divided into a training set and a test set in a ratio of 7:3.

[0015] Preferably, in S2, a mask is added to the target image by first randomly selecting a number of pixels from the target area in the image, and these pixel values ​​will serve as the basis for adding the mask; then, among all the pixels in the target area, a number of pixels are randomly selected as the center of the mask; these randomly selected pixels will serve as the core of the mask in subsequent steps to determine the influence range and position of the mask.

[0016] Preferably, based on the YOLO V9 model, S3 integrates a spatial attention module in the shallow layer of the model to focus on image detail information; in the middle layer of the model, it combines spatial and channel attention to comprehensively improve local details and global information; in the deep layer of the model, it integrates a channel attention module to highlight global features; through this hierarchical design, YOLO V9 can flexibly use spatial and channel attention at different network levels, helping the network to improve the detection accuracy and robustness of targets in various complex scenarios.

[0017] Preferably, the loss function in S4 selects the MPDIoU loss function, and the MPDIoU loss function can be expressed by the following formula:

[0018] LMPDIoU=1-IoUmax+λ·Distance(Bp,Bg);

[0019] Among them, IoUmax is the maximum intersection-over-union ratio between the predicted box and the true box; Distance(Bp,Bg) is the maximum probability distance metric between the predicted box Bp and the true box Bg; λ is a learnable parameter that controls the impact of the distance metric on the loss function.

[0020] Preferably, before pre-training the masked target image in S5, a normalization and standardization method is applied to all images.

[0021] Preferably, S6 includes a fine-tuning phase, which focuses on optimizing the adaptability of the model on sonar images while avoiding overfitting by adjusting the learning rate and freezing some layers.

[0022] The beneficial effects of the present invention are as follows:

[0023] (1) The technical solution of the present invention introduces transfer learning, which uses a large amount of target image data to pre-train neural network parameters, and then uses sonar image data to fine-tune these parameters, thereby effectively solving the under-fitting problem of model training caused by insufficient data;

[0024] (2) In the data preprocessing stage, a mask is randomly added to the target image to block part of the target for enhancement. This method can increase the diversity and complexity of the data and help the model learn the complete semantic information of the target. When processing occluded target images, the model can accurately identify the target category by deeply mining the deep features of the target. This method simulates the situation in actual applications where the image display is incomplete due to signal interference or target occlusion, thereby improving the model's ability to learn target features and enhancing the model's robustness and generalization capabilities. At the same time, by randomly blocking different positions of the target, the data set can be expanded, further alleviating the problem of insufficient data volume.

[0025] (3) The method of the present invention uses YOLO V9 as the basic model and combines it with the attention mechanism for structural optimization to enhance the ability to extract target features, thereby achieving fast and accurate real-time target detection, which is particularly suitable for target recognition in forward-looking sonar images. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic flow diagram of the method of the present invention;

[0027] Figure 2 It is an example diagram of the target image before and after the mask of the method of the present invention;

[0028] Figure 3 It is a model training flow chart involved in the present invention;

[0029] Figure 4 This is the method of integrating the attention mechanism into the model of the present invention. DETAILED DESCRIPTION

[0030] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments of the present invention are provided for the purpose of illustration and description, and are not intended to be exhaustive or to limit the present invention to the disclosed forms. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present invention, and to enable those of ordinary skill in the art to understand the present invention and thereby design various embodiments with various modifications suitable for specific uses.

[0031] As described in the background technology, traditional deep learning models face a series of challenges in underwater target recognition. It is difficult to obtain high-quality training data. The particularity of underwater images and the scarcity of data make the training process more complicated. In addition, underwater targets are incomplete due to occlusion or blur, which increases the difficulty of target detection. At the same time, underwater target recognition has high requirements for real-time and accuracy. In order to solve the problems raised in the background technology, the technical solution adopted by the present invention is: a method that can efficiently detect and identify sonar images with insufficient data volume and target occlusion, a forward-looking sonar image target recognition method based on random masks, comprising the following steps:

[0032] S1: Collect and annotate data sets, and divide target image data and sonar image data into training set and test set in a certain ratio; collect target image data including airplanes, shipwrecks, rocks, etc., and use LabelImg annotation software to draw bounding boxes on the images and annotate target information; use LabelImg annotation software to draw bounding boxes for targets in sonar image data, and indicate target categories and other information; divide the annotated data into training set and test set in a ratio of 7:3.

[0033] In S1, the open source data annotation tool LabelImg is used for data annotation, and the labels are saved in the YOLO format as a txt file; if there are multiple targets in the image, ensure that each target is correctly labeled and the same target is not labeled repeatedly; for the target image data, apply data enhancement techniques such as rotation, cropping, scaling, etc. to generate more training samples, and use different image enhancement methods for the target image data and sonar image data to enhance the diversity of the data; for the sonar image data, enhance the data set by adding Gaussian noise; divide it into training set and test set using the k-fold cross-validation method, divide the data set in a ratio of 7:3, and avoid the instability of the model evaluation results due to unbalanced data division or accidental factors through multiple divisions.

[0034] S2: Preprocess the training set and test set of the target image data, randomly add masks to the target image to simulate the structural defects caused by the occlusion or signal interference of the underwater target; when pre-training the model, randomly add masks to the target image to simulate the structural defects caused by the occlusion or signal interference of the underwater target, so as to prompt the model to deeply explore the deep features of the target and accurately identify the target category, thereby improving the model's learning ability of the target features and enhancing the model's robustness and generalization ability;

[0035] To add a mask to the target image, first randomly select several pixels from the target area in the image. Assume that the bounding box coordinates of the target are (x_min, y_min) and (x_max, y_max), where (x_min, y_min) is the coordinate of the upper left corner and (x_max, y_max) is the coordinate of the lower right corner. Extract the pixels of the target area from the image according to the bounding box. These pixel values ​​will be used as the basis for adding the mask, target_pixels = image[y_min:y_max,x_min:x_max]; then, among all the pixels in the target area, randomly select several pixels as the center of the mask; these randomly selected pixels will be used as the core of the mask in subsequent steps to determine the influence range and position of the mask, numpy.random.choice(target_pixels).

[0036] The mask area is usually filled with black or other values ​​(for example, pure white or transparency is 0), and covers part of the target's pixels. The number is 10 and the size is set to 50*50 pixels. To further enhance the diversity of the dataset, the above steps can be repeated for each image, and a certain random seed can be set. Each run can generate multiple different occlusion samples for each target. This will result in multiple images with different occlusion methods, helping the model learn more complex target features. After completing the above operations, save the image with the mask added and generate a new label file for each image. These new images will be used as part of the dataset for training the model to improve the model's adaptability to occlusions in real environments.

[0037] like Figure 2 The figure shows an example of the target image before and after the mask of the method of the present invention. Ten pixels are randomly selected as the center in the pixels of the target image, and expanded outward to form a rectangle of 50*50 pixels in size, which is filled with black to form the masked image.

[0038] S3: Build an object detection model, using the YOLO V9 model as the backbone and integrating the attention mechanism module;

[0039] In S3, YOLO V9 introduced programmed gradient information (PGI) to provide reliable gradient information, ensuring that the network can fully utilize the features of each layer during training and effectively update the weights of the model; YOLO V9 adopted a new network architecture - Generalized Efficient Layer Aggregation Network (GELAN), which optimizes convolution operations and feature transfer, making the network more efficient, reducing unnecessary calculations, and improving training and reasoning speeds; in YOLO V9, PGI improves the stability of training and the learning effect of the model by enhancing gradient information transfer; and GELAN improves the network's computational efficiency and multi-level feature perception capabilities by optimizing convolution operations and hierarchical feature aggregation. The combination of the two has promoted YOLO V9 to achieve higher performance in target detection tasks, while also ensuring the efficiency and robustness of the model in practical applications.

[0040] Based on the YOLO V9 model, a spatial attention mechanism is inserted after the first few convolutional layers of the model. By learning the importance of the spatial features of the image, the model can automatically focus on the area containing the target in the image. After each convolutional layer in the middle layer of the model, spatial attention is first applied to enhance the spatial features of the image area. Then channel attention is applied to enhance the focus on important channels. The method of fusing spatial and channel attention modules is as follows: Figure 4As shown in the figure. Through this dual attention mechanism (spatial first and channel second), more fine-grained feature extraction is achieved, and the perception of medium-sized objects is improved. In the deep layer of the network model, the weights of the feature map channels are strengthened through channel attention. This can help the model focus on more discriminative features in high-level features, which helps improve the detection of large objects or long-distance targets in complex scenes.

[0041] S4: Select appropriate loss function and optimization method according to the requirements of this technology;

[0042] In S4, the MPDIoU loss function is selected. The MPDIoU loss function maximizes the relative distance and area overlap between the predicted bounding box and the true bounding box. The relative distance optimizes the position of the box by calculating the "maximum probability distance" between the predicted bounding box and the true bounding box; the area overlap is measured by maximizing the ratio of the bounding box overlap area (intersection) to the overall area (union), that is, IoU.

[0043] The MPDIoU loss function involved in step 4 can be expressed by the following formula:

[0044] LMPDIoU=1-IoUmax+λ·Distance(Bp,Bg)

[0045] Among them, IoUmax is the maximum intersection-over-union ratio between the predicted box and the true box; Distance(Bp,Bg) is the maximum probability distance metric between the predicted box Bp and the true box Bg; λ is a learnable parameter that controls the impact of the distance metric on the loss function. In the loss function of the model, λ is set as a learnable parameter, allowing the model to automatically learn the optimal value of λ during training, so that the hyperparameter can be dynamically adjusted according to different data sets and task requirements; lambda_param = nn.Parameter(torch.tensor(init_value)), where init_value is the initial value of λ, set to 1.0. During the training process of the model, the value of λ will be updated through back propagation like other network weights, and the balance between IoU loss and position loss can be automatically learned based on the training data, without manually adjusting the value of λ, thereby optimizing the training process of the model.

[0046] S5: Perform pre-training parameter operations on the target image training set after adding the mask, and use the target image test set to test the pre-trained model.

[0047] In S5, before pre-training the masked target image, the pixel values ​​of all images are scaled to the range of [0,1] for normalization, and standardized according to the statistics of the dataset (minus the mean and divided by the standard deviation) to ensure the consistency of the model input data. However, due to the small sample size of this dataset, when calculating the mean and standard deviation of the dataset, there will be a risk of unstable statistical results and overfitting. Therefore, using the standard ImageNet mean (mean = [0.485, 0.456, 0.406]) and standard deviation (std = [0.229, 0.224, 0.225]) can avoid the model's bias towards the dataset and enhance the model's generalization ability.

[0048] Among them, the batch size is set to 16. A smaller batch size can improve the generalization ability of the model. The epoch is selected as 50. If the loss function continues to decrease during training and there is no sign of overfitting, the number of epochs can be appropriately increased. If the validation loss of the model is no longer decreasing, the training can be stopped in advance. The Adam optimizer is selected to optimize the model. This optimizer is often used for target detection tasks. It performs well in most cases and can adaptively adjust the learning rate. The learning rate uses steplr to dynamically reduce the learning rate, torch.optim.lr_scheduler.StepLR(optimizer,step_size=10,gamma=0.1).

[0049] In S5, the model training is performed in a loop. At the beginning of each Epoch, the training set is used for training, and the network weights are updated through the optimizer. After each Epoch, the performance of the model is tested on the validation set, and the loss, accuracy and other indicators are recorded. If the loss of the validation set is no longer significantly reduced, the training is terminated early. Before pre-training the target image after the mask, normalization and standardization methods are used for all images, which helps to accelerate model training and make the model more stable to changes in the input image; in the pre-training stage, a larger Epoch and a moderate Batch size are used, and the appropriate learning rate and optimizer are selected. By monitoring the loss function, accuracy and other evaluation indicators, ensure that the model performs optimally on the dataset.

[0050] S6: Fine-tune the pre-trained model parameters using the sonar image training set and test the model using the sonar image test set;

[0051] Due to signal interference and complex underwater environment, the target in the sonar image is partially obscured, so no mask operation is required. However, when fine-tuning the model parameters, it is necessary to scale the pixel values ​​of all images to the range of [0,1] for normalization and standardize them according to the statistics of the data set (subtract the mean and divide by the standard deviation) to ensure the consistency of the model input data. However, due to the small sample size of this data set, when calculating the mean and standard deviation of the data set, there will be a risk of unstable statistical results and overfitting. Therefore, using the standard ImageNet mean (mean = [0.485, 0.456, 0.406]) and standard deviation (std = [0.229, 0.224, 0.225]) can avoid the model's deviation from the data set and enhance the generalization ability of the model;

[0052] In S6, since the target image training phase has been well trained, you can choose to freeze the first few layers of the model and only train the parameters of the last few layers. This can effectively reduce the training time while retaining the common features learned in the pre-trained model, for paramin model.backbone.parameters():param.requires_grad=False. In the fine-tuning phase, the learning rate is set to 0.0001, because the model is close to the optimal state, so that the existing knowledge will not be destroyed during fine-tuning. Epoch is selected as 20 and Batch size is selected as 16. In the fine-tuning state, a smaller cycle can get better performance; the model is fine-tuned in a loop, and each Epoch is trained, and at the end of each Epoch, it is tested on the sonar image validation set to see the improvement of the model performance. Before pre-training the sonar image, all images are normalized and standardized, which helps to speed up the model training and make the model more stable to changes in the input image; in the fine-tuning phase, by adjusting the learning rate and freezing some layers, focus on optimizing the model's adaptability on sonar images while avoiding overfitting. By monitoring the loss function, accuracy, and other evaluation metrics, ensure that the model performs optimally on both datasets.

[0053] To overcome the above challenges, this paper proposes an improved target recognition system based on the YOLO V9 algorithm. We adopt a transfer learning strategy to pre-train the model using collected image data of aircraft, shipwrecks, rocks, etc., and then use real sonar data to fine-tune the model parameters to make up for the lack of underwater data sets. In addition, in order to improve the generalization ability of the model and simulate the situation where underwater targets may be obscured due to complex environments, we introduce random mask technology. This technology not only enhances the adaptability of the model in complex scenarios, but also significantly improves the computational efficiency of the model while ensuring recognition accuracy, meeting the needs of real-time target detection.

[0054] like Figure 3 The figure shows the model training flow chart involved in the present invention. First, there is the pre-training parameter stage. After the masked target image data is processed by image pre-processing, it is input into the YOLO V9 model integrated with the attention mechanism module, and finally the target bounding box, category and confidence are output. Then there is the fine-tuning parameter interface. After the unmasked sonar image data is processed by image pre-processing, it is input into the pre-trained model, and finally the target bounding box, category and confidence are output.

[0055] Through the above method, a mask is randomly added to the target image to block part of the target position for enhancement processing. This method can increase the diversity and complexity of the data and help the model learn the complete semantic information of the target; when processing the blocked target image, the model can accurately identify the target category by digging deep into the deep features of the target; this method simulates the situation in which the image display is incomplete due to signal interference or target occlusion in actual applications, thereby improving the model's learning ability for target features and enhancing the model's robustness and generalization ability. At the same time, by randomly blocking different positions of the target, the data set can be expanded, and the YOLO V9 basic model is combined with the attention mechanism for structural optimization to enhance the ability to extract target features, thereby achieving fast and accurate real-time target detection, which is particularly suitable for target recognition in forward-looking sonar images. The present invention can effectively solve the problems of data scarcity and incomplete targets in underwater target detection, and provide solid technical support for autonomous navigation and safe operation of underwater unmanned vehicles.

Claims

1. A method for forward-looking sonar image target recognition based on random masks, characterized in that: The following steps are included: S1: Collect and annotate data sets, and divide target image data and sonar image data into training set and test set according to a certain ratio; S2: Preprocess the training set and test set of the target image data, randomly add masks to the target image to simulate the structural defects caused by the occlusion or signal interference of the underwater target; S3: Build an object detection model, using the YOLO V9 model as the backbone and integrating the attention mechanism module; S4: Select appropriate loss function and optimization method according to solution requirements; S5: perform pre-training parameter operations on the target image training set after adding the mask, and use the target image test set to test the pre-trained model; S6: Fine-tune the pre-trained model parameters using the sonar image training set and test the model using the sonar image test set.

2. The method for forward-looking sonar image target recognition based on random mask according to claim 1, characterized in that: The target image data in S1 uses LabelImg annotation software to draw a bounding box on the image and annotate the target information; the sonar image data uses LabelImg annotation software to draw a bounding box on the target in the sonar image data and indicate the target category and other information; the annotated data is divided into a training set and a test set in a ratio of 7:

3.

3. The method for forward-looking sonar image target recognition based on random mask according to claim 2, characterized in that: In S2, a mask is added to the target image. First, several pixels are randomly selected from the target area in the image. These pixel values ​​will serve as the basis for adding the mask. Then, several pixels are randomly selected from all the pixels in the target area as the center of the mask. These randomly selected pixels will serve as the core of the mask in subsequent steps to determine the influence range and position of the mask.

4. The method for forward-looking sonar image target recognition based on random mask according to claim 3, characterized in that: Based on the YOLO V9 model, S3 integrates the spatial attention module in the shallow layer of the model to focus on the image detail information; in the middle layer of the model, it combines the spatial and channel attention to comprehensively improve the local details and global information; in the deep layer of the model, it integrates the channel attention module to highlight the global features; through this hierarchical design, YOLO V9 can flexibly use spatial and channel attention at different network levels, helping the network to improve the detection accuracy and robustness of targets in various complex scenarios.

5. The method for forward-looking sonar image target recognition based on random mask according to claim 4, characterized in that: The loss function in S4 selects the MPDIoU loss function, which can be expressed by the following formula: LMPDIoU=1-IoUmax+λ·Distance(Bp,Bg); Among them, IoUmax is the maximum intersection-over-union ratio between the predicted box and the true box; Distance(Bp,Bg) is the maximum probability distance metric between the predicted box Bp and the true box Bg; λ is a learnable parameter that controls the impact of the distance metric on the loss function.

6. The method for forward-looking sonar image target recognition based on random mask according to claim 5, characterized in that: Before pre-training on the masked target images in S5, normalization and standardization methods are applied to all images.

7. The method for forward-looking sonar image target recognition based on random mask according to claim 6, characterized in that: S6 includes a fine-tuning phase, which focuses on optimizing the model’s adaptability to sonar images while avoiding overfitting by adjusting the learning rate and freezing some layers.

Images