Sonar image classification method for coping with long tail distribution
By using a two-stage edge detection and sonar image classification network, the long-tail distribution problem in sonar image recognition is solved, the recognition accuracy of rare targets is improved, and the ability to identify and track underwater targets is enhanced.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-18
- Publication Date
- 2026-03-31
AI Technical Summary
The long-tail distribution problem in sonar image recognition technology leads to overfitting of the model to common targets, while its performance is insufficient for recognizing rare but high-value anomalous targets, affecting the effectiveness of practical applications.
A two-stage edge detection algorithm and a sonar image classification network are adopted, including an encoder, a main classification module, an auxiliary classification module, and a boundary extraction module. By combining USAN edge feature coarse detection and multi-scale edge fine detection, and combining residual neural network and multi-loss function optimization, the classification accuracy of long-tailed sonar images is improved.
It effectively overcomes the identification bias caused by long-tail data, improves the accuracy of sonar image classification networks, especially the ability to identify rare targets, and enhances the ability to identify and track underwater targets.
Smart Images
Figure CN121767701A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of underwater image classification and deep learning technology, specifically relating to a sonar image classification method for dealing with long-tailed distribution, applicable to sonar image classification of different targets in low signal-to-noise ratio environments. Background Technology
[0002] Sonar technology has wide applications in the civilian sector, primarily in three areas: marine resource exploration, underwater detection, and engineering maintenance. In marine resource exploration, sonar systems accurately map seabed topography through sound wave reflection, providing crucial data support for waterway planning, oil and gas exploration, and fisheries monitoring. In underwater detection, this technology can efficiently locate underwater targets such as shipwrecks and aircraft wreckage, serving both deep-sea search and rescue and assisting in underwater archaeological research. In engineering maintenance, it is mainly used for infrastructure safety assurance, such as subsea pipeline inspection and dam and bridge monitoring. With technological advancements, sonar systems are developing towards intelligent systems, continuously providing vital technical support for marine economic development and underwater operations.
[0003] In practical applications of sonar image recognition technology, data distribution often exhibits a significant long-tail characteristic. This long-tail characteristic means that there are ample samples of common target categories, while samples of special or rare targets are extremely limited. This data imbalance is prevalent in civilian fields such as marine resource exploration, underwater archaeology, and infrastructure inspection. For example, in subsea pipeline inspection, data on conventional structures is abundant, but samples of abnormal conditions such as corrosion and cracks are scarce; in underwater archaeology, sonar images of common shipwrecks are plentiful, while samples of artifacts with special materials or severe damage are lacking. This long-tail distribution leads to overfitting of trained models to the head categories, while their recognition performance for key tail targets is severely inadequate; however, in practical applications, these rare but high-value anomalous targets are precisely what need to be accurately detected. Therefore, overcoming the recognition bias caused by long-tail data has become a key challenge in improving the practicality of sonar image analysis systems. Summary of the Invention
[0004] The purpose of this invention is to provide a sonar image classification method that addresses long-tailed distribution, thereby improving the training effect of sonar image classification networks on sonar image datasets with long-tailed distribution characteristics, and thus increasing the network's classification accuracy for different categories of sonar images.
[0005] To achieve the above objectives, the present invention employs the following technical solution: A sonar image classification method for dealing with long-tailed distribution includes: Acquire the sonar image to be classified, input the sonar image into the trained sonar image classification network, and use the encoder and main classification module of the sonar image classification network to classify the sonar image. The sonar image classification network includes an encoder, a main classification module, an auxiliary classification module, and a boundary extraction module; the training process of the sonar image classification network includes: Construct a sonar image classification dataset and preprocess the sonar images within it; utilize joint... USAN A two-stage edge detection algorithm, consisting of coarse edge feature detection and fine multi-scale edge detection, is used to perform edge detection on preprocessed sonar images. A residual neural network is used as the encoder to extract features from the preprocessed sonar images and output a deep semantic feature map. The main classification module includes classification units that distinguish between all actual categories, middle categories, and tail categories in the sonar image classification dataset. The final fusion prediction result is obtained using the classification outputs of each classification unit, and the classification loss for each unit is calculated. The auxiliary classification module uses an auxiliary classifier to perform auxiliary classification based on the feature map output from the residual network block in the encoder to calculate the auxiliary loss. The boundary extraction module decodes the deep semantic feature map step by step to obtain a boundary prediction map containing edge information of the sonar images. This boundary prediction map and the edge detection result are used to calculate the edge detection loss. By optimizing the classification loss, auxiliary loss, and edge detection loss, the network parameters of the encoder and the main classification module are optimized.
[0006] Furthermore, the aforementioned USAN The process of coarse edge feature detection is as follows: Each pixel in the preprocessed sonar image is used as a pixel kernel point; for each pixel kernel point, its corresponding neighborhood is defined, and the similarity function between the pixel kernel point and the pixels in the neighborhood is defined using gray values and soft similarity function; Based on the similarity function and the neighborhood of the pixel kernel point, define USAN Area; utilization USAN The response function is constructed based on the area and a preset upper limit threshold of the area; and a binary mask matrix is constructed based on the response function value corresponding to the pixel kernel point; the binary mask matrix represents the location of the suspected feature region in the sonar image, thereby realizing coarse edge feature detection.
[0007] Furthermore, the process of multi-scale edge fine detection is as follows: based on USAN The binary mask matrix obtained from coarse edge feature detection identifies suspected edge pixels and assigns them initial probabilities. Then, multiple different diffusion scales are set, and iterative diffusion is performed sequentially within the corresponding neighborhood size for each diffusion scale. The probability values of each pixel obtained after the iterative diffusion are used to construct the final probability matrix, which serves as the probability response map. Using the mean probability of all pixels in the probability response map as a threshold, a fine edge mask matrix is constructed, and the mask values within it are combined to determine the final edge pixels, thus completing the fine edge detection of the sonar image.
[0008] Furthermore, during the initial probability assignment, pixels with a mask value of 1 in the binary mask matrix are considered as suspected edge pixels and assigned an initial probability value of 1 / N , N This represents the total number of pixels suspected of being at the edge. Multiple diffusion scales are set up, with each diffusion scale corresponding to a neighborhood size. Iterative diffusion is carried out in the neighborhood size corresponding to each diffusion scale in ascending order of diffusion scale. After the iterative diffusion of each diffusion scale is completed, the current probability matrix composed of the probability values of all edge-like pixels is normalized before entering the iteration of the next diffusion scale. After all diffusion scales are iterated, the matrix composed of the current probability values of each pixel is used as the final probability matrix. The final probability matrix is used as the probability response map, and the mean probability of all pixels in the probability response map is calculated. Based on the calculated mean probability and the probability values of each suspected edge pixel at the end of the iterative diffusion, a fine edge mask matrix is constructed. When the probability value of a suspected edge pixel is not less than the mean, its corresponding mask value in the fine edge mask matrix is set to 1; otherwise, it is set to 0. Suspected edge pixels with a mask value of 1 are used as final edge pixels.
[0009] Furthermore, the residual neural network of the encoder includes multiple residual network blocks; The boundary extraction module includes multiple decoding layers corresponding to the residual network blocks, which are used to perform upsampling and feature fusion processing on the deep semantic feature maps step by step, and finally output the boundary prediction map. After each upsampling operation, the feature map output by the corresponding residual network block of the encoder is introduced in a skip connection manner.
[0010] Furthermore, the three classification units in the main classification module include HMT unit, MT unit, T Unit, where: The sonar image classification dataset is divided into head, middle, and tail categories based on the number of sonar images in each actual category. HMT The unit classifies all actual categories of the sonar image. MT The unit distinguishes between the middle category and the tail category. T The unit determines the tail category; For all real-world sonar images, the deep semantic feature map output by the encoder is fed into... HMT After the unit passes through a convolutional layer and a fully connected layer, the classification output is obtained. And based on the normalized probability of the classification output and the actual category label of the sonar image, it utilizes... FocalLoss The loss function calculates the classification loss; For sonar images of the middle and tail categories, the deep semantic feature map output by the encoder is entered into... MT After the unit passes through a convolutional layer and a fully connected layer, the classification output is obtained. The output of this classification is the same as MT The module label of the unit is used to calculate the classification loss; if a sonar image belongs to the head category, the module label is set to 0, and if a sonar image belongs to the middle or tail category, the module label is set to 1. For sonar images of the tail category, the deep semantic feature map output by the encoder is entered into... T After the unit passes through a convolutional layer and a fully connected layer, the classification output is obtained. The output of this classification is the same as T The module label of the unit is used to calculate the classification loss; if a sonar image belongs to the head or middle category, the module label is set to 0, and if a sonar image belongs to the tail category, the module label is set to 1.
[0011] Furthermore, the auxiliary classification module includes two auxiliary classifiers, which are applied to the feature maps output by the second-to-last and third-to-last residual network blocks of the encoder, respectively. Each auxiliary classifier consists of a global average pooling layer, a flattening layer, and a fully connected layer in sequence: the global average pooling layer compresses the feature map into a channel-level global vector and converts it into a tensor; the flattening layer further transforms it into a vector; the fully connected layer performs a dimension mapping of the number of classes and outputs the predicted class probability distribution; the class probability distribution output by each auxiliary classifier is used to calculate the cross-entropy loss with the actual class label as an auxiliary loss.
[0012] Furthermore, in the inference stage, the sonar image to be classified is preprocessed and then input into the encoder. The deep semantic feature map output by the encoder is used to calculate the corresponding classification output by each classification unit in the main classification module, and the sum of these outputs is used as the final fusion prediction result.
[0013] A terminal device includes a processor, a memory, and a computer program stored in the memory; when the processor executes the computer program, it implements the sonar image classification method for dealing with long-tailed distribution.
[0014] A computer-readable storage medium storing a computer program; when executed by a processor, the computer program implements the sonar image classification method for dealing with long-tailed distributions.
[0015] Compared with the prior art, the present invention has the following technical features: This invention effectively addresses the recognition bias problem in training traditional network models for sonar image datasets with long-tailed distribution characteristics by constructing a two-stage edge detection algorithm and a sonar image classification network. It not only provides a new theoretical solution to the underwater image classification problem but also has broad potential in practical applications, better addressing the complex challenges of the marine environment and providing technical and data support for marine monitoring and exploration. Accurate classification of underwater sonar images can effectively improve the identification and tracking capabilities of underwater targets. Attached Figure Description
[0016] Figure 1 This invention relates to a sonar image classification network. UCS - Net ( UnifiedClassificationandStructu re - awareNetwork ) structural diagram; Figure 2 This is a schematic diagram of the edge detection result acquisition method; Figure 3 This is a schematic diagram of the loss function module; Figure 4 This is a comparative method in one embodiment of the present invention. SADE , KCL , UHL - Net The confusion matrix.
[0017] Figure 5 This is a confusion matrix of a method in one embodiment of the present invention. Detailed Implementation
[0018] To address the problems existing in current technologies, this invention provides a sonar image classification method that handles long-tailed data distribution. This method effectively overcomes the recognition bias caused by long-tailed data, thereby improving the practicality of sonar image analysis systems. See also... Figures 1 to 3 The method of the present invention includes the following steps: Step 1: Construct a sonar image classification dataset and preprocess the sonar images in it.
[0019] Based on public SCTD ( SonarCommonTargetDetectionDataset )and KLSG ( KongsbergLabeledSeabedGeodataA sonar image dataset is constructed by selecting sonar images of different categories from two sonar image datasets. The sonar images in the sonar image classification dataset are preprocessed, including data augmentation operations such as random cropping, rotation transformation, and mirror flipping, to improve the diversity of sonar image samples. To facilitate the training and evaluation of the sonar image classification network, the preprocessed sonar image classification dataset is divided into a training set and a test set with a ratio of 7:3. The sonar images in both the training set and the test set retain their category distribution characteristics in the original sonar image dataset.
[0020] In one embodiment of the present invention, the following is integrated: SCTD and KLSG Two publicly available sonar image datasets, containing a total of 1193 sonar images, were used to construct a sonar image classification dataset. These sonar images contain three types of targets: seabed ( floor 583 sheets, airplanes plane 123 sheets and ships ship 487 images were processed and pre-processed. The specific distribution of the number of categories is shown in the table below:
[0021] Step 2, using the combination USAN A two-stage edge detection algorithm, consisting of coarse edge feature detection and multi-scale fine edge detection, performs edge detection on preprocessed sonar images. The edge detection results are then used to calculate the edge detection loss during the subsequent training of a sonar image classification network.
[0022] This invention provides a two-stage edge detection algorithm, which mainly includes: USAN The process involves two steps: coarse detection and multi-scale edge system detection. Figure 2 This reflects the edge detection scheme used in this invention.
[0023] Step 2.1, USAN Coarse edge feature detection.
[0024] This step primarily involves extracting the coarse outline edge of the sonar image by calculating the difference between the pixel value of a given point and the pixel values of its surrounding points. Figure 2 The three subgraphs in the first row.
[0025] Specifically, the present invention uses USAN ( UnivalueSegmentAssimilatingNucleus The method is an edge and corner detection method based on region similarity. The core idea is to count how many similar pixels a pixel has in its neighborhood and use the statistical results to determine the local structural features of the image. Let the sonar image be , Represents the pixel coordinate space of a sonar image, where It is a set containing pixel positions A two-dimensional array of coordinates; The grayscale space representing the pixels of a sonar image. It represents an integer plane.
[0026] When analyzing the local structure of an image, the specific pixel selected as the "analysis center" is used for neighborhood similarity statistics and calculations. USAN The core reference point for operations such as area measurement is called the pixel kernel point. Each pixel in a sonar image becomes a pixel kernel point; for any pixel kernel point... Define the neighborhood of the pixel kernel point as Core Point The similarity function between it and its neighboring pixels is as follows: ; in and Represents pixel kernel point P and neighboring pixels Q The grayscale value, if it is a black and white image, and It is a one-dimensional vector. If it is a color image, and It is a three-dimensional vector. This is called the soft similarity function, which is a monotonically decreasing function, and its expression is as follows: ; in, T This is the similarity threshold; at this point, for a given pixel kernel point... P ,definition USAN The area is: ; USAN The physical meaning of area is the area of the neighborhood of the pixel kernel point. P The sum of the number of pixels with similar gray levels reflects the uniformity and structural complexity of a local region; in order to... USAN Extract edge or corner information from the area and introduce a response function. Its definition is as follows: ; in G It is a preset upper limit threshold for area, indicating USAN The upper limit of the area, in physical terms, is the area of the pixel kernel point. P When the pixel is entirely located in a region of uniform grayscale in the image, the maximum number of pixels in the neighborhood with similar grayscale to the pixel kernel point. A larger response function value indicates a stronger pixel kernel point. PThe less a pixel resembles its surrounding pixels, the more likely it is to be an edge or corner. Subsequently, a binary mask matrix is constructed based on whether the response function value is greater than zero. The binary mask representation is as follows: ; in, H and W The binary mask matrix represents the height and width of the sonar image, respectively. M It represents the location of suspected feature regions in the sonar image and can be regarded as a structural prior for coarse edge localization; this mask will be used as the initial probability distribution for quantum walk in the second stage, and will be used for subsequent diffusion of feature information and structural enhancement.
[0027] Step 2.2, multi-scale edge fine detection.
[0028] This step is based on USAN The binary mask matrix obtained from coarse edge feature detection identifies suspected edge pixels and assigns them initial probabilities. Then, multiple different diffusion scales are set, and iterative diffusion is performed sequentially within the corresponding neighborhood size for each scale. The probability values of each pixel obtained after iterative diffusion are used to construct a final probability matrix, which serves as the probability response map. Using the mean probability of all pixels in the probability response map as a threshold, a fine edge mask matrix is constructed, and the mask values are combined to determine the final edge pixels, thus completing the fine edge detection of the sonar image. This step corresponds to... Figure 2 The three subgraphs in the second row.
[0029] (1) First, the initial probability distribution is constructed.
[0030] USAN The binary mask matrix obtained from coarse detection is , This indicates the pixel For suspected edge pixels, count the total number of suspected edge pixels in the binary mask matrix. For each of the pseudo-edge pixels Assign initial probability values according to the following formula To form the initial probability matrix : ; This operation ensures that the probability is initially distributed only in the suspected edge area, avoiding interference from spreading to non-target areas.
[0031] (2) Subsequently, multi-scale probability diffusion was carried out.
[0032] set up S Groups of different diffusion scales , where each diffusion scale Corresponding to a neighborhood size In this embodiment of the invention, three diffusion scales 3, 4, and 5 are used; that is, the neighborhood size is 3×3, 4×4, and 5×5. Iterative diffusion is performed sequentially within the neighborhood size corresponding to each diffusion scale, in ascending order of diffusion scale. The number of iterations for each diffusion scale is [number missing]. T In this invention, T =5; for the current scale The t The next iteration ( (, with similar edge pixels) Divide by center neighborhood Statistical neighborhood satisfies the first t The probability value at -1 iteration pixels The quantity is denoted as: ; in This indicates an indicator function; it takes the value 1 if the condition within the parentheses is met, and 0 otherwise. Then update the pixels according to the following formula. The t The probability value at the next iteration : ; Conversely, if Then keep .
[0033] After each diffusion scale iteration is completed, the current probability matrix composed of the probability values of all near-edge pixels is normalized to ensure that the sum of the probability values of all near-edge pixels is 1 before proceeding to the next diffusion scale iteration; after all diffusion scale iterations are completed, the current probability values of each pixel are... The constructed matrix serves as the final probability matrix. This enables the probabilistic supplementation of blurred and broken edges.
[0034] (3) Finally, complete the edge pixel filtering.
[0035] The final probability matrix As a probability response map, the mean probability of all pixels in the probability response map is calculated using the following formula: ; Based on the calculated probability mean Combined with the probability values of the similar edge pixels at the end of the iterative diffusion Construct a fine edge mask matrix Pixels in this matrix Corresponding mask value for: ; Among them, the mask value Corresponding suspected edge pixels These are the final edge pixels, thus completing the fine edge detection of the sonar image; the final edge pixels will serve as labels for the sonar image and participate in the classification process.
[0036] Step 3: Design and build a sonar image classification network UCS - Net ( UnifiedClassificationand Structure - awareNetwork This network integrates several advanced modules, including an encoder, a main classification module, an auxiliary classification module, and a boundary extraction module; all modules are presented in [the document / document / etc.]. Figure 1 middle.
[0037] (1) Encoder.
[0038] This step uses a residual neural network as an encoder to extract features from the sonar image; for example... Figure 1 For example, the residual neural network contains five residual network blocks arranged sequentially. In the preprocessing stage, a sonar image of uniform size of 3×224×224 is passed through each residual block of the residual neural network for feature extraction. The feature maps output by the five residual network blocks are 64×56×56, 64×28×28, 128×14×14, 256×7×7, and 512×4×4, respectively. The deep semantic feature map output by the last residual network block enters the main classification module and the boundary extraction module to perform the tasks of classifying the sonar image and outputting the edge.
[0039] The residual neural network can be an existing network; this invention employs... ResNet The network serves as the skeleton; the residual network blocks within it are... ResidualBlock wait.
[0040] (2) Main category module.
[0041] The main classification module is constructed based on the deep semantic feature map output by the encoder; specifically, the encoder outputs a 512×4×4 deep semantic feature map, and the main classification module uses the features of this 512×4×4 dimension for classification.
[0042] To address the long-tail problem, the main classification module has three classification units: HMT ( HeadMediumTail )unit, MT ( MediumTail )unit, T( Tail ) unit, where: First, the training set is divided into head, middle, and tail categories based on the number of sonar images in each actual category. The head category generally refers to the actual category with the most sonar images in the training set; in this embodiment, it is... floor Category; the tail category refers to the actual category with the fewest sonar images in the training set, or the category of interest. In this embodiment, it refers to... plane Categories; the middle category refers to all actual categories other than the head category and the tail category, which in this embodiment is... ship category.
[0043] It should be noted that when there are a large number of categories in the training set, the categories can be divided into head categories, middle categories, and tail categories according to a preset ratio. For example, categories with a number of no less than a certain preset ratio can be used as head categories. Tail categories can be selected from the categories with the fewest number or can be specified by the user. These are the categories that need to be strengthened.
[0044] HMT The unit will identify all actual category labels of the sonar images in the training set and use the actual category labels of the sonar images as module labels; MT The unit distinguishes between the middle and tail categories, unlike... HMT unit, MT The module label of the unit is {0,1}. If a sonar image belongs to the head category, the module label is set to 0. If a sonar image belongs to the middle or tail category, the module label is set to 1. T The unit distinguishes based on the tail category, and... MT Similar to the unit, the sonar image module labels for the head and middle categories are 0, while the module label for the tail category is 1. This design constructs a more refined discrimination module.
[0045] Specifically, HMT The unit operates on all types of sonar images, that is: ; in Refers to all participants HMT The set of category labels for unit classification. These refer to the head category, middle category, and tail category, respectively. HMT The input to the unit is the deep semantic feature map output by the encoder. ,in C Represents the number of actual categories; pass After further extracting discriminative features through convolution, the data is fed into a fully connected layer to obtain the classification output. ; in, represent HMT The fully connected layer of the unit, represent HMT Convolutional layers of units; yes HMT The output of the unit, , It is a set of real numbers, and the result corresponds to the actual class labels (0, 1, 2, ...) of the sonar image. floor , ship and plan The loss is calculated using three actual categories, and the loss function is adopted. FocalLoss The loss function, the expression for this classification loss function is: ; in The classification output of the model, after normalization, corresponds to C The probability distribution of each category It is a class balance parameter. These are actual category tags. y The corresponding predicted probability, i.e. P The first in y One element, It is the focusing parameter, through This module suppresses the loss for easily classified samples and amplifies the loss for difficult-to-classify samples. It provides a unified ability to distinguish all classes and serves as the model's fundamental prediction loss function.
[0046] MT The unit is specifically designed for medium and small class samples. Assuming the actual class label of a sonar image is... y , MT The unit only applies to those that satisfy the actual category label. Activation of the sonar image, where: ; in Refers to all participants MT The unit classification uses a set of category labels, and the module receives the same feature input. And extract the classification output using independent parameters: ; in, represent MT The fully connected layer of the unit, represent MT Convolutional layers of units; yes MT The physical meaning of the network's output is the probability that a sample belongs to the middle or minor class. The result is consistent with MTUnit module tag Treat losses as losses The expression is: ; y These are actual category labels; MT The classification loss function expression for a unit is: ; T A unit is a module designed specifically for small categories. Let's still assume the actual category label of a certain sonar image is... y , T The unit only applies to those that satisfy Activation of the sonar image, where: ; in Refers to all participants T The unit classification uses a set of category labels, and the module receives the same feature input. And extract the classification output using independent parameters: ; in, represent T The fully connected layer of the unit, represent T Convolutional layers of units. yes T The physical meaning of the network's output is the probability that a sample belongs to a minor class. The result is consistent with T Unit module tag Treat losses as losses The expression is: ; y These are actual category labels; T The classification loss function expression for a unit is: ; The final fusion prediction result during training is defined as: ; in and For indicator functions: ; ; Final fusion prediction results This is the predicted probability distribution for each actual category.
[0047] The main loss of the main classification module The combination of losses from the three modules: .
[0048] (3) Auxiliary classification module.
[0049] To improve classification accuracy, this invention introduces an auxiliary supervision module, which supervises the intermediate layer features during the encoder stage, guiding them to learn stronger discriminative abilities, thereby improving the training efficiency and stability of the main classification task.
[0050] The auxiliary classification module contains two auxiliary classifiers ( AuxiliaryClassifier These are applied to the feature maps output by the second and third penultimate residual network blocks of the encoder, respectively, and output the corresponding class probability distributions.
[0051] Specifically, in the encoder structure, stages with feature map sizes of 256×7×7 and 128×14×14 are selected and connected to the auxiliary classifiers respectively. Each auxiliary classifier consists of a global average pooling layer, a flattening layer, and a fully connected layer in sequence: the global average pooling layer compresses the two-dimensional spatial features into channel-level global vectors and converts the input feature map into a tensor; the flattening layer further transforms it into a vector; and the fully connected layer maps the channel-dimensional features to the number of categories dimension and outputs the predicted category probability distribution.
[0052] During training, the class probability distribution output by each auxiliary classifier Individual vs. actual category labels y Calculate cross-entropy loss and This is added to the total loss as a separate auxiliary loss term. The expression for the auxiliary loss is as follows: ; ; in The model corresponds to after normalization C The probability distribution of each category It is a class balance parameter. These are actual category tags. y The corresponding predicted probability, i.e. P The first in y One element, It is the focusing parameter.
[0053] (4) Boundary extraction module.
[0054] This step constructs a boundary extraction module, which progressively upsamples the 512×4×4 deep features generated by the encoder to output a boundary prediction map containing edge information from the sonar image. This boundary prediction map is then compared with the edge detection results output in step 2 to calculate the edge detection loss. (such as using) MSE The loss function is used to supervise the edge information learning process and optimize the network parameters of the encoder and main classification module. The boundary reconstruction process is as follows: Figure 1 As shown, the construction process of the boundary feature map is divided into five decoding stages, corresponding to five encoding modules.
[0055] Specifically, the boundary reconstruction module uses a deep semantic feature map with a size of 512×4×4 output from the top layer of the encoder as the initial input. To gradually restore the image spatial resolution and retain key boundary information, the boundary reconstruction module has five levels of decoding layers to perform progressive upsampling and feature fusion processing on the deep semantic feature map. For example, the first decoding layer uses a transposed convolutional module with a kernel size of 2×2 and a stride of 2 to upsample the deep semantic feature map, enlarging its size from 512×4×4 to 256×7×7. To enhance the boundary reconstruction capability during the decoding process, after each upsampling operation, a skip connection is used to introduce the feature map output by the corresponding residual network block of the encoder. The skip connection specifically involves concatenating the current upsampling result with the feature map output by the residual network block in the channel dimension.
[0056] In this embodiment, the boundary reconstruction module utilizes a five-level decoding layer to sequentially perform five levels of upsampling and feature fusion operations, restoring the size of the deep semantic feature map to 256×7×7, 128×14×14, 64×28×28, and 64×56×56, ultimately outputting a boundary prediction map with a size of 1×224×224. The above multi-level upsampling process fully integrates the multi-scale features contained in the encoder's feature map, realizing the collaborative modeling of semantic information and detailed structure, thereby improving the accuracy, consistency, and robustness of the boundary detection results.
[0057] Therefore, the sonar image classification network in this invention UCS - Net The expression for the total loss is as follows: ; in, The edge detection loss of the boundary extraction module, These are the weighting coefficients.
[0058] In this process, all losses are backpropagated independently, which not only ensures the task-specific optimization of the main classification module, but also forces the intermediate layer features to learn the category discrimination information in advance through auxiliary supervision, thereby accelerating the convergence of the encoder and the main classification module, enhancing the feature generalization ability, and alleviating the gradient vanishing problem in deep networks.
[0059] This invention backpropagates the gradient signal from the boundary extraction module to the encoder, forming a closed-loop optimization link of "task supervision - feature feedback." This design has key technical value: specifically, the gradient generated by the boundary detection task can serve as an additional supervision signal, guiding the encoder to enhance its ability to extract local structural features such as image edges while learning global semantic features, thus avoiding the problem of insufficient detailed feature learning caused by single semantic supervision in the backbone network. Through this mechanism, the backbone network can learn a composite feature representation that combines "global semantic discriminativeness" and "local structural discriminativeness." Such features can not only directly improve the accuracy of the boundary detection task but also feed back into the main classification task, enabling it to achieve more accurate feature matching and target discrimination based on structured features when processing complex scenes or low-quality images, ultimately significantly improving the generalization ability and prediction reliability of the overall model.
[0060] Step 4, based on the integrated sonar image dataset and UCS - Net The architecture was developed, and the optimal training configuration was determined through systematic hyperparameter tuning experiments to complete the classification task of sonar images.
[0061] This invention is based on a constructed sonar image classification dataset and UCS - Net For classification networks, a series of preliminary hyperparameters are first set, and these hyperparameters are continuously adjusted and optimized during network training. Through refined hyperparameter design and adjustment, the optimal experimental configuration is explored, thereby improving the classification accuracy and performance of the network.
[0062] The values of the hyperparameters are shown in the table below:
[0063] After completion UCS - Net After training and optimization, the trained neural network is evaluated using an independent test set to verify its generalization performance in sonar image classification tasks.
[0064] Step 6: In the inference stage using the model, the sonar images to be classified are acquired and preprocessed, then input into the trained sonar image classification network. The encoder and main classification module in the sonar image classification network are used to classify the sonar images. During the inference stage, since the sonar images to be classified lack prior information about their categories, the final fusion prediction result of the main classification module... for: ; The deep semantic feature map output from the sonar image after encoder processing also utilizes the features from the main classification module.HMT unit, MT unit, T The unit calculates the corresponding classification output, and sums them to obtain the final fusion prediction result, which contains the predicted probability distribution of each actual category, thereby realizing the classification of sonar images.
[0065] The fusion of the reasoning stage is achieved under the condition of "no prior knowledge" by transforming the independent outputs of the three modules into a unified conclusion with clear category orientation and hierarchical attributes, thus realizing a closed loop from "multi-dimensional prediction" to "structured decision-making".
[0066] In one embodiment of this invention, a test set is used to evaluate the trained image classification network to verify its generalization performance in sonar image classification tasks; the experimental results are shown in the table below:
[0067] The experiment achieved a classification accuracy of 90.80%, far exceeding traditional methods. ResNet , MetaModelNet Methods and the latest NAS - MFF The method demonstrates its significant advantages in underwater image classification.
[0068] Once trained, the network in this invention can be used for sonar image classification tasks. In practical applications, after acquiring new sonar images, preprocessing operations identical to those performed during model training are required to ensure matching of input feature distributions: First, the sonar images are adjusted to the standard 224×224 pixel network size using algorithms such as bilinear interpolation to ensure spatial dimension compatibility with the network input layer; then, the image pixel values are normalized to map the original dynamic range to the [0,1] interval, eliminating numerical fluctuations caused by differences in device gain and imaging conditions; finally, the processed image is converted into a tensor format recognizable by the network, and after preprocessing, it can be input into the trained network for prediction.
[0069] The experiment achieved a classification accuracy of 90.80%, far exceeding traditional methods. ResNet , MetaModelNet Methods and the latest NAS - MFF This method demonstrates its significant advantages in underwater image classification. It not only provides a new theoretical solution to the underwater image classification problem but also has broad potential in practical applications, better addressing the complex challenges of the marine environment and providing technical and data support for marine monitoring, detection, and other fields. Accurate classification of underwater sonar images can effectively improve the identification and tracking capabilities of underwater targets.
[0070] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A sonar image classification method for dealing with long-tailed distributions, characterized in that, include: Acquire the sonar image to be classified, input the sonar image into the trained sonar image classification network, and use the encoder and main classification module of the sonar image classification network to classify the sonar image. The sonar image classification network includes an encoder, a main classification module, an auxiliary classification module, and a boundary extraction module; The training process of a sonar image classification network includes: Construct a sonar image classification dataset and preprocess the sonar images in it; Utilize joint USAN A two-stage edge detection algorithm, consisting of coarse edge feature detection and fine multi-scale edge detection, is used to perform edge detection on preprocessed sonar images. A residual neural network is used as the encoder to extract features from the preprocessed sonar images and output a deep semantic feature map. The main classification module includes classification units that distinguish between all actual categories, middle categories, and tail categories in the sonar image classification dataset. The final fusion prediction result is obtained using the classification outputs of each classification unit, and the classification loss for each unit is calculated. The auxiliary classification module uses an auxiliary classifier to perform auxiliary classification based on the feature map output from the residual network block in the encoder to calculate the auxiliary loss. The boundary extraction module decodes the deep semantic feature map step by step to obtain a boundary prediction map containing edge information of the sonar images. This boundary prediction map and the edge detection result are used to calculate the edge detection loss. By optimizing the classification loss, auxiliary loss, and edge detection loss, the network parameters of the encoder and the main classification module are optimized.
2. The sonar image classification method for dealing with long-tailed distributions according to claim 1, characterized in that, The USAN The process of coarse edge feature detection is as follows: Each pixel in the preprocessed sonar image is used as a pixel kernel point; for each pixel kernel point, its corresponding neighborhood is defined, and the similarity function between the pixel kernel point and the pixels in the neighborhood is defined using gray values and soft similarity function; Based on the similarity function and the neighborhood of the pixel kernel point, define USAN area; use USAN The response function is constructed based on the area and a preset upper limit threshold of the area; and a binary mask matrix is constructed based on the response function value corresponding to the pixel kernel point; the binary mask matrix represents the location of the suspected feature region in the sonar image, thereby realizing coarse edge feature detection.
3. The sonar image classification method for dealing with long-tailed distributions according to claim 1, characterized in that, The process of multi-scale edge fine detection is as follows: based on USAN The binary mask matrix obtained from coarse edge feature detection is used to identify suspected edge pixels and assign initial probabilities to them. Then, multiple sets of different diffusion scales are set, and iterative diffusion is carried out on each diffusion scale in its corresponding neighborhood size in sequence; the probability values of each pixel obtained after the iterative diffusion are used to construct the final probability matrix, and it is used as the probability response map. By using the mean probability of all pixels in the probability response map as the threshold, a fine edge mask matrix is constructed, and the final edge pixels are determined by combining the mask values in it, thereby completing the fine edge detection of the sonar image.
4. The sonar image classification method for dealing with long-tailed distributions according to claim 1, characterized in that, During the initial probability assignment, pixels with a mask value of 1 in the binary mask matrix are considered as suspected edge pixels and assigned an initial probability value of 1 / N , N This represents the total number of pixels suspected of being at the edge. Multiple diffusion scales are set up, with each diffusion scale corresponding to a neighborhood size. Iterative diffusion is carried out in the neighborhood size corresponding to each diffusion scale in ascending order of diffusion scale. After the iterative diffusion of each diffusion scale is completed, the current probability matrix composed of the probability values of all edge-like pixels is normalized before entering the iteration of the next diffusion scale. After all diffusion scales are iterated, the matrix composed of the current probability values of each pixel is used as the final probability matrix. The final probability matrix is used as the probability response map, and the mean probability of all pixels in the probability response map is calculated. Based on the calculated mean probability and the probability values of each suspected edge pixel at the end of the iterative diffusion, a fine edge mask matrix is constructed. When the probability value of a suspected edge pixel is not less than the mean, its corresponding mask value in the fine edge mask matrix is set to 1; otherwise, it is set to 0. Suspected edge pixels with a mask value of 1 are used as final edge pixels.
5. The sonar image classification method for dealing with long-tailed distributions according to claim 1, characterized in that, The residual neural network of the encoder includes multiple residual network blocks; The boundary extraction module includes multiple decoding layers corresponding to the residual network blocks, which are used to perform upsampling and feature fusion processing on the deep semantic feature maps step by step, and finally output the boundary prediction map. After each upsampling operation, the feature map output by the corresponding residual network block of the encoder is introduced in a skip connection manner.
6. The sonar image classification method for dealing with long-tailed distributions according to claim 1, characterized in that, The three classification units in the main classification module include: HMT unit, MT unit, T Unit, where: The sonar image classification dataset is divided into head, middle, and tail categories based on the number of sonar images in each actual category. HMT The unit classifies all actual categories of the sonar image. MT The unit distinguishes between the middle category and the tail category. T The unit determines the tail category; For all real-world sonar images, the deep semantic feature map output by the encoder is fed into... HMT After the unit passes through a convolutional layer and a fully connected layer, the classification output is obtained. And based on the normalized probability of the classification output and the actual category label of the sonar image, it utilizes... FocalLoss The loss function calculates the classification loss; For sonar images of the middle and tail categories, the deep semantic feature map output by the encoder is entered into... MT After the unit passes through a convolutional layer and a fully connected layer, the classification output is obtained. The output of this classification is the same as MT The module label of the unit is used to calculate the classification loss; if a sonar image belongs to the head category, the module label is set to 0, and if a sonar image belongs to the middle or tail category, the module label is set to 1. For sonar images of the tail category, the deep semantic feature map output by the encoder is entered into... T After the unit passes through a convolutional layer and a fully connected layer, the classification output is obtained. The output of this classification is the same as T The module label of the unit is used to calculate the classification loss; if a sonar image belongs to the head or middle category, the module label is set to 0, and if a sonar image belongs to the tail category, the module label is set to 1.
7. The sonar image classification method for dealing with long-tailed distributions according to claim 1, characterized in that, The auxiliary classification module includes two auxiliary classifiers, which are applied to the feature maps output by the second-to-last and third-to-last residual network blocks of the encoder, respectively. Each auxiliary classifier consists of a global average pooling layer, a flattening layer, and a fully connected layer in sequence: the global average pooling layer compresses the feature map into a channel-level global vector and converts it into a tensor; the flattening layer further transforms it into a vector; the fully connected layer performs a dimension mapping of the number of classes and outputs the predicted class probability distribution; the class probability distribution output by each auxiliary classifier is used to calculate the cross-entropy loss with the actual class label as an auxiliary loss.
8. The sonar image classification method for dealing with long-tailed distributions according to claim 1, characterized in that, During the inference phase, the sonar image to be classified is preprocessed and then input into the encoder. The deep semantic feature map output by the encoder is used to calculate the corresponding classification output by each classification unit in the main classification module. The sum of these outputs is used as the final fusion prediction result.
9. A terminal device, comprising a processor, a memory, and a computer program stored in the memory; characterized in that, When the processor executes a computer program, it implements the sonar image classification method for dealing with long-tailed distributions as described in any one of claims 1-8.
10. A computer-readable storage medium storing a computer program; characterized in that, When the computer program is executed by a processor, it implements the sonar image classification method for dealing with long-tailed distributions as described in any one of claims 1-8.