Generation method, device and equipment for sonar image classification

Abstract

The invention discloses a generation method, device and equipment for sonar image classification, and the method comprises the steps: inputting a tail class with scarce samples in a training set into a trained customized diffusion generation model, and generating an initial synthesis image; performing style migration processing based on a diffusion model on the initial synthetic image, and injecting texture and noise features of a real sonar image to obtain a final synthetic image; and adding the final synthetic image into the training set, and using enhanced data for downstream classification model training. Wherein the customized diffusion generation model integrates a quality perception feature modulation module in a U-Net network, introduces a cross-scale attention and channel expansion mechanism, and adaptively optimizes low-quality image feature extraction; texture injection and structure keeping balance are realized by style migration through hierarchical feature control. According to the method, high-fidelity and diversified synthetic samples are generated, category imbalance is effectively relieved, the classification accuracy and generalization ability are improved, rare category recognition is enhanced, meanwhile, the robustness of the model to noise and adversarial samples is improved, and reliable data support is provided for underwater intelligent sensing.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision and signal processing, and particularly relates to a synthetic data generation method, device and system for low-visibility sonar image classification and a computer readable medium thereof, and is particularly used for generating a long-tailed distribution of sonar data through customized diffusion and multi-reference style injection to alleviate the long-tailed distribution and improve classification performance. BACKGROUND

[0002] Sonar imaging technology has become an indispensable key perception means in underwater environment monitoring, target recognition and seabed mapping tasks due to its physical characteristics of small sound wave attenuation, long transmission distance and independence from light. Compared with optical imaging, which significantly decreases in performance in turbid or lightless water areas, sonar systems can work stably in extreme environments and provide reliable detection capabilities for platforms such as autonomous underwater vehicles (AUVs) and remotely operated vehicles (ROVs).

[0003] With the development of deep learning technology, visual models based on convolutional neural networks (CNNs) have achieved remarkable success in image classification and target detection tasks. However, the high performance of such models is highly dependent on large-scale, high-quality labeled data sets. In the field of sonar, due to the complexity of underwater acquisition conditions, high cost and low efficiency, and the dependence of image labeling on professional knowledge, high-quality samples are extremely limited, which has become a major bottleneck restricting the application of deep learning methods in actual underwater scenarios.

[0004] To alleviate the problem of data scarcity, researchers have proposed various data augmentation and generation strategies. Traditional geometric transformations (such as rotation, flipping, etc.) can increase data diversity, but cannot generate new structural samples, and have limited improvement in model generalization performance. Generative adversarial networks (GANs) can generate synthetic images to expand the training set, but their training process is unstable and prone to mode collapse, and the details and diversity of generated samples are still insufficient. Methods based on physical simulation can simulate sonar imaging mechanisms to some extent, but they rely on high-quality three-dimensional models and have limited ability to simulate complex marine environments, with a cumbersome generation process and high computational cost.

[0005] In recent years, diffusion models have achieved remarkable results in natural image generation tasks as a powerful probabilistic generation framework. Some studies have attempted to apply them to underwater and sonar image generation tasks, proving that they can achieve stable and high-fidelity generation effects through iterative denoising. Compared with GAN, diffusion models effectively alleviate the mode collapse problem and have smoother convergence characteristics. However, there are still limitations in directly migrating natural image diffusion models to the sonar field: sonar images are usually grayscale, low resolution, and have low signal-to-noise ratio, and scattering and multipath effects cause their noise distribution to be inconsistent with the Gaussian degradation mechanism assumed by the model; at the same time, the sonar target structure is sparse and the boundary is fuzzy, and the generated results are prone to problems such as excessive smoothing of texture and loss of details.

[0006] In addition, existing sonar datasets generally exhibit typical long-tail distribution characteristics: common target classes have abundant samples, while rare targets such as sunken ships and aircraft wreckage have very limited samples. Such imbalance makes the model more prone to bias towards majority class features, weakening the recognition ability of minority class targets and affecting its generalization performance and robustness in complex underwater environments.

[0007] Therefore, there is an urgent need for a sonar synthetic image generation method that can generate high-fidelity, high-diversity, and class-distribution-balanced images to simultaneously alleviate the problems of insufficient data size and class imbalance, thereby providing reliable data support for downstream recognition and classification tasks and improving the robustness and generalization performance of the model in real underwater environments. SUMMARY

[0008] To solve the problems of limited size and severely imbalanced class distribution of existing sonar image datasets, which lead to insufficient generalization ability of deep learning models, the present application provides a generation method, device, system and storage medium for sonar image classification.

[0009] The technical problem to be solved by the present application is solved by the following technical scheme: In a first aspect, the present application provides a generation method for sonar image classification, comprising: inputting a sample-scarce tail class in a sonar image training set into a customized diffusion generation model trained to generate an initial synthetic image; wherein the customized diffusion generation model is used to learn the data distribution of real sonar images and generate images with similar structure and semantic features; performing style transfer processing on the initial synthetic image to obtain a final synthetic image; wherein the style transfer processing is used to inject the texture, contrast and noise characteristics of one or more real sonar images into the initial synthetic image to improve visual authenticity; The final synthetic image is added to the sonar image training set to build an enhanced training set to supplement the number of tail category samples and alleviate the class imbalance problem. The image classification model is trained using the enhanced training set to improve the model recognition performance and generalization ability.

[0010] Optionally, the customized diffusion generation model includes a denoising network based on a U-Net architecture, and the training process is used to learn to gradually recover data from noise; The U-Net network includes the following improvements: (1) Enhanced multi-scale feature modeling capability: by expanding the channel number of the down-sampling and up-sampling modules and introducing a self-attention mechanism, the model's ability to capture different scale key structures and long-range dependency features is improved; (2) Quality-aware feature modulation mechanism: integrate a quality-aware feature modulation module in the input preprocessing stage, which receives a quality vector extracted from real sonar images as a conditional input, and dynamically adjusts the network receptive field according to the vector, so that the model can adaptively optimize the feature representation, especially enhancing the structure preservation ability of low signal-to-noise ratio and blurred outline images.

[0011] Optionally, the style transfer processing includes: The initial synthetic image is used as the content image, and one or more real sonar images are selected as the style reference image; The content image and the style reference image are input into a style transfer pipeline based on a diffusion model, which injects style information in the middle and high-level feature space through a hierarchical control mechanism, while maintaining the content structure at the bottom; By optimizing the weighted objective function composed of content loss, style loss and total variation loss, the final synthetic image is generated, which retains the original structure while having the texture and noise distribution characteristics of real sonar images.

[0012] Optionally, to improve the diversity of generated samples, a rotating style matching strategy is adopted, which matches different real style reference images to each initial synthetic image in turn during the style transfer process.

[0013] In a second aspect, the present application provides a generation device for sonar image classification, which comprises: An initial image generation module for generating initial synthetic images using a customized diffusion generation model for tail categories; A style transfer module for performing style transfer on the initial synthetic images to obtain final synthetic images; A data set enhancement module for adding the final synthetic images to the training set to build an enhanced data set; A classification training module for training an image classification model based on the enhanced data set.

[0014] In a third aspect, the present application provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus; The memory is configured to store a computer program, and the processor is configured to execute the program to implement any step of the method for generating sonar image classification according to the first aspect.

[0015] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the program, when executed by a processor, implements any step of the method according to the first aspect.

[0016] The present application has the following advantages: The present application effectively solves the problems of sonar image training data scarcity and class imbalance by customizing the diffusion model combined with the style transfer strategy; the generated high-quality and diversified samples significantly improve the accuracy, recall rate and generalization performance of the classification model; at the same time, the robustness of the model to noise and adversarial interference is enhanced, and the reliability and stability of the model in complex underwater environments are improved, which can meet the needs of intelligent sonar perception and real-time identification applications.

[0017] The present application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0018] Figure 1 is a sonar image generation and classification enhancement flowchart provided by an embodiment of the present application based on a diffusion model Figure 2 is a generation device schematic diagram for sonar image classification provided by an embodiment of the present application DETAILED DESCRIPTION

[0019] In order to better illustrate the implementation of the present application, this section will describe the present application in detail in combination with specific embodiments, but the implementation of the present application is not limited thereto.

[0020] As shown in Figure 1 , the present application provides a sonar image generation method based on quality guidance and multi-reference injection, which specifically includes the following steps: Step S101: Obtain the original sonar training set and extract the tail classes with sample scarcity First, we extract samples from the original sonar image training dataset and perform class distribution statistics and analysis. According to the difference in the number of samples of each class, the dataset is divided into head classes with sufficient samples and tail classes with insufficient samples. The tail classes usually have insufficient samples, which can lead to insufficient learning of the model during training, thereby affecting the classification performance. Therefore, for the tail classes, the generation and enhancement process focuses on optimization to compensate for the performance bottleneck caused by uneven data distribution, laying the foundation for subsequent long-tail recognition optimization.

[0021] Step S102: Extract the quality vector of the tail class image, generate the dynamic modulation weight, and extract the initial convolutional features of the image The purpose of this step is to provide basic support for the generation of subsequent quality-aware features. By quantifying the quality features of the sonar image, these information are used as guiding signals in the generation process, thereby achieving adaptive modulation of the generated features. Specifically, for any input sonar image in the selected tail class, the following two operations are performed: Extract quality vector: Extract the quality vector of the image through a no-reference perceptual quality evaluation model (such as a deep neural network-based, convolutional feature encoder, or visual transformer architecture). This model can extract high-dimensional features such as noise level, blur degree, contrast, and texture integrity to quantify the intrinsic quality of the sonar image. The dimension of the quality vector can be set according to specific task requirements and used as a learnable constraint condition in the generation process to guide the model to generate images with specific quality features.

[0022] Extract initial feature map: At the same time, input the image into a standard two-dimensional convolutional network to extract the basic convolutional feature map, which contains the structure and texture information of the image.

[0023] Step S103: Perform multi-scale adaptive weighting and fusion of the initial convolutional features according to the dynamic modulation weight to obtain the quality-aware feature map This step is one of the key innovations of the present invention, aiming to guide the feature extraction process through the quality vector to generate a quality-aware feature map. The specific implementation is as follows: Multi-scale feature extraction: The initial feature map is divided into multiple feature groups according to the channel, and different sizes of convolutional kernels (such as 3x3, 5x5, 9x9, etc.) are applied to each feature group for processing to capture local details and global structure information in the image.

[0024] Dynamic weight generation: The extracted quality vector is input into a small multi-layer perceptron to generate a set of adaptive dynamic weights through the Softmax function. These weights reflect the importance of different scale features to the quality of the current image.

[0025] Adaptive weighted fusion: The generated dynamic weights are used to weight and sum the multi-scale features, and the final quality-aware feature map is generated through concatenation operation. This adaptive routing mechanism can ensure that when processing low-quality images (such as severe noise or blurred edges), the model focuses on restoring details and reducing noise interference.

[0026] Step S104: The quality-aware feature map is input into the enhanced U-Net backbone network, and the quality condition is used to guide the generation of the initial synthesized image that preserves the structure and semantics in the reverse denoising process This step is the first stage of the generation process, aiming to generate a preliminary synthesized image that is structurally correct and semantically consistent. The generation process is completed by a quality-guided diffusion generator, which specifically includes: Enhanced U-Net backbone network: The quality-aware feature map generated in step S103 is input into the U-Net architecture optimized for sonar image characteristics. This architecture increases the channel capacity of the network and integrates attention enhancement modules in the middle layer, thereby improving the model's ability to capture long-range dependencies. This is crucial for modeling complete underwater target structures from sparse sonar information.

[0027] Quality condition guided diffusion generation: In the reverse denoising stage, the system guides the denoising network of U-Net according to the target quality vector. Each generation iteration, the network generates an image that meets the target quality according to the quality condition. This process continues until an initial synthesized image that is structurally and semantically consistent is generated.

[0028] Step S105: Style transfer processing is performed on the initial synthesized image to inject texture and noise features of real sonar images The purpose of this step is to improve the perceptual realism of the generated image. Through the multi-reference sonar style injection module, the texture, noise, and other style features of multiple real sonar images are transferred to the generated structural image, enhancing the realism of the image. The specific implementation is as follows: Architecture design: The multi-reference style injection module is based on the latent diffusion model, which maps images to a low-dimensional latent space through a variational autoencoder (VAE), and then decouples content and style through U-Net denoising operations and DDIM schedulers. This architecture can effectively integrate structural information and style features of images.

[0029] Style injection mechanism: The texture features of multiple real sonar images are injected into the middle layer of the U-Net denoising network through a controller. This hierarchical injection strategy ensures that structural information is preserved while style features can be transferred at a high level, enhancing the style consistency of the generated image.

[0030] Multi-reference style fusion: Through the multi-reference style injection module, style features are extracted from multiple real sonar images, and the diversity and authenticity of the generated images are enhanced through a rotating injection strategy.

[0031] Loss function guidance: In order to optimize the style transfer effect, a composite loss function is used, which includes content loss, style loss and total variation loss, to ensure the consistency and authenticity of the generated image in terms of content and style.

[0032] Step S106: The generated final synthetic image is supplemented to the original training set to build a class-balanced data augmentation training set The high-quality synthetic images generated in step S105 are added to the original training set to supplement the samples of the tail classes. By generating a sufficient number of synthetic images for each tail class, a class-balanced data augmentation training set is finally built.

[0033] Step S107: Use the augmented training set to train the downstream classification model to obtain the final classification model with improved tail class recognition performance and robustness Finally, the augmented training set built in step S106 is used to train the downstream classification model (such as ResNet, DenseNet or VGG, etc. Convolutional neural network). Through the hybrid training strategy of "real data + synthetic supplement", the recognition accuracy, recall rate and F1 score of the model in the tail class are significantly improved. In addition, this training method also enhances the overall classification performance and adversarial robustness of the model.

[0034] Figure 2 is a structural schematic diagram of a generation device for sonar image classification provided by an embodiment of the present application. As shown in Figure 2 The device 200 can include an initial image generation module 201, a style transfer module 202, a data set augmentation module 203, and a classification training module 204. The device is configured to perform the steps described in the above method embodiment.

[0035] Initial image generation module 201 This module is used to perform steps S102, S103 and S104 in the above method, and its core function is to generate an initial synthetic image that retains the integrity of structure and semantics. Specifically, the module internally integrates hardware or software logic for extracting sonar image quality vectors, units for implementing quality-aware feature modulation, and a quality-guided diffusion generator containing an enhanced U-Net backbone and a quality condition guiding mechanism.

[0036] Style transfer module 202 This module performs step S105 of the above method, refining the image output by the initial image generation module 201 and injecting realistic sonar style features. Specifically, this module implements a multi-reference sonar style injection framework, including a VAE codec, a U-Net denoiser operating in the latent space, a DDIM scheduler, and a controller for injecting style features from one or more reference images into a specified layer of U-Net.

[0037] Dataset Augmentation Module 203 This module performs step S106 of the method described above, and its function is to construct a class-balanced augmented training dataset. This module receives the final synthesized images from the style transfer module 202 and, according to a preset class balancing strategy, supplements these images into the corresponding tail classes in the original training dataset.

[0038] Classification Training Module 204 This module performs step S107 of the method described above, and its function is to train a high-performance final classification model. This module receives an augmented training set from the dataset augmentation module 203 and uses this dataset to train a downstream classifier, ultimately outputting a classification model that significantly improves both tail category recognition performance and overall robustness.

[0039] The method provided in this invention can be applied to electronic devices. Specifically, the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, a server, etc. No limitation is made herein; any electronic device that can implement this invention falls within the protection scope of this invention.

[0040] An electronic device for implementing the method of the present invention may include a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus. The memory is used to store computer programs; when the processor executes the program stored in the memory, it implements the steps of any of the above-described methods for generating sonar image classification.

[0041] The communication bus mentioned in the aforementioned electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. The memory can include Random Access Memory (RAM) or Non-Volatile Memory (NVM), such as at least one disk drive. Optionally, the memory can also be at least one storage device located remotely from the aforementioned processor.

[0042] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0043] The present invention also provides a computer-readable storage medium. A computer program is stored in the computer-readable storage medium, and when executed by a processor, the computer program implements any step of the above-described method.

[0044] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the steps described in any of the above methods.

[0045] It should be noted that the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those described herein. In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example.

[0046] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings and the disclosure in carrying out the claimed invention. In the description of the invention, the word "comprising" does not exclude other components or steps, "a" or "an" does not exclude a plurality, and "a plurality" means two or more, unless otherwise expressly specified.

[0047] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus (devices), or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects.

[0048] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.

[0049] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A generation method for sonar image classification, characterized in that, The method includes: The tail category, which has scarce samples in the sonar image training set, is input into the trained customized diffusion generation model to generate the initial synthetic image. The initial synthesized image is subjected to style transfer processing to inject texture and noise features of a real sonar image to obtain the final synthesized image; The final synthesized image is added to the sonar image training set to construct an enhanced training set; The image classification model is trained using the enhanced training set to improve its recognition performance and generalization ability.

2. The generation method for sonar image classification according to claim 1, characterized in that, The customized diffusion generation model includes a denoising network based on the U-Net architecture, which is used to learn to progressively recover data from noise. The U-Net network includes the following improvements: Enhanced multi-scale feature modeling capabilities are achieved by expanding the number of channels in the downsampling and upsampling modules and introducing a self-attention mechanism to improve the model's ability to capture key structures and long-range dependent features at different scales. The quality-aware feature modulation mechanism integrates a quality-aware feature modulation module in the input preprocessing stage. This module receives a quality vector extracted from real sonar images as a conditional input and dynamically adjusts the network's receptive field based on the vector, thereby optimizing the feature extraction and structure preservation capabilities of low-quality images.

3. The generation method for sonar image classification according to claim 1, characterized in that, The style transfer process includes: Use the initial synthesized image as the content image, and select one or more real sonar images as style reference images; The content graph and the style reference graph are input into a style transfer pipeline based on a diffusion model. The pipeline injects style information into the mid-to-high-level feature space through a hierarchical control mechanism, while maintaining the content structure at the bottom level. By optimizing a weighted objective function consisting of content loss, style loss, and total variation loss, the final synthesized image is generated, which retains the original structure while possessing the texture and noise distribution characteristics of a real sonar image.

4. The generation method for sonar image classification according to claim 3, characterized in that, The style transfer process further includes: A rotating style matching strategy is adopted, in which different real style reference images are matched to each initial synthetic image in turn during the style transfer process, so as to improve the diversity and realism of the generated images.

5. The generation method for sonar image classification according to claim 1, characterized in that, The augmented training set, by supplementing the tail category with samples, constructs a class-balanced dataset, thereby enhancing the model's recognition performance and robustness in the tail category.

6. A generation device for sonar image classification, characterized in that, The device includes: The initial image generation module is used to input tail-type sonar images into a customized diffusion generation model to generate initial synthetic images; The style transfer module is used to perform style transfer processing on the initial synthesized image to generate the final synthesized image; The dataset augmentation module is used to add the final synthesized image to the training set to construct an augmented dataset; The classification training module is used to train an image classification model based on the augmented dataset.

7. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the executable instructions to implement the generation method for sonar image classification as described in any one of claims 1 to 5.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the steps of the generation method for sonar image classification as described in any one of claims 1 to 5.

Images