A SAR image preprocessing method for marine targets

By constructing a distribution migration generation model, the problem of the differences in the distribution of maritime target images in different acquisition channels is solved, the image resolution and detection effect are improved, and the field generalization of maritime target detection is achieved.

CN118799561BActive Publication Date: 2024-12-03NAVAL AVIATION UNIV
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411273295.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-12
Publication Date
2024-12-03
Estimated Expiration
2044-09-12

AI Technical Summary

Technical Problem

The prior art is difficult to effectively solve the problem of distribution differences in marine target images of different acquisition channels, resulting in a decrease in the detection effect of the model on the test set, especially the low resolution of small targets affects the detection effect.

Method used

A marine target SAR image preprocessing method is proposed. By constructing source domain and target domain data sets, the distribution migration generation model is designed, including mapping modules G and F, discriminator Dx and image optimization module DY, and the Generative Adversarial Network (UniGAN) and multi-scale feature extraction networks to achieve the unity of image distribution and semantic alignment of features, reducing distribution differences.

Benefits of technology

It improves the resolution consistency and detection effect of the image, adapts to different data distributions, enhances the generalization ability of the detection algorithm, and provides better data input to improve detection effect.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118799561B_ABST
    Figure CN118799561B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for preprocessing marine target SAR images, comprising the following steps: step S1, constructing a source domain data set and a target domain data set; step S2, constructing a distribution migration generation model, including a mapping module G, a mapping module F, a discriminator D x and image optimization module D Y ; Step S3, training the distribution transfer generation model, and updating the parameters of the model; Step S4, using the mapping module G in the trained distribution transfer generation model to perform distribution uniformity processing on the input image to be processed. The present invention can map multiple source domain SAR marine target detection images with inconsistent distribution and uneven quality to a target domain with concentrated distribution, high resolution and good quality, thereby providing good data input for subsequent detection tasks, and ultimately improving the detection effect of the algorithm on images in different domains, and realizing the field generalization function.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of computer image processing, in particular to a method for preprocessing SAR images of offshore targets. Background Art

[0002] With the development of deep learning related technologies, methods based on convolutional neural networks have achieved obvious advantages in the field of marine target detection. However, due to differences in shooting environment, resolution, etc., the distribution differences between samples collected from different channels are large, and it is difficult for the model to learn to adapt to the characteristics of samples with different distributions in different scenarios. This also leads to distribution differences between the training set and the test set, which makes the model learned from the training set less effective when applied to the test set. Moreover, there are usually many small targets in the marine target data set collected, such as ships. These small targets have low resolution and little available information, which also affects the detection effect. Therefore, it is of great significance to study the preprocessing of marine target images from various acquisition channels to eliminate the distribution differences between samples, enhance the resolution of small targets, and achieve domain generalization for marine target detection tasks, which is of great significance to improving the effect of marine target detection.

[0003] Some solutions have been proposed in the prior art, which are committed to improving image quality from the perspective of data processing. The Chinese invention patent application "A method for detecting small targets in remote sensing images based on resolution enhancement" with publication number CN111709307A aims to solve the problem of small targets having little available feature information. It constructs low-resolution-high-resolution image data pairs by downsampling high-resolution images, realizes supervised super-resolution reconstruction, and uses the perceptual loss function based on the VGG network to calculate the generated loss. The paper "Small Target Detection in Remote Sensing Images Based on Super-Resolution Reconstruction Technology" (Zhang Ziqian, Xiong Zaili, Zhang Biao; Journal of Northeast Electric Power University, 2022(002):042) also constructs low-resolution image data pairs by downsampling, and then realizes high-resolution reconstruction of images by SRGAN. This type of method preliminarily verifies that improving the algorithm resolution and enhancing low-quality image information methods are helpful to improve the detection effect of marine targets. However, these methods construct fixed-resolution data pairs by downsampling to realize supervised learning, which makes it difficult to solve the problem of enhancing images with different distributions collected from multiple acquisition channels.

[0004] There is another type of solution in the existing technology that is dedicated to aligning the source domain and the target domain through domain adaptation methods. For example, the paper "Ship Detection and Recognition in Cross-Modal Domain Adaptive SAR Images" (Song Yucheng, Li Jingrun, Tian Tian, ​​Tian Jinwen; Journal of Huazhong University of Science and Technology (Natural Science Edition), 2022, Vol. 50 (11): 107-113) designed a domain adaptation module to achieve alignment features at the global and instance levels, and to transfer attribute knowledge from optical data to SAR data across modalities. However, this type of algorithm only considers feature-level alignment, not image-level alignment, and only focuses on the alignment of features in the source domain and target domain, which makes it difficult to adapt to the alignment of multiple distributions collected from different channels. Summary of the invention

[0005] The present invention proposes a marine target SAR image preprocessing method, the purpose of which is to reduce the influence of marine image distribution differences and small target samples on marine target detection effects.

[0006] The technical solution of the present invention is as follows:

[0007] A method for preprocessing a SAR image of a marine target, comprising the steps of:

[0008] Step S1, constructing a source domain dataset and a target domain dataset;

[0009] The source domain dataset contains images collected through different platforms and channels, and having different distributions of acquisition parameters, and the marine targets in some images in the source domain dataset are small targets within a specified range of size;

[0010] The images in the target domain dataset are images whose resolution and image quality meet the preset standards;

[0011] Step S2, constructing a distribution migration generation model;

[0012] The distribution migration generation model includes a mapping module G and a mapping module F, and also includes a discriminator D x and image optimization module D Y ;

[0013] The mapping module G and the mapping module F have the same structure; the mapping module G is used to convert the input image into an image with consistent distribution of the target domain data set, and the mapping module F is used to convert the input image into an image with consistent distribution of the source domain data set;

[0014] The image optimization module D Y Used to calculate the perceptual loss value between the image from the target domain dataset and the image output by the mapping module G;

[0015] The discriminator D xUsed to judge the authenticity of the input image, that is, whether the image comes from the source domain dataset or is output by the mapping module F;

[0016] Step S3: construct a training set based on the source domain data set and the target domain data set, input the training set into the distribution transfer generation model, and update the parameters of the model according to the output of the model;

[0017] Step S4: Use the trained mapping module G in the distribution transfer generation model to perform distribution uniformity processing on the input image to be processed.

[0018] As a further improvement of the marine target SAR image preprocessing method: each mapping module comprises a coding part, a conversion part and a decoding part connected in sequence;

[0019] The encoding part includes three convolutional layers, a Batch Normalization layer and a Leaky ReLU layer connected in sequence;

[0020] The conversion part includes three Resnet convolution blocks connected in sequence;

[0021] The decoding part includes 2 DeConvLayers and 1 Conv layer connected in sequence.

[0022] As a further improvement of the marine target SAR image preprocessing method: the image optimization module D Y Includes feature learning part and loss calculation part;

[0023] The feature learning part is used to learn multiple sets of feature maps from the input image;

[0024] The loss calculation part calculates the perceptual loss value based on the feature maps learned from the two input images.

[0025] As a further improvement of the marine target SAR image preprocessing method: the feature learning part includes a multi-scale feature extraction network and an aggregated feature pyramid network;

[0026] The multi-scale feature extraction network is based on the input image , , , and Five groups of feature maps with decreasing sizes;

[0027] The aggregate feature pyramid network includes a top-down pathway and a bottom-up pathway;

[0028] The top-down pathway is based on the feature map output by the multi-scale feature extraction network. , , and Four groups of feature maps with gradually increasing sizes; specifically:

[0029] ;

[0030] in, Represents the numbers of the 4 groups of feature maps; Indicates that After a convolution operation, a convolution layer with a convolution kernel of 1×1 and a step size of s=1 is performed; Indicates that After a convolution operation with a convolution kernel of 1×1 and a step size of s=1, Perform element-by-element addition;

[0031] The bottom-up pathway is based on the feature map output by the top-down pathway. , , and Four groups of feature maps with gradually decreasing sizes; specifically:

[0032] ;

[0033] in, Represents the numbers of the 4 groups of feature maps; Indicates that After a convolution operation, a convolution layer with a convolution kernel of 1×1 and a step size of s=1 is performed; Indicates that After a convolution operation with a convolution kernel of 1×1 and a step size of s=1, Perform element-by-element addition;

[0034] , , and The four sets of feature maps are the output results of the feature learning part.

[0035] As a further improvement of the marine target SAR image preprocessing method: the image optimization module D Y In , let us assume that for a certain input image, , , and The four sets of feature maps are the output results of the feature learning part, denoted as , ;

[0036] For feature map, and its size is , , , Respectively feature map height, width and number of channels, then Expand to get the size The matrix , ; The matrix The inner product is defined as Gram matrix of feature maps , whose size is ;

[0037] Assume that the input is to the image optimization module D Y The sample pairs in which the real image from the target domain dataset and the generated image output by the mapping module G have a common group, the perceptual loss value is calculated as:

[0038] For the input to the image optimization module D Y The A set of real images from the target domain dataset And the generated image output by the mapping module G , the overall distribution loss value is:

[0039] ;

[0040] in, for The first loss weight, and Respectively represent the generated images No. Gram matrix of feature maps and real image No. Gram matrix of feature maps;

[0041] For the input to the image optimization module D Y The A set of real images from the target domain dataset And the generated image output by the mapping module G , the feature loss value is:

[0042] ;

[0043] in, for The second loss weight, and Respectively represent the generated images No. feature maps and real images No. feature maps;

[0044] For the input The generated image output by the real image mapping module G from the target domain dataset is obtained by:

[0045] ;

[0046] in, and are the weights of the overall distribution loss and feature loss, respectively.

[0047] As a further improvement of the marine target SAR image preprocessing method, the specific method of step S3 is as follows:

[0048] Step S3-1: Randomly select from the source domain dataset and the target domain dataset samples, forming a dataset X from the source domain dataset b and dataset Y from the target domain dataset b ;

[0049] Step S3-2: Data set X b Each image in is input into the mapping module G, and the obtained images constitute the data set Y b_1 , and then the data set Y b_1 Each image in is input into the mapping module F, and the obtained images constitute the data set X b_2 ; Then, calculate the data set X b With dataset X b_2 A first reconstruction loss between , and updating parameters of mapping modules G and F according to the first reconstruction loss;

[0050] Step S3-3: Data set Y b Each image in is input into the mapping module F, and the obtained images constitute the data set X b_1 , and then the data set X b_1 Each image in is input into the mapping module G, and the obtained images constitute the data set Y b_2 ; Then, calculate the data set Y b With data set Y b_2 A second reconstruction loss between , and updating parameters of mapping modules G and F according to the second reconstruction loss;

[0051] Step S3-4: For data set X b All image samples are labeled with the real sample label 1, for the dataset X b_2 All image samples are annotated to generate sample label 0, and then the dataset X b and dataset X b_2All image samples are input to the discriminator D x In the above example, the discrimination loss is calculated based on whether the model determines whether the input sample is a real sample or a generated sample, and then the mapping module F and the discriminator D are updated based on the discrimination loss. x Parameters;

[0052] Step S3-5: Data set Y b and dataset Y b_2 Combine to get sample pairs, each of which contains a sample from the dataset Y b The image and the image generated by mapping module F and mapping module G in Y b_2 The corresponding image in , and then all sample pairs are input into the image optimization module D Y In the above example, the perceptual loss value is calculated pairwise and the mapping module F and the discriminator D are updated according to the loss value. x Parameters;

[0053] Steps S3-1 to S3-5 are executed repeatedly until the training times are reached.

[0054] As a further improvement of the marine target SAR image preprocessing method: in step S3-2, the first reconstruction loss is calculated as follows:

[0055] ;

[0056] in, Represents dataset X b Middle The pixel value of the p-th row, q-th column, and r-th channel of an image, Represents dataset X b_2 In the dataset X b Middle The pixel value of the p-th row, q-th column, and r-th channel of the image corresponding to the image; It means summing up the rows, columns and channels in sequence.

[0057] As a further improvement of the marine target SAR image preprocessing method: in step S3-3, the second reconstruction loss is calculated as follows:

[0058] ;

[0059] in, Represents data set Y b Middle The pixel value of the p-th row, q-th column, and r-th channel of an image, Represents data set Y b_2 In the dataset Y b Middle The pixel value of the p-th row, q-th column, and r-th channel of the image corresponding to the image; It means summing up the rows, columns and channels in sequence.

[0060] As a further improvement of the marine target SAR image preprocessing method: in step S3-4, the calculation method of the identification loss is:

[0061] ;

[0062] in, Indicates that the from dataset X b The image is input to the discriminator D x The results obtained in Indicates that the from dataset X b_2 The image is input to the discriminator D x The results obtained in .

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] From the perspective of data processing, the present invention proposes a preprocessing method for SAR images of marine targets, and designs an image distribution transfer generation algorithm (UniGAN) based on a generative adversarial network. This method can improve and unify the resolution of the input image, thereby improving the consistency of the data and further improving the detection effect. In order to make the generated image more semantically similar to the target image, the present invention also introduces an image optimization module based on multi-scale target detection, and finally achieves a distribution that is beneficial to downstream detection tasks and has more consistent semantic information. Specifically:

[0065] 1. The present invention first constructs source domain and target domain datasets for the SAR image distribution migration task. By collecting SAR maritime target images from different channels, it constructs a source domain dataset covering different distributions and a target domain dataset with concentrated distribution and good image quality, which provides a sufficient foundation for model training.

[0066] 2. The distribution transfer generation model constructed by the present invention is based on unsupervised learning. Under the premise of no one-to-one corresponding high-resolution target images, it maps the source domain data with different distributions and different qualities to the target domain with concentrated distribution and better quality, thereby eliminating the domain differences and providing a better data source for the next step of detection, and finally realizing a maritime target detection method that can adapt to different data distributions.

[0067] 3. The image optimization module designed for detection tasks in the present invention can prompt the mapping module G to learn features related to detection, thereby further improving the support of the generated image for subsequent detection tasks and improving the effect of downstream detection algorithms.

[0068] Fourth, the present invention calculates the reconstruction loss after the cyclic generation, so as to keep the image content features consistent as much as possible when the image is distributed transformed between G and F, thereby reducing the drift of semantic information.

[0069] Through the method proposed in this patent, the model can map multiple source domain SAR maritime target detection images with inconsistent distribution and uneven quality into a concentrated, high-resolution and good-quality target domain, thereby providing good data input for subsequent detection tasks, and ultimately improving the algorithm's detection effect on images in different domains and realizing domain generalization function. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 Schematic diagram of the overall framework of the model in the present invention; this figure only represents the basic framework structure of the model and is not consistent with the model training process;

[0071] Figure 2 It is a schematic diagram of the network structure of mapping modules G and F;

[0072] Figure 3 Schematic diagram of the structure of the Resnet convolution block in the conversion part of the mapping module G and F;

[0073] Figure 4 Optimizing Module D for Images Y Schematic diagram of the structure of the feature learning part;

[0074] Figure 5 is the discriminator D x Schematic diagram of the network structure. DETAILED DESCRIPTION

[0075] The technical solution of the present invention is described in detail below with reference to the accompanying drawings:

[0076] A method for preprocessing a SAR image of a marine target, comprising the steps of:

[0077] Step S1: construct source domain dataset and target domain dataset.

[0078] The source domain dataset contains images collected through different platforms and channels, and have distribution differences in acquisition parameters, and the marine targets in some images in the source domain dataset are small targets within a specified range, which are usually obtained by directly cleaning the original dataset.

[0079] The acquisition parameters include acquisition distance, angle, sensor, resolution, etc.

[0080] The images in the target domain dataset are images whose resolution and image quality meet preset standards or above, and are obtained through manual selection. The construction can be completed by selecting marine targets from high-resolution images.

[0081] To facilitate model training, all images are uniformly resized to 256×256×3 through bicubic interpolation.

[0082] Step S2: construct a distribution migration generation model.

[0083] Based on CycleGAN, a distribution migration generative network is designed to achieve the function of unified data distribution. Its framework is as follows Figure 1 shown.

[0084] The distribution migration generation model includes a mapping module G and a mapping module F, and also includes a discriminator D x and image optimization module D Y .

[0085] The mapping module G and the mapping module F have the same structure, and the basic structure is an autoencoder. The mapping module G is used to convert the input image into an image with consistent distribution with the target domain data set, and the mapping module F is used to convert the input image into an image with consistent distribution with the source domain data set.

[0086] Specifically, Figure 2 ,In order to improve the effect of the generated image, this method embeds a ,transformation network between the encoder and the decoder to better transfer the ,features of different layers.,Therefore, each mapping module includes an encoding part, a ,transformation part and a decoding part that are connected in sequence.

[0087] The encoding part includes three sequentially connected convolutional layers ConvLayer, a BatchNormalization layer and a Leaky ReLU layer. Each ConvLayer contains a convolutional layer (the convolution kernel size is 3×3), and the three convolutional layers use 64, 128, and 256 convolution kernels respectively.

[0088] The conversion part includes three sequentially connected Resnet convolution blocks (Resnet Block1, ResnetBlock2, Resnet Block3), such as Figure 3 Each block contains three convolutional layers, whose convolution kernels are 1×1, 3×3, and 1×1, and the number of convolution kernels is 64, 64, and 256 respectively. A ReLu activation function is added after each convolutional layer. In order to better transmit information, the input of the first convolutional layer is connected across layers and integrated into the output of the third convolutional layer by point-by-point addition.

[0089] The decoding part includes two DeConvLayer and one Conv layer connected in sequence. DeConvLayer contains a deconvolution layer (convolution kernel size is 3×3), a Batch Normalization layer, and a Leaky ReLU layer. The two convolution layers use 256 and 128 convolution kernels respectively. The last Conv layer outputs a 3-channel image to align with the input image size.

[0090] The image optimization module D Y It is used to calculate the perceptual loss value between the image Y from the target domain dataset and the image Y' output by the mapping module G, so that the distribution of the image generated by the mapping module G is similar to that of the target domain dataset.

[0091] Traditional perceptual loss networks are usually based on pre-trained classification models such as VGG16, which capture and align the classification features of images by default. In order to make the images reconstructed by the algorithm with uniform distribution more conducive to the subsequent target detection tasks, this paper proposes an image optimization module D based on a multi-scale target detection network. Y ,By combining the multi-scale feature learning network in the offshore target detection model, the image structure and semantic information alignment,which is beneficial to target detection, is achieved.

[0092] Specifically, Figure 4 , the image optimization module D Y It includes feature learning part and loss calculation part.

[0093] The feature learning part is used to learn multiple groups of feature maps from the input image.

[0094] Furthermore, the feature learning part includes a multi-scale feature extraction network and an aggregated feature pyramid network.

[0095] The multi-scale feature extraction network often uses but is not limited to pre-trained networks such as VGG19, ResNet18, and ResNet50. It obtains , , , and Five groups of feature maps with gradually decreasing sizes.

[0096] The aggregated feature pyramid network (PAFPN) includes a top-down pathway and a bottom-up pathway.

[0097] The top-down pathway is based on the feature map output by the multi-scale feature extraction network. , , and Four groups of feature maps with gradually increasing sizes. Specifically:

[0098] ;

[0099] in, Represents the numbers of the 4 groups of feature maps; Indicates that After a convolution operation, a convolution layer with a convolution kernel of 1×1 and a step size of s=1 is performed; Indicates that After a convolution operation with a convolution kernel of 1×1 and a step size of s=1, Perform element-wise addition.

[0100] The bottom-up pathway is based on the feature map output by the top-down pathway. , , and Four groups of feature maps with gradually decreasing sizes. Specifically:

[0101] ;

[0102] in, Represents the numbers of the 4 groups of feature maps; Indicates that After a convolution operation, a convolution layer with a convolution kernel of 1×1 and a step size of s=1 is performed; Indicates that After a convolution operation with a convolution kernel of 1×1 and a step size of s=1, Perform element-wise addition.

[0103] , , and The four sets of feature maps are the output results of the feature learning part, denoted as , .

[0104] The loss calculation part calculates the perceptual loss value based on the feature maps learned from the two input images.

[0105] Furthermore, for the feature map, and its size is , , , Respectively feature map height, width and number of channels, then Expand to get the size The matrix , ; The matrix The inner product is defined as Gram matrix of feature maps , whose size is Since the Gram matrix is ​​position-insensitive, it can be used to quantitatively measure image distribution.

[0106] Assume that the input is to the image optimization module D Y The sample pairs in which the real image from the target domain dataset and the generated image output by the mapping module G have a common group, the perceptual loss value is calculated as:

[0107] For the input to the image optimization module D Y The A set of real images from the target domain dataset And the generated image output by the mapping module G , extract features from each image and calculate the distribution loss according to the sample pairs. The overall distribution loss value is:

[0108] ;

[0109] in, for The first loss weight is uniformly set to 1 in this paper; and Respectively represent the generated images No. Gram matrix of feature maps and real image No. The Gram matrix of the feature map.

[0110] For the input to the image optimization module D Y The A set of real images from the target domain dataset And the generated image output by the mapping module G , extract features from each image and calculate feature loss according to sample pairs. The feature loss value is:

[0111] ;

[0112] in, for The second loss weight is uniformly set to 1 in this paper; and Respectively represent the generated images No. feature maps and real images No. A feature map.

[0113] For the input The generated image output by the real image mapping module G from the target domain dataset is obtained by:

[0114] ;

[0115] in, and are the weights of the overall distribution loss and feature loss respectively. In order to better ensure the accuracy of the content in the generated image, , .

[0116] The discriminator D x It is used to judge the authenticity of the input image (image X of the source domain dataset and image X' output by the mapping module F), that is, whether the image comes from the source domain dataset or is output by the mapping module F, so that the distribution of the image generated by the mapping module F is similar to that of the source domain dataset.

[0117] like Figure 5 , where the convolution layer in the feature extraction module is set to a stride of 2. After 6 convolution operations (including 6 convolution layers with k=3, s=2 and ReLu and BN layers, the number of convolution kernels are: 16, 32, 64, 128, 256, 256 respectively), the 256×256×3 input image is convolved into a 4×4×256 feature map, and then the 4×4×256 feature map is transformed into a one-dimensional vector with a length of 4096 through the Reshape operation. Then, the Normalization function is used to normalize it, and the values ​​in the above one-dimensional vector are mapped to [0,1]. Then, a fully connected layer (FC) containing two nodes is used to map the normalized one-dimensional vector to two nodes, and then the output is output after the sigmoid activation function, which represents a two-dimensional judgment of whether the input image is true / false (True / False). True means that the input image comes from the source domain dataset, and False means that the input image is the image output by the mapping module F.

[0118] Step S3: construct a training set based on the source domain data set and the target domain data set, input the training set into the distribution transfer generation model, and update the parameters of the model according to the output of the model.

[0119] In this embodiment, the stochastic gradient descent method SGD is used as the optimizer, the SGD learning rate parameter is set to 0.0002, the number of training times is 500, the batch size m is 128, and the training is performed according to the following steps:

[0120] Step S3-1: Randomly select from the source domain dataset and the target domain dataset samples, forming a dataset X from the source domain dataset band dataset Y from the target domain dataset b .

[0121] Step S3-2: Data set X b Each image in is input into the mapping module G, and the obtained images constitute the data set Y b_1 , and then the data set Y b_1 Each image in is input into the mapping module F, and the obtained images constitute the data set X b_2 ; Then, calculate the data set X b With dataset X b_2 A first reconstruction loss between , and parameters of mapping module G and mapping module F are updated according to the first reconstruction loss.

[0122] The first reconstruction loss is calculated as:

[0123] ;

[0124] in, Represents dataset X b Middle The pixel value of the p-th row, q-th column, and r-th channel of an image, Represents dataset X b_2 In the dataset X b Middle The pixel value of the p-th row, q-th column, and r-th channel of the image corresponding to the image; It means summing up the rows, columns and channels in sequence.

[0125] Step S3-3: Data set Y b Each image in is input into the mapping module F, and the obtained images constitute the data set X b_1 , and then the data set X b_1 Each image in is input into the mapping module G, and the obtained images constitute the data set Y b_2 ; Then, calculate the data set Y b With dataset Y b_2 A second reconstruction loss between , and the parameters of mapping module G and mapping module F are updated according to the second reconstruction loss.

[0126] The second reconstruction loss is calculated as:

[0127] ;

[0128] in, Represents data set Y b Middle The pixel value of the p-th row, q-th column, and r-th channel of an image, Represents data set Y b_2In the dataset Y b Middle The pixel value of the p-th row, q-th column, and r-th channel of the image corresponding to the image; It means summing up the rows, columns and channels in sequence.

[0129] Step S3-4: For data set X b All image samples are labeled with the real sample label 1, for the dataset X b_2 All image samples are annotated to generate sample label 0, and then the dataset X b and dataset X b_2 All image samples are input to the discriminator D x In the above example, the discrimination loss is calculated based on whether the model determines whether the input sample is a real sample or a generated sample, and then the mapping module F and the discriminator D are updated based on the discrimination loss. x Parameters.

[0130] The discrimination loss is calculated as:

[0131] ;

[0132] in, Indicates that the from dataset X b The image is input to the discriminator D x The results obtained in Indicates that the from dataset X b_2 The image is input to the discriminator D x The results obtained in .

[0133] Step S3-5: Data set Y b and dataset Y b_2 Combine to get sample pairs, each of which contains a sample from the dataset Y b The image and the image generated by mapping module F and mapping module G in Y b_2 The corresponding image in , and then all sample pairs are input into the image optimization module D Y In the above example, the perceptual loss value is calculated pairwise and the mapping module F and the discriminator D are updated according to the loss value. x Parameters.

[0134] Steps S3-1 to S3-5 are executed repeatedly until a preset number of training times is reached.

[0135] Step S4: Use the trained mapping module G in the distribution transfer generation model to perform distribution uniformity processing on the input image to be processed.

Claims

1. A method for preprocessing SAR images of marine targets, characterized by the following steps: include: Step S1, constructing a source domain dataset and a target domain dataset; The source domain dataset contains images collected through different platforms and channels, and having different distributions of acquisition parameters, and the marine targets in some images in the source domain dataset are small targets within a specified range of size; The images in the target domain dataset are images whose resolution and image quality meet the preset standards; Step S2, constructing a distribution migration generation model; The distribution migration generation model includes a mapping module G and a mapping module F, and also includes a discriminator D x and image optimization module D Y ; The mapping module G and the mapping module F have the same structure; the mapping module G is used to convert the input image into an image with consistent distribution of the target domain data set, and the mapping module F is used to convert the input image into an image with consistent distribution of the source domain data set; The image optimization module D Y Used to calculate the perceptual loss value between the image from the target domain dataset and the image output by the mapping module G; The image optimization module D Y Includes feature learning part and loss calculation part; The feature learning part is used to learn multiple sets of feature maps from the input image; The feature learning part includes a multi-scale feature extraction network and an aggregated feature pyramid network; The multi-scale feature extraction network is based on the input image , , , and Five groups of feature maps with decreasing sizes; The aggregate feature pyramid network includes a top-down pathway and a bottom-up pathway; The top-down pathway is based on the feature map output by the multi-scale feature extraction network. , , and Four groups of feature maps with gradually increasing sizes; specifically: ; in, Represents the numbers of the 4 groups of feature maps; Indicates that After a convolution operation, a convolution layer with a convolution kernel of 1×1 and a step size of s=1 is performed; Indicates that After a convolution operation with a convolution kernel of 1×1 and a step size of s=1, Perform element-by-element addition; The bottom-up pathway is based on the feature map output by the top-down pathway. , , and Four groups of feature maps with gradually decreasing sizes; specifically: ; in, Represents the numbers of the 4 groups of feature maps; Indicates that After a convolution operation, a convolution layer with a convolution kernel of 1×1 and a step size of s=1 is performed; Indicates that After a convolution operation with a convolution kernel of 1×1 and a step size of s=1, Perform element-by-element addition; , , and The four sets of feature maps are the output results of the feature learning part; The loss calculation part calculates a perceptual loss value based on feature maps learned from two input images; The image optimization module D Y In , let us assume that for a certain input image, , , and The four sets of feature maps are the output results of the feature learning part, denoted as , ; For feature map, and its size is , , , Respectively feature map height, width and number of channels, then Expand to get the size The matrix , ; The matrix The inner product is defined as Gram matrix of feature maps , whose size is ; Assume that the input is to the image optimization module D Y The sample pairs in which the real image from the target domain dataset and the generated image output by the mapping module G have a common group, the perceptual loss value is calculated as: For the input to the image optimization module D Y The A set of real images from the target domain dataset And the generated image output by the mapping module G , the overall distribution loss value is: ; in, for The first loss weight, and Respectively represent the generated images No. Gram matrix of feature maps and real image No. Gram matrix of feature maps; For the input to the image optimization module D Y The A set of real images from the target domain dataset And the generated image output by the mapping module G , the feature loss value is: ; in, for The second loss weight, and Respectively represent the generated images No. feature maps and real images No. feature maps; For the input The generated image output by the real image mapping module G from the target domain dataset is obtained by: ; in, and are the weights of overall distribution loss and feature loss respectively; The discriminator D x Used to judge the authenticity of the input image, that is, whether the image comes from the source domain dataset or is output by the mapping module F; Step S3: construct a training set based on the source domain data set and the target domain data set, input the training set into the distribution transfer generation model, and update the parameters of the model according to the output of the model; The specific method of step S3 is: Step S3-1: Randomly select from the source domain dataset and the target domain dataset samples, forming a dataset X from the source domain dataset b and dataset Y from the target domain dataset b ; Step S3-2: Data set X b Each image in is input into the mapping module G, and the obtained images constitute the data set Y b_1 , and then the data set Y b_1 Each image in is input into the mapping module F, and the obtained images constitute the data set X b_2 ; Then, calculate the data set X b With dataset X b_2 A first reconstruction loss between , and updating parameters of mapping modules G and F according to the first reconstruction loss; Step S3-3: Data set Y b Each image in is input into the mapping module F, and the obtained images constitute the data set X b_1 , and then the data set X b_1 Each image in is input into the mapping module G, and the obtained images constitute the data set Y b_2 ; Then, calculate the data set Y b With data set Y b_2 A second reconstruction loss between , and updating parameters of mapping modules G and F according to the second reconstruction loss; Step S3-4: For data set X b All image samples are labeled with the real sample label 1, for the dataset X b_2 All image samples are annotated to generate sample label 0, and then the dataset X b and dataset X b_2 All image samples are input to the discriminator D x In the above example, the discrimination loss is calculated based on whether the model determines whether the input sample is a real sample or a generated sample, and then the mapping module F and the discriminator D are updated based on the discrimination loss. x Parameters; Step S3-5: Data set Y b and dataset Y b_2 Combine to get sample pairs, each of which contains a sample from the dataset Y b The image and the image generated by mapping module F and mapping module G in Y b_2 The corresponding image in , and then all sample pairs are input into the image optimization module D Y In the above example, the perceptual loss value is calculated pairwise and the mapping module F and the discriminator D are updated according to the loss value. x Parameters; Steps S3-1 to S3-5 are executed repeatedly until the number of training times is reached; Step S4: Use the trained mapping module G in the distribution transfer generation model to perform distribution uniformity processing on the input image to be processed.

2. The method for preprocessing marine target SAR images as claimed in claim 1, characterized in that: Each mapping module comprises an encoding part, a conversion part and a decoding part connected in sequence; The encoding part includes three convolutional layers, a Batch Normalization layer and a Leaky ReLU layer connected in sequence; The conversion part includes three Resnet convolution blocks connected in sequence; The decoding part includes 2 DeConvLayers and 1 Conv layer connected in sequence.

3. The method for preprocessing marine target SAR images as claimed in claim 1, characterized in that: In step S3-2, the first reconstruction loss is calculated as follows: ; in, Represents dataset X b Middle The pixel value of the p-th row, q-th column, and r-th channel of an image, Represents dataset X b_2 In the dataset X b Middle The pixel value of the p-th row, q-th column, and r-th channel of the image corresponding to the image; It means summing up the rows, columns and channels in sequence.

4. The method for preprocessing marine target SAR images according to claim 1, wherein: In step S3-3, the second reconstruction loss is calculated as follows: ; in, Represents data set Y b Middle The pixel value of the p-th row, q-th column, and r-th channel of an image, Represents data set Y b_2 In the dataset Y b Middle The pixel value of the p-th row, q-th column, and r-th channel of the image corresponding to the image; It means summing up the rows, columns and channels in sequence.

5. The method for preprocessing marine target SAR images as claimed in claim 1, characterized in that: In step S3-4, the calculation method of identification loss is: ; in, Indicates that the from dataset X b The image is input to the discriminator D x The results obtained in Indicates that the from dataset X b_2 The image is input to the discriminator D x The results obtained in .

Citation Information

Patent Citations

  • Remote sensing image small target detection method based on resolution enhancement

    CN111709307A

  • Cross-domain target detection method based on domain self-adaption

    CN114626461A

  • Cross-modal unsupervised domain adaptive medical image segmentation method

    CN115731178A