Construction method of interference SAR sample generation model and image target detection model

Abstract

The application belongs to the technical field of image detection, and discloses a method for constructing an interference SAR sample generation model and an image target detection model. The sample generation model construction method comprises the following steps: obtaining an interference-free SAR image and an interference SAR image of a research object, synthesizing the interference-free SAR image and the interference SAR image to obtain a content image; taking a randomly extracted interference SAR image as a style image, inputting the content image and the style image into a style transfer network for training to generate an interference SAR sample, calculating a training loss and updating network parameters through back propagation to improve the fidelity of the generated interference SAR sample; training multiple times until the training loss converges to obtain an interference SAR sample generation model. The style transfer network comprises a reversible residual module, a channel refinement module and a conversion module. Through the above method, the problem of insufficient SAR data samples can be solved, and more realistic interference SAR samples can be quickly generated.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field related to image detection, and more particularly to a method for constructing an interference SAR sample generation model and an image target detection model. BACKGROUND

[0002] Synthetic Aperture Radar (SAR) may be subject to diversified complex electromagnetic interference in its working frequency band, such as satellite signals, wireless communication signals, and local active interference signals. The interference signals in the SAR image may cover or confuse the target signals, resulting in false positives or false negatives of the target detection algorithm.

[0003] Deep learning is an algorithm based on a large amount of data. Using deep learning to train a target detection model can quickly analyze complex SAR images to deal with various interferences. However, the performance of the detection and recognition model is directly related to the training quantity. To carry out research on interference SAR image target detection algorithms based on deep methods, the lack of interference SAR data samples is the first problem to be solved.

[0004] In order to solve the problem of lack of SAR data samples, most researches obtain interference samples based on simulation. However, the simulation data often has domain differences with the actual data distribution, and the simulation interference samples obtained have large differences with the actual interference samples.

[0005] Therefore, it is urgent to propose a method for quickly generating more realistic interference SAR samples to improve the training effect of the image target detection model. SUMMARY

[0006] In view of the above defects or improvement needs of the prior art, the present application provides a method for constructing an interference SAR sample generation model and an image target detection model, which aims to quickly generate more realistic interference SAR samples to improve the training effect of the image target detection model.

[0007] To achieve the above-mentioned purpose, according to the first aspect of the present application, a method for constructing an interference SAR sample generation model is provided, which comprises:

[0008] Obtaining an interference-free SAR image and an interference SAR image of a research object, synthesizing the interference-free SAR image and the interference SAR image to obtain a content image;

[0009] Taking a randomly extracted interference SAR image as a style image, inputting the content image and the style image into a style transfer network for training to generate an interference SAR sample, calculating a training loss and updating network parameters through back propagation to improve the realism of the generated interference SAR sample;

[0010] Train multiple times until the training loss converges to obtain an interference SAR sample generation model;

[0011] The style transfer network comprises a reversible residual module, a channel refinement module and a conversion module:

[0012] The reversible residual module is configured to perform forward residual calculation on features from different input images respectively to obtain first features corresponding to the different input images;

[0013] The channel refinement module is configured to perform forward residual calculation on the first features of different input images respectively to obtain second features corresponding to the different input images;

[0014] The conversion module is configured to fuse the second features of different input images to obtain third features;

[0015] The channel refinement module is configured to perform backward residual calculation on the third features to obtain fourth features;

[0016] The reversible residual module is configured to perform backward residual calculation on the fourth features to obtain a stylized image, the stylized image being a generated interference SAR sample;

[0017] The forward and backward residual calculation of the reversible residual module increases the number of output channels and reduces the feature size, and the forward and backward residual calculation of the channel refinement module maintains the number of output channels and the feature size unchanged.

[0018] In some embodiments, the style transfer network further comprises an injection padding module;

[0019] Before the reversible residual module performs forward residual calculation on features from different input images respectively, the injection padding module is configured to pad zeros to the content image and the style image respectively to increase the number of channels;

[0020] After the reversible residual module performs backward residual calculation on the fourth features, the injection padding module is configured to remove the zero-padded channels to obtain the stylized image.

[0021] In some embodiments, the expression for forward residual calculation on an arbitrary feature x is:

[0022] y1=x1+F(x2)

[0023] y2=x2

[0024] In the expression, x=x1∪x2, x1 and x2 are two groups of channel features obtained by equally dividing the feature x in the channel dimension;

[0025] The expression for backward residual calculation on an arbitrary feature y is:

[0026] x² = y²

[0027] x1 = y1 - F(x2)

[0028] F() is the function for forming the residual network.

[0029] In some embodiments, the loss function used to calculate the training loss includes at least a style loss L. s Content loss L C Reconstruction loss L recon And similarity loss L MSSSIM The weighted average;

[0030] The style loss L s Measuring the stylized image I cs and the style diagram I s The similarity in style between them;

[0031] The content loss L C Measuring the stylized image I cs and the content diagram I c Content similarity between them;

[0032] The reconstruction loss L recon Measured by the stylized diagram I cs As a content image, with the aforementioned content image I c The reconstructed image I output after being input into the style transfer network as a style map Crecon With Content Image I c Gray-scale similarity between them;

[0033] The similarity loss L MSSSIM Measuring the stylized image I cs and the style diagram I s Similarity in image brightness, image contrast, and image structure between the images.

[0034] In some embodiments, the style loss L is calculated. s This includes calculating the stylized graph I. cs and the style diagram I s The L1 distance between them;

[0035] Calculate the content loss L C This includes calculating the stylized graph I. cs and the content diagram I c MSE loss between;

[0036] Calculate the reconstruction loss L recon Including the calculation and reconstruction of Figure I Crecon With Content Image I c The L1 norm between;

[0037] computing the similarity loss L MSSSIM , comprising computing a multi-scale structural similarity index between the stylized image I cs and the style image I s .

[0038] In some embodiments,

[0039] computing the style loss L s , the stylized image I cs and the style image I s are respectively input into a VGG19 network to obtain the feature map φ i (I cs ) of the corresponding stylized image and the feature map φ i (I s ) of the corresponding style image extracted by the i-th layer of the VGG19 network, and then the style loss L s =∑ i λ i ||(φ i (I cs ))-(φ i (I s ))|| is calculated, wherein λ i is the loss calculation weight of the i-th layer;

[0040] computing the content loss L C , the stylized image I cs and the content image I c are respectively input into the VGG19 network to obtain the feature map P(I cs ) of the corresponding stylized image and the feature map P(I c ) of the corresponding style image extracted by the VGG19 network, and then the MSE loss between the feature map P(I cs ) and the feature map P(I c ) is calculated as the loss L C .

[0041] In some embodiments, when computing the style loss L s , the feature maps of the ReLu1-1 layer, the ReLu2-1 layer, the ReLu3-1 layer, and the ReLu4-1 layer of the VGG19 network are respectively obtained, and the corresponding loss calculation weights are 1, 0.7, 0.5, and 0.3, respectively.

[0042] When computing the content loss L C , the feature map of the ReLu4-1 layer of the VGG19 network is obtained.

[0043] According to a second aspect of the present application, a method for constructing an interference SAR image target detection model is provided, which comprises:

[0044] an interference-free SAR image and an interference SAR image of a target are acquired, and the interference-free SAR image and the interference SAR image are synthesized to obtain a content image;

[0045] a randomly extracted interference SAR image is taken as a style image, and the content image and the style image are input into an interference SAR sample generation model constructed by the method according to any one of the above aspects to obtain an interference SAR sample;

[0046] a neural network is trained with the interference SAR sample and the interference SAR image as a training set, so that the neural network realizes target detection, and an interference SAR image target detection model is obtained.

[0047] According to a third aspect of the present application, an electronic device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method according to any one of the above aspects when executing the computer program.

[0048] According to a fourth aspect of the present application, a computer readable storage medium is provided, which stores a computer program, wherein the computer program is executed by a processor to implement the steps of the method according to any one of the above aspects.

[0049] Overall, compared with the prior art, the interference SAR sample generation model provided by the present application mainly has the following beneficial effects:

[0050] 1. The present application first synthesizes images with and without interference, and the preliminarily synthesized image has certain interference visual features, then the synthesized image (content image) and the interference image are input into a style transfer network, the style transfer network is trained, and through deep learning, the style transfer of the synthesized image can be realized, the feature level correction of the synthesized image is performed, and more realistic interference SAR samples are generated. Through the above method, the problem of lack of SAR data samples can be solved, and more realistic interference SAR samples can be quickly generated. The style transfer network includes a reversible residual module, a channel refinement module and a conversion module, and the network as a whole is based on a reversible framework. Based on the above design, more detailed features can be extracted, content information loss can be reduced, and the content affinity of the generated image can be better preserved, which is conducive to the training of the above model.

[0051] 2. In specific embodiments, a reversible injection filling module is provided, which can convert the input feature map into a higher dimensional representation, thereby providing richer feature expression capability.

[0052] 3. In specific embodiments, by constructing four losses, the similarity of the generated interference SAR sample to the image representation of the input style picture and the similarity of the generated interference SAR sample to the feature content of the input content picture can be comprehensively reflected. Training based on the above loss function can make the generated interference SAR sample more realistic. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 is a step flow chart of the interference SAR sample generation model construction method in an embodiment of the present application;

[0054] Figure 2 is a network structure diagram of the style transfer network in an embodiment of the present application;

[0055] Figure 3 is a forward residual calculation and backward residual calculation process diagram in an embodiment of the present application, (a) represents forward calculation, and (b) represents backward calculation;

[0056] Figure 4 is a generated interference SAR sample of a ship in an embodiment of the present application;

[0057] Figure 5 is a two-dimensional feature distribution diagram of non-interference data, interference data and generated data in an embodiment of the present application;

[0058] Figure 6 is a three-dimensional feature distribution diagram of non-interference data, interference data and generated data in an embodiment of the present application;

[0059] Figure 7 is a FID distance statistical diagram between the distance of the stylized picture and the content picture and the distance of the stylized picture and the style picture in an embodiment of the present application;

[0060] Figure 8 is a PSNR calculation value statistical diagram of the stylized picture and the content picture in an embodiment of the present application;

[0061] Figure 9 is a difference value statistical diagram of the equivalent number of looks ENL of the stylized picture and the style picture in an embodiment of the present application;

[0062] Figure 10 is a difference value statistical diagram of the information entropy of the stylized picture and the information entropy of the content picture in an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not used to limit the present application. In addition, the technical features involved in the various embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0064] Embodiment 1

[0065] As Figure 1 shown is a step flow chart of the interference SAR sample generation model construction method in an embodiment of the present application, and the main steps thereof will be introduced below.

[0066] Step S1: Obtain the non-interference SAR image and the interference SAR image of the research object, synthesize the non-interference SAR image and the interference SAR image to obtain a content image.

[0067] Taking a ship as an example, the non-interference SAR image and the interference data of different types of ships are obtained, and the non-interference SAR image and the interference SAR image are synthesized to obtain a content image. It can be understood that the research object can also be a car, a building or a biological object, etc.

[0068] Obtain the non-interference SAR image of the research object: the non-interference SAR Figure 1 image is generally easy to obtain, if the image size does not meet the setting, the image can be scaled and cropped to the set size to form a non-interference data set. Specifically, the image size is 256x256 pixels, and the image resolution is between 3m and 10m. In order to improve the generalization ability of training, the non-interference data set contains multiple scenes, multiple polarization modes and multiple types of ship targets.

[0069] Obtain the interference SAR image: the interference SAR image can be obtained by simulation or actual measurement. The simulation interference types include noise convolution interference, noise amplitude modulation interference, intermittent sampling and forwarding interference, combined interference and smart noise interference, etc. The actual measurement interference types can be obtained by actual flight shooting or collecting interference data of interference types such as radio frequency interference and noise interference in public satellite data. Finally, the collected and simulated data form an interference scene database.

[0070] Synthesize the non-interference SAR image and the interference SAR image: the image fusion method in the traditional computer vision algorithm can be used to synthesize two images into one image. A specific synthesis algorithm is given below:

[0071] Set the parameters a and β, set the original non-interference SAR image as I ori , set the interference SAR image as I inter , and the merged image I cThe pixel value at the (m, n) coordinate can be written as:

[0072] I com (m, n) = 255 - (255 - I ori (m, n))*(255 - I inter (m, n)*a) / 255) (1+β)

[0073] Step S2: taking the randomly extracted interference SAR image as a style image, inputting the content image and the style image into the style transfer network for training, generating an interference SAR sample, calculating a training loss, and updating network parameters based on back propagation to improve the fidelity of the generated interference SAR sample.

[0074] As Figure 2 Fig. 1 shows a network structure diagram of a style transfer network in an embodiment of the present application, which includes reversibly residual modules, channel refinement modules, and conversion modules connected in sequence. In an embodiment, an injection padding module can also be arranged before the reversibly residual modules.

[0075] Specifically, the reversibly residual modules are used to respectively perform forward residual calculation on the features from the content image and the features from the style image, to obtain a first feature corresponding to the content image and a first feature corresponding to the style image, which have increased channel numbers and reduced feature sizes.

[0076] Specifically, the channel refinement modules are used to respectively perform forward residual calculation on the first feature of the content image and the first feature of the style image, to obtain a second feature corresponding to the content image and a second feature corresponding to the style image, which have unchanged channel numbers and feature sizes.

[0077] Specifically, the conversion modules are used to fuse the second feature of the content image and the second feature of the style image to obtain a third feature.

[0078] Specifically, the channel refinement modules are used to perform reverse residual calculation on the third feature to obtain a fourth feature.

[0079] Specifically, the reversibly residual modules are used to perform reverse residual calculation on the fourth feature to obtain a stylized image, which is the generated interference SAR sample.

[0080] In an embodiment, if the channel numbers of the content image and the style image input into the style transfer network are low, an injection padding module can be arranged, which can perform reversible calculation.

[0081] Before the forward residual calculation of the features from different input images by the reversible residual module, the content image and the style image are zero-padded by the injection padding module to increase the number of channels, and then the features after the channel expansion of the padding module are input into the reversible residual module to perform the forward residual calculation of the features from different input images.

[0082] After the reverse residual calculation of the fourth feature by the reversible residual module, the zero-padded channels are removed by the injection padding module to obtain the stylized image.

[0083] The reversible injection padding module is set, so that the input feature map can be converted into a higher-dimensional representation, thereby providing more rich feature expression capability.

[0084] The processing of the style transfer network includes a forward processing process and a reverse processing process, wherein the reversible residual module and the channel refinement module can perform reversible residual calculation, the reversible residual calculation includes forward residual processing and reverse residual processing, and the specific calculation of the reversible residual can refer to the calculation of an existing reversible residual network.

[0085] In the embodiment, a classic NICE (full name: Non-linear Independent Components Estimation) reversible residual generation network is used. The forward residual calculation and reverse residual calculation process thereof is as shown in Figure 3 The corresponding formula is as follows:

[0086] The expression for forward residual calculation of any feature x is:

[0087] y1=x1+F(x2)

[0088] y2=x2

[0089] In the formula, x=x1∪x2, x1 and x2 are two groups of channel features obtained by equally dividing the channel dimension of the feature x, and F() is a function formed by a residual network.

[0090] The expression for reverse residual calculation of any feature y is:

[0091] x2=y2

[0092] x1=y1-F(x2)

[0093] In the formula, y=y1∪y2, y1 and y2 are two groups of channel features obtained by equally dividing the channel dimension of the feature y.

[0094] The function F() is related to the specific design of the residual network. In the present application, although both the reversible residual module and the channel refinement module perform residual calculation, their purposes are different. In the reversible residual module, the receptive field is increased through residual calculation, and the number of channels of the output of the residual network is increased and the feature size is reduced compared with the input. In the channel refinement module, deep mining of features is achieved through residual calculation, and the number of channels and the feature size of the output of the residual network remain unchanged compared with the input.

[0095] In the present embodiment, the structural framework of the residual network structure in the reversible residual module and the residual network structure in the channel refinement module is the same, but the parameter design is different to adapt to the change requirement of the number of channels and the feature size.

[0096] In an embodiment, the residual network includes first to third sub-networks connected in sequence and a last adjustment layer, each sub-network first performs reflection padding on the edges of the image through a reflection padding layer (reflectionpad2d) and then passes through a convolution layer and a Relu activation function, and the adjustment layer (Squeeze) is used to adjust the number of channels and the size of the features to a set value. reflectionpad2d is a module in PyTorch, which is used for reflection padding on the edges of the image. In a convolutional neural network (CNN), in order to avoid the reduction of image size during convolution operation, it is often necessary to perform padding operation on the edges of the image to keep the size of the output feature map consistent with that of the input feature map.

[0097] Specifically, for the residual network in the reversible residual module, the number of channels of the output of the first sub-network is 1 / r times the number of input channels, the feature scale is unchanged, and r is the sampling rate of the reversible residual network; the second sub-network maintains the number of channels and the feature scale unchanged, and the number of channels of the output of the third sub-network is r times the number of input channels.

[0098] The following Table 1 is a specific processing flow of the residual network in the reversible residual module, the input of the network is (c, h, w), and the output is (2c, h / 2, w / 2), the final number of channels is increased, and the feature size is reduced.

[0099] Table 1 F function operation processing flow in reversible residual module

[0100]

[0101] In the table, Conv+Relu represents convolution operation and Relu activation operation, and Squueze is a size and channel number adjustment (reshape) operation that converts spatial information of a tensor into channel information.

[0102] In an embodiment, the reversible residual module can include a plurality of identical residual networks, and multiple residual calculations are performed through the reversible residual module to gradually increase the receptive field.

[0103] It can be understood that in the channel refinement module, the channels and sizes of the features also need to be adjusted to adapt to the conversion module.

[0104] The following Table 2 shows the processing flow of the channel refinement module in a specific example. The number of channels and the size of the features remain unchanged in each reversible residual calculation, and then the size is adjusted to adapt to the conversion module.

[0105] Table 2 Forward processing flow of channel refinement module

[0106] Input dimensions (c, h, w) Operation Output dimensions (256, h, w) Reversible residual computation (256, h, w) (256, h, w) Reversible residual computation (256, h, w) (256, h, w) Adjust dimensions and number of channels (64, 2h, 2w) (64, 2h, 2w) Adjust dimensions and number of channels (16, 4h, 4w)

[0107] It can be understood that in the reverse processing process, the channel refinement module first adjusts the size and the number of channels and then performs reversible residual calculation in the input residual network.

[0108] In an embodiment, the conversion module is a Cholesky decomposition-based WCT (cWCT), and the processing flow includes whitening and coloring. The conversion module converts the extracted content feature code and style feature code to obtain a stylized feature code representation. Compared with the traditional WCT, the cWCT uses the Cholesky decomposition method, which can effectively reduce the computational complexity; at the same time, the cWCT can better maintain the details and structure of the image. The input of the conversion is the feature representation tensor of the content image and the style image.

[0109] After the style transfer network outputs the interference SAR sample, the training loss between the output sample and the reference sample needs to be calculated, the training loss between the generated sample and the input content image and style image is calculated, and the network parameters are updated based on the back propagation to complete a round of training.

[0110] It can be understood that in order to make the generated sample more realistic, the generated interference SAR sample needs to be as close as possible to the content image in content and as close as possible to the style image in image expression, therefore, the loss function needs to measure the content similarity between the generated interference SAR sample and the content image and the image style similarity between the generated interference SAR sample and the style image.

[0111] In an embodiment, the loss function L total includes at least the weighted average of the style loss L s , the content loss L C , the reconstruction loss L recon , and the similarity loss L MSSSIM , which can be specifically represented as:

[0112] L total = λ1L s + λ2L C + λ3L recon + λ4L MSSSIM

[0113] wherein λ1, λ2, λ3, λ4 are preset proportionality parameters, which can be flexibly adjusted. In the embodiment, λ1, λ2, λ3, λ4 can be respectively taken as 0.4, 0.3, 0.1, 0.2.

[0114] style loss L s measures the style similarity between the stylized image I cs and the style image I s , the smaller the loss, the higher the similarity.

[0115] content loss L C measures the content similarity between the stylized image I cs and the content image I c .

[0116] reconstruction loss L recon measures the grayscale similarity between the reconstructed image I cs output by inputting the stylized image I c as the content image and the content image I Crecon as the style image into the style transfer network. c and the content image I c .

[0117] similarity loss L MSSSIM measures the similarity of image brightness, image contrast and image structure between the stylized image I cs and the style image I s .

[0118] By constructing the above four losses, the similarity of image representation between the generated interference SAR sample and the input style image and the similarity of feature content between the generated interference SAR sample and the input content image can be comprehensively reflected. Based on the above loss function for training, the generated interference SAR sample can be more realistic.

[0119] In an embodiment, the L1 distance between the stylized image I cs and the style image I s is calculated as the style loss L s .

[0120] The L1 distance is not sensitive to abnormal value noise, is more suitable for capturing texture and style features, and has higher calculation efficiency. Therefore, by calculating the L1 distance, the stylized image I cs and the style image I sthe similarity of the image styles between the stylized image I

[0121] In a specific operation, the stylized image I cs and the style image I s are input into the VGG19 network to obtain the feature map φ i (I cs ) of the corresponding stylized image and the feature map φ i (I s ) of the corresponding style image extracted by the i-th layer of the VGG19 network, and then the style loss L s is calculated.

[0122]

[0123] In the formula, λ i is the weight for calculating the loss of the i-th layer.

[0124] In this embodiment, the feature maps of the ReLu1-1 layer, the ReLu2-1 layer, the ReLu3-1 layer and the ReLu4-1 layer of the VGG19 network are obtained for the calculation of the style loss, i.e., i = 4, 9, 16, 23, and the corresponding weights λ i are 1, 0.7, 0.5 and 0.3, respectively.

[0125] The image style includes not only the texture information at a shallow level but also the semantic information at a deep level, and the multi-layer features of the image can be effectively extracted by the VGG19 network to extract the style features of the image.

[0126] In an embodiment, the content loss L C is calculated, including the calculation of the MSE loss between the stylized image I cs and the content image I c .

[0127] The MSE loss can better compare the global structure and is more sensitive to large pixel differences, and better preserves the information of the target. The calculation of the MSE loss can effectively evaluate the content similarity between the stylized image I cs and the content image I c .

[0128] In a specific operation, the stylized image I cs and the content image I c are input into the VGG19 network to obtain the feature map P(I cs ) of the corresponding stylized image and the feature map P(I c ) of the corresponding style image extracted by the VGG19 network, and then the MSE loss between the feature map P(I cs ) and the feature map P(I c ) is calculated as the loss L C .

[0129] L c =mse(P(I cs )-P(I c ))

[0130] More specifically, since content features are generally deep semantic features, we can choose to obtain the feature map of the ReLu4-1 layer of the VGG19 network to calculate the MSE loss, which can effectively extract the content features of the image.

[0131] In one embodiment, the reconstruction loss L is calculated. recon Including the calculation and reconstruction of Figure I Crecon With Content Image I c The L1 norm between them.

[0132] The network of this invention is a reversible framework. The stylized features obtained by forward inference and the fusion encoding are used as content features, and the feature representation of the original content image is used as style features. After inference by the style transfer network, the theoretical output, i.e. the style image, should be the original content image. Thus, the training process of the network can be constrained by the reconstruction loss.

[0133] In specific operations, the content diagram I will be used. C The style transfer network is input through the input channel of the style map, and the stylized map I is processed. cs The content map is input into the style transfer network, which performs forward and backward computations before outputting the reconstructed map I. Crecon Calculate and reconstruct Figure I Crecon and content diagram I C The L1 norm is used as the reconstruction loss L recon :

[0134] L recon =∥I Crecon -I C ∥1

[0135] In one embodiment, the similarity loss L is calculated. MSSSIM Including computational stylized graph I cs and style diagram I s The multiscale structural similarity index MSSSIM between them.

[0136] The similarity loss L can be calculated using conventional methods. MSSSIM This includes: calculating the stylized graph I separately. cs With style diagram I s Brightness similarity between l(I) cs ,I s ), contrast similarity c(I) cs ,I s) and structural similarity s(I cs ,I s ), the product operation of the three differences is obtained Similarity loss L MSSSIM :

[0137]

[0138]

[0139]

[0140]

[0141] wherein, is the mean of the stylized image, is the mean of the style image, is the variance of the stylized image, is the variance of the style image, is the covariance of the stylized image and the style image, and c1, c2, and c3 are parameters to prevent the denominator from being 0.

[0142] Step S3: multiple training until the training loss converges, to obtain the jamming SAR sample generation model.

[0143] The network training of the embodiment uses the sar-ship-dataset dataset as the dataset of the style transfer network. During network training, the initial learning rate is set to 0.0001, the learning rate decay rate is set to 0.00005, the Adam optimization algorithm is used for iterative updating of the network, and the training number is 20 rounds.

[0144] Next, the effectiveness of the technical scheme of the present application is verified.

[0145] The synthetic data is used as the content image, and the randomly sampled image of the jamming scene database is used as the style image. The improved style transfer network is used for style transfer, and the obtained stylized image is used as the final generated jamming data. The generated jamming SAR sample of the ship is shown in Figure 4 .

[0146] The proposed jamming data generation method is demonstrated from five aspects of subjective analysis of generated images, feature space distribution of generated images, feature distance between generated images and target images, image quality evaluation of generated images, and influence of generated images on detection model performance.

[0147] (1) Subjective analysis of generated images

[0148] The generated data is observed, and from the subjective view, the saliency of the target in the generated data by the present application is low, the contrast between the target and the background is low, the recognition of the target in the image is reduced, and it is difficult to accurately identify and analyze. Secondly, the subjective quality effect of the generated image is poor, and the overall image presents features such as blur, distortion and noise points. From the perspective of interference effect, these images visually present similar interference effects to real SAR interference images. These interferences include features such as speckles, stripes and blurring of target edge details in the image. This similarity shows that the present method can simulate the interference characteristics of real SAR images to a certain extent.

[0149] (2) Feature space distribution of generated images

[0150] Suppose that the data without interference is distributed in domain A, the interference data is distributed in domain B, that is, the target domain, and the generated data after the style migration of the synthesized image is distributed in domain C. If the feature set F C of domain C is close to the feature set F B of domain B, it is indicated that in the feature distribution, the distribution of domain B is close to the distribution of domain C, that is, the domain distribution of the generated data is close to the domain distribution of the interference data, which indicates that the generated data has reached the simulation interference effect.

[0151] As shown in Figure 5 , the two-dimensional feature distribution diagram of the data without interference, the interference data and the generated data, Figure 6 is the three-dimensional feature distribution diagram of the data without interference, the interference data and the generated data.

[0152] Observing Figure 5 and Figure 6 , the red points in the figure represent the feature dimension reduction results of the interference scene samples, the blue points represent the feature dimension reduction results of the original non-interference data, and the yellow points are the feature dimension reduction results of the interference data generated by the present method. It can be seen that the style data set composed of the interference scene data of the red data points is closer to the feature distribution space of the generated stylized image, that is, the space composed of the yellow data points, and is farther away from the feature distribution space of the original image, that is, the space composed of the blue data points. That is, the data generated by the SAR interference data generation method of the present application is close to the features of the interference image at the feature level.

[0153] (3) Feature distance between generated images and target images

[0154] From the feature distribution distance between the non-interference data, the generated data and the interference data, the distance measurement method uses FID (Frechet Inception Distance).

[0155] Using the data generated by this invention, FID distance analysis was performed on the content image and style image, respectively, with the stylized image. The calculation results are as follows: Figure 7 As shown. The x-axis in the image represents the index of the image set, and the y-axis represents the difference between the distance between the stylized image and the content image, and the distance between the stylized image and the style image, that is:

[0156] y i =d(I styled ,I ori )-d(I styled ,I style )

[0157] Where d(x,y) represents the fid distance between the two images, I styled To generate images for this invention, I ori For the original image without interference, I style The images were designed to handle interference. Statistical analysis revealed 21,762 image pairs with y-values ​​greater than 0 and 912 image pairs with y-values ​​less than 0. Therefore, the data augmentation method proposed in this invention can, in most cases, reduce the distance between the interference-free image and the interference scene, enabling the generated image to possess the characteristics of the interference image.

[0158] (4) Image quality assessment of generated images

[0159] 1) PSNR

[0160] For the 22,674 stylized data images generated by this invention, the PSNR was calculated for each image pair with its corresponding original, undisturbed image, resulting in the calculation results for 22,674 image pairs as follows: Figure 8 As shown.

[0161] Figure 8 In the diagram, the x-axis represents the image pair index, and the y-axis represents the calculated PSNR values ​​of the stylized image and the content image. According to... Figure 8 It can be seen that the generated stylized data is mostly of lower quality than the original image, consistent with the effect of SAR images subjected to interference, which will affect the effect of target detection and recognition in the image.

[0162] 2) Equivalent visual number

[0163] For the 22,674 stylized data images generated by the method of this invention, the difference in equivalent number of views (ENL) is calculated between each image and its corresponding original, interference-free image, resulting in the calculation results for 22,674 image pairs as follows: Figure 9 As shown.

[0164] Figure 9 In the diagram, the x-axis represents the index of the image pair, and the y-axis represents the difference between the equivalent number of views of the original image and the equivalent number of views of the corresponding generated image. That is:

[0165] y i = enl(I ori )- enl(I styled )

[0166] Through statistics, the number of image pairs with y value greater than 0 is 20548 groups, and the number of image pairs with y value less than 0 is 2126 groups, so it can be obtained that the noise level of the generated image is higher than that of the original image, and the details become blurred, which is consistent with the effect of the disturbed SAR image, which will affect the performance of the target detection and recognition model.

[0167] 3) Image information entropy

[0168] The difference value of information entropy of 22674 style data generated by the application and the corresponding original undisturbed image is calculated, and the calculation results of 22674 image pairs are shown in Figure 10 .

[0169] Figure 10 The x-axis in the figure is the serial number of the image pair, and the y-axis is the difference between the information entropy of the stylized image and the information entropy of the content image. That is:

[0170] y i = H(I styled )- H(I content )

[0171] Through statistics, the number of image pairs with y value greater than 0 is 22190 groups, and the number of image pairs with y value less than 0 is 464 groups, that is, most of the images generated by the application have different degrees of information loss compared with the original image, which corresponds to the image performance of target edge blur, background spot noise and other situations, and the other part of the data with y value less than 0 may be caused by noise points in the image under the condition of lower signal-to-noise ratio. Through the above analysis, the overall quality of the generated data image is reduced, which will limit the performance of the target detection and recognition model.

[0172] Example 2

[0173] The application also relates to a method for constructing a disturbed SAR image target detection model, comprising:

[0174] Obtaining an undisturbed SAR image and a disturbed SAR image of a target; synthesizing the undisturbed SAR image and the disturbed SAR image to obtain a content image;

[0175] Taking a randomly extracted disturbed SAR image as a style image, inputting the content image and the style image into the disturbed SAR sample generation model constructed by the construction method in Example 1 to obtain a disturbed SAR sample;

[0176] The neural network is trained by taking the interference SAR sample and the interference SAR image as a training set, so that the neural network realizes target detection, and a target detection model of the interference SAR image is obtained.

[0177] Embodiment 3

[0178] The application also relates to an electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the above method when executing the computer program.

[0179] The electronic device can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The processor can be a central processing unit (CPU), and can also be another general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or another programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and the like. The memory can be used to store a computer program and / or a module, and the processor can realize various functions of the electronic device by running or executing the computer program and / or the module stored in the memory, and by calling data stored in the memory.

[0180] Embodiment 4

[0181] The application also relates to a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the above method.

[0182] Specifically, the memory can include a high-speed random access memory, and can also include a non-volatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one magnetic disk storage device, a flash memory device, or another volatile solid-state memory device.

[0183] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not contradict each other, they should be considered as the scope of the present application. It should be noted that the present application is intended to be illustrated by the above examples, and is not intended to be limited to the above examples.

[0184] The above embodiments only express several implementation ways of the present application, and the description is specific and detailed, but it should not be understood as a limitation to the patent scope. It should be pointed out that, for ordinary skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are all within the protection scope of the present application.

Claims

1. A method for constructing an interference SAR sample generation model, characterized in that, include: Obtain both interference-free and interference-interference SAR images of the research object, and synthesize the interference-free and interference-interference SAR images to obtain the content map; Using randomly selected interfering SAR images as style maps, the content maps and style maps are respectively input into the style transfer network for training to generate interfering SAR samples. The training loss is calculated and the network parameters are updated through backpropagation to improve the realism of the generated interfering SAR samples. The model for generating interfering SAR samples is obtained by training multiple times until the training loss converges. The style transfer network includes a reversible residual module, a channel refinement module, and a transformation module: The reversible residual module performs forward residual calculation on the features from different input images to obtain the first features corresponding to different input images. The channel refinement module performs positive residual calculation on the first features of different input images to obtain the second features corresponding to different input images. The conversion module fuses the second features of different input images to obtain the third feature; The fourth feature is obtained by performing reverse residual calculation on the third feature through the channel refinement module; The inverse residual calculation is performed on the fourth feature by the invertible residual module to obtain a stylized map, which is a generated interfering SAR sample; The forward and reverse residual calculations of the reversible residual module increase the number of output channels and reduce the feature size, while the forward and reverse residual calculations of the channel refinement module maintain the number of output channels and feature size unchanged. The loss function used to calculate training loss includes at least style loss. Content loss Reconstruction losses and similarity loss The weighted average; The style loss Measuring the stylized image and the style diagram Style similarity between them; The content loss Measuring the stylized image and the content diagram Content similarity between them; The reconstruction loss Measured by the stylized image As a content image, with the aforementioned content image The reconstructed image output after the style transfer network is input as a style map. With content image Gray-scale similarity between them; The similarity loss Measuring the stylized image and the style diagram Similarity in image brightness, image contrast, and image structure between the images.

2. The method for constructing an interference SAR sample generation model as described in claim 1, characterized in that, The style transfer network also includes an injection filling module; Before performing forward residual calculation on the features from different input images through the reversible residual module, the content map and style map are first zero-filled through the injection filling module to increase the number of channels. After the fourth feature is calculated by the inverse residual module, the zero-filled channels are removed by the injection filling module to obtain the stylized map.

3. The method for constructing an interference SAR sample generation model as described in claim 1, characterized in that, For any feature The expression for calculating the forward residual is: In the formula, , , Each will have the following features Two sets of channel features evenly divided according to channel dimension; For any feature The expression for calculating the reverse residual is: This is a function for forming the residual network.

4. The method for constructing an interference SAR sample generation model as described in claim 1, characterized in that, Calculate the style loss This includes calculating the stylized map. and the style diagram The L1 distance between them; Calculate the content loss This includes calculating the stylized map. and the content diagram MSE loss between; Calculate the reconstruction loss Including computational reconstruction maps With content image The L1 norm between them; Calculate the similarity loss This includes calculating the stylized map. and the style diagram The multi-scale structural similarity index between them.

5. The method for constructing an interference SAR sample generation model as described in claim 4, characterized in that, Calculate the style loss At that time, first, the stylized images are respectively... and the style diagram Input the VGG19 network and obtain the feature map of the corresponding stylized map extracted from the i-th layer of the VGG19 network. Feature maps of corresponding style maps Then calculate the style loss. In the formula, Calculate the weights for the loss of the i-th layer; Calculate the content loss At that time, first, the stylized images are respectively... and the content diagram Input the VGG19 network to obtain the feature maps of the corresponding stylized maps extracted by the VGG19 network. Feature maps of corresponding style maps Then calculate the feature map. and feature map The MSE loss between them is considered as a loss. .

6. The method for constructing an interference SAR sample generation model as described in claim 5, characterized in that, Calculate the style loss At that time, feature maps of ReLu1-1, ReLu2-1, ReLu3-1 and ReLu4-1 layers of the VGG19 network were obtained respectively, and the corresponding loss calculation weights were 1, 0.7, 0.5 and 0.3 respectively. Calculate the content loss At that time, the feature map of the ReLu4-1 layer of the VGG19 network is obtained.

7. A method for constructing a target detection model for interfering SAR images, characterized in that, include: Obtain both interference-free and interference-interference SAR images of the target; combine the interference-free and interference-interference SAR images to obtain the content map; Using randomly selected interference SAR images as style images, the content images and style images are respectively input into the interference SAR sample generation model constructed by the construction method described in any one of claims 1 to 6 to obtain interference SAR samples; A neural network is trained using jammed SAR samples and jammed SAR images as a training set to enable the neural network to perform target detection, thus obtaining a target detection model for jammed SAR images.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images