Ship target identification method, device, equipment, medium and product

Abstract

The invention discloses a ship target identification method, device, equipment, medium and product, and relates to the field of artificial intelligence and synthetic aperture radar target identification, and the method comprises the steps: training an S3GAN network model according to an amplitude ship image in a source domain SAR ship image data set and SAR ship phase data in a target domain SAR ship image data set; s3, the GAN network model comprises a first generator network model for carrying out a task of converting from an amplitude domain to a frequency spectrum domain, a second generator network model for carrying out a task of converting from the frequency spectrum domain to the amplitude domain, and a discriminator; generating pseudo-SAR frequency spectrum information of an amplitude ship image to be converted based on the S3GAN network model; and inputting the pseudo SAR frequency spectrum information and the amplitude ship image into a feature fusion recognition network for feature fusion, and recognizing the SAR ship target in the fused image, so that the ship target recognition precision can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence and synthetic aperture radar target recognition, and in particular to a ship target recognition method, device, equipment, medium and product. Background Art

[0002] In ship target recognition, Synthetic Aperture Radar (SAR) is more suitable for marine target recognition due to its all-weather observation and strong penetration. In recent years, with the rapid development of deep learning, deep neural networks have shown extremely excellent performance in target detection, target recognition, and image segmentation. At present, it is very popular and effective to use SAR images and deep learning methods for ship target recognition in the field of ship automatic target recognition. However, in deep learning methods, if the data set is small, the categories are unbalanced, and the representativeness is poor, the target recognition effect is unsatisfactory. Unfortunately, due to the difficulty of SAR image acquisition, the high cost of labeling manpower, and the military restrictions on ship targets, the sample size of SAR ship images is small and the categories are unbalanced. Therefore, it is very important to achieve high-precision SAR ship target recognition under the premise of small samples.

[0003] The main development directions of small sample target recognition can be divided into two categories:

[0004] 1) Improvements to the algorithm structure.

[0005] 2) Improvements to the dataset itself.

[0006] Improvements to the algorithm structure mainly use transfer learning, meta-learning and other methods, which are essentially the use of prior knowledge learned from large-scale data sets to apply to small sample data sets, without really processing the small sample data sets themselves; improvements to the data sets themselves are mainly through expanding the data scale, improving data quality, and enhancing supervisory information. The original data scale is expanded by rotating, scaling, cropping and other data enhancement methods, which is also the most popular data enhancement method in the past few years.

[0007] With the popularity of generative models in recent years, expanding the sample size by generating samples through generative model sampling is also a common data processing method. Since the resolution of public SAR datasets is generally low, super-resolution reconstruction to improve the data quality is also a dataset improvement method.

[0008] However, these methods are all improvements on SAR amplitude single-channel grayscale images, and do not take advantage of the rich spectral information of SAR images. Since SAR images have different imaging mechanisms from optical images, even two completely different targets may be extremely similar in amplitude images. Simply using SAR amplitude images to identify ships is not completely accurate, incomplete, or even biased. This may make it difficult to distinguish objects with similar textures but discriminative scattering patterns.

[0009] Unfortunately, in the field of ships, the currently available SAR ship datasets are almost all datasets that only contain amplitude information and lack spectral information. This means that the unique characteristics of SAR images with rich and complex physical scattering information are not fully utilized, resulting in low accuracy in subsequent ship target recognition. Summary of the invention

[0010] The purpose of this application is to provide a ship target recognition method, device, equipment, medium and product to solve the problem of low accuracy of ship target recognition.

[0011] To achieve the above objectives, this application provides the following solutions:

[0012] In a first aspect, the present application provides a ship target recognition method, comprising:

[0013] The S3GAN network model is trained according to the amplitude ship images in the source domain SAR ship image data set and the SAR ship phase data in the target domain SAR ship image data set; the S3GAN network model includes a first generator network model for performing an amplitude domain to spectrum domain conversion task, a second generator network model for performing a spectrum domain to amplitude domain conversion task and a discriminator; wherein the first generator network model is constructed by adding a spectrum transformation expansion module before the feature encoder of the generative adversarial network U-GAT-IT; the second network model is constructed by adding a multi-scale channel comprehensive processing module before the feature encoder of the generative adversarial network U-GAT-IT; the frequency domain transformation expansion module is used to convert a single-channel or three-channel amplitude ship image into a 128-channel amplitude ship image rich in deep features; the multi-scale channel comprehensive processing module is used to adjust the output image to the same scale as the input image to extract the depth information of the input image at the frequency domain level;

[0014] Inputting the amplitude ship image to be converted into the trained S3GAN network model to generate pseudo SAR spectrum information of the amplitude ship image to be converted;

[0015] The pseudo-SAR spectrum information and the amplitude ship image in the source domain SAR ship image data set are input into a feature fusion recognition network for feature fusion to identify the SAR ship target in the fused image; the feature fusion recognition network is trained based on the amplitude ship image in the source domain SAR ship image data set and the SAR ship phase data in the target domain SAR ship image data set.

[0016] In a second aspect, the present application provides a ship target identification device, comprising:

[0017] The S3GAN network model training module is used to train the S3GAN network model according to the amplitude ship image in the source domain SAR ship image data set and the SAR ship phase data in the target domain SAR ship image data set; the S3GAN network model includes a first generator network model for performing the amplitude domain to spectrum domain conversion task, a second generator network model for performing the spectrum domain to amplitude domain conversion task and a discriminator; wherein the first generator network model is constructed by adding a spectrum transformation expansion module before the feature encoder of the generative adversarial network U-GAT-IT; the second network model is constructed by adding a multi-scale channel comprehensive processing module before the feature encoder of the generative adversarial network U-GAT-IT; the frequency domain transformation expansion module is used to convert a single-channel or three-channel amplitude ship image into a 128-channel amplitude ship image rich in deep features; the multi-scale channel comprehensive processing module is used to adjust the output image to the same scale as the input image to extract the depth information of the input image at the frequency domain level;

[0018] A pseudo SAR spectrum information generation module is used to input the amplitude ship image to be converted into the trained S3GAN network model to generate pseudo SAR spectrum information of the amplitude ship image to be converted;

[0019] The SAR ship target recognition module is used to input the pseudo-SAR spectrum information and the amplitude ship image in the source domain SAR ship image data set into the feature fusion recognition network for feature fusion, and recognize the SAR ship target in the fused image; the feature fusion recognition network is trained based on the amplitude ship image in the source domain SAR ship image data set and the SAR ship phase data in the target domain SAR ship image data set.

[0020] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the ship target identification methods described above.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described ship target identification methods.

[0022] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned ship target identification methods.

[0023] According to the specific embodiments provided by the present application, the present application discloses the following technical effects: the present application trains a first generator network model having an amplitude domain to spectrum domain conversion task, a second generator network model having a spectrum domain to amplitude domain conversion task, and an S3GAN network model of a discriminator according to the amplitude ship images in the source domain SAR ship image dataset and the SAR ship phase data in the target domain SAR ship image dataset, wherein the first generator network model is constructed by adding a spectrum transformation expansion module before the feature encoder of the generative adversarial network U-GAT-IT; the frequency domain transformation expansion module is used to convert a single-channel or three-channel amplitude ship image into a 128-channel one rich in deep features. amplitude ship image, thereby realizing the expansion of the number of channels and alleviating the problem of domain conversion difficulties caused by the large difference in the modalities of the amplitude map and the spectrum information; the second network model is constructed by adding a multi-scale channel comprehensive processing module before the feature encoder of the generative adversarial network U-GAT-IT; the multi-scale channel comprehensive processing module is used to adjust the output image to the same scale as the input image, so as to extract the depth information of the input image at the frequency domain level, and solves the problem of information processing difficulties caused by too many spectrum information channels. The present application combines amplitude information and spectrum information for SAR ship target recognition, and makes full use of the unique characteristics of SAR ship images with rich and complex physical scattering information to improve the accuracy of ship target recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 This is a flow chart of a ship target identification method in an embodiment of the present application;

[0026] Figure 2 This is an overall framework diagram of a ship target recognition method in an embodiment of the present application;

[0027] Figure 3This is a structural diagram of a generator network model in an S3GAN network in an embodiment of the present application;

[0028] Figure 4 Detailed structural diagram of a multi-scale channel comprehensive processing module in one embodiment of the present application;

[0029] Figure 5 This is a schematic diagram of the data flow and loss function of S3GAN in one embodiment of the present application;

[0030] Figure 6 A detailed structural diagram of a feature fusion recognition network in an embodiment of the present application;

[0031] Figure 7 is a detailed structural diagram of a spectral processing attention module in one embodiment of the present application;

[0032] Figure 8 This is a schematic diagram of S3GAN generating pseudo-spectrum information and a heat map of the generation process in one embodiment of the present application. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0034] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0035] If the deep learning network can learn the feature mapping between the amplitude information domain and the spectrum information domain of the SAR ship target, it can generate pseudo-spectral information data for any amplitude map, and can fuse the features of the two to perform target recognition, ultimately achieving a better target recognition result than the result obtained by simply extracting the amplitude map features. Currently, the most common methods between domains are autoencoders and generative adversarial networks. Generative adversarial networks have gradually become the most popular domain-to-domain generation method due to the high robustness and high quality of the generated results they provide. For example, the U-GAT-IT network integrates an encoder into the generator for feature extraction, and introduces an adaptive attention mechanism to enable the generator and discriminator to focus on different areas of the image, thereby achieving high-quality generation results even with fewer samples.

[0036] However, these popular GANs are designed for single-channel or 3-channel image data, while the spectral information data rich in complex information still has 128 channels in its three-dimensional form even if its spatial information is discarded. The single-channel amplitude information data and the 128-channel spectral information data have very different forms, and it is not applicable to directly substitute them into the common popular GAN network and the target recognition network.

[0037] It can be seen that the use of U-GAT-IT network (i.e., directly using the open source generative adversarial network) to achieve the information conversion from SAR amplitude to SAR spectrum domain has the following technical problems:

[0038] 1. The number of channels is not equal. In domain conversion, the conversion from a single channel to 128 channels cannot achieve a one-to-one mapping of pixel relationships between channels.

[0039] 2. Too many spectral information channels. The common domain conversion network for image data has a limited field of view and cannot fully and effectively extract deeper feature information from spectral information data.

[0040] The embodiment of the present application provides a ship target identification method, which is executed by a computer device, specifically, it can be executed by a computer device such as a terminal or a server alone, or it can be executed by a terminal and a server together. In the embodiment of the present application, the method includes the following steps 101 to 103. Among them:

[0041] Step 101: train the S3GAN network model according to the amplitude ship image in the source domain SAR ship image dataset and the SAR ship phase data in the target domain SAR ship image dataset; Figure 2-Figure 3 As shown, the S3GAN network model includes a first generator network model for performing amplitude domain to spectrum domain conversion tasks, a second generator network model for performing spectrum domain to amplitude domain conversion tasks, and a discriminator; wherein, the first generator network model is constructed by adding a spectrum transformation expansion module before the feature encoder of the generative adversarial network U-GAT-IT; the second network model is constructed by adding a multi-scale channel comprehensive processing module before the feature encoder of the generative adversarial network U-GAT-IT; the frequency domain transformation expansion module is used to convert a single-channel or three-channel amplitude ship image into a 128-channel amplitude ship image rich in deep features; the multi-scale channel comprehensive processing module is used to adjust the output image to the same scale as the input image to extract the depth information of the input image at the frequency domain level. Figure 3 In a Denotes the encoder, De s represents the decoder, ω i represents the weight obtained by the auxiliary classifier, Representative aThe kth activation map in i Represents the feature map after weight activation.

[0042] Step 102: input the amplitude ship image to be converted into the trained S3GAN network model to generate pseudo SAR spectrum information of the amplitude ship image to be converted.

[0043] Step 103: Input the pseudo-SAR spectrum information and the amplitude ship image in the source domain SAR ship image dataset into a feature fusion recognition network for feature fusion to identify the SAR ship target in the fused image; the feature fusion recognition network is trained based on the amplitude ship image in the source domain SAR ship image dataset and the SAR ship phase data in the target domain SAR ship image dataset.

[0044] In an exemplary embodiment, step 101 also includes: converting the amplitude ship image into the same size using a linear interpolation method; performing joint time domain analysis on the SAR ship phase data, discarding the spatial information of the data after the time domain analysis, and determining the frequency spectrum information of the target domain SAR ship image with 128 channels.

[0045] Furthermore, the amplitude ship images are uniformly converted into a size of 224*224.

[0046] In an exemplary embodiment, the frequency domain transform expansion module specifically includes: a 7×7 convolution layer, a normalization layer, an activation function layer, a frequency domain processing layer, and a fusion processing layer connected in sequence; wherein the frequency domain processing layer includes a Fourier transform layer, a tensor shape expansion layer, an attention mechanism layer, and an inverse Fourier transform layer connected in sequence;

[0047] The single-channel or three-channel amplitude ship image enters the frequency domain processing layer after being segmented at different scales through the 7×7 convolution layer, the normalization layer and the activation function layer; the fusion processing layer is used to perform feature fusion on the channel dimension of the amplitude ship images of different scales that have passed through the frequency domain processing layer to obtain the output amplitude ship image of the frequency domain exchange expansion module, and the output amplitude ship image is the input of the feature encoder in the first generator network model.

[0048] Furthermore, first, the input data is input into the convolutional layer for preliminary feature extraction.

[0049] Then, after Fourier transform, the real and imaginary frequency domain data are spliced ​​in the frequency domain to obtain frequency domain data, so that further feature enhancement operations can be performed. At the same time, global pooling is performed to obtain channel attention weighting, and the weighted convolution operation is performed to obtain the final spectrum space representation.

[0050] Finally, the frequency domain data is converted to time domain data using inverse Fourier transform, and the spatial scale is transformed to adjust the data to the same scale as the input data, thus realizing the extraction of deep information from the input data at the frequency domain level.

[0051] Next, a portion of the input data is taken to extract the low-frequency component, and is processed in the same manner as the above steps to obtain.

[0052] Finally, the frequency-domain converted data is added to the original input data and the low-frequency component processing results, and the number of channels is adjusted to 128 through a convolution kernel with a kernel size of 1.

[0053] Through such a design, the spectrum transformation expansion module can convert single-channel or three-channel input data into 128-channel data with rich deep features, thereby expanding the number of channels, providing richer feature representation for subsequent domain conversion tasks and alleviating the feature correspondence problem caused by the large difference in the number of channels.

[0054] In an exemplary embodiment, the multi-scale channel comprehensive processing module specifically includes: three processing branches; the input of each branch is the same spectrum information with 128 channels, and the output is the result of splicing the feature information processed by the three branches in the channel dimension; wherein, the first branch includes a normalization layer and an activation function layer connected in sequence, and the output obtained by the first branch is the global information of the spectrum information; the second branch includes a random selection layer, a 1×1 convolution layer, a normalization layer and an activation function layer connected in sequence, and the second branch extracts features by randomly selecting the spectrum information of half of the channels, thereby improving the robustness and stability of the network in extracting spectrum information features; the third branch includes a channel attention layer, a normalization layer and an activation function layer connected in sequence, wherein the channel attention layer includes a global pooling layer and a weight calculation layer, and the third branch extracts key information in the spectrum information through channel attention.

[0055] Further, such as Figure 4 As shown in the figure, first, a convolution kernel with a kernel size of 3 and a padding of 1 is used to perform convolution operations on all input channels, and the output feature map saves the information of all channels. Then, for multi-channel data, half of the channels are randomly selected as the input channels, and the convolution kernel of the same size is used to obtain the information of the selected channels. This process randomly selects half of the channels, so that the model has a certain degree of randomness when processing input data, which helps the generalization ability of the model.

[0056] Subsequently, the real spectrum data is globally pooled and the weight information of each channel is calculated to allow the network to learn the weighted feature information to obtain feature information that enhances the response of important channels and suppresses the response of unimportant channels.

[0057] Finally, the feature information processed by full channel convolution, randomly selected half of the channel convolution and channel attention mechanism is concatenated, and the convolution kernel with kernel size of 1 is used to further extract feature information and adjust the number of channels. The generator integrated into this module extracts features from different scales of the original single data, showing the flexibility of processing data with many channels, thus improving the efficiency and performance of the model.

[0058] In an exemplary embodiment, Figure 5 As shown, the game form between the generator network model and the discriminator in the S3GAN network model is:

[0059]

[0060] in, is the maximum-minimum game between the generator network model and the discriminator; G is the generator network model, and the generator network model is the first generator network model or the second generator network model; D is the discriminator, i.e., the discriminator; is the loss of real data. The goal of this item is to maximize D(v) so that the output of the discriminator for real data is close to 1, so that it is classified as real data; E is the expectation; v~P r (v) is the real data v and the data distribution P of the real data r (v); D(v) is the discriminant output of the discriminator on the real data, which indicates the probability that the discriminator believes that the input data is real; is the loss of pseudo data; z~P g (z) is the pseudo data z and the probability distribution P of the pseudo data z g (z); D(G(z)) represents the discriminant output of the discriminator on the fake data G(z) generated by the generator network model, which represents the probability that the discriminator believes that the input data is real. The goal of this item is to minimize D(G(z)) so that the output of the discriminator on the generated data is close to 0, so that it is classified as fake data. The goal of the generator is to deceive the discriminator and generate realistic target domain images, while the goal of the discriminator is to distinguish between the fake images generated by the generator and the real target domain images as accurately as possible. This competition prompts the generator to learn better image conversion mapping.

[0061] The total loss function of the S3GAN network from amplitude to spectrum domain conversion task is expressed as:

[0062]

[0063] Among them, α, β and γ represent and The weight of represents adversarial loss, represents the auxiliary classification loss, represents the cycle loss, D represents the discriminator, G represents the generator, Represents the Class Activation Map Loss (CAMLoss) function used in the spectral information discriminator.

[0064] Adversarial loss is used to reduce the feature distribution difference between pseudo targets and real target samples:

[0065]

[0066] Among them, x~X s Represents the real spectrum domain sample, x~X a represents the real amplitude domain sample, G a→s (x) represents the generator network model in the adversarial game formation process of the amplitude domain to spectrum domain conversion task, D s represents the discriminator between real SAR spectrum information and pseudo SAR spectrum information, X a and X s represents the ship amplitude image and ship spectrum information from the source domain SAR ship image dataset and the target domain SAR ship image dataset, respectively. represents the expected value of data sampled from the true spectral data distribution, represents the expected value of the data sampled from the true amplitude data distribution, D s (x) represents the discriminant output of the real spectrum information x, D s (G a→s (x)) represents the discriminant output of the pseudo-spectrum information generated by the discriminator on the amplitude information.

[0067] The function is defined as:

[0068]

[0069] Among them, G s→a represents the generator network model in the process of forming the adversarial game from spectrum domain to amplitude domain task, G s→a (G a→s (x)) represents the pseudo spectrum information generated by the real amplitude information through G s→aThe pseudo amplitude information generated by the generator, |·|1 represents the L1 norm, that is, the absolute value error, which measures the conversion loss from the amplitude domain to the spectrum domain and then back to the amplitude domain. By minimizing this loss, it ensures that the data remains intact when it goes from the amplitude domain to the spectrum domain and then back to the amplitude domain.

[0070] It is mainly used to generate heat maps in image classification tasks to visualize the image areas that convolutional neural networks focus on when making decisions.

[0071]

[0072] Among them, η s (x) represents the auxiliary classifier in the generator.

[0073] In an exemplary embodiment, Figure 6 As shown, the feature fusion recognition network specifically includes: two processing branches, a feature fusion module and a target recognition module.

[0074] Among them, Figure 7 As shown, the first processing branch includes a spectral processing attention module and a first feature extraction module; the first processing branch is used to process the pseudo-SAR spectrum information; the spectral attention module includes a global feature extraction module and a channel attention feature extraction module; the input of the global feature extraction module and the channel attention feature extraction module is the same pseudo-SAR spectrum information, and the outputs of the global feature extraction module and the channel attention feature extraction module are spliced ​​in the channel dimension to obtain the output of the spectral processing attention module; the global feature extraction module includes a 7×7 convolution layer, a batch normalization layer, an activation function layer, a pooling layer and a 1×1 convolution layer connected in sequence; the channel attention feature extraction module includes a global pooling layer and a weight acquisition layer, a 1×1 convolution layer, a spectral normalization layer, a batch normalization layer, an activation function layer and a maximum pooling layer and a 1×1 convolution layer connected in sequence.

[0075] The second processing branch includes a second feature processing module; the second processing branch is used to process the amplitude ship image to be converted.

[0076] The feature fusion module is used to fuse the pseudo-SAR spectrum information processed by the first processing branch and the amplitude ship image processed by the second processing branch to obtain a fused image.

[0077] The target recognition module is used to recognize the SAR ship target in the fused image.

[0078] Furthermore, firstly, the present application performs global average pooling on the input spectrum data and obtains the weights of all channels, which is designed to allow the network to continuously update the weights to learn the differences between the importance of different channels, and on this basis, to focus on the channels that are beneficial to the classification effect among a large number of channels.

[0079] Then, this application adds a layer of spectral feature convolution layer group, which is the ordinary two-dimensional convolution, spectral normalization layer, batch normalization layer, and ReLU that comes with pytorch. Spectral normalization constrains the spectral norm of the weight matrix, which helps to reduce the gradient explosion and gradient vanishing problems during model training and improves the stability and generalization ability of the model in the process of processing complex spectral data.

[0080] Subsequently, this application adjusts the spatial size and number of channels of the feature information processed above through a maximum pooling layer and a convolution kernel with a kernel size of 1, extracts features from the original input data through a convolution layer with kernel_size = 7, stride = 2, and padding = 3, and then performs batch normalization and ReLU activation function activation, and then adjusts the spatial size and number of channels, and finally fuses and obtains the output.

[0081] Common target recognition networks are divided into feature extraction and classification output. The feature fusion recognition network of this application extracts features from amplitude information and spectrum information respectively, then concatenates them in the channel dimension, and then performs classification output through a fully connected layer.

[0082] This application uses the pseudo SAR spectrum information generated by the S3GAN network and the amplitude information to perform feature fusion network target recognition, and realizes the reasonable feature fusion of two data of different modalities, and uses the spectral processing attention module to extract the effective features of complex spectrum information, which enhances the training stability and generalization ability of the model, and improves the learning ability of complex spectrum information, and finally achieves better target recognition effect than using simple amplitude map for target recognition.

[0083] In order to illustrate the effectiveness of the generative adversarial network S3GAN network proposed in this application, the real amplitude map and the heat map of the key features in the S3GAN training process and the effect map of the generated spectrum information are shown, such as Figure 8 shown.

[0084] In order to verify the rationality and effectiveness of the ship target recognition method in solving the problems of scarce effective labeled samples and lack of spectrum information in the deep learning network model of SAR ship target recognition, this application tests the ability of pseudo-spectral information fusion amplitude image to improve the recognition accuracy of popular classification network.

[0085] The target recognition accuracy of the feature fusion recognition network after feature fusion achieved an average recognition accuracy improvement of 0.12 under the conditions of three popular classification networks, three types of ships, and 10 samples in each type, which proves the rationality, effectiveness and application value of this application in solving the scarcity of effective labeled samples and lack of spectral information in the deep learning network model for automatic target recognition of SAR ships.

[0086] This application proposes an S3GAN that can generate pseudo-spectral information based on SAR ship amplitude data, and based on the S3GAN, a multi-channel feature fusion automatic recognition framework that supports the fusion of amplitude data and pseudo-spectral data features is proposed. In order to solve the problems of lack of training focus of the generative adversarial network caused by the large number of spectral information data channels and difficulty in feature mapping caused by the large difference in the number of channels from amplitude information to spectral information, a multi-scale channel comprehensive processing module and a frequency domain transformation expansion module are designed; at the same time, in order to realize the optimal processing of spectral information by the feature fusion recognition network, a spectral processing attention module is designed to realize the feature extraction processing of important channels at the spectral level; finally, according to the preset training parameters and loss function, the S3GAN network is trained, and the trained S3GAN network is used to generate pseudo-spectral information and fuse the amplitude. Figure 1 The pseudo-spectrum information generated by this application and the improved fusion feature network are extremely reasonable and valuable in solving the problems of scarcity of effective labeled samples and lack of spectrum information in the deep learning network model of SAR ship automatic target recognition.

[0087] Based on the same inventive concept, the embodiment of the present application also provides a ship target recognition device for implementing the ship target recognition method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more ship target recognition device embodiments provided below can refer to the limitations of the ship target recognition method above, and will not be repeated here.

[0088] In an exemplary embodiment, Figure 5 As shown, a ship target identification device is provided, comprising:

[0089] The S3GAN network model training module is used to train the S3GAN network model according to the amplitude ship images in the source domain SAR ship image data set and the SAR ship phase data in the target domain SAR ship image data set; the S3GAN network model includes a first generator network model for performing the amplitude domain to spectrum domain conversion task, a second generator network model for performing the spectrum domain to amplitude domain conversion task and a discriminator; wherein, the first generator network model is constructed by adding a spectrum transformation expansion module before the feature encoder of the generative adversarial network U-GAT-IT; the second network model is constructed by adding a multi-scale channel comprehensive processing module before the feature encoder of the generative adversarial network U-GAT-IT; the frequency domain transformation expansion module is used to convert a single-channel or three-channel amplitude ship image into a 128-channel amplitude ship image with rich deep features; the multi-scale channel comprehensive processing module is used to adjust the output image to the same scale as the input image to extract the depth information of the input image at the frequency domain level.

[0090] The pseudo SAR spectrum information generation module is used to input the amplitude ship image to be converted into the trained S3GAN network model to generate the pseudo SAR spectrum information of the amplitude ship image to be converted.

[0091] The SAR ship target recognition module is used to input the pseudo-SAR spectrum information and the amplitude ship image in the source domain SAR ship image data set into the feature fusion recognition network for feature fusion, and recognize the SAR ship target in the fused image; the feature fusion recognition network is trained based on the amplitude ship image in the source domain SAR ship image data set and the SAR ship phase data in the target domain SAR ship image data set.

[0092] In an exemplary embodiment, a computer device is provided, which may be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O for short) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store ship target identification data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a ship target identification method is implemented.

[0093] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0094] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0095] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0096] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0097] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ReadOnlyMemory, ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (Magnetoresistive RandomAccess Memory, MRAM), ferroelectric random access memory (Ferroelectric RandomAccess Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (RandomAccess Memory, RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0098] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0099] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0100] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A ship target recognition method, characterized in that: The ship target recognition method comprises: The S3GAN network model is trained according to the amplitude ship images in the source domain SAR ship image data set and the SAR ship phase data in the target domain SAR ship image data set; the S3GAN network model includes a first generator network model for performing an amplitude domain to spectrum domain conversion task, a second generator network model for performing a spectrum domain to amplitude domain conversion task and a discriminator; wherein the first generator network model is constructed by adding a spectrum transformation expansion module before the feature encoder of the generative adversarial network U-GAT-IT; the second network model is constructed by adding a multi-scale channel comprehensive processing module before the feature encoder of the generative adversarial network U-GAT-IT; the frequency domain transformation expansion module is used to convert a single-channel or three-channel amplitude ship image into a 128-channel amplitude ship image rich in deep features; the multi-scale channel comprehensive processing module is used to adjust the output image to the same scale as the input image to extract the depth information of the input image at the frequency domain level; Inputting the amplitude ship image to be converted into the trained S3GAN network model to generate pseudo SAR spectrum information of the amplitude ship image to be converted; The pseudo-SAR spectrum information and the amplitude ship image in the source domain SAR ship image data set are input into a feature fusion recognition network for feature fusion to identify the SAR ship target in the fused image; the feature fusion recognition network is trained based on the amplitude ship image in the source domain SAR ship image data set and the SAR ship phase data in the target domain SAR ship image data set.

2. The ship target recognition method according to claim 1, characterized in that: The S3GAN network model is trained based on the amplitude ship images in the source domain SAR ship image dataset and the SAR ship phase data in the target domain SAR ship image dataset, which also includes: Converting the amplitude ship images to the same size using a linear interpolation method; The SAR ship phase data is subjected to joint time domain analysis and processing, and the spatial information of the data after the time domain analysis and processing is discarded to determine the frequency spectrum information with 128 channels of the target domain SAR ship image.

3. The ship target recognition method according to claim 1, characterized in that: The frequency domain transformation expansion module specifically includes: a 7×7 convolution layer, a normalization layer, an activation function layer, a frequency domain processing layer and a fusion processing layer connected in sequence; wherein the frequency domain processing layer includes a Fourier transform layer, a tensor shape expansion layer, an attention mechanism layer and an inverse Fourier transform layer connected in sequence; The single-channel or three-channel amplitude ship image enters the frequency domain processing layer after being segmented at different scales through the 7×7 convolution layer, the normalization layer and the activation function layer; the fusion processing layer is used to perform feature fusion on the channel dimension of the amplitude ship images of different scales that have passed through the frequency domain processing layer to obtain the output amplitude ship image of the frequency domain exchange expansion module, and the output amplitude ship image is the input of the feature encoder in the first generator network model.

4. The ship target recognition method according to claim 1, characterized in that: The multi-scale channel comprehensive processing module specifically includes: three processing branches; the input of each branch is the same spectrum information with 128 channels, and the output is the result of splicing the feature information processed by the three branches in the channel dimension; wherein, the first branch includes a normalization layer and an activation function layer connected in sequence, and the output obtained by the first branch is the global information of the spectrum information; the second branch includes a random selection layer, a 1×1 convolution layer, a normalization layer and an activation function layer connected in sequence, and the second branch extracts features by randomly selecting the spectrum information of half of the channels; the third branch includes a channel attention layer, a normalization layer and an activation function layer connected in sequence, wherein the channel attention layer includes a global pooling layer and a weight calculation layer, and the third branch extracts key information from the spectrum information through channel attention.

5. The ship target recognition method according to claim 1, characterized in that: The game form between the generator network model and the discriminator in the S3GAN network model is: in, is the maximum-minimum game between the generator network model and the discriminator; G is the generator network model, and the generator network model is the first generator network model or the second generator network model; D is the discriminator; is the loss of real data; E is the expectation; v~P r (v) is the real data v and the data distribution P of the real data r (v); D(v) is the discriminant output of the discriminator for the real data; is the loss of pseudo data; z~P g (z) is the pseudo data z and the probability distribution P of the pseudo data z g (z); D(G(z)) represents the discriminant output of the discriminator on the fake data G(z) generated by the generator network model.

6. The ship target recognition method according to claim 1, characterized in that: The feature fusion recognition network specifically includes: two processing branches, a feature fusion module and a target recognition module; Among them, the first processing branch includes a spectral processing attention module and a first feature extraction module; the first processing branch is used to process the pseudo-SAR spectrum information; the spectral attention module includes a global feature extraction module and a channel attention feature extraction module; the input of the global feature extraction module and the channel attention feature extraction module is the same pseudo-SAR spectrum information, and the output of the global feature extraction module and the channel attention feature extraction module are spliced ​​in the channel dimension to obtain the output of the spectral processing attention module; the global feature extraction module includes a 7×7 convolution layer, a batch normalization layer, an activation function layer, a pooling layer and a 1×1 convolution layer connected in sequence; the channel attention feature extraction module includes a global pooling layer and a weight acquisition layer, a 1×1 convolution layer, a spectral normalization layer, a batch normalization layer, an activation function layer and a maximum pooling layer and a 1×1 convolution layer connected in sequence; The second processing branch includes a second feature processing module; the second processing branch is used to process the amplitude ship image to be converted; The feature fusion module is used to fuse the pseudo SAR spectrum information processed by the first processing branch and the amplitude ship image processed by the second processing branch to obtain a fused image; The target recognition module is used to recognize the SAR ship target in the fused image.

7. A ship target identification device, characterized in that: The ship target identification device comprises: The S3GAN network model training module is used to train the S3GAN network model according to the amplitude ship image in the source domain SAR ship image data set and the SAR ship phase data in the target domain SAR ship image data set; the S3GAN network model includes a first generator network model for performing the amplitude domain to spectrum domain conversion task, a second generator network model for performing the spectrum domain to amplitude domain conversion task and a discriminator; wherein the first generator network model is constructed by adding a spectrum transformation expansion module before the feature encoder of the generative adversarial network U-GAT-IT; the second network model is constructed by adding a multi-scale channel comprehensive processing module before the feature encoder of the generative adversarial network U-GAT-IT; the frequency domain transformation expansion module is used to convert a single-channel or three-channel amplitude ship image into a 128-channel amplitude ship image rich in deep features; the multi-scale channel comprehensive processing module is used to adjust the output image to the same scale as the input image to extract the depth information of the input image at the frequency domain level; A pseudo SAR spectrum information generation module is used to input the amplitude ship image to be converted into the trained S3GAN network model to generate pseudo SAR spectrum information of the amplitude ship image to be converted; The SAR ship target recognition module is used to input the pseudo-SAR spectrum information and the amplitude ship image in the source domain SAR ship image data set into the feature fusion recognition network for feature fusion, and recognize the SAR ship target in the fused image; the feature fusion recognition network is trained based on the amplitude ship image in the source domain SAR ship image data set and the SAR ship phase data in the target domain SAR ship image data set.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the ship target recognition method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the ship target recognition method described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the ship target recognition method described in any one of claims 1 to 6 is implemented.

Images