Synthetic aperture radar image generation method and system based on multi-attribute cooperative processing

Abstract

The invention discloses a synthetic aperture radar image generation method and system based on multi-attribute cooperative processing, and belongs to the technical field of computers, and the method comprises the steps: obtaining a plurality of image attribute values needed for generating an image; performing embedding processing on each image attribute value through a multi-attribute cooperative processing module to obtain a first high-dimensional feature vector; wherein the multi-attribute co-processing module is used for co-processing and jointly controlling each image attribute value so as to guide image generation; performing feature splicing processing on the first high-dimensional feature vector to obtain a second high-dimensional feature vector, inputting the second high-dimensional feature vector into a pre-trained target generation module, and obtaining a target synthetic aperture radar image corresponding to the image attribute value; wherein the target generation module takes a diffusion model as a trunk structure, and comprises a plurality of Transform blocks. According to the method and the device, attribute joint controllable generation of the synthetic aperture radar image can be realized, and the synthetic radar aperture image with high quality, diversity and high accuracy is stably generated.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and system for generating synthetic aperture radar images based on multi-attribute collaborative processing. Background Art

[0002] Synthetic Aperture Radar (SAR) is an advanced active remote sensing technology that generates high-resolution two-dimensional images of the earth by transmitting and receiving microwave signals and using the movement of flying platforms (such as satellites or aircraft). It has the ability to work all day and all weather, has strong penetration ability, and can operate normally in any lighting scene or climate conditions. Therefore, this technology plays an important role in many fields.

[0003] Early SAR image generation technology generated SAR images through electromagnetic simulation. Although the traditional method can achieve the purpose of expanding the data set, it usually has problems such as poor image quality and high consumption of computing resources. With the development of technology, a series of derivatives and improvements based on generative adversarial networks have alleviated the above problems, but it still has problems such as unstable network structure and easy collapse.

[0004] In general, the related technology has the following disadvantages:

[0005] (1) The quality and diversity of the generated images are not good enough. There are deficiencies in processing details such as the complex structure and texture of the target. In addition, the generalization ability of the model is insufficient and it cannot cope with changing environments and target features. It is not realistic and comprehensive enough when simulating complex environments.

[0006] (2) It is impossible to achieve controllable generation of SAR image attributes, which lacks accuracy and consistency, resulting in unreliability of the generated results in terms of vision and application, and even has an adverse impact on the quality of image generation.

[0007] (3) Most of the existing SAR image conditional generation technologies can only achieve controllable generation of target categories, but still have shortcomings in the controllable generation of other key attributes (such as azimuth, elevation and complex background). In addition, it is often difficult to coordinate and control the relationship between various attributes, and it is impossible to achieve stable generation of SAR images with multi-attribute joint controllability. Summary of the invention

[0008] To solve at least one of the above problems, an embodiment of the present application proposes a synthetic aperture radar image generation method and system based on multi-attribute collaborative processing, aiming to stably generate high-quality synthetic aperture radar images.

[0009] To achieve the above-mentioned purpose, an embodiment of the present application provides a synthetic aperture radar image generation method based on multi-attribute collaborative processing, the method comprising:

[0010] Obtain several image attribute values ​​required to generate an image;

[0011] Embedding each of the image attribute values ​​through a multi-attribute collaborative processing module to obtain a first high-dimensional feature vector; wherein the multi-attribute collaborative processing module is used to collaboratively process and jointly control each of the image attribute values ​​to guide image generation;

[0012] Performing feature concatenation processing on the first high-dimensional feature vector to obtain a second high-dimensional feature vector;

[0013] The second high-dimensional feature vector is input into a pre-trained target generation module to obtain a target synthetic aperture radar image corresponding to the image attribute value; wherein the target generation module uses a diffusion model as a main structure and includes a plurality of Transformer blocks.

[0014] In some embodiments, the obtaining of several image attribute values ​​required for generating an image includes:

[0015] Obtain the target category, pitch angle, azimuth angle and background required to generate the image, and obtain the image attribute value.

[0016] In some embodiments, the embedding process of each of the image attribute values ​​by the multi-attribute collaborative processing module to obtain the first high-dimensional feature vector includes the following steps:

[0017] Through the target category neural network embedding function, the target category is embedded to obtain a high-dimensional feature vector of the target category;

[0018] The pitch angle is embedded through the pitch angle neural network embedding function to obtain the high-dimensional feature vector of the pitch angle;

[0019] Through the background neural network embedding function, the background is embedded to obtain the background high-dimensional feature vector;

[0020] Through sine-cosine encoding and linear fully connected layers, the azimuth is mapped to obtain a high-dimensional feature vector of the azimuth;

[0021] The target category high-dimensional feature vector, the pitch angle high-dimensional feature vector, the background high-dimensional feature vector and the azimuth angle high-dimensional feature vector are used as the first high-dimensional feature vector.

[0022] In some embodiments, inputting the second high-dimensional feature vector into a pre-trained target generation module to obtain a target synthetic aperture radar image corresponding to the image attribute value comprises the following steps:

[0023] Performing feature extraction based on long skip connections on the second high-dimensional feature vector through a plurality of Transformer blocks to obtain global context information;

[0024] Combined with the global context information, the standard Gaussian noise is gradually eliminated through the backbone structure of the diffusion model to obtain a target synthetic aperture radar image.

[0025] In some embodiments, the method further comprises the step of training a target generation module, specifically comprising the following steps:

[0026] Acquire a training image and a training image attribute value of the training image; wherein the training image is a synthetic aperture radar image;

[0027] Performing random time step noise processing on the training image to obtain a noisy image;

[0028] Performing a first processing on the noise image and the time step of the noise image to obtain an image feature vector having the time step;

[0029] Embedding the attribute values ​​of the training image to obtain a high-dimensional feature vector of the attribute;

[0030] Concatenate the attribute high-dimensional feature vector and the image feature vector as an input vector;

[0031] Inputting the input vector into a first generation model, performing noise prediction on the noise added to the noise image, and obtaining a noise prediction result;

[0032] The loss between the noise prediction result and the actual noise is evaluated through a loss function, and the model parameters of the first generation model are updated according to the gradient of the loss function until the end condition is met, thereby obtaining a target generation module.

[0033] In some embodiments, the first processing of the noise image and the time step of the noise image to obtain an image feature vector having a time step comprises the following steps:

[0034] Dividing the noisy image into sub-blocks of fixed size;

[0035] Using a convolutional neural network to convert the sub-blocks into a sequence of image vectors of fixed dimension;

[0036] Encoding the position information of the sub-block to obtain a learnable vector;

[0037] Combining the image vector sequence with the learnable vector to obtain an intermediate vector;

[0038] A time step mark is added to the intermediate vector to obtain a first vector feature.

[0039] In some embodiments, the loss function is expressed as:

[0040]

[0041] in, represents the loss function; x0 represents the input image; x t represents the noise image; t represents the time step information; c cat represents the target category attribute condition; c az Indicates the azimuth attribute condition; c pit Indicates the pitch angle attribute condition; c bg Represents background attribute conditions; ∈ θ Represents the noise prediction result; ∈ t Indicates the actual addition of noise.

[0042] To achieve the above purpose, another aspect of the embodiment of the present application proposes a synthetic aperture radar image generation system based on multi-attribute collaborative processing, the system comprising:

[0043] The first module is used to obtain a number of image attribute values ​​required to generate an image;

[0044] A second module is used to embed each of the image attribute values ​​through a multi-attribute collaborative processing module to obtain a first high-dimensional feature vector; wherein the multi-attribute collaborative processing module is used to collaboratively process and jointly control each of the image attribute values ​​to guide image generation;

[0045] A third module is used to perform feature concatenation processing on the first high-dimensional feature vector to obtain a second high-dimensional feature vector;

[0046] The fourth module is used to input the second high-dimensional feature vector into a pre-trained target generation module to obtain a target synthetic aperture radar image corresponding to the image attribute value; wherein the target generation module uses a diffusion model as a main structure and includes a plurality of Transformer blocks.

[0047] It should be noted that the system may also include the following modules:

[0048] The fifth module is used to train the target generation module.

[0049] To achieve the above objective, another aspect of an embodiment of the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above-mentioned method when executing the computer program.

[0050] To achieve the above objective, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0051] The embodiments of the present application include at least the following beneficial effects: The present application provides a method and system for generating a synthetic aperture radar image based on multi-attribute collaborative processing. The scheme obtains image attribute values ​​and embeds the image attribute values ​​to obtain a first high-dimensional feature vector, then concatenates all the first high-dimensional feature vectors to obtain a second high-dimensional feature vector, and then inputs the second high-dimensional feature vector into a pre-trained target generation module to obtain the overall steps of a synthetic radar aperture image. The attributes of the synthetic aperture radar image can be jointly and controllably generated, and while the synthetic radar aperture image is stably generated, the generated image has the characteristics of high quality, diversity and high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings are used to provide further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0053] Figure 1 is a flow chart of a synthetic aperture radar image generation method based on multi-attribute collaborative processing provided in an embodiment of the present application;

[0054] Figure 2 It is an overall architecture diagram of a synthetic aperture radar image generation model based on multi-attribute collaborative processing provided in an embodiment of the present application;

[0055] Figure 3 It is a schematic diagram of a process of applying the diffusion model provided in an embodiment of the present application to SAR image generation;

[0056] Figure 4 : is a module structure diagram of a multi-attribute collaborative processing module for SAR images provided in an embodiment of the present application;

[0057] Figure 5 It is a multi-attribute joint controllable generation result diagram of the SAR image provided by the embodiment of the present application;

[0058] Figure 6 This is a multi-attribute generated image result diagram of the ten major target categories of MSTAR provided in an embodiment of the present application;

[0059] Figure 7 It is a module schematic diagram of a synthetic aperture radar image generation system based on multi-attribute collaborative processing provided in an embodiment of the present application;

[0060] Figure 8 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0062] Although the functional modules are divided in the system schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first / S100", "second / S200", etc. in the specification, claims and the above drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0063] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0064] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0065] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0066] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0067] In view of the shortcomings and improvement needs of SAR image generation technology in the related art, the embodiment of the present application proposes a synthetic aperture radar image generation method based on multi-attribute collaborative processing, which aims to significantly improve the quality and diversity of generated SAR images while achieving stable generation of multi-attribute joint controllability. Specifically, the invention is committed to accurately controlling attributes such as azimuth, pitch angle, and background while controlling the target category to generate more realistic, detailed, accurate, and consistent images. In addition, by coordinating the relationship between multiple attributes, this method can also meet the needs of various complex application scenarios, thereby improving its effectiveness and reliability in applications such as remote sensing, environmental monitoring, and disaster assessment.

[0068] The synthetic aperture radar image generation method based on multi-attribute collaborative processing provided in the embodiment of the present application relates to the field of computer technology. The synthetic aperture radar image generation method based on multi-attribute collaborative processing provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements a synthetic aperture radar image generation method based on multi-attribute collaborative processing, etc., but is not limited to the above forms.

[0069] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0070] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0071] Figure 1 is an optional flowchart of a synthetic aperture radar image generation method based on multi-attribute collaborative processing provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S100 to S400.

[0072] Step S100, obtaining a number of image attribute values ​​required for generating an image.

[0073] Step S200, embedding each of the image attribute values ​​through a multi-attribute collaborative processing module to obtain a first high-dimensional feature vector; wherein the multi-attribute collaborative processing module is used to collaboratively process and jointly control each of the image attribute values ​​to guide image generation.

[0074] Step S300: performing feature concatenation processing on the first high-dimensional feature vector to obtain a second high-dimensional feature vector.

[0075] Step S400, inputting the second high-dimensional feature vector into a pre-trained target generation module to obtain a target synthetic aperture radar image corresponding to the image attribute value; wherein the target generation module uses a diffusion model as a main structure and includes a plurality of Transformer blocks.

[0076] Steps S100 to S400 shown in the embodiment of the present application obtain the first high-dimensional feature vector by acquiring the image attribute values ​​and embedding the image attribute values, then concatenate all the first high-dimensional feature vectors to obtain the second high-dimensional feature vector, and then input them into the pre-trained target generation module to obtain the overall steps of the synthetic radar aperture image. This can realize the joint and controllable generation of the attributes of the SAR image, and while stably generating the synthetic radar aperture image, the generated image has the characteristics of high quality, diversity and high accuracy.

[0077] In some embodiments, step S100 may include, but is not limited to, step S110:

[0078] Step S110, obtaining the target category, pitch angle, azimuth angle and background required for generating the image, and obtaining the image attribute value. It should be noted that, under the premise of being able to obtain a certain amount of data, many attributes of the SAR image can be taken into account, not limited to the four attributes of target category, pitch angle, azimuth angle and background.

[0079] In some embodiments, step S200 may include but is not limited to steps S210 to S250:

[0080] Step S210, embedding the target category through the target category neural network embedding function to obtain a high-dimensional feature vector of the target category.

[0081] Step S220, embedding the pitch angle through the pitch angle neural network embedding function to obtain a high-dimensional feature vector of the pitch angle.

[0082] Step S230, embedding the background through the background neural network embedding function to obtain a background high-dimensional feature vector.

[0083] Step S240, mapping the azimuth angle through sine-cosine coding and a linear fully connected layer to obtain a high-dimensional feature vector of the azimuth angle.

[0084] Step S250: taking the target category high-dimensional feature vector, the elevation angle high-dimensional feature vector, the background high-dimensional feature vector and the azimuth angle high-dimensional feature vector as the first high-dimensional feature vector.

[0085] In some embodiments, step S400 includes but is not limited to the following steps S410 to S420:

[0086] Step S410: performing feature extraction based on long skip connections on the second high-dimensional feature vector through a plurality of Transformer blocks to obtain global context information.

[0087] Step S420, combining the global context information, gradually eliminating noise from the standard Gaussian noise through the backbone structure of the diffusion model, and obtaining a target synthetic aperture radar image.

[0088] In some embodiments, the method may further include step S500 of training the target generation module, specifically including but not limited to the following steps S510 to S570:

[0089] Step S510, obtaining a training image and a training image attribute value of the training image; wherein the training image is a synthetic aperture radar image.

[0090] Step S520, performing random time step noise processing on the training image to obtain a noisy image.

[0091] Step S530: Perform a first process on the noise image and the time step of the noise image to obtain an image feature vector having a time step.

[0092] Step S540: embedding the attribute value of the training image to obtain a high-dimensional attribute feature vector.

[0093] Step S550: concatenate the attribute high-dimensional feature vector and the image feature vector to form an input vector.

[0094] Step S560: input the input vector into the first generation model, perform noise prediction on the noise added to the noise image, and obtain a noise prediction result.

[0095] Step S570, performing loss evaluation between the noise prediction result and the actual noise through a loss function, and updating the model parameters of the first generation model according to the gradient of the loss function until the termination condition is met, thereby obtaining a target generation module.

[0096] In some embodiments, step S530 includes but is not limited to the following steps S531 to S535:

[0097] Step S531, dividing the noise image into sub-blocks of fixed size.

[0098] Step S532: Use a convolutional neural network to convert the sub-blocks into a fixed-dimensional image vector sequence.

[0099] Step S533: Encode the position information of the sub-block to obtain a learnable vector.

[0100] Step S534, combining the image vector sequence and the learnable vector to obtain an intermediate vector.

[0101] Step S535: adding a time step mark to the intermediate vector to obtain a first vector feature.

[0102] In some embodiments, the loss function is expressed as:

[0103]

[0104] in, represents the loss function; x0 represents the input image; x t represents the noise image; t represents the time step information; c cat represents the target category attribute condition; c az Indicates the azimuth attribute condition; c pit Indicates the pitch angle attribute condition; c bg Represents background attribute conditions; ∈ θ Represents the noise prediction result; ∈ t Indicates the actual addition of noise.

[0105] The following describes the method of the embodiment of the present application in detail in combination with a specific radar image generation application scenario:

[0106] The embodiment of the present application mainly uses a diffusion model based on multi-attribute collaborative processing to generate SAR images, wherein the Transformer block is used for feature extraction. Under the premise of ensuring the generation quality, the attribute features of various SAR images (including target category, azimuth, pitch angle and background) are further finely processed and input into the network to jointly guide the image generation process, which can ultimately achieve multi-attribute jointly controllable synthetic aperture radar image generation.

[0107] The method of the embodiment of the present application can be implemented as a synthetic aperture radar image generation model based on multi-attribute collaborative processing. The overall structure of the model is as follows: Figure 2 As shown, it can include a multi-attribute collaborative processing module and a target generation module. In order to improve the quality and diversity of the generated image, the target generation module of the embodiment of the present application adopts a diffusion model (DiffusionModels) as the main structure of the generation process. The diffusion model mainly includes two processes: a forward diffusion process and a reverse denoising process, such as Figure 3As shown in the figure, the forward diffusion process is to add fixed noise to the image, so that the image is gradually converted into standard Gaussian noise, and the reverse denoising process is to gradually remove noise information from the standard Gaussian noise through a neural network, so that it gradually generates a high-quality image. This type of model has good scalability and parallelism, a very robust objective function, and a more stable training process than some methods. In addition, the generated samples have high quality and improved diversity.

[0108] The embodiment of the present application uses multiple Transformer Blocks in the main architecture, and adopts long skip connections between shallow and deep layers to effectively connect the features of shallow and deep layers, integrate more unknown information, and realize information transfer across multiple layers to better capture global context information, thereby improving the performance of the model without bringing too much computational burden to the model. Combining Transformer with the diffusion model for training and reasoning can better capture the long-distance dependencies between images through the advanced attention mechanism of Transformer, perform better when processing images with complex textures and shapes, and significantly improve the generation quality of SAR images. In addition, the Transformer model has higher parallelism and can complete model training more efficiently.

[0109] Furthermore, the embodiment of the present application designs a multi-attribute collaborative processing module (MACPM), whose internal structure is as follows: Figure 4 As shown, the multi-attribute collaborative processing module is used to collaboratively process and jointly control attributes such as target category, azimuth, pitch angle, and background during the entire generation process to achieve multi-attribute joint controllable generation.

[0110] The working steps of the multi-attribute collaborative processing module are as follows: First, the input noisy SAR image data must be preprocessed. The image is block-encoded (Patchify), and the image is divided into a set of fixed-size sub-block patches. The patches are converted into a vector sequence of a certain length with a hidden dimension D using a convolutional neural network. At the same time, position embedding is also required to encode the information of each position into a learnable vector and combine it with the image. At the same time, each time step in the input sequence must be embedded through the corresponding embedding layer E. tGenerate a corresponding time step token to capture the temporal characteristics of the sequence in the model. The image feature vector obtained in this way retains a lot of information of the original image, so it can provide effective image information for subsequent training.

[0111] Secondly, it is also necessary to extract the target category, azimuth, elevation angle, background and other attribute information and time steps from the image data, and perform corresponding embedding layer (Embedding Layer) and normalization processing on the extracted attributes to make them suitable for model training and generation process. It should be noted that many attributes of the SAR image that can be obtained can be taken into account, not limited to the four attributes of target category, elevation angle, azimuth and background.

[0112] Specifically, because the possible values ​​of the attributes such as target type, pitch angle and background are relatively few and discrete, a neural network embedding layer is used to map these discrete attributes to a high-dimensional feature space to facilitate model processing and learning. Furthermore, the expression for high-dimensional feature mapping of target category, pitch angle and background is:

[0113]

[0114] Among them, A category ,A pitch ,A background are the corresponding target categories, elevation angles and background attributes, E cat ,E pit ,E bg denote the neural network embedding functions for target category, pitch angle and background, respectively, v cat ,v pit ,v bg This corresponds to the high-dimensional feature vector obtained after embedding processing of the above attributes.

[0115] As for the azimuth attribute, since its possible values ​​are in the range of [0°, 360°) and the values ​​are continuous, we use sine-cosine encoding plus a linear fully connected layer to map it to a high-dimensional feature space. This not only represents the size of the azimuth angle, but also naturally handles the periodic characteristics of the angle. Based on this, the expression for mapping the azimuth angle is:

[0116] v az =E az ([sin(A azimuth ),cos(A azimuth )]),

[0117] Among them A azimuth is the azimuth attribute, E az represents the fully connected mapping layer of the neural network, vaz The corresponding azimuth is obtained after processing the high-dimensional feature vector.

[0118] In order to make the generated SAR image have high quality and accuracy of multi-attribute joint control, the embodiment of the present application designs an advanced loss function for joint optimization in training. The training process is: get the input image x0, the image diffusion algorithm will gradually add noise to the image, and generate a noise image x t At the same time, given a set of multi-attribute conditions, including time step information t, target category attribute condition c cat , azimuth attribute condition c az , pitch angle attribute condition c pit And background attribute condition c bg , the image diffusion algorithm needs to learn a network to predict the value added to the noise image x t The noise on θ , and make it consistent with the real noise ∈ t As close as possible. The overall loss function The mathematical formula can be expressed as:

[0119]

[0120] In summary, the training process of the synthetic aperture radar image generation model based on multi-attribute collaborative processing proposed in this embodiment may include:

[0121] 1. Data preparation: Collect SAR image datasets containing multiple attributes (target category, azimuth, elevation angle, background, etc.) and perform normalization preprocessing.

[0122] 2. Build model input:

[0123] (1) Noise the SAR image at the corresponding t time steps to obtain a noisy image;

[0124] (2) In the MACPM module, the noisy image and the corresponding time step are processed: the image is divided into a set of fixed-size sub-blocks (patches), and each patch is converted into a feature vector using a convolutional neural network. Finally, the feature vector sequence of the image is formed, and the position embedding information is added and the time step t is processed into the corresponding time step label;

[0125] (3) All attributes of the SAR image are processed collaboratively in the MACPM module: the corresponding attributes of the SAR image (including target category, azimuth, elevation angle, and background, etc.) are extracted respectively, and different embedding processes are performed on them respectively to obtain the corresponding high-dimensional feature vectors. They are then concatenated with the high-dimensional feature vector sequence and time step marker obtained after block encoding of the noisy SAR image, and used as the final model input.

[0126] 3. Model design:

[0127] (1) Introducing the Multi-Attribute Cooperative Processing Module (MACPM) to handle the joint control of various attributes;

[0128] (2) Design a generative architecture that includes the Transformer and the diffusion model. The Transformer is used to capture global features and dependencies, and uses long jump connections between shallow and deep layers to achieve feature information transfer, while the diffusion model is used to gradually eliminate noise and generate high-quality images.

[0129] (3) Construct the corresponding loss function and use the time step and multi-attribute information as joint conditions to guide image generation.

[0130] 4. Model training:

[0131] (1) After the noise image is input into the model, the model needs to predict the noise added to the image, that is, the final output is the predicted noise. The loss function is then calculated by combining the predicted noise with the actual noise added, and the model parameters are updated based on the gradient of the loss function through back propagation.

[0132] The training batch size is 64; the Adamw optimizer is used for training, the weight decay is 0.03, the parameters are set to beta1=0.99, beta2=0.999; the learning rate is set to 0.0002, and the warmup strategy is used in the initial stage of training, that is, the learning rate is increased linearly in the first 5000 steps to avoid the model converging too quickly in the early stage of training.

[0133] The reasoning process of the model in this embodiment is:

[0134] 1. Input the attribute values ​​of the image to be generated, including target category, azimuth, elevation angle and background;

[0135] 2. Perform different embedding processes on the input attribute values ​​through the MACPM module;

[0136] 3. The high-dimensional feature vectors obtained after processing each attribute are spliced ​​and input into the generation model (target generation module), and are subjected to feature extraction of multiple Transformer blocks in the overall model of this embodiment and denoising processing of the diffusion model to generate a high-quality SAR image;

[0137] The final generated SAR image will be able to fully reflect the various attributes of the input and maintain high quality and consistency of attribute joint generation.

[0138] The embodiments of the present application include at least the following beneficial effects:

[0139] 1. Through the joint action of attribute-based collaborative processing modules, Transformer blocks and diffusion models, the quality and diversity of generated SAR images are significantly improved. The attributes such as azimuth, elevation and background can be accurately controlled to generate more realistic, detailed, accurate and consistent images, and realize the stable generation of multi-attribute joint controllable, which significantly improves the robustness and generalization performance of the model, making it more stable in different application scenarios.

[0140] 2. The Transformer is combined with the diffusion model to make it suitable for SAR image generation tasks. The powerful self-attention mechanism of the Transformer can capture the global features in the image, while the diffusion model is good at processing noise and generating high-quality images. The two complement each other, which significantly improves the overall quality and diversity of the generated SAR images and can better preserve the complex structure and texture information of the original image.

[0141] 3. By coordinating the relationship between multiple attributes, the method of the embodiment of the present application can generate SAR images with strong all-attribute availability, so as to effectively alleviate the problem of insufficient measured sample data and meet the needs of various complex application scenarios, thereby improving its effectiveness and reliability in various fields such as remote sensing, environmental monitoring, and disaster assessment.

[0142] 4. A multi-attribute collaborative processing module (MACPM) and corresponding loss functions are designed to process multiple attributes of SAR images and integrate them into the training and inference process of the model. The corresponding embedding or encoding method is selected according to the characteristics of each attribute, so that it can independently and flexibly guide the generation process, so that each attribute can be accurately reflected in the generated image.

[0143] The following is an introduction to some beneficial effects of the embodiments of the present application in combination with experimental data:

[0144] In order to verify the good effect and reliability, this application conducted relevant experiments on a widely recognized measured SAR image dataset. The dataset used is the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset. This dataset contains ten categories of SAR targets. The azimuth angles of all image data are distributed in the range of [0°, 360°), and the pitch angles have several possible values ​​of 15°, 17°, 30° and 45°. Only some target categories contain SAR images with pitch angles of 30° and 45°. There are fewer data with different background attributes in the MSTAR dataset. Therefore, this experiment first clusters the entire dataset according to the energy level of the background area of ​​the SAR image. There are a total of 5 categories, and then the clustering results are used as the background attributes of the SAR data. The effect of the joint controllable generation of multiple attributes of SAR image target categories, azimuth angles, pitch angles and background is as follows. Figure 5 As shown, the generated results include 4 SAR target categories (BMP2, BRDM_2, T72, BTR70), 12 azimuth angles taken at equal intervals in [0°, 360°), 4 pitch angles (15°, 17°, 30°, 45°) and 3 background categories (Class I, Class III, Class V). It can be seen that the generated SAR image is of high quality and stable, and can show the texture changes of different target categories. The changes in azimuth and pitch angles can also be intuitively seen from the direction and shadow changes of the target. The overall energy from low to high also reflects the changes in the background. This fully demonstrates that the SAR image generated by the method proposed in the present invention can not only accurately reflect each attribute, but also maintain the high quality of the generated results and the reliability of joint generation when multiple attributes are specified at the same time.

[0145] It is worth mentioning that for the two types of targets, BMP2 and BTR70, shown in the generated result graph, there is no data with pitch angles of 30° and 45° in the training set, but after training, corresponding and convincing results can be generated. This fully proves that the model has strong generalization and reasoning capabilities. Even in the case of incomplete training data, the model can still generate corresponding high-quality results based on sufficient learning and reasoning of existing data.

[0146] Figure 6 The figure shows the generation of ten types of targets in the MSTAR dataset (each row is a type of target), and each column corresponds to a set of random multi-attribute combinations. As can be seen from the figure, even given the same multi-attribute conditions, the model can show the complex texture differences between different target categories learned, and give very accurate and reliable generation results.

[0147] See also Figure 7 The embodiment of the present application also provides a synthetic aperture radar image generation system based on multi-attribute collaborative processing, which can implement the above-mentioned synthetic aperture radar image generation method based on multi-attribute collaborative processing, and the system includes:

[0148] The first module 101 is used to obtain a number of image attribute values ​​required to generate an image;

[0149] The second module 102 is used to embed each of the image attribute values ​​through a multi-attribute collaborative processing module to obtain a first high-dimensional feature vector; wherein the multi-attribute collaborative processing module is used to collaboratively process and jointly control each of the image attribute values ​​to guide image generation;

[0150] The third module 103 is used to perform feature concatenation processing on the first high-dimensional feature vector to obtain a second high-dimensional feature vector;

[0151] The fourth module 104 is used to input the second high-dimensional feature vector into a pre-trained target generation module to obtain a target synthetic aperture radar image corresponding to the image attribute value; wherein the target generation module uses a diffusion model as a main structure and includes a plurality of Transformer blocks.

[0152] It should be noted that the system may also include the following modules:

[0153] The fifth module is used to train the target generation module.

[0154] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0155] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the synthetic aperture radar image generation method based on multi-attribute collaborative processing when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0156] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0157] See also Figure 8 , Figure 8 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0158] The processor 201 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0159] The memory 202 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 202 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 202, and the processor 201 calls and executes the synthetic aperture radar image generation method based on multi-attribute collaborative processing in the embodiment of this application;

[0160] Input / output interface 203, used to implement information input and output;

[0161] The communication interface 204 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0162] Bus 205 , which transmits information between various components of the device (e.g., processor 201 , memory 202 , input / output interface 203 , and communication interface 204 );

[0163] The processor 201 , the memory 202 , the input / output interface 203 and the communication interface 204 are connected to each other in communication within the device via the bus 205 .

[0164] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned synthetic aperture radar image generation method based on multi-attribute collaborative processing is implemented.

[0165] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0166] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0167] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0168] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0169] The system embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0170] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0171] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0172] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0173] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0174] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0175] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0176] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0177] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A synthetic aperture radar image generation method based on multi-attribute collaborative processing, characterized in that: The following steps are involved: Obtain several image attribute values ​​required to generate an image; Embedding each of the image attribute values ​​through a multi-attribute collaborative processing module to obtain a first high-dimensional feature vector; wherein the multi-attribute collaborative processing module is used to collaboratively process and jointly control each of the image attribute values ​​to guide image generation; Performing feature concatenation processing on the first high-dimensional feature vector to obtain a second high-dimensional feature vector; The second high-dimensional feature vector is input into a pre-trained target generation module to obtain a target synthetic aperture radar image corresponding to the image attribute value; wherein the target generation module uses a diffusion model as a main structure and includes a plurality of Transformer blocks.

2. The method according to claim 1, characterized in that The obtaining of several image attribute values ​​required for generating an image includes: Obtain the target category, pitch angle, azimuth angle and background required to generate the image, and obtain the image attribute value.

3. The method according to claim 1, characterized in that The embedding process of each of the image attribute values ​​by the multi-attribute collaborative processing module to obtain a first high-dimensional feature vector includes the following steps: Through the target category neural network embedding function, the target category is embedded to obtain a high-dimensional feature vector of the target category; The pitch angle is embedded through the pitch angle neural network embedding function to obtain the high-dimensional feature vector of the pitch angle; Through the background neural network embedding function, the background is embedded to obtain the background high-dimensional feature vector; Through sine-cosine encoding and linear fully connected layers, the azimuth is mapped to obtain a high-dimensional feature vector of the azimuth; The target category high-dimensional feature vector, the pitch angle high-dimensional feature vector, the background high-dimensional feature vector and the azimuth angle high-dimensional feature vector are used as the first high-dimensional feature vector.

4. The method according to claim 1, characterized in that: The step of inputting the second high-dimensional feature vector into a pre-trained target generation module to obtain a target synthetic aperture radar image corresponding to the image attribute value comprises the following steps: Performing feature extraction based on long skip connections on the second high-dimensional feature vector through a plurality of Transformer blocks to obtain global context information; Combined with the global context information, the standard Gaussian noise is gradually eliminated through the backbone structure of the diffusion model to obtain a target synthetic aperture radar image.

5. The method according to claim 1, characterized in that The method further comprises the step of training a target generation module, specifically comprising the following steps: Acquire a training image and a training image attribute value of the training image; wherein the training image is a synthetic aperture radar image; Performing random time step noise processing on the training image to obtain a noisy image; Performing a first processing on the noise image and the time step of the noise image to obtain an image feature vector having the time step; Embedding the attribute values ​​of the training image to obtain a high-dimensional feature vector of the attribute; Concatenate the attribute high-dimensional feature vector and the image feature vector as an input vector; Inputting the input vector into a first generation model, performing noise prediction on the noise added to the noise image, and obtaining a noise prediction result; The loss function is used to evaluate the loss between the noise prediction result and the actual added noise. According to the gradient of the loss function, the model parameters of the first generation model are updated until the end condition is met, thereby obtaining the target generation module.

6. The method according to claim 5, characterized in that The first processing of the noise image and the time step of the noise image to obtain an image feature vector having a time step comprises the following steps: Dividing the noisy image into sub-blocks of fixed size; Using a convolutional neural network to convert the sub-blocks into a sequence of image vectors of fixed dimension; Encoding the position information of the sub-block to obtain a learnable vector; Combining the image vector sequence with the learnable vector to obtain an intermediate vector; A time step mark is added to the intermediate vector to obtain a first vector feature.

7. The method according to claim 5, characterized in that The expression of the loss function is: in, represents the loss function; x0 represents the input image; x t represents the noise image; t represents the time step information; c cat represents the target category attribute condition; c az Indicates the azimuth attribute condition; c pit Indicates the pitch angle attribute condition; c bg Represents background attribute conditions; ∈ θ Represents the noise prediction result; ∈ t Indicates the actual addition of noise.

8. A synthetic aperture radar image generation system based on multi-attribute collaborative processing, characterized in that: include: The first module is used to obtain a number of image attribute values ​​required to generate an image; A second module is used to embed each of the image attribute values ​​through a multi-attribute collaborative processing module to obtain a first high-dimensional feature vector; wherein the multi-attribute collaborative processing module is used to collaboratively process and jointly control each of the image attribute values ​​to guide image generation; A third module is used to perform feature concatenation processing on the first high-dimensional feature vector to obtain a second high-dimensional feature vector; The fourth module is used to input the second high-dimensional feature vector into a pre-trained target generation module to obtain a target synthetic aperture radar image corresponding to the image attribute value; wherein the target generation module uses a diffusion model as a main structure and includes a plurality of Transformer blocks.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 7.

10. A computer storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to implement the method according to any one of claims 1 to 7 when executed by the processor.

Images