Synthetic aperture radar image generation method and system based on multi-attribute collaborative processing

Abstract

This application discloses a method and system for generating synthetic aperture radar images based on multi-attribute collaborative processing, belonging to the field of computer technology. The method includes: obtaining a number of image attribute values ​​required for image generation; embedding each image attribute value through a multi-attribute collaborative processing module to obtain a first high-dimensional feature vector; wherein the multi-attribute collaborative processing module is used to collaboratively process and jointly control each image attribute value to guide image generation; performing feature concatenation processing on the first high-dimensional feature vector to obtain a second high-dimensional feature vector, which is input into a pre-trained target generation module to obtain a target synthetic aperture radar image corresponding to the image attribute value; wherein the target generation module uses a diffusion model as the main structure and includes a number of Transformer blocks. This application can realize the joint and controllable generation of synthetic aperture radar image attributes, and stably generate high-quality, diverse, and highly accurate synthetic aperture radar images.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and system for generating synthetic aperture radar images based on multi-attribute collaborative processing. Background Art

[0002] Synthetic Aperture Radar (SAR) is an advanced active remote sensing technology that generates high-resolution two-dimensional images of the Earth by transmitting and receiving microwave signals and utilizing the movement of flying platforms (such as satellites or aircraft). It has the ability to work in all weather conditions and at all times, has strong penetrating power, and can operate normally in any lighting scene or climate conditions. Therefore, this technology plays an important role in many fields.

[0003] Early SAR image generation technologies used electromagnetic simulation to generate SAR images. While these traditional methods could expand datasets, they often suffered from poor image quality and high computing resource consumption. With the development of technology, a series of derivatives and improvements based on generative adversarial networks have alleviated these issues, but they still suffer from unstable network structures and prone to crashes.

[0004] In general, the related technologies have the following disadvantages:

[0005] (1) The quality and diversity of the generated images are not good enough. There are deficiencies in processing details such as the complex structure and texture of the target. In addition, the generalization ability of the model is insufficient and it cannot cope with the changing environment and target characteristics. It is not realistic and comprehensive enough when simulating complex environments.

[0006] (2) It is impossible to achieve well-controlled generation of SAR image attributes, which lacks accuracy and consistency, resulting in unreliable visual and application results, and even has an adverse impact on the quality of image generation.

[0007] (3) Most existing SAR image conditional generation technologies can only achieve controllable generation of target categories, but still have shortcomings in the controllable generation of other key attributes (such as azimuth, pitch angle and complex background). It is often difficult to coordinate and control the relationship between various attributes, and it is impossible to achieve stable generation of SAR images with multi-attribute joint controllability. Summary of the Invention

[0008] To solve at least one of the above problems, the embodiments of the present application propose a synthetic aperture radar image generation method and system based on multi-attribute collaborative processing, aiming to stably generate high-quality synthetic aperture radar images.

[0009] To achieve the above objectives, one aspect of an embodiment of the present application provides a method for generating a synthetic aperture radar image based on multi-attribute collaborative processing, the method comprising:

[0010] Obtain several image attribute values ​​required to generate an image;

[0011] Embedding each of the image attribute values ​​through a multi-attribute collaborative processing module to obtain a first high-dimensional feature vector; wherein the multi-attribute collaborative processing module is used to collaboratively process and jointly control each of the image attribute values ​​to guide image generation;

[0012] Performing feature concatenation processing on the first high-dimensional feature vector to obtain a second high-dimensional feature vector;

[0013] The second high-dimensional feature vector is input into a pre-trained target generation module to obtain a target synthetic aperture radar image corresponding to the image attribute value; wherein the target generation module has a diffusion model as a main structure and includes a plurality of Transformer blocks.

[0014] In some embodiments, obtaining a plurality of image attribute values ​​required for generating an image includes:

[0015] Obtain the target category, pitch angle, azimuth angle and background required to generate the image, and obtain the image attribute value.

[0016] In some embodiments, embedding each of the image attribute values ​​using a multi-attribute collaborative processing module to obtain a first high-dimensional feature vector includes the following steps:

[0017] Through the target category neural network embedding function, the target category is embedded to obtain the high-dimensional feature vector of the target category;

[0018] The pitch angle is embedded through the pitch angle neural network embedding function to obtain the high-dimensional feature vector of the pitch angle;

[0019] Through the background neural network embedding function, the background is embedded to obtain the background high-dimensional feature vector;

[0020] The azimuth is mapped through sine-cosine encoding and linear fully connected layers to obtain a high-dimensional feature vector of the azimuth.

[0021] The target category high-dimensional feature vector, the pitch angle high-dimensional feature vector, the background high-dimensional feature vector and the azimuth angle high-dimensional feature vector are used as the first high-dimensional feature vector.

[0022] In some embodiments, inputting the second high-dimensional feature vector into a pre-trained target generation module to obtain a target synthetic aperture radar image corresponding to the image attribute value includes the following steps:

[0023] Performing feature extraction based on long skip connections on the second high-dimensional feature vector through several Transformer blocks to obtain global context information;

[0024] Combined with the global context information, the standard Gaussian noise is gradually eliminated through the backbone structure of the diffusion model to obtain a target synthetic aperture radar image.

[0025] In some embodiments, the method further includes the step of training a target generation module, specifically including the following steps:

[0026] Acquire a training image and a training image attribute value of the training image; wherein the training image is a synthetic aperture radar image;

[0027] Performing random time step noise processing on the training image to obtain a noisy image;

[0028] Performing a first processing on the noise image and the time step of the noise image to obtain an image feature vector having the time step;

[0029] Embedding the attribute values ​​of the training image to obtain a high-dimensional feature vector of the attribute;

[0030] Concatenate the attribute high-dimensional feature vector and the image feature vector as an input vector;

[0031] Inputting the input vector into a first generation model, performing noise prediction on the noise added to the noise image, and obtaining a noise prediction result;

[0032] The loss between the noise prediction result and the actual noise is evaluated through a loss function, and the model parameters of the first generation model are updated according to the gradient of the loss function until the end condition is met to obtain a target generation module.

[0033] In some embodiments, performing a first processing on the noise image and the time step of the noise image to obtain an image feature vector having a time step includes the following steps:

[0034] dividing the noisy image into sub-blocks of fixed size;

[0035] Using a convolutional neural network to convert the sub-blocks into a sequence of image vectors of fixed dimension;

[0036] Encoding the position information of the sub-block to obtain a learnable vector;

[0037] Combining the image vector sequence and the learnable vector to obtain an intermediate vector;

[0038] A time step mark is added to the intermediate vector to obtain a first vector feature.

[0039] In some embodiments, the loss function is expressed as:

[0040]

[0041] in, represents the loss function; x0 represents the input image; x t represents the noise image; t represents the time step information; c cat represents the target category attribute condition; c az Indicates the azimuth attribute condition; c pit Indicates the pitch angle attribute condition; c bg Represents background attribute conditions; ∈ θ Represents the noise prediction result; ∈ t Indicates the actual addition of noise.

[0042] To achieve the above objectives, another aspect of the present application provides a synthetic aperture radar image generation system based on multi-attribute collaborative processing, the system comprising:

[0043] The first module is used to obtain several image attribute values ​​required to generate an image;

[0044] a second module configured to embed each of the image attribute values ​​using a multi-attribute collaborative processing module to obtain a first high-dimensional feature vector; wherein the multi-attribute collaborative processing module is configured to collaboratively process and jointly control each of the image attribute values ​​to guide image generation;

[0045] A third module is used to perform feature concatenation processing on the first high-dimensional feature vector to obtain a second high-dimensional feature vector;

[0046] A fourth module is configured to input the second high-dimensional feature vector into a pre-trained target generation module to obtain a target synthetic aperture radar image corresponding to the image attribute value; wherein the target generation module has a diffusion model as its main structure and includes a plurality of Transformer blocks.

[0047] It should be noted that the system may also include the following modules:

[0048] The fifth module is used to train the target generation module.

[0049] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned method when executing the computer program.

[0050] To achieve the above objectives, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0051] The embodiments of the present application include at least the following beneficial effects: The present application provides a method and system for generating a synthetic aperture radar image based on multi-attribute collaborative processing. The scheme obtains image attribute values ​​and embeds the image attribute values ​​to obtain a first high-dimensional feature vector, then concatenates all the first high-dimensional feature vectors to obtain a second high-dimensional feature vector, and then inputs the second high-dimensional feature vector into a pre-trained target generation module to obtain the overall steps of a synthetic radar aperture image. This can realize the joint and controllable generation of the attributes of the synthetic aperture radar image, and while stably generating the synthetic radar aperture image, the generated image has the characteristics of high quality, diversity and high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings are used to provide a further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0053] Figure 1 This is a flow chart of a synthetic aperture radar image generation method based on multi-attribute collaborative processing provided by an embodiment of the present application;

[0054] Figure 2 This is a diagram of the overall architecture of a synthetic aperture radar image generation model based on multi-attribute collaborative processing provided by an embodiment of the present application;

[0055] Figure 3 This is a flow chart of applying the diffusion model provided in the embodiment of the present application to SAR image generation;

[0056] Figure 4 1 is a module structure diagram of a multi-attribute collaborative processing module for SAR images provided in an embodiment of the present application;

[0057] Figure 5 This is a result diagram of the multi-attribute joint controllable generation of SAR images provided by an embodiment of the present application;

[0058] Figure 6 This is a multi-attribute generated image result diagram of the ten major target categories of MSTAR provided in the embodiment of the present application;

[0059] Figure 7 Schematic diagram of a module of a synthetic aperture radar image generation system based on multi-attribute collaborative processing provided by an embodiment of the present application;

[0060] Figure 8 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are merely examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0062] Although the system diagrams illustrate functional modules and the flowcharts illustrate a logical sequence, in some cases, the steps shown or described may be performed in a different order than the module divisions in the system or the order in the flowcharts. The terms "first / S100," "second / S200," and the like in the specification, claims, and drawings are used to distinguish similar items and are not necessarily used to describe a specific order or sequence.

[0063] It will be understood that the terms "first", "second", etc. used in this application may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0064] The terms "at least one", "plurality", "each", "any", etc. used in this application include "at least one", "two" or more, "plurality" or "each", "any" or "any one", "each" or "any one" as used herein.

[0065] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0066] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0067] In view of the shortcomings and improvement needs of SAR image generation technology in related technologies, the embodiments of the present application propose a synthetic aperture radar image generation method based on multi-attribute collaborative processing, which aims to significantly improve the quality and diversity of generated SAR images while achieving stable generation of multi-attribute joint controllability. Specifically, the invention is committed to precisely controlling attributes such as azimuth, pitch angle, and background while controlling the target category to generate more realistic, detailed, accurate, and consistent images. In addition, by coordinating the relationships between multiple attributes, this method can also meet the needs of various complex application scenarios, thereby improving its effectiveness and reliability in applications such as remote sensing, environmental monitoring, and disaster assessment.

[0068] The embodiments of the present application provide a method for generating synthetic aperture radar images based on multi-attribute collaborative processing, which relates to the field of computer technology. The method for generating synthetic aperture radar images based on multi-attribute collaborative processing provided in the embodiments of the present application can be applied to a terminal or a server, or can be software running on a terminal or server. In some embodiments, the terminal can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, etc., but is not limited thereto. The server can be configured as an independent physical server, or as a server cluster or distributed system consisting of multiple physical servers. It can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application that implements the method for generating synthetic aperture radar images based on multi-attribute collaborative processing, etc., but is not limited to the above forms.

[0069] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0070] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0071] Figure 1 This is an optional flowchart of the synthetic aperture radar image generation method based on multi-attribute collaborative processing provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S100 to S400.

[0072] Step S100: obtaining a number of image attribute values ​​required for generating an image.

[0073] Step S200, embedding processing is performed on each of the image attribute values ​​through a multi-attribute collaborative processing module to obtain a first high-dimensional feature vector; wherein the multi-attribute collaborative processing module is used to collaboratively process and jointly control each of the image attribute values ​​to guide image generation.

[0074] Step S300: performing feature concatenation processing on the first high-dimensional feature vector to obtain a second high-dimensional feature vector.

[0075] Step S400: Input the second high-dimensional feature vector into a pre-trained target generation module to obtain a target synthetic aperture radar image corresponding to the image attribute value; wherein the target generation module uses a diffusion model as a main structure and includes a plurality of Transformer blocks.

[0076] In steps S100 to S400 shown in the embodiment of the present application, a synthetic radar aperture image is obtained by obtaining image attribute values ​​and embedding the image attribute values ​​to obtain a first high-dimensional feature vector, and then all the first high-dimensional feature vectors are spliced ​​to obtain a second high-dimensional feature vector, which is then input into a pre-trained target generation module. This overall step can achieve the joint and controllable generation of SAR image attributes, and while stably generating the synthetic radar aperture image, the generated image has the characteristics of high quality, diversity, and high accuracy.

[0077] In some embodiments, step S100 may include, but is not limited to, step S110:

[0078] Step S110 : Obtain the target category, elevation angle, azimuth angle, and background required for image generation to obtain image attribute values. It should be noted that, provided a sufficient amount of data is available, a wide range of SAR image attributes can be considered, not limited to the target category, elevation angle, azimuth angle, and background attributes.

[0079] In some embodiments, step S200 may include but is not limited to steps S210 to S250:

[0080] Step S210 , embedding the target category through the target category neural network embedding function to obtain a high-dimensional feature vector of the target category.

[0081] Step S220 , embedding the pitch angle using a pitch angle neural network embedding function to obtain a high-dimensional feature vector of the pitch angle.

[0082] Step S230: embedding the background using a background neural network embedding function to obtain a background high-dimensional feature vector.

[0083] In step S240 , the azimuth is mapped through sine-cosine coding and a linear fully connected layer to obtain a high-dimensional feature vector of the azimuth.

[0084] Step S250 : taking the target category high-dimensional feature vector, the elevation angle high-dimensional feature vector, the background high-dimensional feature vector, and the azimuth angle high-dimensional feature vector as a first high-dimensional feature vector.

[0085] In some embodiments, step S400 includes but is not limited to the following steps S410 to S420:

[0086] Step S410 , performing feature extraction based on long skip connections on the second high-dimensional feature vector through a plurality of Transformer blocks to obtain global context information.

[0087] Step S420 , combining the global context information, and gradually eliminating noise from the standard Gaussian noise through the backbone structure of the diffusion model, to obtain the target synthetic aperture radar image.

[0088] In some embodiments, the method may further include step S500 of generating a training target module, specifically including but not limited to the following steps S510 to S570:

[0089] Step S510: Acquire a training image and a training image attribute value of the training image; wherein the training image is a synthetic aperture radar image.

[0090] Step S520 , performing random time step noise processing on the training image to obtain a noisy image.

[0091] Step S530 : performing a first process on the noise image and the time step of the noise image to obtain an image feature vector having the time step.

[0092] Step S540: embedding the attribute values ​​of the training image to obtain a high-dimensional attribute feature vector.

[0093] Step S550: Concatenate the attribute high-dimensional feature vector and the image feature vector to form an input vector.

[0094] Step S560: input the input vector into a first generation model, perform noise prediction on the noise added to the noise image, and obtain a noise prediction result.

[0095] Step S570: perform a loss evaluation between the noise prediction result and the actual noise through a loss function, and update the model parameters of the first generation model according to the gradient of the loss function until the end condition is met to obtain a target generation module.

[0096] In some embodiments, step S530 includes but is not limited to the following steps S531 to S535:

[0097] Step S531: Divide the noise image into sub-blocks of fixed size.

[0098] Step S532: Use a convolutional neural network to convert the sub-block into a fixed-dimensional image vector sequence.

[0099] Step S533: Encode the position information of the sub-block to obtain a learnable vector.

[0100] Step S534: combining the image vector sequence and the learnable vector to obtain an intermediate vector.

[0101] Step S535 : adding a time step mark to the intermediate vector to obtain a first vector feature.

[0102] In some embodiments, the loss function is expressed as:

[0103]

[0104] in, represents the loss function; x0 represents the input image; x t represents the noise image; t represents the time step information; c cat represents the target category attribute condition; c az Indicates the azimuth attribute condition; c pit Indicates the pitch angle attribute condition; c bg Represents background attribute conditions; ∈ θ Represents the noise prediction result; ∈ t Indicates the actual addition of noise.

[0105] The following describes the method of the embodiment of the present application in detail with reference to a specific radar image generation application scenario:

[0106] The embodiment of the present application mainly uses a diffusion model based on multi-attribute collaborative processing to generate SAR images, in which the Transformer block is used for feature extraction. While ensuring the generation quality, the attribute features of multiple SAR images (including target category, azimuth, pitch angle and background) are further finely processed and input into the network to jointly guide the image generation process, ultimately achieving multi-attribute joint controllable synthetic aperture radar image generation.

[0107] The method of the embodiment of the present application can be implemented as a synthetic aperture radar image generation model based on multi-attribute collaborative processing. The overall structure of the model is as follows: Figure 2 As shown, it can include a multi-attribute collaborative processing module and a target generation module. In order to improve the quality and diversity of the generated image, the target generation module of the embodiment of the present application adopts a diffusion model (DiffusionModels) as the main structure of the generation process. The diffusion model mainly includes two processes: the forward diffusion process and the reverse denoising process, as shown in FIG. Figure 3As shown in the figure, the forward diffusion process adds fixed noise to the image, gradually converting it into standard Gaussian noise. The reverse denoising process uses a neural network to gradually remove noise information from the standard Gaussian noise, gradually generating a high-quality image. This type of model has good scalability and parallelization, a very robust objective function, and a more stable training process than some other methods. It also generates samples with high quality and improved diversity.

[0108] The embodiment of the present application uses multiple Transformer blocks in the main architecture, and adopts long skip connections between shallow and deep layers to effectively connect the features of shallow and deep layers, integrate more unknown information, and realize information transmission across multiple layers to better capture global context information, thereby improving the performance of the model without bringing too much computational burden to the model. Combining Transformer with the diffusion model for training and reasoning can better capture the long-distance dependencies between images through the advanced attention mechanism of Transformer, perform better when processing images with complex textures and shapes, and significantly improve the generation quality of SAR images. In addition, the Transformer model has higher parallelism and can complete model training more efficiently.

[0109] Furthermore, the embodiment of the present application designs a Multi-Attribute Collaborative Processing Module (MACPM), whose internal structure is as follows: Figure 4 As shown in FIG, the multi-attribute collaborative processing module is used to collaboratively process and jointly control attributes such as target category, azimuth angle, pitch angle, and background during the entire generation process, thereby realizing multi-attribute joint controllable generation.

[0110] The working steps of the multi-attribute collaborative processing module are as follows: First, the input noisy SAR image data must be preprocessed. The image is patchified and divided into a set of fixed-size sub-blocks. Each patch is converted into a vector sequence of a certain length with a hidden dimension D using a convolutional neural network. At the same time, position embedding is also required to encode the information of each position into a learnable vector and combine it with the image. At the same time, each time step in the input sequence must be passed through the corresponding embedding layer E. tGenerate a corresponding time step token to capture the temporal characteristics of the sequence in the model. The image feature vector obtained in this way retains a lot of information of the original image, so it can provide effective image information for subsequent training.

[0111] Secondly, it is necessary to extract attribute information such as target category, azimuth, elevation, and background, as well as time steps, from the image data. These extracted attributes are then embedded and normalized to make them suitable for model training and generation. It should be noted that a wide range of attributes of available SAR images can be considered, not just target category, elevation, azimuth, and background.

[0112] Specifically, because the possible values ​​of attributes such as target type, pitch angle, and background are relatively few and discrete, a neural network embedding layer is used to map these discrete attributes into a high-dimensional feature space to facilitate model processing and learning. Furthermore, the expression for high-dimensional feature mapping of target category, pitch angle, and background is:

[0113]

[0114] Among them, A category ,A pitch ,A background are the corresponding target categories, pitch angles and background attributes, E cat ,E pit ,E bg denote the neural network embedding functions for target category, pitch angle and background, respectively, v cat ,v pit ,v bg This corresponds to the high-dimensional feature vector obtained after embedding processing of the above attributes.

[0115] As for the azimuth angle attribute, since its possible values ​​are spread across the range [0°, 360°) and the values ​​are continuous, a sine-cosine encoding plus a linear fully connected layer is used to map it to a high-dimensional feature space. This not only represents the size of the azimuth angle but also naturally handles the periodic characteristics of the angle. Based on this, the expression for mapping the azimuth angle is:

[0116] v az =E az ([sin(A azimuth ),cos(A azimuth )]),

[0117] Among them A azimuth is the azimuth attribute, E az represents the fully connected mapping layer of the neural network, vaz The corresponding azimuth angle is processed to obtain the high-dimensional feature vector.

[0118] In order to ensure that the generated SAR images have high quality and accuracy of multi-attribute joint control, the embodiment of the present application designs an advanced loss function for joint optimization during training. The training process is as follows: the input image x0 is obtained, the image diffusion algorithm gradually adds noise to the image, and generates a noise image x t At the same time, a set of multi-attribute conditions is given, including time step information t, target category attribute condition c cat , azimuth attribute condition c az , pitch angle attribute condition c pit And background attribute condition c bg , the image diffusion algorithm needs to learn a network to predict the value of adding to the noise image x t The noise on θ , and make it consistent with the real noise ∈ t As close as possible. Overall loss function The mathematical formula can be expressed as:

[0119]

[0120] In summary, the training process of the synthetic aperture radar image generation model based on multi-attribute collaborative processing proposed in this embodiment may include:

[0121] 1. Data preparation: Collect SAR image datasets containing multiple attributes (target category, azimuth, elevation angle, background, etc.) and perform normalization preprocessing.

[0122] 2. Build model input:

[0123] (1) Noise the SAR image in corresponding time steps t to obtain a noisy image;

[0124] (2) Processing the noisy image and the corresponding time step in the MACPM module: the image is divided into a set of fixed-size sub-blocks (patches), and each patch is converted into a feature vector using a convolutional neural network. The final combination forms a feature vector sequence of the image, and the position embedding information is added and the time step t is processed into the corresponding time step label;

[0125] (3) All attributes of the SAR image are processed collaboratively in the MACPM module: the corresponding attributes of the SAR image (including target category, azimuth, pitch angle, and background, etc.) are extracted separately, and different embedding processes are performed on them to obtain the corresponding high-dimensional feature vectors. They are then spliced ​​together with the high-dimensional feature vector sequence and time step markers obtained after block encoding of the noisy SAR image, and used as the final model input.

[0126] 3. Model design:

[0127] (1) Introducing the Multi-Attribute Collaborative Processing Module (MACPM) to handle the joint control of various attributes;

[0128] (2) Design a generative architecture that includes a Transformer and a diffusion model. The Transformer is used to capture global features and dependencies, and uses long jump connections between shallow and deep layers to achieve feature information transfer, while the diffusion model is used to gradually eliminate noise and generate high-quality images.

[0129] (3) Construct the corresponding loss function and use the time step and multi-attribute information as joint conditions to guide image generation.

[0130] 4. Model training:

[0131] (1) After the noise image is input into the model, the model needs to predict the noise added to the image, that is, the final output is the predicted noise. The loss function is then calculated by combining the predicted noise with the actual noise added, and the model parameters are updated based on the gradient of the loss function through backpropagation.

[0132] The training batch size is 64; the Adamw optimizer is used for training, with a weight decay of 0.03, and parameters set to beta1 = 0.99 and beta2 = 0.999. The learning rate is set to 0.0002, and a warmup strategy is used in the initial stage of training, that is, the learning rate is linearly increased within the first 5000 steps to avoid the model converging too quickly in the early stages of training.

[0133] The reasoning process of the model in this embodiment is:

[0134] 1. Input the attribute values ​​of the image to be generated, including target category, azimuth, elevation angle, and background;

[0135] 2. Perform different embedding processes on the input attribute values ​​through the MACPM module;

[0136] 3. The high-dimensional feature vectors obtained after processing each attribute are spliced ​​and input into the generative model (target generation module). After feature extraction by multiple Transformer blocks in the overall model of this embodiment and noise reduction processing by the diffusion model, a high-quality SAR image is generated.

[0137] The final generated SAR image will be able to fully reflect the various attributes of the input and maintain high quality and consistency of attribute joint generation.

[0138] The embodiments of the present application include at least the following beneficial effects:

[0139] 1. Through the joint action of the attribute-based collaborative processing module, the Transformer block and the diffusion model, the quality and diversity of the generated SAR images are significantly improved. Attributes such as azimuth, elevation and background can be precisely controlled to generate more realistic, detailed, accurate and consistent images. The stable generation of multi-attribute joint control is achieved, which significantly improves the robustness and generalization performance of the model, making it more stable in different application scenarios.

[0140] 2. The Transformer is combined with the diffusion model to make it suitable for SAR image generation tasks. The Transformer's powerful self-attention mechanism can capture global features in the image, while the diffusion model is good at processing noise and generating high-quality images. The two complement each other, enabling this invention to significantly improve the overall quality and diversity of generated SAR images, and better preserve the complex structure and texture information of the original image.

[0141] 3. By coordinating the relationships between multiple attributes, the method of the embodiment of the present application can generate SAR images with strong all-attribute availability, so as to effectively alleviate the problem of insufficient measured sample data and meet the needs of various complex application scenarios, thereby improving its effectiveness and reliability in various fields such as remote sensing, environmental monitoring, and disaster assessment.

[0142] 4. A Multi-Attribute Co-Processing Module (MACPM) and corresponding loss functions are designed to process the multiple attributes of SAR images and integrate them into the training and inference process of the model. The corresponding embedding or encoding method is selected according to the inherent characteristics of each attribute, so that it can independently and flexibly guide the generation process, so that each attribute can be accurately reflected in the generated image.

[0143] The following describes some beneficial effects of the embodiments of the present application in combination with experimental data:

[0144] In order to verify the good effect and reliability, this application conducted relevant experiments on a widely recognized measured SAR image dataset. The dataset used is the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset. This dataset contains ten categories of SAR targets. The azimuth angles of all image data are distributed in the range of [0°, 360°), and the pitch angles have several possible values ​​of 15°, 17°, 30° and 45°. Only some target categories contain SAR images with pitch angles of 30° and 45°. There are fewer data with different background attributes in the MSTAR dataset, so this experiment first clusters the entire dataset according to the energy level of the SAR image background area, with a total of 5 categories, and then uses the clustering results as the background attributes of the SAR data. The effect of the joint controllable generation of multiple attributes of SAR images, including target category, azimuth, pitch angle and background is as follows. Figure 5 As shown, the generated results include four SAR target categories (BMP2, BRDM_2, T72, BTR70), 12 azimuth angles equally spaced in the range [0°, 360°), four elevation angles (15°, 17°, 30°, 45°), and three background categories (Class I, Class III, Class V). It can be seen that the generated SAR image is of high quality and stable, capable of showing the texture changes of different target categories. The changes in azimuth and elevation angles can also be intuitively seen from the direction and shadow changes of the target. The overall energy from low to high also reflects the changes in the background. This fully demonstrates that the SAR image generated by the proposed method can not only accurately reflect each attribute, but also maintain the high quality of the generated results and the reliability of the joint generation when multiple attributes are specified simultaneously.

[0145] It's worth noting that for the two targets (BMP2 and BTR70) shown in the generated results, there was no data in the training set for pitch angles of 30° and 45°, yet after training, the model was still able to generate corresponding and convincing results. This fully demonstrates the model's strong generalization and reasoning capabilities. Even with incomplete training data, the model can still generate corresponding high-quality results based on sufficient learning and reasoning from the existing data.

[0146] Figure 6 The figure shows the generation of ten object categories from the MSTAR dataset (each row represents a category), while each column corresponds to a random multi-attribute combination. As can be seen from the figure, even given the same multi-attribute conditions, the model is able to demonstrate the complex texture differences between different object categories, providing highly accurate and reliable generation results.

[0147] See also Figure 7 The embodiment of the present application also provides a synthetic aperture radar image generation system based on multi-attribute collaborative processing, which can implement the above-mentioned synthetic aperture radar image generation method based on multi-attribute collaborative processing. The system includes:

[0148] The first module 101 is used to obtain a number of image attribute values ​​required to generate an image;

[0149] A second module 102 is configured to perform embedding processing on each of the image attribute values ​​through a multi-attribute collaborative processing module to obtain a first high-dimensional feature vector; wherein the multi-attribute collaborative processing module is configured to collaboratively process and jointly control each of the image attribute values ​​to guide image generation;

[0150] The third module 103 is configured to perform feature concatenation processing on the first high-dimensional feature vector to obtain a second high-dimensional feature vector;

[0151] The fourth module 104 is configured to input the second high-dimensional feature vector into a pre-trained target generation module to obtain a target synthetic aperture radar image corresponding to the image attribute value; wherein the target generation module has a diffusion model as a main structure and includes a plurality of Transformer blocks.

[0152] It should be noted that the system may also include the following modules:

[0153] The fifth module is used to train the target generation module.

[0154] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0155] The present application also provides an electronic device comprising a memory and a processor. The memory stores a computer program, and the processor, when executing the computer program, implements the aforementioned method for generating synthetic aperture radar images based on multi-attribute collaborative processing. The electronic device can be any intelligent terminal, including a tablet computer and an in-vehicle computer.

[0156] It can be understood that the contents of the above method embodiments are applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0157] See also Figure 8 , Figure 8 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0158] The processor 201 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0159] The memory 202 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 202 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 202 and is called by the processor 201 to execute the synthetic aperture radar image generation method based on multi-attribute collaborative processing in the embodiments of this application.

[0160] Input / output interface 203, used to implement information input and output;

[0161] Communication interface 204, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0162] bus 205 , which transmits information between the various components of the device (e.g., processor 201 , memory 202 , input / output interface 203 , and communication interface 204 );

[0163] The processor 201 , the memory 202 , the input / output interface 203 and the communication interface 204 are connected to each other in communication within the device via the bus 205 .

[0164] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned synthetic aperture radar image generation method based on multi-attribute collaborative processing.

[0165] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiment, the functions specifically implemented by the present storage medium embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0166] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0167] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0168] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0169] The system embodiment described above is merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0170] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0171] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0172] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0173] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the system embodiments described above are merely schematic. For example, the division of the above units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0174] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0175] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0176] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0177] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A synthetic aperture radar image generation method based on multi-attribute collaborative processing, characterized in that: The following steps are involved: Obtain several image attribute values ​​required to generate an image; Embedding each of the image attribute values ​​through a multi-attribute collaborative processing module to obtain a first high-dimensional feature vector; wherein the multi-attribute collaborative processing module is used to collaboratively process and jointly control each of the image attribute values ​​to guide image generation; Performing feature concatenation processing on the first high-dimensional feature vector to obtain a second high-dimensional feature vector; Inputting the second high-dimensional feature vector into a pre-trained target generation module to obtain a target synthetic aperture radar image corresponding to the image attribute value; wherein the target generation module has a diffusion model as a main structure and includes a plurality of Transformer blocks; The embedding process of each of the image attribute values ​​by the multi-attribute collaborative processing module to obtain a first high-dimensional feature vector includes the following steps: Through the target category neural network embedding function, the target category is embedded to obtain the high-dimensional feature vector of the target category; The pitch angle is embedded through the pitch angle neural network embedding function to obtain the high-dimensional feature vector of the pitch angle; Through the background neural network embedding function, the background is embedded to obtain the background high-dimensional feature vector; The azimuth is mapped through sine-cosine encoding and linear fully connected layers to obtain a high-dimensional feature vector of the azimuth. Taking the target category high-dimensional feature vector, the pitch angle high-dimensional feature vector, the background high-dimensional feature vector and the azimuth high-dimensional feature vector as a first high-dimensional feature vector; Inputting the second high-dimensional feature vector into a pre-trained target generation module to obtain a target synthetic aperture radar image corresponding to the image attribute value includes the following steps: Performing feature extraction based on long skip connections on the second high-dimensional feature vector through several Transformer blocks to obtain global context information; Combined with the global context information, the standard Gaussian noise is gradually eliminated through the backbone structure of the diffusion model to obtain a target synthetic aperture radar image.

2. The method according to claim 1, characterized in that The acquisition of several image attribute values ​​required for generating an image includes: Obtain the target category, pitch angle, azimuth angle and background required to generate the image, and obtain the image attribute value.

3. The method according to claim 1, characterized in that The method further includes the step of training a target generation module, specifically including the following steps: Acquire a training image and a training image attribute value of the training image; wherein the training image is a synthetic aperture radar image; Performing random time step noise processing on the training image to obtain a noisy image; Performing a first processing on the noise image and the time step of the noise image to obtain an image feature vector having the time step; Embedding the attribute values ​​of the training image to obtain a high-dimensional attribute feature vector; Concatenate the attribute high-dimensional feature vector and the image feature vector as an input vector; Inputting the input vector into a first generation model, performing noise prediction on the noise added to the noise image, and obtaining a noise prediction result; The loss function is used to evaluate the loss between the noise prediction result and the actual added noise. According to the gradient of the loss function, the model parameters of the first generation model are updated until the end conditions are met, thereby obtaining the target generation module.

4. The method according to claim 3, characterized in that The first processing of the noise image and the time step of the noise image to obtain an image feature vector having a time step includes the following steps: dividing the noisy image into sub-blocks of fixed size; Using a convolutional neural network to convert the sub-blocks into a sequence of image vectors of fixed dimension; Encoding the position information of the sub-block to obtain a learnable vector; Combining the image vector sequence and the learnable vector to obtain an intermediate vector; A time step mark is added to the intermediate vector to obtain a first vector feature.

5. The method according to claim 3, characterized in that The expression of the loss function is: ， in, represents the loss function; represents the input image; represents a noisy image; Represents time step information; Represents the target category attribute condition; Indicates the azimuth attribute condition; Indicates the pitch angle attribute condition; Represents background attribute conditions; represents the noise prediction result; Indicates the actual addition of noise.

6. A system for implementing the synthetic aperture radar image generation method based on multi-attribute collaborative processing according to any one of claims 1 to 5, characterized in that: include: The first module is used to obtain several image attribute values ​​required to generate an image; a second module configured to embed each of the image attribute values ​​using a multi-attribute collaborative processing module to obtain a first high-dimensional feature vector; wherein the multi-attribute collaborative processing module is configured to collaboratively process and jointly control each of the image attribute values ​​to guide image generation; A third module is used to perform feature concatenation processing on the first high-dimensional feature vector to obtain a second high-dimensional feature vector; A fourth module is configured to input the second high-dimensional feature vector into a pre-trained target generation module to obtain a target synthetic aperture radar image corresponding to the image attribute value; wherein the target generation module has a diffusion model as its main structure and includes a plurality of Transformer blocks.

7. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 5.

8. A computer storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to implement the method according to any one of claims 1 to 5 when executed by the processor.

Images