SAR (Synthetic Aperture Radar) image simulation method based on Mama parameter extraction network

Abstract

The invention provides an SAR (Synthetic Aperture Radar) image simulation method based on a Mama parameter extraction network, which takes a 3D (Three-Dimensional) model of a target object and a real high-definition SAR image as input data of a framework, obtains a mapping relation between the SAR image and simulation parameters by training the Mama network, realizes conversion from the simulation parameters to the simulation SAR image by virtue of RaySAR software, and improves the simulation accuracy of the SAR image. Finally, accurate simulation parameters are extracted from a single real SAR image, and a large number of simulation SAR images are generated. Compared with a traditional SAR image simulation method, machine learning and a simulation method based on the imaging principle are combined, SAR image simulation is assisted through machine learning, the simulation speed is higher, the simulation effect is better, and the utilization rate of real SAR image data is increased.

Description

Technical Field

[0001] The invention relates to the technical field of synthetic aperture radar image simulation, and in particular to a SAR image simulation method based on a Mamba parameter extraction network. Background Art

[0002] Synthetic Aperture Radar (SAR) is an advanced remote sensing technology that acquires surface information by actively emitting microwave signals and receiving the reflected signals. SAR has the ability to work around the clock and in all weather conditions. High-resolution SAR images can provide a large amount of detailed information about the target, such as structure, shape, material, etc.

[0003] When we interpret SAR images, especially when using machine learning methods, we often need a large number of labeled target images. However, SAR image data of the same target in multiple scenes and multiple imaging conditions is often scarce. At the same time, the use of SAR image simulation technology based on simulation parameter extraction can quickly provide SAR target simulation images in different scenes and imaging conditions at a low cost, thereby achieving approximate supplementation of the above scarce data. These simulated images can be widely used in many fields such as SAR target recognition, image interpretation, and three-dimensional reconstruction.

[0004] Accurate simulation parameters are the key to obtaining high-quality simulated images. However, traditional parameter acquisition methods, including direct measurement from actual targets and manual adjustment, require a lot of time, energy and resources, and it is difficult to obtain simulation parameters accurately and quickly. In addition, since the analytical relationship between SAR images and their simulation parameters is difficult to express through explicit functional relationships, although there are a large number of real SAR images, it is difficult to use them to assist in the generation of simulated SAR images. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides a SAR image simulation method and system based on the Mamba parameter extraction network to solve the technical problems in the prior art that it is difficult to accurately and quickly obtain simulation parameters and to generate simulated SAR images.

[0006] The present invention provides a SAR image simulation method based on a Mamba parameter extraction network, comprising:

[0007] S1. Preprocessing a given target 3D model, and inputting the target 3D model into SAR image simulation software to produce a SAR image data set;

[0008] S2. Building a parameter inversion network based on the Mamba model, and using the SAR image data set to train the parameter inversion network;

[0009] S3, inputting the real SAR image into the parameter inversion network, outputting the simulation parameters, and importing the simulation parameters into the RaySAR software to output the simulated SAR image.

[0010] Optionally, the preprocessing of the given target 3D model includes:

[0011] The center of mass of the target object in the target 3D model is located at the center of the 3D Studio Max software workspace, and the size of the 3D model is adjusted based on the application scenario. The texture information in all layers of the target 3D model is uniformly erased, and the 3D model is converted into a .pov format file.

[0012] Optionally, the adopting the SAR image data set to train the parameter inversion network includes:

[0013] The simulation parameters in the SAR image data set at least include the diffuse reflection coefficient, roughness, and specular reflection weight of the ground and the target, respectively, and the specular reflection coefficient of the ground.

[0014] Optionally, building a parameter inversion network based on the Mamba model and using the SAR image data set to train the parameter inversion network includes:

[0015] Select a SAR image I from the SAR image dataset, A 2D convolution with a kernel size of k and a stride of s is used to map the local block into a pixel-level feature map, and the feature map is flattened into a 1D sequence, represented as:

[0016] T=Φ Flatten (Φ Conv2D (I,k,s))

[0017] T=T+P

[0018] where Φ Conv2D represents 2D convolution, Φ Flatten represents the flattening operation, represents the input 1D sequence, Indicates positional encoding;

[0019] A dynamic multi-path activation mechanism is used for feature extraction. After feature extraction, it is normalized by layers, average pooling is performed, and finally a linear layer and sigmoid activation are used to complete the regression task.

[0020] Optionally, the adopting a dynamic multi-path activation mechanism to perform feature extraction includes:

[0021] Three copies of the input sequence are copied to establish the forward path, the reverse path, and the random path respectively, and the dependencies in the forward path, the reverse path, and the random path are modeled respectively. Then, the tokens in the sequence are restored to the correct order, and a linear layer is used to compress the sequence information to activate the representation of different information flows. The process of the i-th block is as follows:

[0022]

[0023]

[0024]

[0025]

[0026]

[0027] Where T i represents the input sequence of the i-th layer, k∈{0,1,2} represents the kth sequence path, which are the forward path, the reverse path and the random path, respectively. mixer is the vanilla Mamba mixer with parameters θ, Φ revert-pather represents the operation of restoring all marks to the forward order, Φ cat represents the connection between the sequence and the feature dimension, Φ mean represents the average pooling along the sequence length dimension, Φ gate-proj represents the activation of the sequence information of the linear projection 3D dimension to the forward path, the reverse path and the random path, Φ softmax Represents the Softmax operation.

[0028] Optionally, the step of inputting the real SAR image into the parameter inversion network, outputting the simulation parameters, and importing the simulation parameters into RaySAR software to output the simulated SAR image comprises:

[0029] A real SAR image is selected and input into the parameter inversion network, based on the output simulation parameters, the corresponding simulation parameters in the .pov format file are modified based on the output simulation parameters, and the RaySAR software is used to render the output simulated SAR image.

[0030] Compared with the prior art, the present invention has the following advantages:

[0031] With the 3D model of the target object and the real high-definition SAR image as the input data of the framework, the mapping relationship between the SAR image and the simulation parameters is obtained by training the Mamba network. The conversion between the simulation parameters and the simulated SAR image is realized with the help of RaySAR software. Finally, accurate simulation parameters can be extracted from a single real SAR image and a large number of simulated SAR images can be generated based on it.

[0032] Compared with traditional SAR image simulation methods, combining machine learning with simulation methods based on imaging principles and using machine learning to assist SAR image simulation has faster simulation speed, better simulation effect, and improved the utilization rate of real SAR image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0035] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0036] Figure 2 A method logic diagram of an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of the simulation parameter inversion network structure of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other implementation cases obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. The functional units with the same labels in the examples of the present invention have the same and similar structures and functions.

[0039] See also Figure 1 The present invention provides a SAR image simulation method based on a Mamba parameter extraction network, comprising:

[0040] S1. Preprocessing a given target 3D model, and inputting the target 3D model into SAR image simulation software to produce a SAR image data set;

[0041] S2. Building a parameter inversion network based on the Mamba model, and using the SAR image data set to train the parameter inversion network;

[0042] S3, inputting the real SAR image into the parameter inversion network, outputting the simulation parameters, and importing the simulation parameters into the RaySAR software to output the simulated SAR image.

[0043] In this embodiment, S1, preprocessing a given target 3D model, and inputting the target 3D model into SAR image simulation software to produce a SAR image data set.

[0044] First, use 3D Studio Max software to process the 3D model of the target. At the beginning of the operation, in order to ensure the accuracy and convenience of subsequent simulation, the center of mass of the target object should be accurately located at the center of the 3ds Max workspace. According to the needs of the actual application scenario, the size of the target model should be adjusted to make it the same as the real target size. At the same time, in order to facilitate the setting of the azimuth angle in accordance with the MSTAR dataset, its angle is rotated so that the target is facing the outside of the screen.

[0045] Afterwards, we save the adjusted 3D model in .3ds format or other AccuTrans3D compatible formats so that it can be smoothly imported and parameterized in subsequent simulation software.

[0046] Next, the 3D model was further processed using at3D, a professional 3D model conversion tool. Considering that RaySAR software needs to control the performance of the target surface characteristics through precisely set simulation parameters, it is necessary to uniformly erase the texture information in all layers of the target model in advance, which helps to reduce interference factors and ensure that the geometric properties of the model itself are relied upon during the simulation process to accurately reflect its electromagnetic scattering characteristics.

[0047] The processed model is then converted into a .pov format file, which is widely supported by various ray tracing renderers, ensuring that the target model can obtain efficient and high-precision simulation results after being input into the RaySAR software.

[0048] See also Figure 2 , the present invention mainly predicts several material surface parameters that are difficult to determine during image simulation but have a greater impact on the image. In addition to the parameters to be measured, some other parameters are required for SAR image simulation using RaySAR, including the type and angle of light, the position of the camera, etc. These parameters need to be debugged and set in advance based on experience. During simulation, the type of light is set to parallel light that is more in line with SAR imaging simulation, the camera position is set to the same as the light source position, and the three-dimensional coordinates of the camera position are calculated in combination with the pitch angle, azimuth angle and range distance of the real SAR image provided in the data set.

[0049] Taking the armored transport vehicle BTR70 image in MSTAR as an example, for the image labeled HB03379, its pitch angle is 15°, DesiredRange=4500, and the azimuth angle is 90°, so the camera position is set to (4500,1205.77,0) during simulation.

[0050] Secondly, the SAR image data set is produced. By analyzing the interaction between various parameter variables and their significant influence on the simulation results, the parameters to be tested are sampled parameter by parameter within a specific range.

[0051] Specifically, for those parameters that will cause significant visual changes to the generated image or have a significant impact on the output results, a more intensive sampling strategy is adopted to fully capture the various possible forms brought about by small changes in these parameters. On the contrary, for those parameters that have relatively limited impact on the generated image even if the numerical changes are large, a more sparse sampling method is implemented to avoid excessively increasing the amount of data and ignoring the truly important dimensions. This can not only ensure the validity of the data set, but also reasonably control resource consumption and avoid the redundant accumulation of invalid data.

[0052] When determining the range of parameter values, it should be ensured that it covers a wide and reasonable range, covering all expected situations as much as possible to prevent prediction failure during the training and testing phases. At the same time, the setting of sampling intervals is particularly critical, especially in parameter-sensitive areas. The intervals should be minimized near the values ​​where important changes are most likely to occur, so as to capture subtle change trends and improve the generalization ability and accuracy of the prediction network.

[0053] Multiple experimental comparisons found that among the material surface parameters, the changes in the ground's diffuse reflectance and roughness have a greater impact on the simulation image, so dense sampling is used. For example, the ground's diffuse values ​​are: 0.11, 0.13, 0.15, 0.17, 0.19, 0.21, 0.23, 0.25, 0.27, 0.29, 0.31, 0.33, 0.35, 0.37, 0.39, 0.41, 0.43, 0.45, 0.47, 0.49, 0.6, 0.7, 0.8, 0.9; the target's diffuse reflectance and roughness have a greater impact on the simulation image, so dense sampling is used. The changes in roughness, specular reflection coefficient, and specular reflection weight of the target and the ground have little effect on the simulated image, so sparse sampling is used. For example, the specular values ​​of the ground are: 0.10, 0.20, 0.30, 0.35, 0.40, 0.45, 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, and 0.90; and the value of the scattering attenuation coefficient (ambient) of the target or the ground is easy to determine manually, so it is directly set to fixed values ​​0.1 and 0, and no subsequent prediction is performed.

[0054] S2. Building a parameter inversion network based on the Mamba model, and using the SAR image dataset to train the parameter inversion network.

[0055] The parameter prediction network is based on the Mamba network, which adopts a position-sensitive dynamic multi-path activation mechanism to address the limitations of the original Mamba, converts a two-dimensional image into a one-dimensional sequence, and uses a multi-path state space model (SSM) encoder to capture long-range dependencies.

[0056] The parameter inversion network is based on the Mamba model, and a selection mechanism is used to improve the state space model (SSM), thereby enhancing the expressiveness of the model. A hardware-aware algorithm that integrates kernels and recalculations is designed to avoid the storage of intermediate states and improve speed and memory efficiency. The SSM block in H3 and the MLP block in Transformer are merged into a simplified block, thereby improving the efficiency of training and inference.

[0057] The overall network structure can be found in Figure 3 First, a SAR image is selected from the SAR image dataset. A 2D convolution with a kernel size of k and a step size of s is used to map the local block to a pixel-level feature embedding, after which the feature map is flattened into a 1D sequence. In order to maintain the spatial position relationship within the image, a position encoding P is added. The process can be expressed as:

[0058] T=ΦFlatten (Φ Conv2D (I,k,s))

[0059] T=T+P

[0060] where Φ Conv2D represents 2D convolution, Φ Flatten represents the flattening operation, represents the input 1D sequence, Represents positional encoding.

[0061] A dynamic multi-path activation mechanism is then used for feature extraction. Specifically, three copies of the input sequence are copied to establish three different paths: a forward path, a reverse path, and a random path, and a pure Mamba mixer with shared parameters is used to model the dependencies in the three sequences respectively. The tokens in the sequence are then restored to the correct order, and a linear layer is used to compress the sequence information, thereby establishing a gating mechanism for the three paths, and then using this gating mechanism to activate the representation of different information streams. The process of the i-th block is as follows:

[0062]

[0063]

[0064]

[0065]

[0066]

[0067] Where T i represents the input sequence of the i-th layer, k∈{0,1,2} represents the kth sequence path, including the forward path, the reverse path and the random path Φ mixer is the vanilla Mamba mixer with parameters θ, Φ revert-pather represents the operation of restoring all marks to the forward order, Φ cat represents the connection between the sequence and the feature dimension, Φ mean represents the average pooling along the sequence length dimension, Φ gate-proj represents the activation of the sequence information of the linear projection 3D dimension to the forward path, the reverse path and the random path, Φ softmax Represents the Softmax operation.

[0068] After feature extraction, it is normalized by layers, average pooled, and finally activated by linear layers and sigmoid to complete the regression task, realize regression prediction of each parameter to be tested, and ensure that the value of each predicted parameter is within the range of 0-1.

[0069] S3, inputting the real SAR image into the parameter inversion network, outputting the simulation parameters, and importing the simulation parameters into the RaySAR software to output the simulated SAR image.

[0070] Next, the parameter inversion network built in step S2 is trained. The trained parameter inversion network is an end-to-end network. By inputting a real SAR image into the trained network, a set of predicted simulation parameters can be obtained, including the diffuse reflectance (diffuse), roughness (roughness), specular reflection weight (reflection) of the ground and the target respectively, and the specular reflectance (specular) of the ground.

[0071] Use the POV-Ray part of the RaySAR software to process the .pov file generated in step S1, and modify the corresponding simulation parameters in the file according to the predicted values ​​to render an optical 3D image and an output file "Contributions.txt" containing SAR-related information. Then input the txt file into the MATLAB GUI of RaySAR and set the remaining relevant parameters to simulate and generate a SAR image.

[0072] Compared with the traditional SAR image simulation method, the present invention combines machine learning with a simulation method based on imaging principles and uses machine learning to assist SAR image simulation, which has a faster simulation speed and better simulation effect, and improves the utilization rate of real SAR image data.

[0073] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0074] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A SAR image simulation method based on Mamba parameter extraction network, characterized in that: include: S1. Preprocessing a given target 3D model, and inputting the target 3D model into SAR image simulation software to produce a SAR image data set; S2. Building a parameter inversion network based on the Mamba model, and using the SAR image data set to train the parameter inversion network; S3, inputting the real SAR image into the parameter inversion network, outputting the simulation parameters, and importing the simulation parameters into the RaySAR software to output the simulated SAR image.

2. The SAR image simulation method based on the Mamba parameter extraction network according to claim 1, characterized in that: The preprocessing of the given target 3D model includes: The center of mass of the target object in the target 3D model is located at the center of the 3D Studio Max software workspace, and the size of the 3D model is adjusted based on the application scenario. The texture information in all layers of the target 3D model is uniformly erased, and the 3D model is converted into a .pov format file.

3. The SAR image simulation method based on the Mamba parameter extraction network as claimed in claim 1, characterized in that: The adopting of the SAR image data set to train the parameter inversion network comprises: The simulation parameters in the SAR image data set at least include the diffuse reflection coefficient, roughness, and specular reflection weight of the ground and the target, respectively, and the specular reflection coefficient of the ground.

4. The SAR image simulation method based on the Mamba parameter extraction network as claimed in claim 3, characterized in that: The parameter inversion network is constructed based on the Mamba model, and the parameter inversion network is trained using the SAR image data set, including: Select a SAR image from the SAR image dataset A 2D convolution with a kernel size of k and a stride of s is used to map the local block into a pixel-level feature map, and the feature map is flattened into a 1D sequence, represented as: T=Φ Flatten (F Conv2D (I,k,s)) T=T+P where Φ Conv2D represents 2D convolution, Φ Flatten represents the flattening operation, represents the input 1D sequence, Indicates positional encoding; A dynamic multi-path activation mechanism is used for feature extraction. After feature extraction, it is normalized by layers, average pooling is performed, and finally a linear layer and sigmoid activation are used to complete the regression task.

5. The SAR image simulation method based on the Mamba parameter extraction network as claimed in claim 4, characterized in that: The method of extracting features by using a dynamic multi-path activation mechanism includes: Three copies of the input sequence are copied to establish the forward path, the reverse path, and the random path respectively, and the dependencies in the forward path, the reverse path, and the random path are modeled respectively. Then, the tokens in the sequence are restored to the correct order, and a linear layer is used to compress the sequence information to activate the representation of different information flows. The process of the i-th block is as follows: Where T i represents the input sequence of the i-th layer, represents the kth sequence path, which are the forward path, the reverse path and the random path, Φ mixer is the vanilla Mamba mixer with parameters θ, Φ revert-pather represents the operation of restoring all marks to the forward order, Φ cat represents the connection between the sequence and the feature dimension, Φ mean represents the average pooling along the sequence length dimension, Φ gate-proj represents the activation of the sequence information of the linear projection 3D dimension to the forward path, the reverse path and the random path, Φ softmax Represents the Softmax operation.

6. The SAR image simulation method based on the Mamba parameter extraction network as claimed in claim 2, characterized in that: The method of inputting the real SAR image into the parameter inversion network, outputting the simulation parameters, and importing the simulation parameters into the RaySAR software to output the simulated SAR image comprises: A real SAR image is selected and input into the parameter inversion network, simulation parameters are output, corresponding simulation parameters in a .pov format file are modified based on the output simulation parameters, and the simulated SAR image is rendered and output using RaySAR software.

Images