Arbitrary view angle SAR target simulation sample generation method
Generating simulation samples of SAR targets at any angle through convolutional neural network and SBR methods solves the problems of slow generation speed and poor quality in the prior art, and achieves fast and high-quality generation without the need for high-precision electromagnetic scattering models and large number of sample data sets.
Patent Information
- Application Number
- CN202510506801.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-22
- Publication Date
- 2025-08-08
AI Technical Summary
The prior art is difficult to generate high-quality SAR target simulation samples of any angle of view, and it is necessary to build a high-precision target electromagnetic scattering model and a large number of diverse SAR target sample data sets, resulting in large and slow computing resources consumption.
The target three-dimensional reconstruction network model is constructed using convolutional neural network, combined with SBR method for rendering and optimization, and generate simulation images of SAR targets at any perspective angle, and optimize network parameters through mixed loss functions to avoid building high-precision electromagnetic scattering models and large sample data sets.
It quickly generates high-quality simulation samples of any angle of SAR targets, solves the problems of difficulty in obtaining data and insufficient multi-view samples, and has good application potential.
Smart Images
Figure BDA0005370030390000062 
Figure FDA0005370030370000031 
Figure FDA0005370030370000032
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of radar image processing and deep learning, and in particular to a method for generating simulated SAR target samples from any viewing angle, which is suitable for application scenarios such as generating simulated SAR target samples and preparing data sets. Background Art
[0002] Synthetic Aperture Radar (SAR) has the ability to collect and acquire data around the clock and in all weather conditions. Using SAR to conduct large-scale monitoring and surveillance of land and ocean has great advantages and is of great significance. Therefore, it is widely used in regional reconnaissance, key target monitoring and other fields.
[0003] In recent years, deep learning has been widely used in scenarios such as SAR target detection and recognition. Detecting and identifying targets such as aircraft, vehicles, and ships can provide critical military intelligence information, including enemy weaponry and strategic deployments. This helps guided weapons with precise detection and control capabilities achieve high-precision strikes against these targets in complex battlefield environments. However, the position and attitude of targets such as aircraft, vehicles, and ships change over time, and satellites can only capture SAR imagery from specific perspectives, resulting in incomplete coverage due to the following reasons:
[0004] (1) The routes of aircraft and ships are relatively fixed, and the satellite revisit period is insufficient to obtain SAR image data of aircraft and ship targets under various attitude conditions at different time periods;
[0005] (2) Due to limitations in satellite orbit and imaging viewing angle, imaging cannot cover the target in all directions;
[0006] (3) When an airplane docks at an airport, the parking space has a fixed direction; when a ship docks at a pier, the parking space has a fixed direction; when a vehicle docks at a parking lot, the parking space also has a fixed direction; their directions are similar and lack diversity.
[0007] Therefore, in the process of assisting SAR target detection and recognition, obtaining SAR target image data from any perspective and increasing sample diversity are crucial to improving the recognition accuracy of the network model.
[0008] At present, there are two main methods for generating arbitrary-viewpoint SAR target simulation samples:
[0009] One is a sample generation algorithm based on a generative adversarial network. This algorithm can quickly generate a large number of simulated sample images, but it also has the following shortcomings: model training requires a large number of correctly labeled SAR target sample images, and the quality of the generated simulated sample images is generally poor; at the same time, sample diversity is crucial to the robustness and stability of model training, and a large amount of diverse SAR target sample data is difficult to obtain, which is also a common problem faced in the field of SAR target recognition.
[0010] Another approach is to construct a target electromagnetic scattering model, obtain the target's original echo signal data based on high-frequency electromagnetic field asymptotic methods such as geometric optics (GO), physical optics (PO), geometric theory of diffraction (GTD), or physical theory of diffraction (PTD), and then perform imaging processing to obtain a simulated SAR image of the target. This method can simulate both amplitude / intensity information and phase information, and can fully and realistically reflect and reveal the electromagnetic scattering characteristics of the SAR target without the need to build a data set or train learning. However, it also has some shortcomings: this method requires the construction of a high-precision target model, and the complex electromagnetic calculations make the calculation process very time-consuming and require a large amount of memory space. Summary of the Invention
[0011] The present invention aims to avoid the problems of the aforementioned background methods and provide a method for generating simulated SAR target samples from any viewing angle. This method eliminates the need to construct a high-precision target electromagnetic scattering model or prepare a dataset containing a large number of diverse SAR target samples. It can generate SAR target sample data from any viewing angle, effectively addressing the difficulties in acquiring SAR target data and the shortage of multi-view sample data. The present invention generates samples quickly and with high quality, demonstrating strong potential for practical applications.
[0012] The technical solution adopted by the present invention to solve the technical problem is:
[0013] A method for generating SAR target simulation samples from any viewing angle comprises the following steps:
[0014] Step 1: Construct a target 3D reconstruction network model based on a convolutional neural network. The target 3D reconstruction network model includes two parts: an encoder and a decoder. The encoder is used to extract features of the input 2D SAR image, and the decoder obtains the coordinates of the triangle surface vertices through feature transformation to generate a predicted target 3D model.
[0015] Step 2: training the target 3D reconstruction network model to obtain a trained target 3D reconstruction network model;
[0016] Step 3: Input the two-dimensional SAR target image for three-dimensional reconstruction into the trained target three-dimensional reconstruction network model to obtain a reconstructed target three-dimensional model;
[0017] Step 4: Render the target 3D model obtained in step 3 using the SBR method; obtain observation data of the target 3D model at the corresponding angle according to the viewing angle of each 2D SAR target image, and generate a SAR target simulation image consistent with the viewing angle of the 2D SAR target image;
[0018] Step 5: construct an objective function, calculate the error between the SAR target simulation image and the corresponding two-dimensional SAR target image, and optimize the parameters of the trained target three-dimensional reconstruction network model through back propagation and gradient descent methods;
[0019] Step 6: re-input the two-dimensional SAR target image in step 3 into the optimized target three-dimensional reconstruction network model to obtain the final target three-dimensional model;
[0020] Step 7: Render the final target 3D model using the SBR method; obtain observation data of the final target 3D model at the corresponding angle according to the required viewing angle, and generate a SAR target simulation image at the required viewing angle.
[0021] Furthermore, the two-dimensional SAR image input to the target three-dimensional reconstruction network model contains three channels and at least four different perspectives, and the angles of these four perspectives are distributed in four different quadrants.
[0022] Furthermore, the target 3D reconstruction network model works as follows:
[0023] The encoder uses three convolutional layers to extract the 2D SAR image features for each channel. After normalization, the images are transformed nonlinearly using the ReLU activation function to generate 1D feature information. Three fully connected layers are then used to further extract features. A triangular mesh is used to divide the target 3D model into finite facets, and the resulting triangular facets are combined to form the target 3D model.
[0024] The decoder uses two separate fully connected layers to extract the vertex coordinates P of each triangle face from the features output by the encoder. i,j ,i=1,2,3...,N,j=1,2,3;where N is the number of triangle faces, P i,j Represents the coordinates of the jth vertex of the i-th triangle face; based on the coordinates of each vertex P i,j , generating a predicted target 3D model.
[0025] Furthermore, the specific method of step 4 is:
[0026] Step 401: Project the direction opposite to the incident wave direction according to the geometric shape of the target three-dimensional model to generate an isophase surface, and divide the isophase surface to obtain an initial ray tube;
[0027] Step 402 , using the initial ray tube, performing ray tracing on the target three-dimensional model;
[0028] Step 403: Integrate the surface current of the triangular surface element intersecting the ray tube according to the tracking result to calculate the scattered field;
[0029] Step 404: Perform vector accumulation on all scattered fields obtained in this tracking to obtain the total scattered field of the target 3D model in the current incident direction, and calculate the radar cross section corresponding to the target 3D model in the current incident wave direction;
[0030] Step 405: adjust the direction of the incident wave, update the intersection information between the ray tube and the triangular facets in the target 3D model, as well as the amplitude, phase, and polarization information of the ray tube, and perform ray tracing on the target 3D model again;
[0031] Step 406, repeating steps 403 to 405 until there is no intersection between the ray tube and the target three-dimensional model; at this point, radar cross section data of the target three-dimensional model at multiple observation angles are obtained;
[0032] Step 407 : Based on the viewing angle of each two-dimensional SAR target image and the radar cross section data of the target three-dimensional model at the corresponding observation angle, a SAR target simulation image having the same viewing angle as the two-dimensional SAR target image is generated based on the CS linear frequency modulation scaling imaging method.
[0033] Furthermore, the specific method of step 5 is:
[0034] Step 501: construct the objective function L:
[0035] L=α·L M +(1-α)·G σ ·L s ;
[0036] Among them, α is the weight, G σ is the Gaussian distribution parameter, L M is the MS-SSIM loss function, which is used to characterize the differences in detail features and structural contours between the two-dimensional SAR target image x and the SAR target simulation image y. s is the scattering image loss function, which is used to characterize the difference in brightness and grayscale values of corresponding pixels on the two-dimensional SAR target image x and the SAR target simulation image y; L M Expressed as:
[0037]
[0038] L s Expressed as:
[0039]
[0040] Among them, x i ,y i ~P data (x, y) represents the corresponding coordinate pixel in the two-dimensional SAR target image x and the SAR target simulation image y; represents the average value after accumulation of the entire image; MS_SSIM(·) is the multi-scale structural consistency loss function;
[0041] In step 502, the error between the simulated SAR target image y and the two-dimensional SAR target image x is calculated based on the objective function L, and the error is transferred back to the vertex coordinates and texture information of the triangle facets. The parameters of the trained target 3D reconstruction network model are adjusted through back propagation and gradient descent methods to achieve optimization adjustment of the trained target 3D reconstruction network model.
[0042] The beneficial effects of the present invention are:
[0043] 1. The present invention can generate SAR simulation sample images of typical targets with fast sample generation speed and high quality, and has great application potential.
[0044] 2. The present invention does not require the construction of a high-precision target electromagnetic scattering model, nor does it require the preparation of a data set containing a large number of diverse SAR target samples. It can generate SAR target sample data from any perspective, and can effectively solve the problems of difficulty in obtaining SAR target data and insufficient multi-perspective sample data. DETAILED DESCRIPTION
[0045] The present invention is further described with reference to the following specific examples.
[0046] A method for generating SAR target simulation samples at arbitrary viewing angles, the method specifically comprising the following steps:
[0047] (1) Constructing a target 3D reconstruction network model based on a convolutional neural network; the specific method is as follows:
[0048] (101) The network structure of the target 3D reconstruction network model includes two parts: encoder and decoder;
[0049] (102) The model input is a two-dimensional SAR target image under different viewing angles; this includes at least four different viewing angles, and the viewing angles must be distributed in four different quadrants; the two-dimensional SAR target image contains three channels;
[0050] In the target 3D reconstruction network model, the encoder extracts 2D SAR image features, and the decoder obtains the coordinates of the triangle face vertices through feature transformation to generate the predicted target 3D model. The specific working method of the target 3D reconstruction network model is as follows:
[0051] The encoder first uses three convolutional layers to extract the two-dimensional SAR image features of each channel. After normalization, it uses the ReLU activation function to perform nonlinear transformation to generate one-dimensional feature information. Then, it uses three fully connected layers to further extract features.
[0052] Considering that the target surface is a complex non-planar surface, a triangular mesh is used to divide the target 3D model into finite facets, and the target 3D model is composed of triangular facets one by one; the decoder uses two separate fully connected layers to extract the vertex coordinates P of each triangular mesh facet from the output features of the encoder. i,j ,i=1,2,3...,N,j=1,2,3; where N is the number of triangle elements, P i,j represents the j-th vertex of the i-th triangle face;
[0053] Based on the coordinates P of each vertex i,j , generate a predicted target three-dimensional model;
[0054] (2) training the target 3D reconstruction network model to obtain a trained target 3D reconstruction network model;
[0055] (3) Inputting the two-dimensional SAR target image for three-dimensional reconstruction into the trained target three-dimensional reconstruction network model to obtain a reconstructed target three-dimensional model;
[0056] (4) Use the SBR algorithm to render the target three-dimensional model and obtain a SAR target simulation image that is consistent with the perspective of the two-dimensional SAR target image. The SBR algorithm represents the incident wave with a series of ray tubes and divides the initial ray tubes by generating equal phase surfaces. The data structure of the ray tube contains the amplitude, phase, polarization and other related attribute information of the electromagnetic wave. In the process of ray tracing, the main ray of the ray tube is used to represent the electromagnetic information of the entire ray tube, and the change in the area of the ray tube before and after ray tracing is used to represent the amplitude information. The specific method is:
[0057] (401) According to the geometric shape of the target model, the target model is oriented along the direction of the incident wave. The opposite direction Perform projection to generate an isophase surface, and divide the generated isophase surface into steps of 1 / 10 wavelength to obtain an initial ray tube;
[0058] (402) using the initial ray tube, performing ray tracing on the target three-dimensional model;
[0059] (403) Based on the tracking results, the surface current of the triangular surface element intersecting with the ray tube is integrated to calculate the scattered field
[0060]
[0061] The integration domain S is the area of the ray tube at the last reflection in the ray tracing process, and the vector is the outer normal of the triangle surface element (i.e., target surface element) that intersects the ray tube, and the vector Represents the position of any point on S, vector Indicates the incident direction of the ray tube when it is reflected at a certain ray tracing stage, vector Indicates the scattering direction, vector is the position of the reflection point of the main ray of the ray tube. In backscattering, j, k, R are Green function parameters, is the Green function, is the magnetic field at the last reflection point of the ray tube, and M is the number of reflections of the ray tube.
[0062] The surface integral above can be converted into a line integral by using Green's formula:
[0063]
[0064] Among them, vector is the lth edge vector of the target element, vector is the midpoint of the lth edge. During the calculation process, this formula will have the following two singular cases:
[0065] ① When The above formula can be simplified to:
[0066]
[0067] ②When have:
[0068]
[0069] Among them, S Δ is the area of the target cell.
[0070] (404) Scattering fields for all target elements Perform vector accumulation to obtain the total scattered field of the target three-dimensional model in the current incident direction The target 3D model is in the direction of the current incident wave The corresponding radar cross section RCS can be calculated by the following formula:
[0071]
[0072] Among them, vector is the scattering direction, vector is the polarization of the incident wave, The total scattered field computed for all tube ray traces.
[0073] (405) Adjust the direction of the incident wave Update the intersection information between the ray tube and the triangle surface element in the target 3D model, as well as the amplitude, phase, polarization and other information of the ray tube, and perform ray tracing on the target 3D model again;
[0074] (406) Repeat steps 403 to 405 until there is no intersection between the ray tube and the target three-dimensional model; at this point, the RCS data of the target three-dimensional model under multiple observation angles are obtained;
[0075] (407) According to the viewing angle of each two-dimensional SAR target image, the RCS data of the target three-dimensional model at the corresponding observation angle is used to generate a SAR target simulation image consistent with the viewing angle of the two-dimensional SAR target image based on the CS linear frequency modulation scaling imaging method.
[0076] (5) Construct a hybrid objective function, calculate the error between the SAR target simulation image y and the two-dimensional SAR target image x, and optimize the parameters of the target three-dimensional reconstruction network model through back propagation and gradient descent strategies. The specific method is:
[0077] (501) The image input to the target 3D reconstruction network encoder is denoted as x, and the output of the target 3D reconstruction network (the predicted target 3D model) is denoted as Where Q(·) represents the mapping of the target 3D reconstruction network; the SAR target simulation image obtained by rendering the target 3D model using the SBR algorithm and consistent with the input image perspective is Where F(·) represents the SBR algorithm;
[0078] (502) The MS-SSIM loss value is recorded as:
[0079]
[0080] Among them, x i ,y i ~P data (x,y) means taking the corresponding coordinate pixels in image x and image y respectively; represents the average value after accumulation of the entire image; MS_SSIM(·) is a multi-scale structural consistency loss function, which takes into account the unique electromagnetic scattering characteristics of SAR images, can characterize the image's detailed features and structural contours, and better reflect the structural consistency between training samples and simulated SAR images;
[0081] (503) The scattering image loss value is recorded as:
[0082]
[0083] L s It can represent the difference in brightness and grayscale values of corresponding pixels on image x and image y;
[0084] (504) A hybrid loss function is used to supervise the target 3D model reconstruction, which combines the MS-SSIM loss function with the scatter image loss function L s Combined to form a weighted loss function, recorded as:
[0085] L=α·L M +(1-α)·G σ ·L s ;
[0086] Among them, α is the weight, which is generally 0.7; G σ is the Gaussian distribution parameter;
[0087] (505) Based on the hybrid loss function L, the error between the simulated image and the real image of the SAR target is calculated, the error is transferred back to the coordinates and texture information of the triangle face vertices, the coordinate matrix and the scattering value are adjusted, and the optimization adjustment of the target three-dimensional reconstruction network model is achieved.
[0088] (6) Re-inputting the two-dimensional SAR target image into the optimized target three-dimensional reconstruction network model to obtain the final target three-dimensional model;
[0089] (7) Use the SBR method to render the final target three-dimensional model; according to the required viewing angle, obtain the observation data of the final target three-dimensional model at the corresponding angle and generate the SAR target simulation image of the required viewing angle.
[0090] This method constructs a target 3D reconstruction network model based on a convolutional neural network. The encoder extracts 2D SAR image features, and the decoder obtains the coordinates of triangular facet vertices through feature transformation to generate a predicted target 3D model. The SBR algorithm is used to render the target 3D model to obtain a simulated SAR target image of any viewpoint and a simulated SAR target image consistent with the viewpoint of the training image. A hybrid objective function is constructed to calculate the error between the simulated SAR target image consistent with the viewpoint of the training image and the real image, and the parameters of the target 3D model and the reconstruction network model are optimized through backpropagation and gradient descent strategies. The present invention can generate SAR target sample data of any viewpoint without constructing a high-precision target electromagnetic scattering model or preparing a dataset containing a large number of diverse SAR target samples. This method can effectively solve the problems of difficult acquisition of SAR target data and insufficient multi-viewpoint sample data.
[0091] In summary, this invention can generate simulated SAR target samples from any viewing angle quickly and with high quality, demonstrating strong potential for practical applications. This method eliminates the need to construct a high-precision target electromagnetic scattering model or prepare a dataset containing a large number of diverse SAR target samples. It can generate SAR target sample data from any viewing angle, effectively addressing the challenges of acquiring SAR target data and the lack of multi-viewpoint sample data.
[0092] Finally, it should be noted that the above embodiments are merely illustrative of the spirit of the present invention. Persons skilled in the art may make various modifications, additions, or substitute similar methods to the described embodiments without departing from the spirit of the present invention or exceeding the scope defined by the appended claims.
Claims
1. A method for generating SAR target simulation samples at any viewing angle, characterized in that: The following steps are involved: Step 1: construct a target 3D reconstruction network model based on a convolutional neural network. The target 3D reconstruction network model includes two parts: an encoder and a decoder. The encoder is used to extract features of the input 2D SAR image, and the decoder obtains the coordinates of the triangular surface vertices through feature transformation to generate a predicted target 3D model. Step 2: training the target 3D reconstruction network model to obtain a trained target 3D reconstruction network model; Step 3: Input the two-dimensional SAR target image for three-dimensional reconstruction into the trained target three-dimensional reconstruction network model to obtain a reconstructed target three-dimensional model; Step 4: Render the target 3D model obtained in step 3 using the SBR method; obtain observation data of the target 3D model at the corresponding angle according to the viewing angle of each 2D SAR target image, and generate a SAR target simulation image consistent with the viewing angle of the 2D SAR target image; Step 5: construct an objective function, calculate the error between the SAR target simulation image and the corresponding two-dimensional SAR target image, and optimize the parameters of the trained target three-dimensional reconstruction network model through back propagation and gradient descent methods; Step 6: re-input the two-dimensional SAR target image in step 3 into the optimized target three-dimensional reconstruction network model to obtain the final target three-dimensional model; Step 7: Render the final target 3D model using the SBR method; obtain observation data of the final target 3D model at the corresponding angle according to the required viewing angle, and generate a SAR target simulation image at the required viewing angle.
2. The method for generating SAR target simulation samples at any viewing angle according to claim 1, wherein: The two-dimensional SAR image input to the target three-dimensional reconstruction network model contains three channels and at least four different perspectives, and the angles of these four perspectives are distributed in four different quadrants.
3. The method for generating SAR target simulation samples at any viewing angle according to claim 2, wherein: The target 3D reconstruction network model works as follows: The encoder uses three convolutional layers to extract the two-dimensional SAR image features of each channel. After normalization, it uses the ReLU activation function to perform nonlinear transformation to generate one-dimensional feature information, and then further extracts features through three fully connected layers. The target three-dimensional model is divided into finite surface elements using a triangular mesh, and the triangular surface elements obtained by the division are combined into the target three-dimensional model; The decoder uses two separate fully connected layers to extract the vertex coordinates P of each triangle face from the features output by the encoder. i,j ,i=1,2,3...,N,j=1,2,3;where N is the number of triangle faces, P i,j Represents the coordinates of the jth vertex of the i-th triangle face; based on the coordinates of each vertex P i,j , generating a predicted target 3D model.
4. The method for generating SAR target simulation samples at any viewing angle according to claim 1, wherein: The specific method of step 4 is: Step 401: Project the direction opposite to the incident wave direction according to the geometric shape of the target three-dimensional model to generate an isophase surface, and divide the isophase surface to obtain an initial ray tube; Step 402 , using the initial ray tube, performing ray tracing on the target three-dimensional model; Step 403: Integrate the surface current of the triangular surface element intersecting the ray tube according to the tracking result to calculate the scattered field; Step 404: Perform vector accumulation on all scattered fields obtained in this tracking to obtain the total scattered field of the target 3D model in the current incident direction, and calculate the radar cross section corresponding to the target 3D model in the current incident wave direction; Step 405: adjust the direction of the incident wave, update the intersection information between the ray tube and the triangular facets in the target 3D model, as well as the amplitude, phase, and polarization information of the ray tube, and perform ray tracing on the target 3D model again; Step 406, repeating steps 403 to 405 until there is no intersection between the ray tube and the target three-dimensional model; At this point, the radar cross-section data of the target 3D model at multiple observation angles are obtained; Step 407 : Based on the viewing angle of each two-dimensional SAR target image and the radar cross section data of the target three-dimensional model at the corresponding observation angle, a SAR target simulation image having the same viewing angle as the two-dimensional SAR target image is generated based on the CS linear frequency modulation scaling imaging method.
5. The method for generating SAR target simulation samples at any viewing angle according to claim 1, wherein: The specific method of step 5 is: Step 501: construct the objective function L: L=α·L M +(1-a)·G σ ·L s ; Among them, α is the weight, G σ is the Gaussian distribution parameter, L M is the MS-SSIM loss function, which is used to characterize the differences in detail features and structural contours between the two-dimensional SAR target image x and the SAR target simulation image y. s is the scattering image loss function, which is used to characterize the difference in brightness and grayscale values of corresponding pixels on the two-dimensional SAR target image x and the SAR target simulation image y; L M Expressed as: L s Expressed as: Among them, x i ,y i ~P data (x, y) represents the corresponding coordinate pixel in the two-dimensional SAR target image x and the SAR target simulation image y; represents the average value after accumulation of the entire image; MS_SSIM(·) is the multi-scale structural consistency loss function; In step 502, the error between the simulated SAR target image y and the two-dimensional SAR target image x is calculated based on the objective function L, and the error is transferred back to the vertex coordinates and texture information of the triangle facets. The parameters of the trained target 3D reconstruction network model are adjusted through back propagation and gradient descent methods to achieve optimization adjustment of the trained target 3D reconstruction network model.