A neural radiated field (SAR) image simulation method based on noise self-perception

Abstract

The application discloses a neural radiation field SAR image simulation method based on noise self-perception, and comprises the following steps: S1, constructing an image rendering network model; S2, training the image rendering network model to obtain an optimal image rendering network model; and S3, inputting a required SAR direction angle into the optimal image rendering network model for three-dimensional rendering to obtain a network-simulated SAR image. The original two-dimensional image generation problem is converted into a three-dimensional rendering problem, and the SAR image under any direction angle can be generally obtained. The result generated by the neural radiation field SAR image simulation method based on noise self-perception has high similarity with a real SAR image, and the simulation effect is good.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a neural radiance field SAR image simulation method based on noise self-perception. BACKGROUND

[0002] Synthetic Aperture Radar (SAR) technology has been applied more and more widely in recent years. It obtains ground information by transmitting and receiving radar waves, has the characteristics of all-weather and all-day operation, and the radar waves have a certain penetration ability to ground vegetation. SAR images have been widely used in military and national economic fields, including aircraft target detection, land monitoring, navigation, natural disaster monitoring, etc.

[0003] The existing SAR image simulation method usually generates a two-dimensional image, and the simulation result is quite different from the real SAR image, which is difficult to meet the actual use demand of people. SUMMARY

[0004] The purpose of the present application is to provide a neural radiance field SAR image simulation method based on noise self-perception.

[0005] To achieve the above purpose, the present application adopts the following technical scheme:

[0006] A neural radiance field SAR image simulation method based on noise self-perception, comprising the following steps:

[0007] S1, constructing an image rendering network model;

[0008] S2, training the image rendering network model to obtain the best image rendering network model;

[0009] S3, inputting the required SAR direction angle into the best image rendering network model for three-dimensional rendering to obtain the network simulated SAR image.

[0010] Preferably, the image rendering network model constructed in step S1 is: a deep neural network f θ : Predicting the color value c=(r,g,b), transparency σ and noise degree β of a certain point (x,y,z) in three-dimensional space under a given direction angle (θ,φ).

[0011] Preferably, the specific process of step S2 is:

[0012] S21, dividing the training data set and determining the network parameters;

[0013] S22, sample the sample point list according to uniform distribution, and calculate the sample point list in the Ground Truth plane rendering result;

[0014] S23, sample the difficult sample point list, and calculate the difficult sample point list in the Ground Truth plane rendering result;

[0015] S24, calculate the loss function and update the network parameters of the image rendering network model;

[0016] S25, judge whether the loss function converges, if yes, save it as the best image rendering network model; if not, repeat steps S22-S24 until the loss function converges.

[0017] Preferably, the network parameter determination process in step S21 is: according to the optical principle, the cumulative effect of the pixel value in the 2D imaging of the synthetic aperture radar is calculated by the following formula:

[0018]

[0019] Wherein, C(r) represents the color of the rendering result, r=o+td represents the ray with the starting point o and the direction d, t n and t f respectively represent the near end and the far end of the lens, T(t) represents the cumulative transparency attenuation at point t, σ(r(t)) and c(r(t),d) respectively represent the transparency and color of the network prediction for the point on the ray with a distance of t from the near end, and σ(r(s)) represents the transparency of the point on the ray with a distance of s from the near end, and is solved discretely using quadrature numerical integration method.

[0020] Preferably, the specific process of step S22 is:

[0021] S221, sample N points from the interval [t n , t f ] according to uniform distribution:

[0022]

[0023] S222, use the N discrete points to represent the integral variable on the ray, and estimate the rendering result by the following formula:

[0024]

[0025] Wherein, δ i =t i+1 -t i represents the distance between adjacent sampling points, σ i and c irespectively represent the network's prediction of the transparency and color value at sample point i;

[0026] S223, integrate the network's prediction of the noise on the ray to estimate the degree to which the pixel value is affected by the noise using the following formula:

[0027]

[0028] where β(r(s)) represents the predicted noise value at a distance s from the proximal end on the ray;

[0029] S224, use a fixed mapping to map each element of the 5d input vector to a high-dimensional vector, the mapping is as follows:

[0030] γ(p) = (sin(2 0 πp), cos(2 0 πp), …, sin(2 L-1 πp), cos(2 L-1 πp));

[0031] S225, change the number of networks from 1 to 2, namely the coarse network and the fine network, the coarse network is optimized on uniformly sampled points, and a weight for each sample point is calculated according to the following formula

[0032]

[0033] where the weight represents the attenuation coefficient of the sample point i affected by the transparency and the degree of influence of the noise in the accumulation process, the greater the attenuation or the influence of the noise, the lower the weight of the point.

[0034] Preferably, the specific process of step S23 is:

[0035] S231, obtain the normalized weight from S22 At this time represents the probability density function W of the importance of the sample point on the ray r;

[0036] S232, randomly sample Nc points on the ray from the probability density function W;

[0037] S233, map the Nc points to the high-dimensional space in the same way as step S224.

[0038] Preferably, the calculation process of the loss function in step S24 is: the mean square error of the prediction results of the coarse network and the fine network and the ground truth is used as the loss function:

[0039] ​

[0040] wherein, and respectively represent the prediction results of the coarse network and the fine network, C(r) represents the pixel value of the Ground Truth point corresponding to the ray r, β is the noise degree accumulated by the ray r, the greater the noise, the lower the weight of the point loss function, and β(r) represents the noise value accumulated by the ray.

[0041] After the above technical scheme is adopted, compared with the background art, the present application has the following advantages: the present application converts the original two-dimensional image generation problem into a three-dimensional rendering problem, and can generally obtain SAR images under any direction angle, the result generated by the neural radiance field SAR image simulation method based on noise self-perception has high similarity with the real SAR image, and the simulation effect is good. BRIEF DESCRIPTION OF DRAWINGS

[0042] Fig. 1 is a flowchart of the present application;

[0043] Fig. 2 is a training flowchart of the image rendering network model of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application is further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application, and are not used to limit the present application.

[0045] The present application mainly relates to a neural radiance field SAR image simulation method based on noise self-perception, which is described in detail below in combination with the drawings.

[0046] Please refer to Figs. 1-2 The present application discloses a neural radiance field SAR image simulation method based on noise self-perception, which comprises the following steps:

[0047] S1, constructing an image rendering network model;

[0048] The image rendering network model constructed in step S1 is: a deep neural network f θ : predicting the color value c=(r,g,b), the transparency σ and the noise degree β of a point (x,y,z) in a three-dimensional space under a given direction angle (θ,φ);

[0049] S2, training the image rendering network model to obtain the best image rendering network model;

[0050] The specific process of step S2 is:

[0051] S21, divide the training data set, and determine the network parameters;

[0052] The determination process of the network parameters in step S21 is: according to the optical principle, the cumulative effect of the pixel value in the 2D imaging of the synthetic aperture radar is calculated by the following formula:

[0053]

[0054] Wherein, C(r) represents the color of the rendering result, r=o+td represents the ray with the starting point o and the direction d, t n and t f respectively represent the near end and the far end of the lens, T(t) represents the cumulative transparency attenuation at point t, σ(r(t)) and c(r(t),d) respectively represent the transparency and color predicted by the network for the point on the ray with a distance of t from the near end, and σ(r(s)) represents the transparency of the point on the ray with a distance of s from the near end, and the quadrature numerical integration method is used for discrete solution;

[0055] S22, according to the uniform distribution, randomly sample a sample point column, and calculate the sample point column in the Ground Truth plane rendering result;

[0056] The specific process of step S22 is:

[0057] S221, from the interval [t n ,t f ], according to the uniform distribution, sample N points:

[0058]

[0059] S222, use the N discrete points to represent the integral variable on the ray, and estimate the rendering result by the following formula:

[0060]

[0061] Wherein, δ i =t i+1 -t i represents the distance between adjacent sampling points, σ i and c i respectively represent the transparency and color value predicted by the network for the sampling point i;

[0062] S223, integrate the noise predicted by the network on the ray by the following formula to estimate the degree of influence of the noise on the pixel value:

[0063]

[0064] where β(r(s)) represents the predicted noise value at a distance s from the proximal end on the ray;

[0065] S224, using a fixed mapping to map each element of the 5d input vector into a high-dimensional vector, the mapping is as follows:

[0066] γ(p) = (sin(2 0 πp), cos(2 0 πp), …, sin(2 L-1 πp), cos(2 L-1 πp));

[0067] S225, change the number of networks from 1 to 2, which are coarse network and fine network, the coarse network is optimized on uniformly sampled points, and a weight is calculated for each sampling point according to the following formula

[0068]

[0069] where the weight represents the attenuation coefficient of the sampling point i affected by the transparency in the accumulation process and the degree of influence by the noise, the greater the attenuation or the influence by the noise, the lower the weight of the point ;

[0070] S23, sample the difficult sample point list, and calculate the rendering result of the difficult sample point list on the Ground Truth plane;

[0071] The specific process of step S23 is:

[0072] S231, obtain the normalized weight from S22 At this time represents the probability density function W of the importance degree of the sampling point on the ray r;

[0073] S232, randomly sample Nc points on the ray from the probability density function W;

[0074] S233, map the Nc points to the high-dimensional space in the same way as step S224;

[0075] The calculation process of the loss function in step S24 is: the mean square error of the prediction results of the coarse network and the fine network and the Ground Truth is used as the loss function:

[0076]

[0077] where, and respectively represent the prediction results of the coarse network and the fine network, C(r) represents the pixel value of the Ground Truth point corresponding to the ray r, and β is the noise degree accumulated by the ray r, the greater the noise, the lower the weight of the loss function of the point, and β(r) represents the noise value accumulated by the ray;

[0078] S25, judge whether the loss function converges, if converges, save as the best image rendering network model, if not converges, repeat steps S22-S24 until the loss function converges;

[0079] S3, input the required SAR direction angle into the best image rendering network model for three-dimensional rendering to obtain a network simulated SAR image.

[0080] In order to further confirm the effect of the present application, 200 SAR pictures are collected, and the SAR pictures are used as test data, and the rendering results of the network are compared with the real SAR image one by one, and the average value of SSIM is calculated. Specifically, the neural radiation field SAR image simulation method based on noise self-perception is used, and 30%, 40%, 50% and 100% of the test data are used for testing respectively; the Baseline:DCGAN method is used, and 30% and 100% of the test data are used for testing respectively; the test results are shown in Table 1.

[0081] Table 1 rendering results of different schemes

[0082]

[0083] As shown in Table 1, the neural radiation field SAR image simulation method based on noise self-perception is better than the Baseline:DCGAN method in simulating SAR image, even if only 30% of the test data is used, the result generated by the neural radiation field SAR image simulation method based on noise self-perception still has high similarity with the real SAR image.

[0084] The above is only the preferred specific embodiment of the present application, but the protection scope of the present application is not limited to this, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for simulating neural radiation field SAR images based on noise self-perception, characterized in that, The method comprises the following steps: S1, constructing an image rendering network model; S2, training the image rendering network model to obtain an optimal image rendering network model; The specific process of step S2 is: S21, dividing the training data set and determining network parameters; S22, randomly sampling a sample point column according to a uniform distribution and calculating the sample point column in a Ground Truth plane rendering result; The specific process of step S22 is: S221、From interval [t n ,t f ], sample N points according to uniform distribution: S222, using the N discrete points to represent the integral variable on the ray, and using the following formula to estimate the rendering result: where δ i = t i+1 -t i denotes the distance between adjacent samples, σ i and c i denote the network's prediction of the transparency and color value of sample i, respectively; S223, integrating the noise predicted by the network on the ray to estimate the degree of influence of the pixel value on the noise using the following formula: Where β(r(s)) represents the predicted noise value at a distance of s from the proximal end on the ray; S224, using a fixed mapping to map each element of the 5d input vector into a high-dimensional vector, as follows: y(p) = (sin(2 0 πp), cos(2 0 πp),..., sin(2 L-1 πp), cos(2 L-1 πp)) ; S225, change the number of networks from 1 to 2, coarse network and fine network, respectively, the coarse network is optimized on the uniformly sampled points, and a weight is calculated for each sampled point according to the following formula wherein the weight represents the degree of influence of the sampling point i in the accumulation process by the transparency and the degree of influence by the noise, the greater the attenuation or the influence of the noise, the lower the weight of the point ; S23, sampling a difficult sample point column and calculating the difficult sample point column in the Ground Truth plane rendering result; S24, calculating the loss function and updating the network parameters of the image rendering network model; The calculation process of the loss function in step S24 is: using the mean square error of the prediction results of the coarse network and the fine network and the Ground Truth as the loss function: wherein, and respectively represent the prediction results of the coarse network and the fine network, C(r) represents the pixel value of the Ground Truth point corresponding to the ray r, β is the noise degree accumulated by the ray r, the greater the noise, the lower the weight of the loss function of this point, and β(r) represents the noise value accumulated by the ray. S25, determining whether the loss function converges, if the loss function converges, saving it as the optimal image rendering network model; if the loss function does not converge, repeating steps S22-S24 until the loss function converges; S3, inputting the required SAR direction angle into the optimal image rendering network model for three-dimensional rendering to obtain a network-simulated SAR image.

2. The method of claim 1, wherein the method is based on noise self-awareness. The image rendering network model constructed in step S1 is a deep neural network with a parameter θ Predict the color value c = (R, g, b), transparency σ, and noise level β of a point (x, y, z) in three-dimensional space at a given direction angle (α, φ).

3. The noise self-aware based neural beamforming (NeuralBf) synthetic aperture radar (SAR) image simulation method of claim 2, wherein, The determination process of the network parameters in step S21 is: according to the optical principle, the cumulative effect of the pixel value in the 2D imaging of the synthetic aperture radar is calculated by the following formula: where C(r) denotes the color of the rendering result, r = o + td denotes a ray with origin o and direction d, t n and t f denote the near and far ends of the lens, respectively, T(t) denotes the accumulated transparency attenuation at point t, σ(r(t)) and c(r(t), d) denote the network's prediction of the transparency and color, respectively, of the point on the ray that is t units away from the near end, and σ(r(s)) denotes the transparency of the point on the ray that is s units away from the near end, and is solved discretely using a quadrature numerical integration method.

4. The noise self-aware based neural beamforming (NeuralBf) SAR image simulation method of claim 3, wherein, The specific process of step S23 is: S231, normalizing the weights from S22 At this time W represents a probability density function of the importance of the sampling points on the ray r; S232, randomly sampling Nc points on the ray from the probability density function W; S233, mapping the Nc points to the high-dimensional space in the same way as step S224.

Images