Generative single-frequency millimeter wave self-focusing near-field imaging method based on diffusion model
Through the generative single-frequency millimeter wave self-focusing near-field imaging method based on diffusion model, the spatial convolutional neural network and conditional diffusion model are used to solve the problem that traditional single-frequency millimeter wave imaging method cannot generate high-quality focused images, and the problem is realized in High-quality imaging in complex object scenes.
Patent Information
- Application Number
- CN202411128574.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2024-11-22
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional single-frequency millimeter wave imaging methods cannot generate high-quality focused images, especially in complex object scenes, and the existing self-focusing methods require long-term iterative calculations and complex radar system design.
Using a generative single-frequency millimeter wave self-focusing near-field imaging method based on diffusion model, the spatial structure of three-dimensional out-of-focus images is extracted through spatial convolutional neural network, and input it as conditions into the conditional diffusion model to gradually reconstruct high-quality images .
Under the condition of using only single-frequency millimeter wave echo data, self-focus imaging of high-quality images is achieved, reducing the difficulty of fitting the neural network and improving the imaging capabilities in complex object scenes.
Smart Images

Figure CN119001650A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of millimeter wave human body security inspection, and in particular to a generative single-frequency millimeter wave self-focusing near-field imaging method based on a diffusion model. Background Art
[0002] Millimeter waves have appropriate penetrating power and can detect hidden objects through common materials such as clothing and packages without posing a threat to human health. Active millimeter wave security inspection devices based on millimeter wave radar can achieve millimeter-level resolution in azimuth, which can fully meet the requirements of human security inspection for safety, high precision and contactlessness, and have been piloted in some key places.
[0003] The key technology of active millimeter-wave security inspection devices is the millimeter-wave near-field imaging method. The traditional near-field imaging method is based on the Nyquist sampling theory. It completes the inversion process from echo data to the preset spatial distribution through matched filtering in the spatial domain or spatial frequency domain. Under the requirements of spatial sampling and imaging bandwidth, three-dimensional imaging with high azimuth and range resolution can be achieved. However, the stringent requirements of the traditional near-field imaging method on imaging conditions have brought difficulties to the implementation of the imaging system. In order to obtain higher range resolution, frequency-modulated continuous waves are often used as the waveform system of imaging radars in radar design, which will undoubtedly greatly increase the complexity of the radar system and bring challenges to the specific implementation process. In addition, with the improvement of imaging conditions, the amount of three-dimensional echo data collected by the radar will increase, which will have an adverse effect on real-time data transmission and subsequent imaging processes.
[0004] Using single-frequency echo data imaging can greatly reduce the dimension of echo data while getting rid of the constraints of the broadband radar system. However, due to sampling conditions, directly using traditional millimeter-wave near-field imaging methods cannot obtain good imaging results, and the existing self-focusing single-frequency millimeter-wave imaging methods perform poorly when dealing with target objects with complex surfaces or spatial structures such as the human body.
[0005] In near-field single-frequency millimeter-wave imaging, traditional near-field imaging methods can only obtain imaging results that lack distance resolution, also known as out-of-focus images. Image self-focusing uses the property that the preset forms a special spatial structure in the out-of-focus image to complete the image focusing process and obtain high-quality imaging results. A good image self-focusing method can effectively filter out the diffuse sidelobes around the preset in the out-of-focus image and restore the preset position. The current single-frequency image self-focusing methods can be divided into focal plane selection methods and preset position estimation methods. Among them, the focal plane selection method: first calculate the focus degree of the image on all reference distance planes, draw the focus degree-distance curve, and then set a threshold or manually select the distance corresponding to the focus degree in the curve that is obviously prominent as the focal plane distance, and retain the imaging results at the corresponding distance; the preset position estimation method is divided into two types: one is to first estimate the preset spatial position, image the hypothetical preset, and then use the difference between the imaging result and the actual measurement as the optimization criterion to iteratively update the preset position; the other adopts a sliding window strategy to determine the possible distance of the preset in the window by measuring the focus degree at all reference distances in each sliding window. These traditional self-focusing methods perform well in simple preset scenarios, but often fail when faced with presets with complex spatial structures, and the preset position estimation method requires long iterative calculations.
[0006] Furthermore, traditional near-field imaging methods require imaging on multiple reference distance planes, and the resulting imaging result is a three-dimensional image containing two azimuth dimensions and one distance dimension. The feature extraction neural network that usually uses a two-dimensional convolutional layer only has three dimensions: image length, width, and channel. If the distance dimension is directly input as the channel dimension, the spatial structure of the image may be lost. In addition, the fitting ability of a single neural network structure is often limited, and the choice of training loss function will affect the final imaging quality. Summary of the invention
[0007] The present invention provides a generative single-frequency millimeter-wave self-focusing near-field imaging method based on a diffusion model to solve the problems that the traditional single-frequency millimeter-wave imaging method cannot generate high-quality focused images and the single-frequency imaging method based on self-focusing has insufficient imaging ability in complex object scenes.
[0008] The first aspect of the present invention provides a generative single-frequency millimeter-wave self-focusing near-field imaging method based on a diffusion model, comprising the following steps: using a radar device to acquire single-frequency original echo data; imaging the single-frequency original echo data to obtain a single-frequency imaging defocus result; inputting the single-frequency imaging defocus result into a pre-constructed spatial convolutional neural network to obtain a network coding result; embedding the network coding result into a pre-trained denoising diffusion model to obtain a conditional diffusion model; using the conditional diffusion model to gradually reconstruct the residual value between the network coding result and a preset ideal broadband imaging result to obtain a diffusion model prediction result, and adding the diffusion model prediction result to the network coding result to obtain a final single-frequency self-focusing imaging result.
[0009] Optionally, imaging the single-frequency raw echo data to obtain a single-frequency imaging defocus result includes:
[0010] Based on the phase shift method in the millimeter wave near-field imaging method, the single-frequency raw echo data is imaged on several equally spaced reference planes in a pre-selected imaging space to obtain the single-frequency imaging defocus result.
[0011] Optionally, the solution formula for the single-frequency imaging defocus result is:
[0012]
[0013] in, is the inverse transform of the two-dimensional Fourier transform, FT 2D is the two-dimensional Fourier transform, e is the natural logarithm, is the imaginary unit, k z′ is k in the spatial wave number domain x′ With k y′ The spherical function, z is the plane represented by multiple reference distances, σ s (x, y, z) is the single-frequency imaging defocus result of the scattering point coordinates, s(x′,′, 0 ) is the spatial wave number received by the plane array transceiver element at the coordinate (x′,′) is k 0 Single frequency raw echo data.
[0014] Optionally, inputting the single-frequency imaging defocus result into a pre-constructed spatial convolutional neural network to obtain a network coding result includes:
[0015] Collecting broadband echo data using a broadband imaging device, and extracting single frequency echo data from the broadband echo data;
[0016] Imaging the broadband echo data and the single-frequency echo data respectively to obtain a broadband imaging focusing result and a single-frequency imaging defocusing result;
[0017] Converting the broadband imaging focusing result into a maximum projection result as a training label;
[0018] Obtaining the amplitude value of the single-frequency imaging defocus result as training data;
[0019] Pre-training the pre-constructed spatial convolutional neural network using the training labels and the training data, performing supervised training with mean square error as loss, and obtaining a trained spatial convolutional neural network;
[0020] The single-frequency imaging defocus result is input into the trained spatial convolutional neural network for training to obtain the network coding result.
[0021] Optionally, the solution formula for the network coding result is:
[0022] x c = θ {{(x′,′, 0 )}}
[0023] Among them, x c is the network coding result, F θ {·} is the trained spatial convolutional neural network, PSM{·} is the near-field imaging method, s(x′,′, 0 ) is the spatial wave number received by the plane array transceiver element at the coordinate (x′,′) is k 0 Single frequency raw echo data.
[0024] Optionally, the solution formula for the diffusion model prediction result is:
[0025]
[0026] Among them, x 0 (x 1 ,x c ,1) is the prediction result of the diffusion model, x 1 is the input of the preset diffusion model at time step 1, β 1 Add a hyperparameter, α, for time-correlated noise at time step 1 1 , is the conditional diffusion model hyperparameter when the time step is 1, ∈ θ (x 1 ,x c ,1) is the conditional noise predictor in the conditional diffusion model when the time step is 1.
[0027] The second aspect of the present invention provides a generative single-frequency millimeter-wave self-focusing near-field imaging device based on a diffusion model, including: an acquisition module, used to acquire single-frequency original echo data using radar equipment. An imaging module, used to image the single-frequency original echo data to obtain a single-frequency imaging defocus result. An input module, used to input the single-frequency imaging defocus result into a pre-constructed spatial convolutional neural network to obtain a network coding result. An embedding module, used to embed the network coding result into a pre-trained denoising diffusion model to obtain a conditional diffusion model. A prediction module, used to use the conditional diffusion model to gradually reconstruct the residual value between the network coding result and the preset ideal broadband imaging result to obtain a diffusion model prediction result, and add the diffusion model prediction result to the network coding result to obtain the final single-frequency self-focusing imaging result.
[0028] A third aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the generative single-frequency millimeter-wave self-focusing near-field imaging method based on the diffusion model as described in the above embodiment.
[0029] A fourth aspect of the present invention provides a computer program product, which, when executed by a processor, implements the above-mentioned generative single-frequency millimeter-wave self-focusing near-field imaging method based on a diffusion model.
[0030] A fifth aspect of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned generative single-frequency millimeter-wave self-focusing near-field imaging method based on a diffusion model.
[0031] The generative single-frequency millimeter-wave self-focusing near-field imaging method based on the diffusion model proposed in the embodiment of the present invention can reconstruct high-quality image results by self-focusing under the condition of using only single-frequency millimeter-wave echo data; the echo data is converted into the imaging domain by using the traditional near-field imaging method, and then the focal plane selection and target position estimation process are replaced by a neural network with strong fitting ability to realize self-focusing imaging; a spatial feature extraction neural network using three-dimensional convolution is designed, and the three-dimensional convolution is combined with the feature extraction structure so that the network has four dimensions of image length, width, distance, and channel, which effectively extracts the spatial structure existing in the three-dimensional defocused image and greatly reduces the fitting difficulty of the neural network; combining the advantages of the generative model, a single-frequency millimeter-wave image reconstruction framework based on the conditional diffusion model is proposed, and the feature map extracted by the spatial convolution network in the defocused image is used as a condition to guide the iterative process of high-quality image restoration.
[0032] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0034] Figure 1 A flow chart of a generative single-frequency millimeter-wave self-focusing near-field imaging method based on a diffusion model provided by an embodiment of the present invention;
[0035] Figure 2 A schematic diagram of a specific implementation of a generative single-frequency millimeter-wave self-focusing near-field imaging method based on a diffusion model provided in an embodiment of the present invention;
[0036] Figure 3 A schematic diagram of the structure of a pre-built spatial convolutional neural network provided in an embodiment of the present invention;
[0037] Figure 4 A schematic diagram comparing the imaging effects of the front face, the front left arm, and the front left hand of a human body using the conventional imaging method and the imaging method of the present invention respectively provided in an embodiment of the present invention, wherein (a1), (a2), (a3), and (a4) are all conventional imaging effects, and (b1), (b2), (b3), and (b4) are all imaging effects of the present invention;
[0038] Figure 5 A schematic block diagram of a generative single-frequency millimeter-wave self-focusing near-field imaging device based on a diffusion model provided by an embodiment of the present invention;
[0039] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0040] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.
[0041] The following describes a generative single-frequency millimeter-wave self-focusing near-field imaging method based on a diffusion model of an embodiment of the present invention with reference to the accompanying drawings. In view of the problems mentioned in the above background technology center that the traditional single-frequency millimeter-wave imaging method cannot generate high-quality focused images, and the single-frequency imaging method based on self-focusing has insufficient imaging capabilities in complex object scenes, the present invention provides a generative single-frequency millimeter-wave self-focusing near-field imaging method based on a diffusion model. In this method, the spatial structure of the traditional near-field imaging result is extracted through a spatial convolutional neural network, and the extracted feature map is used as a condition to guide the diffusion model generation process, and finally the reconstruction process from single-frequency echo data to a clear image is completed. As a result, the traditional single-frequency millimeter-wave imaging method cannot generate high-quality focused images, and the single-frequency imaging method based on self-focusing has insufficient imaging capabilities in complex object scenes.
[0042] In order to facilitate those skilled in the art to understand the generative single-frequency millimeter-wave self-focusing near-field imaging method based on the diffusion model of the present application, the parameter symbols involved in the present invention are explained through the following table.
[0043]
[0044]
[0045] Specifically, Figure 1 A schematic flow chart of a generative single-frequency millimeter-wave self-focusing near-field imaging method based on a diffusion model provided in an embodiment of the present invention.
[0046] like Figure 1 As shown, the generative single-frequency millimeter-wave self-focusing near-field imaging method based on the diffusion model includes the following steps:
[0047] In step S101, single-frequency raw echo data is acquired using a radar device.
[0048] Specifically, the preset millimeter wave human body security detector is run to correct the inconsistency of amplitude and delay between signals of different channels, and the corresponding spatial wave number k is obtained. 0 The single-frequency raw echo data s(x′,′, 0 ).
[0049] In step S102, single-frequency raw echo data is imaged to obtain a single-frequency imaging defocus result.
[0050] In some embodiments, imaging the single-frequency raw echo data to obtain a single-frequency imaging defocus result includes:
[0051] Based on the phase shift method in the millimeter-wave near-field imaging method, the single-frequency raw echo data is imaged on several equally spaced reference planes in a pre-selected imaging space to obtain the single-frequency imaging defocus result.
[0052] Specifically, the phase shift PSM method in the millimeter wave near-field imaging method is used to obtain the On the plane represented by 0 The single-frequency raw echo data s(x′,′, 0 ) to obtain the single-frequency imaging defocus result σ s (x,y,z), which is represented as follows:
[0053] The solution formula for the single-frequency imaging defocus result is:
[0054]
[0055] in, is the inverse transform of the two-dimensional Fourier transform, FT 2D is the two-dimensional Fourier transform, e is the natural logarithm, is the imaginary unit, k z′ is k in the spatial wave number domain x′ With k y′ The spherical function is expressed as z is the plane represented by multiple reference distances, σ s (x, y, z) is the single-frequency imaging defocus result of the scattering point coordinates, s(x′,′, 0 ) is the spatial wave number received by the plane array transceiver element at the coordinate (x′,′) is k 0 Single frequency raw echo data.
[0056] In step S103, the single-frequency imaging defocus result is input into a pre-constructed spatial convolutional neural network to obtain a network coding result.
[0057] In some embodiments, the single-frequency imaging defocus result is input into a pre-built spatial convolutional neural network to obtain a network coding result, including:
[0058] During the training process, broadband imaging equipment is used to collect broadband echo data s′(x′,′,), and single-frequency echo data s′(x′,′, 0 );
[0059] The broadband echo data and the single frequency echo data are imaged respectively to obtain the broadband imaging focusing result σ f ′(x,y,z) and single-frequency imaging defocus result σ s ′(x,y,z);
[0060] Convert the broadband imaging focusing result into the maximum projection result as the training label x s ;
[0061] Get the amplitude value of the single-frequency imaging defocus result as training data x f ;
[0062] The pre-built spatial convolutional neural network is pre-trained using training labels and training data, and supervised training is performed using mean square error as loss to obtain the trained spatial convolutional neural network F. θ ;
[0063] The single-frequency imaging defocus result is input into the trained spatial convolutional neural network for training to obtain the network coding result x c .
[0064] Specifically, Figure 2 and 3 As shown in Figure 2, during the training process, broadband imaging equipment is used to collect broadband echo data s′(x′,′,), and single-frequency echo data s′(x′,′, 0 ), respectively imaging to form focusing results and defocusing results, where the solution formula for broadband imaging focusing results is:
[0065]
[0066] Among them, σ f ′(x,y,z) is the broadband imaging focusing result of the scattering point coordinates, PSM{·} is the near-field imaging method, s′(x′,′,) is the original echo data of k spatial wave numbers received by the coordinates of the planar array transceiver element, is the inverse transform of the two-dimensional Fourier transform, FT 2D is the two-dimensional Fourier transform, e is the natural logarithm, is the imaginary unit, k z′ is k in the spatial wave number domain x′ With k y′ The spherical function is expressed as z is the plane represented by multiple reference distances.
[0067] In order to facilitate the calculation without losing the spatial features of the input 3D defocused image, the amplitude value of the single-frequency imaging defocused result in complex form is selected to establish the training data x of the spatial convolutional network. s , and while simplifying the input data, the maximum projection result of the broadband imaging focusing result is used as the training label x f . Network training data x s With training data x s It is expressed as follows, where MIP{·} represents the maximum projection operation of converting a three-dimensional image to a two-dimensional image:
[0068] x s =|σ s ′(x,y,z)|=SM{s′(x′,′, 0 )}
[0069] x f =MIP{σ f ′(x,y,z)}
[0070] When using the input and label pairs (x s ,x f ) After the spatial convolutional network is supervised and trained with mean square error as loss, the single frequency echo data s(x′,′, 0 ), after being imaged by the PSM method, it is input into the pre-trained network to obtain a two-dimensional feature map x c (i.e., the network coding result), is expressed as follows:
[0071] x c = θ {{(x′,′, 0 )}}
[0072] Among them, x c is the network coding result, F θ {·} is the trained spatial convolutional neural network, PsM{·} is the near-field imaging method, s(x′,′, 0 ) is the spatial wave number received by the plane array transceiver element at the coordinate (x′,′) is k 0 Single frequency raw echo data.
[0073] It should be noted that each single-frequency input has a unique corresponding two-dimensional feature map, which can be used repeatedly in subsequent tasks once generated.
[0074] In step S104, the network coding result is embedded into a preset diffusion model as conditional information to obtain a conditional diffusion model.
[0075] Specifically, Figure 3 As shown, the two-dimensional feature map x obtained by spatial convolutional network encoding c The variables transmitted between each time step of the preset diffusion model remain the same in spatial size and number of channels. Therefore, the network coding result can be simply embedded into the preset diffusion model as a condition by adding the matrix element by element to achieve fusion, and the conditional diffusion model is obtained, which is expressed as follows:
[0076]
[0077] in, represents the element-by-element addition of the matrix, x tis the input of the preset diffusion model at time step t. TIN{·} represents the preprocessing of the input, which consists of a two-dimensional convolutional layer with fixed parameters and a Mish activation function.
[0078] In addition, the conditional diffusion model also accepts the specific number of time steps at each time step as the system inherent information input. You can refer to the Transformer network structure and use the sinusoidal position encoding SIN{·} to encode the independent specific time step values into a matrix. The matrix addition method is also used to embed the time code:
[0079]
[0080] The network coding and time step coding results also provide the conditional information required by the diffusion model.
[0081] In step S105, the conditional diffusion model is used to gradually reconstruct the residual value between the network coding result and the preset ideal broadband imaging result to obtain the diffusion model prediction result, and the diffusion model prediction result is added to the network coding result to obtain the final single-frequency self-focusing imaging result.
[0082] Specifically, the core and trainable part of the conditional diffusion model is the conditional noise predictor ∈ θ (x t ,x c ,t), the loss function used by this predictor during training As shown below:
[0083]
[0084] in, Represents the expectation of calculation under the current input conditions, ∥·∥ 2 Represents the two-norm of the calculation matrix, ∈ is the standard Gaussian noise sample added at the current time step.
[0085] After completing the embedding representation in step S104, the residual value between the encoding result and the broadband imaging result is gradually reconstructed. In the pre-trained diffusion model with a total time step of T, the number of time steps and conditional information can be combined to predict the amount of noise that needs to be subtracted from the output of the current time step through the pre-trained noise predictor, and the residual result x of each time step is sequentially predicted. T ,…, 0 The prediction process for each time step is expressed as follows:
[0086]
[0087] For the last time step closest to the clear image, the prediction result does not need to add additional Gaussian noise, which is expressed as follows:
[0088]
[0089] Among them, x 0 (x 1 ,x c ,1) is the prediction result of the diffusion model, x 1 is the input of the preset diffusion model at time step 1, β 1 Add a hyperparameter for time-correlated noise at time step 1, which needs to be manually defined before training or prediction, α 1 , is the conditional diffusion model hyperparameter when the time step is 1, α t =1-β t , is a hyperparameter derived parameter that controls the mixing ratio of the prediction noise and the current time step input. z is a sampling result of the standard Gaussian noise step, ∈ θ (x 1 ,x c ,1) is the conditional noise predictor in the conditional diffusion model when the time step is 1.
[0090] Specifically, Figure 3 As shown in the figure, since the residual prediction strategy is adopted to accelerate the convergence of network training, the network prediction result is a high-quality image and the network coding result x c The difference between them is obtained by iterating the network after T time steps. 0 After that, we need to compare it with the network coding result x c Add element by element to get the final high-quality image result x h (i.e. the final imaging result):
[0091]
[0092] It should be noted that this is the last step in the network inference process and can be omitted during the network training process.
[0093] Therefore, if Figure 4 As shown in the figure, the imaging effects of the front of the human body, the front left arm and the front left hand are respectively obtained by using the traditional near-field imaging method and the imaging method of the present invention. It can be seen that the generative single-frequency millimeter-wave self-focusing near-field imaging method based on the diffusion model proposed in the embodiment of the present invention can easily cope with objects with complex surfaces or spatial structures such as the human body, while reducing the fitting difficulty of the neural network. It can reconstruct high-quality image results that are almost the same as those of traditional near-field imaging by self-focusing under the condition of using only single-frequency millimeter-wave echo data.
[0094] In summary, the generative single-frequency millimeter-wave self-focusing near-field imaging method based on the diffusion model proposed in the embodiment of the present invention has the following beneficial effects:
[0095] (1) Different from the common neural network method that directly maps the input from the echo data domain to the imaging domain, the embodiment of the present invention uses the traditional millimeter wave near-field imaging method to reconstruct the single-frequency raw echo data into a three-dimensional defocused image, which reduces the difficulty of neural network fitting. It also uses three-dimensional convolution to extract the spatial structure in the defocused image, and inputs these spatial structures as conditions into the diffusion model to iteratively reconstruct a high-quality millimeter wave image.
[0096] (2) A spatial feature extraction network using three-dimensional convolution is designed. This network takes into account the spatial structure of the target point spread function in the three-dimensional defocused image. The three-dimensional convolution is used to replace the original two-dimensional convolution in the feature extraction network, which can effectively extract the spatial structure features in the three-dimensional defocused image and map the extracted features into a feature map with a smaller dimension, reducing the number of parameters of the overall network.
[0097] (3) In order to address the problems of insufficient fitting ability of a single neural network and difficulty in selecting a loss function, an embodiment of the present invention embeds a three-dimensional spatial feature extraction network as a conditional encoder into a conditional diffusion model, guides the iterative reconstruction process of the diffusion model at multiple time steps, and ultimately recovers a high-quality focused image from single-frequency echo data.
[0098] Next, a generative single-frequency millimeter-wave self-focusing near-field imaging device based on a diffusion model proposed in an embodiment of the present invention is described with reference to the accompanying drawings.
[0099] Figure 5 It is a block diagram of a generative single-frequency millimeter-wave self-focusing near-field imaging device based on a diffusion model according to an embodiment of the present invention.
[0100] like Figure 5 As shown, the generative single-frequency millimeter-wave self-focusing near-field imaging device 50 based on the diffusion model includes: an acquisition module 501, an imaging module 502, an input module 503, an embedding module 504 and a prediction module 505.
[0101] Among them, the acquisition module 501 is used to acquire single-frequency original echo data using radar equipment. The imaging module 502 is used to image the single-frequency original echo data to obtain a single-frequency imaging defocus result. The training module 503 is used to input the single-frequency imaging defocus result into a pre-constructed spatial convolutional neural network to obtain a network coding result. The embedding module 504 is used to embed the network coding result into a pre-trained noise reduction diffusion model to obtain a conditional diffusion model. The prediction module 505 is used to gradually reconstruct the residual value between the network coding result and the preset ideal broadband imaging result using the conditional diffusion model to obtain the diffusion model prediction result, and add the diffusion model prediction result to the network coding result to obtain the final single-frequency self-focusing imaging result.
[0102] In some embodiments, imaging module 502 includes:
[0103] Based on the phase shift method in the millimeter-wave near-field imaging method, the single-frequency raw echo data is imaged on several equally spaced reference planes in a pre-selected imaging space to obtain the single-frequency imaging defocus result.
[0104] Among them, the solution formula for the single-frequency imaging defocus result is:
[0105]
[0106] in, is the inverse transform of the two-dimensional Fourier transform, FT 2D is the two-dimensional Fourier transform, e is the natural logarithm, is the imaginary unit, k z′ is k in the spatial wave number domain x′ With k y′ The spherical function, z is the plane represented by multiple reference distances, σ s (x, y, z) is the single-frequency imaging defocus result of the scattering point coordinates, s(x′,′, 0 ) is the spatial wave number received by the plane array transceiver element at the coordinate (x′,′) is k 0 Single frequency raw echo data.
[0107] In some embodiments, the input module 503 includes:
[0108] The single-frequency imaging defocus results are input into the pre-built spatial convolutional neural network for training to obtain network coding results, including:
[0109] Using a broadband imaging device to collect broadband echo data, and extracting single frequency echo data from the broadband echo data;
[0110] The broadband echo data and the single-frequency echo data are imaged respectively to obtain the broadband imaging focusing result and the single-frequency imaging defocusing result;
[0111] Convert the broadband imaging focusing results into maximum projection results to serve as training labels;
[0112] Obtaining the amplitude value of the single-frequency imaging defocus result as training data;
[0113] The pre-constructed spatial convolutional neural network is pre-trained using training labels and training data, and supervised training is performed using mean square error as loss to obtain a trained spatial convolutional neural network;
[0114] The single-frequency imaging defocus results are input into the trained spatial convolutional neural network for training to obtain the network coding results.
[0115] In some embodiments, the solution formula for the network coding result is:
[0116] x c = θ {{(x′,′, 0 )}}
[0117] Among them, x c is the network coding result, F θ {·} is the trained spatial convolutional neural network, PSM{·} is the near-field imaging method, s(x′,′, 0 ) is the spatial wave number received by the plane array transceiver element at the coordinate (x′,′) is k 0 Single frequency raw echo data.
[0118] In some embodiments, the solution formula for the diffusion model prediction result is:
[0119]
[0120] Among them, x 0 (x 1 ,x c ,1) is the prediction result of the diffusion model, x 1 is the input of the preset diffusion model at time step 1, β 1 Add a hyperparameter, α, for time-correlated noise at time step 1 1 , is the conditional diffusion model hyperparameter when the time step is 1, ∈ θ (x 1 ,x c ,1) is the conditional noise predictor in the conditional diffusion model when the time step is 1.
[0121] It should be noted that the aforementioned explanation of the embodiment of the generative single-frequency millimeter-wave self-focusing near-field imaging method based on the diffusion model is also applicable to the generative single-frequency millimeter-wave self-focusing near-field imaging device based on the diffusion model of this embodiment, and will not be repeated here.
[0122] The generative single-frequency millimeter-wave self-focusing near-field imaging device based on the diffusion model proposed in the embodiment of the present invention has the following beneficial effects:
[0123] (1) Different from the common neural network method that directly maps the input from the echo data domain to the imaging domain, the embodiment of the present invention uses the traditional millimeter wave near-field imaging method to reconstruct the single-frequency raw echo data into a three-dimensional defocused image, which reduces the difficulty of neural network fitting. It also uses three-dimensional convolution to extract the spatial structure in the defocused image, and inputs these spatial structures as conditions into the diffusion model to iteratively reconstruct a high-quality millimeter wave image.
[0124] (2) A spatial feature extraction network using three-dimensional convolution is designed. This network takes into account the spatial structure of the target point spread function in the three-dimensional defocused image. The three-dimensional convolution is used to replace the original two-dimensional convolution in the feature extraction network, which can effectively extract the spatial structure features in the three-dimensional defocused image and map the extracted features into a feature map with a smaller dimension, reducing the number of parameters of the overall network.
[0125] (3) In order to address the problems of insufficient fitting ability of a single neural network and difficulty in selecting a loss function, an embodiment of the present invention embeds a three-dimensional spatial feature extraction network as a conditional encoder into a conditional diffusion model, guides the iterative reconstruction process of the diffusion model at multiple time steps, and ultimately recovers a high-quality focused image from single-frequency echo data.
[0126] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device may include:
[0127] A memory 601 , a processor 602 , and a computer program stored in the memory 601 and executable on the processor 602 .
[0128] When the processor 602 executes the program, the generative single-frequency millimeter-wave self-focusing near-field imaging method based on the diffusion model provided in the above embodiment is implemented.
[0129] Furthermore, the electronic device further comprises:
[0130] The communication interface 603 is used for communication between the memory 601 and the processor 602 .
[0131] The memory 601 is used to store computer programs that can be executed on the processor 602 .
[0132] The memory 601 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
[0133] If the memory 601, the processor 602 and the communication interface 603 are implemented independently, the communication interface 603, the memory 601 and the processor 602 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0134] Optionally, in a specific implementation, if the memory 601, the processor 602 and the communication interface 603 are integrated on a chip, the memory 601, the processor 602 and the communication interface 603 can communicate with each other through an internal interface.
[0135] The processor 602 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.
[0136] An embodiment of the present invention further provides a computer program product, which, when executed by a processor, implements the above-mentioned generative single-frequency millimeter-wave self-focusing near-field imaging method based on a diffusion model.
[0137] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned generative single-frequency millimeter-wave self-focusing near-field imaging method based on a diffusion model.
[0138] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0139] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0140] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.
[0141] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.
[0142] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0143] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0144] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0145] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.
Claims
1. A generative single-frequency millimeter-wave self-focusing near-field imaging method based on a diffusion model, characterized in that: The following steps are involved: Using radar equipment to obtain single frequency raw echo data; Imaging the single-frequency raw echo data to obtain a single-frequency imaging defocus result; Inputting the single-frequency imaging defocus result into a pre-constructed spatial convolutional neural network to obtain a network coding result; Embedding the network coding result into a pre-trained denoising diffusion model to obtain a conditional diffusion model; The conditional diffusion model is used to gradually reconstruct the residual value between the network coding result and the preset ideal broadband imaging result to obtain the diffusion model prediction result, and the diffusion model prediction result is added to the network coding result to obtain the final single-frequency self-focusing imaging result.
2. The generative single-frequency millimeter-wave self-focusing near-field imaging method based on a diffusion model according to claim 1, characterized in that: The imaging of the single-frequency raw echo data to obtain a single-frequency imaging defocus result includes: Based on the phase shift method in the millimeter wave near-field imaging method, the single-frequency raw echo data is imaged on several equally spaced reference planes in a pre-selected imaging space to obtain the single-frequency imaging defocus result.
3. The generative single-frequency millimeter-wave self-focusing near-field imaging method based on a diffusion model according to claim 2, characterized in that: The solution formula for the single-frequency imaging defocus result is: in, is the inverse transform of the two-dimensional Fourier transform, FT 2D is the two-dimensional Fourier transform, e is the natural logarithm, j is the imaginary unit, k z′ is k in the spatial wave number domain x′ With k y′ The spherical function, z is the plane represented by multiple reference distances, σ s (x, y, z) is the single-frequency imaging defocus result of the scattering point coordinates, and s(x′, y′, k0) is the single-frequency original echo data with a spatial wave number of k0 received by the planar array transceiver element at the coordinate (x′, y′).
4. The generative single-frequency millimeter-wave self-focusing near-field imaging method based on a diffusion model according to claim 1, characterized in that: The step of inputting the single-frequency imaging defocus result into a pre-constructed spatial convolutional neural network to obtain a network coding result comprises: Collecting broadband echo data using a broadband imaging device, and extracting single frequency echo data from the broadband echo data; Imaging the broadband echo data and the single-frequency echo data respectively to obtain a broadband imaging focusing result and a single-frequency imaging defocusing result; Converting the broadband imaging focusing result into a maximum projection result as a training label; Obtaining the amplitude value of the single-frequency imaging defocus result as training data; Pre-training the pre-constructed spatial convolutional neural network using the training labels and the training data, performing supervised training with mean square error as loss, and obtaining a trained spatial convolutional neural network; The single-frequency imaging defocus result is input into the trained spatial convolutional neural network for training to obtain the network coding result.
5. The generative single-frequency millimeter-wave self-focusing near-field imaging method based on a diffusion model according to claim 4, characterized in that: The solution formula for the network coding result is: x c =F θ {PSM{s(x',y',k0)}} Among them, x c is the network coding result, F θ {·} is the trained spatial convolutional neural network, PSM{·} is the near-field imaging method, and s(x′, y′, k0) is the single-frequency original echo data with a spatial wave number of k0 received by the planar array transceiver element at the coordinate (x′, y′).
6. The generative single-frequency millimeter-wave self-focusing near-field imaging method based on a diffusion model according to claim 1, characterized in that: The solution formula for the diffusion model prediction result is: Among them, x0(x1,x c ,1) is the prediction result of the diffusion model, x1 is the input of the preset diffusion model at time step 1, β1 is the time-correlated noise increase hyperparameter when the time step is 1, α1, is the conditional diffusion model hyperparameter when the time step is 1, ∈ θ (x1,x c ,1) is the conditional noise predictor in the conditional diffusion model when the time step is 1.
7. A generative single-frequency millimeter-wave self-focusing near-field imaging device based on a diffusion model, characterized in that: include: An acquisition module, used for acquiring single-frequency raw echo data using radar equipment; An imaging module, used for imaging the single-frequency raw echo data to obtain a single-frequency imaging defocus result; An input module, used for inputting the single-frequency imaging defocus result into a pre-built spatial convolutional neural network to obtain a network coding result; An embedding module, used for embedding the network coding result into a pre-trained denoising diffusion model to obtain a conditional diffusion model; A prediction module is used to use the conditional diffusion model to gradually reconstruct the residual value between the network coding result and the preset ideal broadband imaging result to obtain the diffusion model prediction result, and add the diffusion model prediction result to the network coding result to obtain the final single-frequency self-focusing imaging result.
8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the generative single-frequency millimeter-wave self-focusing near-field imaging method based on a diffusion model as described in any one of claims 1 to 6.
9. A computer program product, characterized in that When the computer program / instruction is executed by a processor, the generative single-frequency millimeter-wave self-focusing near-field imaging method based on a diffusion model as described in any one of claims 1 to 6 is implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the generative single-frequency millimeter-wave self-focusing near-field imaging method based on a diffusion model as described in any one of claims 1 to 6.
