Millimeter wave radar imaging method based on unsupervised network
By optimizing the end-to-end unsupervised network and designing multi-scale convolutional layers, the model mismatch and insufficient feature extraction problems of millimeter-wave radar imaging under sparse sampling are solved, achieving highly robust and high-resolution radar imaging suitable for complex scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-17
- Publication Date
- 2026-04-03
AI Technical Summary
Existing millimeter-wave radar imaging methods suffer from decreased imaging quality under sparse or undersampled conditions, model mismatch, difficulty in effectively extracting multi-scale features of complex scenes, and supervised learning methods rely on a large amount of labeled data, resulting in high data acquisition costs and limited generalization ability.
End-to-end optimization is performed using unsupervised networks. By constructing an untrained image generator network and combining multi-layer nonlinear mapping and loss function optimization, high-quality images can be reconstructed from sparse sampled data. Residual networks and multi-scale convolutional layers are used to improve feature extraction capabilities, and a loss function is constructed to constrain sparsity and data consistency.
Under zero-shot learning conditions, it significantly improves the robustness and resolution of radar imaging, suppresses noise and sidelobe effects, can reconstruct high-resolution images at low sampling rates, and has a flexible and scalable framework.
Smart Images
Figure CN121784743A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of radar signal processing and computational imaging technology, and in particular to a millimeter-wave radar imaging method based on unsupervised networks. Background Technology
[0002] Millimeter-wave radar, with its all-weather, all-day high-resolution imaging capabilities, has been widely used in military, surveying, and environmental monitoring fields. Traditional millimeter-wave radar imaging algorithms, such as matched filtering, while computationally efficient, suffer from significant image quality degradation and high sidelobes under sparse or undersampled conditions. Compressed sensing and other methods can improve image quality to some extent, but they rely on complex parameter tuning and iterative processes, resulting in high computational costs.
[0003] In recent years, artificial intelligence technologies, represented by deep learning, have been introduced into the field of millimeter-wave radar imaging. By learning the mapping from echo to image in a data-driven manner, they have effectively improved imaging resolution and simplified parameter procedures. However, existing supervised deep learning methods typically require a large amount of paired "echo-image" labeled data for training, which is costly in terms of data acquisition, and the trained models have limited generalization ability when faced with new scenarios.
[0004] Implicit Neural Representation (INR) offers a novel solution to this problem. INR can perform adaptive iterative optimization using a single data sample without pre-training, demonstrating good generalization potential. However, existing INR imaging methods still face some challenges when applied to radar imaging: firstly, existing implicit neural network frameworks suffer from model mismatch in millimeter-wave radar imaging; secondly, image quality degrades significantly under low sampling rates; and thirdly, conventional network structures struggle to effectively extract complex multi-scale features from radar scenes, resulting in insufficient resolution for strong scattering points and weak backgrounds, easily leading to reduced resolution or sidelobe effects.
[0005] Therefore, how to design a radar imaging method that does not require training data, can maintain high imaging quality under undersampling conditions, and can effectively characterize multi-scale features is a technical problem that urgently needs to be solved in this field. Summary of the Invention
[0006] The purpose of this invention is to provide a millimeter-wave radar imaging method based on unsupervised networks that does not require a pre-trained dataset, has strong generalization ability, high robustness under sparse sampling, strong feature extraction ability, and high flexibility and scalability.
[0007] The technical solution to achieve the purpose of this invention is: a millimeter-wave radar imaging method based on unsupervised networks, comprising the following steps:
[0008] Step 1: Acquire radar echo data and define an undersampling operator to construct an untrained image generator network parameterized by optimizable parameters;
[0009] Step 2: The fixed low-dimensional latent tensor is forward-propagated through the image generator network and mapped to the scattering rate distribution estimate of the generated scene through multi-layer nonlinear mapping.
[0010] Step 3: Perform a two-dimensional inverse Fourier transform on the estimated scattering rate distribution to obtain the complete echo signal fitted by the image generator network;
[0011] Step 4: Use the undersampling operator to sample the complete echo signal to obtain the reconstructed undersampled echo;
[0012] Step 5: Construct the loss function, including a data consistency term to ensure that the reconstructed undersampled echo is consistent with the actual acquired radar echo data, and a regularization term to constrain the sparsity of the scattering rate distribution estimation.
[0013] Step 6: Based on the loss function, iteratively optimize the parameters of the image generator network using the backpropagation algorithm;
[0014] Step 7: Repeat steps 2 through 6 until the maximum number of iterations is reached;
[0015] Step 8: Output the final optimized image generator network-generated scattering rate distribution estimate as the imaging result of the millimeter-wave radar.
[0016] Furthermore, the acquisition of radar echo data in step 1, the definition of an undersampling operator, and the construction of an untrained image generator network parameterized by optimizable parameters are as follows:
[0017] Step 1.1: Acquire radar echo data And define an undersampling operator M to characterize radar echo data. Valid sampling location;
[0018] Step 1.2: Let the number of iterations n = 1; and n = 1, ..., num; num is the total number of iterations;
[0019] Step 1.3: Construct an untrained image generator network G parameterized by optimizable parameters p. The image generator network G takes a preset low-dimensional matrix C as a fixed input and nonlinearly maps it to a high-dimensional scene scattering rate distribution estimate through a series of upsampling and convolution operations.
[0020]
[0021] in, Let C be the spatial dimension of the low-dimensional matrix. c represents the spatial dimension of the output image, and c represents the number of channels.
[0022] Furthermore, the undersampling operator M described in step 1.1 is a binary mask matrix, where the positions of elements with a value of 1 correspond to radar echo data. The valid sampling points in the data are those where the element value of 0 corresponds to a missing or invalid sampling point.
[0023] Furthermore, the low-dimensional matrix C mentioned in step 1.3 is a randomly initialized noise matrix, or a small-sized matrix randomly sampled from the radar echo data.
[0024] Furthermore, the image generator network G described in step 1.3 contains 6 network layers, each containing 128 channels. Each network layer consists of an upsampling layer, a residual network layer, and a multi-scale convolutional layer. The image generator network gradually improves the spatial resolution and information dimension of the data through layer-by-layer mapping and feature reconstruction, estimates the scattering distribution of the high-dimensional target scene from the input data, and realizes the transformation from low-dimensional input to high-dimensional output.
[0025] The upsampling layer performs a nonlinear mapping on the low-size scattering coefficient output from the previous layer, transforming it into a high-size scattering coefficient. The calculation formula is as follows:
[0026]
[0027] The residual network layer is used to enhance the training stability and feature representation ability of the deep neural network during the deep network iteration process. The calculation formula is as follows:
[0028]
[0029] in, It is the input of this layer. It is the output of the intra-layer nonlinear transformation module. This is the final output;
[0030] The multi-scale convolutional layer processes the input features in parallel using convolutional kernels of various sizes, and then fuses the outputs from each path. The calculation formula is as follows:
[0031]
[0032] Where f(·), g(·), and h(·) represent the operations of different convolution kernels.
[0033] Furthermore, the residual network layer consists of two 1×1 convolutional layers, two regularization layers, a ReLU activation function, and cross-layer skip connections;
[0034] First, the input features from the previous layer are convolved with 1×1 and then normalized to enhance information exchange within the channels.
[0035] Then, the ReLU activation function is used for nonlinear transformation to improve the feature learning ability of the network.
[0036] Next, the second 1×1 convolutional layer further extracts deeper information and is normalized again to ensure the stability of the data distribution;
[0037] Finally, the input features are directly passed to subsequent layers through identity mapping, and the outputs of the skip connections are added to the outputs of the backbone network and then passed through the ReLU activation function again to improve the training stability of the network and promote deep feature learning.
[0038] Furthermore, the multi-scale convolutional layer consists of 1×1, 3×3 and 5×5 convolutional kernels, convolutional fusion layers, regularization layers, and ReLU activation function layers;
[0039] First, 1×1, 3×3 and 5×5 convolutions are applied to the input features of the previous layer to extract feature information within different receptive fields. Among them, 1×1 convolution is used for dimensionality reduction and inter-channel information fusion, 3×3 convolution is used to capture local spatial features, and 5×5 convolution is used to expand the receptive field to extract global information at a larger scale.
[0040] Then, the convolution results at each scale are fused by element-wise addition to achieve adaptive integration of multi-scale features;
[0041] Next, the fused features are standardized by a regularization layer;
[0042] Finally, a nonlinear transformation is performed through a ReLU activation function layer.
[0043] Furthermore, in step 2, the fixed low-dimensional latent tensor is forward-propagated through the image generator network and mapped to the scattering rate distribution estimate of the generated scene through multi-layer nonlinear mapping, as detailed below:
[0044] A fixed low-dimensional latent tensor C is forwarded through network G and mapped to an estimate of the scattering rate distribution of the generated scene through multiple nonlinear mappings. .
[0045] Furthermore, the loss function constructed in step 5 includes a data consistency term to ensure that the reconstructed undersampled echo is consistent with the actual acquired radar echo data, and a regularization term to constrain the sparsity of the scattering rate distribution estimation, as follows:
[0046] A loss function is constructed, consisting of two parts: a data consistency term to ensure consistency between the reconstructed undersampled echo and the actual acquired radar echo data, minimizing the difference between the reconstructed undersampled echo and the actually observed radar echo data; and a regularization term to constrain the sparsity of the scattering rate distribution estimation, thereby suppressing noise and sidelobes. The formula is as follows:
[0047]
[0048] Furthermore, in step 6, the parameters of the image generator network are iteratively optimized using the backpropagation algorithm based on the loss function, as described in the formula:
[0049]
[0050] Compared with existing technologies, the significant advantages of this invention are: (1) Zero-shot learning capability: This method does not require a pre-training dataset and directly performs end-to-end optimization on the single-frame data to be imaged, fundamentally solving the data dependency and generalization problems of supervised learning methods; (2) High robustness under sparse sampling: By fusing the physical model with the prior depth of the deep network, this method can effectively recover complete scene information from extremely sparse measurement data. Its performance is significantly better than traditional methods at low sampling rates. Even when the data is severely undersampled, it can effectively fill in the missing echo information and reconstruct the image. High-quality, high-resolution radar images, and significantly suppress sidelobes and noise; (3) Powerful feature extraction capability: The innovative network structure design, especially the introduction of multi-scale convolutional layers and residual network layers, significantly improves the network's ability to capture features of complex scenes and the stability of training, and can better distinguish strong scatterers from background clutter, making the imaging results more detailed and the outline clearer; (4) Flexibility and scalability: The framework of this invention has good flexibility, and the physical forward model regularization term can be replaced according to the specific radar system and imaging scene, and it is easy to extend to more complex imaging modes. Attached Figure Description
[0051] Figure 1 This is a flowchart illustrating a millimeter-wave radar imaging method based on unsupervised networks according to the present invention.
[0052] Figure 2 This is a schematic diagram of the image generator network in this invention.
[0053] Figure 3 This is a schematic diagram of the residual network layer in this invention.
[0054] Figure 4 This is a schematic diagram of the structure of the multi-scale convolutional layer in this invention.
[0055] Figure 5 This is a schematic diagram of the experimental scenario in an embodiment of the present invention.
[0056] Figure 6 This is a schematic diagram of the imaging results in an embodiment of the present invention. Detailed Implementation
[0057] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0058] like Figure 1 As shown, the present invention provides a millimeter-wave radar imaging method based on unsupervised networks, comprising the following steps:
[0059] Step 1: Acquire radar echo data and define an undersampling operator to construct an untrained image generator network parameterized by optimizable parameters, as follows:
[0060] Step 1.1: Acquire radar echo data And define an undersampling operator M to characterize radar echo data. Valid sampling location;
[0061] The undersampling operator M is a binary mask matrix, where the positions of elements with a value of 1 correspond to radar echo data. The valid sampling points in the data are those where the element value of 0 corresponds to a missing or invalid sampling point.
[0062] Step 1.2: Let the number of iterations n = 1; and n = 1, ..., num; num is the total number of iterations;
[0063] Step 1.3: Construct an untrained image generator network G parameterized by optimizable parameters p. The image generator network G takes a preset low-dimensional matrix C as a fixed input and nonlinearly maps it to a high-dimensional scene scattering rate distribution estimate through a series of upsampling and convolution operations.
[0064]
[0065] in, Let C be the spatial dimension of the low-dimensional matrix. c represents the spatial dimension of the output image, and c represents the number of channels.
[0066] As a specific example, the low-dimensional matrix C is a randomly initialized noise matrix, or a small-sized matrix randomly sampled from the radar echo data.
[0067] As a specific example, such as Figure 2As shown, the image generator network G contains 6 network layers, each containing 128 channels. Each network layer consists of an upsampling layer, a residual network layer, and a multi-scale convolutional layer. The image generator network improves the spatial resolution and information dimension of the data by layer-by-layer mapping and feature reconstruction, and estimates the scattering distribution of the high-dimensional target scene from the input data, thus realizing the transformation from low-dimensional input to high-dimensional output.
[0068] The upsampling layer performs a nonlinear mapping on the low-size scattering coefficient output from the previous layer, transforming it into a high-size scattering coefficient. The calculation formula is as follows:
[0069]
[0070] The residual network layer is used to enhance the training stability and feature representation ability of the deep neural network during the deep network iteration process. The calculation formula is as follows:
[0071]
[0072] in, It is the input of this layer. It is the output of the intra-layer nonlinear transformation module. This is the final output;
[0073] The multi-scale convolutional layer processes the input features in parallel using convolutional kernels of various sizes, and then fuses the outputs from each path. The calculation formula is as follows:
[0074]
[0075] Where f(·), g(·), and h(·) represent the operations of different convolution kernels.
[0076] As a specific example, the residual network layer consists of two 1×1 convolutional layers, two regularization layers, a ReLU activation function, and cross-layer skip connections;
[0077] First, the input features from the previous layer are convolved with 1×1 and then normalized to enhance information exchange within the channels.
[0078] Then, the ReLU activation function is used for nonlinear transformation to improve the feature learning ability of the network.
[0079] Next, the second 1×1 convolutional layer further extracts deeper information and is normalized again to ensure the stability of the data distribution;
[0080] Finally, the input features are directly passed to subsequent layers through identity mapping, and the outputs of the skip connections are added to the outputs of the backbone network and then passed through the ReLU activation function again to improve the training stability of the network and promote deep feature learning.
[0081] As a specific example, the multi-scale convolutional layer consists of 1×1, 3×3 and 5×5 convolutional kernels, convolutional fusion layers, regularization layers, and ReLU activation function layers;
[0082] First, 1×1, 3×3 and 5×5 convolutions are applied to the input features of the previous layer to extract feature information within different receptive fields. Among them, 1×1 convolution is used for dimensionality reduction and inter-channel information fusion, 3×3 convolution is used to capture local spatial features, and 5×5 convolution is used to expand the receptive field to extract global information at a larger scale.
[0083] Then, the convolution results at each scale are fused by element-wise addition to achieve adaptive integration of multi-scale features;
[0084] Next, the fused features are standardized by a regularization layer to accelerate training and improve model stability;
[0085] Finally, a nonlinear transformation is performed through a ReLU activation function layer to enhance the network's feature representation capabilities.
[0086] Step 2: The fixed low-dimensional latent tensor is forward-propagated through the image generator network and mapped to the estimated scattering rate distribution of the generated scene through multiple layers of nonlinear mapping, as follows:
[0087] A fixed low-dimensional latent tensor C is forwarded through network G and mapped to an estimate of the scattering rate distribution of the generated scene through multiple nonlinear mappings. .
[0088] Step 3: Perform a two-dimensional inverse Fourier transform on the estimated scattering rate distribution to obtain the complete echo signal fitted by the image generator network;
[0089] Step 4: Use the undersampling operator to sample the complete echo signal to obtain the reconstructed undersampled echo;
[0090] Step 5: Construct the loss function, including a data consistency term to ensure that the reconstructed undersampled echo is consistent with the actual acquired radar echo data, and a regularization term to constrain the sparsity of the scattering rate distribution estimation, as follows:
[0091] A loss function is constructed, consisting of two parts: a data consistency term to ensure consistency between the reconstructed undersampled echo and the actual acquired radar echo data, minimizing the difference between the reconstructed undersampled echo and the actually observed radar echo data; and a regularization term to constrain the sparsity of the scattering rate distribution estimation, thereby suppressing noise and sidelobes. The formula is as follows:
[0092]
[0093] Step 6: Based on the loss function, iteratively optimize the parameters of the image generator network using the backpropagation algorithm. The formula is as follows:
[0094]
[0095] Step 7: Repeat steps 2 through 6 until the maximum number of iterations is reached;
[0096] Step 8: Output the final optimized image generator network-generated scattering rate distribution estimate as the imaging result of the millimeter-wave radar.
[0097] Example
[0098] This embodiment uses a TI AWR2243 millimeter-wave radar for data acquisition. It is mounted on the side of the vehicle's roof and scans a parking lot scene to obtain data. The radar system's carrier frequency is 77 GHz, and the modulation frequency is 70.295 THz / s. During the experiment, the target vehicle was located within 6-8 meters of the radar.
[0099] Experimental scenarios such as Figure 5 As shown, the experimental results are as follows: Figure 6 As shown. From Figure 6 The visual results demonstrate that the method proposed in this invention performs well in target imaging. The target area within the bounding box exhibits strong signal and clear edges, with a complete vehicle target outline and almost no obvious artifact interference. Simultaneously, it shows significant effectiveness in suppressing background noise, resulting in a clean image background and good contrast between the target and background, making the target stand out more clearly.
[0100] In summary, this embodiment verifies the correctness, effectiveness, and reliability of the present invention. The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A millimeter-wave radar imaging method based on unsupervised networks, characterized in that, Includes the following steps: Step 1: Acquire radar echo data and define an undersampling operator to construct an untrained image generator network parameterized by optimizable parameters; Step 2: The fixed low-dimensional latent tensor is forward-propagated through the image generator network and mapped to the scattering rate distribution estimate of the generated scene through multi-layer nonlinear mapping. Step 3: Perform a two-dimensional inverse Fourier transform on the estimated scattering rate distribution to obtain the complete echo signal fitted by the image generator network; Step 4: Use the undersampling operator to sample the complete echo signal to obtain the reconstructed undersampled echo; Step 5: Construct the loss function, including a data consistency term to ensure that the reconstructed undersampled echo is consistent with the actual acquired radar echo data, and a regularization term to constrain the sparsity of the scattering rate distribution estimation. Step 6: Based on the loss function, iteratively optimize the parameters of the image generator network using the backpropagation algorithm; Step 7: Repeat steps 2 through 6 until the maximum number of iterations is reached; Step 8: Output the final optimized image generator network-generated scattering rate distribution estimate as the imaging result of the millimeter-wave radar.
2. The millimeter-wave radar imaging method based on unsupervised networks according to claim 1, characterized in that, Step 1 involves acquiring radar echo data, defining an undersampling operator, and constructing an untrained image generator network parameterized by optimizable parameters, as detailed below: Step 1.1: Acquire radar echo data And define an undersampling operator M to characterize radar echo data. Valid sampling location; Step 1.2: Let the number of iterations n = 1; and n = 1, ..., num; num is the total number of iterations; Step 1.3: Construct an untrained image generator network G parameterized by optimizable parameters p. The image generator network G takes a preset low-dimensional matrix C as a fixed input and nonlinearly maps it to a high-dimensional scene scattering rate distribution estimate through a series of upsampling and convolution operations. ； in, Let C be the spatial dimension of the low-dimensional matrix. c represents the spatial dimension of the output image, and c represents the number of channels.
3. The millimeter-wave radar imaging method based on unsupervised networks according to claim 2, characterized in that, The undersampling operator M mentioned in step 1.1 is a binary mask matrix, where the positions of elements with a value of 1 correspond to radar echo data. The valid sampling points in the data are those where the element value is 0, and the positions corresponding to the missing or invalid sampling points are the valid sampling points.
4. The millimeter-wave radar imaging method based on unsupervised networks according to claim 2, characterized in that, The low-dimensional matrix C mentioned in step 1.3 is a randomly initialized noise matrix, or a small-sized matrix randomly sampled from the radar echo data.
5. The millimeter-wave radar imaging method based on unsupervised networks according to claim 2, characterized in that, The image generator network G described in step 1.3 contains 6 network layers, each containing 128 channels. Each network layer consists of an upsampling layer, a residual network layer, and a multi-scale convolutional layer. The image generator network improves the spatial resolution and information dimension of the data by layer-by-layer mapping and feature reconstruction, and estimates the scattering distribution of the high-dimensional target scene from the input data, thus realizing the transformation from low-dimensional input to high-dimensional output. The upsampling layer performs a nonlinear mapping on the low-size scattering coefficient output from the previous layer, transforming it into a high-size scattering coefficient. The calculation formula is as follows: ； The residual network layer is used to enhance the training stability and feature representation ability of the deep neural network during the deep network iteration process. The calculation formula is as follows: ； in, It is the input of this layer. It is the output of the intra-layer nonlinear transformation module. This is the final output; The multi-scale convolutional layer processes the input features in parallel using convolutional kernels of various sizes, and then fuses the outputs from each path. The calculation formula is as follows: ； Where f(·), g(·), and h(·) represent the operations of different convolution kernels.
6. The millimeter-wave radar imaging method based on unsupervised networks according to claim 5, characterized in that, The residual network layer consists of two 1×1 convolutional layers, two regularization layers, a ReLU activation function, and cross-layer skip connections; First, the input features from the previous layer are convolved with 1×1 and then normalized to enhance information exchange within the channels. Then, the ReLU activation function is used for nonlinear transformation to improve the feature learning ability of the network. Next, the second 1×1 convolutional layer further extracts deeper information and is normalized again to ensure the stability of the data distribution; Finally, the input features are directly passed to subsequent layers through identity mapping, and the outputs of the skip connections are added to the outputs of the backbone network and then passed through the ReLU activation function again to improve the training stability of the network and promote deep feature learning.
7. The millimeter-wave radar imaging method based on unsupervised networks according to claim 5, characterized in that, The multi-scale convolutional layer consists of 1×1, 3×3 and 5×5 convolutional kernels, convolutional fusion layers, regularization layers, and ReLU activation function layers; First, 1×1, 3×3 and 5×5 convolutions are applied to the input features of the previous layer to extract feature information within different receptive fields. Among them, 1×1 convolution is used for dimensionality reduction and inter-channel information fusion, 3×3 convolution is used to capture local spatial features, and 5×5 convolution is used to expand the receptive field to extract global information at a larger scale. Then, the convolution results at each scale are fused by element-wise addition to achieve adaptive integration of multi-scale features; Next, the fused features are standardized by a regularization layer; Finally, a nonlinear transformation is performed through a ReLU activation function layer.
8. The millimeter-wave radar imaging method based on unsupervised networks according to claim 7, characterized in that, Step 2 involves forward propagating a fixed low-dimensional latent tensor through the image generator network and mapping it to an estimate of the scattering rate distribution of the generated scene via multi-layer nonlinear mapping, as detailed below: A fixed low-dimensional latent tensor C is forwarded through network G and mapped to an estimate of the scattering rate distribution of the generated scene through multiple nonlinear mappings. .
9. The millimeter-wave radar imaging method based on unsupervised networks according to claim 8, characterized in that, The loss function described in step 5 includes a data consistency term to ensure that the reconstructed undersampled echo is consistent with the actual acquired radar echo data, and a regularization term to constrain the sparsity of the scattering rate distribution estimation, as detailed below: A loss function is constructed, consisting of two parts: a data consistency term to ensure consistency between the reconstructed undersampled echo and the actual acquired radar echo data, minimizing the difference between the reconstructed undersampled echo and the actually observed radar echo data; and a regularization term to constrain the sparsity of the scattering rate distribution estimation, thereby suppressing noise and sidelobes. The formula is as follows: 。 10. The millimeter-wave radar imaging method based on unsupervised networks according to claim 9, characterized in that, Step 6 describes the iterative optimization of the image generator network parameters based on the loss function using the backpropagation algorithm, as shown in the formula: 。