Millimeter wave radar anti-attack imaging method and system

Abstract

The present invention provides a millimeter-wave radar anti-attack imaging method that can be applied to the fields of radar signal processing and artificial intelligence technology. The method includes: performing multi-dimensional fast Fourier transform processing on the intermediate frequency signal to obtain a range angle map and a range Doppler map; using a two-stream neural network to aggregate information on the range angle map and the range Doppler map respectively to obtain radar semantic features; using a semantic mapping network to map the radar semantic features to the target domain feature space and remove the attack noise in the radar semantic features to obtain radar target domain features; constructing an isolation mechanism to resist noise disturbance in the latent imaging space, using a feature space codec network to perform multiple gradient discretization processing on the radar target domain features, mapping the radar target domain features to the latent imaging space, and obtaining an imaging map of the target scene. The present invention also provides a multi-neural network training method and a millimeter-wave radar anti-attack imaging system, electronic equipment, and storage medium.

Description

Technical Field

[0001] The present invention relates to the field of radar signal processing and artificial intelligence technology, and in particular to a millimeter wave radar anti-attack imaging method and system, a multi-neural network training method, an electronic device, and a storage medium. Background Art

[0002] In recent years, imaging technology based on millimeter-wave radar signals has rapidly developed, showing significant application prospects in multiple fields. However, with the widespread adoption of millimeter-wave radar imaging systems, this technology faces new challenges. Multipath propagation effects in millimeter-wave radar signals cause coherent interference in complex electromagnetic environments, severely impacting target feature extraction accuracy. Limited by existing array apertures and beamforming algorithms, achieving sub-centimeter spatial resolution is difficult. It is also worth noting that research on defense mechanisms against countermeasures against millimeter-wave radar attacks is still in its infancy, and a comprehensive theoretical framework and security assessment system are urgently needed. Therefore, improving millimeter-wave radar imaging quality and its ability to resist countermeasures has become particularly important.

[0003] While some existing deep learning-based millimeter-wave radar super-resolution imaging algorithms exist, they typically only address multipath imaging and struggle with complex indoor environments and fine-grained perception. Therefore, there is a need for a solution that combines deep learning, adversarial purification, and millimeter-wave radar imaging to achieve high-quality, high-precision, and highly robust millimeter-wave radar anti-attack imaging. Summary of the Invention

[0004] In view of the above problems, the present invention provides a millimeter wave radar anti-attack imaging method and system, a multi-neural network training method, an electronic device and a storage medium.

[0005] According to a first aspect of the present invention, a millimeter wave radar anti-attack imaging method is provided, comprising:

[0006] The transmission signal of the millimeter-wave radar is processed with the echo signal of the target scene to obtain an intermediate frequency signal, and the intermediate frequency signal is processed by multi-dimensional fast Fourier transform to obtain a range angle map and a range Doppler map;

[0007] A dual-stream neural network is used to extract physical layer features from the range angle map and the range Doppler map respectively, and the extracted range angle physical layer features and range Doppler physical layer features are aggregated to obtain radar semantic features.

[0008] The radar semantic features are mapped to the pre-constructed target domain feature space using a semantic mapping network and the attack noise in the radar semantic features is removed to obtain the radar target domain features after knowledge transfer.

[0009] An isolation mechanism against noise disturbance is constructed in the latent imaging space. The radar target domain features are subjected to multiple gradient discretization processes using the feature space encoder-decoder network. The radar target domain features are mapped to the latent imaging space to obtain an imaging map of the target scene.

[0010] According to an embodiment of the present invention, the processing of the transmission signal of the millimeter wave radar and the echo signal of the target scene to obtain the intermediate frequency signal includes:

[0011] Based on the principle of frequency modulated continuous wave radar, the transmission signal of the millimeter wave radar is set to a linear frequency modulated wave, and the echo signal of the target scene is received by the millimeter wave radar;

[0012] Performing complex conjugate mixing processing on the linear frequency modulation wave and the echo signal to obtain a complex conjugate mixing signal;

[0013] The complex conjugate mixed signal is subjected to complex conjugate multiplication, low-pass filtering and orthogonal down-conversion by a preset mixer to obtain an intermediate frequency signal.

[0014] According to an embodiment of the present invention, performing multi-dimensional fast Fourier transform processing on the intermediate frequency signal to obtain the range angle map and the range Doppler map includes:

[0015] By performing fast Fourier transform on the multi-channel intermediate frequency signal along the time dimension, the distance information of the target scene is obtained;

[0016] By performing fast Fourier transform on the multi-channel intermediate frequency signals along the spatial dimension, the angular spectrum of the target scene is obtained;

[0017] Perform fast Fourier transform on the slow time dimension signal of multiple consecutive frequency modulation cycles to obtain the speed information of the target scene;

[0018] The distance information and angle spectrum of the target scene are selected to obtain a distance angle map, and the distance information and speed information of the target scene are selected to obtain a range Doppler map.

[0019] According to an embodiment of the present invention, the above-mentioned dual-stream neural network is used to simultaneously extract physical layer features from the range angle map and the range Doppler map, and the extracted range angle physical layer features and range Doppler physical layer features are aggregated to obtain radar semantic features including:

[0020] The first physical layer feature extraction stream of the two-stream neural network based on self-supervised learning is used to perform pooling or strided convolution operations on the distance angle map to obtain distance angle physical layer features with contextual information;

[0021] The second physical layer feature extraction stream of the two-stream neural network based on self-supervised learning is used to perform pooling or strided convolution operations on the range Doppler map to obtain the range Doppler physical layer features with contextual information;

[0022] The range angle physical layer features and the range Doppler physical layer features are subjected to multimodal information aggregation processing based on the self-attention mechanism to obtain radar semantic features.

[0023] According to an embodiment of the present invention, the radar target domain features obtained after knowledge transfer by using a semantic mapping network to map radar semantic features to a pre-constructed target domain feature space and removing attack noise from the radar semantic features include:

[0024] The semantic mapping network is used to model the mapping path between radar semantic features and pre-constructed target domain feature space as a diffusion process to simulate attack noise and obtain the mapped radar semantic features.

[0025] A scoring function is fitted using a semantic mapping network, and the scoring function is used to iteratively perform a reverse denoising mapping operation on the mapped radar semantic features. During the reverse denoising mapping operation, Wiener process noise is injected to destroy the adversarial perturbation structure and obtain the initial radar target domain features.

[0026] Based on the initial noise sampled by Gaussian distribution, the semantic mapping network is used to iteratively transfer the knowledge of the initial radar target domain features to obtain the radar target domain features.

[0027] According to an embodiment of the present invention, the isolation mechanism against noise disturbance is constructed in the latent imaging space. The feature space codec network is used to perform multiple gradient discretization processes on the radar target domain features, and the radar target domain features are mapped to the latent imaging space. The obtained imaging image of the target scene includes:

[0028] The radar target domain features are encoded and vector quantized using the feature space encoder-decoder network to obtain a low-dimensional continuous latent vector.

[0029] By constraining the distribution of the latent imaging space through variational inference and KL divergence constraints, a latent imaging space with isolation mechanism is obtained;

[0030] A low-dimensional continuous latent vector is subjected to a quadratic gradient discretization process in a latent imaging space with an isolation mechanism to obtain a result of the quadratic gradient discretization process;

[0031] The decoder of the feature space encoder-decoder network is used to perform multi-layer deconvolution reconstruction operations on the results of the secondary gradient discretization processing to obtain an imaging map of the target scene.

[0032] According to a second aspect of the present invention, a multi-neural network training method is provided, comprising:

[0033] A two-stream neural network is iteratively trained in a self-supervised manner using a mean square error loss function, range angle map samples, and range Doppler map samples to obtain a trained two-stream neural network, wherein the trained two-stream neural network is applied to the above-mentioned millimeter-wave radar anti-attack imaging method;

[0034] A semantic mapping network driven by a diffusion model is self-supervised and iteratively trained using an expected estimation loss function and radar semantic feature samples output by a trained two-stream neural network to obtain a trained semantic mapping network. The trained semantic mapping network is then applied to the millimeter-wave radar anti-attack imaging method.

[0035] The feature space codec network based on the discrete variational codec is iteratively trained in a self-supervised manner using the reconstruction loss function, the codebook constraint loss function, and the radar target domain feature samples output by the trained semantic mapping network to obtain the trained feature space codec network. The trained feature space codec network is applied to the above-mentioned millimeter wave radar anti-attack imaging method.

[0036] According to a third aspect of the present invention, there is provided a millimeter wave radar anti-attack imaging system applied to the above-mentioned millimeter wave radar anti-attack imaging method, comprising:

[0037] The radar data image acquisition module is used to process the millimeter-wave radar's transmission signal and the target scene's echo signal to obtain an intermediate frequency signal, and then perform multi-dimensional fast Fourier transform processing on the intermediate frequency signal to obtain a range angle map and a range Doppler map;

[0038] The radar semantic feature acquisition module is used to extract physical layer features from the range angle map and the range Doppler map simultaneously using a dual-stream neural network, and aggregate the extracted range angle physical layer features and range Doppler physical layer features to obtain radar semantic features;

[0039] The radar target domain feature acquisition module is used to map the radar semantic features to the pre-constructed target domain feature space using the semantic mapping network and remove the attack noise in the radar semantic features to obtain the radar target domain features after knowledge transfer;

[0040] The target scene imaging module is used to build an isolation mechanism against noise disturbances in the latent imaging space. It uses the feature space encoder-decoder network to perform multiple gradient discretization processes on the radar target domain features, maps the radar target domain features to the latent imaging space, and obtains the imaging map of the target scene.

[0041] A fourth aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0042] The fifth aspect of the present invention further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the computer program or instructions are executed by a processor.

[0043] The above-mentioned millimeter-wave radar anti-attack imaging method provided by the present invention processes the transmission signal of the millimeter-wave radar and the echo information of the target scene to obtain an intermediate frequency signal, and performs feature extraction, semantic mapping and encoding and decoding on the intermediate frequency information through multiple neural networks to obtain an image of the target scene that is resistant to attack interference, thereby realizing fine-grained high-precision imaging of the target scene; at the same time, due to the use of multiple neural networks for collaborative processing, the generalization of the present invention is improved, and anti-interference imaging of various types of complex scenes can be performed; in addition, the present invention combines deep learning technology, adversarial purification technology and millimeter-wave radar imaging technology, which can accurately image spatial object information at a fine-grained level, is not easily interfered with by the attacker's attack algorithm, and achieves high-precision and high-robust imaging effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0045] Figure 1 2. This is a diagram of an application scenario of the millimeter wave radar anti-attack imaging method according to an embodiment of the present invention;

[0046] Figure 2 is a flowchart of a millimeter wave radar anti-attack imaging method according to an embodiment of the present invention;

[0047] Figure 3 is an architectural diagram of a millimeter-wave radar anti-attack imaging method according to an embodiment of the present invention;

[0048] Figure 4 2 is a schematic diagram of data processing of a millimeter wave radar anti-attack imaging method according to an embodiment of the present invention;

[0049] Figure 5 is a flow chart of a multi-neural network training method according to an embodiment of the present invention;

[0050] Figure 6 2 is a schematic diagram of imaging effects of the millimeter wave radar anti-attack imaging method according to an embodiment of the present invention when applied to different scenarios;

[0051] Figure 7 2. It is a schematic diagram of the defense effect of the millimeter wave radar anti-attack imaging method and other imaging algorithms under attack according to an embodiment of the present invention;

[0052] Figure 8 3. This is a schematic diagram showing the effect of a millimeter wave radar anti-attack imaging method according to an embodiment of the present invention being attacked by a fast symbol method with different intensities;

[0053] Figure 9 is a structural block diagram of a millimeter wave radar anti-attack imaging system according to an embodiment of the present invention;

[0054] Figure 10 4 is a block diagram of an electronic device suitable for implementing a millimeter wave radar anti-attack imaging method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concept of the present invention.

[0056] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise," "include," etc. used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0057] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0058] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0059] In recent years, imaging technology based on millimeter-wave radar signals has rapidly developed, showing significant application prospects in multiple fields. For example, in the field of autonomous driving, millimeter-wave radar perception imaging technology is an essential component of advanced driver assistance systems. It can effectively distinguish objects at close range, detect the spatial extension of traffic participants, and enhance object recognition, thereby effectively reducing human driving errors. In the field of cyberspace security, millimeter-wave radar imaging technology, with its all-weather, high-precision physical space perception characteristics, improves environmental perception capabilities, effectively filling the blind spots of traditional network security technologies in detecting physical layer threats, and is a key support for building active defenses.

[0060] However, with the increasing popularity of millimeter-wave radar imaging systems, millimeter-wave radar imaging technology faces new challenges. Multipath propagation effects in millimeter-wave radar signals cause coherent interference in complex electromagnetic environments, severely impacting target feature extraction accuracy. Limited by existing array apertures and beamforming algorithms, achieving sub-centimeter spatial resolution is difficult. It is also worth noting that research on millimeter-wave radar defense mechanisms against countermeasures is still in its infancy, and a comprehensive theoretical framework and security assessment system are urgently needed. Therefore, improving millimeter-wave radar imaging quality and its ability to resist countermeasures has become particularly important.

[0061] While some existing deep learning-based millimeter-wave radar super-resolution imaging algorithms exist, they typically only address multipath imaging and struggle with complex indoor environments and fine-grained perception. The recent rise of adversarial purification technology has achieved significant results in deep learning, demonstrating its advantages in handling both black-box and white-box attacks using unfamiliar attack modalities. However, applying this technology to millimeter-wave radar imaging remains a work in progress.

[0062] In order to solve at least one of the problems of the existing technology, the present invention provides a millimeter-wave radar anti-attack imaging method, which combines deep learning technology, adversarial purification technology and millimeter-wave radar imaging technology to achieve high-quality, high-precision and high-robust millimeter-wave radar imaging; at the same time, the method provided by the present invention can accurately image spatial object information at a fine-grained level, is not easily interfered with by the attacker's attack algorithm, and achieves high-precision and high-robust imaging effects.

[0063] The millimeter wave radar anti-attack imaging method provided by the present invention is described in detail below through specific embodiments or implementation methods in conjunction with the accompanying drawings.

[0064] Figure 1 This is a diagram of an application scenario of the millimeter wave radar anti-attack imaging method according to an embodiment of the present invention.

[0065] like Figure 1As shown, the application scenario 100 according to this embodiment may include scenarios such as radar signal processing and artificial intelligence. The network 104 is used as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0066] A user may use a first terminal device 101, a second terminal device 102, or a third terminal device 103 to interact with a server 105 via a network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, or the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (for example only).

[0067] The first terminal device 101 , the second terminal device 102 , and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0068] The server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process received data such as user requests, and feed back processing results (e.g., web pages, information, or data obtained or generated based on user requests) to the terminal devices.

[0069] It should be noted that the millimeter-wave radar anti-attack imaging method provided in the embodiments of the present invention can generally be executed by the server 105. Accordingly, the millimeter-wave radar anti-attack imaging system provided in the embodiments of the present invention can generally be set in the server 105. The millimeter-wave radar anti-attack imaging method provided in the embodiments of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the millimeter-wave radar anti-attack imaging system provided in the embodiments of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0070] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0071] The following will be based on Figure 1 The scene described by Figures 2 to 8 The millimeter wave radar anti-attack imaging method of the disclosed embodiment is described in detail.

[0072] Figure 2 4 is a flowchart of a millimeter wave radar anti-attack imaging method according to an embodiment of the present invention.

[0073] like Figure 2 As shown, the above-mentioned millimeter wave radar anti-attack imaging method includes operations S210 to S240.

[0074] In operation S210, the transmission signal of the millimeter wave radar and the echo signal of the target scene are processed to obtain an intermediate frequency signal, and the intermediate frequency signal is subjected to multi-dimensional fast Fourier transform processing to obtain a range angle map and a range Doppler map.

[0075] In operation S220, a dual-stream neural network is used to simultaneously extract physical layer features from the range angle map and the range Doppler map, and information aggregation processing is performed on the extracted range angle physical layer features and range Doppler physical layer features to obtain radar semantic features.

[0076] In the present invention, the two-stream neural network is used to extract semantic information. The two-stream neural network may also be referred to as a semantic information extraction network in subsequent specific embodiments, implementation methods or experiments of the present invention.

[0077] In operation S230 , the radar semantic features are mapped to a pre-constructed target domain feature space using a semantic mapping network and attack noise in the radar semantic features is removed to obtain radar target domain features after knowledge transfer.

[0078] In operation S240, an isolation mechanism against noise disturbance is constructed in the latent imaging space, and the radar target domain features are subjected to multiple gradient discretization processes using a feature space codec network. The radar target domain features are mapped to the latent imaging space to obtain an imaging image of the target scene.

[0079] The above-mentioned millimeter-wave radar anti-attack imaging method provided by the present invention processes the transmission signal of the millimeter-wave radar and the echo information of the target scene to obtain an intermediate frequency signal, and performs feature extraction, semantic mapping and encoding and decoding on the intermediate frequency information through multiple neural networks to obtain an image of the target scene that is resistant to attack interference, thereby realizing fine-grained high-precision imaging of the target scene; at the same time, due to the use of multiple neural networks for collaborative processing, the generalization of the present invention is improved, and anti-interference imaging of various types of complex scenes can be performed; in addition, the present invention combines deep learning technology, adversarial purification technology and millimeter-wave radar imaging technology, which can accurately image spatial object information at a fine-grained level, is not easily interfered with by the attacker's attack algorithm, and achieves high-precision and high-robust imaging effects.

[0080] The following is a specific implementation method and combined with the attached Figure 3 The millimeter wave radar anti-attack imaging method provided by the present invention is further described in detail.

[0081] Figure 3 4 is an architectural diagram of a millimeter-wave radar anti-attack imaging method according to an embodiment of the present invention.

[0082] The millimeter-wave radar anti-attack imaging method provided by the present invention is based on the data of the millimeter-wave radar itself, designs a radar transferable feature learning model based on a self-supervised learning strategy to extract semantic features in the millimeter-wave radar data, designs a semantic mapping network with anti-attack capabilities to realize the knowledge transfer of the semantic features of the millimeter-wave radar, smoothes the attack information through a stochastic differential equation algorithm, and designs a discrete variational inference decoder to realize the homeomorphic mapping from the feature space to the imaging space. Through this method, the present invention can achieve high-resolution imaging of the millimeter-wave radar, and make the model have good anti-attack capabilities, and achieve excellent imaging effects in a variety of scenarios. The innovation of this patent is reflected in the deep embedding of semantic perception into the radar signal processing link, and the construction of a closed-loop defense mechanism of "physical layer feature extraction-semantic space mapping-anti-noise suppression".

[0083] like Figure 3 As shown in the figure, the millimeter-wave radar transmits radar signals to the target scene and receives the echo signals from the target scene, thereby perceiving the spatial information of the target scene; and performs a series of processing on the intermediate frequency signal through RN (two-stream neural network or semantic information extraction network, the same below), F (semantic mapping network) and D (feature space codec), finally obtaining a high-resolution, high-quality point cloud image of the target scene.

[0084] According to an embodiment of the present invention, the above-mentioned processing of the transmission signal of the millimeter-wave radar and the echo signal of the target scene to obtain the intermediate frequency signal includes: based on the principle of frequency-modulated continuous wave radar, setting the transmission signal of the millimeter-wave radar to a linear frequency-modulated wave, and receiving the echo signal of the target scene through the millimeter-wave radar; performing complex conjugate mixing processing on the linear frequency-modulated wave and the echo signal to obtain a complex conjugate mixed signal; and using a preset mixer to perform complex conjugate multiplication operations, low-pass filtering operations, and orthogonal down-conversion operations on the complex conjugate mixed signal to obtain an intermediate frequency signal.

[0085] According to an embodiment of the present invention, the above-mentioned multi-dimensional fast Fourier transform processing of the intermediate frequency signal to obtain the distance angle diagram and the range Doppler diagram includes: performing fast Fourier transform on the multi-channel intermediate frequency signal along the time dimension to obtain the distance information of the target scene; performing fast Fourier transform on the multi-channel intermediate frequency signal along the spatial dimension to obtain the angle spectrum of the target scene; performing fast Fourier transform on the slow time dimension signal of multiple consecutive frequency modulation cycles to obtain the speed information of the target scene; selecting the distance information and angle spectrum of the target scene to obtain the distance angle diagram, and selecting the distance information and speed information of the target scene to obtain the range Doppler diagram.

[0086] The process of obtaining the range angle map and the range Doppler map is further described in detail below through specific implementation methods.

[0087] Before performing millimeter-wave radar anti-attack imaging, it is necessary to generate a range angle map and a range Doppler map associated with the millimeter-wave radar transmit signal and the received signal.

[0088] The range angle diagram and range Doppler diagram are the preliminary spatial information obtained by analyzing radar data. They are the response of the radar imaging system to the point source. Based on the principle of frequency modulated continuous wave (FMCW) radar, the transmitted signal is assumed to be is a linear frequency modulation wave, as shown in formula (1):

[0089] (1),

[0090] in, represents the amplitude of the transmitted signal, is the carrier frequency, is the frequency modulation slope, is the bandwidth, is the frequency modulation period, Indicates time, Indicates imaginary unit. Target reflected signal Delayed After receiving, represents the distance from the target to the millimeter-wave radar, and the target reflection signal is shown in formula (2):

[0091] (2),

[0092] in, represents the amplitude of the received signal, Indicates carrier frequency, Indicates time, represents the imaginary unit, Indicates the frequency modulation slope; the received signal is mixed with the complex conjugate of the transmitted signal to obtain the intermediate frequency signal through the mixer , as shown in formula (3):

[0093] (3),

[0094] in, represents the amplitude of the intermediate frequency signal, Indicates carrier frequency, Indicates time, represents the imaginary unit, Indicates the frequency modulation slope.

[0095] Perform FFT on the multi-channel intermediate frequency signal along the fast time dimension to extract distance information , as shown in formula (4):

[0096] (4),

[0097] in, represents the peak frequency related to the target distance, represents the speed of light, Indicates the frequency modulation slope.

[0098] Then beamforming or FFT is performed along the spatial dimension (antenna array) to obtain the angular spectrum , as shown in formula (5):

[0099] (5),

[0100] in, represents the wavelength of the millimeter-wave radar signal, represents the phase difference between the antennas, Indicates the antenna element spacing.

[0101] For continuous Perform FFT on the slow time dimension signal of the FM cycle to analyze the target speed , as shown in formula (6):

[0102] (6),

[0103] in, represents the wavelength of the millimeter-wave radar signal, Indicates the Doppler shift caused by the relative motion between the target and the radar.

[0104] This operation outputs a feature map containing the target distance, angle, and speed. Selecting the distance-angle dimension generates a range-angle map (RA), and selecting the distance-speed dimension generates a range-Doppler map (RD), providing basic input for subsequent anti-attack algorithms.

[0105] According to an embodiment of the present invention, the above-mentioned use of a dual-stream neural network to simultaneously extract physical layer features from the range angle map and the range Doppler map, and performing information aggregation processing on the extracted distance angle physical layer features and range Doppler physical layer features to obtain radar semantic features includes: using the first physical layer feature extraction stream of the dual-stream neural network based on self-supervised learning to perform pooling or strided convolution operations on the range angle map to obtain distance angle physical layer features with contextual information; using the second physical layer feature extraction stream of the dual-stream neural network based on self-supervised learning to perform pooling or strided convolution operations on the range Doppler map to obtain range Doppler physical layer features with contextual information; and performing multimodal information aggregation processing on the range angle physical layer features and the range Doppler physical layer features based on a self-attention mechanism to obtain radar semantic features.

[0106] The following is a specific implementation method and combined with the attached Figure 4 The process of extracting radar semantic features using a two-stream neural network (RN, or voice information extraction network) is further explained in detail.

[0107] Figure 4 3 is a schematic diagram of data processing of a millimeter wave radar anti-attack imaging method according to an embodiment of the present invention.

[0108] Establish a millimeter wave radar semantic information extraction network RN. In order to effectively understand the semantic information of the imaging physical space, the present invention needs to extract high-dimensional semantic features based on the input of the millimeter wave radar, and convert the millimeter wave radar space into Mapping to millimeter-wave radar semantic features ,in and Much smaller than and , and express Dimensions, and express and The present invention designs a two-stream neural network design strategy based on self-supervision, such as Figure 4The RN architecture shown in the figure shows that the range angle map and the range Doppler map are each extracted using a separate feature extraction chain. At the same time, the image size is compressed using a downsampling layer to enhance the network's receptive field, enabling the network to capture a wider range of contextual information and improving feature extraction accuracy. Given the differences in the feature spaces of the range angle map and the range Doppler map, it is difficult to complement and fuse their information. Therefore, an information aggregation strategy is introduced to aggregate the low-dimensional, high-frequency features extracted from the range angle map and the range Doppler map in the channel dimension, and a self-attention mechanism is used to enhance feature representation.

[0109] According to an embodiment of the present invention, the above-mentioned use of a semantic mapping network to map radar semantic features to a pre-constructed target domain feature space and remove attack noise from the radar semantic features to obtain radar target domain features after knowledge transfer includes: using the semantic mapping network to model the mapping path between the radar semantic features and the pre-constructed target domain feature space as a diffusion process to simulate the attack noise, thereby obtaining the mapped radar semantic features; using the semantic mapping network to fit a scoring function, and using the scoring function to iteratively perform a reverse denoising mapping operation on the mapped radar semantic features, and injecting Wiener process noise during the reverse denoising mapping operation to destroy the adversarial perturbation structure to obtain initial radar target domain features; based on the initial noise sampled from a Gaussian distribution, using the semantic mapping network to iteratively perform knowledge transfer on the initial radar target domain features to obtain the radar target domain features.

[0110] The following is a further detailed description of the acquisition process of the above radar target domain features through specific implementation methods.

[0111] Establish a semantic mapping network F. The design of the semantic mapping network F is based on the consideration of the large difference in the semantic space and the target feature space domain modality and the defense against attack noise. The basic principle of the diffusion model is used to implement the semantic mapping network. Construct the target domain feature space After (where and Represents the target domain feature space dimension), hoping to achieve this through a semantic mapping network arrive knowledge transfer, i.e. , and in this migration process, remove the attacker The attack noise added in . The overall mapping logic is shown in formula (7):

[0112] (7),

[0113] in, represents the Wiener process, Indicates the state of the target feature space mapped by the semantic mapping network at a certain moment, represents the fixed noise parameter in the forward process of the diffusion model, It is fitting The derivative of the distribution, The amplitude of the diffusion term (i.e., random noise) is used to control the change over time and is fitted by iteratively using the semantic mapping network F. Perfect mapping is achieved. This is the process of achieving generation-by-generation convergence, and eventually the network will be stable and proficient. space, The function of the term is to defend against attack perturbations by smoothing them out by adding random perturbations at each iteration. Therefore, after modeling the semantic space of the millimeter-wave radar and the feature space of the target domain through the semantic mapping network F, it is only necessary to sample from the Gaussian space and gradually iteratively update and migrate it to the target domain using the semantic mapping network.

[0114] According to an embodiment of the present invention, the above-mentioned isolation mechanism for resisting noise disturbance is constructed in the latent imaging space, and the radar target domain features are subjected to multiple gradient discretization processes using a feature space codec network, and the radar target domain features are mapped to the latent imaging space to obtain an imaging image of the target scene. The method includes: using the feature space codec network to perform encoding operations and vector quantization compression operations on the radar target domain features to obtain a low-dimensional continuous latent vector; constraining the distribution of the latent imaging space through variational inference and KL divergence constraints to obtain a latent imaging space with an isolation mechanism; performing secondary gradient discretization processing on the low-dimensional continuous latent vector in the latent imaging space with the isolation mechanism to obtain a result of the secondary gradient discretization processing; and performing a multi-layer deconvolution reconstruction operation on the result of the secondary gradient discretization processing using a decoder of the feature space codec network to obtain an imaging image of the target scene.

[0115] The following is a detailed description of the process of acquiring radar target domain features through specific implementation methods.

[0116] A feature space encoder-decoder network D is established. This network maps the target domain feature space to the final imaging space. This architecture relies on a feature space encoder-decoder structure based on the discrete variational encoder-decoder (DVAE) framework. This architecture establishes a robust mapping relationship between the discrete encoder and the target domain feature space, and the decoder. First, the encoder compresses the input features into a low-dimensional latent vector. It constrains the latent space distribution through variational inference and performs discretization operations to ensure the generalizability of the feature representation and the discreteness of the target domain space. The decoder reconstructs the high-resolution imaging result through multi-layer deconvolution. This network incorporates a KL divergence constraint and a joint optimization of the reconstruction loss to establish an isolation mechanism in the latent space that is resistant to noise perturbations. In use, the target domain features obtained by the semantic mapping network are discretized using a quadratic gradient technique to block the propagation path of attack noise in the decoding chain, significantly reducing the impact of typical attacks such as the fast gradient sign method, and improving the stability and reliability of the imaging system in adversarial environments.

[0117] Figure 5 4 is a flowchart of a multi-neural network training method according to an embodiment of the present invention.

[0118] like Figure 5 As shown, the above-mentioned multi-neural network training method includes operations S510 to S530.

[0119] In operation S510, the two-stream neural network is iteratively trained in a self-supervised manner using a mean square error loss function, range angle map samples, and range Doppler map samples to obtain a trained two-stream neural network, wherein the trained two-stream neural network is applied to the above-mentioned millimeter wave radar anti-attack imaging method.

[0120] In operation S520, a semantic mapping network driven by a diffusion model is self-supervised iteratively trained using an expected estimation loss function and radar semantic feature samples output by the trained dual-stream neural network to obtain a trained semantic mapping network, wherein the trained semantic mapping network is applied to the above-mentioned millimeter-wave radar anti-attack imaging method.

[0121] In operation S530, a feature space codec network based on a discrete variational codec is iteratively trained in a self-supervised manner using a reconstruction loss function, a codebook constraint loss function, and radar target domain feature samples output by the trained semantic mapping network to obtain a trained feature space codec network. The trained feature space codec network is applied to the above-mentioned millimeter wave radar anti-attack imaging method.

[0122] The following is a further detailed description of the training process of the multiple neural networks, namely the neural networks RN, F and D, through a specific implementation method.

[0123] RN, F, and D are trained online. This method uses self-supervised learning (SSL) to optimize the RN network, which enables this method to optimize millimeter-wave radar semantic extraction without labeled data. Mean square error loss function It consists of two components, as shown in formula (8):

[0124] (8),

[0125] in, represents the distance angle graph, represents the range Doppler map, represents the range Doppler map recovered by the semantic decoder, represents the distance angle map recovered by the semantic decoder, Represents the square of the L2 norm. The semantic decoder is only used in training and is not used in practical applications. For the training of F, the expectation estimation algorithm is used, and the training loss function of the semantic mapping network is As shown in formula (9):

[0126] (9),

[0127] in, Represents the F network, the input is the current iteration time Target domain features , number of iterations , and millimeter-wave radar semantic features , Represents the original, noise-free target domain features. This loss function ensures that the F network converges to the target domain feature space in a probabilistic manner. ,because to The first step is decoded by stochastic differential equations, so the adversarial noise can be smoothed, and the derivative of the Wiener distribution is Gaussian distribution with an expectation of 0, so it will not affect the convergence process of network training and ensure its stability and feasibility.

[0128] Finally, the self-supervision strategy is also adopted for the training of the target domain feature space, and the training codec weight parameters are frozen, and the training loss of the feature space codec is As shown in formula (10):

[0129] (10),

[0130] in, is the imaging loss function, which represents the restored target domain imaging map, Is a semantic feature loss function used to constrain continuous features and quantized discrete features As close as possible, represents the real picture of the original target domain, Represents the weight ratio between different loss functions, Represents the square of the L2 norm.

[0131] The training process of the present invention was performed on a workstation equipped with an NVIDIA GeForce RTX A100 GPU and implemented using the PyTorch deep learning framework. The present invention used the AdamW optimizer, with an initial learning rate set to 5e-6 and a cosine annealing strategy for learning rate adjustment. The optimizer momentum was set to 0.99, the exponential moving average (EMA) parameter was set to 0.9999, and the L2 regularization strategy was selected with its parameter set to 0.0002. Mixed precision training was disabled, and training was performed on both datasets for 100,000 steps. The iteration step size of the training set was set to 1000; in practical applications, the iteration step size can be 200 steps.

[0132] The following experiments and combined with the Figures 6-8 Verify the effectiveness of the method provided by the present invention in imaging and anti-attack.

[0133] Figure 6 2 is a schematic diagram of imaging effects when the millimeter wave radar anti-attack imaging method according to an embodiment of the present invention is applied in different scenarios.

[0134] Figure 7 3. It is a schematic diagram of the defense effect of the millimeter wave radar anti-attack imaging method and other imaging algorithms under attack according to an embodiment of the present invention.

[0135] Figure 8 3 is a schematic diagram showing the effect of the millimeter wave radar anti-attack imaging method according to an embodiment of the present invention being attacked by fast symbol methods with different intensities.

[0136] Verify the imaging effect and anti-attack effect: The imaging effect of the present invention is verified by comparing the imaging results of the input millimeter-wave radar signal processed by this algorithm with imaging using other algorithms, and the anti-attack effect of the present invention is verified by comparing the imaging performance of the present invention before and after the attack algorithm.

[0137] Experiment 1: Experiment 1 is used to verify the effect of the present invention on high-resolution imaging of millimeter-wave radar signals. Figure 6 As shown, this example applies the inventive method to multiple imaging scenarios. Figure 6 From left to right are the imaging images of three other imaging algorithms and the algorithm of the present invention, as well as an imaging reference image, ie, a target image. Figure 6 There are five sub-figures, marked as (a), (b), (c), (d) and (e), showing the different imaging effects of different imaging algorithms.

[0138] Figure 6 (a) Imaging results of other imaging algorithm 1: From top to bottom are the imaging results of three different scenes based on other algorithm 1. This algorithm is a discriminative model. Compared with the reference image, it has a better ability to capture structure, but poor performance in details.

[0139] Figure 6 (b) Imaging results of other imaging algorithm 2: From top to bottom are the imaging results of three different scenes based on other imaging algorithm 2. This algorithm is a generative model. Compared with the reference image, it can capture certain details but cannot obtain the overall structural information.

[0140] Figure 6 (c) Imaging results of other imaging algorithm 3: From top to bottom are the imaging results of three different scenes based on other imaging algorithms 3. Compared with other imaging algorithms 1 and 2, this algorithm combines the advantages of discriminative models and generative models, achieving a certain balance between structure and information, but there is still the problem of information loss.

[0141] Figure 6 (d) Imaging results of the algorithm proposed in this paper: From top to bottom, the imaging results of three different scenes using the algorithm proposed in this paper are shown. Compared with other imaging algorithms and the reference image (e), the imaging results of the algorithm proposed in this paper are superior in terms of structure, level of detail, and ability to utilize all information captured by the radar. It achieves optimal utilization of all radar information. This demonstrates that the algorithm designed in this paper successfully improves millimeter-wave radar imaging quality and achieves generalized imaging in a variety of scenes.

[0142] Figure 6 (e) Imaging algorithm reference image: This image shows the target reference images of the three scenes, which are point cloud images generated using scene information collected by a high-precision sensor lidar.

[0143] The implementation process of Experiment 1 followed the steps described in the Summary of the Invention, namely:

[0144] (1) The input range angle map and range Doppler map suitable for the millimeter wave radar system used in this experiment were generated offline.

[0145] (2) A millimeter-wave radar semantic information extraction network RN was established, and a pre-trained self-supervised learning method was adopted to optimize the network. The loss function was used to accurately extract the semantic information of the millimeter-wave radar.

[0146] (3) A semantic mapping network F is established, which is a generation-by-generation convergence strategy that maps the millimeter-wave radar semantic space to the target domain feature space through multiple iterations.

[0147] (4) The lidar decoder D1 in the feature space encoder-decoder is constructed to establish a differentiable relationship between the target domain feature space and the final imaging result L.

[0148] (5) During the online training phase, the present invention adopts the expected regression calculation strategy to update the iteration F, adopts the self-supervision strategy to train D1, and freezes the parameters of D1.

[0149] (6) Based on the training results, the present invention obtains semantic features through RN , using F to Map to , and finally decoded to L through D1.

[0150] (7) The imaging effect was verified through comparative experiments.

[0151] By comparison Figure 6 (a) in Figure 6 (b) Figure 6 (c) and Figure 6 In the four images (d), we can clearly see the imaging effect of the present invention. Radar images obtained by other imaging algorithms or lack of detailed information ( Figure 6 (a) in ), or lack of structural information ( Figure 6 (b) in ), or with information loss ( Figure 6 (c) in the figure), and after using the millimeter-wave radar imaging algorithm designed by the present invention, the millimeter-wave radar imaging result has better structural detail information and has the least information loss ( Figure 6 (d) in the figure), the imaging results are different from the reference imaging results ( Figure 6 The results are very similar to those in (e), which verifies the effectiveness of the present invention.

[0152] Experiment 2: Experiment 2 is used to verify the defense effect of the present invention against the attack algorithm. Figure 7 As shown, this example applies the inventive method to a defense task against an attack algorithm, and compares the effects of other imaging algorithms facing the same attack algorithm. Figure 7 There are five subgraphs in Figure 7 (a), (b), (c), (d) and (e) in the figure show the effect of the attack algorithm on the algorithm provided by the present invention and other algorithms.

[0153] Figure 7 (a) Other algorithms in 1 are not vulnerable to attacks: Figure 7 (a) shows the imaging result of other algorithms 1, which is in a non-attacked state. Figure 7 Compared with (e) in the figure, it can capture certain structural information.

[0154] Figure 7 (b) The algorithm of the present invention is not attacked: Figure 7 Panel (b) shows the imaging results of the proposed algorithm under a millimeter-wave radar system, when it is not attacked. As can be seen from the figure, spatial contours are clear and spatial information is well displayed. Thanks to the proposed imaging algorithm, structures in space are highly visible and recognizable under the radar system.

[0155] Figure 7 (c) Other algorithms in 1 are attacked: Figure 7 (c) in the figure shows the imaging results of other algorithms 1 after being attacked by the fast gradient sign method. Figure 7 Compared with (b) in the figure, the model imaging performance completely collapses and the structural information is completely lost compared to the imaging when not attacked, which proves the fragility of the millimeter-wave radar imaging model structure proposed in existing research.

[0156] Figure 7 (d) After the algorithm of the present invention is attacked: Figure 7 Panel (d) shows the imaging result of the proposed algorithm after the fast signed gradient method attack. Compared with the original imaging result, the original imaging information is better preserved. This proves that the model structure with purification defense designed by the proposed algorithm has strong resistance to model attacks and is very robust.

[0157] Figure 7 (e) Imaging algorithm reference diagram: Figure 7 (e) in the figure shows the point cloud image generated by using the current scene information collected by the high-precision sensor lidar.

[0158] In addition to following the first six steps described in the content of the invention, including offline generation of range angle maps and range Doppler maps, establishment of a semantic information extraction network RN, establishment of a semantic mapping network F, establishment of a target domain space decoder D2 in the feature space codec, online training of the network, and generation of imaging results, the implementation process of this example also adds a fast symbol method to attack the imaging results and verify the defense results.

[0159] By comparison Figure 7 Images (a), (b), (c), and (d) clearly demonstrate the effectiveness of the present invention in resisting attacks. This example demonstrates the effectiveness of the present invention in resisting model attack algorithms, which have little impact on the model's imaging performance and demonstrate high robustness.

[0160] Example 3: Experiment 3 verifies the point cloud performance change diagram after the FGSM (Fast GSM) attack of different intensities of the present invention. The attack intensity increases from left to right, as shown in FIG. Figure 8As shown in the figure, the changes in the point cloud evaluation indicators CD, EMD, and HD show that the point cloud distance loss of the algorithm of the present invention has not increased significantly and remains at a low value, which further proves the effectiveness of the algorithm of the present invention in defending against model attacks.

[0161] Based on the above experiments 1 to 3, it can be seen that the semantic-based millimeter radar anti-attack imaging method proposed in the present invention can effectively realize imaging in different scenarios and can effectively defend against model attacks of different intensities.

[0162] Experiments show that the present invention performs well in the fine-grained imaging and anti-attack imaging of millimeter-wave radar. For different scenarios, the imaging algorithm designed by the present invention can always effectively achieve fine-grained imaging and obtain information in space. For a variety of unfamiliar attack algorithms, it is possible to achieve adversarial purification of attack noise without knowing the attack method in advance. Compared with existing methods, the present invention has significant improvements in imaging effects and anti-attack capabilities. In particular, for typical adversarial attacks such as general adversarial perturbations, the anti-attack algorithm involved in the present invention can effectively reduce the damage of attack samples to imaging performance, making the attack algorithm invalid, verifying the wide applicability of the semantic-based millimeter-wave radar anti-attack algorithm.

[0163] In summary, through the semantic-based millimeter-wave radar anti-attack algorithm, the present invention can perform accurate imaging for a variety of complex and unfamiliar scenes, so that it can accurately obtain information about objects in space, while effectively resisting attacks from attackers and realizing full-domain imaging and robust imaging of millimeter-wave radar imaging.

[0164] Based on the above-mentioned millimeter wave radar anti-attack imaging method, the present invention also provides a millimeter wave radar anti-attack imaging system. Figure 9 The system is described in detail.

[0165] Figure 9 4 is a structural block diagram of a millimeter wave radar anti-attack imaging system according to an embodiment of the present invention.

[0166] like Figure 9 As shown, the above-mentioned millimeter wave radar anti-attack imaging system 900 includes a radar data image acquisition module 910, a radar semantic feature acquisition module 920, a radar target domain feature acquisition module 930 and a target scene imaging module 940.

[0167] The radar data image acquisition module 910 is used to process the transmission signal of the millimeter-wave radar and the echo signal of the target scene to obtain an intermediate frequency signal, and perform multi-dimensional fast Fourier transform processing on the intermediate frequency signal to obtain a range angle map and a range Doppler map. In one embodiment, the radar data image acquisition module 910 can be used to perform the operation S210 described above, which will not be repeated here.

[0168] The radar semantic feature acquisition module 920 is used to use a dual-stream neural network to simultaneously extract physical layer features from the range angle map and the range Doppler map, and to perform information aggregation processing on the extracted range angle physical layer features and range Doppler physical layer features to obtain radar semantic features. In one embodiment, the radar semantic feature acquisition module 920 can be used to perform the operation S220 described above, which will not be repeated here.

[0169] The radar target domain feature acquisition module 930 is used to map the radar semantic features to a pre-constructed target domain feature space using a semantic mapping network and remove attack noise from the radar semantic features to obtain the radar target domain features after knowledge transfer. In one embodiment, the radar target domain feature acquisition module 930 can be used to perform the operation S230 described above and will not be repeated here.

[0170] The target scene imaging module 940 is used to establish an isolation mechanism against noise disturbances in the latent imaging space, perform multiple gradient discretization processes on the radar target domain features using the feature space codec network, map the radar target domain features to the latent imaging space, and obtain an imaging image of the target scene. In one embodiment, the target scene imaging module 940 can be used to perform the operation S240 described above, which will not be repeated here.

[0171] According to embodiments of the present invention, any multiple modules among the radar data image acquisition module 910, radar semantic feature acquisition module 920, radar target domain feature acquisition module 930, and target scene imaging module 940 may be combined into a single module, or any one of these modules may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of other modules and implemented in a single module. According to embodiments of the present invention, at least one of the radar data image acquisition module 910, radar semantic feature acquisition module 920, radar target domain feature acquisition module 930, and target scene imaging module 940 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or may be implemented in hardware or firmware through any other suitable means of circuit integration or packaging, or implemented in any one of software, hardware, and firmware, or any suitable combination of these. Alternatively, at least one of the radar data image acquisition module 910, the radar semantic feature acquisition module 920, the radar target domain feature acquisition module 930, and the target scene imaging module 940 can be at least partially implemented as a computer program module, which can perform corresponding functions when executed.

[0172] Figure 10 4 is a block diagram of an electronic device suitable for implementing a millimeter wave radar anti-attack imaging method according to an embodiment of the present invention.

[0173] like Figure 10 As shown, an electronic device 1000 according to an embodiment of the present invention includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage unit 1008 into a random access memory (RAM) 1003. Processor 1001 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. Processor 1001 may also include onboard memory for caching purposes. Processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0174] RAM 1003 stores various programs and data required for the operation of electronic device 1000. Processor 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. Processor 1001 executes the programs in ROM 1002 and / or RAM 1003 to perform various operations according to the method flow of the embodiment of the present invention. It should be noted that the programs may also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 may also execute the programs stored in one or more memories to perform various operations according to the method flow of the embodiment of the present invention.

[0175] According to an embodiment of the present invention, electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to bus 1004. Electronic device 1000 may also include one or more of the following components connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 1008 including a hard disk; and a communication section 1009 including a network interface card such as a LAN card or modem. Communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 1010 as needed, so that computer programs read from the removable media can be installed into storage section 1008 as needed.

[0176] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0177] According to an embodiment of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium, and may include, for example, but not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present invention, a computer-readable storage medium may include ROM 1002 and / or RAM 1003 described above, and / or one or more memories other than ROM 1002 and RAM 1003.

[0178] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0179] It will be understood by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention may be combined and / or coupled in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or couplings fall within the scope of the present invention.

[0180] The above describes embodiments of the present invention. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A millimeter wave radar anti-attack imaging method, characterized in that: The method comprises: The transmission signal of the millimeter-wave radar and the echo signal of the target scene are processed to obtain an intermediate frequency signal, and the intermediate frequency signal is subjected to multi-dimensional fast Fourier transform processing to obtain a range angle map and a range Doppler map; A dual-stream neural network is used to simultaneously extract physical layer features from the range angle map and the range Doppler map, and the extracted range angle physical layer features and range Doppler physical layer features are aggregated to obtain radar semantic features. Mapping the radar semantic features to a pre-constructed target domain feature space using a semantic mapping network and removing attack noise from the radar semantic features to obtain radar target domain features after knowledge transfer; An isolation mechanism against noise disturbance is constructed in the latent imaging space, and the radar target domain features are subjected to multiple gradient discretization processes using a feature space codec network. The radar target domain features are mapped to the latent imaging space to obtain an imaging image of the target scene.

2. The method according to claim 1, characterized in that The millimeter-wave radar's transmitted signal and the target scene's echo signal are processed to obtain the intermediate frequency signal including: Based on the principle of frequency modulated continuous wave radar, the transmission signal of the millimeter wave radar is set to a linear frequency modulated wave, and the echo signal of the target scene is received by the millimeter wave radar; Performing complex conjugate mixing processing on the linear frequency modulation wave and the echo signal to obtain a complex conjugate mixing signal; A preset mixer is used to perform a complex conjugate multiplication operation, a low-pass filtering operation, and an orthogonal down-conversion operation on the complex conjugate mixed signal to obtain the intermediate frequency signal.

3. The method according to claim 2, characterized in that The intermediate frequency signal is subjected to a multi-dimensional fast Fourier transform process to obtain a range angle map and a range Doppler map, including: By performing fast Fourier transform on the multi-channel intermediate frequency signal along the time dimension, the distance information of the target scene is obtained; Obtaining an angular spectrum of the target scene by performing fast Fourier transform on the multi-channel intermediate frequency signal along the spatial dimension; Performing fast Fourier transform on the slow time dimension signal of multiple consecutive frequency modulation cycles to obtain speed information of the target scene; The distance information and angle spectrum of the target scene are selected to obtain the distance angle diagram, and the distance information and speed information of the target scene are selected to obtain the range Doppler diagram.

4. The method according to claim 1, wherein A dual-stream neural network is used to simultaneously extract physical layer features from the range angle map and the range Doppler map, and the extracted range angle physical layer features and range Doppler physical layer features are aggregated to obtain radar semantic features including: Performing pooling or strided convolution operations on the distance angle map using the first physical layer feature extraction stream of a two-stream neural network based on self-supervised learning to obtain distance angle physical layer features with contextual information; Performing a pooling or strided convolution operation on the range Doppler map using a second physical layer feature extraction stream of a two-stream neural network based on self-supervised learning to obtain a range Doppler physical layer feature with context information; The range angle physical layer feature and the range Doppler physical layer feature are subjected to multimodal information aggregation processing based on a self-attention mechanism to obtain the radar semantic feature.

5. The method according to claim 1, characterized in that The radar semantic features are mapped to a pre-constructed target domain feature space using a semantic mapping network and the attack noise in the radar semantic features is removed. The radar target domain features obtained after knowledge transfer include: Using the semantic mapping network, the mapping path between the radar semantic feature and the pre-constructed target domain feature space is modeled as a diffusion process to simulate the attack noise, thereby obtaining the mapped radar semantic feature; fitting a score function using the semantic mapping network, iteratively performing a reverse denoising mapping operation on the mapped radar semantic features using the score function, and injecting Wiener process noise during the reverse denoising mapping operation to destroy the adversarial perturbation structure, thereby obtaining an initial radar target domain feature; Based on the initial noise sampled by Gaussian distribution, the semantic mapping network is used to iteratively perform knowledge migration on the initial radar target domain features to obtain the radar target domain features.

6. The method according to claim 1, characterized in that An isolation mechanism against noise disturbance is constructed in the latent imaging space, and the radar target domain features are subjected to multiple gradient discretization processes using a feature space codec network. The radar target domain features are mapped to the latent imaging space, and an imaging image of the target scene is obtained, including: Using the feature space codec network to perform encoding operations and vector quantization compression operations on the radar target domain features to obtain a low-dimensional continuous latent vector; Constraining the distribution of the latent imaging space through variational inference and KL divergence constraints to obtain a latent imaging space with the isolation mechanism; performing a secondary gradient discretization process on the low-dimensional continuous latent vector in a latent imaging space having the isolation mechanism to obtain a result of the secondary gradient discretization process; A decoder of the feature space codec network is used to perform a multi-layer deconvolution reconstruction operation on the result of the secondary gradient discretization processing to obtain an imaging image of the target scene.

7. A method for training multiple neural networks, characterized in that: The method comprises: Performing self-supervised iterative training on the two-stream neural network using a mean square error loss function, a range angle map sample, and a range Doppler map sample to obtain a trained two-stream neural network, wherein the trained two-stream neural network is applied to the method according to any one of claims 1 to 6; A semantic mapping network driven by a diffusion model is self-supervisedly trained iteratively using an expected estimation loss function and radar semantic feature samples output by the trained dual-stream neural network to obtain a trained semantic mapping network, wherein the trained semantic mapping network is applied to the method according to any one of claims 1 to 6; A feature space codec network based on a discrete variational codec is self-supervisedly iteratively trained using a reconstruction loss function, a codebook constraint loss function, and radar target domain feature samples output by the trained semantic mapping network to obtain a trained feature space codec network, wherein the trained feature space codec network is applied to the method according to any one of claims 1 to 6.

8. A millimeter wave radar anti-attack imaging system applied to the method according to any one of claims 1 to 6, characterized in that: The system comprises: The radar data image acquisition module is used to process the transmission signal of the millimeter wave radar and the echo signal of the target scene to obtain an intermediate frequency signal, and perform multi-dimensional fast Fourier transform processing on the intermediate frequency signal to obtain a range angle map and a range Doppler map; a radar semantic feature acquisition module, configured to simultaneously extract physical layer features from the range angle graph and the range Doppler graph using a dual-stream neural network, and perform information aggregation processing on the extracted range angle physical layer features and range Doppler physical layer features to obtain radar semantic features; A radar target domain feature acquisition module is used to map the radar semantic features to a pre-constructed target domain feature space using a semantic mapping network and remove attack noise from the radar semantic features to obtain radar target domain features after knowledge transfer; The target scene imaging module is used to construct an isolation mechanism to combat noise disturbances in the latent imaging space, perform multiple gradient discretization processes on the radar target domain features using a feature space codec network, map the radar target domain features to the latent imaging space, and obtain an imaging image of the target scene.

9. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images