Radio frequency signal background noise elimination method based on multi-mode space-time decoupling

Abstract

The radio frequency signal background noise elimination method based on multi-mode space-time decoupling comprises the following steps: S1, collecting an original radio frequency reflection signal of a target object, and synchronously collecting a depth image containing scene information; s2, encoding the original radio frequency reflection signal and the depth image, and extracting the spatial-temporal characteristics of the original radio frequency reflection signal and the spatial-temporal characteristics of the depth image; s3, by taking the spatial-temporal characteristics of the depth image as guidance, aligning the spatial-temporal characteristics of the original radio frequency reflection signal with the spatial-temporal characteristics of the depth image through a multi-head cross attention mechanism to obtain multi-modal fusion characteristics; and S4, inputting the multi-modal fusion feature into a pure signal decoder module, enhancing the original radio frequency reflection signal through a predictive attention mask, and reconstructing a pure signal from the original radio frequency reflection signal in combination with residual connection to realize elimination of the background noise of the radio frequency signal. According to the invention, a multi-mode deep learning technology is adopted, and low-cost and high-efficiency radio frequency signal background noise elimination is realized.

Description

Technical Field

[0001] This invention relates to the field of wireless sensing and radio frequency signal processing technology, specifically to a method for eliminating background noise in radio frequency signals based on multimodal spatiotemporal decoupling. Background Technology

[0002] With the rapid development of IoT and AI technologies, wireless sensing technology has shown great application potential in many fields such as security imaging, logistics monitoring, human behavior recognition, and indoor positioning due to its advantages such as non-line-of-sight propagation capability, good privacy protection, and insensitivity to lighting conditions. The core of radio frequency sensing technology lies in obtaining information such as the position, material, or motion state of a target object by analyzing the reflection characteristics of radio frequency signals on the object's surface.

[0003] However, in practical applications, the propagation of radio frequency signals is highly susceptible to interference from environmental factors. When a radar transmits a signal to a target object, the signal received by the receiver not only contains the direct reflected signal from the target object but also inevitably overlaps with background reflected signals from the surrounding environment (such as the ground, walls, furniture, etc.). This overlap of target and environmental signals severely obscures the true characteristics of the target object, greatly limiting the robustness of radio frequency sensing technology in complex scenarios and significantly impacting the accuracy and reliability of subsequent sensing tasks.

[0004] In existing technologies, a common method for eliminating environmental interference is background subtraction, which involves pre-collecting an empty scene signal without target objects as a background reference and then subtracting it from the signal containing the target. However, this method assumes that the environment is completely static, cannot cope with dynamic environmental changes, and struggles to handle the complex secondary reflection problem between the target and the environment. This results in incomplete signal decoupling, and residual noise still affects the accuracy and reliability of various sensing tasks.

[0005] Therefore, how to effectively decouple a pure target signal from aliased signals with strong environmental interference is a key technical problem that urgently needs to be solved in the field of radio frequency sensing. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies, this invention provides a method, system, medium, and device for eliminating background noise in radio frequency signals based on multimodal spatiotemporal decoupling. This method employs multimodal deep learning technology to achieve low-cost and high-efficiency background noise elimination of radio frequency signals. It addresses the problem that existing radio frequency sensing technologies struggle to separate clean target signals from aliased signals with strong environmental interference in real-world, complex environments.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for eliminating background noise in radio frequency signals based on multimodal spatiotemporal decoupling includes the following steps: S1. Acquire the original radio frequency reflection signal of the target object using a single-transmitter single-receiver commercial pulse ultra-wideband (IR-UWB) radar, and simultaneously acquire a depth image containing scene information corresponding to the area where the target object is located; the original radio frequency reflection signal contains an aliasing component of the target object's radio frequency reflection signal and the environmental background radio frequency reflection signal. S2. Construct a dual-modal spatiotemporal coding network model for signal and image, encode the original radio frequency reflection signal and the depth image respectively, and extract the spatiotemporal features of the original radio frequency reflection signal and the spatiotemporal features of the depth image; S3. Construct intra-modal and cross-modal fusion modules. Guided by the spatiotemporal features of the depth image, the spatiotemporal features of the original radio frequency reflection signal and the spatiotemporal features of the depth image are fused through a multi-head cross-attention mechanism to obtain multimodal fusion features. S4. Construct a signal decoder and a composite loss function. Input the multimodal fusion features into the clean signal decoder module. Enhance the original radio frequency reflection signal by predicting the attention mask. Combined with residual connection, reconstruct the clean signal from the original radio frequency reflection signal to achieve radio frequency signal background noise elimination. In S1, the radar and depth camera are fixedly installed. The target object moves through the radar detection area with the conveyor belt. During this process, the radar continuously transmits detection pulses to the moving target object and receives the echo, thereby completing the acquisition of the original radio frequency reflection signal. The aliasing component is the superposition of the target object's radio frequency reflection signal and the environmental background radio frequency reflection signal in the radio frequency signal received by the radar; the target object's radio frequency reflection signal is used to extract the target object's spatial position and geometric structure; the environmental background radio frequency reflection signal is used to characterize irrelevant interference components in the scene, which are noise components to be eliminated. In the multimodal feature fusion stage, background noise is identified and eliminated by spatiotemporally aligning it with the environmental features of the depth image.

[0008] In S2, the signal-image dual-modal spatiotemporal coding network model adopts a feature extraction architecture based on a parallel convolutional neural network. The feature extraction architecture includes a single-modal encoder module that processes data of different modalities, namely a signal time encoder module, a signal space encoder module, a depth image time encoder module, and a depth image space encoder module, which are used to extract the spatiotemporal features of the original radio frequency reflection signal and the spatiotemporal features of the depth image, respectively.

[0009] The processing procedures of the signal time encoder module and the signal space encoder module are as follows: The original radio frequency reflection signal is divided into real and imaginary parts, which are then input into an encoder with a shared structure. The encoder consists of three one-dimensional convolutional sub-blocks, one residual sub-block, and one two-dimensional convolutional sub-block. The encoder is used to extract the spatiotemporal features of the original radio frequency reflection signal. and ; The processing steps of the depth image temporal encoder module and the depth image spatial encoder module are as follows: the depth image is input into three two-dimensional convolutional sub-blocks to extract depth features, and then features are extracted along the time dimension through two one-dimensional convolutional blocks to generate image spatiotemporal features. and .

[0010] In S3, the intra-modal and cross-modal fusion module is used to extract cross-modal spatial fusion features, cross-modal temporal fusion features, intra-modal signal features, and intra-modal image features; The intra-modal and cross-modal fusion module includes intra-modal fusion units and cross-modal fusion units, and its calculation process includes: For cross-modal fusion units in the spatial dimension, multi-head cross-attention (MCA) is used to compute spatial fusion features. The calculation process is as follows: based on the spatial characteristics of the signal As a query vector, the spatial features of the depth image As key and value vectors, multi-head cross-attention is used to compute the first attention term; based on deep image spatial features As a query vector, the spatial characteristics of the signal As key and value vectors, a second attention term is computed using multi-head cross-attention; the two attention terms are then summed with the deep image spatial features as residual connectivity terms. The formula for addition is: For cross-modal fusion units in the temporal dimension, multi-head cross-attention (MCA) is used to compute temporal fusion features. The calculation process is as follows: based on the signal time characteristics As a query vector, the temporal features of the depth image As key and value vectors, multi-head cross-attention is used to compute the first attention term; based on the temporal features of the depth image. As a query vector, the time characteristics of the signal As key and value vectors, a second attention term is computed using multi-head cross-attention; the two attention terms are then summed with the depth image temporal features as residual connectivity terms. The formula for addition is: The MCA operation is defined as follows: in, Represents the query matrix. Represents the key matrix, Represents a value matrix, The scaling factor representing the feature dimension. This represents the matrix transpose operation. It is a normalized exponential function; For intramodal fusion units, calculate intramodal features. and The calculation method is similar to that of cross-modal fusion, except that the attention calculation is performed on the same modality. The formulas are as follows: Adaptive weighting, calculating the final fused features : in, Defined as: .

[0011] In step S4, the processing procedure of the clean signal decoder module is as follows: Fusion features Perform two 2D convolutions and PReLU activations, then generate an attention mask using the Sigmoid function. ;The original input signal S is enhanced using a mask to obtain Simultaneously, convolution processing is performed on the original signal to obtain residual features. The final predicted pure signal for: In step S4, the loss function The calculation formula is: in, Cosine similarity loss over time: ; Cosine similarity loss for the distance dimension: ; Extreme value mean square error loss; ; This is a hyperparameter.

[0012] In step S4, during the network training phase, a clean signal based on a physical propagation model is used to generate a corresponding label. The composite loss function includes measures of the cosine similarity loss between the predicted clean RF signal and the true clean RF signal in the time and space dimensions, as well as the mean square error loss of the signal amplitude.

[0013] A radio frequency signal background noise cancellation system based on multimodal spatiotemporal decoupling includes a signal image encoding module, a feature fusion module, a signal decoding module, and a model training module. The signal image encoding module is used to extract the spatiotemporal features of the real and imaginary parts of the input radio frequency signal and the spatiotemporal features of the depth image through a multi-layer convolutional network. The feature fusion module is used to calculate cross-modal features and intra-modal features based on the spatiotemporal features of the signal and image extracted by the signal image encoding module, namely cross-modal spatial fusion features, cross-modal temporal fusion features, intra-modal signal features, and intra-modal image features. The module then performs weighted fusion of the above four features using adaptive weights to obtain the final fused features. The signal decoding module is used to generate a mask based on the fusion features and reconstruct a clean radio frequency signal of the target object; The model training module is used to train the network using a simulation dataset and optimize the model parameters using a joint loss function.

[0014] A computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform the steps of the above-described method for eliminating background noise of radio frequency signals based on multimodal spatiotemporal decoupling.

[0015] A computing device, comprising: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include steps for performing the above-described method for eliminating background noise of radio frequency signals based on multimodal spatiotemporal decoupling.

[0016] The beneficial effects of this invention are: This invention solves the fundamental problem of obtaining pure radio frequency (RF) signals unaffected by environmental interference as training labels in the real physical world. The scheme utilizes depth images to guide the RF signal and combines a multi-head cross-attention mechanism to align the RF signal's spatiotemporal features, achieving efficient decoupling of aliased signals in dynamic and complex environments. This significantly improves the signal-to-noise ratio and robustness of RF sensing, overcoming the limitations of traditional background subtraction methods in handling complex environments.

[0017] Furthermore, this invention proposes a unique intra-modal and cross-modal feature fusion mechanism. It extracts the spatiotemporal features of the radio frequency signal and the depth image separately using a single-modal encoder, and performs depth alignment in both spatial and temporal dimensions using a multi-head cross-attention mechanism. This mechanism not only leverages the spatial resolution advantage of the visual modality but also fully exploits the temporal continuity features of the radio frequency modality, thereby achieving accurate capture of the target object signal and effective suppression of background noise.

[0018] In summary, this invention, by designing a radio frequency (RF) signal denoising network model and utilizing the spatiotemporal features of depth images to guide the spatiotemporal feature alignment and noise stripping of RF signals, achieves accurate extraction of target object signals in complex dynamic environments. This invention not only significantly reduces the data acquisition cost of RF sensing systems but also greatly improves the robustness and accuracy of signal processing, making it widely applicable in fields such as wireless sensing signal preprocessing and enhancement. Attached Figure Description

[0019] Figure 1 A schematic diagram of the overall architecture of the radio frequency signal denoising network model.

[0020] Figure 2 This is a schematic diagram of the structure of an RF signal encoder module.

[0021] Figure 3 This is a schematic diagram of the depth image encoder module.

[0022] Figure 4 A comparison chart showing the decoupling effect of aliased signals. Detailed Implementation

[0023] The present invention will now be described in further detail with reference to the accompanying drawings.

[0024] This invention provides a method for eliminating background noise in radio frequency (RF) signals based on multimodal spatiotemporal decoupling. The invention aims to address the problem of severe environmental background interference in real-world RF signals and the lack of clean signals as training labels. This method constructs an RF signal denoising network, extracting spatiotemporal features from the RF signal and depth image using a single-modal encoder; Utilizing intramodal and cross-modal fusion modules, it aligns the features of the two modalities in the spatial and temporal dimensions through a multi-head cross-attention mechanism; finally, it predicts an attention mask using a clean signal decoder and combines it with residual connections to reconstruct a clean target object signal from the aliased signal. This invention achieves efficient decoupling and denoising of RF signals in complex environments through multimodal fusion technology, significantly improving signal quality.

[0025] This invention discloses a method for eliminating background noise in radio frequency signals based on multimodal spatiotemporal decoupling, comprising the following steps: S1. Construct a radio frequency signal denoising network model; This step constructs an end-to-end deep learning processing framework, providing the structural foundation for subsequent signal denoising. Please refer to [link / reference]. Figure 1 A radio frequency (RF) signal denoising network is constructed, with the input being a matrix of aliased RF signals. and depth images The model includes a single-mode encoder module, an intra-modal and cross-modal fusion module, and a clean signal decoder module.

[0026] S2. Extracting single-modal spatiotemporal features; This step maps the original low-dimensional signal to a high-dimensional feature space, providing rich feature representation for subsequent signal denoising. Simultaneously, the spatiotemporal features extracted in this step can be used as a reference to guide the identification of background noise in the radio frequency signal. By inputting real or simulated aliased radio frequency signals and depth images into the single-mode encoder module, the spatiotemporal features of the radio frequency signal and the spatiotemporal features of the depth image are extracted respectively.

[0027] Please see Figure 2 For radio frequency signals, their complex form is separated into real parts. and the virtual part Both extract the spatial features of the radio frequency signal through a structurally shared encoder, which contains three one-dimensional convolutional sub-blocks, one residual sub-block, and one two-dimensional convolutional block. and time characteristics .

[0028] Please see Figure 3 For depth images, the spatial depth features at the pixel level for each frame are first extracted using three two-dimensional convolutional blocks. Then, the features are aggregated along the time dimension using two one-dimensional convolutional blocks to generate the image spatial features. and time characteristics .

[0029] S3, Multimodal Feature Fusion The single-modal features extracted in step S2 are input into the intra-modal and cross-modal fusion module. A multi-head cross-attention mechanism aligns the features of the radio frequency signal and the depth image in both spatial and temporal dimensions, outputting the fused multimodal features. This step utilizes the geometric features of the depth map to guide the radio frequency signal in identifying background noise and target objects, achieving feature alignment and noise recognition. Furthermore, the adaptive weighting mechanism allows the network to automatically adjust the contribution of different feature components based on the complexity of the current scene, thereby maximizing denoising performance and significantly improving the model's adaptability to various complex environments.

[0030] For cross-modal fusion units in the spatial dimension, multi-head cross-attention (MCA) is used to compute spatial fusion features. The calculation process includes: using signal spatial features The query vector is the spatial features of the depth image. Given a key vector and a value vector, multi-head cross-attention is used to compute the first attention term. This utilizes deep image spatial features. The query vector is based on the spatial characteristics of the signal. Given a key vector and a value vector, a second attention term is computed using multi-head cross-attention. The two attention terms are then summed with the depth image spatial features as a residual connectivity term. The formula for addition is: For cross-modal fusion units in the temporal dimension, multi-head cross-attention (MCA) is used to compute temporal fusion features. The calculation process includes: using signal time characteristics The query vector is a temporal feature of the depth image. Given a key vector and a value vector, multi-head cross-attention is used to compute the first attention term. This is based on the temporal features of the depth image. The query vector is based on the time characteristics of the signal. Given a key vector and a value vector, a second attention term is computed using multi-head cross-attention. The two attention terms are then summed with the temporal features of the depth image as a residual connectivity term. The formula for addition is: The MCA operation is defined as follows: in, Represents a query item matrix. Represents the key matrix, Represents the value term matrix, The scaling factor representing the feature dimension. This represents the matrix transpose operation. It is a normalized exponential function.

[0031] For intramodal fusion units, calculate intramodal features. and The calculation method is similar to that of cross-modal fusion, except that the attention calculation is performed on the same modality. The formulas are as follows: Adaptive weighting, calculating the final fused features : in, Defined as: S4, Pure Signal Reconstruction The fused features obtained in step S4 are input into the clean signal decoder module. By predicting the attention mask and combining it with residual connections, the abstract high-dimensional features are mapped back to the physical signal space, which enables the recovery of the predicted clean radio frequency signal from the aliased radio frequency signal.

[0032] Fusion features Perform two 2D convolutions and PReLU activations, then generate an attention mask using the Sigmoid function. ;The original input signal S is enhanced using a mask to obtain Simultaneously, convolution processing is performed on the original signal to obtain residual features. The final predicted pure signal for: S5, Model Training and Optimization The core of this step lies in supervised training and parameter optimization of the RF signal denoising network model using a composite loss function. Using a clean signal generated from a physical propagation model simulation as the ground truth, the loss function is calculated and the RF signal denoising network model is trained. The trained model is then used to separate the clean target object signal from the real-world noisy RF signal.

[0033] loss function It consists of three weighted parts, and the calculation formula is as follows: in, Cosine similarity loss over time: ; Cosine similarity loss for the distance dimension: ; Mean square error loss ; This is a hyperparameter.

[0034] By minimizing this loss function, the model parameters are optimized to accurately decouple the clean target signal from the aliased signal acquired in real-world sampling. See also... Figure 4 The figure illustrates a visual comparison of the effectiveness of the method of this invention and the traditional method in eliminating background noise in radio frequency signals under the same experimental conditions. Specifically, it compares the original received signal, the signal processed using the traditional static background removal method, the clean signal output by the method of this invention, and the true signal. The signal processed by the method of this embodiment effectively removes environmental background noise, restores the true reflection characteristics of the target object, and highly matches the true signal.

[0035] In another embodiment of the present invention, a radio frequency signal background noise cancellation system based on multimodal spatiotemporal decoupling is provided. This system can be used to implement the above-mentioned radio frequency signal background noise cancellation method based on multimodal spatiotemporal decoupling. Specifically, the radio frequency signal background noise cancellation system based on multimodal spatiotemporal decoupling includes a signal encoding module, a feature fusion module, a signal decoding module, and a model training module.

[0036] Among them, the signal image encoding module is used to extract the real and imaginary part features of the input radio frequency signal and the spatiotemporal features of the depth image through a multi-layer convolutional network; The feature fusion module is used to calculate cross-modal features and intramodal features, and to perform weighted fusion using adaptive weights; The signal decoding module is used to generate a mask based on the fusion features and reconstruct the clean radio frequency signal of the target object. The model training module is used to train the network using a simulation dataset and optimize the model parameters using a joint loss function.

[0037] In another embodiment of the present invention, a terminal device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used in the operation of a radio frequency signal background noise cancellation method based on multimodal spatiotemporal decoupling, including: A radio frequency (RF) signal denoising network model is constructed, comprising a single-modal encoder module, an intra-modal and cross-modal fusion module, and a clean signal decoder module. Real or simulated aliased RF signals and depth images are input into the single-modal encoder module to extract spatiotemporal features of the RF signal and the depth image, respectively. The extracted single-modal features are input into the intra-modal and cross-modal fusion module, which aligns the features of the RF signal and the depth image in the spatial and temporal dimensions using a multi-head cross-attention mechanism, outputting fused multimodal features. The obtained fused features are input into the clean signal decoder module, which recovers the predicted clean RF signal from the aliased RF signal by predicting an attention mask and combining it with residual connections. Using the generated clean RF signal as the ground truth, a loss function is calculated and the RF signal denoising network model is trained. The trained model is then used to separate the clean target object signal from the real-world noisy RF signal.

[0038] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory). This computer-readable storage medium is a memory device in a terminal device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and extended storage media supported by the terminal device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device.

[0039] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the radio frequency signal background noise cancellation method based on multimodal spatiotemporal decoupling in the above embodiments; one or more instructions in the computer-readable storage medium are loaded and executed by the processor to perform the following steps: A radio frequency (RF) signal denoising network model is constructed, comprising a single-modal encoder module, an intra-modal and cross-modal fusion module, and a signal decoder module. Real or simulated aliased RF signals and depth images are input into the single-modal encoder module to extract the spatiotemporal features of the RF signal and the depth image, respectively. The extracted single-modal features are input into the intra-modal and cross-modal fusion module, which aligns the features of the RF signal and the depth image in the spatial and temporal dimensions using a multi-head cross-attention mechanism, outputting the fused multimodal features. The obtained fused features are input into the clean signal decoder module, which recovers the predicted clean RF signal from the aliased RF signal by predicting an attention mask and combining it with residual connections. Using the generated clean RF signal as the ground truth, a loss function is calculated and the RF signal denoising network model is trained. The trained model is then used to separate the clean target object signal from the real-world noisy RF signal.

[0040] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0041] The radio frequency signal background noise cancellation method based on multimodal spatiotemporal decoupling of this invention was tested in the laboratory.

[0042] Please see Figure 4 The figure shows a comparison of the original received signal, the signal processed by the traditional static background removal method, the clean signal output by the method of this invention, and the true signal under the same experimental conditions. Experimental results show that the original received signal contains a large amount of clutter caused by environmental reflections, severely obscuring the target features. While the traditional background removal method can filter out some static clutter, it is weak in handling dynamic interference, leaving noticeable signal residue. In contrast, the signal output by the method of this invention clearly preserves the reflection characteristics of the target object, and background noise is effectively suppressed, showing a high degree of consistency with the true signal.

[0043] Experimental results show that when trained using only the real dataset, the model's RRMSE is 0.0833 and SSIM is 0.9289; however, after introducing the simulated dataset for joint training, the RRMSE significantly decreases to 0.0274 and the SSIM increases to 0.9861. This demonstrates the excellent decoupling accuracy of this invention. Furthermore, in non-line-of-sight scenarios where the object is completely obscured by a cardboard box, the method of this invention still maintains extremely high performance, with the RRMSE increasing by only 0.0086 and the SSIM decreasing by only 0.0078 compared to the line-of-sight scenario. This proves that the depth image-guided denoising mechanism of this invention has extremely strong robustness against occlusion and environmental interference.

[0044] In summary, this invention presents a method, system, medium, and device for eliminating background noise in radio frequency (RF) signals based on multimodal spatiotemporal decoupling. The proposed RF signal denoising network model fully utilizes information from the deep visual modality for guidance, and achieves accurate decoupling and denoising of RF aliased signals through a cross-modal attention mechanism and an adaptive weighted fusion strategy. Experimental results show that this method not only performs excellently in line-of-sight environments but also maintains extremely high signal denoising accuracy in non-line-of-sight scenarios where objects are completely occluded, demonstrating strong environmental adaptability and robustness.

Claims

1. A method for eliminating background noise in radio frequency signals based on multimodal spatiotemporal decoupling, characterized in that, Includes the following steps: S1. Acquire the original radio frequency reflection signal of the target object using a single-transmitter, single-receiver commercial pulse ultra-wideband radar, and simultaneously acquire a depth image containing scene information corresponding to the area where the target object is located; the original radio frequency reflection signal contains an aliasing component of the target object's radio frequency reflection signal and the environmental background radio frequency reflection signal; S2. Construct a dual-modal spatiotemporal coding network model for signal and image, encode the original radio frequency reflection signal and the depth image respectively, and extract the spatiotemporal features of the original radio frequency reflection signal and the spatiotemporal features of the depth image; S3. Construct intra-modal and cross-modal fusion modules. Guided by the spatiotemporal features of the depth image, the spatiotemporal features of the original radio frequency reflection signal and the spatiotemporal features of the depth image are fused through a multi-head cross-attention mechanism to obtain multimodal fusion features. S4. Construct a signal decoder, input the multimodal fusion features into the clean signal decoder module, enhance the original radio frequency reflection signal through the prediction attention mask, and reconstruct the clean signal from the original radio frequency reflection signal by combining residual connections, thereby achieving the elimination of radio frequency signal background noise; construct a composite loss function training model.

2. The method for eliminating background noise of radio frequency signals based on multimodal spatiotemporal decoupling according to claim 1, characterized in that, In S1, the radar and depth camera are fixedly installed. The target object moves through the radar detection area with the conveyor belt. During this process, the radar continuously transmits detection pulses to the moving target object and receives the echo, thereby completing the acquisition of the original radio frequency reflection signal. The aliasing component is the superposition of the target object's radio frequency reflection signal and the environmental background radio frequency reflection signal in the radio frequency signal received by the radar. The target object's radio frequency reflection signal is used to extract the target object's spatial position and geometric structure; the environmental background radio frequency reflection signal is used to characterize irrelevant interference components in the scene, which are noise components to be eliminated. In the multimodal feature fusion stage, the background noise is identified and eliminated by spatiotemporally aligning it with the environmental features of the depth image.

3. The method for eliminating background noise of radio frequency signals based on multimodal spatiotemporal decoupling according to claim 2, characterized in that, In S2, the signal-image dual-modal spatiotemporal coding network model adopts a feature extraction architecture based on a parallel convolutional neural network. The feature extraction architecture includes a single-modal encoder module that processes data of different modalities, namely a signal time encoder module, a signal space encoder module, a depth image time encoder module, and a depth image space encoder module, which are used to extract the spatiotemporal features of the original radio frequency reflection signal and the spatiotemporal features of the depth image, respectively.

4. The method for eliminating background noise of radio frequency signals based on multimodal spatiotemporal decoupling according to claim 3, characterized in that, The processing procedures of the signal time encoder module and the signal space encoder module are as follows: The original radio frequency reflection signal is divided into real and imaginary parts, which are then input into an encoder with a shared structure. The encoder consists of three one-dimensional convolutional sub-blocks, one residual sub-block, and one two-dimensional convolutional sub-block. The encoder is used to extract the spatiotemporal features of the original radio frequency reflection signal. and ; The processing steps of the depth image temporal encoder module and the depth image spatial encoder module are as follows: the depth image is input into three two-dimensional convolutional sub-blocks to extract depth features, and then features are extracted along the time dimension through two one-dimensional convolutional blocks to generate image spatiotemporal features. and .

5. The method for eliminating background noise of radio frequency signals based on multimodal spatiotemporal decoupling according to claim 4, characterized in that, In S3, The intra-modal and cross-modal fusion module includes intra-modal fusion units and cross-modal fusion units, and its calculation process includes: For cross-modal fusion units in the spatial dimension, multi-head cross-attention (MCA) is used to compute spatial fusion features. The calculation process is as follows: based on the spatial characteristics of the signal As a query vector, spatial features of the depth image As key and value vectors, the first attention term is calculated using multi-head cross-attention; based on deep image spatial features... As a query vector, based on the spatial characteristics of the signal As key and value vectors, a second attention term is computed using multi-head cross-attention; the two attention terms are then summed as the deep image spatial features as the residual connectivity term. The formula for addition is: For cross-modal fusion units in the temporal dimension, multi-head cross-attention (MCA) is used to compute temporal fusion features. The calculation process is as follows: based on the signal time characteristics As a query vector, the temporal features of the depth image As key and value vectors, the first attention term is calculated using multi-head cross-attention; based on the temporal features of the depth image. As a query vector, the time characteristics of the signal As key and value vectors, a second attention term is computed using multi-head cross-attention; the two attention terms are then summed as the depth image temporal features used as the residual connectivity term. The formula for addition is: The MCA operation is defined as follows: in, Represents the query matrix. Represents the key matrix. Represents a value matrix, The scaling factor representing the feature dimension. This represents the matrix transpose operation. It is a normalized exponential function; For intramodal fusion units, calculate intramodal features. and The calculation method is similar to that of cross-modal fusion, except that the attention calculation is performed on the same modality. The formulas are as follows: Adaptive weighting, calculating the final fused features : in, Defined as: 。 6. The method for eliminating background noise of radio frequency signals based on multimodal spatiotemporal decoupling according to claim 5, characterized in that, In step S4, the processing procedure of the clean signal decoder module is as follows: Fusion features Perform two 2D convolutions and PReLU activations, then generate an attention mask using the Sigmoid function. ; Using a mask to enhance the original input signal S, we obtain Simultaneously, convolution processing is performed on the original signal to obtain residual features. The final predicted pure signal for: In step S4, the loss function The calculation formula is: in, Cosine similarity loss over time: ； Cosine similarity loss for the distance dimension: ； Extreme value mean square error loss; ； This is a hyperparameter.

7. The method for eliminating background noise of radio frequency signals based on multimodal spatiotemporal decoupling according to claim 6, characterized in that, In S4, during the network training phase, a clean signal based on a physical propagation model is used to generate the corresponding label. The composite loss function includes measures of the cosine similarity loss between the predicted clean RF signal and the true clean RF signal in the time and space dimensions, as well as the mean square error loss of the signal amplitude.

8. A radio frequency signal background noise cancellation system based on multimodal spatiotemporal decoupling for implementing the method of any one of claims 1-7, characterized in that, It includes a signal image encoding module, a feature fusion module, a signal decoding module, and a model training module; The signal image encoding module is used to extract the spatiotemporal features of the real and imaginary parts of the input radio frequency signal and the spatiotemporal features of the depth image through a multi-layer convolutional network. The feature fusion module is used to calculate cross-modal features and intra-modal features based on the spatiotemporal features of the signal and image extracted by the signal image encoding module, namely cross-modal spatial fusion features, cross-modal temporal fusion features, intra-modal signal features, and intra-modal image features. The module then performs weighted fusion of the above four features using adaptive weights to obtain the final fused features. The signal decoding module is used to generate a mask based on the fusion features and reconstruct a clean radio frequency signal of the target object; The model training module is used to train the network using a simulation dataset and optimize the model parameters using a joint loss function.

9. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform the steps of the radio frequency signal background noise cancellation method based on multimodal spatiotemporal decoupling as described in any one of claims 1-8.

10. A computing device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including steps for performing the radio frequency signal background noise cancellation method based on multimodal spatiotemporal decoupling according to any one of claims 1-8.

Images