A method for joint phase synchronization of long sequences of wireless signals and its ISAC action recognition method and device

Abstract

This invention discloses a method for joint synchronization of long-sequence phase of wireless signals and its ISAC action recognition method and device, belonging to the interdisciplinary field of radar signal processing and wireless communication. This invention aims to solve the problem of high-performance sensing with low-cost hardware in ISAC. Specifically, multi-carrier frequency-shift linear frequency modulation technology is used at the transmitting end to embed communication data while maintaining the characteristics of the radar detection waveform. To address the random carrier frequency offset, random sampling frequency offset, and random sampling time offset introduced by hardware non-ideals, a joint synchronization neural network is designed. A long-sequence feature extraction module captures long-sequence phase drift, achieving high-precision parameter decoupling and compensation in a single operation. Subsequently, the synchronized single-channel time-frequency features are combined with an environment-aware feature fusion network, and an adaptive gating fusion mechanism is used to eliminate static environmental clutter, achieving highly robust action gesture recognition.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of radar signal processing, wireless communication, and artificial intelligence, specifically relating to a method for achieving long-sequence phase joint synchronization of wireless signals and its ISAC action recognition method and device. This technology adds communication capabilities to a radar system, utilizing multi-carrier frequency-shift linear frequency modulated signals and deep learning technology to eliminate hardware time-frequency synchronization errors and achieve a highly robust action and posture recognition integrated sensing and communication (ISAC) device and method. Background Technology

[0002] With the evolution of electronic information technology, radar systems are undergoing a transformation from single detection functions to integrated sensing and communication (ISAC). Traditional radar systems utilize linear frequency modulation (LFM / Chirp) signals for target ranging and velocity measurement, offering excellent pulse compression performance but lacking data transmission capabilities. To endow radar with communication capabilities without altering its hardware architecture, frequency-shift chirp modulation (FFM / Chirp Modulation) has emerged as a highly promising technology. FFM / Chirp modulates communication data by fine-tuning the starting frequency of the chirp signal, preserving the radar waveform's anti-jamming characteristics while embedding information.

[0003] However, upgrading existing radar equipment to ISAC nodes (especially low-cost, distributed radar IoT nodes) presents a significant challenge in terms of hardware time-frequency synchronization errors, specifically:

[0004] 1. Random carrier frequency offset: The local oscillators between radar transceiver nodes cannot be perfectly synchronized, which will result in random deviations in the center frequency.

[0005] 2. Random sampling frequency offset: Due to the inconsistency of the clock source, there is a non-negligible frequency difference between the sampling clock at the receiving end and the transmitting end, which will cause the received frequency-shifted linear frequency modulated radar echo to randomly stretch and shrink on the time axis.

[0006] 3. Random sampling time offset: Due to the uncertainty of transmission distance and the randomness of system startup, the sampling window may not be aligned with the radar pulse boundary.

[0007] In traditional monostation radar systems, the transmitter and receiver share the same clock source (local oscillator), and the transmitted and received signals naturally maintain coherence, thus eliminating significant frequency and time synchronization errors. However, to extend sensing coverage and enable communication between nodes, ISAC systems often employ a distributed or bistationary architecture. In this architecture, the transmitting and receiving nodes are physically separated, each using an independent, low-cost clock source, inevitably leading to hardware non-ideal errors between the transmitting and receiving ends.

[0008] Currently, existing solutions for synchronization issues in distributed or bistatic radar typically rely on costly hardware or complex system architectures. For example, maintaining high-precision synchronization involves adding a GPS disciplined clock (GPSDO) or atomic clock, but this significantly increases hardware costs and power consumption; alternatively, a dedicated synchronization receiving channel (reference channel) is used to acquire direct-wave signals for differential calibration, which requires additional antennas and RF links and increases signal processing complexity. These methods are clearly unsuitable for cost-sensitive, lightweight consumer-grade ISAC IoT nodes.

[0009] Therefore, in low-cost ISAC systems, the hardware-introduced random carrier frequency offset, random sampling frequency offset, and random sampling time offset become key bottlenecks limiting performance. Traditional radar signal processing algorithms mainly focus on the detection of strong targets, often ignoring these random errors. However, in ISAC systems, the non-negligible random sampling frequency offset disrupts the demodulation orthogonality of frequency-shift linear frequency modulation, causing distortion of the chirp signal's frequency modulation slope, which in turn leads to a sharp increase in the communication bit error rate. At the same time, these time-frequency synchronization errors introduce non-stationary phase noise, severely degrading the radar's Doppler resolution and significantly reducing the radar's signal-to-noise ratio, severely limiting the accuracy of target detection and robustness of identification.

[0010] Existing technologies typically employ a step-by-step estimation method to handle random carrier frequency offset and random sampling time offset separately. However, in low-cost hardware, the presence of random sampling frequency offset introduces nonlinear time scaling, which can lead to severe deviations in the estimation of carrier frequency offset (i.e., error coupling effect).

[0011] In summary, existing methods either have excessively high hardware costs, treat communication synchronization and radar sensing separately, or can only address one type of error individually. There is a lack of a unified framework that can simultaneously solve the multi-dimensional time-frequency synchronization error coupling problem under conditions of a single receiving antenna and low-cost hardware. Summary of the Invention

[0012] This invention aims to address the technical problem of existing radar systems' inability to effectively solve multi-dimensional time-frequency synchronization error coupling under conditions of single receiving antenna and low-cost hardware. To this end, this invention proposes a method for long-sequence phase joint synchronization of wireless signals and its ISAC action recognition method and device. Under a low-cost, single-receiver-channel multi-carrier sensing integrated system, it proposes for the first time to treat three types of strongly coupled physical errors—random carrier frequency offset, random sampling frequency offset, and random sampling time offset—as a whole for single-run end-to-end collaborative prediction. The proposed deep joint synchronization network breaks through the limitations of traditional iterative algorithms, enabling the decoupling of these three strongly coupled physical variables in a single inference, significantly reducing computational latency. Furthermore, based on the high-precision characteristics of deep synchronization, the need for multi-antenna differential calibration is eliminated, requiring only a single receiving channel to achieve high-precision sensing, greatly reducing hardware costs.

[0013] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0014] On one hand, the present invention provides a method for joint phase synchronization of long sequences of wireless signals, comprising the following steps:

[0015] Step S1, Signal Acquisition: The radar receiver acquires the radio frequency echo signal transmitted through the wireless channel and converts it into a discrete baseband sequence; the discrete baseband sequence contains various coupling time-frequency synchronization errors introduced by hardware non-ideals, and the coupling time-frequency synchronization errors include at least random carrier frequency offset, random sampling frequency offset and random sampling time offset;

[0016] Step S2, Joint Error Estimation: Construct an input tensor based on the synchronization pilot band in the discrete baseband sequence, input it into a pre-trained joint synchronization neural network, and output the joint error estimate. , These are the random carrier frequency offsets. Random sampling frequency offset and random sampling time offset The estimated value;

[0017] The joint synchronous neural network includes a local feature extraction module, a long-term feature extraction module, and a regressor to capture the local phase texture and long-term nonlinear phase drift features of the signal.

[0018] Step S3, Inverse Compensation: Construct an inverse compensation operator based on the joint error estimate, and perform frequency offset correction, time resampling and translation alignment on the discrete baseband sequence to obtain the synchronous baseband signal.

[0019] Optionally, the local feature extraction module of the joint synchronous neural network adopts a convolutional neural network (CNN), which has multiple stacked 2D convolutional layers, and each 2D convolutional layer is followed by batch normalization and activation functions.

[0020] The long temporal feature extraction module employs a Transformer encoder and incorporates multiple head self-attention mechanisms—feedforward neural network layers—to capture the global dependencies of long sequences.

[0021] Optionally, the mathematical model of the discrete baseband sequence is:

[0022]

[0023] in:

[0024] The sequence is a discrete baseband sequence, where n is the number of discrete sampling points;

[0025] This is the ideal sampling interval for the radar receiver. Nominal sampling rate;

[0026] Random carrier frequency offset: caused by local oscillator frequency drift, manifested as rapid rotation of the signal phase;

[0027] Random sampling frequency offset: caused by the sampling clock deviation of the ADC (Analog-to-Digital Converter), manifested as a linear scaling factor on the time axis. ;

[0028] Random sampling time offset: This manifests as a time delay for non-integer sampling points, affecting the accuracy of the radar's ranging starting point;

[0029] It is additive white Gaussian noise;

[0030] The multi-carrier frequency-shift linear frequency modulated baseband signal generated for the radar transmitter;

[0031] This represents the gain coefficient for the complex channel.

[0032] Optionally, the process of performing frequency offset correction, time resampling, and translation alignment on the discrete baseband sequence in step S3 is as follows:

[0033] Frequency offset correction: Based on the estimate of random carrier frequency offset Frequency offset correction is performed on the discrete baseband sequence to eliminate random carrier frequency offset:

[0034] ;

[0035] In the formula, It is a discrete baseband sequence. To eliminate the discrete baseband sequence after random carrier frequency offset, Where n is the nominal sampling rate, and n is the discrete sampling point.

[0036] Resampling correction: an estimate based on the random sampling frequency offset Fractional interpolation resampling is performed to map the discrete baseband sequence signal onto a new time grid. Used to correct for time axis scaling:

[0037] ;

[0038] The above equation represents the remapping of the time coordinates for each discrete sampling point n, where, This is the ideal sampling interval for the receiver;

[0039] Delay alignment: an estimate based on random sampling time offset By applying a linear phase shift in the frequency domain, a subsampling-point precision shift is achieved, resulting in a standard orthogonal multicarrier frequency-shifted linear frequency modulated signal.

[0040] ;

[0041] In the formula, It is a standard orthogonal multicarrier frequency shift linear frequency modulation signal, eliminating interference from hardware damage; This is a discrete baseband sequence after frequency offset correction and resampling correction; FFT is Fast Fourier Transform, and IFFT is Inverse Fast Fourier Transform. The number of computation points for the Fast Fourier Transform (FFT) For discrete frequency indexes.

[0042] Optionally, the pre-training process of the joint synchronous neural network adopts a composite loss function based on dynamic weight constraints, which is composed of a weighted sum of a supervised regression loss term, a coarse-grained classification loss term, and a physical perception loss term.

[0043] The physical perception loss term is as follows:

[0044]

[0045] in, The amplitude of the frequency spectrum after demodulation. It is a smoothing constant. Represents the energy of the spectral peak. Represents total energy. This is a discrete frequency index, i.e., a frequency point number.

[0046] Optionally, the method further includes: the radar transmitter using multi-carrier frequency-shift linear frequency modulation (MFLM) technology to embed communication data into the radar MFLM pulse by controlling the discrete frequency shift, wherein the multi-carrier frequency-shift linear frequency modulation full-wideband signal used for transmission... for:

[0047]

[0048]

[0049] in:

[0050] Here, K is the power normalization coefficient, and K is the number of subcarriers. For the first The transmitted signal of each subcarrier, where t is time;

[0051] The duration of the pulse is defined for a rectangular window function;

[0052] The symbol period;

[0053] For the first The center frequency of each subcarrier ensures that the subcarriers do not interfere with each other in the frequency domain;

[0054] For linear frequency modulation slope, The sweep bandwidth for a single subcarrier;

[0055] For discrete frequency shifts controlled by communication data, satisfying , For discrete frequency offset set The first, second, and Mth frequency offset values ​​in the droplet; for the... Subcarriers, Each time, truncate A bit as a communication data symbol And then based on the current communication symbols From discrete frequency offset sets Select a corresponding frequency offset M is a positive integer.

[0056] Secondly, the present invention provides an ISAC action recognition method. After obtaining the synchronization baseband signal using steps S1-S3 of the aforementioned method, the ISAC action recognition method further performs the following steps:

[0057] Step A: Feature extraction, performing time-frequency transformation on the input baseband signal to generate a time-frequency feature matrix containing the target micro-Doppler effect;

[0058] Step B: Feature separation and gating fusion. The time-frequency feature matrix is ​​input into an environment-aware feature fusion network. The environment-aware feature fusion network has static branches, dynamic branches, and an adaptive gating fusion unit.

[0059] The static branch and the dynamic branch are used to extract static environmental features that characterize environmental complexity and dynamic action features that characterize the motion law of the target from the time-frequency feature matrix, respectively.

[0060] The adaptive gating fusion unit generates gating weights based on the static environment features to characterize the complexity of the current environment; then, the gating weights are used to adaptively weight the dynamic action features to suppress noise interference in complex environments.

[0061] Step C: Action recognition, outputting classification labels for action postures based on gated fusion features.

[0062] In three aspects, the present invention provides a multi-carrier FSCM sensing integrated device, comprising at least:

[0063] Radar transmitter: Utilizing multi-carrier frequency shift linear frequency modulation (FSCM) technology, communication data is embedded into the radar linear frequency modulation pulse by controlling the discrete frequency shift amount, thereby generating a multi-carrier FSCM signal;

[0064] Radar receiver: Collects radio frequency echo signals transmitted via wireless channels;

[0065] The sensing and communication controller is connected to the radar receiver and processes the radio frequency echo signal using the aforementioned method to obtain a synchronous baseband signal; or processes the radio frequency echo signal using the aforementioned method to obtain a classification label for the ISAC action posture.

[0066] In four aspects, the present invention provides a system based on the above method, comprising:

[0067] The signal acquisition module is used to acquire radio frequency echo signals transmitted through a wireless channel using a radar receiver and convert them into discrete baseband sequences. The discrete baseband sequences contain various coupling time-frequency synchronization errors introduced by hardware non-ideals. The coupling time-frequency synchronization errors include at least random carrier frequency offset, random sampling frequency offset, and random sampling time offset.

[0068] The joint error estimation module is used to construct an input tensor based on the synchronization pilot band in the discrete baseband sequence, input it into a pre-trained joint synchronization neural network, and output a joint error estimate. , These are the random carrier frequency offsets. Random sampling frequency offset and random sampling time offset The estimated value;

[0069] The joint synchronous neural network includes a local feature extraction module, a long-term feature extraction module, and a regressor to capture the local phase texture and long-term nonlinear phase drift features of the signal.

[0070] The inverse compensation module is used to construct an inverse compensation operator based on the joint error estimate, and perform frequency offset correction, time resampling and translation alignment on the discrete baseband sequence to obtain the synchronous baseband signal.

[0071] In five aspects, the present invention provides a computer-readable storage medium, characterized in that: it stores a computer program, which is called by a processor to implement: the steps of a method for joint synchronization of long sequence phases of wireless signals or an ISAC action recognition method.

[0072] Compared with the prior art, the present invention has the following beneficial effects:

[0073] 1. The technical solution of this invention is based on a radar system. To address the hardware time-frequency synchronization error problem that is common in low-cost radar nodes, a joint synchronization neural network is constructed to achieve single joint decoupling of random carrier frequency offset, random sampling frequency offset, and random sampling time offset. Then, inverse compensation is performed based on the decoupling error value to ensure signal accuracy.

[0074] In existing technologies, traditional signal processing algorithms typically rely on "serial step-by-step estimation" or "iterative search" under high signal-to-noise ratio (SNR) conditions to address three types of hardware non-ideal errors: random carrier frequency offset, random sampling frequency offset, and random sampling time offset. However, under low-cost hardware systems, the non-negligible sampling frequency offset introduces nonlinear scaling of the time axis, leading to deep nonlinear coupling of these three types of physical errors, thus requiring joint estimation. The joint synchronous neural network design overcomes the deadlock obstacles of "strong physical coupling effect" and "step-by-step iterative error propagation" faced by existing technologies when dealing with multidimensional time-frequency non-ideals. This invention innovatively constructs a network architecture based on physical mechanism fit: it utilizes the high-frequency receptive field of the local feature extraction module to accurately lock the local instantaneous frequency offset and pulse edges; simultaneously, to address the nonlinear drift problem caused by conventional network failure, a long-time-series feature extraction module is introduced, utilizing its multi-head global attention mechanism to directly construct feature maps of the beginning and end of the sequence, accurately fitting the secondary phase drift caused by the sampling frequency offset. This network architecture design is not available in existing solutions that simply apply neural networks, achieving high-precision collaborative correction of multidimensional coupling errors under extremely low SNR conditions. The technical solution of this invention makes full use of the long-time feature extraction module's ability to model long-distance dependencies and capture the long-time phase drift of the signal. Under the conditions of low signal-to-noise ratio and multi-dimensional damage coupling, the synchronization accuracy and robustness are better than those of traditional iterative algorithms.

[0075] 2. In the further optimized technical solution of this invention, efficient data transmission and high-precision synchronization are achieved while retaining the advantages of radar detection. Specifically, this invention utilizes multi-carrier frequency shift linear frequency modulation (MFCM) technology to covertly embed communication data into the radar MFCM pulses by controlling the discrete frequency shift. Unlike traditional OFDM sensing schemes, this solution fully preserves the radar signal's pulse compression gain and anti-interference capability, ensuring the core performance of ranging and velocity measurement, and achieving deep waveform-level fusion of sensing and communication.

[0076] 3. In a further optimized technical solution of this invention, the training mechanism of the joint synchronous neural network is innovatively optimized by introducing a composite loss function that includes supervised regression, coarse-grained classification, and physical perception loss. In particular, the physical perception loss utilizes the sharpness of the demodulated frequency spectrum as an unsupervised self-testing index, deeply integrating the physical laws of radar signal processing into the gradient update of the neural network. This "physically guided" training strategy forces the network to learn parameters that conform to the signal generation mechanism, effectively avoiding overfitting and significantly improving the robustness of the model under different signal-to-noise ratios and frequency offset ranges.

[0077] 4. In the further optimized technical solution of this invention, based on the high robustness requirement of weak motion recognition, the feature construction and network structure of the motion perception link have been optimized. This invention breaks through the limitation of traditional low-cost radar perception relying on multi-antenna differential (such as RoR) to eliminate phase noise. Thanks to the "deep self-healing" capability of the joint synchronization neural network for time-frequency synchronization errors, the signal already has extremely high purity before entering the perception network. Therefore, only a single receiving antenna is needed to extract clear micro-Doppler features, reducing the cost of the RF link by 50% compared to the dual-channel solution. Addressing the pain point of radar perception being susceptible to environmental clutter interference, this invention constructs a feature fusion network based on environmental perception. This network innovatively designs a dynamic and static dual-stream architecture and an adaptive gating fusion mechanism, which can automatically perceive scene complexity (such as open or crowded) based on the extracted static environmental features and dynamically adjust the attention weight of dynamic motion features accordingly. This mechanism effectively suppresses background noise amplification in complex environments while focusing on weak motion details in pure environments, significantly solving the problem of "poor environmental adaptability" in traditional methods and achieving high-precision motion recognition across scenes. Attached Figure Description

[0078] Figure 1 This is a scene diagram provided in an embodiment of the present invention.

[0079] Figure 2 This is a flowchart provided in an embodiment of the present invention.

[0080] Figure 3 This is a schematic diagram of the structure of the long-sequence phase joint synchronization neural network and the feature fusion network based on environment perception provided in the embodiments of the present invention. Detailed Implementation

[0081] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. The technical features involved in the various embodiments of the invention described below can be combined with each other as long as they do not conflict with each other.

[0082] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0083] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0084] This invention provides a method for joint phase synchronization of long sequences of wireless signals and its ISAC action recognition method and device. To overcome time-frequency synchronization errors caused by low-cost hardware and improve perception robustness, this invention improves the signal processing flow by introducing a joint synchronization neural network to jointly decouple multi-dimensional time-frequency synchronization errors in the radio frequency echo signal, addressing the three random variables (random carrier frequency offset, random carrier frequency offset, and random carrier frequency offset). Random sampling frequency offset and random sampling time offset The single-shot high-precision estimation and compensation provides a high-quality synchronous baseband signal for subsequent processing. Furthermore, based on the "deep self-healing" capability of the joint synchronization neural network for time-frequency synchronization errors, and addressing the vulnerability of radar sensing to environmental clutter interference, an innovative dynamic-static dual-stream architecture and adaptive gating fusion mechanism are designed. This mechanism can automatically perceive scene complexity (such as open or crowded areas) based on extracted static environmental features and dynamically adjust the attention weight for dynamic action features accordingly. This mechanism effectively suppresses background noise amplification in complex environments while focusing on subtle action details in clean environments, significantly solving the problem of "poor environmental adaptability" in traditional methods and achieving high-precision action recognition across scenes. Moreover, this invention also utilizes a multi-carrier frequency-shift linear frequency modulation scheme to covertly embed communication data into the radar detection waveform, thereby enabling parallel operation of radar detection and data transmission.

[0085] The technical concept of the long-sequence phase joint synchronization method for wireless signals provided by this invention is as follows:

[0086] Step 1: For the radar transmitter, a multi-carrier frequency-shift linear frequency modulation (MFLM) baseband signal is generated using MFLM technology. In some embodiments, the preferred MFLM baseband signal is generated by applying discrete frequency offsets controlled by communication data to N orthogonal subcarriers based on the radar MFLM pulse, as will be explained in detail below.

[0087] Step 2: Acquire the radio frequency echo signal transmitted via the wireless channel using a multi-channel radar receiver, and down-convert and digitize the radio frequency echo signal into a discrete baseband sequence. In some embodiments, the discrete baseband sequence is a discrete baseband IQ sequence. The discrete baseband IQ sequence contains three random variables introduced by the non-ideals of the radar transceiver hardware: random carrier frequency offset. Random sampling frequency offset and random sampling time offset .

[0088] Step 3: Joint error estimation, specifically: A synchronization pilot segment (reference signal segment) is extracted from the discrete baseband IQ sequence to construct an input tensor, which is then input into a pre-trained joint synchronization neural network; this joint synchronization neural network outputs the joint estimate of the three random variables mentioned above. , These are the random carrier frequency offsets. Random sampling frequency offset and random sampling time offset The estimated value.

[0089] Step 4: Inverse Compensation: Construct an inverse compensation operator based on the joint estimate, and perform phase rotation correction (frequency offset correction), time resampling, and translation alignment on the discrete baseband IQ sequence of the entire frame to obtain the synchronous baseband signal. .

[0090] Furthermore, this invention also provides an ISAC action recognition method, the technical idea of ​​which is to obtain the synchronization baseband signal through steps 1-4, and then perform:

[0091] Step 51: Feature extraction. Perform time-frequency transformation on the input baseband signal to generate a time-frequency feature matrix containing the target micro-Doppler effect.

[0092] Step 52: Feature separation and gating fusion. The time-frequency feature matrix is ​​input into the environment-aware feature fusion network; the environment-aware feature fusion network is provided with static branches, dynamic branches, and adaptive gating fusion units.

[0093] Step 53: Action recognition, outputting classification labels for action postures based on gated fusion features.

[0094] Among them, the synchronous baseband signal The process is divided into two parallel paths: one path is used for communication demodulation, which performs time-frequency transformation on the synchronous baseband signal and extracts the instantaneous frequency offset of each subcarrier. One channel is used to recover communication binary data; another channel is used for radar perception: the synchronous baseband signal of one receiving channel is used to construct a time-frequency feature matrix, which is input into the feature fusion network based on environmental perception. After dynamic and static feature separation and adaptive gating fusion, the classification label of the action posture is output.

[0095] The embodiments of the present invention will be described in detail below. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operation processes to further explain the technical solution of the present invention.

[0096] In some embodiments, step 1 employs a multi-carrier frequency shift linear frequency modulation integrated sensing signal generation method, in which the radar transmitter transmits a multi-carrier frequency shift linear frequency modulation baseband signal.

[0097] This embodiment comprehensively illustrates the method for generating multi-carrier frequency-shift linear frequency modulation (LFM) integrated sensing and communication signals from the perspective of a radar transmitter. This embodiment aims to address how to efficiently embed high-speed communication data while maintaining the excellent pulse compression characteristics and anti-interference capabilities of the radar detection waveform (linear frequency modulation pulse, LFM / Chirp). See also... Figure 1 In the transmitter section, unlike traditional OFDM communication waveforms, the waveform basis of this embodiment is radar linear frequency modulated pulse (LFM / Chirp), which ensures that the signal has a constant envelope and excellent pulse compression characteristics.

[0098] The specific implementation steps are as follows:

[0099] 1. System parameter configuration and orthogonality design.

[0100] When constructing a multi-carrier system, the primary task is to ensure the orthogonality between subcarriers to avoid inter-carrier interference (ICI / ISI). In this embodiment, the total baseband bandwidth of the radar system is set to [value missing]. (For example According to radar principles, range resolution Determined by bandwidth, i.e. Where c is the speed of light. To ensure that the ranging accuracy of the radar is not lost after embedding communication data, this invention utilizes the full bandwidth for chirp signal coverage, rather than the traditional frequency division multiple access. The symbol period is set to... (For example Total bandwidth Divided into Subcarriers (e.g.) To satisfy the orthogonality condition, the frequency spacing of adjacent subcarriers... It must be a symbol period. Integer multiples of the reciprocal, i.e. ,in It is a positive integer. This embodiment preferably uses... That is, the subcarrier spacing is It should be understood that the values ​​of the above parameters are illustrative examples in this embodiment. In other feasible embodiments, the values ​​of each parameter can be adjusted adaptively according to the application environment and the object.

[0101] 2. Communication data mapping and frequency shift linear frequency modulation.

[0102] Traditional frequency-shift linear frequency modulation (FSFM) typically employs binary frequency shift keying (FSK). To improve spectral efficiency, this embodiment preferably uses M-ary FSK. The high-speed binary communication bitstream to be transmitted undergoes serial-to-parallel conversion and is allocated to... On the nth parallel subcarrier. Subcarriers ( The communication data is intercepted each time. bits (e.g.) (i.e., fourth-order modulation) as a communication data symbol Define a discrete set of frequency offsets. , For discrete frequency offset set The first, second, and Mth frequency offset values ​​are designed according to the orthogonality principle of multi-level orthogonal frequency shift keying (M-FSK). The frequency interval between adjacent discrete frequency offsets... Set as symbol period multiples of the reciprocal (i.e.) This can be directly achieved using existing standard communication modulation techniques, ensuring that when the receiver demodulates using Fast Fourier Transform (FFT) or matched filter banks, the discrete frequency points remain strictly orthogonal and do not interfere with each other, thus achieving communication data extraction with extremely low bit error rate while ensuring the radar detection waveform structure.

[0103] According to current communication symbols From the set through symbol mapping rules Select a corresponding frequency offset To minimize the demodulation error rate, this invention preferably employs a Gray code mapping rule, where adjacent frequency offsets correspond to binary data that differ by only one bit.

[0104] 3. Generation of frequency-shifted linear frequency modulated subcarrier signals.

[0105] For the Each subcarrier transmits a signal. The mathematical model is defined as follows:

[0106]

[0107] in:

[0108] The duration of the pulse is defined for a rectangular window function;

[0109] The symbol period;

[0110] For the first The center frequency of each subcarrier ensures that the subcarriers do not interfere with each other in the frequency domain;

[0111] This is the slope of the linear frequency modulation. It is worth noting that this slope... All data symbols remain unchanged. This design is crucial, as it gives the signal "radar properties." This is because all signal components share the same frequency modulation slope. When the radar receiver performs matched filtering (pulse compression), the first-order phase term (frequency shift) caused by the communication data will only cause a slight shift in the position of the matched filter peak, without destroying the shape of the peak, thus ensuring high signal-to-noise ratio detection capability and ranging accuracy.

[0112] For discrete frequency shifts controlled by communication data, satisfying This is the core of frequency-shift linear frequency modulation (FSFM), which means that communication data is represented by a jump in the starting frequency of a linear frequency modulation signal.

[0113] In summary, this invention proposes a distributed multi-carrier FSCM architecture. As can be seen from the mathematical model of the subcarriers mentioned above, unlike the existing OFDM sensing scheme (which would destroy the radar pulse compression characteristics), all subcarriers in this invention share the same linear frequency modulation slope. The communication data only changes the initial frequency of the signal, without altering the slope. Therefore, the radar receiver can still obtain a radar matched filter peak with an extremely high signal-to-noise ratio, achieving a significant increase in communication speed without sacrificing the radar's ability to detect weak targets.

[0114] 4. Multicarrier superposition and radio frequency transmission.

[0115] Finally, all the parallel-generated The subcarrier signals are superimposed in the digital baseband domain to obtain the final multi-carrier frequency-shift linear frequency modulation full-wideband signal used for transmission. :

[0116]

[0117] in, The power normalization factor is usually set to 1. To maintain a constant total transmission power.

[0118] The synthesized signal It is then converted into an analog signal by a high-speed digital-to-analog converter (DAC), and up-converted to an radio frequency carrier (e.g., via a quadrature modulator). or Ultimately, it is radiated into the space channel through the transmitting antenna.

[0119] It should be understood that although the above formula describes a continuous-time signal, in actual digital systems, signals are generated in discrete form, i.e., discrete sequences are generated. Sampling rate To prevent aliasing and improve waveform quality, oversampling is preferred. .

[0120] Through the above steps, each pulse emitted by the radar can not only be used to detect targets (utilizing LFM characteristics), but also transmit data in parallel. The communication data (using frequency-shift linear frequency modulation characteristics to directly map bit data into the physical parameters of the signal, thus carrying it in the transmitted discrete baseband sequence) achieves deep waveform-level fusion of communication and sensing. This design differs from simple time-division multiplexing or frequency-division multiplexing, maximizing the utilization of time and frequency resources.

[0121] In some embodiments, after the radar receiver receives the radio frequency echo signal, steps 2-5 involve signal acquisition and processing of the radio frequency echo signal using a signal reception and processing method based on deep joint synchronization and gated perception. This will be explained in detail how deep learning technology can be used to eliminate time-frequency synchronization errors and achieve highly robust action recognition under low-cost hardware conditions. Figure 2 The signal processing flow shown covers the entire process from signal reception model establishment, deep joint synchronization, signal reverse reconstruction to synesthetic parallel processing.

[0122] 1. Construction of a signal reception model with time-frequency synchronization error.

[0123] In distributed IoT radar nodes, because the transceiver typically uses low-cost incoherent crystal oscillators and there is no physical clock synchronization line, the received discretized baseband IQ sequence... It inevitably contains three serious random hardware impairments. This embodiment establishes the following accurate signal reception model:

[0124]

[0125] in:

[0126] The ideal sampling interval for the receiver. Nominal sampling rate;

[0127] Random carrier frequency offset: caused by local oscillator frequency drift, manifested as rapid rotation of the signal phase. In multicarrier systems, it leads to inter-carrier orthogonality violation (ICI).

[0128] Random sampling frequency offset: caused by the sampling clock deviation of the ADC (Analog-to-Digital Converter), manifested as a linear scaling factor on the time axis. In frequency-shift linear frequency modulation (FLM), random sampling frequency offset is a critical source of interference. It causes a slight distortion in the modulation slope of the chirp signal, disrupting the orthogonality between multiple carriers, leading to a surge in communication bit error rate and blurring of radar micro-Doppler characteristics. Specifically, since FLM demodulates data by detecting frequency shift (i.e., changes in the slope intercept), a weak random sampling frequency offset can easily be misinterpreted as a frequency shift in communication modulation. This causes serious communication errors; and traditional iterative algorithms cannot decouple this multidimensional nonlinear distortion at the same time, so it is necessary to introduce a joint synchronization neural network with long time-series phase to achieve high-precision single-error decoupling.

[0129] Random sampling time offset: This manifests as a time delay for non-integer sampling points, affecting the accuracy of the radar's ranging starting point;

[0130] It is additive white Gaussian noise;

[0131] This represents the gain coefficient for the complex channel.

[0132] 2. Joint synchronization neural network with long-term phase.

[0133] To accurately decouple the three random variables from the noisy signal, this embodiment deploys a pre-trained long-term phase joint synchronization neural network, the specific network architecture of which is as follows: Figure 3 As shown on the left. This network operates in an "end-to-end" manner, taking the original discretized baseband IQ sequence as input. Directly output random carrier frequency offset Random sampling frequency offset and random sampling time offset The estimated value .

[0134] Input data preprocessing: Extracting the synchronization pilot sequence from the data frame header (discrete baseband IQ sequence) It consists of two parts: a frame header (synchronization pilot) and a payload (communication data); from (The starting position is used to extract a pre-agreed, unmodulated pure Chirp pulse sequence of communication data as a synchronization pilot band) Double oversampling (e.g.) To improve temporal resolution, the construction size is... The input tensor; The number of chirp symbols contained in the extracted synchronization pilot sequence is represented, such as 4 or 8; L represents the number of discrete sampling points contained in a single chirp symbol after oversampling, and its value satisfies the following condition. .

[0135] Network structure: It includes a local feature extraction module, a long-term feature extraction module, and a regressor.

[0136] The local feature extraction module, in this embodiment, adopts a CNN structure and constructs a front-end CNN.

[0137] In this embodiment, the front-end CNN consists of three stacked 2D convolutional layers (Conv2D), each followed by batch normalization and ReLU activation. The kernel size is designed to be... or (Along the time axis). The front-end CNN utilizes the translation invariance of convolution operations on the time-frequency graph (IQ path) to extract features, for the following reasons:

[0138] Capturing instantaneous frequency offset (for random carrier frequency offset): Random carrier frequency offset manifests as a linear phase rotation within an extremely short time slice. The local feature extraction module, through local convolutional kernel operations, can extremely sensitively perceive the rate of change of phase difference between adjacent sampling points, thereby locking in instantaneous frequency features.

[0139] Pulse edge extraction (for random sampling time offset): Random sampling time offset manifests as a shift in the signal envelope. The edge detection capability of the local feature extraction module can effectively locate the start and end boundaries of the chirp signal, providing coarse-grained features for time delay estimation.

[0140] Use Case: Suitable for processing raw IQ data with high sampling rates. Compared to fully connected layers, the local feature extraction module significantly reduces the number of parameters at the front end and effectively suppresses the interference of additive white Gaussian noise (AWGN) on local features.

[0141] The long-term feature extraction module uses a Transformer encoder.

[0142] The long-term feature extraction module follows the local feature extraction module and receives the feature sequence extracted by the local feature extraction module. For example... Figure 3 As shown, the Transformer encoder in this embodiment features a two-layer stacked architecture of multi-head self-attention and feedforward neural network (FFN). While multi-head attention and feedforward networks are existing technologies, this invention is the first to apply them to phase-drift data.

[0143] In particular, this invention features a specific data stream design tailored to the phase drift characteristics of radar signals.

[0144] (1) Input embedding: Input sequence The feature sequences output by the CNN are superimposed using positional encoding (PE) to preserve temporal order information.

[0145]

[0146] The length of the generated position code must be greater than or equal to the total length of the input pilot sequence to ensure that the absolute time order of the radar signal is not disrupted or truncated throughout the entire time frame. The position code is generated by using the existing standard sine / cosine absolute position coding method on the feature sequence data output by the front-end local feature extraction module to generate an absolute position coordinate matrix PE that is completely identical in dimension to the input feature sequence.

[0147] (2) Global Relevance Modeling: Utilizing a multi-head self-attention mechanism to capture long-distance dependencies. For the first... Each attention head, calculate the query ( ),key( ),value( )matrix:

[0148]

[0149] In the formula, , , In order to target the Attention, query ( ),key( ),value( The corresponding learnable projection parameter matrix, , , In order to target the A query for individual attention points ( ),key( ),value( ) Corresponding matrix.

[0150] Then the attention weight matrix is ​​calculated. This matrix essentially fits the phase drift over time. The nonlinear cumulative relationship (i.e., random sampling frequency offset) Caused phase drift (correlation)

[0151]

[0152]

[0153] in, is the feature dimension, and T is the matrix transpose symbol.

[0154] (3) Output: After concatenating the outputs of all heads, the output is processed by LayerNorm and Feedforward Network (FFN) to finally output a feature vector containing the global phase drift pattern.

[0155] Functional principle:

[0156] Addressing the cumulative effect of random sampling frequency offset: In a multi-carrier frequency shift linear frequency modulation scheme, random sampling frequency offset... The resulting phase error is not constant, but accumulates nonlinearly in a quadratic manner over time. Traditional temporal feature extraction modules suffer from the vanishing gradient problem when processing long sequences (such as long radar frames), making it difficult to capture subtle correlations between the beginning and end of the sequence. This is due to random sampling frequency offsets. Caused phase drift It accumulates non-linearly over time, and its mathematical form approximately satisfies a quadratic function relationship:

[0157]

[0158] Global attention mechanism: The long-term feature extraction module calculates the global attention mechanism between any two time points in the sequence. and Correlation weight matrix between This means the network can directly focus on the start and end points of the pilot sequence. By comparing the cumulative phase difference between the beginning and end, it can fit the nonlinear phase drift curve caused by random sampling frequency shift with extremely high accuracy, achieving frequency offset estimation at the ppm level. This invention utilizes the global attention mechanism of the Transformer to fit the specific physical phase drift caused by random sampling frequency shift. This requires specific system-level technical design based on this invention, specifically addressing the self-attention calculation formula... The long-span phase differential operation of radar signals is equivalently mapped in the feature space. Specifically, before entering the Transformer, each feature vector needs to be added with an absolute position code (PE). Characteristics of information and with Characteristics of information Multiply by the learnable matrix respectively and Then, their inner product operation It possesses the ability to calculate in mathematics. The material basis of (i.e., nonlinear phase difference).

[0159] During the model training phase, this invention relies on a composite loss function that includes physical perception constraints for directional gradient guidance.

[0160] It should be noted that the multi-head self-attention mechanism used in this invention follows the standard Transformer architecture in its basic mathematical expression, but it can accurately fit radar signals. The core of nonlinear phase drift characteristics lies in: parameter orientation mapping under specific input representation design and physical constraints (referring to the I / Q dual-channel splitting of complex baseband signals and the superposition of absolute position encoding to enable network parameters to be oriented for fitting). (due to nonlinear drift), therefore, the radar baseband sequence must be superimposed with absolute position encoding before entering the long-time-series feature extraction module. The phase drift caused by random sampling frequency offset is a time index... Deterministic physical functions, learnable projection parameter matrices in the network and Under the supervision and constraints of the composite loss function of this invention, it is forced into directional optimization. This means that the conventional self-attention weight matrix... When processing the radar sequences of this invention, the internal inner product operation essentially calculates different time points. and The secondary phase correlation between them gives the general network structure a specific radar physical meaning, and achieves high-precision fitting of long-term phase drift.

[0161] Meanwhile, regarding random carrier frequency offset and random sampling time offset The problems that arise are also applicable to Transformer encoder networks, and the strong physical coupling requires the networks to share underlying features. Using the same network to perform joint feature extraction on these three types of entanglement damage avoids feature loss caused by manual stripping, and deep nonlinear coupling for multi-task learning has advantages.

[0162] Application scenarios: Particularly suitable for long pilot sequence or high bandwidth signal processing scenarios, as well as for capturing weak nonlinear errors in low signal-to-noise ratio environments.

[0163] End (Hybrid Head):

[0164] The output layer contains three parallel regressors, which respectively map the high-dimensional features output by the long-time feature extraction module to... The physical values.

[0165] 3. On the construction and training of joint synchronous neural networks.

[0166] In order to achieve high-precision synchronization under low-cost hardware conditions and to solve the problem of difficulty in obtaining the true label of synchronization error in real-world scenarios, this embodiment adopts the strategy of "simulation pre-training + physical self-supervised fine-tuning".

[0167] Simulation data generation:

[0168] Since the signal reception model (i.e., the mathematical model in this embodiment) is known, massive amounts of labeled training data are generated through computer simulation. Specifically, based on the formula:

[0169]

[0170] In pure transmission signal Above, artificially superimposed random sampling frequency offsets that follow a specific distribution. Carrier frequency offset and latency This allows the network to construct a pairwise dataset of {input signal, true parameter labels}. This enables the network to learn the mapping from distorted signals to physical parameters without the need for expensive calibration equipment. It should be understood that the labels are random sampling frequency offsets. Carrier frequency offset and latency The parameter value.

[0171] 4. Multi-task composite loss function and training strategy: To balance convergence speed and physical consistency in multi-dimensional parameter decoupling tasks, this embodiment designs a total loss function containing three components. :

[0172]

[0173] The components are defined as follows:

[0174] (1) Supervision regression loss Supervised learning of three random variables is performed using weighted mean squared error (MSE).

[0175]

[0176] in, These are the weighting coefficients for random carrier frequency offset, random sampling frequency offset, and random sampling time offset, defined to balance different physical dimensions and gradients, respectively. The predictions are generated by combining a synchronization neural network to address random carrier frequency offset, random sampling frequency offset, and random sampling time offset. The actual label value;

[0177] (2) Coarse-grained classification loss : Used for a coarse estimation of auxiliary carrier frequency offset, employing the cross-entropy loss function:

[0178]

[0179] in, The preset number of frequency offset classification categories is used, which is the total number of carrier frequency offset ranges for random carrier frequency offset. The first output of the network The softmax probability of the class; The target distribution is encoded using one-hot encoding. This encoding is constructed using simulated real carrier frequency offset labels. It determines which preset interval the real carrier frequency offset value falls into and sets the index position of the corresponding interval to 1, while setting the positions of all other intervals to 0. This serves as an unbiased standard probability distribution to guide network updates.

[0180] Specifically, the carrier frequency offset range is divided into several preset intervals to assist the network in making a rough classification estimate of the carrier frequency offset. The one-dimensional array data, obtained by mapping the output global feature vector through a fully connected layer, is then passed through a Softmax layer to obtain the classification result. .

[0181] (3) Physical perception loss function Self-supervised principle: To further improve the model's generalization ability on real data, this invention introduces a physical perception loss function. This loss function is based on a core physical fact: a perfectly synchronized linear frequency modulated (LFM / Chirp) signal, after demodulation, should have its energy highly focused in the frequency domain, forming a sharp spectral peak. Therefore, even with unlabeled measured data, synchronization quality can be evaluated by maximizing the "sharpness" of the demodulated spectrum. The calculation formula is as follows:

[0182]

[0183] in, The frequency spectrum amplitude after demodulation of the linear frequency modulated signal. It is a smoothing constant, typically a very small positive number such as... To prevent errors; Represents the energy of the spectral peak. This represents the total energy. Minimizing this loss function essentially forces the network to find a set of parameters that best conforms to the physical characteristics of LFM (Linear Frequency Modulation). This is an unsupervised physical guidance mechanism that greatly enhances the robustness of the algorithm. Specifically, it utilizes the network's predicted parameters to perform pre-compensation and demodulation operations on the input signal. The physical perception loss term is constructed by calculating the energy focusing degree or spectral sharpness of the demodulated signal in the frequency domain, forcing the network to output an estimate that conforms to the physical characteristics of energy focusing in a linear frequency modulated signal.

[0184] In other embodiments, the loss function can also be constructed using information entropy or spectral contrast. A sharply focused spectrum has the minimum information entropy and the maximum variance.

[0185] (4) Dynamic weight adjustment strategy: In the early stage of training, set... The network converges quickly using only supervised loss; as the number of training rounds increases, the loss gradually increases. To reach the preset value, the parameter estimates are fine-tuned using physical constraints. Similarly, It is to prevent getting trapped in local minima in the early stages, and it is a process of decreasing from large to small, which complements it.

[0186] 3. Inverse compensation and reconstruction of the full-frame signal (implementation of step 4).

[0187] After obtaining the joint estimate, this embodiment constructs an inverse compensation operator to "clean" and reconstruct the full-frame signal containing the data load. The steps are as follows:

[0188] Frequency offset correction: Eliminates random carrier frequency offset. The received signal is multiplied by an inverse complex exponential factor.

[0189]

[0190] Resampling correction: Eliminates random sampling frequency offset. Fractional interpolation resampling is performed using Farrow structure filters or polyphase filter banks to map the signal to a new time grid. This corrects for the scaling of the time axis.

[0191]

[0192] Delay alignment: Eliminates random sampling time offsets. A linear phase shift is applied in the frequency domain to achieve subsampling point precision shifting.

[0193]

[0194] After the above processing The signal has been restored to a standard orthogonal multicarrier frequency shift linear frequency modulation signal, eliminating interference from hardware damage.

[0195] 4. Feature fusion network based on environmental perception (implementation of step 5).

[0196] The synchronized signal is input into the motion-aware neural network. Addressing the vulnerability of radar sensing to environmental interference, this embodiment introduces "video motion recognition" and "domain adaptation" from the vision field, and innovatively designs a dual-stream architecture that separates static and dynamic signals.

[0197] Feature construction: Directly targeting a single synchronous baseband signal Perform a short-time Fourier transform (time-frequency transform) to generate a time-frequency graph, such as... Figure 3 The STFT spectrum is shown in the image. Because the deep synchronization network has eliminated nonlinear sampling frequency offset and random carrier frequency offset, the micro-Doppler texture on this time-frequency graph is clearly visible and is no longer affected by the "smearing" or "jitter" caused by hardware oscillator drift.

[0198] like Figure 3 As shown on the right, the environment-aware feature fusion network is a deep network containing static and dynamic branches as well as adaptive gated fusion units, used to process single-channel time-frequency feature matrices. .

[0199] Static Branch: Recent studies have shown that radar signals not only contain dynamic reflections of targets but also carry a large amount of background information (such as wall distances and multipath structures). Traditional methods often filter out static clutter, but this invention argues that static information is an important basis for judging the "complexity of the current environment." This static branch extracts background static features from the time-frequency graph. Global average pooling (GAP) is used to compress the time series into a static vector, which... In effect, it has become a “fingerprint” or “embedded representation” of the current physical environment (such as an empty room vs. a crowded office).

[0200] In this embodiment, the mathematical processing of static branches is as follows:

[0201] Let the input time-frequency feature map be... (in For frequency dimension, (Time dimension).

[0202] First, features are compressed in the time dimension using global average pooling (GAP) to extract the statistical steady-state features of the environment:

[0203]

[0204] Then, The input is fed into a fully connected layer (MLP), and environmental feature vectors are generated through nonlinear transformation. :

[0205]

[0206] in, For network weights, For network bias, For example, ReLU is an activation function.

[0207] Dynamic branch: It has a mean-reduction operation and a time-series feature extraction layer. This dynamic branch focuses on extracting the components in the signal that change drastically over time, namely the micro-Doppler effect produced by human movement.

[0208] In this embodiment, the mean removal operation involves subtracting the mean from the single-antenna time-frequency features (removing the DC component). Then, a multi-layer 1D-CNN or a time-series feature extraction module (LSTM network, etc.) is used to extract the time-series feature vector. The time-series feature vector The frequency variation patterns of actions such as waving and falling are encoded.

[0209] In this embodiment, the mathematical processing of dynamic branches is as follows:

[0210] The input is the time-frequency features after removing the mean. We utilize one-dimensional temporal convolution (Conv1D) to extract the variation patterns of micro-Doppler frequencies along the time axis. Let the... The convolution kernel of the layer is Then output feature map The calculation is as follows:

[0211]

[0212] In the formula, i is the output channel index, k is the time window size of the one-dimensional convolution kernel, and l is the network layer index. For time frame indexing, This is the input channel index.

[0213] After extraction through multiple layers of convolution stacking, the dynamic action feature vector is obtained. This vector encodes the frequency evolution pattern of the target action (such as falling or waving) over time.

[0214] Adaptive Gated Fusion: To address the problem of "decreased cross-domain recognition rate" (i.e., a model trained in room A fails in room B), this invention introduces a gating mechanism. It utilizes the environment vector extracted via static branching. Generate a computational gate weight :

[0215]

[0216] In the formula, and These represent the weights and biases of the gating mechanism, respectively.

[0217] Scenario A (Strong Clutter): If the environmental background is extremely complex (strong static features), the gating weights... It will automatically reduce and suppress noise that may be amplified in dynamic branches to prevent false alarms.

[0218] Scenario B (Pure Environment): If the environment is simple, the gating weights... As the sensitivity increases, the network will focus intently on the subtle features of dynamic branches, thus enhancing its sensitivity.

[0219] Final fusion features The results are fed into the classifier. This mechanism gives the device a human-like ability to "adjust attention." It allows the device to automatically adjust its strategy based on the complexity of the environment: if the background is extremely complex (strong static features), the gating weights are automatically reduced to suppress noise amplification; conversely, if the environment is clean, it focuses on subtle movement features. Finally, the action classification result is output through a Softmax layer.

[0220] It should be understood that existing technologies typically filter out static clutter or reconstruct the channel separately by separating static and dynamic characteristics, with the two branches being independent or mutually exclusive. However, this invention does not simply discard static features but instead designs an "interactive gating mechanism." The substantial breakthrough of this invention lies in using static environmental features as a "controller" to dynamically adjust the network's focus on action features.

[0221] Unlike existing technologies that only treat static components as interference filters or independently modeled, this invention innovatively constructs an environment-action interaction gating system. This is achieved through the formula... The calculated gating weights establish a nonlinear mapping between environmental complexity and feature confidence. This enables the model to "understand" the signal-to-noise ratio (SNR) of the current environment: automatically "shrinking" the sensing range (reducing noise) in cluttered corner environments. It automatically "expands" its sensitivity in open environments. This dynamic weighting mechanism based on environmental context is something that existing simple motion-static separation techniques lack, effectively solving the technical problem of the sharp performance drop of action recognition models when deployed across scenarios (Cross-Domain).

[0222] In some embodiments, the present invention also provides a multi-carrier frequency shift linear frequency modulation inductance integrated system, comprising:

[0223] The signal acquisition module is used to acquire radio frequency echo signals transmitted through a wireless channel using a radar receiver and convert them into discrete baseband sequences. The discrete baseband sequences contain various coupling time-frequency synchronization errors introduced by hardware non-ideals. The coupling time-frequency synchronization errors include at least random carrier frequency offset, random sampling frequency offset, and random sampling time offset.

[0224] The joint error estimation module is used to construct an input tensor based on the synchronization pilot band in the discrete baseband sequence, input it into a pre-trained joint synchronization neural network, and output a joint error estimate. , These are the random carrier frequency offsets. Random sampling frequency offset and random sampling time offset The estimated value;

[0225] The joint synchronous neural network includes a local feature extraction module, a long-term feature extraction module, and a regressor to capture the local phase texture and long-term nonlinear phase drift features of the signal.

[0226] The inverse compensation module is used to construct an inverse compensation operator based on the joint error estimate, and perform frequency offset correction, time resampling and translation alignment on the discrete baseband sequence to obtain the synchronous baseband signal.

[0227] In other embodiments, the system further includes an ISAC action recognition module for recognizing ISAC actions using a synchronous baseband signal. Specifically, this includes:

[0228] The feature extraction unit performs time-frequency transformation processing on the input baseband signal to generate a time-frequency feature matrix that includes the target micro-Doppler effect;

[0229] The feature separation and gating fusion unit inputs the time-frequency feature matrix into the environment-aware feature fusion network; the environment-aware feature fusion network is provided with static branches, dynamic branches, and an adaptive gating fusion unit.

[0230] The action recognition unit outputs classification labels for action postures based on the gated fusion features.

[0231] It should be understood that the specific implementation process of each module is described in the foregoing method embodiments, and will not be repeated here. The division of functional modules in the above device is merely illustrative. In some embodiments, some functional modules may be combined, some functional modules may be split, and each functional module may be implemented in software, hardware, or a combination of software and hardware.

[0232] In some embodiments, a multi-carrier FSCM sensing and communication integrated device includes at least: a radio frequency transceiver front end (radar transmitter and radar receiver) and a sensing and communication controller. In other embodiments, a digital signal processor (DSP) is also provided, and the radio frequency transceiver front end is connected to the DSP and the sensing and communication controller respectively (or interconnected via a bus);

[0233] The radio frequency transceiver front end is used to transmit multi-carrier frequency-shift linear frequency modulated signals and to collect radio frequency echo signals containing time-frequency synchronization errors transmitted through the wireless channel, converting them into discrete baseband IQ sequences.

[0234] A digital signal processor (DSP) is used to execute a long-sequence phase joint synchronization method for wireless signals. Specifically, the truncated synchronization pilot band is input into a pre-trained joint synchronization neural network (including a local feature extraction module front-end and a long-sequence feature extraction module). The single-time decoupling output is a joint estimate of random carrier frequency offset, random sampling frequency offset, and random sampling time offset. Then, an inverse compensation operator is constructed to perform frequency offset correction, resampling correction, and time delay alignment on the entire frame signal, and output a clean synchronization baseband signal.

[0235] The sensing and communication controller is used to perform parallel sensing processing. Specifically, it is configured to split the synchronous baseband signal into two paths. One path performs time-frequency analysis to demodulate the frequency-shift linear frequency modulation communication data. The other path constructs the received time-frequency feature matrix and inputs it into the motion sensing neural network. After separation of dynamic and static features and adaptive gating fusion, it outputs the action posture classification result of the target road segment.

[0236] It should be understood that, in the embodiments of the present invention, the digital signal processor and the sensing communication controller may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. In particular, for the deep neural network (joint synchronization network and action perception network) involved in the present invention, it is preferable to use an AI acceleration chip or graphics processing unit (GPU) with parallel computing capabilities to achieve efficient inference. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0237] In some embodiments, the digital signal processor is embedded in the communication sensing controller, and the sensing communication controller also performs the method for joint synchronization of long sequence phases of wireless signals.

[0238] In some embodiments, the present invention provides a computer-readable storage medium, characterized in that it stores a computer program, which is invoked by a processor to implement: the steps of a method for joint synchronization of long sequence phases of wireless signals or the steps of an ISAC action recognition method.

[0239] For details on the implementation of each step, please refer to the description of the control method embodiment above.

[0240] The readable storage medium is a computer-readable storage medium, which can be an internal storage unit of the hardware and software device described in any of the foregoing embodiments, such as the hard drive or memory of the controller. The readable storage medium can also be an external storage device of the controller, such as a plug-in hard drive, Smart MediaCard (SMC), Secure Digital (SD) card, or Flash Card equipped on the controller. Further, the readable storage medium can include both internal storage units and external storage devices of the controller. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium can also be used to temporarily store data that has been output or will be output.

[0241] Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0242] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application refers to flowchart illustrations and / or instructions executed by a processor of a method, apparatus (system), and computer program product according to embodiments of this application to create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams. These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowchart illustrations and / or one or more block diagrams. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more blocks of a block diagram.

[0243] It should be emphasized that the examples of this invention are illustrative rather than limiting. Therefore, this invention is not limited to the examples in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solutions of this invention, without departing from the spirit and scope of this invention, whether modifications or substitutions, are also within the protection scope of this invention.

Claims

1. A method for joint phase synchronization of long sequences of wireless signals, characterized in that: Includes the following steps: Step S1, Signal Acquisition: The radar receiver acquires the radio frequency echo signal transmitted through the wireless channel and converts it into a discrete baseband sequence; the discrete baseband sequence contains various coupling time-frequency synchronization errors introduced by hardware non-ideals, and the coupling time-frequency synchronization errors include at least random carrier frequency offset, random sampling frequency offset and random sampling time offset; Step S2, Joint Error Estimation: Construct an input tensor based on the synchronization pilot band in the discrete baseband sequence, input it into a pre-trained joint synchronization neural network, and output the joint error estimate. , These are the random carrier frequency offsets. Random sampling frequency offset and random sampling time offset The estimated value; The joint synchronous neural network includes a local feature extraction module, a long-term feature extraction module, and a regressor to capture the local phase texture and long-term nonlinear phase drift features of the signal. Step S3, Inverse Compensation: Construct an inverse compensation operator based on the joint error estimate, and perform frequency offset correction, time resampling and translation alignment on the discrete baseband sequence to obtain the synchronous baseband signal.

2. The method according to claim 1, characterized in that: The local feature extraction module of the joint synchronous neural network adopts a convolutional neural network (CNN), which has multiple stacked 2D convolutional layers, and each 2D convolutional layer is followed by batch normalization and activation functions. The long temporal feature extraction module employs a Transformer encoder and incorporates multiple head self-attention mechanisms—feedforward neural network layers—to capture the global dependencies of long sequences.

3. The method according to claim 1, characterized in that: The mathematical model for the discrete baseband sequence is: ； in: The sequence is a discrete baseband sequence, where n is the number of discrete sampling points; This is the ideal sampling interval for the radar receiver. Nominal sampling rate; Random carrier frequency offset: caused by local oscillator frequency drift, manifested as rapid rotation of the signal phase; Random sampling frequency offset: caused by ADC sampling clock deviation, manifested as a linear scaling factor on the time axis. ; Random sampling time offset: This manifests as a time delay for non-integer sampling points, affecting the accuracy of the radar's ranging starting point; It is additive white Gaussian noise; The multi-carrier frequency-shift linear frequency modulated baseband signal generated for the radar transmitter; This represents the gain coefficient for the complex channel.

4. The method according to claim 1, characterized in that: The process of performing frequency offset correction, time resampling, and translation alignment on the discrete baseband sequence in step S3 is as follows: Frequency offset correction: Based on the estimate of random carrier frequency offset Frequency offset correction is performed on the discrete baseband sequence to eliminate random carrier frequency offset: ； In the formula, It is a discrete baseband sequence. To eliminate the discrete baseband sequence after random carrier frequency offset, Where n is the nominal sampling rate, and n is the discrete sampling point. Resampling correction: an estimate based on the random sampling frequency offset Fractional interpolation resampling is performed to map the discrete baseband sequence signal onto a new time grid. Used to correct for time axis scaling: ； The above equation represents the remapping of the time coordinates for each discrete sampling point n, where, This is the ideal sampling interval for the receiver; Delay alignment: an estimate based on random sampling time offset By applying a linear phase shift in the frequency domain, a subsampling point precision shift is achieved, resulting in a standard orthogonal multicarrier frequency-shifted linear frequency modulated signal. ； In the formula, It is a standard orthogonal multicarrier frequency shift linear frequency modulation signal, eliminating interference from hardware damage; This is a discrete baseband sequence after frequency offset correction and resampling correction; FFT is Fast Fourier Transform, and IFFT is Inverse Fast Fourier Transform. The number of computation points for the Fast Fourier Transform (FFT) For discrete frequency indexes.

5. The method according to claim 1, characterized in that: The pre-training process of the joint synchronous neural network adopts a composite loss function based on dynamic weight constraints. The composite loss function is composed of a weighted sum of a supervised regression loss term, a coarse-grained classification loss term, and a physical perception loss term. The physical perception loss term is as follows: ； in, The amplitude of the frequency spectrum after demodulation. It is a smoothing constant. Represents the energy of the spectral peak. Represents total energy. This is a discrete frequency index, i.e., a frequency point number.

6. The method according to claim 1, characterized in that: The method further includes: the radar transmitter using multi-carrier frequency-shift linear frequency modulation (MFLM) technology, embedding communication data into the radar MFLM pulse by controlling the discrete frequency shift, wherein the multi-carrier frequency-shift linear frequency modulation full-bandwidth signal used for transmission... for: 、 ； in: Here, K is the power normalization coefficient, and K is the number of subcarriers. For the first The transmitted signal of each subcarrier, where t is time; The duration of the pulse is defined for a rectangular window function; The symbol period; For the first The center frequency of each subcarrier ensures that the subcarriers do not interfere with each other in the frequency domain; For linear frequency modulation slope, The sweep bandwidth for a single subcarrier; For discrete frequency shifts controlled by communication data, satisfying , For discrete frequency offset set The first, second, and Mth frequency offset values; for the... Subcarriers, Each time, truncate A bit as a communication data symbol And then based on the current communication symbols From discrete frequency offset set Select a corresponding frequency offset M is a positive integer.

7. An ISAC action recognition method, characterized in that: After obtaining the synchronization baseband signal using steps S1-S3 of the method described in any one of claims 1-6, the ISAC action recognition method further performs the following steps: Step A: Feature extraction, performing time-frequency transformation on the input baseband signal to generate a time-frequency feature matrix containing the target micro-Doppler effect; Step B: Feature separation and gating fusion, inputting the time-frequency feature matrix into an environment-aware feature fusion network; The environment-aware feature fusion network is equipped with static branches, dynamic branches, and an adaptive gating fusion unit. The static branch and the dynamic branch are used to extract static environmental features that characterize environmental complexity and dynamic action features that characterize the motion law of the target from the time-frequency feature matrix, respectively. The adaptive gating fusion unit generates gating weights based on the static environment features to characterize the complexity of the current environment; then, the gating weights are used to adaptively weight the dynamic action features to suppress noise interference in complex environments. Step C: Action recognition, outputting classification labels for action postures based on gated fusion features.

8. A multi-carrier FSCM integrated sensing device, characterized in that: At least including: Radar transmitter: Utilizing multi-carrier frequency shift linear frequency modulation (FSCM) technology, communication data is embedded into the radar linear frequency modulation pulse by controlling the discrete frequency shift amount, thereby generating a multi-carrier FSCM signal; Radar receiver: Collects radio frequency echo signals transmitted via wireless channels; The sensing communication controller is connected to the radar receiver and processes the radio frequency echo signal using the method described in any one of claims 1-6 to obtain a synchronous baseband signal; or processes the radio frequency echo signal using the method described in claim 7 to obtain a classification label for the ISAC action posture.

9. A system based on the method of any one of claims 1-6, characterized in that: include: The signal acquisition module is used to acquire radio frequency echo signals transmitted via wireless channels using a radar receiver and convert them into discrete baseband sequences. The discrete baseband sequence contains various coupling time-frequency synchronization errors introduced by hardware non-ideals. These coupling time-frequency synchronization errors include at least random carrier frequency offset, random sampling frequency offset, and random sampling time offset. The joint error estimation module is used to construct an input tensor based on the synchronization pilot band in the discrete baseband sequence, input it into a pre-trained joint synchronization neural network, and output a joint error estimate. , These are the random carrier frequency offsets. Random sampling frequency offset and random sampling time offset The estimated value; The joint synchronous neural network includes a local feature extraction module, a long-term feature extraction module, and a regressor to capture the local phase texture and long-term nonlinear phase drift features of the signal. The inverse compensation module is used to construct an inverse compensation operator based on the joint error estimate, and perform frequency offset correction, time resampling and translation alignment on the discrete baseband sequence to obtain the synchronous baseband signal.

10. A computer-readable storage medium, characterized in that: A computer program is stored, which is invoked by a processor to implement the steps of the method according to any one of claims 1-6 or the steps of the method according to claim 7.

Images