Identification method and electronic device based on radio frequency signal

Abstract

The present invention provides a method and electronic device for identifying an object based on radio frequency signals. The method comprises: inputting to-be-processed radio frequency echo data into a learning-driven embedding module of a recognition model to obtain a coded feature vector; inputting the coded feature vector into a scale-mixed encoder of the recognition model to obtain a multi-granularity coded feature; inputting the multi-granularity coded feature into a task-specific decoder of the recognition model to obtain a feature representation of the task; and inputting the feature representation of the task into a task prediction head of the recognition model to obtain a recognition result of the object to be identified, the recognition result including the identity information of the object to be identified. The present invention obtains the multi-granularity coded feature through the scale-mixed encoder, enabling feature extraction at multiple scales, thereby improving the robustness of the recognition model in determining the identity of the object to be identified.

Description

Technical Field

[0001] The present invention relates to the field of radio frequency human body perception, and in particular to an identity recognition method based on radio frequency signals. Background Art

[0002] Vision-based authentication technologies have made significant progress in recent years, but they still face fundamental limitations in obstructed and low-light environments. In contrast, radio frequency (RF) signals have the ability to penetrate non-metallic obstacles like walls and are unaffected by changes in lighting, making RF-based authentication a highly attractive solution.

[0003] Currently, in identity authentication technology based on radio frequency signals, human identity features are extracted from radio frequency echoes to identify the required information; however, this method lacks robustness and is difficult to cope with the complex and changing identity authentication needs in actual scenarios.

[0004] Therefore, a new identification method based on radio frequency signals is needed. Summary of the Invention

[0005] In view of this, the present invention provides an identity recognition method and electronic device based on radio frequency signals, in an effort to solve or at least alleviate the above problems.

[0006] According to one aspect of the present invention, a method for identity recognition based on radio frequency signals is provided, comprising: inputting radio frequency echo data to be processed into a learning-driven embedding module of a recognition model to obtain a coding feature vector, wherein the radio frequency echo data to be processed includes multiple radio frequency echo data spliced ​​according to acquisition time, the radio frequency echo data is obtained based on a radio frequency echo signal, and the radio frequency echo signal is obtained based on a radio frequency signal used to detect a target to be identified; inputting the coding feature vector into a scale hybrid encoder of the recognition model to obtain multi-granularity coding features; inputting the multi-granularity coding features into a task-specific decoder of the recognition model to obtain a feature representation of the task; and inputting the feature representation of the task into a task prediction head of the recognition model to obtain a recognition result of the target to be identified, wherein the recognition result includes identity information of the target to be identified.

[0007] Optionally, in the method according to the present invention, the RF echo data includes amplitude spectra and phase spectra of frequencies received by multiple virtual antenna elements, and the RF echo data to be processed is input into the learning-driven embedding module of the recognition model to obtain the encoded feature vector, including: generating a feature vector based on the amplitude spectrum and the phase spectrum; generating a spatial position code based on the virtual antenna elements corresponding to the feature vector and the frequency points corresponding to the frequency; generating a time position code based on the acquisition time of the RF echo data; and generating a coding feature vector based on the spatial position code, the time position code and the feature vector.

[0008] Optionally, in the method according to the present invention, the number of virtual antenna elements is determined according to the number of transmitting antennas and the number of receiving antennas.

[0009] Optionally, in the method according to the present invention, the scale hybrid encoder includes a cascaded first-scale hybrid attention module, a second-scale hybrid attention module, and a third-scale hybrid attention module; inputting the encoded feature vector into the scale hybrid encoder of the recognition model to obtain multi-granularity encoding features includes: inputting the encoded feature vector into the first-scale hybrid attention module to generate a first multi-granularity encoding feature; inputting the first multi-granularity encoding feature into the second-scale hybrid attention module to generate a second multi-granularity encoding feature; and inputting the second multi-granularity encoding feature into the third-scale hybrid attention module to generate a third multi-granularity encoding feature.

[0010] Optionally, in the method according to the present invention, the first-scale hybrid attention module includes a first-scale hybrid attention layer, and the encoded feature vector is input into the first-scale hybrid attention module to generate a first multi-granularity encoded feature, including: performing convolution operations on the query matrix, key matrix and value matrix of different grouped feature vectors according to multiple three-dimensional complex convolution kernels of different sizes provided by the first-scale hybrid attention layer to obtain query matrices, key matrices and value matrices of multiple scales; performing multi-head self-attention calculation based on the query matrix, key matrix and value matrix of multiple scales to obtain an attention output vector.

[0011] Optionally, in the method according to the present invention, the first-scale hybrid attention module also includes a first feature interaction feedforward layer, the first feature interaction feedforward layer includes feedforward layer convolution kernels of multiple sizes, and the encoded feature vector is input into the first-scale hybrid attention module to generate the first multi-granularity encoding feature, which also includes: performing a convolution operation on the first and second groups of attention output vectors after grouping according to the first feedforward layer convolution kernel to obtain a fifth group of attention output vectors; performing a convolution operation on the fifth group of attention output vectors and the third group of attention output vectors after grouping according to the second feedforward layer convolution kernel to obtain a sixth group of attention output vectors; performing a convolution operation on the sixth group of attention output vectors and the fourth group of attention output vectors after grouping according to the third feedforward layer convolution kernel to obtain a seventh group of attention output vectors; and splicing the first, fifth, sixth and seventh groups of attention output vectors to generate the first multi-granularity encoding feature.

[0012] Optionally, in the method according to the present invention, the task-specific decoder includes a task interaction module, and the multi-granularity coding features are input into the task-specific decoder of the recognition model to obtain the feature representation of the task, including: adding different weight parameters to the multi-granularity coding features according to the task interaction module, and splicing them to obtain the task interaction features.

[0013] Optionally, in the method according to the present invention, the task-specific decoder also includes a first task query module, a second task query module and a third task query module, and inputting the multi-granularity coding features into the task-specific decoder of the recognition model to obtain the feature representation of the task also includes: using the first multi-granularity coding features as the query matrix, the task interaction features as the key matrix and the value matrix, inputting them into the first task query module for cross-attention calculation to obtain the first feature representation; using the second multi-granularity coding features as the query matrix, the task interaction features as the key matrix and the value matrix, inputting them into the second task query module for cross-attention calculation to obtain the second feature representation; using the third multi-granularity coding features as the query matrix, the task interaction features as the key matrix and the value matrix, inputting them into the third task query module for cross-attention calculation to obtain the third feature representation.

[0014] Optionally, in the method according to the present invention, the recognition model includes a first task prediction head, a second task prediction head and a third task prediction head, and the feature representation of the task is input into the task prediction head of the recognition model to obtain the recognition result of the target to be identified, including: inputting the first feature representation into the first task prediction head to obtain the three-dimensional human joint point information of the target to be identified; inputting the second feature representation into the second task prediction head to obtain the action type of the target to be identified; inputting the third feature representation into the third task prediction head to obtain the identity information of the target to be identified, and the feature granularity of the third feature representation and the second feature representation is coarser than that of the first feature representation.

[0015] According to a second aspect of the present invention, an electronic device is provided, comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute an identity recognition method based on a radio frequency signal.

[0016] In the radio frequency signal-based identity recognition method of the present invention, radio frequency echo data is spliced ​​according to acquisition time to obtain unprocessed radio frequency echo data. This unprocessed radio frequency echo data is then input into a recognition model, enabling the recognition model to learn the time domain characteristics of the radio frequency echo signal. Furthermore, a scale hybrid encoder is used to encode multi-granularity coding features, fully exploiting feature information at multiple scales to obtain a feature representation of the task, thereby improving the robustness of the recognition model in determining the identity information of the target to be identified. Finally, the recognition model can output the identity information of the target to be identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0018] Figure 1 1 shows a schematic diagram of a multimodal data acquisition system 100 according to an embodiment of the present invention;

[0019] Figure 2 A schematic diagram of a camera node according to an embodiment of the present invention is shown;

[0020] Figure 3a A schematic diagram showing a configuration of a camera node, a transmitting antenna, and a receiving antenna according to an embodiment of the present invention is shown;

[0021] Figure 3b A schematic diagram of a virtual antenna array according to an embodiment of the present invention is shown;

[0022] Figure 3c A schematic diagram showing a configuration of a camera node, a transmitting antenna, and a receiving antenna according to another embodiment of the present invention is shown;

[0023] Figure 4 shows a schematic diagram of a learning driving module according to one embodiment of the present invention;

[0024] Figure 5 shows a schematic diagram of a recognition model 500 according to one embodiment of the present invention;

[0025] Figure 6 A schematic diagram of a radio frequency signal-based identity recognition method 600 according to an embodiment of the present invention is shown;

[0026] Figure 7 FIG2 shows a schematic diagram of a learning driven embedded module processing radio frequency echo data according to an exemplary embodiment of the present invention;

[0027] Figure 8 Schematic diagram of a scale-mixed encoder and a task-specific decoder according to one embodiment of the present invention is shown;

[0028] Figure 9 FIG2 shows a schematic diagram of a scale hybrid encoder according to another embodiment of the present invention;

[0029] Figure 10 A schematic diagram of a scale-mixed attention layer according to one embodiment of the present invention is shown;

[0030] Figure 11 A schematic diagram of a feature interaction feed-forward layer according to an embodiment of the present invention is shown;

[0031] Figure 12 A schematic diagram of a task interaction module according to an embodiment of the present invention is shown;

[0032] Figure 13 A schematic diagram of a task query module according to an embodiment of the present invention is shown;

[0033] Figure 14a 、 Figure 14b 、 Figure 14c 、 Figure 14d 、 Figure 14e and Figure 14f Schematic diagrams respectively showing a first recognition situation, a second recognition situation, a third recognition situation, a fourth recognition situation, a fifth recognition situation and a sixth recognition situation of a recognition model according to an embodiment of the present invention;

[0034] Figure 15a and Figure 15b Schematic diagrams respectively showing a seventh recognition situation and an eighth recognition situation of a recognition model according to an embodiment of the present invention;

[0035] Figure 16a 、 Figure 16b and Figure 16c Schematic diagrams respectively show the ninth recognition case, the tenth recognition case and the eleventh recognition case of the recognition model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] Embodiments of the present invention are described below with reference to the accompanying drawings. Various details of the embodiments of the present invention are included to facilitate understanding and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted from the following description.

[0037] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure and application of the data involved (including but not limited to user personal information) comply with the provisions of relevant laws and regulations, take necessary confidentiality measures, and do not violate public order and good morals.

[0038] Currently, RF-based identity authentication technologies fall into two main categories. The first involves a multi-stage approach, whereby a heat map is generated from the RF echoes and then fed into a neural network to extract the person's identity information. However, this indirect approach, requiring the generation of a heat map, requires complex data preprocessing, significantly reducing computational efficiency.

[0039] To address this issue, researchers further proposed a single-stage, end-to-end Transformer-based solution that can directly extract human identity features from RF echoes. However, this solution directly adopts the general attention mechanism originally designed for images or text, only utilizing the amplitude information of the RF signal and ignoring the frequency domain characteristics of the signal itself. In addition, this method fails to fully consider important characteristics inherent to RF perception tasks, such as signal sparsity and specular reflection properties. These shortcomings limit its overall performance.

[0040] To develop more robust RF-based authentication models, it is necessary to fully exploit the characteristic information in RF signals. RF signals contain both amplitude and phase information. Amplitude reflects the reflective properties of a target, while phase accurately describes its motion. However, existing single-stage methods only utilize amplitude information and fail to fully exploit the potential of phase information, thus hindering further improvement in model performance. Furthermore, at low frequencies, the human body behaves more like a reflector than an ideal scatterer, resulting in a single frame of RF echoes only capturing reflection information from a localized limb and lacking overall dynamic posture perception. Therefore, fully capturing motion information across multiple consecutive frames to expand temporal perception is a key task in improving identity recognition accuracy.

[0041] Existing methods lack sufficient consideration of the frequency domain characteristics and sparse structure of radio frequency signals during the modeling process, making it difficult to cope with the complex and ever-changing identity authentication needs in real-world scenarios. The present invention proposes a novel method for simultaneously modeling the frequency domain characteristics and sparse structure of radio frequency signals to improve the generalization and robustness of the model. It can continuously authenticate identities based on radio frequency signals and further identify the behavioral categories and human postures of the identified targets, providing more comprehensive reference information for identity authentication. The present invention solves the above-mentioned problems and is of great significance to radio frequency-based identity authentication technology.

[0042] To obtain training and validation data for the model, the present invention provides a multi-camera subsystem and a multi-input multi-output radar subsystem, which can be included in a multimodal data acquisition system. Figure 1 FIG. 1 is a schematic diagram of a multimodal data acquisition system 100 according to an embodiment of the present invention. Figure 1 As shown, the multi-camera subsystem 110 includes a plurality of camera nodes: a first camera node 111 , a second camera node 112 , . . . , an nth camera node 11 n .

[0043] Each camera node consists of a control module, an image acquisition module, and a communication module, all of which are interconnected. The communication module provides power to the control and image acquisition modules and provides communication functionality for the camera node. The image acquisition module captures and identifies targets, such as images of different people in action. The control module receives instructions and controls the image acquisition module.

[0044] Figure 2 FIG. 1 shows a schematic diagram of a camera node according to an embodiment of the present invention. Figure 2As shown, in the first camera node 111, the control module can be implemented as a computing device, such as a Raspberry Pi 201. The image acquisition module can be implemented as a camera 202. The communication module can be implemented as an Ethernet power supply module 203. Each camera node is easy to carry and deploy.

[0045] Back to Figure 1 The MIMO radar subsystem 120 includes a radio frequency simulator 121, a transmitting antenna module 122, and a receiving antenna module 123. The radio frequency simulator 121 can be implemented as a vector network analyzer, which generates radio frequency signals and transmits them into space via the transmitting antenna module 122. The vector network analyzer can generate a stepped frequency continuous wave from 0.8 GHz to 2.8 GHz as a transmitting signal, achieving a range resolution of approximately 75 mm.

[0046] The transmitting antenna module 122 includes at least one transmitting antenna, and the receiving antenna module 123 includes at least one receiving antenna. The at least one transmitting antenna and the at least one receiving antenna form a virtual antenna array.

[0047] Figure 3a FIG1 shows a schematic diagram of the arrangement of the camera node, the transmitting antenna and the receiving antenna according to an embodiment of the present invention. Figure 3a As shown, a first camera node 111 is exemplarily provided in a room, and multiple antennas are provided outside the room according to an antenna arrangement 310, including 12 receiving antennas in a receiving antenna module 123 and 12 transmitting antennas in a transmitting antenna module 122. The blocks in the receiving antenna module 123 represent receiving antennas, and the blocks in the transmitting antenna module 122 represent transmitting antennas.

[0048] According to one embodiment, the 12 transmitting antennas in the receiving and transmitting antenna module 123 are arranged in three rows, with four transmitting antennas in each row. In the transmitting antenna module 122, two adjacent transmitting antennas are arranged at a preset distance from each other.

[0049] According to one embodiment, the preset distance between the transmitting antennas can be set to n , to suppress the sidelobe effect; λ is the wavelength of the transmitting antenna, and n is an integer greater than or equal to 1.

[0050] According to one embodiment, when multiple receiving antennas and multiple transmitting antennas are co-located (e.g., in the same plane), a virtual antenna array can be formed. A virtual antenna array is not a set of real antennas, but rather a mathematical equivalent that describes the behavior of an antenna arrangement. Figure 3bA schematic diagram of a virtual antenna array according to one embodiment of the present invention is shown. The 12 transmit antennas in transmit antenna module 122 and the 12 receive antennas in receive antenna module 123 form a virtual antenna array 320. Each hollow square in virtual antenna array 320 represents a virtual antenna element. The 12 transmit antennas and 12 receive antennas form a total of 144 (12×12) virtual antenna elements. Number of virtual antenna elements in a virtual antenna array According to the number of transmitting antennas and the number of receiving antennas Calculation yields:

[0051]

[0052] Figure 3c FIG. 1 shows a schematic diagram of a camera node, a transmitting antenna, and a receiving antenna arrangement according to another embodiment of the present invention. Figure 3c As shown, multiple camera nodes are positioned around the room to capture images of the target 330 to be identified. Transmitting antenna module 122 and receiving antenna module 123 are separated from the target 330 by a wall 340. After the camera nodes are positioned, adjacent cameras are calibrated to align them relative to the same world coordinate system. This allows for the collection of identity tags, action tags, and 3D human joint information, providing cross-modal supervision for model training.

[0053] The image acquisition module primarily captures images of the identified target, such as different human bodies. The captured images include information such as the body's behavior and posture. The present invention can recognize images and obtain 3D human joint information. This 3D human joint information includes the coordinates of the 3D human joints in 3D space, after 3D spatial modeling of the target's preset range of motion.

[0054] The preset range of movement can be set to be within the image acquisition area of ​​the image acquisition module and the detection range of the MIMO radar subsystem. According to one embodiment, the preset range of movement has an altitude range of -2.56 meters to +2.56 meters and a distance range of 0 to 5.12 meters.

[0055] According to one embodiment, when the multimodal data acquisition system acquires data, the image acquisition frame rate of the image acquisition module of the camera node is 12 Hz, that is, 12 images are acquired within 1 second, and the optical image resolution is 1640×1248, unit: pixel.

[0056] According to one embodiment, a multiple-input, multiple-output radar subsystem transmits a radio frequency signal into space, where there are targets to be identified, such as different human bodies. It then receives radio frequency echo signals, which are derived from the radio frequency signals used to detect the targets to be identified. The radio frequency echo signals are acquired at a frame rate of 12 Hz, meaning 12 frames of radio frequency echo signals are collected per second. The dimensions of each frame of the radio frequency echo signal are equal to the number of virtual antenna elements in the virtual antenna array multiplied by the number of frequency bins.

[0057] According to one embodiment, the number of virtual antenna elements in the virtual antenna array is 144. The frequency of the RF echo is 0.8 GHz to 2.8 GHz. A frequency point is a number of fixed frequencies, and each frequency point is set at a frequency point every 0.01 GHz and sampled. For this reason, the total number of frequency points included is 201. The dimensions of each frame of the RF echo signal are 144 × 201.

[0058] Thanks to its low frequency band, wide bandwidth, and high virtual antenna density, the MIMO radar subsystem achieves high spatial resolution while maintaining excellent penetration. According to one embodiment, synchronization between the MIMO radar subsystem and multiple cameras is achieved via the Network Time Protocol. Before data collection, all devices are pre-synchronized with a local time server. During the collection process, the server sends a specified timestamp to each device via a Transmission Control Protocol connection, ensuring millisecond-level synchronization. Specifically, the server can be implemented as a server that sends collection instructions and receives collected data.

[0059] Before the radio frequency echo signal is input into the recognition model, the radio frequency echo signal is first processed to obtain radio frequency echo data.

[0060] According to one embodiment, the dimension of each frame of RF echo signal is the number of virtual antenna elements ( ) × number of frequency points (F). Each virtual antenna element is considered to be able to independently transmit and receive RF signals, and therefore can receive signals at different frequencies. When the number of virtual antenna elements is 144 and the number of frequency points is 201, the dimension of each frame of RF echo signal is 144 × 201.

[0061] According to one embodiment, the radio frequency echo signal X(u,v) is determined based on the amplitude spectrum A and phase spectrum θ of the radio frequency echo signal. The value range of u is 1- , the value range of v is 1-F.

[0062] X(u,v)= Re[X(u,v)]+ jIm[X(u,v)]

[0063] X(u,v) is a function of u and v, j is an imaginary number, =-1.

[0064] The real part Re[X(u,v)] and the imaginary part Im[X(u,v)] are calculated based on the amplitude spectrum A and the phase spectrum θ:

[0065] Re[X (u,v)]= A(u,v)·cos[θ(u,v)]

[0066] Im[X(u,v)]=A(u,v)·sin[θ(u,v)]

[0067] Solving the above formula yields the real part Re[X(u,v)] and the imaginary part Im[X(u,v)], allowing X(u,v) to be used as the eigenvector to obtain the RF echo data for each frame. X(u,v) is a complex number, and the RF echo data includes the amplitude and phase spectra of the frequencies received by multiple virtual antenna elements.

[0068] Existing standard attention mechanisms, when directly applied to RF signals, often ignore the rich frequency domain information in RF signals, resulting in degraded model performance. To address this issue, the signal's amplitude and phase information is converted into a real-imaginary representation, thereby fully preserving the frequency domain characteristics of the RF signal.

[0069] According to one embodiment, the present invention also provides a learning-driven embedding module that splices RF echo data along the time dimension and implicitly maps it to a latent space representation of the recognition target information, which includes richer time information to alleviate the problem of specular reflection of RF signals; it can also effectively utilize time domain information to improve the overall performance of the recognition model.

[0070] Figure 4 FIG. 1 shows a schematic diagram of a learning driven embedding module according to an embodiment of the present invention. Figure 4 As shown, RF echo signal 401 is preprocessed to generate RF echo data 402. This data is then input into a learning-driven embedding module 410. This module concatenates each frame of RF echo data (including two dimensions, namely the number of virtual antenna elements 403 and the number of frequencies 404) along time 405, implicitly mapping it into a three-dimensional spatiotemporal latent space 406. This 3D spatiotemporal latent space 406 includes the time, amplitude, and phase information of the RF echo signal, implicitly containing the feature information needed to acquire the target for identification. The recognition model can extract this feature information to achieve various recognition tasks.

[0071] A single RF echo can only capture information about a portion of a person's limbs. Given the continuity of human motion, RF echoes at different time points contain information about different body parts. Therefore, it's necessary to fully utilize time series information to mitigate the pathological underdetermination caused by specular reflections of RF signals. By introducing long-term time domain information, the underdetermination caused by specular reflections of low-frequency signals is effectively mitigated, enabling accurate identification of the target and reconstruction of their behavior and posture.

[0072] Figure 5 FIG. 5 shows a schematic diagram of a recognition model 500 according to an embodiment of the present invention. Figure 5 As shown, the recognition model to be trained in the present invention includes a learning-driven embedding module 510 (which can be specifically implemented as a three-dimensional patch embedding module, 3Dpatch Embedding), a scale-mix encoder 520 (Scale-Mix Encoder), a task-specific decoder 530 (Task-specific Decoder) and a task prediction head 540.

[0073] Figure 6 FIG. 6 shows an identity recognition method 600 based on radio frequency signals according to an embodiment of the present invention. Figure 6 As shown, first, step 610 is executed to input the radio frequency echo data to be processed into the learning driven embedding module of the recognition model to obtain a coding feature vector.

[0074] The learning-driven embedding module receives an input data set. The data set can be divided into a training set and a test set. The training set includes training samples, and the test set includes test samples. The recognition model can be pre-trained using the training samples in the training set. The samples in the data set are radio frequency echo data annotated with annotation information. When the recognition model is used to perform a recognition task, the radio frequency echo data to be processed is input into the recognition model. The present invention does not impose any restrictions on the number of radio frequency echo data specifically included in the radio frequency echo data to be processed; for example, it can be implemented as including 12 frames of radio frequency echo data.

[0075] According to one embodiment, multiple RF echo data sets containing annotated information can be used as the RF echo data to be processed. This data set can be implemented as a single sample (training sample or test sample) when training a recognition model. For example, 12 consecutive frames of RF echo data can be used as a single sample. The present invention does not limit the number of RF echo data sets included in a sample. Furthermore, multiple samples can be collected as a dataset, and the dataset can be divided into a training set and a test set in a 4:1 ratio.

[0076] According to one embodiment, a training sample includes 12 frames of RF echo data, annotated with annotation information. The annotation information includes an identity tag, an action tag, and 3D human joint information. The identity tag includes a unique identifier of at least one identified target detected by the RF signal, such as the ID of each identified target. The action tag includes at least one tag indicating the current posture or action of the identified target. The 3D human joint information includes the coordinates of the 3D human joints of the identified target at the time corresponding to the current frame. 3D human joints can be points that contain human motion information. Their number and type can be preset, and can specifically include highly flexible human joints, such as wrists and ankles, which can be set as joints.

[0077] Each training sample can be specifically implemented as a A three-dimensional array M of size t × H × W, where M[t][u][v] represents the element in the tth frame, uth row, and vth column. T is the number of frames within the selected preset time interval. The preset time interval can be 1 second. Based on the frame rate of the acquired RF echo signal (12 frames / second), the number of frames T is calculated to be 12. The value of t ranges from 1 to T. H is determined by the number of virtual antenna elements, and W is determined by the total number of frequency points. Each element contains a complex number of RF echo data, including the amplitude spectrum and phase spectrum.

[0078] Figure 7 FIG. 1 shows a schematic diagram of a learning driven embedded module performing an encoding operation according to an exemplary embodiment of the present invention. Figure 7 As shown, first, step 710 is executed to perform a block operation on the radio frequency echo data to be processed. According to one embodiment, the dimension of the radio frequency echo data to be processed is ×H×W, divide the RF echo data to be processed into multiple patches, and generate feature vectors for the patches: each patch includes a complex number containing the amplitude spectrum and phase spectrum, and the complex number containing the amplitude spectrum and phase spectrum is used as the feature vector .

[0079] Then, step 720 is executed to generate a spatial position code according to the virtual antenna element corresponding to the eigenvector and the frequency point corresponding to the frequency, generate a time position code according to the acquisition time of the RF echo data, and add a spatial position code to the small block. , add the information corresponding to the spatial position encoding to the feature vector In the new feature vector Spatial Position Encoding This may include the coordinate value u of each small block in the dimension of the number of virtual antenna elements, and the coordinate value v in the dimension of the number of frequency points. According to one embodiment, if the feature vector is generated based on the 87th virtual antenna element and the acquisition frequency is 0.9 GHz, it corresponds to the 11th frequency point, and the spatial position code is generated based on 87 and 11. If the feature vector is the 10th frame of RF echo data, the temporal position code is generated based on 10.

[0080] Then, step 730 is executed to add time position coding to the small block. , add the information corresponding to the time position encoding to the feature vector Finally, we get the encoding feature vector generated by spatial position encoding, temporal position encoding and feature vector . Temporal Position Encoding It may include the frame number of the corresponding frame of the radio frequency echo data where each small block is located, that is, the coordinate value t in the time dimension.

[0081] Then, step 740 is executed to output the first Padding is performed. Padding is required to prevent convolution operations on the RF echo data where the length of at least one dimension is not divisible by the side length of the convolution kernel. The padding size is determined based on the remainder and the side length of the convolution kernel. The remainder is the remainder left when the length of the different dimensions of the RF echo data is not divisible by the side length of the convolution kernel. The padding size is determined by subtracting the remainder from the side length of the convolution kernel. Padding the input sequence allows processing of input sequences of varying lengths.

[0082] According to one embodiment, when performing a filling operation, the first output The size is ×144×201( ×H×W), including ×144×201 encoded feature vectors , the first complex three-dimensional convolution kernel size for convolution is 1×20×24 (respectively Dimension, H dimension and W dimension), the remainders when the number of virtual antenna elements (H dimension) and the number of frequency points (W dimension) cannot be divided by the convolution kernel are 4 and 9 respectively, so the padding size is 16 and 15 respectively. After padding, the second output is 12×160×216. .

[0083] Finally, step 750 is executed to output the second The convolution operation is performed using the convolution kernel. According to one embodiment, the padded second output The size of the convolution operation is 12×160×216. The convolution kernel is the first complex three-dimensional convolution kernel, which has a size of 1×20×24 ( ×H×W), with a step size of 1×20×24 (respectively dimension, stride in H dimension and W dimension). The second output is processed according to the first complex 3D convolution kernel. Perform convolution to get the third output .

[0084] According to one embodiment, the RF echo data is normalized by using LayerNorm, such as normalizing the real and imaginary parts of the plurality of RF echo data respectively. Specifically, the third output Each encoded feature vector of The real and imaginary parts are normalized separately to obtain the fourth output ; which includes multiple normalized encoded feature vectors .

[0085] The learning-driven embedding module outputs the encoding of each small block in the processed RF echo data. Each block's encoding includes both spatial and temporal position encodings. The output training sample consists of N normalized encoding feature vectors. To preserve spatial and temporal position information, learnable spatial and temporal position encodings are introduced. These position encodings help the recognition model better understand spatiotemporal structure. The recognition model learns how to utilize these position encodings during training, dynamically adjusting the spatial and temporal representations during the learning process, thereby enhancing the model's ability to model complex spatiotemporal relationships. By effectively combining spatiotemporal information, the recognition model can more accurately capture subtle changes in the RF signal.

[0086] Back to Figure 6 ,Then, step 620 is executed to input the encoded feature vector into the scale hybrid encoder of the recognition model to obtain multi-granularity encoded features.

[0087] The scale-mixture encoder may include multiple cascaded scale-mixture attention modules. Figure 8 FIG. 1 shows a schematic diagram of a scale-mixed encoder and a task-specific decoder according to an embodiment of the present invention. Figure 8 As shown, the scale-mixed encoder 520 includes a cascaded first-scale-mixed attention module 521 , a second-scale-mixed attention module 522 , and a third-scale-mixed attention module 523 . Figure 8 The construction method of the scale-mixed encoder shown is only exemplary, and the present invention does not limit the number of scale-mixed attention modules specifically included in the scale-mixed encoder.

[0088] As indicated by arrow 81, the scale-mixed encoder 520 receives the output of the learning-driven embedding module 510, and this output is first processed by the first-scale-mixed attention module 521. As indicated by arrow 82, the output of the first-scale-mixed attention module 521 serves as the input to the second-scale-mixed attention module 522. As indicated by arrow 83, the output of the second-scale-mixed attention module 522 serves as the input to the third-scale-mixed attention module 523. The shallow-level scale-mixed attention modules output finer-grained features for pose estimation, while the deep-level scale-mixed attention modules output coarser-grained features for action recognition and person re-identification.

[0089] According to one embodiment, the scale-mixed attention module includes a downsampling layer, at least one normalization layer, an attention layer, and a feed-forward layer. The attention and feed-forward layers are used to learn to generate multi-granular features, which are then fed into a task-specific decoder to generate feature representations related to characteristics such as person identity.

[0090] Figure 9 FIG. 4 shows a schematic diagram of a scale hybrid encoder according to another embodiment of the present invention. Figure 9 As shown, the first-scale hybrid attention module 521 includes a first downsampling layer 911, a first hybrid normalization layer 912, a first-scale hybrid attention layer 913, a second hybrid normalization layer 914, and a first feature interaction feedforward layer 915. The first-scale hybrid attention module 521 generates a first multi-granularity encoding feature 916 based on the input encoding feature vector.

[0091] In the first scale hybrid attention module 521, the first downsampling layer 911 can be implemented as a stem layer, which is suitable for downsampling the output of the learning driven embedding module 510; in specific implementation, a convolution kernel can be used to downsample the fifth output Perform convolution. The size of the second complex three-dimensional convolution kernel can be realized as 3×3×3 ( ×H×W), the step size can be realized as 2×2×2 ( × H × W), and get the sixth output .

[0092] Then, the sixth output Input to the first hybrid normalization layer 912, the first hybrid normalization layer 912 can be specifically implemented as a Complex layerNorm layer, and the sixth output Each eigenvector in is normalized to obtain the seventh output .

[0093] Then, the seventh output Input to the first scale mixed attention layer 913. Figure 10FIG. 5 shows a schematic diagram of a scale-mixed attention layer according to an embodiment of the present invention. Figure 10 As shown, first the seventh output The encoded feature vector in generates the query matrix 1002 (Q matrix), key matrix 1003 (K matrix) and value matrix 1004 (V matrix) through the weight matrix 1001. The weight matrix 1001 includes the query weight matrix ( ), key weight matrix ( ) and the value weight matrix ( ). When generating the query matrix, key matrix, and value matrix, the following formula can be used:

[0094]

[0095]

[0096]

[0097] in, 、 and Represent the query matrix, key matrix and value matrix of the i-th encoded feature vector respectively; 、 and Represent the query weight matrix, key weight matrix and value weight matrix of the i-th encoded feature vector respectively; () represents multi-granularity patch aggregation, which uses complex 3D convolution kernels of different sizes to achieve feature aggregation at different scales. is the i-th encoded feature vector of the input, where is a 3D complex convolution kernel of different sizes, and o is 1-S, where S is the number of 3D complex convolution kernels. In this way, the scale-mixed attention layer can capture richer spatiotemporal features at multiple scales, thereby enhancing its ability to process complex scenes.

[0098] According to one embodiment, when generating the query matrix, key matrix, and value matrix, the encoded feature vector may be processed according to a multi-head attention mechanism. Multiple attention heads have different weight matrices. For example, when a multi-head attention mechanism is used to process the feature vector, there are 8 different attention heads. Then the weight matrix of the first attention head is the first query weight matrix ( ), the first bond weight matrix ( ) and the first value weight matrix ( ), ..., the weight matrix of the eighth attention head is the eighth query weight matrix ( ), the eighth key weight matrix ( ) and the eighth value weight matrix ( ). For the encoded feature vector, a query matrix, a key matrix, and a value matrix are generated based on the weight matrices of different attention heads. Specifically, the weight matrix of the first attention head can be used to generate a first query matrix, a second key matrix, and a third value matrix for the encoded feature vector, and so on, and the weight matrix of the eighth attention head can be used to generate an eighth query matrix, an eighth key matrix, and an eighth value matrix.

[0099] Subsequently, the convolution kernel is used to perform a convolution operation on the query matrix, key matrix, and value matrix of the coded feature vector. According to one embodiment, the convolution operation is performed on the query matrix, key matrix, and value matrix of the coded feature vectors of different groups using a plurality of three-dimensional complex convolution kernels of different sizes provided by the first scale mixed attention layer. The plurality of three-dimensional complex convolution kernels include a size of 1×1×1 ( ×H×W) of the third complex three-dimensional convolution kernel 1005, the size of which is 3×3×3 ( ×H×W) of the fourth complex three-dimensional convolution kernel 1006, the size of 5×5×5 ( ×H×W) and a fifth complex three-dimensional convolution kernel 1007 of size 7×7×7 ( ×H×W).

[0100] When convolution kernels are used to convolve the query, key, and value matrices of the encoded feature vectors, all encoded feature vectors of the RF echo data are grouped. These groups can be divided into a corresponding number of groups based on the number of three-dimensional complex convolution kernels used, for example, four groups containing 216 feature vectors. Three-dimensional complex convolution kernels of varying sizes are used to convolve the query, key, and value matrices of the feature vectors in different groups, downsampling them at varying scales and focusing on features at different scales. Each attention head can also capture feature information at different scales, enabling the recognition model to simultaneously process spatiotemporal features at multiple granularities and effectively combine low-level and high-level feature information.

[0101] During specific implementation, the third complex three-dimensional convolution kernel 1005 can be assigned to perform convolution operations on the query matrix, key matrix and value matrix of the first group of encoded feature vectors, the fourth complex three-dimensional convolution kernel 1006 can perform convolution operations on the query matrix, key matrix and value matrix of the second group of encoded feature vectors, the fifth complex three-dimensional convolution kernel 1007 can perform convolution operations on the query matrix, key matrix and value matrix of the third group of encoded feature vectors, and the sixth complex three-dimensional convolution kernel 1008 can perform convolution operations on the query matrix, key matrix and value matrix of the fourth group of encoded feature vectors.

[0102] Since the query matrix, key matrix, and value matrix obtained according to the complex three-dimensional convolution kernels of different sizes are of different sizes, padding operations are performed on the query matrices, key matrices, and value matrices of different sizes to obtain the query matrix, key matrix, and value matrix of the same size after convolution.

[0103] Subsequently, the query matrix, key matrix and value matrix after convolution of multiple scales and the same size are input into the scale mixed attention module 1010 to perform multi-head self-attention calculation to obtain the eighth output .

[0104] According to one implementation, the scale-mixed attention module 1010 uses a complex softmax function to calculate the attention weights, ensuring that features of different scales have different importance in the information fusion process:

[0105]

[0106] Where, represents the attention weight of the i-th encoded feature vector, is the complex softmax function, represents the transposed matrix of the key matrix of the i-th encoded feature vector, represents the dimension of the key vector, is the scaling factor.

[0107] Back to Figure 9 , as shown by arrow 91, the eighth output With the sixth output After the residual connection, they are used as the input of the second hybrid normalization layer 914. The second hybrid normalization layer 914 can be specifically implemented as a Complex layerNorm layer, which performs a normalization operation on the input to obtain the ninth output , which is the attention output vector.

[0108] Then, the ninth output Input the first feature interaction feed-forward layer 915. Figure 11 FIG. 4 shows a schematic diagram of a feature interaction feedforward layer according to an embodiment of the present invention. Figure 11 As shown, the first feature interaction feed-forward layer includes a first convolutional layer 1110 and a second convolutional layer 1120. Traditional feed-forward networks limit the interactive flow of features between different scales. Therefore, the feature interaction feed-forward layer designed in the present invention allows finer-grained features to flow to coarser-grained paths, thereby more effectively utilizing information at different scales.

[0109] According to one embodiment, the first convolution layer 1110 and the second convolution layer 1120 include a 1×1×1 complex three-dimensional convolution kernel. First, the ninth output is processed according to the first convolution layer 1110. Perform convolution to get the tenth output .

[0110] Then the tenth output The attention output vectors in are grouped to obtain four groups of attention output vectors, including the first group, the second group, the third group and the fourth group of attention output vectors.

[0111] The first, second, and third feed-forward layer convolution kernels are also provided in the first feature interaction feed-forward layer 915, which can be specifically implemented as a seventh complex three-dimensional convolution kernel 1101, an eighth complex three-dimensional convolution kernel 1102, and a ninth complex three-dimensional convolution kernel 1103. According to one embodiment, the size of the seventh complex three-dimensional convolution kernel 1101 is 3×3×3, the size of the eighth complex three-dimensional convolution kernel 1102 is 5×5×5, and the size of the ninth complex three-dimensional convolution kernel 1103 is 7×7×7.

[0112] The first and second groups of attention output vectors after grouping are convolved according to the convolution kernel of the first feedforward layer to obtain the fifth group of attention output vectors; the fifth group of attention output vectors and the third group of attention output vectors after grouping are convolved according to the convolution kernel of the second feedforward layer to obtain the sixth group of attention output vectors; the sixth group of attention output vectors and the fourth group of attention output vectors after grouping are convolved according to the convolution kernel of the third feedforward layer to obtain the seventh group of attention output vectors; finally, the first, fifth, sixth and seventh groups of attention output vectors are spliced ​​to generate the first multi-granularity encoding feature.

[0113] When the first, fifth, sixth and seventh groups of attention output vectors are concatenated, the first, fifth, sixth and seventh groups of attention output vectors can be padded to make the size of each group of attention output vectors the same, so that the first, fifth, sixth and seventh groups of attention output vectors of the same size can be concatenated to obtain the eleventh output .

[0114] Then, the eleventh output Input the second convolutional layer 1120, perform convolution operation to obtain the twelfth output .

[0115] Back to Figure 9 As shown by arrow 92, the eighth output after the residual connection and the sixth output With the twelfth output Perform another residual connection together to generate the first multi-granularity encoding feature 916.

[0116] Subsequently, the first multi-granularity encoded features 916 are input into the second-scale hybrid attention module 522 to generate second multi-granularity encoded features 926. The second hybrid attention module 522 includes a second downsampling layer 921, a third hybrid normalization layer 922, a second-scale hybrid attention layer 923, a fourth hybrid normalization layer 924, and a second feature interaction feedforward layer 925. The second-scale hybrid attention module 522 processes the first multi-granularity encoded features 916 in a similar manner to the first-scale hybrid attention module 521. Please refer to the processing flow of the first-scale hybrid attention module 521 and will not be repeated here.

[0117] Subsequently, the second multi-granularity encoded features 926 are input into the third-scale hybrid attention module 523 to generate third multi-granularity encoded features 936. The third-scale hybrid attention module 523 includes a third downsampling layer 931, a fifth hybrid normalization layer 932, a third-scale hybrid attention layer 933, a sixth hybrid normalization layer 934, and a third feature interaction feedforward layer 935. The third-scale hybrid attention module 523 processes the second multi-granularity encoded features 926 in a manner similar to that of the first-scale hybrid attention module 521. Please refer to the processing flow of the first-scale hybrid attention module 521 and will not be repeated here.

[0118] Based on the scale-mixed attention module, this paper designs a multi-stage scale-mixed encoder to encode spatiotemporal features of different granularities; by capturing more diverse features in RF signals, the robustness of the recognition model is improved.

[0119] Back to Figure 8 ,like Figure 8 As shown, the task-specific decoder 530 includes a task interaction module 531 , a first task query module 532 , a second task query module 533 and a fourth task query module 534 . Figure 8 The construction method of the task-specific decoder 530 shown is only exemplary, and the present invention does not limit the number of task query modules included in the task-specific decoder 530.

[0120] Back to Figure 6 , then execute step 630, input the multi-granularity encoding features into the task-specific decoder of the recognition model to obtain the feature representation of the task; specifically, first add different weight parameters to the multi-granularity encoding features according to the task interaction module, and splice them to obtain the task interaction features.

[0121] Figure 12 FIG. 1 shows a schematic diagram of a task interaction module according to an embodiment of the present invention. Figure 12As shown, task interaction module 531 receives first multi-granularity encoded features 916, second multi-granularity encoded features 926, and third multi-granularity encoded features 936 as input. Considering that different human perception tasks share common features of human structure, the task interaction module fuses features from different tasks. By interactively fusing features from multiple tasks, the module is able to extract shared structural information while preserving task-specific detail features.

[0122] The task interaction module 531 includes at least one fully connected layer. Figure 12 As shown, the task interaction module 531 includes a first fully connected layer 1210, a second fully connected layer 1220, and a third fully connected layer 1230. The first multi-granularity encoded feature 916 is first input into the first fully connected layer 1210 to obtain a fourth multi-granularity encoded feature. The second multi-granularity encoded feature 926 is input into the second fully connected layer 1220 to obtain a fifth multi-granularity encoded feature. The third multi-granularity encoded feature 936 is input into the third fully connected layer 1230 to obtain a sixth multi-granularity encoded feature. The first fully connected layer 1210, the second fully connected layer 1220, and the third fully connected layer 1230 can be specifically implemented as liner layers. The present invention does not limit the specific implementation of the fully connected layers.

[0123] Subsequently, the fourth, fifth, and sixth multi-granularity coding features are concatenated to obtain the first task interaction feature 1201. When concatenating the multi-granularity coding features, if the feature dimensions are of different sizes, the multi-granularity coding features are padded to make them of the same size before concatenating them. According to one embodiment, when concatenating the fourth, fifth, and sixth multi-granularity coding features, the value of the imaginary part of the multi-granularity coding features is extracted as a new feature vector in the multi-granularity coding features, thereby converting the complex feature vector included in the multi-granularity coding features into two real feature vectors, reducing the computational complexity of the subsequent task-specific decoder, thereby realizing a lightweight decoder capable of decoding multi-granularity features and achieving robust identity authentication.

[0124] Finally, weight parameters 1202 are added to the fourth, fifth, and sixth multi-granularity coding features in the first task interaction feature 1201 to obtain the second task interaction feature 1203. According to one embodiment, the weight parameters added to each multi-granularity coding feature can be different; the specific process is shown in the following formula:

[0125]

[0126] represents the generated second task interaction feature, Represents the mth multi-granularity coding feature, and the value range of m is 1- , Indicates the number of multi-granularity encoding features of the input; Represents the weight parameter of the m-th multi-granularity encoding feature.

[0127] Back to Figure 8 As shown by arrows 84 and 87, the first multi-granularity encoding features generated by the first-scale hybrid attention module 521 are used as the query matrix, and the second-task interaction features are used as the key matrix and value matrix, and are input into the first-task query module for cross-attention calculation to obtain a first feature representation. As shown by arrows 85 and 88, the second multi-granularity encoding features generated by the second-scale hybrid attention module 522 are used as the query matrix, and the second-task interaction features are used as the key matrix and value matrix, and are input into the second-task query module for cross-attention calculation to obtain a second feature representation. As shown by arrows 86 and 89, the third multi-granularity encoding features generated by the third-scale hybrid attention module 523 are used as the query matrix, and the second-task interaction features are used as the key matrix and value matrix, and are input into the third-task query module for cross-attention calculation to obtain a third feature representation. The third feature representation and the second feature representation undergo more convolutions during the generation process, so the feature granularity is coarser than that of the first feature representation.

[0128] According to one embodiment, the task query module uses multi-granularity encoding features as the query matrix (Query), task interaction features as the key matrix (Key) and value matrix (Value) to generate the feature representation required for a specific task, as shown in the following formula:

[0129]

[0130]

[0131] Where, Represents multi-granularity coding features, x takes the value of 1-R, R is the number of multi-granularity coding features, represents the second task interaction feature; () represents the normalization operation performed by the normalization layer, () represents the crisscross attention calculation performed by the crisscross attention layer; represents the cross attention output, () represents the operation performed by the multilayer perceptron, Represents the feature representation output by the task query module, x takes the value of 1-R, where R is the number of multi-granularity encoding features.

[0132] Figure 13 FIG1 shows a schematic diagram of a task query module according to an embodiment of the present invention. Taking the first task query module 532 as an example, the data processing process of the task query module is described. Figure 13As shown, the task query module 532 includes a first normalization layer 1310 and a second normalization layer 1320. The task query module 532 converts the first multi-granularity encoding feature 916 (i.e., when x is 1) into ) is input into the first normalization layer 1310 to obtain the first normalized output, i.e. ; The second task interaction feature 1203 ( ) is input into the second normalization layer 1320 to obtain the second normalized output, that is, .

[0133] Subsequently, the first normalized output and the second normalized output are input into the cross attention layer 1330 to obtain the cross attention output, i.e. As shown by arrow 1301, the cross attention output and the first multi-granularity encoding feature are then residually connected and input together into the third normalization layer 1340 to obtain the third normalized output, i.e. The third normalized output is input into a multilayer perceptron (MLP) 1350 to obtain an MLP output. As indicated by arrow 1302, the residual-connected cross-attention output, the first multi-granularity encoded feature, and the MLP output are then subjected to another residual connection, and together serve as the first feature representation.

[0134] Back to Figure 6 ,like Figure 6 As shown, step 640 is finally executed to input the feature representation of the task into the task prediction head of the recognition model to obtain the recognition result of the target to be recognized, and the recognition result includes the identity information of the target to be recognized.

[0135] Back to Figure 8 ,like Figure 8 As shown, the first feature representation output by the first task query module 532 is input into the first task prediction head 541 to obtain the first task prediction output 551, which can be specifically implemented as the three-dimensional human joint point information of the identified target; the second feature representation output by the second task query module 533 is input into the second task prediction head 542 to obtain the second task prediction output 552, which can be specifically implemented as the action type of the identified target; the third feature representation output by the third task query module 543 is input into the third task prediction head 543 to obtain the third task prediction output 553, which can be specifically implemented as the action type of the identified target.

[0136] According to one embodiment, the task prediction head can be implemented as a lightweight task-specific prediction head to reduce the computational overhead of the task; specifically, it includes one or more linear layers for predicting identity categories, behavior categories, and human posture.

[0137] According to one embodiment, the recognition model of the present invention was trained on a workstation equipped with an NVIDIA GeForce RTX4090 GPU using the PyTorch deep learning framework. The model employed the Adam optimizer with a batch size of 40 and an initial learning rate of 2e-3, which was decayed by a factor of 0.5 every 10 epochs using a step-size learning rate scheduler. The input consisted of 12 consecutive frames of RF echo data (i.e., a 1-second time window).

[0138] To evaluate the recognition model's performance for single targets in wall-occluded scenarios, we collected 231,000 pairs of RF echo signal frames, optical images, and corresponding person identity labels, action category labels, and 3D human joint coordinates. Data collection took place in a room with 23 cm thick concrete walls. The trained recognition model performed as follows on the test set:

[0139] In the person re-identification (ReID) task, the recognition model achieved a mean average precision (mAP) of 0.9899 and a Rank-1 matching rate of 0.9941. Its multi-scale attention mechanism, by fusing global behavioral features (such as gait style) with local skeletal features (such as limb length ratio), enabled accurate identification of individuals across scenarios, demonstrating the recognition model's ability to uniformly learn multi-granular human representations.

[0140] For action recognition tasks, the recognition model achieved an accuracy of 0.9985 and an F1 score of 0.9974, demonstrating its ability to accurately capture the temporal evolution of actions. By processing 12 frames of continuous RF signal (covering a 1-second time window), the recognition model effectively extracted long-term motion patterns and successfully distinguished 19 categories of behaviors, including static postures and dynamic movements. Even for static action categories with a relatively small sample size, the model demonstrated stable classification performance.

[0141] In the pose estimation task, the recognition model demonstrated excellent 3D human joint reconstruction capabilities, with an average joint position error of 49.92 mm. The model maintained high accuracy even in dynamic occlusion scenarios (such as random walking), with evenly distributed errors across joints. Core joints such as the neck (41.26 mm) and hip (41.49 mm) had the lowest errors, while end joints such as the wrist (70.21 mm) had manageable errors, demonstrating its robust modeling capabilities for complex human motion.

[0142] Figure 14a 、 Figure 14b 、 Figure 14c 、 Figure 14d 、 Figure 14e and Figure 14f Schematic diagrams respectively show a first recognition situation, a second recognition situation, a third recognition situation, a fourth recognition situation, a fifth recognition situation and a sixth recognition situation of a recognition model according to an embodiment of the present invention.

[0143] like Figure 14a As shown, in the first recognition case, according to the optical image, the first recognition target (ID: 1) photographed by the camera node is in a walking state; according to the radio frequency echo data, the recognition model outputs the identity information as ID: 1, the action type is walking, and also outputs the three-dimensional human joint information; based on the three-dimensional human joint information, a three-dimensional model of the recognition target in the three-dimensional coordinate system is drawn, which is the same as the shape of the recognition target in the optical image.

[0144] like Figure 14b As shown, in the second recognition situation, according to the optical image, the second recognition target (ID: 2) photographed by the camera node is from standing to sitting down; according to the radio frequency echo data, the recognition model outputs the identity information as the recognition target ID: 2, the action type is sitting down, and also outputs the three-dimensional human joint point information; based on the three-dimensional human joint point information, a three-dimensional model of the recognition target in the three-dimensional coordinate system is drawn, which is the same as the shape of the recognition target in the optical image.

[0145] like Figure 14c As shown, in the third recognition situation, according to the optical image, the third recognition target (ID: 3) photographed by the camera node is in the state of open arms, and the action type label is posture 18; according to the radio frequency echo data, the recognition model outputs the identity information as the recognition target ID: 3, the action type is posture 18, and also outputs the three-dimensional human joint point information; based on the three-dimensional human joint point information, a three-dimensional model of the recognition target in the three-dimensional coordinate system is drawn, which is the same as the shape of the recognition target in the optical image.

[0146] like Figure 14d As shown, in the fourth recognition situation, according to the optical image, the first recognition target (ID: 1) photographed by the camera node is in a walking state; according to the radio frequency echo data, the recognition model outputs identity information as recognition target ID: 1, the action type is walking, and three-dimensional human joint point information is also output; based on the three-dimensional human joint point information, a three-dimensional model of the recognition target in the three-dimensional coordinate system is drawn, which is the same as the shape of the recognition target in the optical image, and the position in the three-dimensional coordinate system also changes as the recognition target moves in the real scene.

[0147] like Figure 14e As shown, in the fifth recognition situation, according to the optical image, the first recognition target (ID: 1) photographed by the camera node is from a sitting to a standing state; according to the radio frequency echo data, the recognition model outputs the identity information as the recognition target ID: 1, the action type is standing, and the three-dimensional human joint point information is also output; based on the three-dimensional human joint point information, a three-dimensional model of the recognition target in the three-dimensional coordinate system is drawn, which is the same as the shape of the recognition target in the optical image.

[0148] like Figure 14f As shown, in the sixth recognition situation, according to the optical image, the second recognition target (ID: 2) photographed by the camera node is in a walking state; according to the radio frequency echo data, the recognition model outputs the identity information as the recognition target ID: 2, the action type is walking, and also outputs the three-dimensional human joint point information; based on the three-dimensional human joint point information, a three-dimensional model of the recognition target in the three-dimensional coordinate system is drawn, which is the same as the shape of the recognition target in the optical image.

[0149] The first recognition case, the second recognition case, the third recognition case, the fourth recognition case, the fifth recognition case and the sixth recognition case are merely exemplary single-person recognition cases, which verify the recognition effect of the recognition model for single-person unobstructed scenes.

[0150] The generalization capability of the proposed recognition model and method was evaluated in an unobstructed environment. The scene data contained approximately 120,000 pairs of RF echo signal frames and their corresponding optical images, person identity labels, action category labels, and 3D human joints. All data was divided into a training set and a test set in a 4:1 ratio. The recognition model's performance on the test set is as follows:

[0151] In this scenario, the recognition model achieved an average precision of 1 in the pedestrian re-identification (ReID) task and an accuracy of 1 in the action recognition task, with an average joint position error of 60.12 mm. This result proves that the recognition model can still maintain excellent performance in an environment without concrete wall obstruction.

[0152] according to Figure 14a-14f Results show that the 3D human joint information output by the recognition model can accurately reconstruct human poses and correctly identify action categories and person identities. In particular, in dynamic walking scenes, the recognition model consistently outputs accurate pose estimates and identity information even when the person's position and joint posture continuously change. This fully demonstrates the robustness of the recognition model in complex dynamic scenes.

[0153] To further evaluate the model's performance in multi-person scenarios, we collected data for two-person scenarios. This data contains approximately 32,000 pairs of RF signal frames and their corresponding optical images and labels. Furthermore, we expanded the output dimension of the linear layer in the task prediction head of the recognition model, extending it from single-person prediction to dual-person prediction.

[0154] In the two-person scenario, the average precision of the recognition model in the pedestrian re-identification (ReID) task and the accuracy in the action recognition task are both 1, indicating that the recognition model can effectively distinguish different actions and individuals.

[0155] Figure 15a and Figure 15bSchematic diagrams respectively show the seventh recognition situation and the eighth recognition situation of the recognition model according to an embodiment of the present invention.

[0156] like Figure 15a As shown, in the seventh recognition scenario, two recognition targets are set simultaneously. As shown in the optical image, the first recognition target (ID: 1) captured by the camera node is walking; the second recognition target (ID: 2) is standing still. Based on the RF echo data, the recognition model outputs identity information for the recognition targets ID: 1 and ID: 2, with the action types walking and standing still, respectively. It also outputs 3D human joint information. Based on this 3D human joint information, a 3D model of the recognition target is drawn in a 3D coordinate system, which is identical to the morphology of the recognition target in the optical image.

[0157] like Figure 15b As shown, in the eighth recognition scenario, according to the optical image, the first identified target (ID: 1) captured by the camera node is walking; the second identified target (ID: 2) is stepping in place. Based on the RF echo data, the recognition model outputs identity information for the identified targets ID: 1 and ID: 2, with the action types walking and stepping in place, respectively. It also outputs 3D human joint information. Based on this 3D human joint information, a 3D model of the identified targets in a 3D coordinate system is drawn, which is identical to the morphology of the identified targets in the optical image. The seventh and eighth recognition scenarios show that the first identified target has a higher degree of motion complexity. The average joint position error of the model is calculated to be 92.42 mm. Specifically, the average joint position error for the first identified target is 95.22 mm, while the average joint error for the second identified target is 89.63 mm. Due to the higher motion complexity of the first identified target, its joint reconstruction error is also relatively large.

[0158] To further evaluate the generalization capabilities of the recognition model, the evaluation was extended to more complex scenarios involving three people. To this end, a comprehensive dataset consisting of 108,000 pairs of RF echo signal frames was constructed, collected from various time periods, environmental conditions, and occlusion settings (e.g., free space, wooden board occlusion, and concrete wall occlusion). The dataset also annotated the corresponding optical images, person identity labels, action categories, and human key points. To ensure robustness of the evaluation, the dataset was split into training and test sets in a 4:1 ratio. In these scenarios, the recognition targets simultaneously performed three dynamic activities: free walking, waving, and transitioning between standing and sitting. This experimental setup introduced significant challenges, enabling a more rigorous evaluation of the model's ability to track and analyze multiple targets with diverse and complex motion patterns.

[0159] In an evaluation using three people, the recognition model performed exceptionally well in both action recognition and person re-identification (ReID) tasks, achieving an accuracy and average precision of 1, demonstrating its robustness in complex dynamic scenes. In terms of pose reconstruction, the model performed differently under different occlusion conditions.

[0160] Figure 16a 、 Figure 16b and Figure 16c Schematic diagrams respectively show the ninth recognition case, the tenth recognition case and the eleventh recognition case of the recognition model according to an embodiment of the present invention.

[0161] like Figure 16a As shown in the figure, the ninth recognition scenario involves three people in an unobstructed scene. The recognition model outputs the identity information and action type of each identified target, including: ID: 1, walking; ID: 2, sitting; ID: 3, waving. In the unobstructed scene, the average pose reconstruction error is 95.64 mm.

[0162] like Figure 16b The tenth recognition scenario is shown in Figure 1, which illustrates three-person recognition in a template-occluded scenario. The recognition model outputs the identity and action type of each identified target, including: ID: 1, walking; ID: 2, walking; and ID: 3, sitting. In the wooden board-occluded scenario, the average pose reconstruction error is 109.14 mm.

[0163] like Figure 16c As shown in the figure, the eleventh recognition scenario involves three people in a concrete occlusion scenario. The recognition model outputs the identity information and action type of each identified target, including: ID: 1, walking; ID: 2, waving; ID: 3, sitting. In the concrete occlusion scenario, the average pose reconstruction error is 86.36 mm.

[0164] Figure 16a 、 Figure 16b and Figure 16c The robustness of the recognition model in dealing with different types of occlusions is demonstrated, and it can be seen that the recognition model can accurately capture identity, behavior, and posture information.

[0165] Those skilled in the art will understand that the features described in the various embodiments and / or claims of the present invention may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments and / or claims of the present invention may be combined and / or coupled in various ways. All of these combinations and / or couplings fall within the scope of the present invention. The present invention also provides an electronic device for executing the radio frequency signal-based identity recognition method of the present application, comprising one or more processors, a storage device for storing one or more programs, wherein, when the one or more programs are executed by one or more processors, the one or more processors execute the radio frequency signal-based identity recognition method of the present application.

[0166] The embodiments of the present invention have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. The scope of the present invention is defined by the appended claims and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for identifying an individual based on radio frequency signals, characterized in that: include: Inputting to-be-processed radio frequency echo data into a learning-driven embedding module of a recognition model to obtain a coded feature vector, wherein the to-be-processed radio frequency echo data includes a plurality of radio frequency echo data spliced ​​according to acquisition time, the radio frequency echo data being obtained based on a radio frequency echo signal, and the radio frequency echo signal being obtained based on a radio frequency signal used to detect a target to be identified; Inputting the encoded feature vector into a scale hybrid encoder of the recognition model to obtain a multi-granularity encoded feature; Inputting the multi-granularity encoded features into a task-specific decoder of the recognition model to obtain a feature representation of the task; Inputting the feature representation of the task into the task prediction head of the recognition model to obtain a recognition result of the target to be recognized, wherein the recognition result includes identity information of the target to be recognized; The scale hybrid encoder includes a cascaded first scale hybrid attention module, a second scale hybrid attention module, and a third scale hybrid attention module; the scale hybrid encoder inputting the encoded feature vector into the recognition model to obtain multi-granularity encoded features includes: Inputting the encoded feature vector into a first-scale hybrid attention module to generate a first multi-granularity encoded feature; Inputting the first multi-granularity encoding feature into the second-scale mixed attention module to generate a second multi-granularity encoding feature; Inputting the second multi-granularity encoding feature into the third-scale mixed attention module to generate a third multi-granularity encoding feature; The first-scale hybrid attention module includes a first-scale hybrid attention layer, and inputting the encoded feature vector into the first-scale hybrid attention module to generate a first multi-granularity encoded feature includes: Dividing the encoded feature vector into a corresponding number of groups according to the number of three-dimensional complex convolution kernels, and performing convolution operations on the query matrix, key matrix, and value matrix of the feature vectors of different groups according to multiple three-dimensional complex convolution kernels of different sizes provided by the first-scale mixed attention layer to obtain query matrices, key matrices, and value matrices of multiple scales, wherein the zth three-dimensional complex convolution kernel is used to perform convolution operations on the query matrix, key matrix, and value matrix of the zth group feature vector; Multi-head self-attention calculation is performed based on the query matrix, key matrix and value matrix of multiple scales to obtain the attention output vector.

2. The method according to claim 1, wherein The radio frequency echo data includes the amplitude spectrum and phase spectrum of the frequency received by multiple virtual antenna elements. The radio frequency echo data to be processed is input into the learning-driven embedding module of the recognition model to obtain the encoded feature vector including: generating a feature vector according to the amplitude spectrum and the phase spectrum; Generate a spatial position code according to the virtual antenna element corresponding to the eigenvector and the frequency point corresponding to the frequency; generating a time position code according to the acquisition time of the radio frequency echo data; A coding feature vector is generated according to the spatial position code, the temporal position code and the feature vector.

3. The method according to claim 2, wherein The number of the virtual antenna elements is determined according to the number of transmitting antennas and the number of receiving antennas.

4. The method according to claim 1, wherein The first-scale hybrid attention module further includes a first feature interaction feedforward layer, the first feature interaction feedforward layer including feedforward layer convolution kernels of multiple sizes, and inputting the encoded feature vector into the first-scale hybrid attention module to generate the first multi-granularity encoded feature further includes: Perform a convolution operation on the first and second groups of attention output vectors after grouping according to the convolution kernel of the first feed-forward layer to obtain a fifth group of attention output vectors; Perform a convolution operation on the fifth group of attention output vectors and the grouped third group of attention output vectors according to the convolution kernel of the second feedforward layer to obtain a sixth group of attention output vectors; Perform a convolution operation on the sixth group of attention output vectors and the grouped fourth group of attention output vectors according to the convolution kernel of the third feedforward layer to obtain a seventh group of attention output vectors; The first, fifth, sixth and seventh groups of attention output vectors are concatenated to generate the first multi-granularity encoding feature.

5. The method according to any one of claims 1 to 3, wherein The task-specific decoder includes a task interaction module, and the step of inputting the multi-granularity encoded features into the task-specific decoder of the recognition model to obtain a feature representation of the task includes: According to the task interaction module, different weight parameters are added to the multi-granularity coding features and weighted fusion is performed to obtain the task interaction features.

6. The method according to claim 5, wherein The task-specific decoder further includes a first task query module, a second task query module, and a third task query module. Inputting the multi-granularity encoding features into the task-specific decoder of the recognition model to obtain a feature representation of the task further includes: The first multi-granularity encoding feature is used as a query matrix, and the task interaction feature is used as a key matrix and a value matrix, and inputted into the first task query module to perform cross attention calculation to obtain a first feature representation; The second multi-granularity encoding feature is used as a query matrix, and the task interaction feature is used as a key matrix and a value matrix, and inputted into the second task query module to perform cross attention calculation to obtain a second feature representation; The third multi-granularity encoding feature is used as a query matrix, and the task interaction feature is used as a key matrix and a value matrix, which are input into the third task query module for cross-attention calculation to obtain a third feature representation.

7. The method according to claim 6, wherein The recognition model includes a first task prediction head, a second task prediction head, and a third task prediction head. Inputting the feature representation of the task into the task prediction head of the recognition model to obtain the recognition result of the target to be recognized includes: Inputting the first feature representation into the first task prediction head to obtain three-dimensional human body joint point information of the target to be identified; Inputting the second feature representation into the second task prediction head to obtain the action type of the target to be identified; The third feature representation is input into the third task prediction head to obtain the identity information of the target to be identified. The feature granularity of the third feature representation and the second feature representation is coarser than that of the first feature representation.

8. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to perform the method according to any one of claims 1 to 7.

Images