Millimeter wave radar gesture recognition method and device based on Fourier-Transform network

Abstract

The invention relates to the technical field of gesture recognition, and provides a millimeter-wave radar gesture recognition method and device based on a Fourier-Transform network, and the method comprises the steps: constructing a Fourier network through employing a complex value full connection layer method, carrying out the linear transformation of the sample dimension and chirp dimension of radar original data through two complex value full connection layers, and carrying out the linear transformation of the sample dimension and chirp dimension of the radar original data, distance-Doppler features obtained through Fourier network processing are subjected to blocking and linear embedding, then Patch Merging is carried out, the distance-Doppler features are input into a Swin Transform block, attention calculation is carried out, and a gesture classification result is output through an MLP layer. According to the end-to-end gesture recognition method, the Fourier-Transform model is designed for the original data collected by the millimeter wave radar, the original radar data are processed through the Fourier-Transform model, the gesture classification result is obtained, and therefore the end-to-end gesture recognition method is achieved.

Description

Technical Field

[0001] Embodiments of the present invention generally relate to the technical field of gesture recognition, and more particularly to a millimeter-wave radar gesture recognition method and device based on a Fourier-Transformer network. Background Art

[0002] As an intuitive and expressive interpersonal communication method, gesture is also a highly potential means of human-computer interaction. It can replace physical buttons and touchscreens, providing users with a more flexible and convenient operation experience, and has broad application prospects in fields such as smart home, driving assistance, virtual reality games, and telemedicine. With the rapid development of computer technology, extensive research has been conducted on gesture recognition methods. Among them, methods based on visual images and sensors are particularly common. Gesture recognition methods based on visual images are affected by environmental conditions such as light and dust, and there are limitations in terms of privacy protection. Gesture recognition methods based on sensors, such as electromyogram (EMG) sensors, require users to perform complex operations to wear the device, and are limited by factors such as devices and materials, which to a certain extent affect the user's wearing experience. Although LiDAR can solve some problems, its high cost limits its widespread application.

[0003] In contrast, millimeter-wave radar has gradually emerged in the field of gesture recognition due to its characteristics of small size, low power consumption, easy embedding in devices, and stable operation under harsh conditions, attracting the attention of many researchers. There are currently two main problems in millimeter-wave radar gesture recognition. One is that complex operations are required in the data preprocessing part. For some mobile devices or low-power hardware, the computational resources consumed by using DFT (Discrete Fourier Transform) to process distance, Doppler, and angle information are relatively large. Moreover, compared with optical image data, gesture datasets in the form of range-Doppler maps, range-time maps, and micro-Doppler maps of current millimeter-wave radars are not common. Therefore, directly processing the raw data collected from the radar to extract effective gesture features is a direction worthy of exploration. The other is that the classifiers used in current millimeter-wave radar gesture recognition models are generally based on CNN, RNN, or their hybrid architectures (such as CNN-LSTM). These algorithms generate redundant features during calculation, resulting in low computational efficiency and also lacking in the recognition accuracy of complex gestures. Summary of the Invention

[0004] To solve the above problems, the present invention designs a Fourier-Transformer model for the raw data collected by millimeter-wave radar, processes the raw radar data through the Fourier-Transformer model, and obtains a gesture classification result, thereby realizing an end-to-end gesture recognition method.

[0005] According to an embodiment of the present invention, a method and device for millimeter-wave radar gesture recognition based on a Fourier-Transformer network are provided.

[0006] In a first aspect of the present invention, a method for millimeter-wave radar gesture recognition based on a Fourier-Transformer network is provided. The method includes:

[0007] Step S01: Construct a Fourier network by using the method of a complex-valued fully-connected layer, including: two complex-valued fully-connected layers and a swapping layer, where the first layer is a range fully-connected layer, the second layer is a swapping layer, and the third layer is a Doppler fully-connected layer;

[0008] Step S02: After the original millimeter-wave radar data is processed by the range fully-connected layer, it enters the swapping layer to swap the data of the sample and chirp dimensions of the millimeter-wave radar data, and after being processed by the Doppler fully-connected layer, multi-frame range-Doppler features are obtained;

[0009] Step S03: Perform block division and linear embedding on the obtained range-Doppler features, perform feature mapping and input them into a Swin Transformer. The Swin Transformer is divided into four stages and includes Patch Merging and Swin Transformer blocks;

[0010] Step S04: Downsample the range-Doppler features in Patch Merging, then input them into the Swin Transformer block and perform attention calculation, and the MLP module of the last Swin Transformer block outputs the gesture classification result.

[0011] Further, the complex-valued fully-connected layer described in step S01 is constructed using the Cplx library in Python.

[0012] Further, step S02 further includes initializing the weights of the complex-valued fully-connected layer. Regarding the complex-valued fully-connected layer as a matrix of size M×M, using prior knowledge, initialize its weights in the form of DFT, and the weights are:

[0013]

[0014] In the formula: ω is the weight, N is the number of sampling points, and a and b respectively represent the indexes of rows and columns;

[0015] The output after passing through the complex-valued fully-connected layer is expressed as:

[0016]

[0017] Where output is the output of the complex-valued fully connected layer, and input is the input of the complex-valued fully connected layer.

[0018] Furthermore, the specific steps of performing block segmentation and linear embedding on the obtained range-Doppler features in step S03 are as follows: the obtained range-Doppler features are segmented into blocks of 4*4 in size, each block is feature mapped by linear embedding and input into the Swin Transformer.

[0019] Furthermore, the Swin Transformer block described in step S03 is composed of a multi-layer perceptron MLP, a window self-attention mechanism W-MSA, a shifted window self-attention mechanism SW-MSA, a feedforward and a normalized LN. The features are divided into windows. Assuming that each window contains M×M blocks, the computational complexity of MSA and W-MSA for blocks with a height of h and a width of w is:

[0020] Ω(MSA)=4hwC 2 +2(hw) 2 C

[0021] Ω(W-MSA)=4hwC 2 +2M 2 HkDJ

[0022] Furthermore, the Swin Transformer uses a shift window partitioning method, and the following formula represents two consecutive blocks x k and x k+1 Output stream between:

[0023]

[0024] Where x k and x k+1 Represent the output features of the (S)W-MSA module and the MLP module respectively.

[0025] In a second aspect of the present invention, a device for millimeter-wave radar gesture recognition based on a Fourier-Transformer network is provided. The device comprises:

[0026] Fourier network construction module: used to construct a Fourier network using the complex-valued fully connected layer method, including: two complex-valued fully connected layers and one switching layer, where the first layer is a distance fully connected layer, the second layer is a switching layer, and the third layer is a Doppler fully connected layer;

[0027] Feature acquisition module: The raw millimeter-wave radar data is processed by the range fully connected layer and then enters the exchange layer. The sample and chirp dimensions of the millimeter-wave radar data are exchanged, and the range-Doppler features of multiple frames are obtained after being processed by the Doppler fully connected layer.

[0028] Linear Embedding Module: It is used to block and linearly embed the obtained range-Doppler features, perform feature mapping and input them into the Swin Transformer. The Swin Transformer is divided into four stages and includes Patch Merging and Swin Transformer blocks;

[0029] Gesture Classification Module: It is used to downsample the range-Doppler features in Patch Merging, then input them into the Swin Transformer block and perform attention calculation. The MLP module of the last Swin Transformer block outputs the gesture classification result.

[0030] In the third aspect of the present invention, an electronic device is provided. The electronic device includes: a memory and a processor. A computer program is stored on the memory, and when the processor executes the program, the method according to the first aspect of the present invention is implemented.

[0031] In the fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method according to the first aspect of the present invention is implemented.

[0032] The present invention designs a Fourier-Transformer model for the raw data collected by a millimeter-wave radar, processes the original radar data through the Fourier-Transformer model and obtains the gesture classification result, thereby realizing an end-to-end gesture recognition method.

[0033] It should be understood that the content described in the Summary of the Invention section is not intended to limit the key or important features of the embodiments of the present invention, nor to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description.

[0034] Advantages of the present invention:

[0035] A Fourier-Transformer network is proposed for the raw data collected by a millimeter-wave radar to process the millimeter-wave radar data. Through this network module, it is not necessary to preprocess the radar raw data into point clouds, nor to perform traditional DFT processing, nor to perform normalization before input. This network ensures accuracy while effectively reducing the recognition time. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In combination with the accompanying drawings and with reference to the following detailed description, the above and other features, advantages and aspects of the embodiments of the present invention will become more obvious. Among them:

[0037] Figure 1 The flowchart of the millimeter-wave radar gesture recognition method based on the Fourier-Transformer network according to an embodiment of the present invention is shown;

[0038] Figure 2 The network block diagram of the human gesture recognition by the Fourier-Transformer network according to an embodiment of the present invention is shown;

[0039] Figure 3 The structural diagram of the Fourier network according to an embodiment of the present invention is shown

[0040] Figure 4 The structural diagram of the constructed Swin Transformer module according to an embodiment of the present invention is shown;

[0041] Figure 5 The result comparison diagram with other methods according to an embodiment of the present invention is shown;

[0042] Figure 6 The block diagram of the device for millimeter-wave radar gesture recognition based on the Fourier-Transformer network according to an embodiment of the present invention is shown;

[0043] Figure 7 The schematic diagram of the device for millimeter-wave radar gesture recognition based on the Fourier-Transformer network according to an embodiment of the present invention is shown. Detailed implementation manners

[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the scope of protection of the present invention.

[0045] According to the embodiments of the present invention, a method and device for millimeter-wave radar gesture recognition based on the Fourier-Transformer network are proposed. For the original data collected by the millimeter-wave radar, a Fourier-Transformer model is designed, and the original radar data is processed through the Fourier-Transformer model to obtain the gesture classification result, thereby realizing an end-to-end gesture recognition method.

[0046] Next, with reference to several representative embodiments of the present invention, the principles and spirits of the present invention will be elaborated in detail.

[0047] Figure 1It is a schematic flowchart of a method for millimeter-wave radar gesture recognition based on a Fourier-Transformer network according to an embodiment of the present invention. The method includes:

[0048] Step S01: Construct a Fourier network using the method of complex-valued fully connected layers, including: two complex-valued fully connected layers and one exchange layer, where the first layer is the range fully connected layer, the second layer is the exchange layer, and the third layer is the Doppler fully connected layer;

[0049] Step S02: The original millimeter-wave radar data is processed by the range fully connected layer and then enters the exchange layer to exchange the data of the sample and chirp dimensions of the millimeter-wave radar data. After being processed by the Doppler fully connected layer, multi-frame range-Doppler features are obtained;

[0050] Step S03: The obtained range-Doppler features are segmented and linearly embedded, feature mapped and input into the Swin Transformer. The Swin Transformer is divided into four stages and includes Patch Merging and Swin Transformer blocks;

[0051] Step S04: Downsample the range-Doppler features in Patch Merging, then input them into the Swin Transformer block and perform attention calculation. The MLP module of the last Swin Transformer block outputs the gesture classification result.

[0052] It should be noted that although the operations of the method of the present invention are described in a specific order in the above embodiments and accompanying drawings, this does not require or imply that these operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution.

[0053] In order to more clearly explain the above method for millimeter-wave radar gesture recognition based on a Fourier-Transformer network, a specific embodiment will be described below. However, it should be noted that this embodiment is only for better explaining the present invention and does not constitute an improper limitation of the present invention.

[0054] The following uses a specific example to further illustrate the method for millimeter-wave radar gesture recognition based on a Fourier-Transformer network in more detail:

[0055] As Figure 2 shown, the method for human body gesture recognition based on a Fourier-Transformer network includes the following steps:

[0056] Step S01: Since the original radar signal is complex-valued, a Fourier network is constructed using the method of complex-valued fully connected layers. The complex-valued fully connected layers are constructed using the Cplx library in Python. Different from a single real-valued linear layer, each neuron in the complex-valued fully connected layer is a complex number, so it can be used to imitate the way radar signals use DFT to extract information, simplifying the preprocessing steps.

[0057] As Figure 3 shown, the Fourier network includes: two complex-valued fully connected layers and a swapping layer. The first layer is the range fully connected layer, the second layer is the swapping layer, and the third layer is the Doppler fully connected layer. Considering the deployment of this network on low-power hardware such as mobile devices and smart furniture, we use single-antenna radar data, which mainly includes data in two dimensions of sample and chirp, containing range and Doppler information respectively.

[0058] Step S02: The original data of the millimeter-wave radar enters the swapping layer after being processed by the range fully connected layer. The sample and chirp dimension data of the millimeter-wave radar are swapped, and after being processed by the Doppler fully connected layer, multi-frame range-Doppler features are obtained.

[0059] The weights of the complex-valued fully connected layers are initialized. Regarding the complex-valued fully connected layer as a matrix of size M×M, using prior knowledge, its weights are initialized in the form of DFT, and the weights are:

[0060]

[0061] where: ω is the weight, n is the number of sampling points, and a and b represent the indices of rows and columns respectively;

[0062] The output after passing through the complex-valued fully connected layer is expressed as:

[0063]

[0064] where output is the output of the complex-valued fully connected layer and input is the input of the complex-valued fully connected layer.

[0065] After the weight parameters of the Fourier network are backpropagated by the loss function and iteratively updated, they are no longer the standard DFT. Using complex-valued fully connected layers to imitate DFT can not only allow the classification network to utilize the main frequency components caused by object reflection, but also allow the network to make potentially better transformations.

[0066] Step S03: The obtained range-Doppler features are partitioned and linearly embedded. The range-Doppler features obtained after being processed by the Fourier network are segmented into blocks of size 4*4. Subsequently, each block is feature-mapped through linear embedding and input into the Swin Transformer. The Swin Transformer is divided into four stages, and the steps of the four stages are roughly the same, including Patch Merging and Swin Transformer blocks.

[0067] Step S04: Downsample the range-Doppler features in Patch Merging, and then input them into the Swin Transformer block for attention calculation.

[0068] Construct the Swin Transformer block. As Figure 4 shown, there are two consecutive Swin Transformer blocks. Among them, LN is layer normalization, W-MSA is window self-attention mechanism, SW-MSA is shifted window self-attention mechanism, and MLP is multi-layer perceptron. In order to reduce the computational complexity, the Swin Transformer uses a window self-attention mechanism W-MSA different from the standard Transformer. First, the features are divided into windows. Assuming that each window contains M×M blocks, the computational complexity of MSA and W-MSA for blocks with height h and width w is:

[0069] Ω(MSA) = 4hwC 2 +2(hw) 2 C

[0070] Ω(W-MSA) = 4hwC 2 +2M 2 hwC

[0071] To solve the problem that W-MSA lacks cross-window connections, the Swin Transformer uses the method of shifted window partitioning. The following formula represents the output stream between two consecutive blocks x k and x k+1 :

[0072]

[0073] In the formula, x k and x k+1 represent the output features of the (S)W-MSA module and the MLP module respectively.

[0074] The MLP module of the last Swin Transformer block outputs the gesture classification result.

[0075] Experimental process:

[0076] The experimental dataset uses the human gesture dataset collected by the Peijun Zhao team. Radar data is collected using relevant devices such as the Texas Instruments IWR6843ISK millimeter-wave radar, MMWAVEICBOOST, and DCA 1000EVM. The sensor is installed on a tripod at a height of approximately 110 cm. The raw radar data is continuously stored on the PC in a streaming manner.

[0077] This dataset collects data from eight users. The users perform 12 different gesture activities in front of the radar, with each action lasting 1 second. Each second contains 10 frames, 128 chirps, and 256 samples. 30 samples are collected for each person for each action. The ratio of the training set, validation set, and test set is 15:5:10. Specifically, the training set contains 1920 samples for model training and learning; the validation set and test set each contain 480 samples for model validation and final testing, respectively.

[0078] The experimental platform used in this paper is the Windows 11 operating system, with an Intel Core i5-13490F CPU, an NVIDIA GeForce RTX 4070 graphics card, 32 GB of running memory, the PyTorch deep learning framework version 2.2.1, and the CUDA version 11.8. The initial learning rate of the Fourier network is set to 0.001, and the initial learning rate of other networks is set to 0.0003. The model is trained using the Adam optimizer, with cross-entropy used as the loss function, and the Batchsize is set to 32.

[0079] Different from other datasets, the gesture activities in this dataset mainly reflect subtle finger movements rather than palm and arm movements. Therefore, the difficulty of using this dataset for gesture recognition is much higher than that of arm or palm activities.

[0080] To verify the effectiveness of the present invention, a confusion matrix and a comparative experiment are conducted here. The comparative experiment refers to a comparison with the original method in the dataset paper. The results of the confusion matrix are as Figure 5 shown. In the vast majority of gesture categories, the recognition accuracy of the present invention has reached over 97.5%, fully confirming the effectiveness of the network model proposed by the present invention.

[0081] The results of the comparative experiment are as Figure 5As shown in the figure. Compared with RT-3DCNN, DT-3DCNN, RDT-3DCNN, and RD-2DCNNLSTM, the Fourier-Transformer proposed in the present invention has achieved significant improvements in accuracy by 9.79%, 20.42%, 3.75%, and 1.88% respectively. Although the accuracy of RDAT-2DCNNLSTM is slightly higher than that of the method of the present invention, its excessively long recognition time limits its application on low-power hardware or mobile devices with limited computing resources. Moreover, this algorithm relies on MIMO (multiple-input multiple-output) hardware support for angle estimation, which may not be achievable in some single-antenna radar systems. Compared with RDT-3DCNN and RDT-2DCNNLSTM that also utilize range-Doppler information, the FAST network not only outperforms in terms of accuracy but also has a shorter recognition time, being 9.38 ms and 1.39 ms faster than RDT-3DCNN and RDT-2DCNNLSTM respectively, and is more suitable for single-antenna radars as well as smart homes and mobile devices with limited computing resources.

[0082] Source paper of the comparative method: CubeLearn: End-to-end Lear-ning for Human MotionRecognition from Raw mmWave Radar Signals.

[0083] Based on the same inventive concept, the present invention also proposes a device for millimeter-wave radar gesture recognition based on the Fourier-Transformer network. The implementation of this device can refer to the implementation of the above method, and the repeated parts will not be elaborated. As Figure 6 shown, the device 100 includes:

[0084] Fourier network construction module 101: used to construct a Fourier network by using the method of complex-valued fully connected layers, including: two complex-valued fully connected layers and a swapping layer, where the first layer is the range fully connected layer, the second layer is the swapping layer, and the third layer is the Doppler fully connected layer;

[0085] Feature acquisition module 102: used to process the raw data of the millimeter-wave radar through the range fully connected layer and then enter the swapping layer to swap the data of the sample and chirp dimensions of the millimeter-wave radar data, and obtain multi-frame range-Doppler features after being processed by the Doppler fully connected layer;

[0086] Linear embedding module 103: used to block and linearly embed the obtained range-Doppler features, perform feature mapping and input them into the Swin Transformer. The Swin Transformer is divided into four stages and includes PatchMerging and Swin Transformer blocks;

[0087] Gesture Classification Module 104: It is used to downsample the range-Doppler features in Patch Merging, then input them into the Swin Transformer block for attention calculation, and the MLP module of the last Swin Transformer block outputs the gesture classification result.

[0088] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working process of the described module can refer to the corresponding process in the foregoing method embodiments and will not be elaborated herein.

[0089] As Figure 7 shown, the device includes a central processing unit (CPU), which can execute various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or computer program instructions loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The CPU, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0090] Multiple components in the device are connected to the I / O interface, including: an input unit, such as a keyboard, a mouse, etc.; an output unit, such as various types of displays, speakers, etc.; a storage unit, such as a disk, an optical disc, etc.; and a communication unit, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit allows the device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0091] The processing unit executes the various methods and processes described above, such as method steps S01 to step S04. For example, in some embodiments, method steps S01 to step S04 can be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device via the ROM and / or the communication unit. When the computer program is loaded into the RAM and executed by the CPU, one or more of the method steps S01 to step S04 described above can be executed. Alternatively, in other embodiments, the CPU can be configured to execute method steps S01 to step S04 in any other appropriate manner (for example, by means of firmware).

[0092] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chip (SOCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0093] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, executed partially on the machine and partially on a remote machine as an independent software package, or executed entirely on a remote machine or server.

[0094] In the context of the present invention, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include electrical connections based on one or more wires, portable computer disks, hard disks, Random Access Memory (RAM), Read Only Memory (ROM), Erasable Programmable Read Only Memory (EPROM or Flash Memory), optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0095] Furthermore, although the operations are depicted in a particular order, this should be understood as requiring that the operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed to achieve the desired result. In certain circumstances, multitasking and parallel processing may be advantageous. Likewise, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present invention. Certain features described in the context of separate embodiments can also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation can also be implemented separately or in any suitable sub-combination in multiple implementations.

[0096] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A method for millimeter-wave radar gesture recognition based on a Fourier-Transformer network, characterized in that The method includes: Step S01: Construct a Fourier network using the method of complex-valued fully connected layers, including: two complex-valued fully connected layers and a swapping layer, where the first layer is the range fully connected layer, the second layer is the swapping layer, and the third layer is the Doppler fully connected layer; Step S02: The original data of the millimeter-wave radar enters the swapping layer after being processed by the range fully connected layer, swapping the sample and chirp dimension data of the millimeter-wave radar data, and obtaining multi-frame range-Doppler features after being processed by the Doppler fully connected layer; Step S03: Perform block division and linear embedding on the obtained range-Doppler features, perform feature mapping and input them into the Swin Transformer. The Swin Transformer is divided into four stages and includes Patch Merging and Swin Transformer blocks; Step S04: Downsample the range-Doppler features in Patch Merging, then input them into the Swin Transformer block and perform attention calculation. The MLP module of the last Swin Transformer block outputs the gesture classification result.

2. The method for millimeter-wave radar gesture recognition based on the Fourier-Transformer network according to claim 1, wherein The complex-valued fully connected layer described in Step S01 is constructed using the Cplx library in Python.

3. The method for millimeter-wave radar gesture recognition based on the Fourier-Transformer network according to claim 1, wherein Step S02 also includes initializing the weights of the complex-valued fully connected layer. Regarding the complex-valued fully connected layer as a matrix of size M×M, using prior knowledge, initialize its weights in the form of DFT, and the weights are: In the formula: ω is the weight, N is the number of sampling points, and a and b represent the row and column indices respectively; The output after passing through the complex-valued fully connected layer is expressed as: In the formula, output is the output of the complex-valued fully connected layer, and input is the input of the complex-valued fully connected layer.

4. The method for millimeter-wave radar gesture recognition based on the Fourier-Transformer network according to claim 1, wherein The specific steps of performing block division and linear embedding on the obtained range-Doppler features described in Step S03 are: dividing the obtained range-Doppler features into blocks of size 4*4, and each block performs feature mapping through linear embedding and is input into the Swin Transformer.

5. The method for millimeter-wave radar gesture recognition based on the Fourier-Transformer network according to claim 1, wherein The Swin Transformer block described in Step S03 is composed of a multi-layer perceptron MLP, a window self-attention mechanism W-MSA, a shifted window self-attention mechanism SW-MSA, a feed-forward and normalization LN. Divide the features into windows. Assume that each window contains M×M blocks. The computational complexity of MSA and W-MSA for blocks with height h and width w is: Ω(MSA)=4hwC 2 +2(hw) 2 C Ω(W-MSA) = 4hwC 2 + 2M 2 hwC 6. The method for millimeter-wave radar gesture recognition based on the Fourier-Transformer network according to claim 5, wherein The Swin Transformer uses a shifted window partitioning method, and the following formula represents the output stream between two consecutive blocks x k and x k+1 : where x k and x k+1 represent the output features of the (S)W-MSA module and the MLP module, respectively.

7. An apparatus for millimeter-wave radar gesture recognition based on a Fourier-Transformer network, characterized in that The device implements the method described in any one of claims 1 to 6, including: A Fourier network construction module: used to construct a Fourier network using the method of complex-valued fully connected layers, including: two complex-valued fully connected layers and a swapping layer, where the first layer is the range fully connected layer, the second layer is the swapping layer, and the third layer is the Doppler fully connected layer; A feature acquisition module: used for the original data of the millimeter-wave radar to enter the swapping layer after being processed by the range fully connected layer, swapping the sample and chirp dimension data of the millimeter-wave radar data, and obtaining multi-frame range-Doppler features after being processed by the Doppler fully connected layer; Linear Embedding Module: It is used to block and linearly embed the obtained range-Doppler features, perform feature mapping and input them into the Swin Transformer. The Swin Transformer is divided into four stages and includes Patch Merging and Swin Transformer blocks; Gesture Classification Module: It is used to downsample the range-Doppler features in Patch Merging, then input them into the Swin Transformer block for attention calculation, and the MLP module of the last Swin Transformer block outputs the gesture classification result.

8. An electronic device, comprising a memory and a processor, wherein a computer program is stored on the memory, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method described in any one of claims 1 to 6.

Images