Radar human body action recognition method based on lightweight model

Abstract

The invention discloses a radar human body action recognition method based on a lightweight model, and the method comprises the steps: 1, constructing an FMCW radar model, and collecting and obtaining the radar echo data of a human body under different postures; 2, constructing a micro-Doppler graph data set, and dividing the micro-Doppler graph data set into a training set, a verification set and a test set according to a set proportion; step 3, constructing an improved Faster Net network model, and inputting the preprocessed training set and verification set into the improved Faster Net network model for action recognition; 4, training the improved Faster Net network model: using an AdamW optimizer, using a cross entropy loss function as a loss function, and continuously updating model parameters to obtain an optimal model; and 5, processing radar sampling data to be detected to obtain micro-Doppler feature data to be detected, and inputting the optimal model to obtain a classification result of human body action recognition. The method has remarkable advantages in the aspect of action recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of human motion recognition, and in particular to a radar human motion recognition method based on a lightweight model. Background Art

[0002] With the acceleration of the aging process of the population, the issue of monitoring the elderly is gradually becoming a hot social issue. In daily life, falls are one of the main causes of injuries to the elderly, so it is necessary to monitor the daily lives of the elderly. In recent years, fall detection for the elderly has become an active area of ​​research and development in elderly monitoring.

[0003] At present, the commonly used fall detection devices can be divided into two categories: wearable and non-wearable. Wearable detection devices include pressure insoles, bracelets, buttons and smartphones. Non-wearable detection devices include visual sensors, sound, pressure and vibration sensors.

[0004] However, wearable devices can easily cause discomfort to users, affect their normal lives, and have insufficient recognition accuracy. Visual sensors can expose people to cameras, which can easily leak privacy. Sound, pressure, and vibration sensors are greatly affected by the environment and have weak anti-interference capabilities. Summary of the invention

[0005] The purpose of the present invention is to provide a radar human motion recognition method based on a lightweight model, which has significant advantages in motion recognition and can better meet the needs of practical applications.

[0006] The technical solution adopted by the present invention is:

[0007] A radar human action recognition method based on a lightweight model, firstly, an IWR1843Boost radar is used to collect radar echo data of human body in five states: walking, sitting, standing, bending, and falling, with 300 groups of radar echoes for each type of action, for a total of 1500 groups of radar echoes. Secondly, the collected radar data is freed from environmental noise and interference from other objects in the room, and a distance-time map (RTM) is constructed using fast Fourier transform. The spectrum map is short-time Fourier transformed along the distance dimension to form a micro-Doppler map. Finally, the collected micro-Doppler map is used to construct its data set using data enhancement, in which the training set accounts for 60%, the validation set accounts for 20%, and the test set accounts for 20%. The training set is used for model training and provides input features. During the training process, the model performance indicators obtained from the validation set are used to select the best model hyperparameters. Finally, the human action time-frequency map to be tested is sent to the trained model as a test set for classification and recognition to identify the specific action type of the radar echo signal of the human action to be tested. The method specifically includes the following steps:

[0008] Step 1: construct an FMCW radar model, and acquire radar echo data of a human body in different postures based on the FMCW radar model;

[0009] Furthermore, the specific steps of constructing the FMCW radar model in step 1 are:

[0010] Step 1-1, the radar transmission signal is expressed as:

[0011]

[0012] In the formula, A T represents the transmission power, B represents the Chirp bandwidth, f c Indicates the starting frequency of Chirp, T c Indicates the duration of Chirp. represents phase noise;

[0013] Step 1-2, the mathematical expression of the echo information received by the radar after delay and scaling is:

[0014]

[0015] Where, t d =2R / c is used to represent the round-trip time between the signal and the target at a distance R from the radar, and α is the attenuation of the amplitude during the signal propagation process;

[0016] Step 1-3, the radar transmit signal x T (t) and receive x R (t) is mixed, and after I / Q demodulation, the mathematical expression that approximates the beat signal is obtained:

[0017]

[0018] In the formula, f b =2BR / (cT c ) represents the beat frequency, A R represents the received power, Indicates phase;

[0019] Furthermore, in experiments of detecting human motion at close range, the residual phase noise can be ignored. In addition, due to the actual situation πBt 2 d / T c is very small and can be ignored, so the phase φ b (t) = 4πR / λ.

[0020] Steps 1-4 describe the resulting beat signal in the form of a radar data matrix:

[0021]

[0022] Where n represents the fast time sampling axis, m represents the slow time sampling axis, T f Represents the ADC sampling time interval on the fast time sampling axis, T S Indicates the sampling time interval on the slow time sampling axis.

[0023] Furthermore, in step 1, radar echo data of the human body in different postures are collected, and several groups of radar echo data are collected for each posture.

[0024] Specifically, the IWR1843Boost radar is used to collect radar echo data in five states: walking, sitting, standing, bending over, and falling. There are 300 groups of radar echoes for each type of action, for a total of 1,500 groups of radar echoes.

[0025] Step 2: construct a micro-Doppler map dataset using radar echo data, and divide the micro-Doppler map dataset into a training set, a validation set, and a test set according to a set ratio;

[0026] Specifically, step 2 includes the following steps:

[0027] Step 2-1: After removing noise and interference from the radar echo data, use fast Fourier transform to construct a range-time map (RTM), and perform short-time Fourier transform on the spectrum along the distance dimension to form a micro-Doppler map.

[0028] Step 2-2, using data augmentation to construct a corresponding micro-Doppler image dataset for the collected micro-Doppler images, and dividing the micro-Doppler image dataset into a training set, a validation set, and a test set according to a set ratio;

[0029] Furthermore, in step 2, the ratio is set to 6:2:2.

[0030] Specifically, the collected radar data is freed from environmental noise and interference from other objects in the room, and a range-time map (RTM) is constructed using fast Fourier transform. The spectrum is short-time Fourier transformed along the distance dimension to form a micro-Doppler map. The collected micro-Doppler maps are used to construct their dataset using data augmentation, with the training set accounting for 60%, the validation set accounting for 20%, and the test set accounting for 20%. The training set is used for model training and provides input features. During the training process, the model performance indicators obtained from the validation set are used to select the best model hyperparameters. Finally, the test set is used to evaluate the model performance and obtain the action recognition results.

[0031] For the data matrix collected by the radar, add a Hanning window to all column data in its fast time dimension, perform Range-FFT to obtain the distance-time diagram, then use STFT to add a window to the distance dimension in the distance-time two-dimensional matrix, and finally superimpose them to obtain the micro-Doppler-time characteristic information of human body movements, that is, the micro-Doppler-time diagram. Short-time Fourier transform (STFT) is a linear time-frequency method for detecting time-varying signals. Divide a given radar signal into M segments of length L:

[0032]

[0033] According to the requirements, the window function is set to 128 to obtain the micro-Doppler images of different human body movements.

[0034] Step 3, construct an improved FasterNet network model, and input the preprocessed training set and the validation set into the constructed improved FasterNet network model for action recognition; the improved FasterNet network model includes an embedding layer, a multi-scale feature fusion module, and at least four FasterNet Bolcks; a merging layer is set between two adjacent FasterNet Bolcks; the output of the last FasterNet Bolck is sequentially converted and classified through a global average pooling layer, a 1×1 convolution layer, and a fully connected layer;

[0035] Among them, the multi-scale feature fusion module divides the input features into three parts for parallel processing. One part performs global average pooling on each channel of the feature map through the global average pooling layer to compress the channel information and extract global features; the other two parts perform 1×1 convolution and 3×3 convolution operations respectively, and use depth-separable convolution instead of traditional convolution to reduce the number of network parameters; the extracted features of the three parts are then processed by a PWConv layer, normalized by a BN layer, and then spliced ​​and output;

[0036] FasterNet Bolck includes one PConv layer and two PWConv layers in sequence, forming an inverted residual structure; the input of FasterNet Bolck is divided into two paths, one path is processed by one PConv layer and two PWConv layers in sequence to obtain the output features and concatenate them with the other path as the output of FasterNet Bolck; the merging layer is used to expand the number of channels;

[0037] Specifically, each FasterNet Bolck contains an embedding layer or a merging layer for spatial downsampling and channel number expansion. A multi-scale feature fusion module is designed after the embedding layer, using multi-scale convolution kernels to achieve perception of different scales and enrich the feature information of the feature map. This module divides the input features into three parts for parallel processing. First, global average pooling is performed on each channel of the feature map to compress the channel information and extract global features. The subsequent two parts perform 1×1 convolution and 3×3 convolution operations, and use depth-separable convolution instead of traditional convolution to reduce the number of network parameters. FasterNet Bolck consists of a PConv layer and two PWConv layers, which form an inverted residual structure. Finally, a global average pooling layer, a 1×1 convolution layer, and a fully connected layer are used for feature conversion and classification.

[0038] Furthermore, a SimAM module is set between the two PWConv layers of FasterNet Bolck. The SimAM module learns the correlation of each position of the extracted features from the spatial and channel dimensions, so that the network can remain lightweight and focus more on extracting important feature information of the micro-Doppler image, thereby improving the network recognition accuracy.

[0039] Furthermore, the SimAM module performs the following steps:

[0040] S1, define the energy function for each neuron, the energy function expression is as follows:

[0041]

[0042] In the formula, and are t and x respectively i The linear transformation of w t and b t are weight and bias respectively; M is the number of neurons on the corresponding channel;

[0043] S2, introduce the regularization term to get the final energy function expression:

[0044]

[0045] In the formula, λ is the weight value, x i is the input feature, w t and b t Respectively represent:

[0046]

[0047] In the formula, are the mean and variance of all neurons in this channel except t.

[0048] S3, calculate the minimum energy, the calculation formula is as follows:

[0049]

[0050] It can be seen from equation 3-4 that the lower the energy, the greater the difference between neuron t and the surrounding neurons, and the more attention is paid to neuron t in the model.

[0051] S4, enhance the input features to obtain enhanced features. The enhancement calculation formula is as follows:

[0052]

[0053] Among them, X is the input feature, is the enhanced output feature, E is e t * The sum over all channels and spatial dimensions, ⊙ is the dot product operation.

[0054] Step 4: Train the improved FasterNet network model: Use the AdamW optimizer, adopt the cross-entropy loss function, and continuously update the model parameters to obtain the optimal model.

[0055] Furthermore, the batch size of each training in step 4 is 16, the training is performed for 100 epochs, and the learning rate is 0.001.

[0056] Furthermore, the weight decay of the AdamW optimizer in step 4 is set to 0.0005.

[0057] Step 5: Process the radar sampling data to be tested to obtain the micro-Doppler feature data to be tested, and input the micro-Doppler feature data to be tested into the optimal model to obtain the classification result of human motion recognition.

[0058] The present invention adopts the above technical solution and has the following technical features: (1) The camera is easily affected by light conditions and there is a risk of personal privacy leakage, which is not suitable for some home places such as bedrooms and living rooms. The present invention processes the radar sampling data to obtain micro-Doppler features and uses short-time Fourier transform to construct micro-Doppler maps of different actions. (2) Traditional action recognition methods ignore the size of the model, resulting in too much computational complexity in network operation. The present invention classifies the micro-Doppler maps of human body movements based on the lightweight convolutional neural network FasterNet, so that the network has lower parameters and computational complexity, which can better meet actual requirements. (3) The present invention is provided with a multi-scale feature fusion module, which extracts richer features through average pooling layers and different-scale convolutions, and combines the module with the lightweight convolutional neural network FasterNet model to prevent information loss. In order to avoid the decrease in lightweight classification accuracy, the attention mechanism is integrated into the FasterNet model to improve the network recognition accuracy.

[0059] According to the technical solution of the present invention, in the smart home, users can operate devices more conveniently through actions, and the system can also intelligently adjust the working status according to the user's actions. For example, in the elderly care, the system can analyze the elderly's physical condition in real time and issue an alarm in time when necessary. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments;

[0061] Figure 1 A schematic diagram of a flow chart of a radar human motion recognition method based on a lightweight model of the present invention;

[0062] Figure 2 This is a schematic diagram of distance-FFT before processing;

[0063] Figure 3 This is the distance-FFT diagram after MTI processing;

[0064] Figure 4 This is a schematic diagram of the generation process of micro-Doppler;

[0065] Figure 5 Schematic diagram of the comparison of micro-Doppler of different human body movements;

[0066] Figure 6 Schematic diagram of the Mixup data enhancement principle;

[0067] Figure 7 It is a schematic diagram of the PConv module structure;

[0068] Figure 8 It is a structural diagram of the improved FasterNet network model;

[0069] Fig. 9 This is a schematic diagram of the FasterNet Block structure;

[0070] Fig.10 Schematic diagram of the accuracy comparison curve of different models;

[0071] Fig.11 Schematic diagram of the confusion matrix for action recognition. Implementation

[0073] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0074] Cameras are easily affected by light conditions and there is a risk of personal privacy leakage, which makes them unsuitable for some home places such as bedrooms and living rooms. The present invention proposes a human motion recognition method based on millimeter wave radar to solve such problems. The key to this technology is to obtain micro-Doppler features from radar sampling data processing, and use short-time Fourier transform to construct micro-Doppler maps of different actions. Traditional motion recognition methods ignore the size of the model, resulting in too much computational complexity in network operation. The present invention classifies human motion micro-Doppler maps based on the lightweight convolutional neural network FasterNet, so that the network has lower parameters and computational complexity, which can better meet actual requirements. The present invention designs a multi-scale feature fusion module, extracts richer features through average pooling layers and convolutions of different scales, and combines the module with the lightweight convolutional neural network FasterNet model to prevent information loss. In order to avoid the decrease in lightweight classification accuracy, the attention mechanism is integrated into the FasterNet model to improve the network recognition accuracy.

[0075] like Figures 1 to 11 As shown, the present invention discloses a radar human motion recognition method based on a lightweight model, which comprises the following steps:

[0076] Step 1: construct an FMCW radar model, and acquire radar echo data of a human body in different postures based on the FMCW radar model;

[0077] Furthermore, the specific steps of constructing the FMCW radar model in step 1 are:

[0078] Step 1-1, the radar transmission signal is expressed as:

[0079]

[0080] In the formula, A T represents the transmission power, B represents the Chirp bandwidth, f c Indicates the starting frequency of Chirp, T cIndicates the duration of Chirp. represents phase noise;

[0081] Step 1-2, the mathematical expression of the echo information received by the radar after delay and scaling is:

[0082]

[0083] Where, t d =2R / c is used to represent the round-trip time between the signal and the target at a distance R from the radar, and α is the attenuation of the amplitude during the signal propagation process;

[0084] Step 1-3, the radar transmit signal x T (t) and receive x R (t) is mixed, and after I / Q demodulation, the mathematical expression that approximates the beat signal is obtained:

[0085]

[0086] In the formula, f b =2BR / (cT c ) represents the beat frequency, A R represents the received power, Indicates phase;

[0087] Furthermore, in experiments of detecting human motion at close range, the residual phase noise can be ignored. In addition, due to the actual situation πBt 2 d / T c is very small and can be ignored, so the phase φ b (t) = 4πR / λ.

[0088] Steps 1-4 describe the resulting beat signal in the form of a radar data matrix:

[0089]

[0090] Where n represents the fast time sampling axis, m represents the slow time sampling axis, T f Represents the ADC sampling time interval on the fast time sampling axis, T S Indicates the sampling time interval on the slow time sampling axis.

[0091] Furthermore, in step 1, radar echo data of the human body in different postures are collected, and several groups of radar echo data are collected for each posture.

[0092] Specifically, the IWR1843Boost radar is used to collect radar echo data in five states: walking, sitting, standing, bending over, and falling. There are 300 groups of radar echoes for each type of action, for a total of 1,500 groups of radar echoes.

[0093] Step 2: construct a micro-Doppler map dataset using radar echo data, and divide the micro-Doppler map dataset into a training set, a validation set, and a test set according to a set ratio;

[0094] Specifically, step 2 includes the following steps:

[0095] Step 2-1: After removing noise and interference from the radar echo data, use fast Fourier transform to construct a range-time map (RTM), and perform short-time Fourier transform on the spectrum along the distance dimension to form a micro-Doppler map.

[0096] Step 2-2, using data augmentation to construct a corresponding micro-Doppler image dataset for the collected micro-Doppler images, and dividing the micro-Doppler image dataset into a training set, a validation set, and a test set according to a set ratio;

[0097] Furthermore, in step 2, the ratio is set to 6:2:2.

[0098] Specifically, the collected radar data is freed from environmental noise and interference from other objects in the room, and a range-time map (RTM) is constructed using fast Fourier transform. The spectrum is short-time Fourier transformed along the distance dimension to form a micro-Doppler map. The collected micro-Doppler maps are used to construct their dataset using data augmentation, with the training set accounting for 60%, the validation set accounting for 20%, and the test set accounting for 20%. The training set is used for model training and provides input features. During the training process, the model performance indicators obtained from the validation set are used to select the best model hyperparameters. Finally, the test set is used to evaluate the model performance and obtain the action recognition results.

[0099] For the data matrix collected by the radar, add a Hanning window to all column data in its fast time dimension, perform Range-FFT to obtain the distance-time diagram, then use STFT to add a window to the distance dimension in the distance-time two-dimensional matrix, and finally superimpose them to obtain the micro-Doppler-time characteristic information of human body movements, that is, the micro-Doppler-time diagram. Short-time Fourier transform (STFT) is a linear time-frequency method for detecting time-varying signals. Divide a given radar signal into M segments of length L:

[0100]

[0101] According to the requirements, the window function is set to 128 to obtain the micro-Doppler images of different human body movements.

[0102] Step 3, construct an improved FasterNet network model, and input the preprocessed training set and the validation set into the constructed improved FasterNet network model for action recognition; the improved FasterNet network model includes an embedding layer, a multi-scale feature fusion module, and at least four FasterNet Bolcks; a merging layer is set between two adjacent FasterNet Bolcks; the output of the last FasterNet Bolck is sequentially converted and classified through a global average pooling layer, a 1×1 convolution layer, and a fully connected layer;

[0103] Among them, the multi-scale feature fusion module divides the input features into three parts for parallel processing. One part performs global average pooling on each channel of the feature map through the global average pooling layer to compress the channel information and extract global features; the other two parts perform 1×1 convolution and 3×3 convolution operations respectively, and use depth-separable convolution instead of traditional convolution to reduce the number of network parameters; the extracted features of the three parts are then processed by a PWConv layer, normalized by a BN layer, and then spliced ​​and output;

[0104] FasterNet Bolck includes one PConv layer and two PWConv layers in sequence, forming an inverted residual structure; the input of FasterNet Bolck is divided into two paths, one path is processed by one PConv layer and two PWConv layers in sequence to obtain the output features and concatenate them with the other path as the output of FasterNet Bolck; the merging layer is used to expand the number of channels;

[0105] Specifically, each FasterNet Bolck contains an embedding layer or a merging layer for spatial downsampling and channel number expansion. A multi-scale feature fusion module is designed after the embedding layer, using multi-scale convolution kernels to achieve perception of different scales and enrich the feature information of the feature map. This module divides the input features into three parts for parallel processing. First, global average pooling is performed on each channel of the feature map to compress the channel information and extract global features. The subsequent two parts perform 1×1 convolution and 3×3 convolution operations, and use depth-separable convolution instead of traditional convolution to reduce the number of network parameters. FasterNet Bolck consists of a PConv layer and two PWConv layers, which form an inverted residual structure. Finally, a global average pooling layer, a 1×1 convolution layer, and a fully connected layer are used for feature conversion and classification.

[0106] Furthermore, a SimAM module is set between the two PWConv layers of FasterNet Bolck. The SimAM module learns the correlation of each position of the extracted features from the spatial and channel dimensions, so that the network can remain lightweight and focus more on extracting important feature information of the micro-Doppler image, thereby improving the network recognition accuracy.

[0107] Furthermore, the SimAM module performs the following steps:

[0108] S1, define the energy function for each neuron, the energy function expression is as follows:

[0109]

[0110] In the formula, and are t and x respectively i The linear transformation of w t and b t are weight and bias respectively; M is the number of neurons on the corresponding channel;

[0111] S2, introduce the regularization term to get the final energy function expression:

[0112]

[0113] In the formula, λ is the weight value, x i is the input feature, w t and b t Respectively represent:

[0114]

[0115] In the formula, are the mean and variance of all neurons in this channel except t.

[0116] S3, calculate the minimum energy, the calculation formula is as follows:

[0117]

[0118] It can be seen from equation 3-4 that the lower the energy, the greater the difference between neuron t and the surrounding neurons, and the more attention is paid to neuron t in the model.

[0119] S4, enhance the input features to obtain enhanced features. The enhancement calculation formula is as follows:

[0120]

[0121] Among them, X is the input feature, is the enhanced output feature, E is e t *The sum over all channels and spatial dimensions, ⊙ is the dot product operation.

[0122] Step 4: Train the improved FasterNet network model: Use the AdamW optimizer, adopt the cross-entropy loss function, and continuously update the model parameters to obtain the optimal model.

[0123] Furthermore, the batch size of each training in step 4 is 16, the training is performed for 100 epochs, and the learning rate is 0.001.

[0124] Furthermore, the weight decay of the AdamW optimizer in step 4 is set to 0.0005.

[0125] Step 5: Process the radar sampling data to be tested to obtain the micro-Doppler feature data to be tested, and input the micro-Doppler feature data to be tested into the optimal model to obtain the classification result of human motion recognition.

[0126] The specific principle of the present invention is described in detail below:

[0127] The architecture used by the radar human action recognition method consists of an FMCW radar model, a micro-Doppler map, and an improved FasterNet network. The transmission signal of the FMCW radar model radar can be expressed as:

[0128]

[0129] In the formula, A T represents the transmission power, B represents the Chirp bandwidth, f c Indicates the starting frequency of Chirp, T c Indicates the duration of Chirp. Represents phase noise. The mathematical expression of the echo information received by the radar after delay and scaling is:

[0130]

[0131] Where t d =2R / c is used to represent the round trip time between the signal and the target at a distance R from the radar, and α is the attenuation of the amplitude during the signal propagation. T (t) and receive x R (t) is mixed, and after I / Q demodulation, the mathematical expression that approximates the beat signal can be obtained:

[0132]

[0133] Where the beat frequency f b =2BR / (cT c), A R Indicates received power, phase In experiments to detect human motion at close range, residual phase noise can be ignored In addition, due to the actual situation πBt 2 d / T c is very small and can be ignored, so the phase φ b (t) = 4πR / λ. Finally, the obtained beat signal is described in the form of a radar data matrix:

[0134]

[0135] Where n represents the fast time sampling axis, m represents the slow time sampling axis, T f Represents the ADC sampling time interval on the fast time sampling axis, T S Indicates the sampling time interval on the slow time sampling axis.

[0136] Constructing micro-Doppler images: The basic principle of MTI technology is that the phase of a stationary target remains constant, while the phase of a moving target is different. By performing differential operations on the phases of two adjacent pulses in a continuous pulse, the phase signal of the moving target can be accurately extracted, and filtering processing for stationary targets can be achieved.

[0137] Since the acquisition environment of this embodiment is indoors, the clutter of objects such as walls, chairs, and tables is strong, which will cause certain interference to the target. The distance-FFT result of the measured data of the target at 1m away from the radar is as follows Figure 2 As shown, it can be seen that there are a lot of clutter nearby and the target cannot be distinguished.

[0138] After MTI processing on the target Figure 3 As shown in the figure, it can be seen that the clutter near the target is well suppressed, making the target stand out. In short, the zero-frequency or low-frequency components are suppressed after passing through the MTI high-pass filter, achieving the purpose of eliminating static clutter.

[0139] The human body echo data collected by the present invention uses TI's IWR1843Boost radar and the data acquisition evaluation board DCA1000EVM. The radar data of the human body in five states, namely walking, sitting, standing, bending, and falling, are measured. There are 300 groups of each type of action, and the measurement time of each group of action is 6s, for a total of 1500 groups of radar echoes. The radar parameters of this experiment are as follows Table 1:

[0140] Table 1 Radar parameter design

[0141]

[0142]

[0143] For the data matrix collected by the radar, a Hanning window is added to all column data in its fast time dimension, and Range-FFT is performed to obtain a range-time diagram. Then, the range dimension in the range-time two-dimensional matrix is ​​windowed using STFT, and finally superimposed to obtain the micro-Doppler-time characteristic information of human body movements, namely the micro-Doppler-time diagram. The schematic diagram of the generation process is shown in Figure 4 shown.

[0144] Short-time Fourier transform (STFT) is a linear time-frequency method for detecting time-varying signals. Given a radar signal, divide the signal into M segments of length L:

[0145]

[0146] According to the requirements, select the window function and set it to 128 to obtain the micro-Doppler images of different human body movements. Figure 5 Shown

[0147] Improved FasterNet model: In this embodiment, the image pixels are 224×224, and the training set accounts for 60%, the validation set accounts for 20%, and the test set accounts for 20%. In order to deal with the problem of small data sets, Mixup data enhancement technology is used to reduce the possibility of overfitting of deep learning models, as follows Figure 6 shown.

[0148] PConv is used in FasterNet to reduce computational redundancy and memory access. PConv modules are as follows: Figure 7 shown.

[0149] When performing spatial feature extraction, it only applies the conventional Conv operation to some of the input channels, and the rest of the channels remain unchanged. p The channel is calculated as a representative of the whole feature map, ensuring its generality while maintaining the same number of channels in the input and output feature maps. Therefore, the FLOPs of PConv is For a typical partial ratio r = c p / c=1 / 4, the computational complexity of PConv is only 1 / 16 of that of conventional Conv. In order to fully and effectively utilize the information from all channels, a point-wise convolution (PWConv) is added to PConv.

[0150] This network is divided into four Bolcks, each of which contains an embedding layer or a merging layer for spatial downsampling and channel number expansion. A multi-scale feature fusion module (MSFF) is designed after the embedding layer, using multi-scale convolution kernels to achieve perception of different scales and enrich the feature information of the feature map. This module divides the input features into three parts for parallel processing. First, global average pooling is performed on each channel of the feature map to compress the channel information and extract global features. The subsequent two parts perform 1×1 convolution and 3×3 convolution operations, and use depth-separable convolution instead of traditional convolution to reduce the number of network parameters. FasterNet Bolck consists of a PConv layer and two PWConv layers, which form an inverted residual structure. Finally, a global average pooling layer, a 1×1 convolution layer and a fully connected layer are used for feature conversion and classification. The improved FasterNet model structure is as follows Figure 8 shown.

[0151] The SimAM module is introduced in FasterNet Block to learn the correlation of each position of the extracted features from the spatial and channel dimensions, so that the network can be kept lightweight and focus more on extracting important feature information of the micro-Doppler image, thereby improving the network recognition accuracy. Fig. 9 shown.

[0152] The SimAM module is a parameter-free lightweight attention module, so it does not increase the complexity of the network. The pseudo code is shown in Table 2.

[0153] Table 2 SimAM module pseudo code

[0154]

[0155] SimAM defines an energy function for each neuron. The energy function is as follows:

[0156]

[0157] In the formula and is t and x i The linear transformation of w t and b t are weight and bias respectively; M is the number of neurons on the channel; regularization term is introduced in equation 3-1, and the final energy function expression is:

[0158]

[0159] Where λ is the weight value, x i is the input feature, w t and b tRespectively represent:

[0160]

[0161] In the formula is the mean and variance of all neurons in the channel except t. Therefore, the minimum energy can be calculated as follows:

[0162]

[0163] From equation 3-4, we can see that the lower the energy, the greater the difference between neuron t and the surrounding neurons, and the more attention is paid to neuron t in the model. Finally, the input feature is enhanced through equation 3-5:

[0164]

[0165] Among them, X is the input feature, is the enhanced output feature, E is e t * The sum over all channels and spatial dimensions, ⊙ is the dot product operation.

[0166] Comparison of experimental results: This paper uses NVIDIA GeForce RTX 3060Laptop GPU graphics card for training, and the software environment is python3.9, pytorch1.13. The batch size of each training in the model is 16, 100 epochs are trained, and the learning rate is 0.001; the AdamW optimizer is used, the weight decay is set to 0.0005, and the loss function is the cross entropy loss function (CrossEntropy Loss).

[0167] In order to verify the effectiveness of the improved FasterNet model in this paper, it is experimentally compared with MobileNet V3, ResNet-50 and the original network model FasterNet. This experiment uses accuracy (ACC), precision (Precision), recall rate (recall), F1-score, Params, FLOPs, and Latency to evaluate the model effect. The comparative experimental results are shown in Table 3:

[0168] Table 3 Model comparison results

[0169] Model Accuracy Accuracy Recall F1-score Params FLOPs Latency MobileNet V3 97.66％ 98.07％ 98.04％ 98.02％ 4.208M 231.4M 13.1ms ResNet-50 98.05％ 98.37％ 98.33％ 98.34％ 23.518M 4.13G 35.9ms FasterNet 97.27％ 97.68％ 97.68％ 97.68％ 2.631M 339.3M 9.7ms Ours 98.83％ 99.03％ 99.02％ 99.01％ 2.637M 352.7M 13.5ms

[0170] The ResNet-50 network has improved accuracy compared to other lightweight models. However, due to its complex model structure, the training and inference processes require a lot of computing resources and cannot meet the needs of practical applications. Compared with MobileNetV3, FasterNet has reduced the number of model parameters and inference speed, but the accuracy has also slightly decreased. The network model proposed in this paper has improved the accuracy by 1.17%, 0.78%, and 1.56% compared to MobileNet V3, ResNet-50, and FasterNet networks, respectively. At the same time, the improved FasterNet network model only increased the number of parameters and computational complexity by 0.006M and 13.4M compared to the original model, and the recognition accuracy is better than other models with only an increase of 3.9ms in delay time.

[0171] Fig.10 (a)-(d) respectively describe the accuracy curves of MobileNet V3, ResNet-50, FasterNet and improved FasterNet network models. It can be seen that the MobileNet V3 and FasterNet models converge slowly, and the curves still show large swings around 60 iterations. The curve of the improved model in this paper fluctuates greatly in the first 30 rounds, and begins to stabilize around 60 iterations, with the accuracy rate stabilizing at more than 90%. In summary, compared with the curves of the other three models, the improved model in this paper converges faster, the overall fluctuations of the training set and the validation set are small, and the loss in the later stage is always maintained at a low level.

[0172] Fig.11 Describe the confusion matrix of the improved FasterNet model for action recognition. In the recognition of walking, sitting, and falling, the accuracy rate reached 100%. In the bending action, 3% were misjudged as standing actions, and 2% of standing actions were misjudged as sitting actions. Overall, the improved FasterNet network model has significant advantages in action recognition and can better meet the needs of practical applications.

[0173] The present invention adopts the above technical solution and has the following technical features: (1) The camera is easily affected by light conditions and there is a risk of personal privacy leakage, which is not suitable for some home places such as bedrooms and living rooms. The present invention processes the radar sampling data to obtain micro-Doppler features and uses short-time Fourier transform to construct micro-Doppler maps of different actions. (2) Traditional action recognition methods ignore the size of the model, resulting in too much computational complexity in network operation. The present invention classifies the micro-Doppler maps of human body movements based on the lightweight convolutional neural network FasterNet, so that the network has lower parameters and computational complexity, which can better meet actual requirements. (3) The present invention is provided with a multi-scale feature fusion module, which extracts richer features through average pooling layers and different-scale convolutions, and combines the module with the lightweight convolutional neural network FasterNet model to prevent information loss. In order to avoid the decrease in lightweight classification accuracy, the attention mechanism is integrated into the FasterNet model to improve the network recognition accuracy.

[0174] According to the technical solution of the present invention, in the smart home, users can operate devices more conveniently through actions, and the system can also intelligently adjust the working status according to the user's actions. For example, in the elderly care, the system can analyze the elderly's physical condition in real time and issue an alarm in time when necessary.

[0175] Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of the present application is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians of the art without making creative work are within the scope of protection of the present application.

Claims

1. A radar human motion recognition method based on a lightweight model, characterized in that: It includes the following steps: Step 1: construct an FMCW radar model, and acquire radar echo data of a human body in different postures based on the FMCW radar model; Step 2: construct a micro-Doppler map dataset using radar echo data, and divide the micro-Doppler map dataset into a training set, a validation set, and a test set according to a set ratio; Step 3, construct an improved FasterNet network model, and input the preprocessed training set and the validation set into the constructed improved FasterNet network model for action recognition; the improved FasterNet network model includes an embedding layer, a multi-scale feature fusion module, and at least four FasterNet Bolcks; a merging layer is set between two adjacent FasterNet Bolcks; the output of the last FasterNet Bolck is sequentially converted and classified through a global average pooling layer, a 1×1 convolution layer, and a fully connected layer; Among them, the multi-scale feature fusion module divides the input features into three parts for parallel processing. One part performs global average pooling on each channel of the feature map through the global average pooling layer to compress the channel information and extract global features; the other two parts perform 1×1 convolution and 3×3 convolution operations respectively, and use depth-separable convolution instead of traditional convolution to reduce the number of network parameters; the extracted features of the three parts are then processed by a PWConv layer respectively, and then normalized by the BN layer, and then spliced ​​and output; FasterNet Bolck includes a PConv layer and two PWConv layers in sequence, forming an inverted residual structure; the input of FasterNet Bolck is divided into two paths, one of which is processed by a PConv layer and two PWConv layers in sequence to obtain the output features and spliced ​​with the other path as the output of FasterNet Bolck; the merging layer is used to expand the number of channels; Step 4: Train the improved FasterNet network model: Use the AdamW optimizer, adopt the cross entropy loss function as the loss function, and continuously update the model parameters to obtain the optimal model; Step 5: Process the radar sampling data to be tested to obtain the micro-Doppler feature data to be tested, and input the micro-Doppler feature data to be tested into the optimal model to obtain the classification result of human motion recognition.

2. The radar human motion recognition method based on a lightweight model according to claim 1, characterized in that: The specific steps to build the FMCW radar model in step 1 are: Step 1-1, the radar transmission signal is expressed as: In the formula, A T represents the transmission power, B represents the Chirp bandwidth, f c Indicates the starting frequency of Chirp, T c Indicates the duration of Chirp. represents phase noise; Step 1-1, the mathematical expression of the echo information received by the radar after delay and scaling is: Where, t d =2R / c is used to represent the round-trip time between the signal and the target at a distance R from the radar, and α is the attenuation of the amplitude during the signal propagation process; Step 1-3, the radar transmit signal x T (t) and receive x R (t) is mixed, and after I / Q demodulation, the mathematical expression that approximates the beat signal is obtained: In the formula, f b =2BR / (cT c ) represents the beat frequency, A R represents the received power, Indicates phase; Steps 1-4 describe the resulting beat signal in the form of a radar data matrix: Where n represents the fast time sampling axis, m represents the slow time sampling axis, T f Represents the ADC sampling time interval on the fast time sampling axis, T S Indicates the sampling time interval on the slow time sampling axis.

3. The radar human motion recognition method based on a lightweight model according to claim 2, characterized in that: Step 1-3 Ignore the residual phase noise in the experiment of detecting human motion at close range and πBt 2 d / T c , so the phase φ b (t) = 4πR / λ.

4. The radar human motion recognition method based on a lightweight model according to claim 1, characterized in that: In step 1, radar echo data of the human body in different postures are collected, and several groups of radar echo data are collected for each posture to obtain radar echo data.

5. The radar human motion recognition method based on a lightweight model according to claim 1, characterized in that: Step 2 includes the following steps: Step 2-1: After removing noise and interference from the radar echo data, use fast Fourier transform to construct a range-time map (RTM), and perform short-time Fourier transform on the spectrum along the distance dimension to form a micro-Doppler map. In step 2-2, the collected micro-Doppler images are used to construct a corresponding micro-Doppler image dataset using data augmentation, and the micro-Doppler image dataset is divided into a training set, a validation set, and a test set according to a set ratio.

6. The radar human motion recognition method based on a lightweight model according to claim 1, characterized in that: In step 2, set the ratio to 6:2:

2.

7. The radar human motion recognition method based on a lightweight model according to claim 1, characterized in that: A SimAM module is set between the two PWConv layers of FasterNet Bolck. The SimAM module learns the correlation of each position of the extracted features from the spatial and channel dimensions, so that the network can remain lightweight and focus more on extracting important feature information of the micro-Doppler image, thereby improving the network recognition accuracy.

8. The radar human motion recognition method based on a lightweight model according to claim 7, characterized in that: The SimAM module performs the following steps: S1, define the energy function for each neuron, the energy function expression is as follows: In the formula, and are t and x respectively i The linear transformation of w t and b t are weight and bias respectively; M is the number of neurons on the corresponding channel; S2, introduce the regularization term to get the final energy function expression: In the formula, λ is the weight value, x i is the input feature, w t and b t Respectively represent: In the formula, is the mean and variance of all neurons in this channel except t; S3, calculate the minimum energy, the calculation formula is as follows: in, S4, enhance the input features to obtain enhanced features. The enhancement calculation formula is as follows: Among them, X is the input feature, is the enhanced output feature, E is e t * The sum over all channels and spatial dimensions, ⊙ is the dot product operation.

9. The radar human motion recognition method based on a lightweight model according to claim 1, characterized in that: In step 4, the batch size for each training is 16, the training is done for 100 epochs, and the learning rate is 0.

001.

10. The radar human motion recognition method based on a lightweight model according to claim 1, characterized in that: The weight decay of the AdamW optimizer in step 4 is set to 0.0005.

Images