Activity recognition system and method based on millimeter wave radar Doppler characteristics

Abstract

The present invention discloses an activity recognition system and method based on millimeter-wave radar Doppler characteristics, which relates to the fields of radar signal processing and artificial intelligence technology. The system is characterized in that it includes a human behavior radar acquisition module, a human behavior information transmission module, a human behavior information preprocessing module, a micro-Doppler feature extraction module, a data classification and recognition module, and a human behavior information application module connected in sequence: the human behavior information acquisition module is used to collect reflected signals within the target area, uses a millimeter-wave radar to transmit high-frequency millimeter-wave signals, and obtains information on human movements by receiving signals reflected from the human body surface; the human behavior information transmission module is used to transmit the collected user behavior data to an edge device or a cloud server.

Description

Technical Field

[0001] The present invention relates to the field of radar signal processing and artificial intelligence technology, and in particular to an activity recognition system and method based on millimeter-wave radar Doppler characteristics. Background Art

[0002] Human activity recognition has significant application potential in areas such as smart healthcare, smart homes, and security monitoring. Existing human behavior recognition technologies are primarily categorized into three types: vision-based, wearable device-based, and radio frequency-based. Vision-based methods use cameras to capture images or videos for recognition, but they are sensitive to lighting conditions and pose privacy risks. Wearable devices, while capable of achieving high accuracy, require users to wear them continuously, disrupting their daily activities. In contrast, millimeter-wave radar, with its strong environmental adaptability, lack of light effects, and contactless sensing, has become an ideal solution for human behavior recognition. Millimeter-wave radar analyzes radio frequency echo signals to extract point clouds, range-Doppler maps, and micro-Doppler features. Micro-Doppler features are widely used in classification tasks because they can accurately capture subtle vibration patterns of the human body.

[0003] Traditional radar-based human activity recognition methods rely on manual feature extraction, such as multi-layer perceptrons, principal component analysis, and support vector machines. While effective in specific scenarios, these methods require domain expertise and are susceptible to environmental interference, limiting classification performance. Deep learning technology has significantly improved recognition accuracy, but its reliance on large amounts of data, coupled with the high cost of acquiring radar data, limits its practical application. While Transformer networks can analyze spatiotemporal micro-Doppler characteristics through self-attention mechanisms, their high computational complexity makes them difficult to meet real-time requirements. Hybrid networks, while integrating spatiotemporal features, still face challenges such as high data requirements, high resource consumption, and insufficient real-time performance.

[0004] While recent research has improved the generalization capabilities of micro-Doppler features through lightweight modules and transfer learning, these methods are often designed for scenarios with abundant data. In practice, public radar datasets are scarce and unevenly distributed, leading to significant performance degradation when data is insufficient or unbalanced. Existing methods still struggle to balance feature extraction efficiency, noise robustness, and computational overhead. A human behavior recognition solution that balances high accuracy, low resource consumption, and strong generalization is urgently needed. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide an activity recognition system and method based on millimeter-wave radar Doppler characteristics. Through dynamic feature screening and cross-channel information fusion technology, the recognition accuracy and robustness in data-deficient and unbalanced scenarios are significantly improved, while reducing computational complexity and achieving efficient real-time reasoning. It is suitable for scenarios such as smart homes, medical monitoring and industrial safety.

[0006] The present invention proposes an activity recognition system and method based on millimeter-wave radar Doppler characteristics. Through a full-process technical design, from data acquisition and signal processing to efficient feature extraction and classification, the recognition capability in complex environments is comprehensively improved. This method uses a millimeter-wave radar system to collect echo signals of human activities, covering a variety of daily activities and complex behaviors, ensuring data diversity and adaptability to actual scenarios. In the preprocessing stage, the distance-time graph is extracted through fast Fourier transform, and the static background interference is eliminated by combining the moving target indication technology. The time domain signal is converted into a high-resolution micro-Doppler feature map using short-time Fourier transform. The input image containing frequency domain energy distribution information is generated through color mapping, effectively capturing the subtle vibration patterns of the limbs and providing a data basis for subsequent analysis.

[0007] In terms of network architecture design, the masked splicing module uses a dynamically generated random masking matrix to locally mask and weight the input features, suppressing noise and enhancing the expression of key spatiotemporal regions. This module combines depthwise separable convolution with hierarchical pooling operations to gradually compress feature dimensions, reduce redundant calculations, and retain high-frequency detail information. The gated convolutional linear unit module uses a dual-branch structure to extract local spatial features and global channel features respectively, and uses a gating mechanism to adaptively fuse multi-scale information to enhance sensitivity to energy distribution and contour changes in micro-Doppler features. The overall network adopts a four-stage pyramid architecture, fusing multi-scale spatiotemporal features layer by layer, and finally outputs the classification results through global average pooling and a fully connected layer, achieving millisecond-level real-time inference while ensuring high accuracy.

[0008] Tests on multiple public datasets and self-built datasets show that this method exhibits significant advantages in scenarios with changing lighting, noise interference, and motion similarity. Through dynamic masking and cross-channel interaction, the network can accurately distinguish subtle differences in motion, and the clarity of classification boundaries is significantly improved compared to traditional methods. For scenarios with scarce data, the adaptive masking mechanism effectively alleviates the problem of model overfitting. Combined with a lightweight design, it maintains stable performance with low computing resource consumption. In practical applications, this method can be integrated into smart home systems to monitor user activities in real time and trigger security responses; in the medical field, it supports contactless rehabilitation motion tracking and quantitative evaluation; in industrial scenarios, it accurately identifies dangerous behaviors of workers, including irregular operations and high-risk actions, and provides timely warnings to reduce accident risks.

[0009] By integrating dynamic feature optimization, lightweight architecture, and multi-scale information fusion, this invention provides a highly robust, low-cost solution for millimeter-wave radar human activity recognition systems, promoting the widespread application of contactless sensing technology in smart devices, industrial safety, and healthcare. Future applications could be further expanded by combining multimodal data fusion with edge computing optimization to expand its potential in areas such as unmanned surveillance and intelligent transportation.

[0010] The present invention adopts the following technical solutions to achieve the invention objectives:

[0011] The activity recognition system and method based on millimeter-wave radar Doppler characteristics are characterized by comprising a human behavior radar acquisition module, a human behavior information transmission module, a human behavior information preprocessing module, a micro-Doppler feature extraction module, a data classification and recognition module, and a human behavior information application module connected in sequence:

[0012] The human behavior information acquisition module is used to collect reflected signals in the target area, using millimeter wave radar to transmit high-frequency millimeter wave signals, and obtain information about human movements by receiving signals reflected from the surface of the human body;

[0013] The human behavior information transmission module is used to transmit the collected user behavior data to the edge device or cloud server;

[0014] The human behavior information preprocessing module is used to receive the radar signal data transmitted from the human behavior information transmission module, perform formatting processing, and then use Fourier transform, moving target vital sign extraction and short-time Fourier transform to analyze the frequency distribution of the processed data to extract features related to human behavior;

[0015] The micro-Doppler feature extraction module is used to input the user's behavior data preprocessed by the human behavior information preprocessing module into the trained micro-Doppler feature extraction module to perform high-level feature extraction;

[0016] The data classification and recognition module is used to train and accurately identify the behavior data input into the recognition model.

[0017] The human behavior information application module is used to transmit the obtained human behavior classification and recognition results to the user information interface, thereby realizing the display of human body information.

[0018] As a further limitation of the present technical solution, the micro-Doppler feature extraction module adopts a four-layer pyramid stacking structure, which is a data downsampling unit, a four-layer feature extraction unit and a fully connected layer classifier connected in sequence. Each layer of feature extraction unit includes a mask splicing unit, a two-dimensional maximum pooling unit, and a convolutional linear gating unit connected in sequence.

[0019] As a further limitation of the present technical solution, the data classification and identification module includes a loss function calculation and update module and a classification and identification output module connected in sequence;

[0020] The loss function calculation and update module is used to calculate the mean and variance of the known category behavior data during the training process, thereby generating a loss function to update the parameters of the recognition model;

[0021] The classification recognition output module is used for calculating the determination threshold of known categories and for category mapping.

[0022] As a further limitation of the present technical solution, the human behavior information acquisition module includes a millimeter-wave radar transmitting unit, a millimeter-wave radar receiving unit and a signal amplification and processing unit, which are integrated into a radar acquisition board. The acquisition board is installed in a designated position and transmits the processed intermediate frequency signal original data by transmitting the human body echo within the frequency-modulated continuous wave detection area.

[0023] As a further limitation of the present technical solution, the human behavior information preprocessing module includes a signal storage unit, a window division unit, a Chirp matrix data reshaping unit, a range Fourier transform unit, a moving target feature extraction unit and a short-time Fourier transform unit connected in sequence, which are used to obtain a micro-Doppler map from the millimeter-wave radar raw data.

[0024] As a further limitation of the present technical solution, the human behavior information transmission module is used to transmit the collected user behavior data to the edge device or cloud server via any transmission method including WiFi, serial communication or Lora.

[0025] An identification method for an activity identification system based on millimeter wave radar Doppler characteristics, characterized by comprising the following steps:

[0026] S1: Human behavior data collection;

[0027] S2: Human behavior data transmission;

[0028] S3: human behavior data preprocessing;

[0029] S4: Micro-Doppler feature extraction and recognition;

[0030] The micro-Doppler images pre-processed by S3 are input into the recognition model in batches, and a fully connected layer classifier is obtained after training;

[0031] The fully connected layer classifier includes the GlobalAvgPool2D two-dimensional global average pooling layer and the Linear linear layer connected in sequence;

[0032] S5: Loss function calculation and network update;

[0033] In S4, the micro-Doppler feature extraction module generates feature vectors through forward propagation and inputs them into the data classification and recognition module for processing. In order to achieve efficient training and optimization, the cross-entropy loss function is used as the main optimization target, and the Adam optimizer is combined to update the model parameters. Specifically, the cross-entropy loss function is used to measure the difference between the model's predicted probability distribution and the true label. Its mathematical expression is shown in Equation (1):

[0034] (1);

[0035] in: represents the cross entropy loss value, is the total number of categories, is the one-hot encoded vector of the true label, is the probability distribution predicted by the model;

[0036] In each round of training, data is input into the model micro-Doppler feature extraction module in batches. The data size of each batch is set to B, and B is any positive integer less than the total data size. The Adam optimizer dynamically adjusts the learning rate according to the gradient information of the current batch, thereby efficiently updating the model parameters. The core idea of ​​the Adam optimizer is to combine the advantages of the momentum method and the adaptive learning rate. Its parameter update rules are shown in Equations (2), (3), (4) and (5):

[0037] (2);

[0038] (3);

[0039] (4);

[0040] (5);

[0041] in: represents the number of iteration steps, represents the gradient of the current batch, and are the first-order moment estimate and the second-order moment estimate of the gradient, and is the bias correction term for the original first-order moment and second-order moment estimates, and is the exponential decay rate, is the initial learning rate, To prevent division by zero for small constants, are model parameters;

[0042] S6: Determine the threshold and classification output;

[0043] After the micro-Doppler feature extraction module is trained, the central mean statistics of the feature vectors of known categories are performed, and the credibility threshold is set to judge the reliability of the prediction results when output. First, for each category The feature vector set is processed and its category center mean vector is calculated , as shown in formula (6):

[0044] (6);

[0045] in: Representation category The central mean vector of For category The number of samples in Representation category Middle The feature vector of each sample in the feature space;

[0046] Further calculate the distance between all samples in each category and the mean of its category center, and count the mean and standard deviation of these distances, which are recorded as and , as shown in Equation (7) and Equation (8):

[0047] (7);

[0048] (8);

[0049] in: Represents the feature vector To the category center mean The Euclidean distance of

[0050] In order to set the confidence threshold, the mean distance Add a certain multiple of the standard deviation To define the credible range of each category, as shown in formula (9):

[0051] (9);

[0052] in: Representation category The credibility threshold, is a hyperparameter;

[0053] In the model inference phase, for the input sample , the model first predicts the category to which it belongs , and then calculate the distance from the sample to the central mean of the predicted category , the distance is less than or equal to the corresponding credibility threshold , the prediction result is considered reliable; otherwise, the prediction result is considered unreliable, belongs to the unknown category or needs further confirmation;

[0054] Step S7: Determine the application of the output result;

[0055] The results of the closed set output and the open set output are transmitted to the corresponding application platform in real time.

[0056] As a further limitation of this technical solution, the specific implementation process of S3 is as follows:

[0057] S31: Reshaping of human behavior information data;

[0058] When processing the raw intermediate frequency millimeter wave radar data, in order to facilitate subsequent feature extraction and model training, it is first necessary to rearrange the continuous radar signal into a two-dimensional matrix according to the Chirp method. The raw millimeter wave radar data is represented as ,in Indicates the The intermediate frequency signal value of the sampling point, is the total number of sampling points, and each Chirp contains sampling points, and a total of Chirp, the total number of sampling points of the original data meets ;

[0059] The original data S is first divided into multiple Chirps, each Chirp corresponds to a length of The sampling point sequence is arranged in sequence to form a two-dimensional matrix , whose dimensions are , the two-dimensional matrix is ​​expressed as:

[0060] (10);

[0061] in: Indicates the Chirp's Sampling point values;

[0062] S32: radar data window segmentation;

[0063] The two-dimensional matrix generated in S31 On this basis, the data is further divided according to the time window, and the window length Defines the number of Chirps contained in each window and the step size Determines the degree of overlap or spacing between adjacent windows. The starting Chirp and ending Chirp of a window can be calculated by equations (11) and (12):

[0064] (11);

[0065] (12);

[0066] in: Indicates the segment number, the value range is , Indicates the The starting position of the segment, Indicates the The end position of the segment, and satisfy ; Indicates the fixed length of each window,

[0067] Indicates the The actual length of the segment;

[0068] The total number of window fragments finally generated It can be determined by formula (13):

[0069] (13);

[0070] S33: micro-Doppler processing;

[0071] The window matrix obtained in step S32 , each row represents a Chirp sampling point data. By performing fast Fourier transform on each row, the time domain signal is converted into a frequency domain signal, thereby extracting the target distance information, as shown in formula (14):

[0072] (14);

[0073] in: is an imaginary unit, is the first Row, No. The value of the column, Indicates the in the first Complex values ​​on distance cells, represents the frequency at which the signal is sampled, is the number of sampling points for each Chirp, and also the number of points in FFT;

[0074] After fast Fourier transform, the distance dimension data contains static clutter. In order to highlight the dynamic target signal and suppress the static clutter, a high-pass Butterworth filter is used to filter the distance dimension data for moving target indication. For discrete time signals, the difference equation of the Butterworth filter is shown in Equation (15):

[0075] (15);

[0076] in: is the input signal, is the output signal, and are the forward and feedback coefficients of the filter respectively; and Indicates the forward order of the filter and the feedback order of the filter;

[0077] After the moving target indicator filtering, the distance dimension data is shown in formula (16):

[0078] (16);

[0079] in: represents the distance dimension data after Butterworth filtering, represents the Butterworth filtering operation, The original distance dimension data, in the distance unit, The value at the time sample, represents the forward coefficient vector, represents the feedback coefficient vector;

[0080] The data after moving target indication filtering is subjected to short-time Fourier transform. The core idea of ​​short-time Fourier transform is to divide the signal into multiple short-time windows and perform FFT in each window to obtain the time-frequency distribution of the signal as shown in formula (17):

[0081] (17);

[0082] in: is an imaginary unit, represents the input signal, represents the window function, represents the time-frequency distribution, represents the local window center of the current analysis, Indicates the frequency component currently being analyzed;

[0083] After short-time Fourier transform processing, the micro-Doppler spectrum The dimension is ,in Indicates the frequency axis resolution, Indicates the time axis resolution;

[0084] Finally, the micro-Doppler spectrum is expressed by formula (18):

[0085] (18);

[0086] in: Indicates the frequency and time Energy distribution at

[0087] S34: Visualization of micro-Doppler features;

[0088] The micro-Doppler map described in S33 is converted into a logarithmic form and visualized using an image drawing tool, and label information is added by naming.

[0089] As a further limitation of this technical solution, the specific implementation process of S4 is as follows:

[0090] S41: data downsampling;

[0091] The micro-Doppler feature map input by S34 is first processed through the data downsampling module to reduce its spatial resolution and expand the number of channels. The process is expressed as Equation (19):

[0092] (19);

[0093] in: represents the output of the data downsampling module, is a Gaussian error linear unit, represents the input feature map, represents the block size used for downsampling, function The inputs are feature maps, convolution kernel size and stride respectively;

[0094] S42: micro-Doppler feature extraction;

[0095] The data after S41 processing is passed to four identical stages for layer-by-layer feature extraction. Each stage consists of a mask splicing module, a two-dimensional maximum pooling module, and a convolutional linear gating unit connected in sequence. The process is expressed as formula (20):

[0096] (20);

[0097] in: Represents the output of this stage, represents the operation of the convolutional linear gated unit, represents the operation of the mask concatenation unit, Represents a two-dimensional maximum pooling operation;

[0098] After four stages of layer-by-layer processing, the input feature map is converted into a high-level feature representation for the final classification or regression task, as shown in Equation (21):

[0099] (twenty one);

[0100] in: Represents the extracted high-level features, It is the initial feature map after being processed by the data downsampling module; Represents an operation that is processed through four stages;

[0101] S43: Classification and identification.

[0102] As a further limitation of this technical solution, the specific implementation process of S42 is as follows:

[0103] S421: mask splicing unit;

[0104] The mask splicing unit captures spatiotemporal features through multi-stage feature fusion and enhances the perception of key areas. Suppose the input feature map is , first adopt The deep convolution layer with a convolution kernel and a step size of 1 extracts basic features, and the intermediate features are obtained after batch normalization. , the adaptive mask mechanism generates a binary mask matrix dynamically right The process of implementing regional masking is shown in formula (22):

[0105] (twenty two);

[0106] in: is a learnable scaling factor, the mask matrix By the initial mask rate Control generation;

[0107] Features after masking After compression by multi-layer perceptron and the original input Add features to generate enhanced features As shown in formula (23):

[0108] (twenty three);

[0109] in: Represents a multilayer perceptron, which is a feedforward neural network structure used to perform nonlinear transformations on input data;

[0110] Final module output This is achieved by concatenating the original input and enhanced features along the channel dimension, as shown in Equation (24):

[0111] (twenty four);

[0112] The principle of the adaptive mask generation mechanism is shown in Equations (25) and (26):

[0113] (25);

[0114] (26);

[0115] in: Represents the initial mask vector, whose length is equal to the initial resolution , random permutation function Used for The elements in are randomly shuffled, Used to reshape the one-dimensional vector into the target feature map Matrices of the same shape;

[0116] S422: convolutional linear gated unit;

[0117] Convolutional linear gated units enhance feature expression capabilities through cross-channel interaction, input features After two independent linear layers and After projection, one of the branches extracts spatial features through 3×3 depth convolution and modulates them element by element with the gate weight of the other branch to generate implicit abstract features. , as shown in formula (27):

[0118] (27);

[0119] in: is the Gaussian error linear unit, represents a 3×3 depthwise convolution with a stride of 1, It is an element-level multiplication. After the second depth convolution and linear transformation, the feature is triple-added with the original input, as shown in formula (28):

[0120] (28);

[0121] in: is a linear layer, is element-wise addition, Represents the output of the convolutional linear gated unit.

[0122] The MC-GCLU network proposed in the present invention addresses the problem of insufficient feature extraction capabilities in scenarios with limited data scale or uneven distribution, and significantly improves feature extraction efficiency and generalization performance by enhancing the model's adaptive learning ability for sparse data. The network exhibits stronger robustness under conditions of low sample size or class imbalance, effectively alleviating the constraints of data scarcity on model performance. In order to solve the problem of balancing computational overhead and accuracy, the present invention proposes a joint design scheme of a pyramid architecture and a mask splicing module. This module reduces redundant calculations and optimizes feature processing paths through a dynamic feature selection mechanism, thereby reducing computational costs while maintaining recognition accuracy and meeting the computing power constraints of real-time reasoning. The present invention proposes a method for human behavior recognition based on millimeter-wave radar. By setting a suitable loss function, adjusting model parameters, and setting an open set threshold, the problem that existing classification models cannot recognize data of unknown categories can be solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0123] Figure 1 It is a schematic diagram of the module composition and connection relationship of the present invention.

[0124] Figure 2 Schematic diagram of the flow of the human activity recognition method of the present invention.

[0125] Figure 3 Schematic diagram of the principle of the micro-Doppler feature extraction module in the local human activity recognition method of the present invention. DETAILED DESCRIPTION

[0126] A specific embodiment of the present invention is described in detail below with reference to the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific embodiment.

[0127] Example 1

[0128] Traditional visual monitoring for elderly people faces privacy violations and insufficient nighttime monitoring capabilities. This technology, deployed in nursing home monitoring systems, uses millimeter-wave radar to non-contactly sense micro-movements, enabling all-weather, non-invasive monitoring. Mounted on the ceiling of the room, the millimeter-wave radar transmits a frequency-modulated continuous wave (FMCW) signal to the target area. Reflected echoes from the human body are then generated into an intermediate-frequency (IF) signal, which is then pre-processed to produce a micro-Doppler spectrum. When high-risk behavior, such as a fall, is detected, the system automatically triggers an alarm and notifies caregivers. It also supports long-term behavioral data analysis to assess health status.

[0129] like Figure 1 As shown, the present invention includes a human behavior radar acquisition module, a human behavior information transmission module, a human behavior information preprocessing module, a micro-Doppler feature extraction module, a data classification and recognition module, and a human behavior information application module connected in sequence:

[0130] The human behavior information acquisition module is used to collect reflected signals within the target area. It uses a millimeter-wave radar to transmit high-frequency millimeter-wave signals and obtain information about human movement by receiving signals reflected from the human body surface. The module first activates the millimeter-wave radar transmitting unit to emit electromagnetic waves toward the target area and then receives the reflected echo signal through the millimeter-wave radar receiving unit. The radar signal is amplified by a low-noise amplifier at the front end and then enters the signal amplification and processing unit, where preliminary time-domain and frequency-domain filtering is performed to remove environmental noise. The data acquisition process continues continuously. When the collected raw radar data meets processing requirements, it is stored and cached in preparation for subsequent transmission and processing. The types of data collected primarily include frequency variations, echo strength, and arrival time difference of the radar echo signal.

[0131] The human behavior information transmission module is used to transmit the collected user behavior data to an edge device or cloud server. Depending on the user's scenario, an appropriate transmission method can be selected to transmit the collected user behavior data to the cloud server via either WiFi or Lora, and then forwarded to the designated device. Different transmission methods differ in power consumption, transmission distance, and transmission speed, and are therefore suitable for different application scenarios. For application scenarios directly involving edge devices, rapid transmission can also be achieved through serial ports for data processing.

[0132] The human behavior information preprocessing module is used to receive the radar signal data transmitted from the human behavior information transmission module and perform formatting processing, including time alignment and window division, to ensure the consistency of the millimeter wave radar data in the time dimension. The processed data is then subjected to Fourier transform, moving target sign extraction, and short-time Fourier transform to analyze the frequency distribution of the signal and extract features related to human behavior.

[0133] The micro-Doppler feature extraction module is used to input the user's behavior data preprocessed by the human behavior information preprocessing module into the trained micro-Doppler feature extraction module to perform high-level feature extraction.

[0134] The data classification and recognition module is used to train and accurately identify the behavior data input into the recognition model.

[0135] The human behavior information application module is used to transmit the obtained human behavior classification and recognition results to the user information interface, thereby realizing the display of human body information.

[0136] This system accurately monitors both high-risk fall behaviors and everyday behaviors by analyzing the time-frequency characteristics of millimeter-wave radar echo signals. In the micro-Doppler feature extraction module, a four-layer pyramid stacking structure is constructed. A multi-resolution fusion strategy, combining an adaptive masking mechanism with a convolutional linear gating unit, significantly improves the accuracy of complex human motion recognition by dynamically sensing the distribution of micro-Doppler features in key motion areas. A dynamic threshold determination method based on statistical distribution modeling is also proposed to effectively avoid the risk of misjudgment.

[0137] Example 2:

[0138] The micro-Doppler feature extraction module adopts a four-layer pyramid stacking structure, which consists of a data downsampling unit, four layers of feature extraction units and a fully connected layer classifier connected in sequence. Each layer of feature extraction units includes a mask splicing unit, a two-dimensional maximum pooling unit, and a convolutional linear gating unit connected in sequence.

[0139] The main function of the data downsampling unit is to sample and process the pre-processed user behavior data. This is to achieve efficient data compression and feature extraction. This module downsamples the input data through a specific sampling size and activation function to generate a feature representation suitable for subsequent processing. In the specific implementation, the data sampling layer downsamples the original input data with a dimension of 3×224×224. The data sampling layer uses a 4×4 convolution kernel and combines it with a Gaussian error linear unit for downsampling, thereby effectively compressing the amount of data in the data processing process. By performing a convolution operation on the input data, the sampling layer significantly reduces the spatial dimension of the original data while retaining important feature information. In addition, in order to adapt to the requirements of the subsequent feature extraction layer, the data sampling layer encodes the number of channels to 32, thereby providing a standardized input format for subsequent modules.

[0140] The main function of the mask splicing unit is to enhance the robustness of feature extraction and the sensitivity of feature contours through an adaptive masking mechanism. This mechanism dynamically adjusts the input data, thereby enhancing the network's resistance to various input noise and interference, while optimizing the extraction of feature edges and detail information. Furthermore, through the splicing operation, the mask splicing unit effectively preserves the original characteristics of the data, ensuring that key information is not lost throughout the data processing process, thereby improving the precision and accuracy of feature extraction. The downsampled data is first fed into the deep convolutional layer, whose primary function is to extract the basic features of the data. This process ensures that the most basic patterns and regularities are captured during further processing, laying the foundation for subsequent deep feature extraction and ensuring the reliability of feature extraction results. Next, the adaptive masking mechanism generates a mask matrix that meets the required masking rate and is combined with the original data to complete the data masking process. The masked data is then fed into the convolutional layer to extract features from the incomplete data. This stage effectively captures the missing data caused by the masking mechanism. The data's hidden features are then extracted through a multi-layer perceptron (MLP), and the original features are effectively preserved through a residual structure. This structure effectively mitigates the information loss that can occur during deep network training, ensuring stable network performance. Finally, the original input data and the processed data are concatenated along the channel dimension. This concatenation preserves rich contextual information during data processing, facilitating a comprehensive understanding of the features.

[0141] In its implementation, the parameter design of the mask concatenation unit fully considers the balance between feature extraction efficiency and model performance. The module's input channels default to 64, and the initial masking ratio is set to 0.7, controlling the proportion of masked regions in the mask matrix. The masking ratio is dynamically adjusted via a learnable parameter, ensuring that the model can adaptively optimize the mask distribution based on feedback during training. The downsampled data is first fed into a depthwise convolutional layer, which uses a 3×3 convolution kernel with a stride of 1 and employs grouped convolution to ensure independent convolution operations for each channel. This design not only reduces computational effort but also enhances feature spatial localization. The data is then normalized by a convolution layer with a 3×3 convolution kernel and a stride of 1, followed by a batch normalization layer, further optimizing data distribution. To further mine hidden features, the module incorporates a multilayer perceptron, whose hidden layer dimension is half the number of input channels. Residual connections are used to sum the original and hidden features to ensure information integrity. Finally, the original input data and the processed data are concatenated in the channel dimension to form a feature representation containing rich contextual information.

[0142] The main function of the adaptive mask mechanism is to enhance the local feature representation through the generation and feature extraction of random mask matrices. A dynamic mask generator is constructed based on the two-dimensional spatial dimension of the input feature map. For the input feature map, a basic mask template is generated in the original high-width resolution space through parameterized initialization. This template randomly selects effective perception areas within the preset mask ratio range, randomly rearranges the spatial positions of the feature map, and forms a binary mask matrix that matches the spatial dimension of the input feature map. Subsequently, the generated mask matrix is ​​multiplied pixel by pixel with the original input feature map to achieve selective masking of features in the specified spatial area. In order to maintain the integrity of the feature expression, the masked local features are weightedly fused with the original features, and the contribution strength of the masked features is dynamically adjusted through a learnable scaling coefficient, ultimately forming a composite feature representation that preserves global structure and enhances local features. This process enables the model to adaptively strengthen the feature response of key areas while suppressing irrelevant background noise by dynamically adjusting the mask distribution pattern and feature fusion weights.

[0143] The two-dimensional maximum pooling unit is used to compress input data, reduce the consumption of computing resources, and compress the data value of the data space dimension to one-fourth of the original value through a maximum pooling layer.

[0144] The convolutional linear gating unit is used to coordinate the local detail perception and global feature dependency, significantly improving the discriminability of feature representation. First, a two-way parallel processing architecture is implemented for the input features. The first branch extracts spatial features through the cascade operation of linear transformation and deep convolution, and generates fine-grained feature responses in combination with nonlinear activation functions; the second branch adopts a two-level linear layer stacking structure to capture high-order correlation features of the channel dimension through linear projection transformation. Subsequently, the two-way processing results are subjected to cross-modal feature interaction fusion, and implicit abstract feature representations are generated by element-by-element multiplication. In order to enhance the feature expression capability, linear transformation and deep convolution operations are applied to the fused features respectively, and the outputs of the two are weightedly superimposed to form a composite feature with both local receptive field characteristics and global context perception. Finally, by introducing the residual connection mechanism, the composite feature is adaptively fused with the original input feature, while retaining the integrity of the initial features, the cross-channel information interaction capability is dynamically enhanced.

[0145] The core function of the convolutional linear gated unit (CLGU) is to significantly improve the discriminability of feature representations by coordinating the perception of local details with global feature dependencies. The module first implements a two-way parallel processing architecture for the input features. The first branch extracts fine-grained features in the spatial dimension through a cascade of linear transformations and depthwise convolutions. The linear layer has input and output dimensions of 32. The depthwise convolutional layer uses a 3×3 convolution kernel with a stride of 1 and employs grouped convolutions, with the number of groups equal to the number of input channels, or 32. The second branch captures high-order correlation features in the channel dimension using a stacked structure of two linear layers. The linear layer also has input and output dimensions of 32, and a nonlinear transformation is introduced using a Gaussian error linear unit (GELU). The results of the two-way processing are then fused through element-wise multiplication to generate implicit and abstract feature representations. To enhance feature representation, the fused features undergo a linear transformation and a depthwise convolution, respectively. The linear layer also has input and output dimensions of 32. The depthwise convolutional layer also uses a 3×3 convolution kernel with a stride of 1 and a grouping of 32. The outputs of the two are combined through weighted superposition to form composite features that combine local receptive field characteristics with global context awareness. Ultimately, the composite features are adaptively fused with the original input features through a residual connection mechanism, dynamically enhancing cross-channel information interaction while preserving the integrity of the initial features, thereby achieving efficient modeling and expression of complex features.

[0146] The multi-resolution attention parallel convolution module is used to extract deep features from sensor data of different resolutions. The multi-resolution attention parallel convolution module includes a same-resolution data splicing unit, a parallel self-attention unit, and a parallel convolution unit connected in sequence. The same-resolution data splicing unit generates four types of resolution data from the nine-axis sensor data processed by the first data multi-resolution fusion convolution unit, namely, the X, Y, Z three-axis data of the acceleration sensor, the X, Y, Z three-axis data of the angular velocity sensor, and the X, Y, Z three-axis data of the magnetometer. The data is spliced ​​together according to the principle of the same resolution so that the data of different resolutions all include the nine-axis sensor data. The parallel self-attention unit automatically weights the important features in the data of different resolutions, giving more attention to the important features. The parallel convolution unit further performs deep feature extraction on the data processed by the self-attention unit.

[0147] The fused attention convolution module is used to deeply fuse feature data of different resolutions; the fused attention convolution module includes a second data multi-resolution fused convolution unit, a fused self-attention unit and a fused convolution unit connected in sequence; the second data multi-resolution fused convolution unit is calculated in the same way as the first data multi-resolution fused convolution unit, and is used to fuse parallel feature data of different resolutions into feature data with a length of 80 feature data of the original sensor data window size; the fused self-attention unit is calculated in the same way as the above-mentioned parallel self-attention unit, and is used to automatically extract important features from the fused deep feature data; the fused convolution unit further performs deep extraction of data features;

[0148] The closed set output module is used to output the closed set classification recognition results and the open set feature vector output. The closed set output module includes a fully connected unit and a closed set classification and discrimination output unit connected in sequence; the fully connected unit is a multi-layer fully connected layer that transforms the data into a feature space; the closed set classification and discrimination output unit calculates and outputs the recognition result through the Softmax layer.

[0149] The fully connected classifier is used to classify the extracted abstract features. First, global feature aggregation is performed on the high-dimensional feature map processed by the feature extraction network, and the two-dimensional feature matrix is ​​compressed into a one-dimensional channel feature vector through a two-dimensional global average pooling operation, thereby eliminating spatial position sensitivity while retaining the discriminative information of the channel dimension. Subsequently, the channel dimension of the feature vector after dimensionality reduction is calibrated, and the key channel features with category discrimination are screened through the feature compression operation. Finally, the refined feature representation is input into the learnable linear projection layer, and the high-dimensional feature space is mapped to the target classification dimension to generate a probability distribution output that matches the preset number of categories. This classification architecture uses a space-channel collaborative optimization mechanism to effectively suppress redundant feature interference while reducing the number of model parameters, thereby improving the reliability and robustness of classification decisions.

[0150] The data classification and identification module includes a loss function calculation and update module and a classification and identification output module connected in sequence;

[0151] The loss function calculation and update module is used to calculate the mean and variance of known category behavior data during the training process, thereby generating a loss function to update the parameters of the recognition model; in the model training stage, after each batch of forward inference is completed, the module first receives the predicted probability distribution and the labeled true value output by the model, and calculates the feature discrimination error of the current batch through the cross entropy criterion. This error value represents the model's ability to distinguish category features, and then triggers the reverse error propagation mechanism to analyze the contribution of each network parameter to the error layer by layer along the calculation graph to generate the corresponding gradient tensor. The module's built-in adaptive optimizer is started synchronously to dynamically smooth the original gradient generated by the current batch and construct a continuous gradient correction. After each round of training, the constructed loss function is used to adjust the parameters of the recognition model, and this process is repeated until convergence.

[0152] The classification recognition output module is used for the calculation of the judgment threshold and category mapping of known categories. In the category threshold determination unit, all the correctly classified feature data points in each category are first extracted from the training data, and the distance between these data points and the central mean point of the category is calculated. The calculated distance value is then normalized, and the distance value is limited to a uniform range to eliminate the impact of distribution differences. Based on the normalized distance distribution, a suitable statistic is selected as the judgment threshold of the category. In the recognition output unit, the distance between the data points to be classified and the central mean point of each category is calculated in turn, and these distance values ​​are compared with the judgment threshold of the corresponding category to determine the optimal matching category to which the data point belongs. Finally, the determined category is mapped to the corresponding behavior label to generate the final output result. Through the above process, the module realizes the complete classification and mapping process from feature data to behavior labels.

[0153] The human behavior information acquisition module includes a millimeter-wave radar transmitting unit, a millimeter-wave radar receiving unit and a signal amplification processing unit, which are integrated into a radar acquisition board. The acquisition board is installed in a designated position and transmits the processed intermediate frequency signal raw data by transmitting human body echoes within the frequency-modulated continuous wave detection area.

[0154] The human behavior information preprocessing module includes a signal storage unit, a window division unit, a Chirp matrix data reshaping unit, a range Fourier transform unit, a moving target feature extraction unit and a short-time Fourier transform unit connected in sequence, and is used to obtain a micro-Doppler map from the millimeter wave radar raw data.

[0155] The signal storage unit is used to store the millimeter-wave radar behavior data collected in real time. This data includes the user's behavioral characteristics at different time points, and the reflected signals captured by the radar sensor record the human body's movement status in space.

[0156] The window division unit is used to divide the continuous long-term millimeter-wave radar data into several data segments. By setting the appropriate window length and step size, the original data stream can be effectively divided into multiple independent data segments.

[0157] The chirp matrix data reshaping unit is used to convert one-dimensional input data into a two-dimensional matrix. Since millimeter-wave radar uses a chirp waveform, each chirp wave corresponds to a specific time period and frequency range. Therefore, during processing, the one-dimensional data must be reorganized into a two-dimensional matrix based on the number of chirp waves to facilitate subsequent micro-Doppler map generation and analysis. This reshaping process not only preserves the time and frequency information of the original data but also makes the data structure more regular, facilitating subsequent calculations and visualization.

[0158] The distance Fourier transform unit is used to perform a Fourier transform on the two-dimensional raw data to convert it into distance-time data. This process can separate target information at different distances, allowing subsequent processing to more clearly identify the position changes of each target.

[0159] The moving target feature extraction unit is used to suppress static or slowly changing background clutter using a Butterworth filter. By filtering out these interfering signals, the features of the dynamic target can be highlighted, thereby improving the detection accuracy of the moving target.

[0160] The short-time Fourier transform (STFT) unit performs a short-time Fourier transform on the moving target feature-extracted data to obtain frequency-time data, i.e., a micro-Doppler map. This transformation method can capture the target's speed changes over a short period of time, further refining the description of its behavioral characteristics.

[0161] The human behavior information application module is oriented to various application platforms, and the classification and recognition results are transmitted to the database of each application platform in real time for storage and management, so as to visualize and manage the user's behavior in real time.

[0162] The human behavior information transmission module is used to transmit the collected user behavior data to the edge device or cloud server via WiFi, serial communication or any transmission method of Lora.

[0163] Example 3: A recognition method of an activity recognition system based on millimeter wave radar Doppler characteristics is implemented by the open set human behavior recognition system based on multi-resolution fusion convolution described in any one of Examples 1 or 2, such as Figure 2 As shown, taking the management application platform as an example, the following steps are included:

[0164] S1: Human behavior data collection;

[0165] The millimeter-wave radar transmitter transmits a frequency-modulated continuous wave (FMCW) signal toward the target area via the transmitting antenna. The receiving antenna simultaneously captures the echo signal reflected from the human body surface. This signal is amplified by a low-noise amplifier and then enters the signal processing unit. At this stage, time-domain and frequency-domain filtering are used to remove ambient noise, and the processed intermediate frequency data is then received.

[0166] S2: Human behavior data transmission;

[0167] The compressed radar raw data packets are transmitted to edge computing nodes or cloud servers via WiFi or LoRa. UDP is used as the transmission protocol to reduce latency, and the packet header includes a timestamp, device ID, and checksum to ensure transmission integrity. If signal strength is insufficient, serial communication can also be used for wired transmission, directly to the edge device for processing.

[0168] S3: human behavior data preprocessing;

[0169] The collected user radar data is segmented, reshaped, Doppler processed, visualized, normalized and labeled in sequence to obtain a behavioral micro-Doppler map with dynamic energy characteristic information and labels.

[0170] S4: Micro-Doppler feature extraction and recognition;

[0171] The micro-Doppler images pre-processed by S3 are input into the recognition model in batches, and a fully connected layer classifier is obtained after training;

[0172] The fully connected layer classifier includes the GlobalAvgPool2D two-dimensional global average pooling layer and the Linear linear layer connected in sequence;

[0173] S5: Loss function calculation and network update;

[0174] In S4, the micro-Doppler feature extraction module generates feature vectors through forward propagation and inputs them into the data classification and recognition module for processing. In order to achieve efficient training and optimization, the cross-entropy loss function is used as the main optimization target, and the Adam optimizer is combined to update the model parameters. Specifically, the cross-entropy loss function is used to measure the difference between the model's predicted probability distribution and the true label. Its mathematical expression is shown in Equation (1):

[0175] (1);

[0176] in: represents the cross entropy loss value, is the total number of categories, is the one-hot encoded vector of the true label, is the probability distribution predicted by the model;

[0177] In each round of training, data is input into the model micro-Doppler feature extraction module in batches. The data size of each batch is set to B, and B is any positive integer less than the total data size. The Adam optimizer dynamically adjusts the learning rate according to the gradient information of the current batch, thereby efficiently updating the model parameters. The core idea of ​​the Adam optimizer is to combine the advantages of the momentum method and the adaptive learning rate. Its parameter update rules are shown in Equations (2), (3), (4) and (5):

[0178] (2);

[0179] (3);

[0180] (4);

[0181] (5);

[0182] in: represents the number of iteration steps, represents the gradient of the current batch, and are the first-order moment estimate and the second-order moment estimate of the gradient, and is a bias correction term for the original first-order moment (mean) and second-order moment (variance) estimates, and is the exponential decay rate, is the initial learning rate, To prevent division by zero for small constants, are model parameters;

[0183] During training, the model is first supervised using the cross-entropy loss function and the Softmax function to ensure that the model can accurately predict the labels of known categories. Ultimately, through continuous iterative optimization, the model gradually converges to the optimal state, demonstrating higher classification accuracy and robustness on the test set.

[0184] S6: Determine the threshold and classification output;

[0185] After the micro-Doppler feature extraction module is trained, in order to improve the discrimination ability in practical applications, the central mean statistics of the feature vectors of known categories are performed, and the credibility threshold is set based on this to judge the reliability of the prediction results when outputting. Specifically, first, for each category The feature vector set is processed and its category center mean vector is calculated , as shown in formula (6):

[0186] (6);

[0187] in: Representation category The central mean vector of For category The number of samples in Representation category Middle The feature vector of each sample in the feature space;

[0188] On this basis, we further calculate the distances from all samples in each category to the mean of the category center, and calculate the mean and standard deviation of these distances, which are recorded as and , as shown in Equation (7) and Equation (8):

[0189] (7);

[0190] (8);

[0191] in: Represents the feature vector To the category center mean The Euclidean distance of

[0192] In order to set the confidence threshold, the mean distance Add a certain multiple of the standard deviation To define the credible range of each category, as shown in formula (9):

[0193] (9);

[0194] in: Representation category The credibility threshold, It is a hyperparameter used to control the strictness of the threshold and is usually adjusted according to actual needs;

[0195] In the model inference phase, for the input sample , the model first predicts the category to which it belongs , and then calculate the distance from the sample to the central mean of the predicted category , the distance is less than or equal to the corresponding credibility threshold , the prediction result is considered reliable; otherwise, the prediction result is considered unreliable, belongs to an unknown category or needs further confirmation; this method can effectively improve the robustness of the model in open set scenarios while reducing the risk of misclassification.

[0196] Step S7: Determine the application of the output result;

[0197] The results of the closed set output and the open set output are transmitted to the corresponding application platform in real time.

[0198] The specific implementation process of S3 is as follows:

[0199] S31: Reshaping of human behavior information data;

[0200] When processing the raw intermediate frequency millimeter wave radar data, in order to facilitate subsequent feature extraction and model training, it is first necessary to rearrange the continuous radar signal into a two-dimensional matrix according to the Chirp method. The raw millimeter wave radar data is represented as ,in Indicates the The intermediate frequency signal value of the sampling point, is the total number of sampling points, and each Chirp contains sampling points, and a total of Chirp, the total number of sampling points of the original data meets ;

[0201] The original data S is first divided into multiple Chirps, each Chirp corresponds to a length of The sampling point sequence is arranged in sequence to form a two-dimensional matrix , whose dimensions are , the two-dimensional matrix is ​​expressed as:

[0202] (10);

[0203] in: Indicates the Chirp's Sampling point values;

[0204] S32: radar data window segmentation;

[0205] The two-dimensional matrix generated in S31 On this basis, the data is further divided according to the time window, and the window length Defines the number of Chirps contained in each window and the step size Determines the degree of overlap or spacing between adjacent windows. The starting Chirp and ending Chirp of a window can be calculated by equations (11) and (12):

[0206] (11);

[0207] (12);

[0208] in: Indicates the segment number, the value range is , Indicates the The starting position of the segment, Indicates the The end position of the segment, and satisfy ; Indicates the fixed length of each window,

[0209] Indicates the The actual length of the segment;

[0210] The total number of window fragments finally generated It can be determined by formula (13):

[0211] (13);

[0212] S33: micro-Doppler processing;

[0213] The window matrix obtained in step S32 , each row represents a Chirp sampling point data. By performing fast Fourier transform on each row, the time domain signal is converted into a frequency domain signal, thereby extracting the target distance information, as shown in formula (14):

[0214] (14);

[0215] in: is an imaginary unit, is the first Row, No. The value of the column, Indicates the in the first Complex values ​​on distance cells, represents the frequency at which the signal is sampled, is the number of sampling points for each Chirp, and also the number of points in FFT;

[0216] After fast Fourier transform, the distance dimension data contains static clutter. In order to highlight the dynamic target signal and suppress the static clutter, a high-pass Butterworth filter is used to filter the distance dimension data for moving target indication. For discrete time signals, the difference equation of the Butterworth filter is shown in Equation (15):

[0217] (15);

[0218] in: is the input signal, is the output signal, and are the forward and feedback coefficients of the filter respectively; and Indicates the forward order of the filter and the feedback order of the filter;

[0219] After the moving target indicator filtering, the distance dimension data is shown in formula (16):

[0220] (16);

[0221] in: represents the distance dimension data after Butterworth filtering, represents the Butterworth filtering operation, The original distance dimension data, in the distance unit, The value at the time sample, represents the forward coefficient vector, represents the feedback coefficient vector;

[0222] In order to further extract the micro-Doppler characteristics of the target, the data after the moving target indication filter is subjected to short-time Fourier transform. The core idea of ​​short-time Fourier transform is to divide the signal into multiple short-time windows and perform FFT in each window to obtain the time-frequency distribution of the signal as shown in formula (17):

[0223] (17);

[0224] in: is an imaginary unit, represents the input signal, represents the window function, represents the time-frequency distribution, represents the local window center of the current analysis, Indicates the frequency component currently being analyzed;

[0225] After short-time Fourier transform processing, the micro-Doppler spectrum The dimension is ,in Indicates the frequency axis resolution, Indicates the time axis resolution;

[0226] Finally, the micro-Doppler spectrum is expressed by formula (18):

[0227] (18);

[0228] in: Indicates the frequency and time Energy distribution at

[0229] S34: Visualization of micro-Doppler features;

[0230] In order to intuitively display the micro-Doppler characteristics, the micro-Doppler map described in S33 is converted into a logarithmic form and visualized using an image drawing tool, and label information is added by naming.

[0231] The specific implementation process of S4 is as follows:

[0232] S41: data downsampling;

[0233] The micro-Doppler feature map input by S34 is first processed by the data downsampling module to reduce its spatial resolution and expand the number of channels. The process can be expressed as Equation (19):

[0234] (19);

[0235] in: represents the output of the data downsampling module, is a Gaussian error linear unit, represents the input feature map, Indicates the block size used for downsampling, which is set to 4; function The inputs are the feature map, convolution kernel size and step size, which together determine the downsampling ratio;

[0236] After the convolution operation, the spatial resolution of the feature map is effectively compressed, while layer normalization is introduced to ensure the stability of the numerical distribution. A Gaussian error linear unit further performs nonlinear transformations on the features, enhancing the model's expressive power. Ultimately, the number of channels in the feature map is expanded to 32, reducing computational complexity while preserving key feature information, laying the foundation for subsequent feature extraction.

[0237] S42: micro-Doppler feature extraction;

[0238] After the downsampled feature map, the input feature map is composed of becomes , which allows the resolution of the feature map to be compressed, effectively reducing the amount of computation in subsequent calculations while increasing the speed of computation. The data processed by S41 is passed to four identical stages for layer-by-layer feature extraction. Each stage consists of a mask splicing module, a two-dimensional maximum pooling, and a convolutional linear gating unit connected in sequence. The process is expressed as formula (20):

[0239] (20);

[0240] in: Represents the output of this stage, represents the operation of the convolutional linear gated unit, represents the operation of the mask concatenation unit, Represents a two-dimensional max pooling operation. During micro-Doppler feature extraction, the spatial resolution of the input feature map is halved while the number of channels is doubled after each stage of processing. In each stage, the two-dimensional max pooling operation downsamples the feature map, halving the spatial resolution. The mask stitching module doubles the number of channels by concatenating the previous and next data.

[0241] After four stages of layer-by-layer processing, the input feature map is converted into a high-level feature representation for the final classification or regression task. This process can be expressed as shown in Equation (21):

[0242] (twenty one);

[0243] in: Represents the extracted high-level features, usually used to predict the target activity or behavior category; It is the initial feature map after being processed by the data downsampling module; Represents an operation processed in four stages, and the size of the feature map is determined by becomes ;

[0244] S43: Classification and identification.

[0245] The fully connected layer classifier includes a GlobalAvgPool2D two-dimensional global average pooling layer and a Linear linear layer connected in sequence.

[0246] In S42, upsampling and downsampling methods are used again to perform fusion convolution calculations at different resolutions, and feature extraction at different resolutions is further performed on the sensor single-axis data. The specific implementation process is as follows:

[0247] S421: mask splicing unit;

[0248] The mask splicing unit captures spatiotemporal features through multi-stage feature fusion and enhances the perception of key areas. Suppose the input feature map is , first adopt The deep convolution layer with a convolution kernel and a step size of 1 extracts basic features, and the intermediate features are obtained after batch normalization. , the adaptive mask mechanism generates a binary mask matrix dynamically right The process of implementing regional masking is shown in formula (22):

[0249] (twenty two);

[0250] in: is a learnable scaling factor, the mask matrix By the initial mask rate Control generation;

[0251] Specifically, it is achieved by random arrangement and matrix reshaping to ensure random shielding of non-critical areas. The convolution layer with a convolution kernel and a stride of 1 extracts mask features;

[0252] Features after masking After compression by multi-layer perceptron and the original input Add features to generate enhanced features As shown in formula (23):

[0253] (twenty three);

[0254] in: Represents a multilayer perceptron, which is a feedforward neural network structure used to perform nonlinear transformations on input data;

[0255] Final module output This is achieved by concatenating the original input and enhanced features along the channel dimension, as shown in Equation (24):

[0256] (twenty four);

[0257] The principle of the adaptive mask generation mechanism is shown in Equations (25) and (26):

[0258] (25);

[0259] (26);

[0260] in: Represents the initial mask vector, whose length is equal to the initial resolution , random permutation function Used for The elements in are randomly shuffled, Used to reshape the one-dimensional vector into the target feature map Matrices of the same shape;

[0261] exist middle, front The positions are set to 1 and the rest are set to 0, indicating the selective activation areas of the initial mask. After random permutation and reshaping, the resulting mask matrix is With the same input feature map The same spatial dimension can be used for subsequent mask operations or feature selection tasks.

[0262] S422: convolutional linear gated unit;

[0263] Convolutional linear gated units enhance feature expression capabilities through cross-channel interaction, input features After two independent linear layers and After projection, one of the branches extracts spatial features through 3×3 depth convolution and modulates them element by element with the gate weight of the other branch to generate implicit abstract features. , as shown in formula (27):

[0264] (27);

[0265] in: is the Gaussian error linear unit, represents a 3×3 depthwise convolution with a stride of 1, It is an element-level multiplication. After the second depth convolution and linear transformation, the feature is triple-added with the original input, as shown in formula (28):

[0266] (28);

[0267] in: is a linear layer, is element-wise addition, Represents the output of the convolutional linear gated unit.

[0268] This design fuses multi-scale features through a gating mechanism, models local context using 3×3 convolutional kernels, and enhances semantic associations while preserving original features. Compared to traditional attention mechanisms, this design significantly reduces complexity through parameter sharing and parallel computation.

[0269] The above disclosure is only a specific embodiment of the present invention, but the present invention is not limited thereto. Any changes that can be conceived by those skilled in the art should fall within the scope of protection of the present invention.

Claims

1. A recognition method for an activity recognition system based on millimeter wave radar Doppler characteristics, characterized in that: The following steps are involved: S1: Human behavior data collection; S2: Human behavior data transmission; S3: human behavior data preprocessing; S4: Micro-Doppler feature extraction and recognition; The micro-Doppler images pre-processed by S3 are input into the recognition model in batches, and a fully connected layer classifier is obtained after training; The fully connected layer classifier includes the GlobalAvgPool2D two-dimensional global average pooling layer and the Linear linear layer connected in sequence; S5: Loss function calculation and network update; In S4, the micro-Doppler feature extraction module generates feature vectors through forward propagation and inputs them into the data classification and recognition module for processing. In order to achieve efficient training and optimization, the cross-entropy loss function is used as the main optimization target, and the Adam optimizer is combined to update the model parameters. Specifically, the cross-entropy loss function is used to measure the difference between the model's predicted probability distribution and the true label. Its mathematical expression is shown in Equation (1): （1）； in: represents the cross entropy loss value, is the total number of categories, is the one-hot encoded vector of the true label, is the probability distribution predicted by the model; In each round of training, data is input into the model micro-Doppler feature extraction module in batches. The data size of each batch is set to B, and B is any positive integer less than the total data size. The Adam optimizer dynamically adjusts the learning rate according to the gradient information of the current batch, thereby efficiently updating the model parameters. The core idea of ​​the Adam optimizer is to combine the advantages of the momentum method and the adaptive learning rate. Its parameter update rules are shown in Equations (2), (3), (4) and (5): （2）； （3）； （4）； （5）； in: represents the number of iteration steps, represents the gradient of the current batch, and are the first-order moment estimate and the second-order moment estimate of the gradient, and is the bias correction term for the original first-order moment and second-order moment estimates, and is the exponential decay rate, is the initial learning rate, To prevent division by zero for small constants, are model parameters; S6: Determine the threshold and classification output; After the micro-Doppler feature extraction module is trained, the central mean statistics of the feature vectors of known categories are performed, and the credibility threshold is set to judge the reliability of the prediction results when output. First, for each category The feature vector set is processed and its category center mean vector is calculated , as shown in formula (6): （6）； in: Representation category The central mean vector of For category The number of samples in Representation category Middle The feature vector of each sample in the feature space; Further calculate the distance between all samples in each category and the mean of its category center, and count the mean and standard deviation of these distances, which are recorded as and , as shown in Equation (7) and Equation (8): （7）； （8）； in: Represents the feature vector To the category center mean The Euclidean distance of In order to set the confidence threshold, the mean distance Add a certain multiple of the standard deviation To define the credible range of each category, as shown in formula (9): （9）； in: Representation category The credibility threshold, is a hyperparameter; In the model inference phase, for the input sample , the model first predicts the category to which it belongs , and then calculate the distance from the sample to the central mean of the predicted category , the distance is less than or equal to the corresponding credibility threshold , the prediction result is considered reliable; otherwise, the prediction result is considered unreliable, belongs to the unknown category or needs further confirmation; Step S7: Determine the application of the output result; The results of the closed set output and the open set output are transmitted to the corresponding application platform in real time; The specific implementation process of S4 is as follows: S41: data downsampling; The micro-Doppler feature map input by S34 is first processed through the data downsampling module to reduce its spatial resolution and expand the number of channels. The process is expressed as Equation (19): （19）； in: represents the output of the data downsampling module, is a Gaussian error linear unit, represents the input feature map, represents the block size used for downsampling, function The inputs are feature maps, convolution kernel size and stride respectively; S42: micro-Doppler feature extraction; After S41 processing, the data is passed to four identical stages for layer-by-layer feature extraction. Each stage consists of a mask splicing module, a two-dimensional maximum pooling module, and a convolutional linear gating unit connected in sequence. The process is expressed as formula (20): （20）； in: represents the output of this stage, represents the operation of the convolutional linear gated unit, represents the operation of the mask concatenation unit, Represents a two-dimensional maximum pooling operation; After four stages of layer-by-layer processing, the input feature map is converted into a high-level feature representation for the final classification or regression task, as shown in Equation (21): （21）； in: Represents the extracted high-level features, It is the initial feature map after being processed by the data downsampling module; Represents an operation that is processed through four stages; S43: classification and identification; The specific implementation process of S42 is as follows: S421: mask splicing unit; The mask splicing unit captures spatiotemporal features through multi-stage feature fusion and enhances the perception of key areas. Suppose the input feature map is , first adopt The deep convolution layer with a convolution kernel and a stride of 1 extracts basic features, and the intermediate features are obtained after batch normalization. , the adaptive mask mechanism generates a binary mask matrix dynamically right The process of implementing regional masking is shown in formula (22): （22）； in: is a learnable scaling factor, the mask matrix By the initial mask rate Control generation; Features after masking After compression by multi-layer perceptron and the original input Add features to generate enhanced features As shown in formula (23): （23）； in: Represents a multilayer perceptron, which is a feedforward neural network structure used to perform nonlinear transformations on input data; Final module output This is achieved by concatenating the original input and enhanced features along the channel dimension, as shown in Equation (24): （24）； The principle of the adaptive mask generation mechanism is shown in Equations (25) and (26): （25）； （26）； in: Represents the initial mask vector, whose length is equal to the initial resolution , random permutation function Used for The elements in are randomly shuffled, Used to reshape the one-dimensional vector into the target feature map Matrices of the same shape; S422: convolutional linear gated unit; Convolutional linear gated units enhance feature expression capabilities through cross-channel interaction, input features After two independent linear layers and After projection, one of the branches extracts spatial features through 3×3 depth convolution and modulates them element by element with the gate weight of the other branch to generate implicit abstract features. , as shown in formula (27): （27）； in: is the Gaussian error linear unit, represents a 3×3 depthwise convolution with a stride of 1, It is an element-level multiplication. After the second depth convolution and linear transformation, the feature is triple-added with the original input, as shown in formula (28): （28）； in: is a linear layer, is element-wise addition, Represents the output of the convolutional linear gated unit.

2. The identification method according to claim 1, wherein: The specific implementation process of S3 is as follows: S31: Reshaping of human behavior information data; When processing the raw intermediate frequency millimeter wave radar data, in order to facilitate subsequent feature extraction and model training, it is first necessary to rearrange the continuous radar signal into a two-dimensional matrix according to the Chirp. The raw millimeter wave radar data is represented as ,in Indicates the The intermediate frequency signal value of the sampling point, is the total number of sampling points, and each Chirp contains sampling points, and a total of Chirp, the total number of sampling points of the original data meets ; The original data S is first divided into multiple Chirps, each Chirp corresponds to a length of The sampling point sequence is arranged in sequence to form a two-dimensional matrix , whose dimensions are , the two-dimensional matrix is ​​expressed as: （10）； in: Indicates the Chirp's Sampling point values; S32: radar data window segmentation; The two-dimensional matrix generated in S31 On the basis of this, the data is further divided according to the time window, and the window length Defines the number of Chirps contained in each window and the step size Determines the degree of overlap or spacing between adjacent windows. The starting Chirp and ending Chirp of a window can be calculated by equations (11) and (12): （11）； （12）； in: Indicates the segment number, the value range is , Indicates the The starting position of the segment, Indicates the The end position of the segment, and satisfy ; Indicates the fixed length of each window, Indicates the The actual length of the segment; The total number of window fragments finally generated It can be determined by formula (13): （13）； S33: micro-Doppler processing; The window matrix obtained in step S32 , each row represents a Chirp sampling point data. By performing fast Fourier transform on each row, the time domain signal is converted into a frequency domain signal, thereby extracting the target distance information, as shown in formula (14): （14）； in: is an imaginary unit, is the first Row, No. The value of the column, Indicates the in the first Complex values ​​on distance cells, represents the frequency at which the signal is sampled, is the number of sampling points for each Chirp, and also the number of points in FFT; After fast Fourier transform, the distance dimension data contains static clutter. In order to highlight the dynamic target signal and suppress the static clutter, a high-pass Butterworth filter is used to filter the distance dimension data for moving target indication. For discrete time signals, the difference equation of the Butterworth filter is shown in Equation (15): （15）； in: is the input signal, is the output signal, and are the forward and feedback coefficients of the filter respectively; and Indicates the forward order of the filter and the feedback order of the filter; After the moving target indicator filtering, the distance dimension data is shown in formula (16): （16）； in: represents the distance dimension data after Butterworth filtering, represents the Butterworth filtering operation, The original distance dimension data, in the distance unit, The value at the time sample, represents the forward coefficient vector, represents the feedback coefficient vector; The data after moving target indication filtering is subjected to short-time Fourier transform. The core idea of ​​short-time Fourier transform is to divide the signal into multiple short-time windows and perform FFT in each window to obtain the time-frequency distribution of the signal as shown in formula (17): （17）； in: is an imaginary unit, represents the input signal, represents the window function, represents the time-frequency distribution, represents the local window center of the current analysis, Indicates the frequency component currently being analyzed; After short-time Fourier transform processing, the micro-Doppler spectrum The dimension is ,in Indicates the frequency axis resolution, Indicates the time axis resolution; Finally, the micro-Doppler spectrum is expressed by formula (18): （18）； in: Indicates the frequency and time Energy distribution at S34: Visualization of micro-Doppler features; The micro-Doppler map described in S33 is converted into a logarithmic form and visualized using an image drawing tool, and label information is added by naming.

Images