A low-power-consumption millimeter wave radar human behavior recognition method and system

Abstract

The application relates to the technical fields of computer vision, sensor signal processing and artificial intelligence, and specifically provides a low-power millimeter wave radar human behavior recognition method and system, which comprises the following steps: preprocessing point cloud data collected by a millimeter wave radar to obtain a structured feature sequence; extracting multi-level manual features from the structured feature sequence, and performing feature selection and normalization to generate an optimized feature vector; inputting the optimized feature vector into a heterogeneous integrated classifier, combining a dynamic pseudo-label strategy to perform semi-supervised training on the heterogeneous integrated classifier, and obtaining an enhanced recognition model; and using the enhanced recognition model to perform human activity classification and recognition on millimeter wave radar data to be recognized. The application significantly improves the recognition accuracy and robustness of behavior type recognition from millimeter wave radar data.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision, sensor signal processing and artificial intelligence technology, and specifically relates to a low-power millimeter-wave radar human behavior recognition method and system. Background Technology

[0002] Human Activity Recognition (HAR) is a core enabling technology for fields such as intelligent health monitoring, natural human-computer interaction, and security systems. Traditional HAR solutions mainly rely on wearable sensors or visual cameras, but the former is limited by user compliance and comfort, while the latter is easily constrained by changes in lighting, occlusion, and privacy protection requirements.

[0003] In recent years, millimeter-wave radar has shown significant application potential in indoor intelligent sensing (such as smart homes, medical monitoring, and elderly care) due to its advantages such as non-contact sensing, strong privacy protection, insensitivity to light, and certain penetration capabilities. However, HAR based on millimeter-wave radar still faces several key challenges in practical deployment: Scarcity of labeled data: Manual labeling of radar point cloud data is costly and time-consuming, making it difficult to obtain large-scale, high-quality labeled datasets; Noise and domain shift interference: Radar signals are susceptible to multipath effects, environmental clutter, and equipment noise, and the signal feature distribution varies in different scenarios (such as living rooms, bedrooms, and corridors) (domain shift), leading to a decrease in model generalization ability; Computational resource limitations: Current mainstream methods are mostly based on deep learning models (such as the unsupervised domain adaptation method CLUDA based on contrastive learning, the TD-CNN-LSTM model which combines temporal convolutional and long short-term memory networks, the PointNet++ point cloud processing network, and the MAK-GCN adaptive kernel graph convolutional network). These methods typically rely on GPUs for training and inference, resulting in high computational overhead and making efficient deployment on resource-constrained edge devices.

[0004] While deep learning has significantly improved HAR performance, its reliance on large-scale labeled data and high computational cost have limited its application in edge environments. Meanwhile, traditional machine learning methods based on signal processing and statistical analysis, while offering advantages in interpretability and computational efficiency, are often overlooked in current radar HAR research, especially in scenarios with scarce labels and cross-domain applications, where their potential remains largely untapped.

[0005] In summary, existing millimeter-wave radar HAR solutions have not yet achieved a good balance between accuracy, robustness, computational efficiency, and dependence on labeled data. Therefore, there is an urgent need for a lightweight semi-supervised identification solution that balances recognition performance and deployment efficiency, effectively utilizes limited labeled data, and adapts to complex indoor environments. Summary of the Invention

[0006] In view of the above-mentioned shortcomings of the prior art, the present invention provides a low-power millimeter-wave radar human behavior recognition method and system to solve the above-mentioned technical problems.

[0007] In a first aspect, the present invention provides a low-power millimeter-wave radar method for human behavior recognition, comprising:

[0008] The point cloud data acquired by millimeter-wave radar is preprocessed to obtain a structured feature sequence; Multi-level handcrafted features are extracted from the structured feature sequence, and feature selection and normalization are performed to generate an optimized feature vector. The optimized feature vector is input into the heterogeneous ensemble classifier, and the heterogeneous ensemble classifier is semi-supervised by the dynamic pseudo-label strategy to obtain the enhanced recognition model. The enhanced recognition model is used to classify and identify human activities in the millimeter-wave radar data to be identified.

[0009] In one optional implementation, the point cloud data acquired by the millimeter-wave radar is preprocessed, including: The second-level and nanosecond-level timestamps in the radar data are merged to generate a unified timestamp. Calculate the instantaneous velocity of the target point based on the change in its position on the plane at adjacent time points; and calculate the instantaneous acceleration of the target point based on the rate of change of its instantaneous velocity at adjacent time points. The mean and standard deviation of velocity and azimuth angle of all point clouds are calculated respectively, and outliers with velocity or azimuth angle deviating from the mean by more than twice the standard deviation are removed; the DBSCAN clustering algorithm with adaptive neighborhood radius is used to filter out the sparse noise of the point cloud after removing outliers, and the denoised point cloud is obtained. The radar detection plane area is uniformly divided into a grid of 10 rows and 8 columns. For each grid cell, based on the denoised point cloud data falling into the cell at the current moment, eight statistical features are calculated, including: point cloud number density, mean and variance of velocity, mean and variance of acceleration, and mean and variance of azimuth sine and cosine values. The statistical features of the grid cells extracted at each time point are arranged in chronological order to form multidimensional time-series feature data. The multidimensional time-series feature data is segmented using a sliding window of fixed duration, wherein the window length is set to 5 seconds and there is a 50% overlap between adjacent windows, in order to generate a structured feature sequence for model training and recognition.

[0010] In one optional implementation, the second-level timestamps and nanosecond-level timestamps in the radar data are merged to generate a unified timestamp, including: Multiply the value of the nanosecond-level timestamp by 10 to the power of negative 9 to convert it into a decimal part in seconds; Then, the decimal part is added to the value of the second-level timestamp to synthesize a unified timestamp.

[0011] In an optional implementation, the DBSCAN clustering algorithm with adaptive neighborhood radius is used to filter out sparse noise in the point cloud after outlier removal, including: For the point cloud data obtained after outlier filtering, calculate the Euclidean distance between each point and its third nearest neighbor. The 95th percentile value is determined based on the third nearest neighbor distance of all points. Set the neighborhood radius parameter of the DBSCAN clustering algorithm to the larger of the 95th percentile value and a preset minimum radius value; The DBSCAN clustering algorithm is run to retain only the point cloud data that is determined to belong to the core points, thereby completing the filtering of sparse noise.

[0012] In an optional implementation, multi-level handcrafted features are extracted from the structured feature sequence, including: Time-domain statistical features, frequency-domain features, multi-order time-series difference features, multi-hysteresis autocorrelation coefficients, and cross-channel correlation coefficients are extracted from the structured feature sequence, and the extracted features are fused into a high-dimensional feature vector. The time-domain statistical features include mean, standard deviation, skewness, and kurtosis. The frequency domain features include the spectral centroid, spectral flatness, and energy distribution in multiple preset frequency bands, calculated based on the fast Fourier transform. The cross-channel correlation coefficient refers to the coefficient obtained by performing pairwise correlation calculations on different data channels representing position, velocity, and orientation in the sequence.

[0013] In one optional implementation, feature selection and normalization are performed to generate an optimized feature vector, including: Based on the mutual information criterion, the correlation between each feature dimension in the high-dimensional feature vector and the human activity category label is calculated, and the key feature subset is selected according to the correlation. The key feature subset is subjected to power transform normalization to generate the optimized feature vector.

[0014] In an optional implementation, the method for constructing the heterogeneous ensemble classifier includes: The model integrates multiple different types of base classifiers; the multiple different types of base classifiers include at least two or more of the following models: decision tree ensemble model, support vector machine model, multilayer perceptron neural network model, and gradient boosting ensemble model.

[0015] In an optional implementation, the optimized feature vector is input into a heterogeneous ensemble classifier, and the heterogeneous ensemble classifier is semi-supervised trained using a dynamic pseudo-label strategy, including: Based on the optimized feature vectors corresponding to the labeled point cloud data, the heterogeneous ensemble classifier is initially trained. Using the pre-trained classifier, the optimized feature vectors corresponding to the unlabeled point cloud data are predicted, and a corresponding confidence score is generated for each predicted category. For each action category, an initial confidence threshold and a minimum threshold are set, and the confidence threshold is applied to all predictions for that action category. From the prediction results corresponding to the action category, the top 50% of the prediction results in terms of confidence are selected. The point cloud data and the corresponding prediction category in the prediction results that simultaneously meet the current confidence threshold and the top 50% condition are added to the training set as pseudo-label data. The heterogeneous ensemble classifier is retrained using the expanded training set, and the confidence threshold is reduced by a preset step size after each round of training iterations until the confidence threshold drops to the minimum threshold or a preset iteration termination condition is met.

[0016] In an optional implementation, the enhanced recognition model is used to classify and identify human activities in the millimeter-wave radar data to be identified, including: For the raw point cloud data of the millimeter-wave radar to be identified, preprocessing operations and feature extraction and optimization operations are performed to generate the corresponding optimized feature vector to be identified; The optimized feature vector to be identified is input into the enhanced recognition model; The enhanced recognition model outputs the classification result generated by the optimized feature vector to be identified, and the classification result includes at least the predicted human activity category and its corresponding confidence level. The categories of human activities include boxing, sitting, standing, walking, and waving.

[0017] Secondly, the present invention provides a low-power millimeter-wave radar human behavior recognition system, comprising: The preprocessing module is used to preprocess the point cloud data acquired by the millimeter-wave radar to obtain a structured feature sequence; The feature extraction module is used to extract multi-level handcrafted features from the structured feature sequence, and to perform feature selection and normalization to generate an optimized feature vector. The model training module is used to input the optimized feature vector into the heterogeneous ensemble classifier and perform semi-supervised training on the heterogeneous ensemble classifier by combining a dynamic pseudo-label strategy to obtain an enhanced recognition model. The model application module is used to classify and identify human activities using the enhanced recognition model on the millimeter-wave radar data to be identified.

[0018] The low-power millimeter-wave radar human behavior recognition method and system provided by this invention have the following beneficial effects: Significantly improved recognition accuracy and robustness: Through meticulously designed multi-level handcrafted feature engineering (fusing temporal, frequency, time-series, and cross-channel related features) and a two-level adaptive noise filtering mechanism, this method effectively extracts essential motion features that are insensitive to noise and scene changes. Combining soft-voting fusion with a heterogeneous ensemble classifier and a dynamic pseudo-label strategy, it fully utilizes information from unlabeled data, significantly enhancing the model's generalization ability in complex indoor environments and cross-domain scenarios. On a custom dataset containing multiple scenes and actions, this method achieves an accuracy of 96.40% and a macro-average F1 score of 0.9640, demonstrating a significant performance improvement over existing mainstream deep learning methods (such as CLUDA, TD-CNN-LSTM, etc.).

[0019] This method achieves highly efficient edge deployment capabilities: The entire process is built upon classic machine learning algorithms, requiring no GPU acceleration and running efficiently on CPU-only resources. Compared to computationally intensive deep neural network models, this method significantly reduces the number of model parameters and computational complexity (FLOPs), solving the key problem of deployment difficulties on computationally limited edge devices (such as embedded monitoring terminals, home gateways, and mobile robots). It provides a feasible lightweight solution for the widespread application of millimeter-wave radar HAR technology.

[0020] A remarkable balance is achieved between performance and efficiency: This invention abandons the purely data-driven deep learning paradigm and innovatively combines systematic signal processing, statistical feature engineering, and efficient ensemble learning. This design enables the model to achieve high recognition performance while maintaining extremely low training and inference overhead, successfully breaking the traditional trade-off between accuracy and efficiency. It is particularly suitable for IoT and edge computing scenarios that are sensitive to real-time performance, power consumption, and cost.

[0021] The positive synergistic effect between components is demonstrated: through systematic ablation experiments, a strong positive synergy exists between the preprocessing module, multi-dimensional feature engineering, heterogeneous integration framework, and semi-supervised learning mechanism of this invention. Removal of any core component (such as frequency domain features, cross-channel correlation, or dynamic pseudo-label modules) leads to a significant decrease in model performance. This proves that this invention is not a simple stacking of multiple independent technologies, but an organically integrated and mutually reinforcing systematic innovation, whose overall effect is superior to the independent application of its components. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic flowchart of a method according to an embodiment of the present invention.

[0024] Figure 2 This is a schematic diagram of the preprocessing flow of a method according to an embodiment of the present invention.

[0025] Figure 3 This is a schematic diagram of the feature extraction process of a method according to an embodiment of the present invention.

[0026] Figure 4 This is a schematic block diagram of a system according to an embodiment of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0029] The method provided in this embodiment of the invention is executed by a computer device, and correspondingly, the low-power millimeter-wave radar human behavior recognition system runs in the computer device.

[0030] Figure 1 This is a schematic flowchart illustrating a method according to an embodiment of the present invention. Wherein, Figure 1 The implementing entity can be a low-power millimeter-wave radar human behavior recognition system. Depending on different requirements, the order of the steps in this flowchart can be changed, and some can be omitted.

[0031] like Figure 1 As shown, the method includes: S1. Preprocess the point cloud data acquired by the millimeter-wave radar to obtain a structured feature sequence; S2. Extract multi-level handcrafted features from the structured feature sequence, and perform feature selection and normalization to generate an optimized feature vector; S3. Input the optimized feature vector into the heterogeneous ensemble classifier, and perform semi-supervised training on the heterogeneous ensemble classifier using a dynamic pseudo-label strategy to obtain an enhanced recognition model. S4. Use the enhanced recognition model to classify and identify human activities in the millimeter-wave radar data to be identified.

[0032] In one embodiment of the present invention, based on step S1, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0033] The input data consists of raw point cloud flows continuously acquired by millimeter-wave radar. From a practical application perspective, this data comprises two types during the model development and training phases: Labeled point cloud data refers to radar data segments that have been manually or reliably automatically labeled with corresponding human activity categories (such as "walking," "sitting," etc.). This type of data is limited in quantity and forms the basis for supervised learning.

[0034] Unlabeled point cloud data refers to a large number of radar data segments that have not been categorized. Its acquisition cost is far lower than labeled data, and it contains rich pattern information, making it a key resource for this method to achieve semi-supervised learning and improve the model's generalization ability.

[0035] Please refer to Figure 2 Preprocessing methods for arbitrary point cloud data include: S101. Merge the second-level and nanosecond-level timestamps in the radar data to generate a unified timestamp; including: The nanosecond-level timestamp value is multiplied by 10 to the power of -9 to convert it into a decimal part in seconds; then, the decimal part is added to the second-level timestamp value to synthesize a unified timestamp. The specific calculation formula is as follows: .

[0036] S102. Calculate the instantaneous velocity of the target point based on the change in its position on the plane at adjacent time points; and calculate the instantaneous acceleration of the target point based on the rate of change of its instantaneous velocity at adjacent time points.

[0037] Planar velocity ; acceleration .

[0038] in, Let be the x-coordinate of the target point at time i. Let be the ordinate of the target point at time i.

[0039] S103. Calculate the mean and standard deviation of velocity and azimuth for all point clouds respectively, and remove outliers whose velocity or azimuth deviates from the mean by more than twice the standard deviation; use the DBSCAN clustering algorithm with adaptive neighborhood radius to filter out the sparse noise of the point cloud after removing outliers, and obtain the denoised point cloud.

[0040] Raw radar point clouds often contain outliers and sparse noise caused by multipath effects, environmental clutter, and equipment noise. This step employs a two-stage filtering strategy to clean the data: Level 1: Outlier removal based on statistics. Calculate the velocity and azimuth of all point clouds. mean ( , ) and standard deviation ( , Subsequently, those velocity values ​​that satisfy... Or the azimuth value satisfies This step can quickly remove noise points that are significantly off-center from the main distribution.

[0041] Second stage: Sparse noise filtering based on adaptive DBSCAN.

[0042] For the point cloud dataset P = {p1, p2, ..., p...} after the first-level filtering, n}, calculate each point p i Euclidean distance to its third nearest neighbor .Right now:

[0043] Choosing the third nearest neighbor distance (instead of the first nearest neighbor) as a local density indicator can more stably reflect the neighborhood density of a point and avoid estimation bias caused by individual extremely nearest neighbor points.

[0044] Collect the third nearest neighbor distances of all points to form a set. To ensure that the clustering algorithm can capture the main, normally-density clusters of points in the data, while avoiding setting excessively large radii due to a few points with abnormally large distances, this invention selects... The 95th percentile was used as the baseline for setting the radius. .

[0045]

[0046] This operation means that the algorithm will use the local density level covering 95% of the data points as the primary clustering scale. Ultimately, the neighborhood radius parameter of the DBSCAN algorithm... Determine by the following formula:

[0047] in, This is a preset minimum radius value (e.g., 0.01 meters). This operation ensures that even in extremely dense point clouds (leading to...), the minimum radius value is maintained. In extreme cases (where the calculated value is too small), the cluster radius will not fall below a physically meaningful threshold, ensuring the basic effectiveness of the algorithm.

[0048] Another core parameter of the DBSCAN algorithm is the minimum number of points required to form the core points. Based on experience, this value is usually set to twice the data dimension (in this example, the spatial coordinate dimension of the point, which is 2 or 3). A typical implementation of this invention sets this value accordingly. .

[0049] The process of running the DBSCAN algorithm is as follows: For each point, the statistics are as follows: The number of points in the neighborhood of the radius.

[0050] If a certain point -The neighborhood contains at least If a point (including itself) is a core point, then that point is marked as a core point.

[0051] All points that originate from the core point and are connected by density-reachable relationships are grouped into the same cluster.

[0052] Points that do not belong to any cluster are marked as noise points.

[0053] In this invention, only point cloud data identified as core points are retained. All noise points and boundary points that belong to a cluster but are not core points themselves are filtered out. This is because sparse noise and many unstable spurious points generated by multipath effects usually cannot form a core region that meets the density requirements, and are therefore identified and removed.

[0054] S104. Divide the radar detection plane area into a grid of 10 rows and 8 columns. For each grid cell, calculate eight statistical features based on the denoised point cloud data falling into the cell at the current moment, including: point cloud number density, mean and variance of velocity, mean and variance of acceleration, and mean and variance of azimuth sine and cosine values.

[0055] To transform the spatiotemporal information of the point cloud into a structured representation, the radar's detection plane region (e.g., a forward fan-shaped region) is uniformly divided into a grid of 10 rows (range dimension) and 8 columns (azimuth dimension). For each frame (current time) of the denoised point cloud, each grid cell is traversed, and eight statistical features are calculated based on all points falling within that cell: Point cloud number density:

[0056] ρ represents density, and in practice, the number of point clouds NN within this grid cell is directly used as its density metric. It reflects the salience of the target in this spatial region.

[0057] Mean and variance of velocity:

[0058]

[0059] in, Let k be the instantaneous velocity of the k-th point in the grid content. It is the arithmetic mean of the velocities of all points within the grid, describing the average speed and fluctuations of the target's motion within that region.

[0060] Mean and variance of acceleration:

[0061]

[0062] Describe the average severity and fluctuations of the changes in the motion state of the target within this region.

[0063] Mean sine of azimuth: ; Azimuth sine variance: ; Mean cosine of azimuth: ; Azimuth cosine variance: ; in, It is the azimuth angle (in radians) of the k-th point in the grid relative to the radar.

[0064] The average orientation and its dispersion of the target are encoded in a statistical form, thus avoiding the problem of angular periodicity.

[0065] Through this operation, each frame of point cloud is encoded into a spatial feature map of a fixed dimension (10×8×8=640 dimensions), realizing a quantitative description of the spatial distribution and motion state of the target.

[0066] S105. Arrange the statistical features of the grid cells extracted at each time point in chronological order to form multidimensional time-series feature data; use a sliding window of fixed duration to segment the multidimensional time-series feature data, wherein the window length is set to 5 seconds and there is a 50% overlap between adjacent windows, in order to generate a structured feature sequence for model training and recognition.

[0067] The grid features output from each frame in step S104 are arranged in chronological order according to their timestamps to form a multidimensional temporal feature data stream. To generate samples suitable for model input and increase data diversity, a sliding window method is used to segment this temporal stream. The window length is set to 5 seconds to cover the duration of a complete action; the overlap rate between adjacent windows is set to 50% (i.e., a step size of 2.5 seconds). This operation not only generates a large number of training samples, achieving data augmentation, but also ensures the smoothness of the model during continuous recognition. Finally, a series of fixed-length structured feature sequences are output.

[0068] In one embodiment of the present invention, based on step S2, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0069] S201. Extract multi-level handcrafted features from the structured feature sequence. Please refer to [reference needed]. Figure 3 ,include: A basic statistical description is performed on the time series value of each feature channel (such as the velocity mean channel of a certain grid) in the structured feature sequence.

[0070] Feature extraction: Calculate the mean, standard deviation, skewness, and kurtosis of the time series value. The mean reflects the average intensity of the feature within the time window. The standard deviation reflects the degree of fluctuation of the feature value around the mean, characterizing the stability of the movement. Skewness measures the asymmetry of the data distribution. For example, a movement that is predominantly slow with occasional rapid bursts may have a positive skewness in its speed distribution. Kurtosis measures the sharpness of the data distribution, i.e., the thickness of the tails. High kurtosis means that the feature values ​​are more concentrated around the mean, or have thicker tails than a normal distribution, which may correspond to consistency in movement patterns or regular sudden changes.

[0071] For each feature channel generated in step 1 (a total of 640-dimensional grid features), these four statistics are calculated independently to generate 640×4=2560-dimensional time-domain statistical features.

[0072] The time-domain feature sequence is transformed to the frequency domain using a Fast Fourier Transform (FFT) to capture the periodicity, rhythm, and spectral energy distribution information of the activity. Specifically, the spectral centroid, spectral flatness, and energy distribution of multiple preset frequency bands are calculated. The spectral centroid is the "center of gravity" frequency of the spectrum, reflecting the concentrated location of energy within the spectrum. High-frequency centroids typically correspond to rapid, abrupt movements. Spectral flatness describes the flatness of the spectrum. A value close to 1 indicates a flat spectrum similar to white noise; smaller values ​​indicate that energy is more concentrated at certain frequencies, usually implying regular, periodic movement (such as the gait frequency of walking). Multi-band energy distribution is used to divide the entire spectrum (e.g., 0-5Hz) into multiple sub-bands (e.g., 8), calculating the energy percentage of each sub-band. This allows for a fine-grained characterization of the energy distribution patterns of different activities within a specific frequency range.

[0073] Perform FFT on the time series of each feature channel and calculate the above features based on the power spectrum.

[0074] Extracting multi-order time series differences and autocorrelation features aims to capture the dynamic change patterns and inherent periodicity / correlation of time series.

[0075] Multi-order temporal differencing: Calculates statistics (such as mean and variance) of the original sequence and its first-order, second-order (or even higher-order) differencing sequences. Difference operations highlight the rate of change of the sequence, while higher-order differencing further captures changes in acceleration, which is particularly important for describing the beginning, end, and transition phases of an action.

[0076] Multi-lag autocorrelation coefficient: Calculates the autocorrelation coefficient of the original sequence at different time lags (e.g., lag of 1 frame, 2 frames...k frames). The autocorrelation coefficient reveals the similarity of the sequence itself at different time intervals and is a key indicator for identifying periodicity and repetitive patterns of movement (e.g., waving, walking).

[0077] Extracting cross-channel correlation coefficients: Human activity typically manifests as coordinated spatial changes in multiple physical quantities (such as position, velocity, and orientation). This step aims to quantify these coordinated relationships. Specifically, Pearson correlation coefficients are calculated between different data channels representing position (x, y), velocity (v), and azimuth (represented by sinθ and cosθ). For example, calculating the correlation coefficient between the x-coordinate channel and the velocity channel can reveal the association between "lateral movement" and "velocity"; calculating the correlation coefficient between velocity channels across different grids can reveal the coordination of movements in different parts of the body. These correlation coefficients can capture complex spatial motion patterns.

[0078] All features extracted from the four levels of time domain, frequency domain, time series dynamics, and channel correlation (time domain statistical features, frequency domain features, time series difference and autocorrelation features, and cross-channel correlation coefficients) are concatenated to form a comprehensive high-dimensional feature vector.

[0079] S202. Perform feature selection and normalization to generate optimized feature vectors, including: Using mutual information as a metric, we calculate each individual feature dimension in the high-dimensional feature vector. Statistical dependency between the label Y and the category of human activity.

[0080] Mutual information formula (characterizing the nonlinear correlation between two random variables):

[0081] The higher the mutual information value, the better the feature. Includes about categories The more information a system has, the stronger its discriminative ability. Compared to correlation coefficients, which can only measure linear relationships, mutual information can capture more complex nonlinear relationships and is better suited for handling the complex mappings that may exist between motion features and activity categories.

[0082] Calculate the mutual information values ​​between all feature dimensions and labels, and sort them in descending order of value. Based on a preset threshold (e.g., the top-K features) or the optimal number determined by validation set performance, select a subset of key features. This step significantly reduces feature dimensionality, removes noise and redundancy, and improves the model's generalization ability and computational efficiency.

[0083] The selected subset of key features is normalized using PowerTransformer (specifically Yeo-Johnson transform).

[0084] The Yeo-Johnson transform is a parameterized power transform that automatically handles data containing zero and negative values. Its form is:

[0085] in The parameters are obtained through maximum likelihood estimation optimization, with the aim of making the transformed data as close as possible to a normal distribution. X is the original value of any feature dimension (i.e., a specific statistical feature) in any training sample from the key feature subset after feature selection.

[0086] Many machine learning algorithms (such as SVM and linear models) are sensitive to the scale and distribution of features. PowerTransformer uses a data-driven approach to find the optimal transformation parameters. It not only performs scaling, but more importantly, it stabilizes variance and reduces data skewness, making it closer to a Gaussian distribution.

[0087] Compared to simple Min-Max or Z-Score normalization, power transformation can better handle features with heavy-tailed or skewed distributions, thus more effectively satisfying the potential assumptions of subsequent classifiers about the distribution of input data and further improving model performance.

[0088] In one embodiment of the present invention, based on step S3, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0089] In this embodiment, a combination of multiple models from the following four categories is preferred, but not limited to: Decision tree ensemble models, such as Random Forest and ExtraTrees, excel at capturing nonlinear relationships and interactions between features by constructing multiple decision trees and integrating their results. They are insensitive to data scale and can provide feature importance assessments. Random Forest increases diversity through bootstrap sampling and random feature selection; ExtraTrees employ a more randomized segmentation strategy. Although they share a common origin, they have different bias-variance characteristics.

[0090] Support Vector Machine (SVM) models include linear SVM and SVM with radial basis function kernel (RBF-SVM). The core idea of ​​SVM is to find a maximum margin hyperplane to separate different classes, making it particularly adept at handling classification problems in high-dimensional feature spaces. Linear SVM is suitable for approximately linearly separable data and is computationally efficient; RBF-SVM, on the other hand, can handle complex nonlinear decision boundaries through kernel tricks. The decision-making mechanism of SVM is fundamentally different from tree-based methods.

[0091] Multilayer Perceptron Neural Network Model: As a simple fully connected feedforward neural network, the MLP can learn complex hierarchical combinations and abstract representations of features through multiple nonlinear hidden layers. Its gradient descent-based optimization method and ability to capture distributed representations provide a modeling paradigm completely different from the aforementioned statistical learning models.

[0092] Gradient boosting ensemble models, such as XGBoost and LightGBM, construct decision trees sequentially, with each new tree working to correct the residuals of the previous tree. They typically achieve high prediction accuracy and control overfitting through efficient algorithms and regularization terms. While XGBoost and LightGBM differ in their growth strategies and engineering optimizations, they both belong to the gradient boosting framework.

[0093] The principle of heterogeneity: By combining, for example, "Random Forest (Bagging Decision Tree) + SVM (Maximum Margin Classification) + MLP (Neural Network) + LightGBM (Boosting Decision Tree)," this ensemble system simultaneously encompasses four significantly different learning paradigms: Bagging ensemble, maximum margin theory, neural network representation learning, and Boosting ensemble. This diversity ensures that when one model performs poorly due to data characteristics (such as noise type or distribution bias), other models based on different principles may still remain stable, thus achieving a more reliable collective decision through ensemble integration.

[0094] The optimized feature vector is input into the heterogeneous ensemble classifier, and the heterogeneous ensemble classifier is semi-supervised trained using a dynamic pseudo-label strategy, including: 1. Preliminary model training First, the constructed heterogeneous ensemble classifier is initially trained using existing, limited amounts of labeled data (corresponding optimized feature vectors and true class labels).

[0095] Input: Optimized feature vectors of labeled data and corresponding human activity category labels (such as "walking", "sitting", etc.).

[0096] Training process: Each base classifier in the heterogeneous ensemble classifier (such as random forest, SVM, etc.) is trained independently, taking into account the inverse class frequency weighting (to alleviate class imbalance). Subsequently, the macro-average F1 score of each base classifier is evaluated by 5-fold cross-validation, and the top 5 models with the best performance are selected to form the initial ensemble model.

[0097] Frequency reciprocal weighting:

[0098] The formula for calculating the F1 score is: .

[0099] 2. Pseudo-tag generation and dynamic filtering mechanism After the initial model training is complete, the core iterative pseudo-label learning loop begins. In each iteration, the following operations are performed: a. Prediction and Confidence Acquisition Using the heterogeneous ensemble classifier in the current iteration, predict the optimized feature vectors for all unlabeled data. For each unlabeled sample, the model outputs a probability vector, where each element represents the prediction confidence (probability value, ranging from 0 to 1) that the sample belongs to a specific activity category.

[0100] b. Class-adaptive dynamic threshold filtering This invention maintains an independent, dynamically decaying confidence threshold for each action category c. , where t represents the current iteration round.

[0101] Threshold initialization: Initial threshold for each category Set it to a high conservative value (e.g., 0.85) to ensure that only highly confident pseudo-labels are introduced in the initial stage.

[0102] Threshold decay rule: After each iteration, the threshold decays by a preset fixed step size α (e.g., 0.02), i.e. .in, It is a preset minimum threshold (e.g., 0.70) to prevent the threshold from being too low and introducing too many noise labels.

[0103] In-category filtering: For category c, from all unlabeled samples predicted as belonging to this category, filter out those samples whose prediction confidence simultaneously meets the following two conditions: The confidence level is not lower than the current dynamic threshold for that category. .

[0104] The confidence level is among the top 50% of all samples predicted to be in that category (i.e., the highest confidence level of 50% for class independence screening).

[0105] c. Pseudo-label data generation and training set augmentation Samples that meet the above strict screening criteria are then used to form new training data pairs with their optimized feature vectors and the class predicted by the model (i.e., "pseudo-labels"), and added to the existing training set. This step aims to introduce only high-quality, high-confidence pseudo-labeled samples to safely augment the training data.

[0106] 3. Model Iteration, Retraining, and Termination Conditions Model Update: The heterogeneous ensemble classifier is retrained using the expanded training set (containing the original labeled data and the newly added pseudo-label data). During training, the class weights for the pseudo-label data are calculated in the same way as for the real labeled data (based on the inverse of class frequency), but all pseudo-label data are treated as a single "new" batch for weighted calculation.

[0107] Iterative loop: Repeat steps 2 (pseudo-label generation and filtering) and 3 (model update) to form a closed loop of "prediction-filtering-training".

[0108] Iteration Termination: The semi-supervised training process terminates when any of the following conditions are met: Lower threshold: The dynamic confidence thresholds for all categories have decayed to the preset minimum value. .

[0109] No new samples added: In two consecutive iterations, no new unlabeled samples met the selection criteria and were added to the training set.

[0110] Maximum number of rounds: Reach the preset maximum number of iteration rounds (e.g., 12 rounds).

[0111] Unlabeled sample depletion: The number of remaining unlabeled samples is below a very small threshold (e.g., less than 10).

[0112] In one embodiment of the present invention, based on step S4, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0113] When the enhanced recognition model is deployed in real-world applications and needs to identify new, unknown millimeter-wave radar raw point cloud data, it executes an end-to-end automated process strictly aligned with the training workflow. The entire process begins with the real-time capture and buffering of the input raw point cloud stream. First, a complete set of preprocessing operations is performed on the raw data, which is completely consistent with the workflow and parameters defined during the model training phase to ensure consistency in the feature space. Preprocessing includes merging second-level and nanosecond-level timestamps from radar data to generate a high-precision unified timestamp; then calculating instantaneous velocity and acceleration based on the changes in the planar coordinates of the target point; subsequently, a two-stage noise reduction mechanism, based on statistical filtering using a double standard deviation criterion and clustering filtering using adaptive neighborhood radius DBSCAN, effectively removes outliers and sparse noise to obtain clean point cloud data; next, the detection plane is divided into a fixed grid of 10 rows and 8 columns, and eight statistical features are calculated for each grid cell, including point cloud density, mean and variance of velocity, mean and variance of acceleration, and mean and variance of azimuth sine and cosine, thereby encoding each frame of point cloud into a structured spatial feature map; finally, the temporally arranged feature map sequence is segmented using a sliding window with a length of 5 seconds and an overlap rate of 50%, generating a structured feature sequence to be identified.

[0114] Next, feature extraction and optimization operations are performed on the sequence. This process also replicates the feature engineering pipeline of the training phase: extracting time-domain statistical features, frequency-domain features, multi-order time-series differences and autocorrelation coefficients, and cross-channel correlation coefficients from the sequence, and fusing them into a high-dimensional feature vector; then, applying the feature selector (a subset index of key features selected based on mutual information criteria) and normalizer (a PowerTransformer with fitted parameters) that were pre-learned and saved during the training phase, performing the same dimensionality reduction and distribution transformation on the high-dimensional vector, and finally generating an optimized feature vector to be identified that strictly corresponds to the feature space of the training data.

[0115] Subsequently, the optimized feature vector to be identified is input into the trained and solidified augmented recognition model. This model is an ensemble of multiple heterogeneous classical machine learning classifiers (such as random forests, support vector machines, gradient boosting models, etc.) through a soft voting mechanism. After receiving the input, the individual base classifiers within the model operate in parallel, each outputting a probability distribution vector for five predetermined human activity categories—boxing, sitting, standing, walking, and waving. The ensemble module fuses these probability vectors and calculates the average confidence score for each category to arrive at the final collective decision.

[0116] Finally, the structured results of this identification are output. These results contain at least two core pieces of information: first, the predicted category of human activity, selected from the five categories mentioned above and possessing the highest average confidence level; and second, the confidence level corresponding to this prediction, i.e., the average probability value obtained for that category, which quantifies the model's degree of certainty regarding its current judgment. This output is completed with extremely low latency, and the entire process is executed efficiently on the CPU, thus achieving real-time, accurate, and reliable human activity identification of millimeter-wave radar signals on resource-constrained edge devices.

[0117] In some embodiments, the system may include multiple functional modules composed of computer program segments. The computer programs for each program segment in the low-power millimeter-wave radar human behavior recognition system may be stored in the memory of a computer device and executed by at least one processor to perform (see details). Figure 1 (Description) Functionality of millimeter-wave radar behavior recognition based on a lightweight semi-supervised pipeline.

[0118] In this embodiment, the low-power millimeter-wave radar human behavior recognition system can be divided into multiple functional modules according to its functions, such as... Figure 4 As shown. The module referred to in this invention is a series of computer program segments that can be executed by at least one processor and perform a fixed function, and is stored in memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0119] The preprocessing module is used to preprocess the point cloud data acquired by the millimeter-wave radar to obtain a structured feature sequence; The feature extraction module is used to extract multi-level handcrafted features from the structured feature sequence, and to perform feature selection and normalization to generate an optimized feature vector. The model training module is used to input the optimized feature vector into the heterogeneous ensemble classifier and perform semi-supervised training on the heterogeneous ensemble classifier by combining a dynamic pseudo-label strategy to obtain an enhanced recognition model. The model application module is used to classify and identify human activities using the enhanced recognition model on the millimeter-wave radar data to be identified.

[0120] The same or similar parts between the various embodiments in this specification can be referred to mutually. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0121] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or modules may be electrical, mechanical, or other forms.

[0122] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0123] In addition, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0124] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the present invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the present invention by those skilled in the art without departing from the spirit and essence of the invention, and such modifications or substitutions should all be within the scope of the present invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should also be covered within the protection scope of the present invention.

Claims

1. A low-power millimeter-wave radar method for human behavior recognition, characterized in that, include: The point cloud data acquired by millimeter-wave radar is preprocessed to obtain a structured feature sequence; Multi-level handcrafted features are extracted from the structured feature sequence, and feature selection and normalization are performed to generate an optimized feature vector. The optimized feature vector is input into the heterogeneous ensemble classifier, and the heterogeneous ensemble classifier is semi-supervised by the dynamic pseudo-label strategy to obtain the enhanced recognition model. The enhanced recognition model is used to classify and identify human activities in the millimeter-wave radar data to be identified.

2. The method according to claim 1, characterized in that, Preprocessing of point cloud data acquired by millimeter-wave radar includes: The second-level and nanosecond-level timestamps in the radar data are merged to generate a unified timestamp. Calculate the instantaneous velocity of the target point based on the change in its position on the plane at adjacent time points; and calculate the instantaneous acceleration of the target point based on the rate of change of its instantaneous velocity at adjacent time points. The mean and standard deviation of velocity and azimuth angle of all point clouds are calculated respectively, and outliers with velocity or azimuth angle deviating from the mean by more than twice the standard deviation are removed; the DBSCAN clustering algorithm with adaptive neighborhood radius is used to filter out the sparse noise of the point cloud after removing outliers, and the denoised point cloud is obtained. The radar detection plane area is uniformly divided into a grid of 10 rows and 8 columns. For each grid cell, based on the denoised point cloud data falling into the cell at the current moment, eight statistical features are calculated, including: point cloud number density, mean and variance of velocity, mean and variance of acceleration, and mean and variance of azimuth sine and cosine values. The statistical features of the grid cells extracted at each time point are arranged in chronological order to form multidimensional time-series feature data. The multidimensional time-series feature data is segmented using a sliding window of fixed duration, wherein the window length is set to 5 seconds and there is a 50% overlap between adjacent windows, in order to generate a structured feature sequence for model training and recognition.

3. The method according to claim 2, characterized in that, The second-level and nanosecond-level timestamps in the radar data are merged to generate a unified timestamp, including: Multiply the value of the nanosecond-level timestamp by 10 to the power of negative 9 to convert it into a decimal part in seconds; Then, the decimal part is added to the value of the second-level timestamp to synthesize a unified timestamp.

4. The method according to claim 2, characterized in that, The DBSCAN clustering algorithm with adaptive neighborhood radius is used to filter out sparse noise in the point cloud after outlier removal, including: For the point cloud data obtained after outlier filtering, calculate the Euclidean distance between each point and its third nearest neighbor. The 95th percentile value is determined based on the third nearest neighbor distance of all points. Set the neighborhood radius parameter of the DBSCAN clustering algorithm to the larger of the 95th percentile value and a preset minimum radius value; The DBSCAN clustering algorithm is run to retain only the point cloud data that is determined to belong to the core points, thereby completing the filtering of sparse noise.

5. The method according to claim 1, characterized in that, Extracting multi-level handcrafted features from the structured feature sequence includes: Time-domain statistical features, frequency-domain features, multi-order time-series difference features, multi-hysteresis autocorrelation coefficients, and cross-channel correlation coefficients are extracted from the structured feature sequence, and the extracted features are fused into a high-dimensional feature vector. The time-domain statistical features include mean, standard deviation, skewness, and kurtosis. The frequency domain features include the spectral centroid, spectral flatness, and energy distribution in multiple preset frequency bands, calculated based on the fast Fourier transform. The cross-channel correlation coefficient refers to the coefficient obtained by performing pairwise correlation calculations on different data channels representing position, velocity, and orientation in the sequence.

6. The method according to claim 5, characterized in that, Feature selection and normalization are performed to generate optimized feature vectors, including: Based on the mutual information criterion, the correlation between each feature dimension in the high-dimensional feature vector and the human activity category label is calculated, and the key feature subset is selected according to the correlation. The key feature subset is subjected to power transform normalization to generate the optimized feature vector.

7. The method according to claim 1, characterized in that, The method for constructing the heterogeneous ensemble classifier includes: The model integrates multiple different types of base classifiers; the multiple different types of base classifiers include at least two or more of the following models: decision tree ensemble model, support vector machine model, multilayer perceptron neural network model, and gradient boosting ensemble model.

8. The method according to claim 1, characterized in that, The optimized feature vector is input into the heterogeneous ensemble classifier, and the heterogeneous ensemble classifier is semi-supervised trained using a dynamic pseudo-label strategy, including: Based on the optimized feature vectors corresponding to the labeled point cloud data, the heterogeneous ensemble classifier is initially trained. Using the pre-trained classifier, the optimized feature vectors corresponding to the unlabeled point cloud data are predicted, and a corresponding confidence score is generated for each predicted category. For each action category, an initial confidence threshold and a minimum threshold are set, and the confidence threshold is applied to all predictions for that action category. From the prediction results corresponding to the action category, the top 50% of the prediction results in terms of confidence are selected. The point cloud data and the corresponding prediction category in the prediction results that simultaneously meet the current confidence threshold and the top 50% condition are added to the training set as pseudo-label data. The heterogeneous ensemble classifier is retrained using the expanded training set, and the confidence threshold is reduced by a preset step size after each round of training iterations until the confidence threshold drops to the minimum threshold or a preset iteration termination condition is met.

9. The method according to claim 1, characterized in that, The enhanced recognition model is used to classify and identify human activities in the millimeter-wave radar data to be identified, including: For the raw point cloud data of the millimeter-wave radar to be identified, preprocessing operations and feature extraction and optimization operations are performed to generate the corresponding optimized feature vector to be identified; The optimized feature vector to be identified is input into the enhanced recognition model; The enhanced recognition model outputs the classification result generated by the optimized feature vector to be identified, and the classification result includes at least the predicted human activity category and its corresponding confidence level. The categories of human activities include boxing, sitting, standing, walking, and waving.

10. A low-power millimeter-wave radar human behavior recognition system, characterized in that, include: The preprocessing module is used to preprocess the point cloud data acquired by the millimeter-wave radar to obtain a structured feature sequence; The feature extraction module is used to extract multi-level handcrafted features from the structured feature sequence, and to perform feature selection and normalization to generate an optimized feature vector. The model training module is used to input the optimized feature vector into the heterogeneous ensemble classifier and perform semi-supervised training on the heterogeneous ensemble classifier by combining a dynamic pseudo-label strategy to obtain an enhanced recognition model. The model application module is used to classify and identify human activities using the enhanced recognition model on the millimeter-wave radar data to be identified.

Images