A continuous multi-directional human behavior recognition method and system, and a storage medium

Abstract

The application discloses a continuous multi-direction human behavior recognition method and system and a storage medium, and is applied to the field of human behavior recognition in an indoor scene. In view of the problem that a current human behavior recognition method based on a millimeter wave radar can recognize only a single human behavior movement direction, and a recognition mechanism cannot support continuous operation, the application splits continuous human actions by using extracted movement parameters, obtains real-time split single actions, and then realizes continuous multi-direction human behavior recognition based on the millimeter wave radar by using a parameter estimation method and a mixed feature extraction model of a light-weight deep learning model. The application can realize continuous and stable multi-direction human behavior recognition in an indoor scene, and can be conveniently deployed on a small edge computing platform by using a light-weight recognition method, so that a human behavior recognition system constructed by the application has the advantages of convenient deployment, simple system structure, and easy productization, and can effectively control system cost.

Description

A method, system and storage medium for continuous multi-directional human behavior recognition Technical Field

[0001] This invention belongs to the field of radar human behavior recognition technology, and specifically relates to a continuous multi-directional human behavior recognition technology and system for indoor environments. Background Technology

[0002] In the field of radar-based human behavior recognition, due to its inherent advantages of not being sensitive to ambient light and not infringing on user privacy, radar-based human behavior recognition has the advantages of high reliability, high recognition accuracy, and no contact required. It is widely used in fields such as smart healthcare, medical care, human-computer interaction, and intelligent security.

[0003] Because the motion features captured by radar are relatively abstract, and the direction of motion can cause significant changes in these features, most existing radar-based human behavior recognition methods can only identify a single behavior in a fixed direction. To achieve multi-directional human behavior recognition, S. Waqua et al. proposed using two radars to capture multi-directional human motion and extracting the radial velocity distribution features to identify five types of human activities (S. Waqua, M. Muaaz and M. Muaaz). "Direction-Independent Human Activity Recognition Using a Distributed MIMORadar System and Deep Learning," in IEEE Sensors Journal, vol. 23, no. 20, pp. 24916-24929, 15 Oct. 15, 2023. In the research of Y Zhao et al., a method was proposed to use 4D imaging radar combined with a proposed hierarchical processing and classification pipeline to achieve the recognition of six types of human behavior at any angle (Y. Zhao, A. Yarovoy and F. Fioranelli, "Angle-Insensitive Human Motion and Posture Recognition Based on 4D Imaging Radar and Deep Learning Classifiers," in IEEE Sensors Journal, vol. 22, no. 12, pp. 12173-12182, 15 June 15, 2022). However, the above models are quite complex and difficult to deploy. To address this issue, a lightweight model building method is used to reduce model complexity (Q.Wang, B.Wu, P.Zhu, P.Li, W.Zuo and Q.Hu, "ECA-Net: Efficient Channel Attention for DeepConvolutional Neural Networks," 2020 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp.11531-11539). J.Zhu et al. proposed using one-dimensional deep convolution and point-directed convolution to build a lightweight CNN architecture, which achieved the classification of seven types of human activities (J.Zhu, X.Louand W.Ye, "Lightweight Deep Learning Model in Mobile-Edge Computing for Radar-Based Human Activity Recognition," in IEEE Internet of Things Journal, vol.8, no.15, pp.12350-12359, 1 Aug.1, 2021).However, existing radar-based human behavior recognition methods do not simultaneously address the issues of lightweight and multidirectional recognition models, as well as the complexity and practicality of the radar systems used. Furthermore, human behavior in real-world operating environments is continuous. Therefore, researching a continuous multidirectional human behavior recognition system based on millimeter-wave radar has significant practical implications. Summary of the Invention

[0004] To address the aforementioned technical issues, this invention proposes a continuous multi-directional human behavior recognition method and system based on millimeter-wave radar. By constructing a multi-directional human behavior dataset and combining the features of the multi-directional behavior data, a parameter estimation and deep learning hybrid extraction model that can effectively extract multi-directional behavior features is constructed. Furthermore, radar signal parameters are used to prune the continuous behavior data, enabling real-time recognition of targets continuously performing multiple behaviors in indoor scenes.

[0005] One of the technical solutions adopted in this invention is: a continuous multi-directional human behavior recognition method based on millimeter-wave radar, comprising:

[0006] S1. Establish a radar signal echo model of human kinematics;

[0007] S2. Preprocess the radar signal echo and extract the time-Doppler spectrum that characterizes the behavior of human targets;

[0008] S3. Real-time acquisition of human behavior data stream through millimeter-wave radar. Based on the human target behavior features extracted in step S2, Doppler energy intensity is used to divide the motion state and the stationary state, and the human behavior data stream is split into individual behavior data.

[0009] S4. Construct a discrete multi-directional human behavior feature database using millimeter-wave radar, and combine the multi-directional human behavior features to construct a hybrid feature extraction model based on parameter estimation and lightweight deep learning.

[0010] S5. Input the single behavioral data obtained in step S3 into the hybrid feature extraction model established in step S4 to realize continuous multi-directional human behavior recognition based on millimeter-wave radar.

[0011] The second technical solution adopted in this invention is: a continuous multi-directional human behavior recognition system based on millimeter-wave radar, comprising: a radar echo model construction module, a preprocessing module, a continuous human behavior data segmentation module, and a human behavior recognition module; the radar echo model construction module obtains a radar echo model of human behavior based on a radar transmission signal model and a human reflection model; the preprocessing module extracts a time-Doppler spectrum characterizing human target behavior features based on the modeled radar echo; the continuous human behavior data segmentation module uses Doppler energy intensity to segment the characterizing human target behavior features into individual behavior data; the human behavior recognition module uses a hybrid feature extraction model based on parameter estimation and a lightweight deep learning model to recognize the individual behavior data.

[0012] The third technical solution adopted in this invention is: an electronic device, including: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of a continuous multi-directional human behavior recognition method based on millimeter-wave radar are performed.

[0013] The fourth technical solution adopted in this invention is: a computer-readable storage medium storing a computer program, which, when run by a processor, executes the steps of a continuous multi-directional human behavior recognition method based on millimeter-wave radar.

[0014] The beneficial effects of this invention are as follows: This invention utilizes a single-chip FMCW millimeter-wave radar for human behavior perception, offering advantages such as a simple perception system, no infringement on user privacy, and insensitivity to lighting conditions. Furthermore, the lightweight human behavior recognition method constructed allows for convenient deployment on small edge computing platforms. The resulting human behavior recognition system is easy to deploy, has a simple system structure, and is readily commercializable, effectively controlling system costs. Moreover, the recognition method considers the feature differences caused by multi-directional human behavior. A parameter estimation method extracts time-Doppler spectrum features of human behavior independent of movement direction, while a deep learning model extracts the shape features of the time-Doppler spectrum, obtaining multi-dimensional human behavior features and achieving effective recognition of multi-directional human behavior. During the recognition process, real-time data acquisition, processing, and recognition are employed, effectively enabling the recognition of continuous behavior and making it more suitable for deployment in real-world production and living environments. The method of this invention has the following advantages:

[0015] 1. The proposed multi-directional human behavior recognition model extracts multi-directional human behavior features through parameter estimation methods and deep learning models, which effectively improves the robustness of multi-directional human behavior recognition.

[0016] 2. The constructed lightweight human behavior recognition method effectively reduces system complexity and system construction costs;

[0017] 3. The proposed continuous behavior data segmentation method effectively achieves the cropping of activity data, thereby enabling continuous recognition of human behavior;

[0018] 4. This invention can be used in indoor settings for smart elderly care, smart security, human-computer interaction, and other fields. Attached Figure Description

[0019] Figure 1 shows the processing flow of the proposed method.

[0020] Figure 2 is a schematic diagram of human behavior echoes in close-range scenes.

[0021] Figure 3 is a flowchart of radar signal preprocessing.

[0022] Figure 4 shows a schematic diagram of the original data matrix before and after radar signal preprocessing, and a set of measured data after processing.

[0023] Figure 4(a) is a schematic diagram of the original radar data matrix, and Figure 4(b) is a time-Doppler spectrum of a set of measured data obtained after processing.

[0024] Figure 5 shows the time-Doppler spectrum of a set of measured data and its Doppler energy intensity;

[0025] Figure 5(a) shows the time-Doppler spectrum of a set of measured data for two consecutive waving gestures, and Figure 5(b) shows the corresponding Doppler energy intensity waveform.

[0026] Figure 6 shows the raw time-Doppler spectrum of continuous human behavior data and the time-Doppler spectrum of a single behavior after being split.

[0027] Figure 6(a) shows the time-Doppler spectrum of continuous human behavior data, Figure 6(b) shows the time-Doppler spectrum of the first behavior, and Figure 6(c) shows the time-Doppler spectrum of the second behavior.

[0028] Figure 7 shows the overall block diagram of the hybrid feature extraction model combining parameter estimation and deep learning.

[0029] Figure 8 shows the overall structure of the lightweight deep learning feature extraction module.

[0030] Figure 9 shows the overall block diagram of the parameter estimation module.

[0031] Figure 10 shows the confusion matrix of the four models on the constructed dataset;

[0032] Figure 10(a) shows the confusion matrix of ResNet18, Figure 10(b) shows the confusion matrix of MobileNetV2, Figure 10(c) shows the confusion matrix of CNN, and Figure 10(d) shows the confusion matrix of parameter estimation and deep learning hunter feature extraction model.

[0033] Figure 11 shows the GUI interface of a continuous multi-directional human behavior recognition system based on millimeter-wave radar.

[0034] Figure 12 is a schematic diagram of a continuous multi-directional human behavior recognition system based on millimeter-wave radar. Detailed Implementation

[0035] To facilitate understanding of the technical content of this invention by those skilled in the art, the following description, in conjunction with the accompanying drawings, further illustrates the invention.

[0036] Example 1

[0037] The present invention provides a continuous multi-directional human behavior recognition system based on millimeter-wave radar, comprising the following steps:

[0038] Step 1: Construction of Human Behavior Echo Signal Model

[0039] Considering the radar echo model of a human target in a close-range scenario, a schematic diagram is shown in Figure 2. This diagram illustrates a radar perception scenario of a target waving. The radar is installed 2.8m above the ground, with the radar antenna plate parallel to the ground. First, the time-domain expression for the radar's transmitted signal is:

[0040]

[0041] Where A is the signal amplitude, t represents time, f0 and B are the original frequency and signal bandwidth of the FMCW (Frequency Modulated Continuous Wave) radar, and T is the signal bandwidth. c This represents the period of the FMCW signal. In Figure 2, "Waving" represents waving, "radar" represents radar, and "ground" represents the ground.

[0042] Next, consider a near-field scenario where a human target is stationary and standing at a distance *d* meters from the radar. In this case, the human body serves as an extended target, and the radar receives a signal from the target's multi-scattering point echoes as follows:

[0043]

[0044] In the formula, β is the attenuation factor of the radar signal during propagation, k represents the number of scattering points of the assumed human target, and τ iThis represents the propagation delay of the i-th scattering point of the human target.

[0045] Next, the received target echo signal and the transmitted signal are mixed by a mixer to obtain the intermediate frequency signal, the time-domain expression of which is:

[0046]

[0047] Its center frequency It is directly proportional to the distance to the target. This represents the phase of the intermediate frequency signal.

[0048] Based on the above, considering the target moving at a speed of v meters per second in the radial direction of the radar, its received signal at time t and the mixed intermediate frequency signal are as shown before. After one chirp period T c Then, the distance between the target and the radar is d + Δd, where Δd = vT c At this point, the expression for the intermediate frequency signal is:

[0049]

[0050] in This indicates that due to the phase shift caused by velocity, the phase of different chirps contains micro-Doppler information about the target motion.

[0051] Step 2: Radar signal preprocessing

[0052] Before preprocessing the radar data, the intermediate frequency (IF) signal acquired by the radar needs to be sampled to obtain the raw radar data. First, the IF signal is sampled to obtain the corresponding raw radar data vector, such as...

[0053] Z i [n] = [z] i (1),z i (2),...z i (Ns)] T

[0054] In the formula, the superscript T indicates transpose, z i (k) represents the value obtained by sampling the intermediate frequency signal of the i-th chirp at the k-th time, and Ns represents the number of sampling points in one chirp period.

[0055] After N c After sampling for one cycle, a frame of raw radar data is formed. Based on this, N data are collected. f The frame, whose data matrix is ​​shown below

[0056] Z = [Z1[n], Z2[n], ... ZM [n]

[0057] In the formula, M = N c ×N f , which represents the number of chirps in the original radar data matrix.

[0058] Then, the raw radar data is processed to obtain the time-Doppler spectrum representing human target behavior. Figure 3 shows the flowchart of the preprocessing, with the raw radar data matrix as input and the human behavior time-Doppler spectrum as output. Specifically, on the raw radar data matrix Z, an FFT is performed on the data of each chirp signal to extract the target's range spectrum, calculated as follows:

[0059]

[0060] In the formula, N represents the number of points in the FFT, W is a preset Hamming window function, Z[u,m] represents the data of the u-th sampling point of the m-th chirp, and p represents the p-th distance cell.

[0061] The radar range spectrum is obtained after the above calculations. This range spectrum includes the range information of all reflectors within the scene. Therefore, to suppress the influence of static reflectors and obtain the target's range information, pulse cancellation is used to achieve Moving Target Indication (MTI). The calculation method is as follows:

[0062] R'[p,m]=R[p,m]-R[p,m+b]

[0063] In the formula, R'[p,m] represents the distance spectrum after pulse cancellation, and b represents the interval of each pulse cancellation.

[0064] On the canceled range spectrum, target location information is extracted to obtain the range cell where the target is located. Within the target's range cell, an N-point FFT (Fast Fourier Transform) is performed frame-by-frame on the original range image data to acquire Doppler information. Assuming the target is located in the l-th range cell, the Doppler calculation method for its s-th frame is as follows:

[0065]

[0066] After preprocessing, a frame of raw data is obtained and converted into Doppler information. This is followed by processing all N... f After frame processing, the target's time-Doppler spectrum is obtained. Figure 4 shows a schematic diagram of the structure of the original radar data matrix and a set of time-Doppler spectra obtained after preprocessing of the measured data of a target falling.

[0067] Step 3: Real-time data acquisition and data splitting

[0068] To achieve continuous human behavior recognition, the system breaks down the continuous human behavior data acquired by radar to obtain feature representations of individual behaviors and distinguish behavior categories. Specifically, the classification of human behaviors mainly relies on the existence of a certain time gap between two behaviors. During this gap, the human body is in a relatively static state, thus the data of continuous behaviors is broken down by utilizing the time gap between two behaviors.

[0069] To distinguish between the continuous state of a behavior and the relatively static state during the time interval between behavior transitions, Doppler energy intensity was used to divide the motion and static states based on human behavior features extracted from radar signal preprocessing. Figure 5 shows the time-Doppler spectrum and corresponding Doppler energy intensity of a set of measured data for two consecutive hand waves. As can be seen from the figure, the target is in a static state before 0.2 seconds, at which point the Doppler energy intensity is low. Between 0.2 and 2 seconds, the target is in the continuous state of the behavior, at which point the Doppler energy intensity is much higher than in the static state. Finally, between 2 and 3.1 seconds, during the interval between the two behaviors, the target is in a relatively static state. From 3.1 to 4.6 seconds, the target is in the continuous state of the second behavior. From 4.6 seconds until the end, the target is in a static state. This example demonstrates that by segmenting the behavior using Doppler energy intensity, individual behaviors can be extracted.

[0070] The formula for calculating Doppler energy intensity is:

[0071]

[0072] Where Q[s] is the Doppler energy intensity of the s-th frame.

[0073] For the continuously acquired behavioral data in real time, a signal interception method is used. Based on the energy intensity detection, two behaviors with durations of 0.2 to 2 seconds and 3.1 to 4.6 seconds are identified. The time-Doppler spectra within the duration of these behaviors are retained and then zero-padded to create a 5-second time-Doppler spectrum. The specific processing method is shown in Figure 6. This method effectively achieves the segmentation and interception of continuous behavioral features.

[0074] Step 4: Construct multi-directional human behavior data and recognition model

[0075] For data acquisition, the first step is to construct an experimental scenario for multi-directional human activity based on millimeter-wave radar. To effectively acquire human motion characteristics, the radar is mounted on a bracket at a height of 2.8 meters above the ground, with the radar's antenna array kept parallel to the ground, enabling behavioral perception within a 5m x 5m indoor area. This experimental scenario effectively avoids the loss of behavioral characteristics caused by target movement perpendicular to the radar axis when the radar is mounted laterally, thus achieving effective capture of multi-directional human behavior.

[0076] To mitigate the impact of motion direction on recognition performance in multi-directional human behavior recognition tasks, a novel human behavior recognition network, named the Parameter Estimation and Deep Learning Hybrid Feature Extraction Model, is proposed, as shown in Figure 7. The model consists of a lightweight deep learning feature extraction module, a parameter estimation module, and a classification module. This network enhances the feature representation capability of multi-directional human activities by incorporating direction-independent parameter information extracted by the parameter estimation module, based on the depth information extracted by the deep learning model, thereby improving the performance of multi-directional human behavior recognition.

[0077] Specifically, the framework of the lightweight deep learning feature extraction module is shown in Figure 8. First, the input spectral data is upscaled using a regular convolutional layer, and then normalized using Batch Normalization to improve network training efficiency. Next, multi-dimensional feature extraction is performed using group convolutional layers. Based on this, channel shuffling is applied to the feature maps output by different group convolutional kernels, ensuring that the feature maps of each group in the next group convolutional layer come from the outputs of different group convolutional kernels in the previous layer, thus achieving information exchange between different groups. Then, Ghost Conv is used to generate more feature maps on this output feature map, achieving a high-dimensional representation of the time-Doppler spectrum. Finally, fully connected layers are used for feature mapping to complete the extraction of deep information from the time-Doppler spectrum.

[0078] Figure 9 shows the overall processing flowchart of the time-Doppler spectrum feature analysis module based on the parameter estimation method. This method extracts three types of features from the time-Doppler spectrum: the number of Doppler frequency components, the duration of the action, and the Doppler area, for characterizing human activities. Furthermore, a fall feature is extracted from the time-Doppler spectrum to distinguish between falls and non-falls. Specifically, to extract the parametric features of the time-Doppler spectrum, image binarization is first performed. To reduce computational complexity, the mean of the time-Doppler spectrum is used as the binarization threshold. The threshold is calculated as follows:

[0079]

[0080] In the formula, α is the scaling factor, which is 4 in this embodiment, and max() is the maximum value function. After binarizing the time-Doppler spectrum using the above threshold, a binarized time-Doppler spectrum is obtained.

[0081] Based on the binarized time-Doppler spectrum, we perform parameter estimation. The parameter estimation method mainly extracts four parameters: the number of Doppler frequency components, the duration of the behavior, the Doppler area, and the fall feature. The number of Doppler frequency components is taken into account because different activities involve different limb movements, leading to variations in the number of Doppler frequency components. The duration of the behavior is considered, taking into account the differences in duration between different behaviors; for example, the duration of a fall can differ by up to two times from that of sleeping. The Doppler area is taken into account the differences in velocity between different behaviors; for example, the velocity of sleeping is significantly lower than that of a fall. Regarding the fall feature, because the area of ​​the target's body illuminated by radar is larger in the latter half of the fall, the target's radar cross section (RCS) increases. Furthermore, the higher velocity at this point causes a broadening phenomenon in the time-Doppler spectrum, which is not observed in other behaviors. Therefore, the fall feature can effectively distinguish between falling and non-falling activities.

[0082] As can be seen from the parameter estimation algorithms above, their statistical features are independent of the shape features of the time-Doppler spectrum. Therefore, they avoid feature differences caused by shape changes in the time-Doppler spectrum due to motion direction, achieving human activity feature extraction independent of motion direction. However, although parameter estimation algorithms effectively extract global statistical features from the time-Doppler spectrum, these methods lack depth information of the image. Therefore, features extracted solely through parameter estimation methods cannot achieve accurate multi-directional human behavior recognition.

[0083] The fully connected classification module based on concatenated feature vectors consists of two main steps. First, the features output by the parameter estimation method and the deep learning model are concatenated to obtain human activity feature vectors. Then, on the classifier, a fully connected layer maps the feature vectors to the activity categories, completing human behavior recognition based on Doppler spectra.

[0084] Step 5: Model Reasoning and Behavior Recognition

[0085] After cropping the data of continuous behaviors in step 3, the time-Doppler spectrum of a single behavior is obtained. This spectrum is then fed into the constructed behavior recognition model. Through the model's inference, the behavior recognition result is obtained.

[0086] The following describes a specific implementation of the present invention using a constructed human behavior dataset and an experimental system.

[0087] The duration of a single action was set to 5 seconds. A multi-directional human behavior dataset was collected, consisting of data from 10 targets performing 6 types of actions, with 600 samples per type, totaling 3600 sets of multi-directional human behavior data. After preprocessing the data, the time-Doppler spectrum of each set was obtained. Data from 8 targets was used for training, and data from 2 targets was used for testing, in an 8:2 ratio, to obtain the model's test performance.

[0088] In the experimental validation on the dataset, ResNet18 (K. He, X. Zhang, S. Ren, J. Sun, “Deep Residual Learning for Image Recognition,” Arxiv, 2015. [Online].), MobileNetV2 (M. Sandler, A. Howard, M. Zhu, A. Zhmoginov and L.-C. Chen, “MobileNetV2: Inverted Residuals and Linear Bottlenecks,” 2018 IEEE / CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 4510-4520.), and CNN (H. Sadreazami, M. Bolic and S. Rajan, “Contactless Fall Detection Using Time-Frequency Analysis and Convolutional Neural Networks,” IEEE Transactions on Industrial Networks) were validated. (Informatics, vol. 17, no. 10, pp. 6842-6851, Oct. 2021.) and the recognition performance of the parameter estimation and deep learning hybrid feature extraction model proposed in this invention are shown in Figure 10. The confusion matrices of the four models on the entire dataset are also shown. As can be seen from the figure, the parameter estimation and deep learning hybrid feature extraction model constructed in this invention has the best recognition performance for multi-directional human behavior, with an overall recognition accuracy of 96.67%. In comparison, the recognition accuracy of ResNet18 is 93.75%, MobileNetV2 is 92.50%, and CNN is 92.78%.

[0089] In Figure 10, "hand" represents waving, "sleep" represents sleeping, "standup" represents standing up, "sit down" represents sitting down, "fall" represents falling down, "getup" represents getting up, "predicted label" represents the predicted label, and "True label" represents the true label.

[0090] Finally, a radar-based continuous multi-directional human behavior recognition system was constructed. The overall system architecture and graphical user interface are shown in Figure 11. The system acquires raw radar data in real time via a PC, performs radar signal preprocessing, continuous behavior segmentation, model inference, and recognition on the PC, and then obtains real-time human behavior recognition results. In Figure 11, the horizontal axis "time" represents the time period.

[0091] Example 2

[0092] As shown in Figure 12, this embodiment provides a continuous multi-directional human behavior recognition system based on millimeter-wave radar, including: a radar signal echo model construction module, a preprocessing module, a continuous human behavior data segmentation module, and a human behavior recognition module; the radar signal echo model construction module obtains a radar echo model of human behavior based on a radar transmission signal model and a human reflection model; the preprocessing module extracts a time-Doppler spectrum representing the human target behavior features based on the modeled radar echo; the continuous human behavior data segmentation module uses Doppler energy intensity to segment the human target behavior features into individual behavior data; the human behavior recognition module uses a hybrid feature extraction model based on parameter estimation and a lightweight deep learning model to recognize the individual behavior data.

[0093] Example 3

[0094] This embodiment provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, a continuous multi-directional human behavior recognition task based on millimeter-wave radar is performed.

[0095] Example 4

[0096] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs a continuous multi-directional human behavior recognition task based on millimeter-wave radar.

[0097] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of the claims of the invention.

Claims

1. A method for continuous multi-directional human behavior recognition, characterized in that, include: S1. Establish a radar signal echo model based on human kinematics; the implementation process of step S1 is as follows: For the transmitted signal of millimeter-wave radar, its time-domain expression is: ;in, The signal amplitude, Indicates time, 、 These are the original frequency and signal bandwidth of the millimeter-wave radar, respectively. The period of the millimeter-wave radar signal is given; then, a human body standing still at a distance from the millimeter-wave radar is considered. At a position of meters; at this moment, with the human body as an extended target, the signal of the target's multi-scattering point echo received by the millimeter-wave radar is: In the formula, This is the attenuation factor of the radar signal during propagation. This indicates the number of hypothetical scattering points of the human body. The first one represents the human body The propagation delay of each scattering point; then, the received signal of the target multi-scattering point echo is mixed with the millimeter-wave radar transmitted signal by a mixer to obtain the intermediate frequency signal, the time domain expression of which is: Among them, the frequency of the intermediate frequency signal It is directly proportional to the distance to the target. The phase of the intermediate frequency signal. The number of scattering points on the human body; considering the target's radial direction relative to the millimeter-wave radar. Motion at a speed of meters per second, its The received signal at a given time and the intermediate frequency signal after mixing After one chirp cycle Afterwards, the distance between the target and the millimeter-wave radar is ,in At this point, the time-domain expression of the intermediate frequency signal is: ;in This indicates the phase shift caused by velocity. S2. Preprocess the radar signal echo model to extract the time-Doppler spectrum representing the human target behavior characteristics; S3. Acquire human behavior data stream in real time through millimeter-wave radar, and based on the human target behavior characteristics extracted in step S2, use Doppler energy intensity to divide the motion state and stationary state, splitting the human behavior data stream into individual behavior data; S4. Construct a discrete multi-directional human behavior feature database through millimeter-wave radar, and construct a hybrid feature extraction model based on parameter estimation and lightweight deep learning in combination with the multi-directional human behavior feature database; S5. Input the individual behavior data obtained in step S3 into the hybrid feature extraction model established in step S4 to realize continuous multi-directional human behavior recognition based on millimeter-wave radar.

2. The continuous multi-directional human behavior recognition method according to claim 1, characterized in that, Before preprocessing the radar signal echo model in step S2, the method further includes: sampling the intermediate frequency signal acquired by the millimeter-wave radar to obtain the original radar data. Specifically: first, the intermediate frequency signal is sampled to obtain the original radar data vector: In the formula, Indicates the first The intermediate frequency signal of the chirp cycle is at the first... The values ​​obtained by sampling at each time point This represents the number of sampling points within one chirp period; after After sampling for one cycle, a frame of raw radar data is formed. Based on this, data is collected... The original radar data matrix is ​​shown below: In the formula, This represents the number of chirp periods in the original radar data matrix. 。 3. The continuous multi-directional human behavior recognition method according to claim 2, characterized in that, The implementation process of step S2 is as follows: in the radar raw data matrix Above, perform FFT on the data of each chirp cycle signal to extract the target's range spectrum: In the formula, N represents the number of points in the FFT, and W is a preset Hamming window function. Indicates the first The first chirp cycle Data from each sampling point Indicates the first Each distance unit; pulse cancellation is used to indicate moving targets, and the calculation method is as follows: In the formula, This represents the distance spectrum after pulse cancellation. This indicates the interval between each pulse cancellation; on the cancelled range spectrum, target location information is extracted using a detection algorithm to obtain the range cell where the target is located; within the range cell where the target is located, the original range image data is processed frame by frame. The point-wise FFT is used to acquire Doppler information. Assuming the target is located in the l-th range cell, the Doppler information of its s-th frame is calculated as follows: ;in, For the first One Doppler unit; after preprocessing, the acquisition of Doppler information from a frame of raw data is completed, and after processing all... After processing the frames, the time-Doppler spectrum of the target is obtained.

4. The continuous multi-directional human behavior recognition method according to claim 3, characterized in that, The continuous behavior data splitting method described in step S3 splits the data by extracting the Doppler energy intensity of the millimeter-wave radar. Specifically, when the target is moving, the Doppler energy intensity of the millimeter-wave radar is higher than that in the stationary state. The formula for calculating the Doppler energy intensity is as follows: ;in, For the first The Doppler energy intensity of each frame is calculated to determine whether the target has moved. Valid frames in which the target has moved are then extracted, thus splitting continuous behavioral data.

5. The continuous multi-directional human behavior recognition method according to claim 4, characterized in that, The hybrid feature extraction model based on parameter estimation and a lightweight deep learning model described in step S4 includes: a lightweight deep learning feature extraction module, a parameter estimation module, a concatenation module, and a classification module. The lightweight deep learning feature extraction module takes a time-Doppler spectrum as input and outputs the depth information of the time-Doppler spectrum. The parameter estimation module takes a time-Doppler spectrum as input and outputs the number of Doppler frequency components, behavior duration, Doppler area, and fall features. The concatenation module concatenates the depth information of the time-Doppler spectrum with the output features of the parameter estimation module to obtain a human activity feature vector. The classification module takes the human activity feature vector as input and outputs the human behavior recognition result of the Doppler spectrum.

6. A continuous multi-directional human behavior recognition system, employing the continuous multi-directional human behavior recognition method according to any one of claims 1-5, characterized in that, include: Radar echo model building module, preprocessing module, continuous human behavior data splitting module, and human behavior recognition module; The radar echo model construction module obtains the radar echo model of human behavior based on the radar transmission signal model and the human body reflection model. The preprocessing module extracts time-Doppler spectra representing human target behavior features based on the modeled radar echoes; The continuous human behavior data segmentation module uses Doppler energy intensity to segment the characteristics representing human target behavior into individual behavior data; The human behavior recognition module uses a hybrid feature extraction model based on parameter estimation and a lightweight deep learning model to recognize individual behavioral data.

7. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the continuous multi-directional human behavior recognition method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the continuous multi-directional human behavior recognition method according to any one of claims 1-5.

Images