A human activity detection method capable of self-expanding radar data
By combining data collected by radar sensors and Kinect cameras, and utilizing a dual-branch feature extraction module and a meta-learning framework to self-expand radar data, the problem of the lack of open-source databases in the field of radar detection is solved. This enables rapid training and efficient recognition of human activities, especially fall behavior, thereby improving the recognition rate and robustness.
Patent Information
- Application Number
- CN202410975761.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-07-19
AI Technical Summary
In the current technology, the lack of open-source databases in the field of radar detection makes it difficult to quickly obtain reliable neural network models because the construction of neural network models requires a long time for sample collection and training.
By combining radar sensors and Kinect cameras, raw radar data and simulated radar data are collected. Using a dual-branch feature extraction module and a meta-learning framework, the human activity detection model is trained by self-expanding radar data, including the collection and feature extraction of indoor human behavior. Simulated radar data is used to supplement the raw radar data for meta-learning training.
It enables rapid training of human activity detection models with small sample sizes, improves the accuracy and robustness of fall detection, reduces training time, and protects personal privacy from environmental influences.
Smart Images

Figure CN118520305B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of human activity detection methods, and specifically discloses a human activity detection method capable of self-expanding radar data. Background Art
[0002] With the advent of the intelligent era, various intelligent recognition technologies for detecting human activities such as falls have flourished. Indoor human activity detection and recognition has important applications in daily life scenarios, such as smart homes, health monitoring, motion analysis and many other fields.
[0003] For example, for traditional fall behavior recognition technology, there are detection methods that use wearable devices combined with computer vision, and detection methods that use radar equipped with neural networks. At the same time, there is also a hybrid method that combines radar detection technology with computer vision.
[0004] Radar detection methods equipped with neural networks often involve the construction of neural networks and the comparison of data collected by the radar with historical data to determine falling behavior. The generation of historical data requires a large number of samples.
[0005] In the existing technology, the field of visual images has been well developed, and there are a large number of open source databases. When the amount of data is sufficient, the performance of the neural network can be significantly improved. However, the lack of open source databases in the radar field makes it difficult to obtain a large number of labeled samples. Therefore, after the neural network model based on radar detection is built, long-term sample collection is required to train the neural network and obtain a sufficient database and a reliable neural network model in the future. In the existing technology, there is no method for self-expansion of samples for radar detection data.
[0006] In view of this, the inventor provides a human activity detection method capable of self-expanding radar data to solve the above-mentioned problem. Summary of the invention
[0007] The purpose of the present invention is to solve the problem that in the traditional field of radar detection technology, there is a lack of open source databases, which leads to the need for long-term sampling to enrich and train neural network models at the beginning of building various neural network models based on radar detection.
[0008] In order to achieve the above object, the basic scheme of the present invention provides a method for detecting human activities that can self-expand radar data, comprising the following steps:
[0009] Step S1: collecting indoor personnel behavior detection samples, wherein the collection includes collecting at least two groups of detection samples in mutually perpendicular directions in a plane;
[0010] Step S2: The collected behavior detection samples are used as inputs in different directions and input into a human activity detection model. The human activity detection model is a neural network model and is internally equipped with a dual-branch feature extraction module for distinguishing human action categories. The dual-branch feature extraction module is used to extract features of different inputs respectively, and correspondingly obtains different features of the same number as the input;
[0011] Step S3: The obtained different features are spliced and input into the classifier for decision making, and the classifier is used to determine the human action category to which the behavior represented by the fused input belongs;
[0012] The human activity detection model is trained based on a local data set, which includes original radar data and simulated radar data. The original radar data is obtained by radar sampling, and the simulated radar data is specifically expanded radar data simulated based on human skeleton data collected by a kinect camera.
[0013] Further, the acquisition of the simulated radar data comprises the following steps:
[0014] Step A1: Building a sample collection scene, including building an indoor venue, selecting a target person, installing at least two sets of ultra-wideband impulse radars in the indoor venue in perpendicular directions in a plane, installing a wearable device on the target person, and installing a kinect camera in the indoor venue;
[0015] Step A2: The wearable device maps the coordinates corresponding to different joints of the target person in the Kinect camera coordinate system, and uses each different ultra-wideband impulse radar as the coordinate origin to transform the coordinates corresponding to different joints of the target person in the Kinect camera coordinate system, and the coordinates form the human skeleton data;
[0016] Step A3: using the human skeleton data to calculate corresponding radar echo signals of multiple frames of human skeleton data;
[0017] Step A4: performing short-time Fourier transform on multiple frames of human skeleton data radar echo signals respectively to obtain time-frequency diagrams in corresponding directions as simulated radar data.
[0018] Further, in step A3, the calculation process of radar echo signals of multiple frames of human skeleton data includes the calculation of radar echo signals of human joints at different radar frequencies in each frame collected by the Kinect camera.
[0019] Further, the implementation of step A3 specifically includes the following steps:
[0020] Step A301: Calculate the human target signals of all human joints at different radar frequencies and obtain the human target signals;
[0021] Step A302: Calculate the noise impact in the environment, including the calculation of thermal noise and environmental noise in the environment;
[0022] Step A303: combining the human target signal and the noise signal calculated in the same frame to obtain the radar echo signal in the same frame;
[0023] Step A304: looping through step A301 to step A303 to obtain multiple frames of human skeleton data radar echo signals corresponding to ultra-wideband impulse radars in multiple different directions.
[0024] Further, in step S2, the steps of extracting features of different inputs by the dual-branch feature extraction module are as follows:
[0025] The single-branch feature extraction module receives the input and obtains the corresponding input features through the convolution layer, pooling layer and softmax layer in sequence.
[0026] Furthermore, in step S1, the sampling process of indoor personnel behavior samples includes the forward sample collection of personnel behavior and the lateral sample collection of personnel behavior.
[0027] Furthermore, the human activity detection model is meta-learned and trained based on the local data set, and the steps of the meta-learning process are as follows:
[0028] Step C1: collecting original radar data and simulated radar data, and preprocessing the collected original radar data and simulated radar data to obtain a data set;
[0029] Step C2: Establish a meta-learning framework and initialize the meta-learning framework parameters;
[0030] Step C3: construct and divide meta-tasks based on the data set;
[0031] Step C4: Perform inner-loop based meta-learning training on the meta-task after planning is completed until the training of the human activity detection model is completed.
[0032] Further, in step C1, the preprocessing step for the original radar data and the simulated radar data includes preprocessing operations of cleaning, standardizing and enhancing the original radar data and the simulated radar data.
[0033] Further, in step C3, after the meta-task is constructed, the meta-task is divided based on human activity detection in different environments and for different groups of people, and the meta-task includes a training phase and a test phase and corresponding training sets and test sets obtained based on the data set.
[0034] The principle and effect of this scheme are:
[0035] 1. Compared with the prior art, the present invention uses a radar sensor as a sensing terminal for obtaining human behavior information, and uses the expanded radar data obtained by simulating the human skeleton data obtained based on the Kinect camera as simulated radar data. The original radar data and the simulated radar data obtained by the radar sensor are input into the human activity detection model, and the training of the human activity detection model is completed through a meta-learning process. It can be applied in the case of insufficient detected samples. By combining the radar sensor with the Kinect camera, the data set can be supplemented and the radar data can be self-expanded without the input of new original radar data. This solves the problem in the field of traditional radar detection technology that there is a lack of open source databases, resulting in the need for long-term sampling to enrich and train the neural network model at the beginning of building each neural network model based on radar detection.
[0036] 2. Compared with the prior art, the present invention can realize accurate recognition of fall detection while protecting personal privacy and not being affected by the environment. It has the characteristics of strong robustness, good stability and high recognition rate. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0038] Figure 1 A flow chart of a method for detecting human activities that can self-expand radar data proposed in an embodiment of the present application is shown;
[0039] Figure 2 A flow chart of a dual-branch feature extraction module proposed in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0040] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.
[0041] A method for detecting human activities that can self-expand radar data, for example Figure 1 As shown:
[0042] The steps include:
[0043] Step S1: collecting indoor personnel behavior detection samples, wherein the collection includes collecting at least two groups of detection samples in mutually perpendicular directions in a plane;
[0044] Step S2: The collected behavior detection samples are used as inputs in different directions and input into a human activity detection model. The human activity detection model is a neural network model and is internally equipped with a dual-branch feature extraction module for distinguishing human action categories. The dual-branch feature extraction module is used to extract features of different inputs respectively, and correspondingly obtains different features of the same number as the input;
[0045] Step S3: The obtained different features are spliced and input into the classifier for decision making, and the classifier is used to determine the human action category to which the behavior represented by the fused input belongs;
[0046] The human activity detection model is trained based on a local data set, which includes original radar data and simulated radar data. The original radar data is obtained by radar sampling, and the simulated radar data is specifically expanded radar data simulated based on human skeleton data collected by a kinect camera.
[0047] Specifically, in this embodiment, it is for the detection of falling actions. For the actual falling behavior detection process, multiple ultra-wideband impulse radars in different directions are installed to perform sampling with multiple ultra-wideband impulse radars in different directions. In this embodiment, two ultra-wideband impulse radars in different directions are provided to sample data in two different directions respectively, and the two sampled data are used as inputs in two different directions respectively. The features of the two different inputs are extracted respectively through a dual-branch feature extraction module to obtain two different types of features, and the two different types of features are spliced and input into the classifier for decision making. The classifier is used to determine whether the behavior exhibited by the fused input is a falling behavior. Figure 2 As shown, in this embodiment, the classifier is a softmax layer.
[0048] In this embodiment, the dual-branch feature extraction module is composed of two parallel single-branch feature extraction modules, and the single-branch feature extraction modules are composed of a convolution layer and a pooling layer.
[0049] The steps for obtaining simulated radar data are as follows:
[0050] Step A1: Building a sample collection scene. The building of the sample collection scene includes indoor venue construction, target person selection, ultra-wideband impulse radar installation in the venue, human wearable device installation, and Kinect camera installation.
[0051] Specifically, the construction of the indoor venue is arranged according to the actual indoor scene. The installation of ultra-wideband impulse radars in the venue includes the installation of forward ultra-wideband impulse radars and lateral ultra-wideband impulse radars. The forward ultra-wideband impulse radar is used to detect and sample the forward lean and forward lean trend of the human body, and the lateral ultra-wideband impulse radar is used to detect and sample the lateral tilt and lateral tilt trend of the human body. The target person wears wearable human body devices, and the kinect camera is used to collect the position information of the human body joints emitted by the wearable human body devices.
[0052] Step A2: The wearable device maps the coordinates corresponding to different joints of the target person in the Kinect camera coordinate system, and uses each forward ultra-wideband impulse radar and side ultra-wideband impulse radar as the coordinate origin to transform the coordinates corresponding to different joints of the target person in the Kinect camera coordinate system, and the coordinates constitute the human skeleton data.
[0053] Step A3: Calculate the corresponding radar echo signal using the human skeleton data. The calculation process of the radar echo signal includes calculating the radar echo signal at different radar frequencies at the human joint points in each frame collected by the Kinect camera.
[0054] Specifically, for any single ultra-wideband impulse radar, selecting the mth frame of human skeleton data to calculate radar echo signals of all human joints in the frame at different radar frequencies includes the following specific steps:
[0055] Step A301: Calculate the human target signal of all human joints at different radar frequencies and obtain the human target signal u target . Human target signal u target The calculation formula is as follows:
[0056] ;
[0057] Where D atten (m,i) represents the distance attenuation term of the i-th human joint point in the m-th frame, , which is used to indicate that the reflected signal of the radar wave attenuates as the distance increases, where A is a constant and σ i represents the scattering cross-sectional area corresponding to the i-th joint point, represents the position information from the ith joint point to the simulated radar, {x m,i ,y m,i ,z m,i} respectively represent the X-axis coordinate, Y-axis coordinate and Z-axis coordinate of the i-th human joint point in the three-dimensional coordinate system in the m-th frame.
[0058] Where, I represents the total number of human joints;
[0059] Where G(m,i) represents the Gaussian pulse envelope term, , used to represent the Gaussian pulse envelope of the radar transmission signal, where represents pulse width, which is used to indicate the duration of a signal, C0 represents the speed of light, and t represents time.
[0060] Where DT(m,i) represents the phase term, , which is used to represent the phase change of the target due to the propagation path. This formula introduces the Doppler effect and is applicable to the case where the target (human body) is a dynamic target, that is, the target's velocity v and acceleration a are added to model the dynamic target, where f c , f d , f e They all represent different carrier frequencies, and j is an imaginary number.
[0061] Where F(m,i) represents the frequency dependence term, F(m,i)=exp(-βR m,i ), which is used to represent the attenuation change of the signal during propagation at different radar frequencies, and β represents the attenuation factor.
[0062] Step A302: Calculate the noise impact in the environment, including the calculation of thermal noise and environmental noise in the environment.
[0063] Among them, the thermal noise u in the environment ther and environmental noise u sca The calculation is carried out as follows:
[0064] ;
[0065] In the formula, k B is the Boltzmann constant, which is approximately 1.38×10 -23 J / K;
[0066] Where T is the absolute temperature of the conductor resistor in the environment, R1 is the resistance value, and Δf is the noise bandwidth.
[0067] Where, L i is the noise level of the ith noise source.
[0068] Specifically, the thermal noise u ther and environmental noise u sca The complex noise signal is obtained by unified calculation. The calculation formula of the complex noise signal is as follows:
[0069] ;
[0070] Step A303: combining the human target signal and the noise signal calculated in the same frame to obtain the radar echo signal in the same frame.
[0071] Specifically, the calculation formula of the radar echo signal is as follows:
[0072] ;
[0073] Step A304: looping step A301 to step A303, respectively obtaining radar echo signals of multiple frames of human skeleton data corresponding to the forward ultra-wideband impulse radar and the lateral ultra-wideband impulse radar.
[0074] Step A305: Perform short-time Fourier transform processing on the radar echo signals corresponding to the forward ultra-wideband impulse radar and the lateral ultra-wideband impulse radar respectively, generate a forward time-frequency diagram corresponding to the forward ultra-wideband impulse radar, and generate a lateral time-frequency diagram corresponding to the lateral ultra-wideband impulse radar, and use the forward time-frequency diagram and the lateral time-frequency diagram as the basic sample input of the human activity detection model, and at the same time use the forward time-frequency diagram and the lateral time-frequency diagram as the basic input samples of the single-branch feature extraction module.
[0075] like Figure 2 As shown in FIG, after obtaining the forward time-frequency map and the lateral time-frequency map, the forward time-frequency map and the lateral time-frequency map are respectively used as the input of the single-branch feature extraction module, and the same convolution and pooling process is performed.
[0076] Specifically, take the forward time-frequency graph as an example: after the single-branch feature extraction module receives the forward time-frequency graph, the forward time-frequency graph first performs a 2D convolution maximum pooling operation, then performs an average pooling operation of a 2D convolution on the result of the previous step, then performs a 2D convolution conversion on the result of the previous step, then performs an average pooling operation of a 2D convolution on the result of the previous step, then performs a 2D convolution conversion on the result of the previous step, and finally performs an average pooling operation on the result of the previous step to obtain the final operation result of the single-branch feature extraction module.
[0077] The lateral time-frequency diagram is used as input and the final calculation result of the single-branch feature extraction module is obtained through the same process. Finally, the final calculation results of the two single-branch feature extraction modules are spliced and the spliced calculation results are input into the softmax layer, that is, the classifier to judge the fall behavior. In this embodiment, the value output by the softmax layer is a probability value. When the output probability value is greater than 0.5, it is judged as a fall behavior, and when it is less than 0.5, it is judged as a non-fall behavior.
[0078] In this embodiment, during the process of detecting indoor human activities, a Kinect camera is also placed indoors, and simulated radar data obtained based on human skeleton data simulation is obtained through the Kinect camera. In this way, the process of detecting human activities will not be affected by environmental factors.
[0079] After the basic samples, that is, the original radar data and the complex simulated radar data obtained based on the human skeleton data, are collected, the data set is still a small sample data set, and the data volume is not extensive enough. Therefore, in this embodiment, the human activity detection model with the original radar data and the complex simulated radar data as samples is trained through a meta-learning process, so that the human activity detection model can be applied to human activity detection with a small sample data set.
[0080] In the present invention, the meta-learning process is performed based on the original radar data and the expanded simulated radar data to complete the training of the human activity detection model, which specifically includes the following steps:
[0081] Step C1: collect data and preprocess the collected data to obtain a data set D.
[0082] Specifically, it includes collecting original radar data and simulated radar data, and performing preprocessing operations such as cleaning, standardization, and enhancement on the collected original radar data and simulated radar data to obtain a data set D. The data set D contains multiple human motion categories, including falling motion and daily motion categories.
[0083] Step C2: Establish a meta-learning framework and initialize the meta-learning framework parameters.
[0084] Specifically, the established meta-learning framework is the Model-Agnostic Meta-Learning (MAML) meta-learning framework.
[0085] Step C3: Divide and construct meta-tasks.
[0086] Specifically, the fall detection task is divided into M meta-tasks, and the total meta-task set is represented as: {TA1,TA2,⋯,TA M}, different meta-tasks involve fall detection in different environments and among different populations.
[0087] Each meta-task includes a training phase and a testing phase, including its own training set and test set, and both the training set and the test set contain a Support Set and a Query Set. All training sets and test sets are randomly sampled from the dataset D.
[0088] Step C4: Perform meta-learning training.
[0089] Specifically, the steps include:
[0090] Step C401: Initialize the parameters of the human activity detection model and record them as Φ 0 , and set the learning rate to α mate .
[0091] Step C402: Perform inner loop training. For each meta-task, use the training data to update the model parameters through the gradient descent method, calculate the task loss function, and update the model parameters through back propagation.
[0092] In this embodiment, the meta-task TA m For example, the process of inner loop training is as follows:
[0093] Step Z1: Based on TA m Support Set, use the corresponding Support learning rate a m About TA m MAML network parameters Make updates;
[0094] Step Z2: Use TA m Query Set based on the updated Calculate TA m The query loss (denoted as ), and use the learning rate α mate The human activity detection model parameters (i.e. Φ m-1 ) to update;
[0095] Step Z3: Use the support set of the test task to update Φ m-1 Perform fine-tuning and then use the query set of the test task to evaluate the effect of meta-learning;
[0096] Step Z4: Repeat steps Z1 to Z3 until the parameters of the human activity detection model are updated M times, and obtain
[0097] Φ m .
[0098] Step C403: Perform outer loop training through M query losses, i.e. aggregation Calculate the overall meta-loss and update Φ based on it m .
[0099] Step C404: Repeat steps C402 to C403 until the training of the human activity detection model is completed.
[0100] At this point, the training of the human activity detection model is completed. By simulating radar data, the sample database is supplemented and increased, and the training of the human activity detection model for a small sample database is completed through the constructed meta-learning process.
[0101] The following is an application example of the human activity detection method based on the meta-learning process with self-expanded radar data provided by the present invention:
[0102] In this embodiment, the number of original samples collected by the installed ultra-wideband impulse radar is 446, which are used to generate simulated radar data and expand the total radar data. The total radar data consists of original radar data and simulated radar data, totaling 6850.
[0103] In this embodiment, each type of action in the collected data set is divided into 3:1 according to practical experience, wherein the training samples account for 75% of each type of data set, and the test samples account for 25% of the remaining data set.
[0104] In this embodiment, a large-scale database is also built by self-sampling. By constructing a real experimental scene, the experimental scene includes a forward radar and a side radar, and the forward radar and the side radar are placed 5 meters in front and to the side of the experimenter. The collected forward radar time-frequency diagram and side radar time-frequency diagram are used as input. The experimental results of the human activity detection model network performance comparison are shown in the following table:
[0105]
[0106] It can be seen from the table that: using the self-collected database, the network model proposed in the present invention has a 1.87% higher recognition rate in human fall detection compared with the two-dimensional feature fusion network; compared with the feature fusion CNN network, the recognition rate is increased by 0.5%; compared with the single-branch CNN network, the recognition rate is increased by 1.03%, but the time is increased by 0.022s; compared with the two-branch CNN network, the recognition rate is increased by 0.76% but the time is increased by 0.005s; compared with the single-branch DenseNet network, the recognition rate is increased by 0.87%.
[0107] In order to fully demonstrate the performance of this method, in addition to detecting falling behaviors, this example also performs detection on another 5 daily behaviors. The detection method and steps are the same as those for detecting falling actions. Specific daily behaviors include: slight movement, lying down, getting up, turning over, and waving. The number of original samples collected by Kinect is 885, which are used to generate simulated radar data and to expand the total radar data. The total radar data consists of original radar data and simulated radar data, totaling 16722. Similarly, each type of action in the collected data set is divided into 3:1, where the training samples account for each type of data set.
[0108] The test samples account for 75% of the remaining data set. The experimental results of the performance comparison of different network models are shown in the following table:
[0109]
[0110] According to the content of the table, it can be concluded that the self-collected database collected by the present invention performs well on the double-branch DenseNet network model, and can effectively improve the recognition rate of human falling actions and other daily actions collected by radar.
[0111] The present invention uses a radar sensor as a perception terminal for acquiring human behavior information, and uses a radar echo signal of human skeleton data obtained by the radar sensor in combination with a kinect camera as simulated radar data. The original radar data and the simulated radar data obtained by the radar sensor are input into a human activity detection model, and the training of the human activity detection model is completed by constructing a meta-learning framework and a meta-learning process. The present invention can be applied to the case where the detected samples are insufficient. By combining the radar sensor with a kinect camera, the data set can be supplemented automatically without the input of new original radar data, and the radar data can be self-expanded, thereby shortening the training time of the neural network model.
[0112] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.
Claims
1. A method for detecting human activity by self-expanding radar data, characterized in that: The steps include: Step S1: collecting indoor personnel behavior detection samples, wherein the collection includes collecting at least two groups of detection samples in mutually perpendicular directions in a plane; Step S2: The collected behavior detection samples are used as inputs in different directions and input into a human activity detection model. The human activity detection model is a neural network model and is internally equipped with a dual-branch feature extraction module for distinguishing human action categories. The dual-branch feature extraction module is used to extract features of different inputs respectively, and correspondingly obtains different features of the same number as the input; Step S3: The obtained different features are spliced and input into the classifier for decision making, and the classifier is used to determine the human action category to which the behavior represented by the fused input belongs; The human activity detection model is trained based on a local data set, wherein the local data set includes original radar data and simulated radar data, wherein the original radar data is obtained by radar sampling, and the simulated radar data is specifically expanded radar data obtained by simulating human skeleton data collected by a kinect camera; The human activity detection model is meta-learned and trained based on the local dataset. The steps of the meta-learning process are as follows: Step C1: collecting original radar data and simulated radar data, and preprocessing the collected original radar data and simulated radar data to obtain a data set; Step C2: Establish a meta-learning framework and initialize the meta-learning framework parameters; Step C3: construct and divide meta-tasks based on the data set; In step C1, the preprocessing step for the original radar data and the simulated radar data includes preprocessing operations of cleaning, standardizing and enhancing the original radar data and the simulated radar data; In step C3, after the meta-task is constructed, the meta-task is divided based on human activity detection in different environments and for different groups of people. The fall detection task is divided into M meta-tasks. The total meta-task set is expressed as: {TA1,TA2,TA m ,…,TA M }, where m represents the number of meta-tasks, and 1<m<M, and each meta-task includes a training phase and a test phase and a corresponding training set and test set obtained based on the data set; Step C4: Performing inner-loop meta-learning training on the meta-task after planning is completed until the training of the human activity detection model is completed; Step C4: Perform meta-learning training, including the following steps: Step C401: Initialize the parameters of the human activity detection model and record them as Φ 0 , and set the learning rate to α mate ; Step C402: Perform inner loop training. For each meta-task, use the training data to update the model parameters through the gradient descent method, calculate the task loss function, and update the model parameters through back propagation; Meta-task TA m For example, the process of inner loop training is as follows: Step Z1: Based on TA m Support Set, use the corresponding Support learning rate a m About TA m MAML network parameters Make updates; Step Z2: Use TA m Query Set based on the updated Calculate TA m The query loss is recorded as , and use the learning rate α mate The human activity detection model parameters Φ obtained by training m-1 meta-tasks m-1 Make updates; Step Z3: Use the support set of the test task to update Φ m-1 Perform fine-tuning and then use the query set of the test task to evaluate the effect of meta-learning; Step Z4: Repeat steps Z1 to Z3 until the parameters of the human activity detection model are updated M times, and obtain F m ; Step C403: Perform outer loop training through M query losses, i.e. aggregation Calculate the overall meta-loss and update Φ based on it m ; Step C404: Repeat steps C402 to C403 until the training of the human activity detection model is completed.
2. The method for detecting human activity with self-expandable radar data according to claim 1, characterized in that: The acquisition of the simulated radar data comprises the following steps: Step A1: Building a sample collection scene, including building an indoor venue, selecting a target person, installing at least two sets of ultra-wideband impulse radars in the indoor venue in perpendicular directions in a plane, installing a wearable device on the target person, and installing a kinect camera in the indoor venue; Step A2: The wearable device maps the coordinates corresponding to different joints of the target person in the Kinect camera coordinate system, and uses each different ultra-wideband impulse radar as the coordinate origin to transform the coordinates corresponding to different joints of the target person in the Kinect camera coordinate system, and the coordinates form the human skeleton data; Step A3: using the human skeleton data to calculate corresponding radar echo signals of multiple frames of human skeleton data; Step A4: performing short-time Fourier transform on multiple frames of human skeleton data radar echo signals respectively to obtain time-frequency diagrams in corresponding directions as simulated radar data.
3. The method for detecting human activity with self-expandable radar data according to claim 2, characterized in that: In step A3, the process of calculating radar echo signals of multiple frames of human skeleton data includes calculating radar echo signals of human joints at different radar frequencies in each frame collected by a Kinect camera.
4. The method for detecting human activity with self-expandable radar data according to claim 3, characterized in that: The implementation of step A3 specifically includes the following steps: Step A301: Calculate the human target signals of all human joints at different radar frequencies and obtain the human target signals; Step A302: Calculate the noise impact in the environment, including the calculation of thermal noise and environmental noise in the environment; Step A303: combining the human target signal and the noise signal calculated in the same frame to obtain the radar echo signal in the same frame; Step A304: looping through step A301 to step A303 to obtain multiple frames of human skeleton data radar echo signals corresponding to ultra-wideband impulse radars in multiple different directions.
5. The method for detecting human activity capable of self-expanding radar data according to claim 1, characterized in that: In step S2, the steps of extracting features of different inputs by the dual-branch feature extraction module are as follows: The single-branch feature extraction module receives the input and obtains the corresponding input features through the convolution layer, pooling layer and softmax layer in sequence.
6. The method for detecting human activity capable of self-expanding radar data according to claim 1, characterized in that: In step S1, the sampling process of indoor personnel behavior samples includes the forward sample collection of personnel behavior and the lateral sample collection of personnel behavior.
Citation Information
Patent Citations
Neural network recognition model training method and device, server and storage medium
CN111401180A
Radar human body action recognition method and device, electronic equipment and storage medium
CN112241001A
CNN and radar image-based human body action and identity multi-task identification method
CN112364689A
Human body action recognition method based on deep learning and ultra wide band radar
CN113850204A
Human face action unit detection method based on meta-learning
CN114708637A
