A domain-adaptive method for human posture recognition using through-wall radar

By using convolutional neural network model and adversarial learning method in wall-passing radar human posture recognition, the problem of decreasing recognition accuracy in different environments is solved, high-performance recognition in different environments is achieved, and recognition accuracy and generalization are improved. ability to transform.

CN113869397BActive Publication Date: 2024-11-08AEROSPACE INFORMATION RES INST CAS
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111127646.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-26
Publication Date
2024-11-08
Estimated Expiration
2041-09-26

AI Technical Summary

Technical Problem

The existing wall-through radar human posture recognition method has decreased in different environments, and there is a lack of methods that can generalize to various testing environments. The statistical-based method has low accuracy and depends on humans, and the deep learning-based method has in the environment. Performance deteriorates when changes.

Method used

Using a convolutional neural network model, combining feature extraction module, classifier and discriminator, migrate information between the source domain and the target domain through adversarial learning methods, realizing domain-adaptive wall-through radar human posture recognition, and using loss function to optimize features Extract module and discriminator parameters to maintain high recognition performance in different environments.

Benefits of technology

It realizes the high wall-through radar human posture recognition performance in different test environments, can effectively identify source and target domain data, reduce manual intervention, and improve recognition accuracy and generalization capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113869397B_ABST
    Figure CN113869397B_ABST
Patent Text Reader

Abstract

The present invention relates to a domain adaptive through-wall radar human posture recognition method, comprising the following steps: step 1, collecting source domain data in a first environment, making a source data set, and marking it; collecting target domain data in a second environment, making a target data set; step 2, sending data in the source data set into a convolutional neural network model, extracting features containing human posture information through a feature extraction module, sending them into a classifier and classifying them into different posture categories, and updating parameters of the feature extraction module and the classifier; step 3, sending data in the source data set and the target data set into the model, respectively extracting features of the source data set and features of the target data set through a feature extraction module, inputting the features into a discriminator, obtaining a prediction result of the domain category, and updating parameters of the feature extraction module and the discriminator; step 4, sending data in the target data set into a neural network model, extracting features containing human posture information and sending them into the classifier, and outputting the posture category of the human target.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of radar technology, and in particular to a domain adaptive through-wall radar human body posture recognition method. Background Art

[0002] Human posture recognition methods aim to identify different types of human target postures (such as standing, sitting, squatting, etc.), and are an important part of human perception tasks. Most of the existing human posture recognition methods are designed based on optical systems, which can directly recognize the posture of human targets from photos or videos. However, optical systems such as cameras cannot solve the occlusion problem. When obstacles (such as walls) block human targets, the recognition method cannot work properly. The through-wall radar system can penetrate non-metallic obstacles and detect human targets behind them. If a human posture recognition method can be designed for the through-wall radar system, it will have a significant impact on rescue, security inspection, national defense and other fields.

[0003] At present, the human posture recognition methods designed for through-wall radar systems can be divided into two types: statistical-based methods and deep learning-based methods. The statistical through-wall radar human posture recognition method refers to extracting statistical features (such as echo signal strength, density, etc.) from the radar echo signal, and manually distinguishing the posture of the human target based on these statistical features; this type of method relies on manually designed statistical features and requires manual judgment, which is time-consuming and labor-intensive, and the recognition accuracy is not high. Deep learning is a type of data-driven machine learning method. Due to its powerful feature extraction capabilities, it has been widely used in computer vision, natural language processing and other fields. The through-wall radar human posture recognition method based on deep learning refers to using representative networks such as convolutional neural networks to automatically extract features from radar echo signals, and use these features for automatic recognition and classification. This type of method reduces manual intervention and achieves a higher recognition accuracy.

[0004] However, most of these deep learning-based methods can only achieve high recognition accuracy in the environment where their training sets are collected. When they are applied to a new test environment, the recognition accuracy will drop significantly due to differences such as different occluder materials, ambient temperature changes, and different sensor parameters. The reason for this phenomenon is domain shift, that is, the distribution between the domain where the data collected in the training set is located and the domain where the test data is located is not exactly the same. The data space where the data collected during training is located is called the source domain, and the data space where the data collected during testing is called the target domain. When the network trained in the source domain is directly applied to the target domain, performance will be degraded, and repeated training in the target domain will consume additional time and resources.

[0005] There are two widely used human posture recognition methods for through-wall radar systems. Both methods have some defects, which can be summarized as follows:

[0006] (1) Statistical recognition methods can distinguish different human postures, but such methods usually require manual construction of statistical features and manual recognition, resulting in low recognition accuracy and slow speed.

[0007] (2) The recognition method based on deep learning can automatically extract features and has higher recognition accuracy and speed, but its generalization ability is poor and its performance degrades when applied to new test environments;

[0008] (3) There is still a lack of suitable through-wall radar human posture recognition methods that can be generalized to a variety of different test environments and maintain good performance. Summary of the invention

[0009] Based on the above viewpoints, the present invention considers improving the existing method and migrating the information of the source domain to the target domain to realize a domain adaptive through-wall radar human posture recognition method. Specifically, a convolutional neural network model is established, which consists of three parts: a feature extraction module, a classifier and a discriminator.

[0010] The feature extraction module is responsible for extracting human posture information from radar signals, and the classifier is responsible for classifying the extracted features into corresponding posture categories.

[0011] Due to the different distributions between different domains, the features extracted from the source domain and the features extracted from the target domain are usually quite different. Through the adversarial learning method, the discriminator is introduced to perform adaptive domain adjustment, so that the features extracted from different domains by the feature extraction module fall into a similar feature space. Through supervised training on labeled source domain datasets, the model can predict the human posture categories in unlabeled target domain datasets.

[0012] The technical solution of the present invention is: a domain adaptive through-wall radar human posture recognition method, comprising the following steps:

[0013] Step 1: Collect source domain data in the first environment, create a source data set, and annotate it; collect target domain data in the second environment and create a target data set;

[0014] Step 2: Send the data in the source data set into the convolutional neural network model, extract the features containing human posture information through the N-layer feature extraction module, send these features into the classifier and classify them into different posture categories, and update the parameters of the feature extraction module and the classifier by using the back propagation algorithm on the loss function;

[0015] Step 3: Input the data in the source dataset and the target dataset into the model, extract the features of the source dataset and the target dataset respectively through the feature extraction module, input these features into the discriminator, obtain the prediction results of the domain category, and use the back propagation algorithm to update the parameters of the discriminator and the feature extraction module;

[0016] Step 4: After the model training is completed, the data in the target data set is fed into the neural network model, and the features containing human posture information are extracted through the feature extraction module. After these features are fed into the classifier, the posture category of the human target can be output.

[0017] Furthermore, the step 1, collecting source domain data in the first environment, making a source data set, and annotating it; collecting target domain data in the second environment, making a target data set; specifically includes:

[0018] Collect source domain data in an obstacle-free environment: human targets pose in different postures, radar collects and receives signals, collects these signals and creates source data sets; pre-specify k types of target action categories and label these data, that is, specify the human posture category corresponding to each radar signal;

[0019] Collect target domain data in an environment with wall obstruction: human targets pose in different postures, radar collects and receives signals, collects these signals and creates a target data set.

[0020] Furthermore, in step 2, the data in the source data set is fed into the convolutional neural network model, and features containing human posture information are extracted through an N-layer feature extraction module. These features are fed into a classifier to classify them into different posture categories, and the parameters of the feature extraction module and the classifier are updated by using a back propagation algorithm on the loss function;

[0021] The data in the source data set is fed into the convolutional neural network model, and features containing human posture information are extracted through the N-layer feature extraction module; these features are fed into the classifier and classified into different posture categories; since the source data set has been labeled, the supervised learning method is used to construct the labeled category y and the model's predicted category The loss function between them is the posture category loss function L 1 , by using the back propagation algorithm on the loss function, update the parameters of the feature extraction module and the classifier, record N s is the total number of samples in the source data set, I is an indicative function, which is 1 when the condition is met and 0 otherwise; then L 1 It is expressed as:

[0022]

[0023] c and i are intermediate variables, indicating the prediction result of the current processing i-th sample for the c-th job category, k is the total number of specified categories, and y i Represents the predicted category of the i-th sample, y ic Represents the score of the cth category output by the ith sample, i.e., the confidence level.

[0024] Furthermore, the step three, inputting the data in the source data set and the target data set into the model, extracting the features of the source data set and the features of the target data set respectively through the feature extraction module, inputting the features into the discriminator, obtaining the prediction result of the domain category, and updating the parameters of the discriminator and the feature extraction module using the back propagation algorithm, specifically includes:

[0025] The data in the source dataset and the target dataset are fed into the model, and the features of the source dataset are extracted by the feature extraction module. s and the features f of the target dataset t , input this feature into the discriminator to obtain the prediction result of the domain category; for the discriminator, it is hoped that it can correctly distinguish the feature f of the source data set s and the features f of the target dataset t , construct the loss function L 2 ; Note N t is the total number of samples in the target data set, Disc() represents the output of the discriminator, then L 2 It is expressed as:

[0026]

[0027] n is an intermediate variable, indicating that the nth feature is currently being processed. For the feature extraction module, it is hoped that the feature distribution of the target dataset extracted by it is close enough to the feature distribution of the source dataset so that the discriminator can t Misclassification, construct loss function L 3 :

[0028]

[0029] Alternately optimize the loss function L according to the adversarial learning method 2 And the loss function L 3 , so that it reaches Nash equilibrium, and uses the back propagation algorithm to update the parameters of the discriminator and feature extraction module. By repeating steps one and two, the neural network model can accurately classify the data in the source data set, and the model can extract features from the source data set and the target data set with similar distribution.

[0030] Beneficial effects:

[0031] (1) The present invention proposes a method for domain adaptive posture recognition of a human body behind a wall using a through-wall radar, which achieves the effect of posture recognition of the collected through-wall radar human body echo data, and can be applied to practical tasks such as safety detection and disaster relief;

[0032] (2) The present invention proposes to use a domain adaptive method to process the through-wall radar human echo signal, which can be generalized to various test environments and maintain good performance;

[0033] (3) The method of the present invention uses the recognition loss L 1 and adversarial loss L 2 ,L 3 As the optimization goal, through reasonable network model design, the method can not only effectively identify human posture in the source domain dataset, but also extract accurate posture prediction from the target domain dataset that has not been manually annotated. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Schematic diagram of the proposed through-wall radar human posture recognition model structure;

[0035] Figure 2 Schematic diagram of the feature extraction module structure;

[0036] Figure 3 Schematic diagram of the classifier module structure;

[0037] Figure 4 Schematic diagram of the discriminator module structure;

[0038] Figure 5 The source domain dataset (left) and target domain dataset (right) collection process are shown. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.

[0040] According to an embodiment of the present invention, a domain adaptive through-wall radar human posture recognition method is proposed, comprising the following steps:

[0041] Step 1: Collect source domain data in the first environment, create a source data set, and annotate it; collect target domain data in the second environment and create a target data set; specifically include:

[0042] Source domain data is collected in an experimental environment (such as an environment without obstacles): the human target poses in different postures, the radar collects and receives signals, these signals are collected and a source data set is produced; k types of action categories of the target are specified in advance, and these data are labeled, that is, the human posture category corresponding to each radar signal is specified.

[0043] In another experimental environment (such as an environment with wall obstruction), target domain data is collected: the human target poses in different postures, the radar collects and receives signals, and these signals are collected to create a target data set.

[0044] Construct a convolutional neural network model. The structural diagram of the model is as follows Figure 1 As shown in the figure, the whole model consists of three parts: feature extraction module, classifier and discriminator. The structural diagrams of the three parts are shown in Figure 2 , Figure 3 , Figure 4 shown.

[0045] The feature extraction module consists of N identical submodules, each of which consists of a convolutional layer and a fully connected layer, followed by a normalization layer. The classifier and discriminator have similar structures, both consisting of an average pooling layer and two fully connected layers.

[0046] Step 2: Send the data in the source data set into the convolutional neural network model, extract the features containing human posture information through the N-layer feature extraction module, send these features into the classifier and classify them into different posture categories, and update the parameters of the feature extraction module and classifier by using the back propagation algorithm on the loss function.

[0047] like Figure 1 As shown in the figure (arrows are divided into thick arrows, normal arrows and thin arrows according to thickness), the data in the source data set is fed into the convolutional neural network model, and features containing human posture information are extracted through the N-layer feature extraction module. These features are fed into the classifier and classified into different posture categories. Since the source data set has been labeled, the supervised learning method is used to construct the labeled category y and the model's predicted category The loss function between them is the posture category loss function L 1 By using the back propagation algorithm on the loss function, the parameters of the feature extraction module and the classifier are updated. s is the total number of samples in the source data set, and I is an indicative function, which is 1 when the condition is met and 0 otherwise. Then L 1 It is expressed as:

[0048]

[0049] c and i are intermediate variables, indicating the prediction result of the current processing i-th sample for the c-th job category, k is the total number of specified categories, and yi Represents the predicted category of the i-th sample, y ic Represents the score of the cth category output by the ith sample, i.e., the confidence level.

[0050] Step 3: Send the data in the source dataset and the target dataset into the model, extract the features of the source dataset and the target dataset respectively through the feature extraction module, input the features into the discriminator, obtain the prediction results of the domain category, and use the back propagation algorithm to update the parameters of the discriminator and the feature extraction module.

[0051] like Figure 1 As shown by the normal arrows in the figure, the data in the source dataset and the target dataset are fed into the model, and the features of the source dataset are extracted by the feature extraction module. s and the features f of the target dataset t This feature is input into the discriminator to obtain the prediction result of the domain category. For the discriminator, it is hoped that it can correctly distinguish the feature f of the source data set. s and the features f of the target dataset t , construct the loss function L 2 . Remember N t is the total number of samples in the target data set, Disc() represents the output of the discriminator, then L 2 It is expressed as:

[0052]

[0053] n is an intermediate variable, indicating that the nth feature is currently being processed. For the feature extraction module, it is hoped that the feature distribution of the target dataset extracted by it is close enough to the feature distribution of the source dataset so that the discriminator can t 0 misclassification. Construct loss function L 3 :

[0054]

[0055] Alternately optimize the loss function L according to the adversarial learning method 2 And the loss function L 3 , so that it reaches Nash equilibrium, and uses the back propagation algorithm to update the parameters of the discriminator and feature extraction module. By repeating steps one and two, the neural network model can accurately classify the data in the source data set, and the model can extract features from the source data set and the target data set with similar distribution.

[0056] Step 4: Figure 1 As shown by the thinner arrow in the figure, after the model training is completed, the data in the target data set is sent to the neural network model, and the features containing human posture information are extracted through the feature extraction module. After these features are sent to the classifier, the posture category of the human target can be output.

[0057] According to a specific embodiment of the present invention, the domain adaptive through-wall radar human posture recognition method of the present invention is applied to perform human posture recognition to recognize the human posture behind the wall. The details are as follows:

[0058] 1. The source domain dataset was collected in an open space without wall obstruction, and the target domain dataset was collected in a narrow space with wall obstruction. The human targets posed in several postures during the collection. The size of the collected radar echo signal is (2048, 32), and k = 5 different human postures are manually defined (standing, sitting, squatting, lying horizontally with the radar normal, lying vertically with the radar normal, etc.), and the source domain dataset is manually labeled accordingly; the collection process of the two datasets is as follows Figure 5 shown.

[0059] 2. The feature extraction network uses a 3-layer structure (N=3) to balance the computational complexity and computational speed. The classifier network and the discriminator network are the same in structure. The output length of the classifier network is 5, which is used to distinguish different posture categories. The output length of the discriminator network is 2, which is used to distinguish whether the data comes from the source domain or the target domain. The specific structure of the entire network is shown in Table 1.

[0060] Table 1 Convolutional neural network structure table for example training

[0061]

[0062] 3. The total number of training rounds is 1000. The Adam algorithm is used for gradient update. The initial learning rate is set to 0.0001 and decays by 5% every 10 rounds. The rest of the hyperparameters use the default values. The entire system is implemented using the Pytorch framework, and an RTX3090 is used in the training and testing process.

[0063] 4. After the training is completed, the radar signal is actually collected in the target domain for testing. Table 2 shows the recognition accuracy in the source domain and the target domain. Even if there is no manual labeling in the target domain, the recognition accuracy can reach 83.53%.

[0064] Table 2 Test accuracy in two data domains after instance training

[0065] Data Field source domain Target domain Accuracy 89.60% 83.53%

[0066] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, and it should be clear that the present invention is not limited to the scope of the specific embodiments, for those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A domain adaptive through-wall radar human posture recognition method, It is characterized in that The steps include: Step 1: Collect source domain data in the first environment, create a source data set, and annotate it; collect target domain data in the second environment and create a target data set; Step 2: Send the data in the source data set into the convolutional neural network model, extract the features containing human posture information through the N-layer feature extraction module, send these features into the classifier and classify them into different posture categories, and update the parameters of the feature extraction module and the classifier by using the back propagation algorithm on the loss function; Step 3: Input the data in the source dataset and the target dataset into the model, extract the features of the source dataset and the target dataset respectively through the feature extraction module, input the features into the discriminator, obtain the prediction results of the domain category, and use the back propagation algorithm to update the parameters of the discriminator and the feature extraction module; Step 4: After the model training is completed, the data in the target data set is fed into the neural network model, and the features containing human posture information are extracted through the feature extraction module. After these features are fed into the classifier, the posture category of the human target is output; In the step 2, the data in the source data set is fed into a convolutional neural network model, and features containing human posture information are extracted through an N-layer feature extraction module, and these features are fed into a classifier to classify them into different posture categories, and the parameters of the feature extraction module and the classifier are updated by using a back propagation algorithm on the loss function; The data in the source dataset is fed into the convolutional neural network model, and features containing human posture information are extracted through the N-layer feature extraction module; these features are fed into the classifier and classified into different posture categories; since the source dataset has been labeled, the supervised learning method is used to construct a loss function between the labeled category y and the model's predicted category ŷ, that is, the posture category loss function , by using the back propagation algorithm on the loss function, update the parameters of the feature extraction module and the classifier, record N s is the total number of samples in the source data set, I is an indicative function, which is 1 when the condition is met and 0 otherwise; It is expressed as: , c and i are intermediate variables, indicating the prediction result of the current processing i-th sample for the c-th job category, k is the total number of specified categories, and y i Represents the predicted category of the i-th sample, y ic Represents the score of the cth category output by the ith sample, i.e., the confidence level; The step three, inputting the data in the source data set and the target data set into the model, extracting the features of the source data set and the features of the target data set respectively through the feature extraction module, inputting the features into the discriminator, obtaining the prediction results of the domain category, and updating the parameters of the discriminator and the feature extraction module using the back propagation algorithm, specifically includes: The data in the source dataset and the target dataset are fed into the model, and the features of the source dataset are extracted through the feature extraction module. and the characteristics of the target dataset , input this feature into the discriminator to obtain the prediction result of the domain category; for the discriminator, it is hoped that it can correctly distinguish the features of the source data set and the characteristics of the target dataset , construct the loss function ; Note N t is the total number of samples in the target data set, Disc() represents the output of the discriminator, then It is expressed as: , n is an intermediate variable, indicating that the nth feature is currently being processed. For the feature extraction module, it is hoped that the feature distribution of the target dataset extracted by it is close enough to the feature distribution of the source dataset so that the discriminator can Misclassification, constructing loss function : , Alternately optimize the loss function according to the adversarial learning method And the loss function , so that it reaches Nash equilibrium, and uses the back propagation algorithm to update the parameters of the discriminator and feature extraction module. By repeating steps one and two, the neural network model can accurately classify the data in the source data set, and the model can extract features from the source data set and the target data set with similar distribution.

2. According to the domain adaptive through-wall radar human posture recognition method of claim 1, It is characterized in that The step 1 is to collect source domain data in the first environment, create a source data set, and annotate it; In the second environment, target domain data is collected to create a target data set, including: Collect source domain data in an obstacle-free environment: human targets pose in different postures, radar collects and receives signals, collects these signals and creates source data sets; pre-specify k types of target action categories and label these data, that is, specify the human posture category corresponding to each radar signal; Collect target domain data in an environment with wall obstruction: human targets pose in different postures, radar collects and receives signals, collects these signals and creates a target data set.