Method for detecting falling of personnel by millimeter wave radar based on partition parameter adjustment CNN model

Abstract

The present invention relates to posture recognition technology, which discloses a method for detecting a person falling by using a millimeter-wave radar based on a partitioned parameter-adjusted CNN model, which solves the problem that the echo energy of the millimeter-wave radar is uneven in different areas, resulting in low robustness of point cloud data in different areas, thus affecting the accuracy of fall detection. The method includes: S1, acquiring the point cloud data of the millimeter-wave radar and performing preprocessing to obtain the point cloud track of the tracking target; S2, normalizing the point cloud track of the tracking target, and judging the sub-area where the detection target is located in the detection area based on the target point cloud track; S3, calling the corresponding model parameters according to the sub-area where the tracking target is located in the detection area, and using the pre-trained person fall detection model to perform person fall posture detection according to the normalized point cloud data.

Description

Technical Field

[0001] The present invention relates to a posture recognition technology, and in particular to a method for detecting a person falling by using a millimeter wave radar based on a partition parameter adjustment CNN model. Background Art

[0002] With the improvement of medical conditions and the improvement of living standards, the prospect of millimeter-wave radar in the home market is in the ascendant, especially in today's society where the problem of population aging is aggravated. Millimeter-wave radar has unprecedented development prospects for the problem of indoor personnel, especially the elderly falling. Compared with other posture recognition technologies (infrared recognition, image recognition, force sensor recognition, etc.), the use of millimeter-wave radar to achieve indoor personnel posture detection is gradually expanding its civilian application prospects and playing an increasingly important role due to its unique advantages such as small size, light weight, low cost, high precision, high resolution, and strong anti-interference ability.

[0003] Due to the performance limitations of the millimeter-wave radar hardware system, the radar signal's own echo signal intensity decays with increasing distance and angle, and this attenuation has no obvious regularity, resulting in inconsistent target information content represented by the radar point cloud data generated in different areas, which in turn affects the accuracy of fall detection. Summary of the invention

[0004] The technical problem to be solved by the present invention is: to propose a method for detecting people falling by using millimeter-wave radar based on a partitioned parameter-adjusted CNN model, so as to solve the problem that the echo energy of the millimeter-wave radar is uneven in different areas, resulting in low robustness of point cloud data in different areas, thus affecting the accuracy of fall detection.

[0005] The technical solution adopted by the present invention to solve the above technical problems is:

[0006] The method for detecting a person falling by using a millimeter wave radar based on a partitioned parameter adjustment CNN model includes the following steps:

[0007] S1, obtain the point cloud data of the millimeter wave radar and perform preprocessing to obtain the point cloud track of the tracking target;

[0008] S2, normalizing the tracked target point cloud track, and judging the sub-area where the detected target is located in the detection area based on the target point cloud track;

[0009] S3. According to the sub-area where the tracking target is located in the detection area, the corresponding model parameters are called, and the pre-trained personnel fall detection model is used to perform personnel fall posture detection based on the normalized point cloud data.

[0010] Furthermore, in step S1, the preprocessing includes: Fourier transform in distance dimension, Fourier transform in Doppler dimension, dynamic point cloud detection, static clutter elimination, Fourier transform in angle dimension, point cloud clustering and tracking.

[0011] Furthermore, in step S2, normalizing the tracked target point cloud track includes:

[0012] Normalize all point clouds of the target tracked in the current time frame according to each dimension:

[0013]

[0014] Among them, x i Represents the value of a certain dimension of the i-th point, represents the normalized result, u is the mean of the corresponding dimension, σ is the standard deviation of the corresponding dimension, ε is a very small number to prevent the divisor from being 0, and N is the number of target point clouds;

[0015] The various dimensions include: distance dimension, azimuth dimension, Doppler dimension and elevation dimension.

[0016] Furthermore, in step S2, the sub-areas are divided in the following manner: taking the position of the millimeter-wave radar as the origin of the system absolute coordinate system, the detection area is divided into 10 segments along the x-axis and the y-axis to form corresponding sub-areas.

[0017] Further, in step S2, judging the sub-area where the tracked target is located in the detection area based on the tracked target point cloud track specifically includes:

[0018] Assume that the tracking point cloud trajectory contains N points, and the position of each point is expressed as follows:

[0019]

[0020] The N-dimensional correlation coefficient matrix P of the target points is obtained as:

[0021]

[0022] Where Cov(X,Y) represents the covariance matrix of the position coordinates X and Y, and D(X) and D(Y) represent the variance of the position coordinates X and Y respectively;

[0023] The final regional matrix W of the sub-region where the tracking target is located is:

[0024] W=APA T

[0025] Among them, A is a 10*N matrix, which is used to represent the distribution state of the point cloud of the tracking target in the area.

[0026] Furthermore, in step S3, when the person falling posture detection is performed based on the normalized point cloud data, the input data is obtained in the following manner:

[0027] For the normalized point cloud data, retain 10 consecutive frames of radar point cloud data, sort them by the point cloud height of the tracked target in the absolute coordinate system with the radar position as the origin, select the position coordinates and Doppler information of the highest 4 points and the lowest 4 points in the height direction, and add the average position coordinates of the tracked target point cloud track to form an input data of size 10*35.

[0028] Furthermore, in step S3, the personnel fall detection model is obtained based on CNN network training, and the CNN network includes: a first one-dimensional convolutional layer, a first normalization layer, a second one-dimensional convolutional layer, a second normalization layer, a third one-dimensional convolutional layer, a third normalization layer, a first fully connected layer, a fourth normalization layer, a second fully connected layer and a fifth normalization layer connected in sequence.

[0029] Furthermore, in the CNN network, the number of convolution kernels of the first one-dimensional convolution layer is 42, the size is 3, and the step size is 1; the number of convolution kernels of the second one-dimensional convolution layer is 28, the size is 3, and the step size is 1; the number of convolution kernels of the third one-dimensional convolution layer is 14, the size is 3, and the step size is 1; the first normalization layer, the second normalization layer, the third normalization layer, and the fourth normalization layer all use the relu function as the activation function; the fifth normalization layer uses the softmax function as the activation function.

[0030] Furthermore, during the training process of the personnel fall detection model, the regional matrix W is used as a weight to add the cross entropy as the loss function L of the model training, and the formula is as follows:

[0031]

[0032] Where M is the total number of samples in a batch, K is the number of model classification categories, T is the true label of the corresponding sample, Z is the calculated output value of the model, q is the parameter to be identified during the model training process, and Q is the set of all parameters to be identified;

[0033] After training, the model parameters for each sub-region are obtained.

[0034] Furthermore, in step S3, the detection of a person's falling posture includes two judgment conditions: ① the average height of the tracking target point cloud is lower than a certain threshold; ② the model classification output result is judged as a falling behavior.

[0035] The beneficial effects of the present invention are:

[0036] (1) Using the millimeter-wave radar point cloud data in a continuous time series, a CNN model with regional parameter adjustment is constructed. The fall posture detection of people is performed according to the indoor environment. The CNN model is configured using the model parameter identification results in different regions. In practical applications, the application of this model for fall detection can determine the partition according to the tracked target trajectory, and then call the model configuration parameters of the corresponding partition for detection, ensuring that the fall posture detection in different regions has a high accuracy.

[0037] (2) The CNN model with regional parameter adjustment proposed in the present invention is designed for embedded devices and realizes the online fall detection function on the embedded platform. The model has the characteristics of small memory usage, strong modifiability, and low computing power requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flow chart of the method for detecting a person falling in the present invention;

[0039] Figure 2 It is a schematic diagram of the detection area and target position coordinates in the present invention;

[0040] Figure 3 It is the overall network structure of the CNN model in the present invention;

[0041] Figure 4 It is a schematic diagram of the detection results in a detection example performed according to the scheme of the present invention. DETAILED DESCRIPTION

[0042] The present invention aims to propose a method for detecting human falls using a millimeter-wave radar based on a partitioned parameter-adjusted CNN model, so as to solve the problem that the echo energy of the millimeter-wave radar is uneven in different areas, resulting in low robustness of point cloud data in different areas, thus affecting the accuracy of fall detection. This scheme uses millimeter-wave radar point cloud data under a continuous time series to construct a CNN model with partitioned parameter adjustment, performs human fall posture detection according to the indoor environment, configures the CNN model using the model parameter identification results in different areas, and deploys the model to an embedded platform to realize online judgment of indoor human fall behavior. The present invention generates a CNN model in combination with a specific indoor environment, effectively improving the accuracy of indoor human posture detection, and at the same time, adaptively identifies model parameters in combination with specific scenarios, thereby improving the engineering application efficiency of the model.

[0043] Example:

[0044] This embodiment is based on a closed test environment with boundaries. The length of the test space boundary is 4 meters and the width is 4 meters. The millimeter-wave radar installation mode is wall-mounted. The installation position is the midpoint of the bottom edge of the test space top-down angle width. The effective angle range is set to 120, and the radar is installed at a height of 2.6 meters.

[0045] By implementing the above-mentioned algorithm for detecting a person falling down of the present invention on an embedded platform, the overall detection process is as follows: Figure 1 As shown, point cloud data is first acquired and preprocessed, and then the data normalization and area determination processes are performed in parallel based on the preprocessed point cloud data. The normalized point cloud data is sent to a pre-trained CNN network. The parameters of this CNN network are adjustable, that is, the corresponding model parameters are called according to the area where the point cloud data is located, so as to detect the fall behavior of the tracked target occurring in the corresponding area.

[0046] In the above scheme, the preprocessing of point cloud data specifically includes: Fourier transform in distance dimension, Fourier transform in Doppler dimension, dynamic point cloud detection, static clutter elimination, Fourier transform in angle dimension, point cloud clustering and tracking. The final tracking target point cloud data is stored on the embedded platform as a floating point number containing its own position coordinates and Doppler information in the current time frame.

[0047] After the point cloud data is preprocessed, normalization and region determination are performed simultaneously. Normalization is to provide the model with relatively uniform input data in terms of distribution and data scale. In this embodiment, the target point cloud data normalization method is: all point clouds of the target in the current time frame are normalized according to various dimensions (distance dimension, azimuth dimension, Doppler dimension, and pitch angle dimension). The specific calculation formula is:

[0048]

[0049] Among them, x i Represents the value of a certain dimension of the i-th point, It represents the normalized result, u is the mean of the corresponding dimension, σ is the standard deviation of the corresponding dimension, ε is a very small number to prevent the divisor from being 0, and N is the number of target point clouds.

[0050] The area determination module uses the position coordinates of the tracking point cloud track after data preprocessing as input data and outputs the area matrix W representing the area where the target is located. The area matrix W here is a parameter matrix used to characterize the corresponding sub-areas. In this embodiment, the sub-areas are divided based on the entire detection area, with the radar position as the origin of the system absolute coordinate system, and the detection area is divided into 10 segments along the x-axis and y-axis, such as Figure 2 As shown, the final output area matrix W is a 10*10 square matrix. Assuming that the tracking point cloud trajectory after data preprocessing contains N points, the position of each point is expressed as follows:

[0051]

[0052] Then, the N-dimensional correlation coefficient matrix P about the target points is obtained:

[0053]

[0054] Among them, Cov(X,Y) represents the covariance matrix of the position coordinates X and Y, and D(X) and D(Y) represent the variance of the position coordinates X and Y respectively.

[0055] The final region matrix W is:

[0056] W=APA T

[0057] Among them, A is a 10*N matrix, which is used to represent the distribution state of the target point cloud in the area.

[0058] Taking into account the completeness of the observation of falling behavior and the accuracy of recognition, the observation time given in this embodiment is 10 frames of data, that is, for the target point cloud data after normalization, 10 consecutive frames of data are retained, and in the absolute coordinate system with the radar position as the origin, the target point cloud height is used as the index for sorting, and the position coordinates and Doppler information of the highest 4 points and the lowest 4 points in the height direction are selected, and the average position coordinates of the target track are added to form an input data of size 10*35: the coordinates of the 4 highest points (x, y, z) plus Doppler information (v), the coordinates of the 4 lowest points (x, y, z) plus Doppler information (v), and the average position coordinates of the track itself (x, y, z), that is, 4*4+4*4+3=35 information dimensions.

[0059] The personnel fall detection model used in this embodiment is built and trained based on the CNN network, and its network structure is as follows: Figure 3 As shown: the input layer requires normalized radar point cloud data of size 10*35 as input: then output to the first one-dimensional convolution layer, the number of convolution kernels is 42, the size is 3, and the step size is 1; then output to the first normalization layer, with the relu function as the activation function; then output to the second one-dimensional convolution layer, the number of convolution kernels is 28, the size is 3, and the step size is 1; then output to the second normalization layer, with the relu function as the activation function; then output to the third one-dimensional convolution layer, the number of convolution kernels is 14, the size is 3, and the step size is 1; then output to the third normalization layer, with the relu function as the activation function; then output to the first fully connected layer; then output to the fourth normalization layer, with the relu function as the activation function; then output to the second fully connected layer; then output to the fifth normalization layer, and finally with the softmax function as the activation function.

[0060] During the model training process, the cross entropy function is added with the region matrix W as the weight as the loss function L of the model training. The specific formula is as follows:

[0061]

[0062] Among them, M is the total number of samples in a batch, K is the number of model classification categories, T is the true label of the corresponding sample, Z is the calculated output value of the model, q is the parameter to be identified in the model training process, and Q is the set of all parameters to be identified.

[0063] Since each sub-region has a corresponding unique regional matrix W, the model parameters that can be obtained by each sub-region through training are also different, so a CNN model with adjustable parameters according to the region can be trained.

[0064] In the application of fall detection using the above model, after obtaining the point cloud track of the tracking target, the model parameters of the corresponding area can be called according to the sub-area where the track is located, so as to make a fall detection judgment based on the point cloud data. The judgment conditions include: ① The average height of the target point cloud is less than 0.3m; ② The model classification output result is judged as a fall behavior. The model output is the probability of the corresponding classification label of the softmax function. In this embodiment, the probability of the fall behavior label output greater than 0.5 is used as the judgment condition for the model to judge the fall behavior.

[0065] In order to verify the algorithm of the present invention, the algorithm was tested using an evaluation module test platform based on the IWR6843 millimeter wave sensor as a hardware platform. The test results are as follows: Figure 4 As shown. In a 4×4 meter square test space, the fall behavior was tested in different areas. Ten different subjects performed a total of 240 falls, 225 of which were successfully detected. The fall detection accuracy rate reached 93.75%, and the processing time of the CNN model on the test platform was about 7 milliseconds. It can be seen that the algorithm in the present invention has high detection accuracy and can be carried on an embedded platform to realize online detection.

[0066] Finally, it should be noted that the above embodiments are only preferred implementations and are not intended to limit the present invention. It should be pointed out that for those skilled in the art, several modifications, equivalent replacements, improvements, etc. can be made without departing from the scope of the present invention and the scope of protection of the claims, and all of these should be included in the protection scope of the present invention.

Claims

1. A method for detecting a person falling using a millimeter wave radar based on a partitioned parameter adjustment CNN model, characterized in that: The following steps are involved: S1, obtain the point cloud data of the millimeter wave radar and perform preprocessing to obtain the point cloud track of the tracking target; S2, normalizing the tracked target point cloud track, and judging the sub-area where the detected target is located in the detection area based on the target point cloud track; S3. According to the sub-area where the tracking target is located in the detection area, the corresponding model parameters are called, and the pre-trained personnel fall detection model is used to perform personnel fall posture detection according to the normalized point cloud data. The personnel fall posture detection includes two judgment conditions: ① the average height of the tracking target point cloud is lower than a certain threshold; ② the model classification output result is judged as a fall behavior; In step S2, the sub-area where the tracking target is located in the detection area is determined based on the tracking target point cloud track. Specifically include: Assume that the tracking point cloud trajectory contains N points, and the position of each point is expressed as follows: The N-dimensional correlation coefficient matrix P of the target points is obtained as follows: in, represents the covariance matrix of position coordinates X and Y, and Represent the variance of position coordinates X and Y respectively; The final regional matrix W of the sub-region where the tracking target is located is: Among them, A is a 10*N matrix, which is used to represent the distribution state of the point cloud of the tracking target in the area; In step S3, when the person falling posture detection is performed based on the normalized point cloud data, the input data is obtained in the following manner: For the normalized point cloud data, keep 10 consecutive frames of radar point cloud data, sort them by the point cloud height of the tracked target in the absolute coordinate system with the radar position as the origin, select the position coordinates and Doppler information of the highest 4 points and the lowest 4 points in the height direction, and add the average position coordinates of the tracked target point cloud track to form input data of size 10*35; During the training process of the fall detection model, the regional matrix W is used as the weight and the cross entropy is added as the loss function of the model training. L , the formula is as follows: Where M is the total number of samples in a batch, K is the number of model classification categories, T is the true label of the corresponding sample, Z is the calculated output value of the model, q is the parameter to be identified during the model training process, and Q is the set of all parameters to be identified; After training, the model parameters for each sub-region are obtained.

2. The method for detecting a person falling by using a millimeter wave radar based on a partitioned parameter adjustment CNN model as claimed in claim 1, characterized in that: In step S1, the preprocessing includes: Fourier transform in distance dimension, Fourier transform in Doppler dimension, dynamic point cloud detection, static clutter elimination, Fourier transform in angle dimension, point cloud clustering and tracking.

3. The method for detecting a person falling using a millimeter wave radar based on a partitioned parameter adjustment CNN model as claimed in claim 1 or 2, characterized in that: In step S2, normalizing the tracked target point cloud track includes: Normalize all point clouds of the target tracked in the current time frame according to each dimension: in, Indicates i The value of a certain dimension of a point, represents the normalized result, is the mean of the corresponding dimension, is the standard deviation of the corresponding dimension, To prevent the divisor from being a very small number of 0, N is the number of target point clouds; The various dimensions include: distance dimension, azimuth dimension, Doppler dimension and elevation dimension.

4. The method for detecting a person falling using a millimeter wave radar based on a partitioned parameter adjustment CNN model as claimed in claim 1 or 2, characterized in that: In step S2, the sub-areas are divided in the following manner: taking the position of the millimeter-wave radar as the origin of the system absolute coordinate system, the detection area is divided into 10 segments along the x-axis and the y-axis to form corresponding sub-areas.

5. The method for detecting a person falling using a millimeter wave radar based on a partitioned parameter adjustment CNN model as claimed in claim 1 or 2, characterized in that: In step S3, the personnel fall detection model is obtained based on CNN network training, and the CNN network includes: a first one-dimensional convolutional layer, a first normalization layer, a second one-dimensional convolutional layer, a second normalization layer, a third one-dimensional convolutional layer, a third normalization layer, a first fully connected layer, a fourth normalization layer, a second fully connected layer and a fifth normalization layer connected in sequence.

6. The method for detecting a person falling by using a millimeter wave radar based on a partitioned parameter adjustment CNN model as claimed in claim 5, characterized in that: In the CNN network, the number of convolution kernels of the first one-dimensional convolution layer is 42, the size is 3, and the step size is 1; the number of convolution kernels of the second one-dimensional convolution layer is 28, the size is 3, and the step size is 1; the number of convolution kernels of the third one-dimensional convolution layer is 14, the size is 3, and the step size is 1; the first normalization layer, the second normalization layer, the third normalization layer, and the fourth normalization layer all use the relu function as the activation function; the fifth normalization layer uses the softmax function as the activation function.

Images