Millimeter wave radar fall detection method based on lightweight neural network
Through a lightweight neural network-based method, the speed-time spectrum envelope of the millimeter wave radar is extracted and logically discriminated, which solves the problems of low detection accuracy and instability in traditional methods, and achieves high accuracy and robust fall detection.
Patent Information
- Application Number
- CN202510003702.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-02
- Publication Date
- 2025-05-02
AI Technical Summary
The traditional millimeter-wave radar fall detection method requires manual extraction of features, low detection accuracy, unstable in complex scenarios, and lack generalization capabilities.
Using a lightweight neural network method, enriching the data sets through original signal acquisition, extracting the envelope of the speed-time spectrum as the input of the neural network, reducing the data dimension, improving the detection accuracy, and further improving robustness through logical discrimination.
It realizes fall detection with high accuracy with small calculations, is suitable for environments with limited resources, and improves the accuracy and robustness of detection.
Smart Images
Figure CN119916320A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of fall detection, and in particular to a millimeter wave radar fall detection method based on a lightweight neural network. Background Art
[0002] The traditional millimeter-wave radar fall detection method manually extracts effective features from radar data based on experience and domain knowledge, and uses threshold division, traditional machine learning and other methods to achieve fall detection. This method requires manual feature extraction. If the extracted features are not obvious, it will greatly affect the detection accuracy. In addition, it is unstable in complex scenarios and lacks generalization ability. Summary of the invention
[0003] In view of this, the present invention provides a millimeter-wave radar fall detection method based on a lightweight neural network, which can enrich the data set category and data volume through original signal acquisition, reduce the dimension of the neural network input data by envelope extraction of the velocity-time spectrogram, and improve the detection accuracy of the specific data set, so that the neural network can learn the data features and achieve high accuracy with a small amount of computation. It is suitable for resource-limited environments such as millimeter-wave radars, and further improves the accuracy and robustness of fall detection through logical judgment.
[0004] To achieve the above object, the present invention provides the following technical solutions:
[0005] A millimeter wave radar fall detection method based on a lightweight neural network includes the following steps:
[0006] S100, collects the original signal of millimeter wave radar;
[0007] S200, data preprocessing, obtaining a speed-time spectrum, and extracting a speed-time spectrum envelope;
[0008] S300, marking the falling and getting up actions, and establishing a fall detection data set;
[0009] S400, build a neural network model;
[0010] S500, training and testing of neural network models;
[0011] S600: Perform logical judgment on the output result of the neural network.
[0012] Preferably, step S100 specifically includes: selecting a plurality of experimenters with different heights, weights and ages, and collecting falling, getting up and other actions in single-person scenes and multi-person scenes.
[0013] Preferably, step S200 specifically includes: performing two-dimensional FFT on the original signal to obtain a range-Doppler spectrum, calculating the absolute value along the distance dimension, and calculating after combining 30 frames to form a velocity-time spectrum.
[0014] Preferably, step S200 specifically includes: extracting the velocity-time spectrogram envelope as the neural network input, and the specific steps are as follows:
[0015] S210, performing 64-point distance FFT on the original signal to obtain a distance signal;
[0016] S220, performing a 512-point Doppler FFT on the range signal to obtain a range-Doppler spectrum;
[0017] S230, adding the absolute values along the distance dimension to obtain the speed data of one frame, and then combining 30 frames to obtain a speed-time spectrum, the speed-time spectrum spectrum dimension is 512*30;
[0018] S240, extracting the velocity-time spectrum envelope.
[0019] Preferably, step S240 specifically includes:
[0020] S241, sorting all cells in the speed-time spectrum according to amplitude, selecting half of the cells from small to large and calculating the average value as the background noise;
[0021] S242, dividing the amplitude of each cell in the speed-time spectrum by the background noise to obtain a signal-to-noise ratio;
[0022] S243, setting cells in the velocity-time spectrum whose signal-to-noise ratio is less than 2 to zero;
[0023] S244, respectively record the indexes that are not 0 in each column, and obtain the maximum value, minimum value, and median of the index of each column; if the index array is empty, the maximum value, minimum value, and median are all recorded as 256;
[0024] S245. Subtract 256 from the maximum value, minimum value, and median of each column index to obtain a speed-time spectrum envelope, where the dimension of the speed-time spectrum envelope is 3*30.
[0025] Preferably, step S300 specifically includes: the envelope data of falling, getting up and other actions are all saved in the form of a MAT file, wherein the number of falling actions: the number of getting up actions: the number of other actions = 1:1:5.
[0026] Preferably, the neural network model includes a convolution layer and two fully connected layers, the convolution layer has 5 channels, a 3*9 convolution kernel and serves as the first layer, a fully connected layer has a data dimension of 110*200 and serves as the second layer, and another fully connected layer has a data dimension of 200*3 and serves as the third layer. A ReLU activation function is set between every two hidden layers, and the neural network model outputs a probability value after a Softmax activation function. The amount of computation of the neural network model is 22570 times.
[0027] Preferably, the specific implementation process of the neural network model includes:
[0028] S410, dividing the velocity-time spectrum envelope by 256 for normalization, and obtaining a normalized data dimension of 1*3*30;
[0029] S420, the normalized data passes through the convolution layer of the neural network model, and the dimension of the obtained convolution data is 5*1*22;
[0030] S430, flatten the convolution data dimension to 1*110, and pass it through the fully connected layer of the second layer of the neural network model, so that the first connection data dimension obtained is 1*200;
[0031] S440, after the third fully connected layer of the neural network model, the dimension of the second connection data obtained is 1*3;
[0032] S450, after the Softmax activation function, output the probabilities of falling, getting up and other actions respectively, and the maximum probability is the fall detection result;.
[0033] Preferably, step S600 specifically includes: when the neural network predicts falling for five consecutive times, it is determined to be a fall; when the neural network predicts standing up for six consecutive times, it is determined to be a standing up.
[0034] Preferably, step S600 specifically includes:
[0035] S610, initializing parameters, setting state s = 0, falling matrix f = [0, 0, 0, 0], and standing up matrix u = [0, 0, 0, 0, 0];
[0036] S620, the fall matrix f stores the five most recent prediction results of the neural network, and the stand-up matrix stores the six most recent prediction results of the neural network; 1 represents fall, 2 represents stand-up, and 0 represents other actions;
[0037] S630, when it is determined that the fall matrix f = [1, 1, 1, 1, 1], set the state s = 1, indicating that someone has fallen, and do nothing in other cases;
[0038] S640. When state s=1 and the standing-up matrix u=[2, 2, 2, 2, 2, 2], set state s=0, indicating that the fallen person has stood up. No action is taken in other cases.
[0039] It can be seen from the above technical solutions that the millimeter-wave radar fall detection method based on lightweight neural network provided by the present invention enriches the data set category and data volume through original signal acquisition, and reduces the dimension of neural network input data by envelope extraction of velocity-time spectrogram. The specific data set can improve the detection accuracy, enable the neural network to learn the data features, and achieve high accuracy with a small amount of computation. It is suitable for resource-limited environments such as millimeter-wave radars, and further improves the accuracy and robustness of fall detection through logical judgment. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0041] Figure 1 is a flowchart showing a method for fall detection using a millimeter-wave radar based on a lightweight neural network according to an exemplary embodiment;
[0042] Figure 2 is a flowchart showing data preprocessing according to an exemplary embodiment;
[0043] Figure 3 is a flow chart showing the extraction of a velocity-time spectrogram envelope according to an exemplary embodiment;
[0044] Figure 4 is a flowchart showing a specific implementation process of a neural network model according to an exemplary embodiment;
[0045] Figure 5 is a flow chart showing logic determination according to an exemplary embodiment;
[0046] Figure 6 It is a logic diagram showing logic discrimination according to an exemplary embodiment. DETAILED DESCRIPTION
[0047] The present invention discloses a millimeter wave radar fall detection method based on a lightweight neural network, which can enrich the data set category and data volume through original signal acquisition, reduce the dimension of neural network input data by envelope extraction of velocity-time spectrogram, improve the detection accuracy of a specific data set, enable the neural network to learn data features, and achieve high accuracy with a small amount of computation. The method is suitable for resource-limited environments such as millimeter wave radars, and further improves the accuracy and robustness of fall detection through logical discrimination.
[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0049] In an exemplary embodiment of the present disclosure, a millimeter wave radar fall detection method based on a lightweight neural network is provided. Figure 1 As shown, Figure 1 is a flowchart showing a method for fall detection using a millimeter-wave radar based on a lightweight neural network according to an exemplary embodiment; Figure 2 is a flowchart showing data preprocessing according to an exemplary embodiment; Figure 3 is a flow chart showing the extraction of a velocity-time spectrogram envelope according to an exemplary embodiment; Figure 4 is a flowchart showing a specific implementation process of a neural network model according to an exemplary embodiment; Figure 5 is a flow chart showing logic determination according to an exemplary embodiment; Figure 6 FIG. 1 is a logic diagram for representing logic discrimination according to an exemplary embodiment. Figures 1 to 6 Provide explanation.
[0050] Some specific implementation modes described below are intended to facilitate those skilled in the art to understand the present embodiment, and the present embodiment is not limited to some specific implementation modes described below.
[0051] Reference Figure 1 An exemplary embodiment of the present disclosure provides a millimeter wave radar fall detection method based on a lightweight neural network. The millimeter wave radar fall detection method based on a lightweight neural network includes the following steps:
[0052] Step S100: collecting millimeter wave radar original signals.
[0053] Step S200: preprocess the data to obtain a speed-time spectrum and extract the speed-time spectrum envelope.
[0054] Step S300: Mark the falling and getting up actions, and establish a fall detection data set.
[0055] Step S400: building a neural network model.
[0056] Step S500: training and testing the neural network model.
[0057] Step S600: Perform logical judgment on the output result of the neural network.
[0058] For example, refer to Figure 1 , wherein step S100, collecting the original signal of the millimeter wave radar specifically includes: selecting multiple experimenters with different heights, weights, and ages, and collecting the falling, getting up and other actions in single-person scenes and multi-person scenes. For example, a 64GHz frequency modulated continuous wave millimeter wave radar is used to collect data, and the specific parameters of the continuous wave millimeter wave radar are shown in Table 1:
[0059] parameter Value parameter Value Start frequency 64GHz TX number 1 bandwidth 1.76GHz RX number 1 ADC sampling points 64 Distance resolution 8.48cm Chirp 224 Speed resolution 3.38cm / s
[0060] Table 1
[0061] In the process of raw signal acquisition, multiple actions such as falling, standing up, standing still, walking, running, sitting down, squatting, etc. are designed in single-person and multi-person scenarios. These action sets include most of the basic actions in daily life. Since the technical problem to be solved in this application lies in the action of falling, these actions are divided into falling, standing up and other actions as a whole. Selecting experimenters with different heights, weights and ages to conduct data collection can ensure the generalization of the data. In addition, in order to establish a fall detection data set later, the video is recorded synchronously during the raw signal acquisition process.
[0062] Reference Figure 2 , step S200, data preprocessing, obtaining a speed-time spectrum, extracting the envelope of the speed-time spectrum specifically includes: for the two-dimensional data collected by the continuous wave millimeter wave radar, first perform a two-dimensional FFT to obtain a distance-Doppler spectrum, and further extract the envelope of the speed-time spectrum as the neural network input. For example, perform a two-dimensional FFT on the original signal collected by the continuous wave millimeter wave radar to obtain a distance-Doppler spectrum, calculate the absolute value along the distance dimension, and combine 30 frames to calculate and form a speed-time spectrum.
[0063] In step S200, two-dimensional FFT is performed on the original signal collected by the continuous wave millimeter wave radar to obtain a range-Doppler spectrum. The specific steps are as follows:
[0064] Step S210: Perform 64-point distance FFT on the original signal to obtain a distance signal.
[0065] Step S220: Perform a 512-point Doppler FFT on the range signal to obtain a range-Doppler spectrum.
[0066] Among them, in step S200, the absolute value is calculated along the distance dimension, and the speed-time spectrogram is calculated after combining 30 frames, and the envelope of the speed-time spectrogram is further extracted as the neural network input. The specific steps are as follows:
[0067] Step S230, add the absolute values along the distance dimension to obtain the speed data of one frame, and then combine 30 frames to obtain a speed-time spectrum. The speed-time spectrum has a spectrum dimension of 512*30.
[0068] Step S240: extracting the velocity-time spectrum envelope.
[0069] Reference Figure 3 , wherein the aforementioned step S240, extracting the velocity-time spectrum envelope specifically includes:
[0070] Step S241: sort all cells in the speed-time spectrogram by amplitude, select half of the cells from small to large and calculate the average value as the background noise.
[0071] Step S242: Divide the amplitude of each cell in the speed-time spectrum by the background noise to obtain a signal-to-noise ratio.
[0072] Step S243: Set the cells in the velocity-time spectrum whose signal-to-noise ratio is less than 2 to zero.
[0073] Step S244, respectively record the non-zero indexes in each column, and obtain the maximum value, minimum value and median of each column index; if the index array is empty, the maximum value, minimum value and median are all recorded as 256.
[0074] Step S245: subtract 256 from the maximum value, minimum value and median of each column index to obtain a speed-time spectrogram envelope, the dimension of which is 3*30.
[0075] Among them, step S300, marking the falling and getting up actions, and establishing a fall detection data set, specifically includes:
[0076] The envelope data of falling, getting up and other actions are all saved in the form of MAT files. Among them, the number of falling actions: the number of getting up actions: the number of other actions = 1:1:5. The envelope data of the three categories are all saved in the form of MAT files, for example, 1000 sets of falling actions, 1000 sets of getting up actions, and 5000 sets of other actions, a total of 7000 sets of data.
[0077] Reference Figure 4, wherein, in step S400, the neural network model formed after building the neural network model includes a convolution layer and two fully connected layers, the convolution layer has 5 channels, 3*9 convolution kernels and is used as the first layer, the data dimension of a fully connected layer is 110*200 and is used as the second layer, and the data dimension of another fully connected layer is 200*3 and is used as the third layer, and the neural network model outputs probability values after the Softmax activation function. In order to effectively extract features and complete classification, a ReLU activation function is set between every two hidden layers, and the amount of calculation of the neural network model disclosed in this embodiment is 22570 times.
[0078] The specific implementation process of the neural network model in this embodiment is:
[0079] S410, dividing the velocity-time spectrum envelope by 256 for normalization, and obtaining a normalized data dimension of 1*3*30 to enhance generalization capability;
[0080] S420, the normalized data passes through the convolution layer of the neural network model, and the dimension of the obtained convolution data is 5*1*22;
[0081] S430, flatten the convolution data dimension to 1*110, and pass it through the fully connected layer of the second layer of the neural network model, so that the first connection data dimension obtained is 1*200;
[0082] S440, after the third fully connected layer of the neural network model, the dimension of the second connection data obtained is 1*3;
[0083] S450: After the Softmax activation function, the probabilities of falling, getting up and other actions are output respectively, and the maximum probability is the fall detection result.
[0084] Among them, S500, training and testing the neural network model specifically includes: randomly selecting some samples from the falling, getting up and other action data sets as training sets, and the remaining samples as test sets for training. For example: randomly selecting 4 / 5 samples from the falling, getting up and other action data sets as training sets, and the remaining samples as test sets for training and testing. When training the neural network, select the Adam optimizer, the learning rate is 0.001, the batch number is 64, and the training is 400 rounds.
[0085] Reference Figure 5 and Figure 6 , wherein step S600, logically judging the output result of the neural network specifically includes: when the neural network predicts falling for five consecutive times, judging it as falling, and when the neural network predicts standing up for six consecutive times, judging it as standing up, for example, by the following steps:
[0086] S610, initializing parameters, setting state s = 0, falling matrix f = [0, 0, 0, 0], and standing up matrix u = [0, 0, 0, 0, 0];
[0087] S620, the fall matrix f stores the five most recent prediction results of the neural network, and the stand-up matrix stores the six most recent prediction results of the neural network; 1 represents fall, 2 represents stand-up, and 0 represents other actions;
[0088] S630, when it is determined that the fall matrix f = [1, 1, 1, 1, 1], the fall alarm is turned on, and the state s = 1, indicating that someone has fallen, and no action is taken in other cases;
[0089] S640. When state s=1 and it is judged that the standing-up matrix u=[2, 2, 2, 2, 2, 2], cancel the fall alarm and set state s=0, indicating that the fallen person has stood up. No action is taken in other cases.
[0090] After the radar collects each frame of data, it will update the velocity-time spectrum envelope in real time and input it into the neural network for prediction. However, an action from start to finish is a continuous process and cannot be determined by a single prediction. Therefore, the accuracy and robustness of fall detection can be further improved through the aforementioned step S600.
[0091] In this embodiment, the data set category and data volume are enriched through the original signal collection, and the dimension of the neural network input data is reduced by performing envelope extraction on the speed-time spectrogram. The specific data set can improve the detection accuracy, enable the neural network to learn the data features, and achieve high accuracy with a small amount of computation. It is suitable for resource-limited environments such as millimeter-wave radar, and the accuracy and robustness of fall detection can be further improved through logical judgment.
[0092] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A millimeter wave radar fall detection method based on lightweight neural network, characterized in that: The following steps are involved: S100, collects the original signal of millimeter wave radar; S200, data preprocessing, obtaining a speed-time spectrum, and extracting a speed-time spectrum envelope; S300, marking the falling and getting up actions, and establishing a fall detection data set; S400, build a neural network model; S500, training and testing the neural network model; S600: Performing logical judgment on the output result of the neural network.
2. The millimeter wave radar fall detection method based on lightweight neural network according to claim 1 is characterized in that: The step S100 specifically includes: selecting a plurality of experimenters with different heights, weights, and ages, and collecting the falling, getting up, and other actions in single-person scenes and multi-person scenes.
3. The millimeter wave radar fall detection method based on lightweight neural network according to claim 1 is characterized in that: The step S200 specifically includes: performing a two-dimensional FFT on the original signal to obtain a range-Doppler spectrum, calculating an absolute value along the distance dimension, and calculating after combining 30 frames to form the velocity-time spectrum.
4. The millimeter wave radar fall detection method based on lightweight neural network according to claim 3 is characterized in that: The step S200 specifically includes: extracting the velocity-time spectrum envelope as a neural network input, and the specific steps are as follows: S210, performing 64-point distance FFT on the original signal to obtain a distance signal; S220, performing a 512-point Doppler FFT on the range signal to obtain the range-Doppler spectrum; S230, adding absolute values along the distance dimension to obtain speed data of one frame, and then combining 30 frames to obtain the speed-time spectrogram, wherein the speed-time spectrogram has a spectrogram dimension of 512*30; S240: extracting the velocity-time spectrum envelope.
5. The millimeter wave radar fall detection method based on lightweight neural network according to claim 4 is characterized in that: The step S240 specifically includes: S241, sorting all cells in the speed-time spectrogram by amplitude, selecting half of the cells from small to large and calculating the average value as the background noise; S242, dividing the amplitude of each cell in the speed-time spectrum by the background noise to obtain a signal-to-noise ratio; S243, setting cells in the velocity-time spectrum whose signal-to-noise ratio is less than 2 to zero; S244, respectively record the indexes that are not 0 in each column, and obtain the maximum value, minimum value, and median of the index of each column; if the index array is empty, the maximum value, minimum value, and median are all recorded as 256; S245. Subtract 256 from the maximum value, the minimum value and the median of the index in each column to obtain a speed-time spectrum envelope, where the dimension of the speed-time spectrum envelope is 3*30.
6. The millimeter wave radar fall detection method based on lightweight neural network according to claim 1 is characterized in that: The step S300 specifically includes: the envelope data of falling, getting up and other actions are all saved in the form of a MAT file, wherein the number of falling actions: the number of getting up actions: the number of other actions = 1:1:
5.
7. The millimeter wave radar fall detection method based on lightweight neural network according to claim 1 is characterized in that: The neural network model includes a convolution layer and two fully connected layers. The convolution layer has 5 channels and a 3*9 convolution kernel and serves as the first layer. The data dimension of one fully connected layer is 110*200 and serves as the second layer. The data dimension of the other fully connected layer is 200*3 and serves as the third layer. A ReLU activation function is set between every two hidden layers. The neural network model outputs a probability value after a Softmax activation function. The amount of calculation of the neural network model is 22570 times.
8. The millimeter wave radar fall detection method based on lightweight neural network according to claim 7 is characterized in that: The specific implementation process of the neural network model includes: S410, dividing the velocity-time spectrum envelope by 256 for normalization, and obtaining a normalized data dimension of 1*3*30; S420, the normalized data passes through the convolution layer of the neural network model, and the dimension of the obtained convolution data is 5*1*22; S430, flattening the convolution data dimension to 1*110, passing through the fully connected layer of the second layer of the neural network model, and obtaining a first connection data dimension of 1*200; S440, passing through the fully connected layer of the third layer of the neural network model, the dimension of the second connection data obtained is 1*3; S450: After the Softmax activation function, the probabilities of falling, getting up and other actions are output respectively, and the maximum probability is the fall detection result.
9. The millimeter wave radar fall detection method based on lightweight neural network according to claim 8 is characterized in that: The step S600 specifically includes: when the neural network predicts a fall for five consecutive times, it is determined to be a fall; when the neural network predicts a stand up for six consecutive times, it is determined to be a stand up.
10. The millimeter wave radar fall detection method based on lightweight neural network according to claim 9 is characterized in that: The step S600 specifically includes: S610, initializing parameters, setting state s = 0, falling matrix f = [0, 0, 0, 0], and standing up matrix u = [0, 0, 0, 0, 0]; S620, the fall matrix f stores the five most recent prediction results of the neural network, and the stand-up matrix stores the six most recent prediction results of the neural network; 1 represents fall, 2 represents stand-up, and 0 represents other actions; S630, when it is determined that the fall matrix f = [1, 1, 1, 1, 1], set the state s = 1, indicating that someone has fallen, and do nothing in other cases; S640. When state s=1 and the standing-up matrix u=[2, 2, 2, 2, 2, 2], set state s=0, indicating that the fallen person has stood up. No action is taken in other cases.