Double-threshold radar human body continuous action segmentation recognition method based on digital feature assistance

Abstract

A double-threshold radar human body continuous action segmentation recognition method based on digital feature assistance comprises the steps that human body actions and radar parameters are designed, a millimeter wave radar system platform is built, and a human body stands in front of a platform collection position to change the human body actions; a millimeter wave radar is used for transmitting a linear frequency modulation signal, receiving an echo signal containing action information, performing frequency mixing processing on the signal to obtain an intermediate frequency signal, and generating a Doppler time diagram along slow time axis data based on a velocity dimension projection method after preprocessing; and performing accumulation on radar signal energy at the same time to obtain an energy diagram of the radar signals, performing action cutting division through a digital characteristic auxiliary energy threshold value and a time threshold value of the energy diagram to obtain a time-frequency diagram and an energy diagram of a single action, and finally sending the time-frequency diagram and the energy diagram into a double-flow multi-scale attention network to perform action recognition. The method is convenient and fast to operate, low in cost and high in privacy protection, and has a wide application prospect and a remarkable practical value.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction technology, and in particular to a method for segmenting and recognizing continuous human actions of a dual-threshold radar assisted by digital features. Background Art

[0002] In today's society, with the continuous increase in the proportion of the elderly population, the problem of population aging has become increasingly prominent. The traditional elderly care service model is facing the need for transformation, thus bringing severe challenges to countries around the world. In particular, more and more elderly people choose to live alone and may face various safety risks such as dangerous actions and abnormal vital signs without the accompaniment of guardians. At the same time, the life pressure of young people increases, making it difficult to accompany the elderly all day long. The number of empty-nest elderly people has increased rapidly, and the intelligent elderly care and nursing model has emerged. Currently, many researchers use radar for fall behavior recognition and detection, but there are various deficiencies: the features extracted from radar signals are relatively single, all being micro-Doppler time-frequency diagrams; most studies only collect radar signals of discrete actions for research, ignoring the research of continuous action radar signals. Therefore, in this experiment, multiple features such as micro-Doppler time-frequency diagrams and energy distribution diagrams are proposed in terms of features, and continuous action radar signals and their segmentation algorithms are considered. At the same time, deep learning and neural networks are widely and maturely applied in the field of human pose recognition, and fall detection algorithms based on convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have emerged as the times require. CNN is widely used in image recognition and classification tasks due to its powerful feature extraction ability and can effectively extract key features from images. Furthermore, a multi-scale attention mechanism and a dual-input network architecture are introduced to improve the feature extraction ability and classification performance of the model for input data. The multi-scale attention module extracts features of different scales through global average pooling and global max pooling, and generates attention weights through convolutional layers and activation functions. Summary of the Invention

[0003] In order to overcome the deficiencies of the prior art, the present invention proposes a method for segmenting and recognizing continuous human actions of a dual-threshold radar assisted by digital features, for continuous action fall detection based on an FMCW millimeter-wave radar, which is convenient, fast, safe to operate, does not require wearing, is not affected by distance and obstacles, avoids privacy leakage, and is applicable to various occasions.

[0004] The technical solution proposed by the present invention is as follows:

[0005] A method for segmenting and recognizing continuous human actions of a dual-threshold radar assisted by digital features, the method comprising the following steps:

[0006] Step 1: Design discrete and continuous human actions, including seven actions and their combinations: boxing, falling, raising hands, sitting down, squatting, standing up, and walking. Build a millimeter-wave radar experimental system platform and collect human echo data;

[0007] Step 2: Perform clutter preprocessing on the intermediate-frequency signal. Use the 2D-FFT algorithm to obtain the range-Doppler map of human actions, and based on this, accumulate the micro-Doppler time-frequency map of human actions using the velocity-dimensional projection method; then accumulate the energy values of each point at the same time on the micro-Doppler time-frequency map to obtain the energy distribution map of human action signals;

[0008] Step 3: Segment continuous actions based on the digital features, energy threshold, and time threshold of the energy distribution map to obtain the time-frequency map and energy distribution map of individual actions;

[0009] Step 4: Divide the action time-frequency map and energy distribution map datasets obtained in Step 3 into training sets, validation sets, and test sets according to a certain proportion. Input the training set into the dual-stream multi-scale attention network to extract the features of actions, and use the fully connected layer to complete the classification task to achieve human action recognition. Thus, a parameter model is trained, and then this model is verified through the validation set, and the optimal model is saved;

[0010] Step 5: Apply the above-saved optimal model to the test set to obtain the human action recognition result.

[0011] Furthermore, in Step 1, the millimeter-wave radar experimental system platform is deployed with the millimeter-wave frequency-modulated continuous-wave (FMCW) radar module IWR1843BOOST developed by Texas Instruments. The working range of the FMCW radar is from 76 GHz to 81 GHz, and the parameter settings are shown in Table 1:

[0012]

[0013] Table 1

[0014] One transmitting antenna and four receiving antennas are adopted, and the equivalent bandwidth is:

[0015] B eq =β×N / f s (1)

[0016] where B eq refers to the equivalent bandwidth, β is the linear frequency modulation slope, N is the effective ADC sample number in the linear frequency modulation, and f s is the ADC sampling rate. The resulting range resolution ΔR is approximately 9.8 cm, and the calculation formula is as follows:

[0017]

[0018] where c is the speed of light. The maximum speed V max can be calculated as

[0019]

[0020] The operating frequency of the FMCW radar changes within the chirp period. The transmitted signal is expressed as

[0021] x T (t) = A T cos(2πf c t + πβt 2 ) (4)

[0022] where A T is the amplitude of the transmitted wave, f c is the starting frequency of the chirp, and β is the chirp slope.

[0023] Furthermore, in the step 1, the millimeter-wave radar is used to transmit signals. A human body performs 7 kinds of motion changes, namely boxing, falling, raising hands, sitting down, squatting, standing up, and walking, in front of the radar. Each motion lasts about 1 - 2 s. In addition, there are 6 kinds of continuous motion changes, namely sitting - standing up - falling, walking - falling - standing up, walking - sitting down - raising hands, sitting down - standing up - walking, bowing - sitting down - standing up, and boxing - walking - bowing. Each continuous motion lasts about 3 - 4 s. The DAC1000EVM acquisition board is used to transmit the received intermediate-frequency radar human body motion echo data to a computer for storage, and the data is stored in complex form. [0000078]

[0024] Furthermore, the process of the step 2 is as follows:

[0025] (2.1) According to the set radar parameters, the original intermediate-frequency signal data collected by the acquisition card is divided into multi-channel data and saved as I / Q data in complex form, that is, the.bin file of the intermediate-frequency echo is obtained. Then, the data of each antenna is read from it and organized into a three-dimensional array of [n_sample, n_chirp, n_frame], where n_frame represents the number of sampling frames, n_chirp represents the total number of chirps included in each frame, and n_sample represents the number of sampling points included in each chirp. Then, frame difference processing is performed on it, that is, the data of the latter frame is subtracted from the data of the previous frame. Finally, the two-dimensional echo sequences of the I channel and the Q channel after frame difference are output;

[0026] (2.2) Perform FFT on each frame of data after frame difference processing in the fast time dimension to obtain the distance information of the motion; then perform FFT in the slow time dimension on this basis to obtain the distance and speed information of the motion, that is, the RDM (range - Doppler map). Here, RD(i, j, t) is used to represent the signal power value at the i-th range bin and the j-th velocity bin of the RDM at the t-th frame;

[0027] (2.3) Project the RDM onto the velocity dimension, that is, compress the i dimension, and finally obtain the desired micro-Doppler time-frequency map. The formula for projecting onto the velocity dimension is as follows:

[0028] DT(t,j) = ∑ i RDM(i,j,t);

[0029] (2.4) Accumulate the energy values of each point at the same time on the micro-Doppler time-frequency map to obtain the energy distribution map of a single human action signal. The specific formula is as follows

[0030] EnergyMap(j) = ∑ t DT(t,j).

[0031] Furthermore, in step 3, for the obtained continuous-action micro-Doppler time-frequency map and energy distribution map, by finding the minimum value points of the digital feature energy distribution and supplemented with an energy threshold and a time threshold, the continuous action is segmented to obtain the data unit of a single action. The process is as follows:

[0032] (3.1) First, set parameters including the number of frames read each time, action identifier, energy threshold, and time threshold. First, judge whether it is the start or end of an action by whether the energy value is a minimum value point. At the same time, the energy threshold is an important basis for judging whether an action starts and ends. On the premise that the energy value of this frame is a minimum value, only the points with an energy value less than the energy threshold will be used as the start or end identification points of the action. To facilitate the system's detection of actions, an action identifier is used to help distinguish whether the current state is in an action or not. At the same time, considering the errors existing in practical applications, in order to prevent misjudging actions, a time threshold is added to ensure the validity of the action. The action cut out by the method of the energy threshold must meet the time threshold to be regarded as a real action. Considering the real-time nature of fall detection, data needs to be read in real time. Therefore, a number of frames read each time is set to meet the real-time requirements;

[0033] (3.2) Then, perform preprocessing on the radar data. Read a certain number of frames each time, process each frame to obtain the micro-Doppler map, take out the current frame in the micro-Doppler map, compare the energy of the 128 sampling points in the current frame with the background energy, set those greater than the background energy to 1, and then sum them up. In this way, the calculated energy sum value reflects the action energy size of the current frame to a certain extent. Store this energy sum value in an array for smoothing processing;

[0034] (3.3) Since the energy sum values of the previous frames are stored in the array, exponential smoothing can be used to gradually approximate the true value by weighted averaging the energy sum value of the current frame and the previous smoothed value;

[0035] (3.4) Check the energy sum value of the current frame. If a minimum value point appears in the energy points, and this point is less than the energy threshold and the action identifier indicates that the current is not in an action, then mark the current frame as the start frame of the action and set the action identifier to 1 to indicate the start of the action;

[0036] (3.5) If a minimum value point appears in the current frame, and it is less than the energy threshold and the current identifier indicates that it is in an action, then calculate the duration between the current frame and the start frame of the action. If it is greater than the time threshold, then set the end frame of the action to the previous frame of the current frame, update the start frame of the action, set the action identifier to 0 to indicate the end of the action, and crop, grayscale, and save the micro-Doppler map from the start frame to the end frame of the action;

[0037] (3.6) If a minimum value point appears in the current frame, but it is less than the energy threshold and indicates that it is in an action but the duration does not meet the time threshold, then continue to set the action identifier to 1 to indicate that the action continues, and continue to find the next frame whose sum value is less than the energy threshold;

[0038] (3.7) At the same time, in order to prevent the action time from being too long and the action identifier not being found, when the number of frames is sufficient, use the current frame as the end frame;

[0039] (3.8) Through the above steps, continuous human actions are segmented into individual actions.

[0040] The process of step 4 is as follows:

[0041] (4.1) Integrate the time-frequency map and energy distribution map of the individual actions obtained in step 3 into a data set, and divide it into a training set, a validation set, and a test set according to a preset ratio. The training set is used for model training, the validation set is used for hyperparameter tuning and performance verification of the model, and the test set is used for evaluating the final performance of the model;

[0042] (4.2) Preprocess the data in the training set, including operations such as normalization and data augmentation, to improve the generalization ability and training efficiency of the model;

[0043] (4.3) Input the preprocessed training set into the dual-stream multi-scale attention network, and extract the features of individual actions through the forward propagation of the network. The network structure includes a feature extraction module, a multi-scale attention module, and a fully connected layer. The feature extraction module is used to extract key features from the time-frequency map and energy distribution map, the multi-scale attention module is used to enhance the expression ability of the features, and the fully connected layer is used to map the extracted features to the category label space to complete the classification task;

[0044] (4.4) During the training process, the cross-entropy loss function is used as the optimization objective, and the network parameters are updated through backpropagation. At the same time, the Adam optimizer is adopted for gradient descent to minimize the loss function value;

[0045] (4.5) After each training epoch, the model is validated using the validation set, the loss value and accuracy on the validation set are recorded, and the optimal model parameters are selected and saved according to the performance metrics of the validation set. The optimal model refers to the model with the highest accuracy and the lowest loss value on the validation set;

[0046] Finally, an optimized parameter model is obtained for subsequent human action recognition tasks.

[0047] The beneficial effects of the present invention are as follows: convenient and fast operation, low cost, and strong privacy protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is the flowchart of the implementation process of the method for continuous action segmentation and recognition of human body by dual-threshold radar based on digital feature assistance of the present invention;

[0049] Figure 2 is the Doppler time graph and its corresponding energy graph of 7 human actions defined in the embodiments of the present invention, where (a) raising the hand, (b) punching, (c) squatting, (d) falling, (e) standing up, (f) walking, (g) sitting down, (h) sitting down

[0050] Figure 3 is the schematic diagram of the segmentation of the continuous action time-frequency graph and energy graph of the present invention;

[0051] Figure 4 is the flowchart of the principle of the continuous action segmentation algorithm of the present invention;

[0052] Figure 5 is the structural diagram of the dual-stream multi-scale attention network of the present invention;

[0053] Figure 6 is the confusion matrix graph of the human action recognition result of the present invention;

[0054] Figure 7 is the schematic diagram of the result of segmenting the continuous action Doppler time graph of the human body and feeding it into the network to predict the label. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0055] The following elaborates on the preferred embodiments of the present invention in conjunction with the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making the protection scope of the present invention more clearly defined.

[0056] Refer to Figures 1 to 4, A method for continuous action segmentation and recognition of radar human body assisted by digital features, comprising the following steps:

[0057] (1) Design human discrete actions, continuous actions and radar parameters, and build a millimeter-wave radar experimental system platform;

[0058] In step 1, the millimeter-wave radar experimental system platform is deployed with the millimeter-wave frequency-modulated continuous-wave FMCW radar module IWR1843BOOST developed by Texas Instruments. The working range of the FMCW radar is from 76 GHz to 81 GHz, and the parameter settings are shown in Table 1:

[0059]

[0060] Table 1

[0061] One transmitting antenna and four receiving antennas are adopted, and the equivalent bandwidth is:

[0062] B eq = β × N / f s (1)

[0063] Among them, B eq refers to the equivalent bandwidth, β is the linear frequency modulation slope, N is the effective ADC sample number in the linear frequency modulation, and f s is the ADC sampling rate. The resulting range resolution ΔR is approximately 9.8 cm, and the calculation formula is as follows:

[0064]

[0065] where c is the speed of light. The maximum speed V max can be calculated as

[0066]

[0067] The working frequency of the FMCW radar changes within the linear frequency modulation period, and the transmitted signal is expressed as

[0068] x T (t) = A T cos(2πf c t + πβt 2 ) (4)

[0069] Among them, A T is the amplitude of the transmitted wave, f c is the starting frequency of the linear frequency modulation, and β is the frequency modulation slope.

[0070] The radar of the experimental system platform deployed in this embodiment is a millimeter-wave FMCW radar developed by Texas Instruments. The working range of the FMCW radar is from 76 GHz to 81 GHz. It uses one transmitting antenna and four receiving antennas. The starting frequency and frequency modulation slope are set to 77 GHz and 33 MHz / μm respectively, the ADC sampling rate is 5 MHz, and the number of effective ADC samples per chirp is 256. It uses one transmitting antenna and four receiving antennas;

[0071] A human performs 7 motion changes in front of the radar, namely punching, falling, raising hands, sitting down, squatting down, standing up, and walking. Each motion lasts about 1 - 2 s. There are also 6 continuous motion changes, namely sitting - standing up - falling, walking - falling - standing up, walking - sitting down - raising hands, sitting down - standing up - walking, bowing - sitting down - standing up, and punching - walking - bowing. Each continuous motion lasts about 3 - 4 s. The millimeter-wave radar emits signals and then receives the echo signals containing motion information. The transmitted signal and the received signal are mixed to obtain the intermediate-frequency signal of the human motion echo. The ADC1000EVM acquisition board is used to transmit the received intermediate-frequency radar human motion echo data to the computer for storage, and it is stored in complex form;

[0072] (2) First, perform clutter preprocessing on the intermediate-frequency signal, then use the 2D-FFT algorithm to obtain the range-Doppler map of the human motion. Based on this, accumulate using the velocity dimension projection method to obtain the micro-Doppler time-frequency map of the human motion. Then, accumulate the energy values of each point at the same time on the micro-Doppler time-frequency map to obtain the energy distribution map of the human motion signal. The process is as follows:

[0073] (2.1) According to the set radar parameters, divide the original intermediate-frequency signal data collected by the acquisition card into multiple channel data, save it as complex I / Q data, that is, obtain the.bin file of the intermediate-frequency echo. Then read the data of each antenna from it and organize it into a three-dimensional array of [n_sample, n_chirp, n_frame], where n_frame represents the number of sampling frames, n_chirp represents the total number of chirps per frame, and n_sample represents the number of sampling points per chirp. Then perform frame difference processing on it, that is, subtract the data of the previous frame from the data of the next frame, and finally output the two-dimensional echo sequences of the I-channel and Q-channel after frame difference;

[0074] (2.2) Perform FFT on each frame of data after frame difference processing in the fast time dimension to obtain the range information of the motion; then perform FFT in the slow time dimension on this basis to obtain the range and velocity information of the motion, that is, obtain the RDM (range-Doppler map). Here, use RD(i, j, t) to represent the signal power value at the i-th range bin and the j-th velocity bin of the RDM at the t-th frame;

[0075] (2.3) Project the RDM onto the velocity dimension, that is, compress it in the i - dimension, and finally obtain the desired micro - Doppler time - frequency map. The formula for projecting onto the velocity dimension is as follows:

[0076] DT(t,j)=∑ i RDM(i,j,t);

[0077] (2.4) Accumulate the energy values of each point at the same time on the micro - Doppler time - frequency map to obtain the energy distribution map of a single human motion signal. The specific formula is as follows

[0078] EnergyMap(j)=∑ t DT(t,j);

[0079] (3) Segment continuous actions based on digital features supplemented by energy thresholds and time thresholds to obtain the time - frequency map and energy distribution map of a single action; the process is as follows:

[0080] (3.1) First, set parameters including the number of frames read each time, action identifier, energy threshold, and time threshold. First, judge whether it is the start or end of an action by whether the energy value is a minimum point. At the same time, the energy threshold is an important basis for judging whether an action starts and ends. On the premise that the energy value of this frame is a minimum, only the points with energy values less than the energy threshold will be used as the start or end identification points of the action. To facilitate the system's detection of actions, an action identifier is used to help distinguish whether the current state is in an action or not. At the same time, considering the errors in actual applications, in order to prevent misjudging actions, a time threshold is added to ensure the validity of the action. The action cropped by the energy threshold method must meet the time threshold to be regarded as a real action. Considering the real - time nature of fall detection, data needs to be read in real - time. Therefore, set the number of frames read each time to meet the real - time requirements;

[0081] (3.2) Then, perform pre - processing on the radar data. Read a certain number of frames each time, process each frame to obtain the micro - Doppler map. Take out the current frame from the micro - Doppler map, compare the energy of the 128 sampling points in the current frame with the background energy, set those greater than the background energy to 1, and then sum them up. In this way, the calculated energy sum value reflects the action energy size of the current frame to a certain extent. Store this energy sum value in an array for smoothing processing;

[0082] (3.3) Since the array stores the energy sum values of the previous frames, exponential smoothing can be used to gradually approximate the true value by weighted - averaging the current frame energy sum value and the previous smoothed value;

[0083] (3.4) Check the energy and value of the current frame. If a minimum value point appears in the energy points, and this point is less than the energy threshold and the action identifier indicates that the current is not in an action, then mark the current frame as the start frame of the action and set the action identifier to 1 to indicate the start of the action;

[0084] (3.5) If a minimum value point appears in the current frame, and it is less than the energy threshold and the current identifier indicates that the action is in progress, then calculate the duration between the current frame and the start frame of the action. If it is greater than the time threshold, then set the end frame of the action to the previous frame of the current frame, update the start frame of the action, set the action identifier to 0 to indicate the end of the action, and crop, grayscale, and save the micro-Doppler map from the start frame to the end frame of the action;

[0085] (3.6) If a minimum value point appears in the current frame, it is less than the energy threshold and it indicates that the action is in progress but the duration does not meet the time threshold, then continue to set the action identifier to 1 to indicate that the action continues, and continue to find the next frame whose sum value is less than the energy threshold;

[0086] (3.7) At the same time, in order to prevent the action time from being too long and the action identifier not being found, when the number of frames is sufficient, use the current frame as the end frame;

[0087] (3.8) Segment continuous human actions into individual actions through the above steps.

[0088] (4) Divide the action time-frequency map dataset obtained in step 3 into a training set, a validation set, and a test set according to a certain proportion, and input the training set into the dual-input fusion network to extract the features of individual actions; then use the fully connected layer in deep learning to complete the classification task to achieve human action recognition, and thus train a parameter model. Then verify this model through the validation set and save the optimal model; the process is as follows:

[0089] (4.1) Integrate the time-frequency map and energy distribution map of individual actions obtained in step 3 into a dataset, and divide it into a training set, a validation set, and a test set according to a preset proportion. The training set is used for model training, the validation set is used for hyperparameter tuning and performance verification of the model, and the test set is used for evaluating the final performance of the model;

[0090] (4.2) Preprocess the data in the training set, including operations such as normalization and data augmentation, to improve the generalization ability and training efficiency of the model;

[0091] (4.3) Input the preprocessed training set into the dual-input fusion network, and extract the features of individual actions through the forward propagation of the network. The network structure includes a feature extraction module, a multi-scale attention module, and a fully connected layer. The feature extraction module is used to extract key features from the time-frequency map and the energy distribution map. The multi-scale attention module is used to enhance the expression ability of the features. The fully connected layer is used to map the extracted features to the category label space to complete the classification task;

[0092] (4.4) During the training process, use the cross-entropy loss function as the optimization objective, update the network parameters through backpropagation, and at the same time use the Adam optimizer for gradient descent to minimize the loss function value;

[0093] (4.5) After each training epoch, use the validation set to verify the model, record the loss value and accuracy on the validation set, and select the optimal model parameters for saving according to the performance metrics of the validation set. The optimal model refers to the model with the highest accuracy and the lowest loss value on the validation set;

[0094] Finally, an optimized parameter model is obtained for subsequent human action recognition tasks.

[0095] The human actions defined in this embodiment are as Figure 2 shown, including 7 actions: boxing, falling, raising hands, sitting down, squatting, standing up, and walking. The human stands at a position about 150 cm from the platform acquisition point and makes action changes to obtain radar echo signals. To ensure the universality of the data, multiple people are arranged to sample non-uniformly at different angles and distances for each action, with a total of 2415 groups of data, as shown in Table 2.

[0096]

[0097] Table 2

[0098] In this embodiment, 2D-FFT is used to select and process the data along the slow time axis. The Doppler time maps of 7 human actions and the energy distribution maps corresponding to the actions are as Figure 2 shown.

[0099] In the network training stage of this embodiment, the present invention mainly adopts a continuous action segmentation method based on digital feature-assisted energy threshold and time threshold. The process is as Figure 3 shown, and the principle of the segmentation algorithm is as Figure 4 shown. Input the segmented time-frequency map and energy distribution map into the dual-stream multi-scale attention network for training. The structure is as Figure 5 shown. This embodiment tests 7 human actions, and the results are as Figure 6As shown. Finally, continuously collect human motion data again, generate a micro-Doppler time-frequency map and an energy distribution map for recognition testing, and the results are as Figure 7 shown. The test results verify the effectiveness of the method for detecting and recognizing continuous human actions based on digital feature-assisted energy threshold and time threshold proposed by the present invention. The content described in the embodiments of this specification is only a list of implementation forms of the inventive concept and is only for illustrative purposes. The protection scope of the present invention should not be regarded as limited to the specific forms stated in this embodiment, and the protection scope of the present invention also extends to equivalent technical means that can be conceived by those of ordinary skill in the art based on the inventive concept of the present invention.

[0100] The content described in the embodiments of this specification is only a list of implementation forms of the inventive concept and is only for illustrative purposes. The protection scope of the present invention should not be regarded as limited to the specific forms stated in this embodiment, and the protection scope of the present invention also extends to equivalent technical means that can be conceived by those of ordinary skill in the art based on the inventive concept of the present invention.

Claims

1. A method for continuous action segmentation and recognition of radar human body with dual thresholds assisted by digital features, characterized in that, The method includes the following steps: Step 1: Design discrete and continuous human actions, including 7 actions and their combinations such as boxing, falling, raising hands, sitting down, squatting, standing up, and walking. Build a millimeter-wave radar experimental system platform and collect human echo data. Step 2: Perform clutter preprocessing on the intermediate-frequency signal. Use the 2D-FFT algorithm to obtain the range-Doppler map of human actions, and based on this, accumulate to obtain the micro-Doppler time-frequency map of human actions using the velocity-dimensional projection method. Then, accumulate the energy values of each point at the same time on the micro-Doppler time-frequency map to obtain the energy distribution map of human action signals. Step 3: Segment continuous actions based on the digital features, energy threshold, and time threshold of the energy distribution map to obtain the time-frequency map and energy distribution map of individual actions. Step 4: Divide the action time-frequency map and energy distribution map datasets obtained in Step 3 into a training set, a validation set, and a test set according to a certain proportion. Input the training set into a dual-stream multi-scale attention network to extract the features of actions, and use a fully connected layer to complete the classification task to achieve human action recognition. Thus, a parameter model is trained, and then this model is verified through the validation set, and the optimal model is saved. Step 5: Apply the above saved optimal model to the test set to obtain the human action recognition result.

2. The method for continuous action segmentation and recognition of a radar human body assisted by digital features as claimed in claim 1, wherein In the above Step 1, the millimeter-wave radar experimental system platform is deployed with the millimeter-wave frequency-modulated continuous-wave (FMCW) radar module IWR1442BOOST developed by Texas Instruments. The working range of the FMCW radar is from 76 GHz to 81 GHz, and the parameter settings are shown in Table 1: Table 1 One transmitting antenna and four receiving antennas are adopted, and the equivalent bandwidth is: B eq = β × N / f s (1) Among them, B eq refers to the equivalent bandwidth, β is the chirp slope, N is the effective ADC samples in the chirp, f s is the ADC sampling rate, and the resulting range resolution ΔR is approximately 9.8 cm, and the calculation formula is as follows: where c is the speed of light and the maximum speed V max can be calculated as The working frequency of the FMCW radar changes within the linear frequency modulation period, and the transmitted signal is expressed as x T x(t) = A T cos(2πft c + πβt 2 ) (4) Among them, A T is the amplitude of the transmitted wave, f c is the starting frequency of the chirp signal, and β is the chirp slope.

3. The method for continuous action segmentation and recognition of a radar human body assisted by digital features as claimed in claim 1 or 2, characterized in that, In the above Step 1, using the millimeter-wave radar to transmit signals, a human performs 7 action changes such as boxing, falling, raising hands, sitting down, squatting, standing up, and walking in front of the radar. Each action lasts for 1 - 2 s, and there are 6 continuous action changes such as sitting-standing up-falling, walking-falling-standing up, walking-sitting down-raising hands, sitting down-standing up-walking, bowing-sitting down-standing up, and boxing-walking-bowing. Each continuous action lasts for 3 - 4 s. Use the DAC1000EVM acquisition board to transmit the received intermediate-frequency radar human action echo data to the computer for storage, and store it in complex form.

4. The method for continuous action segmentation and recognition of radar human body assisted by digital features as claimed in claim 1, wherein The process of the above Step 2 is: (2.1) According to the set radar parameters, the original intermediate frequency signal data collected by the acquisition card is divided into multiple channel data and saved as complex I / Q data, that is, the.bin file of the intermediate frequency echo is obtained. Then, the data of each antenna is read from it and organized into a three-dimensional array of [n_sample, n_chirp, n_frame], where n_frame represents the number of sampling frames, n_chirp represents the total number of chirps included in each frame, and n_sample represents the number of sampling points included in each chirp. Then, frame difference processing is performed on it, that is, the data of the latter frame is subtracted from the data of the previous frame. Finally, the two-dimensional echo sequences of the I-channel and Q-channel after frame difference are output; (2.2) Perform FFT on each frame of data after frame difference processing in the fast time dimension to obtain the distance information of the action; then perform FFT in the slow time dimension on this basis to obtain the distance and speed information of the action, that is, the RDM is obtained. Here, RD(i, j, t) is used to represent the signal power value at the i-th distance gate and the j-th speed gate of the RDM at the t-th frame; (2.3) Perform velocity dimension projection on the RDM, that is, compress the i dimension, and finally obtain the required micro-Doppler time-frequency diagram. The formula for projecting the velocity dimension is as follows: DT(t,j) = ∑ i RDM(i,j,t); (2.4) Accumulate the energy values of each point at the same time on the micro-Doppler time-frequency diagram to obtain the energy distribution diagram of a single human action signal. The formula is as follows EnergyMap(j) = ∑ t DT(t, j).

5. The method for segmenting and recognizing continuous human actions of a dual-threshold radar assisted by digital features according to claim 4, wherein In step 3, the obtained continuous action micro-Doppler time-frequency diagram and energy distribution diagram are segmented for continuous actions by finding the minimum value points of the digital feature energy distribution, supplemented by energy thresholds and time thresholds, to obtain the data units of individual actions. The process is as follows: (3.1) First, set parameters including the number of frames read each time, action identifier, energy threshold, and time threshold. First, judge whether it is the start or end of an action by whether the energy value is a minimum value point. At the same time, the energy threshold is an important basis for judging whether an action starts and ends. On the premise that the energy value of this frame is a minimum value, only the points with energy values less than the energy threshold will be used as the start or end identification points of the action. To facilitate the system's detection of actions, an action identifier is used to help distinguish whether the current is in an action or not. At the same time, considering the errors existing in practical applications, to prevent misjudging actions, a time threshold is added to ensure the validity of the action. The action cut out by the method of the energy threshold must meet the time threshold to be regarded as a real action. Considering the real-time nature of fall detection, data needs to be read in real time. Therefore, a number of frames read each time is set to meet the real-time requirements; (3.2) Then, preprocess the radar data. Read a certain number of frames each time, process each frame to obtain the micro-Doppler map. Take out the current frame in the micro-Doppler map, compare the energy of the 128 sampling points in the current frame with the background energy. Those greater than the background energy are set to 1, and then sum them up. The calculated energy sum value reflects the action energy magnitude of the current frame to a certain extent. Store this energy sum value in an array for smoothing processing; (3.3) Since the array stores the energy sum values of the previous frames, exponential smoothing can be used to gradually approximate the true value by weighted averaging the energy sum value of the current frame and the previous smoothed value; (3.4) Check the energy sum value of the current frame. If a minimum value point appears in the energy points, and this point is less than the energy threshold and the action identifier indicates that the current is not in an action, then mark the current frame as the start frame of the action and set the action identifier to 1 to indicate the start of the action; (3.5) If a minimum value point appears in the current frame, and it is less than the energy threshold and the current identifier indicates that it is in an action, then calculate the duration between the current frame and the start frame of the action. If it is greater than the time threshold, set the end frame of the action to the previous frame of the current frame, update the start frame of the action, set the action identifier to 0 to indicate the end of the action, and crop, grayscale, and save the micro-Doppler map from the start frame to the end frame of the action; (3.6) If a minimum value point appears in the current frame, but it is less than the energy threshold and indicates that it is in an action but the duration does not meet the time threshold, then continue to set the action identifier to 1 to indicate that the action continues, and continue to find the next frame with a sum value less than the energy threshold; (3.7) At the same time, to prevent the action time from being too long and the action identifier not being found, when the number of frames is sufficient, use the current frame as the end frame; (3.8) Through the above steps, continuous human actions are segmented into individual actions.

6. A method for segmenting and recognizing continuous human actions of a dual-threshold radar assisted by digital features as claimed in claim 1 or 2, characterized in that The process of step 4 is as follows: (4.1) Integrate the time-frequency map and energy distribution map of the individual actions obtained in step 3 into a dataset, and divide it into a training set, a validation set, and a test set according to a preset ratio. The training set is used for model training, the validation set is used for hyperparameter tuning and performance verification of the model, and the test set is used to evaluate the final performance of the model; (4.2) Preprocess the data in the training set, including operations such as normalization and data augmentation, to improve the generalization ability and training efficiency of the model; (4.3) Input the preprocessed training set into the dual-stream multi-scale attention network. Extract the features of individual actions through the forward propagation of the network. The network structure includes a feature extraction module, a multi-scale attention module, and a fully connected layer. The feature extraction module is used to extract key features from the time-frequency map and energy distribution map, the multi-scale attention module is used to enhance the expression ability of the features, and the fully connected layer is used to map the extracted features to the category label space to complete the classification task; (4.4) During the training process, use the cross-entropy loss function as the optimization objective, update the network parameters through backpropagation, and at the same time use the Adam optimizer for gradient descent to minimize the loss function value; (4.5) After each training cycle, the model is validated using the validation set, the loss value and accuracy on the validation set are recorded, and the optimal model parameters are selected and saved according to the performance metrics of the validation set, where the optimal model refers to the model with the highest accuracy and the lowest loss value on the validation set; Finally, an optimized parameter model is obtained for subsequent human action recognition tasks.

Images