Millimeter wave radar fall detection method based on improved ChanVese model
By improving the Chan_Vese model and transfer learning method, the problem of strong trunk echo interference and small data sets in millimeter wave radar fall detection is solved, and the high-accuracy fall detection is achieved, which improves the recognition rate and model generalization ability.
Patent Information
- Application Number
- CN202410933110.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-12
- Publication Date
- 2024-11-12
AI Technical Summary
In the existing millimeter-wave radar fall detection technology, strong trunk echo interference is difficult to suppress, resulting in low extraction accuracy of micro Doppler effect, and the overfitting problem caused by small batch data sets during model training.
The improved Chan_Vese model is used to combine Canny edge detection and median filter to automatically extract the initialization profile, and the dual attention mechanism network is trained through transfer learning method to improve the extraction accuracy of the micro Doppler effect and the generalization ability of the model.
It significantly improves the recognition rate and edge extraction accuracy of fall detection, reduces the cross-entropy loss during training, and improves the generalization ability and recognition rate of the model, especially in the recognition of fall behaviors, which achieves 100% accuracy.
Smart Images

Figure CN118942151A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of fall detection, and in particular to a millimeter wave radar fall detection method based on an improved Chan-Vese model. Background Art
[0002] As an important branch in the field of radar signal processing, the human body micro-Doppler effect has played an increasingly critical role in applications such as fall detection and behavior recognition in recent years. However, the presence of strong torso echoes, on the one hand, masks subtle movement characteristics, making it difficult to distinguish between fall behavior and other daily behaviors; on the other hand, it also increases the complexity of signal processing and places higher demands on the real-time and robustness of the algorithm. Therefore, how to effectively suppress the interference of strong torso echoes and improve the extraction accuracy of the micro-Doppler effect has become a key issue that needs to be urgently addressed in the field of fall detection.
[0003] In actual environments, radar data collection and preprocessing is a time-consuming process, so when training models, we often face the challenge of small batch data sets. Small data sets make model training difficult and lead to overfitting problems during model training. Summary of the invention
[0004] The present invention provides a millimeter wave radar fall detection method based on an improved Chan_Vese model, which effectively suppresses the interference of strong echoes from the trunk and improves the extraction accuracy of the micro-Doppler effect.
[0005] In order to achieve the purpose of the present invention, the technical solution adopted is: a millimeter wave radar fall detection method based on an improved Chan_Vese model, comprising:
[0006] 1) Obtain the radar echo data of the target through the radar equipment and convert it into the time-frequency image of the radar echo;
[0007] 2) After obtaining the time-frequency image of the radar echo, the improved Chan_Vese model is used to extract the rough contour of the target area. The improved Chan_Vese model introduces the Canny edge detection algorithm to automatically extract the initialization contour;
[0008] 3) After the rough outline is extracted, a median filter is used for smoothing to obtain a binary image after time-frequency image preprocessing;
[0009] 4) Classification network: Divide the preprocessed images into training set and test set. Send the training set to the dual attention mechanism network and use the transfer learning method to complete the network training;
[0010] 5) Fall detection: Send the test data to the trained network to obtain the activity type of the human body. As an optimization solution of the present invention, in step 2), the steps of automatically extracting the initialization contour are as follows:
[0011] A. Use Canny edge detection to detect the edge of the time-frequency image of the radar echo and output the edge detection result;
[0012] B. Further process the results of Canny edge detection, traverse all edge points, analyze their distribution and continuity, and create a polygon that encloses all edge features of interest based on the distribution of edge points;
[0013] C. Convert the polygon constructed in the previous step into the initialization contour format required by the Chan-Vese algorithm. The Chan-Vese algorithm will optimize the position and shape of the contour through iterative calculation based on this initialization contour and image information to accurately segment different areas in the image.
[0014] As an optimization solution of the present invention, the process of Canny edge detection is:
[0015] a. Convert the time-frequency color image of radar echo into a grayscale image. The formula used for grayscale conversion is:
[0016] Gray=0.299R+0.587G+0.114B
[0017] Among them: Gray represents the pixel value of the grayscale image, R, G, and B represent the values of the red, green, and blue color channels of the corresponding pixels in the original image respectively;
[0018] b. Perform Gaussian filtering on the grayscale image;
[0019] c. Calculate the gradient magnitude and direction, and use the template of the Sobel operator to calculate the horizontal difference G x and vertical difference G y , then the formula for the gradient magnitude G and direction is as follows:
[0020]
[0021] The gradient angle θ ranges from [-π,π];
[0022] d. Non-maximum suppression: After obtaining the magnitude and direction of the gradient, traverse the pixels in the image and remove non-edge points. Non-maximum suppression can refine the edges.
[0023] e. Double-threshold edge connection. The Canny edge detection algorithm uses a double-threshold mechanism to accurately define edge pixels. For pixels whose gradient values are lower than the set low threshold, the algorithm will suppress them and assign them a value of 0, thereby effectively eliminating non-edge information; and for pixels whose gradient values are between the low threshold and the high threshold, they will be marked as weak edge points for further processing; only when the gradient value of the pixel exceeds the high threshold, it will be clearly classified as a strong edge point.
[0024] As an optimization scheme of the present invention, after obtaining the preliminary edge detection result through Canny edge detection, the Canny edge detection algorithm will determine whether each weak edge point is connected to the strong edge point. If the virtual edge is connected to the strong edge, then the virtual edge point will be processed as a real edge point; otherwise, it will be suppressed to avoid interference from non-edge information.
[0025] As an optimization scheme of the present invention, the radar echo data is rearranged and expressed as a two-dimensional matrix S(m,n), wherein m=0,1,2…M-1, n=0,1,2…N-1, M refers to the number of sampling points of the radar in one frequency modulation cycle, and N refers to the total number of frequency modulation cycles of the radar. Each column of the two-dimensional matrix S(m,n) is subjected to FFT transformation to obtain distance distribution information. A Butterworth high-pass filter with an order of 4 and a normalized cutoff frequency of 0.008 is used for clutter processing. The distance distribution information after clutter processing is accumulated over time into a distance-time spectrogram. The two-dimensional matrix of the distance-time spectrogram is summed along the slow time dimension to obtain the amplitude information of the one-dimensional time series signal after normalization processing, and a time series diagram in the time domain is obtained. The time series diagram in the time domain is converted into a time-frequency diagram in the time-frequency domain for further feature extraction.
[0026] As an optimization solution of the present invention, STFT and SPWVD time-frequency analysis is performed on the radar echo data of the target to obtain the corresponding time-frequency image.
[0027] As an optimization scheme of the present invention, the median filter smoothing processing includes: first, defining a moving window of an odd size, and aligning the center point of the window with the target pixel in the image; then, reading the grayscale values of all pixels in the window, and sorting them in order from small to large or from large to small; then, finding the grayscale value of the pixel at the middle position from the sorted grayscale sequence; finally, assigning this median grayscale value to the pixel at the center position of the window in the original image, and repeating this process for the entire image, that is, moving the window to cover each pixel of the image, to complete the median filtering of the entire image.
[0028] As an optimization scheme of the present invention, a model training method of transfer learning is introduced, and the dual attention model is pre-trained using the ImageNet dataset. After the pre-training process is completed, the parameters of the model are saved, and all network layers except the fully connected layer and the network input end are frozen to retain the learned feature representation; subsequently, this pre-trained model is migrated to the human fall dataset and used as the initial parameter setting for the human fall detection task; finally, after the trained features are processed by the fully connected layer, the SoftMax classifier is used to output the classification results, and the classification results include walking, sitting down, standing up, picking up things, drinking water and falling.
[0029] The present invention has positive effects: 1) The present invention significantly improves the recognition rate: The C-Chan-Vese model shows significant advantages in human activity recognition, especially in fall detection, the recognition rate is improved, especially in the recognition of fall behavior, the accuracy rate reaches 100%;
[0030] 2) The present invention improves edge extraction: the Cchan-Vese model can converge to the edge of human behavior more accurately, avoiding the error of color inversion inside and outside the contour, thereby improving the accuracy of feature extraction;
[0031] 3) The present invention enhances the generalization ability of the model: The binary graph dataset processed by the Cchan-Vese model, combined with the attention mechanism model, not only reduces the cross entropy loss during the training process, but also shows a good recognition rate on both the training and test sets, confirming the generalization ability and stability of the model;
[0032] 4) The present invention optimizes the performance of different data sets: Whether on the time-frequency graph data set generated by STFT or SPWVD, the recognition rate of the binary graph data set processed by the CChan-Vese model is significantly improved. The recognition rates of the STFT and SPWVD methods are increased to 96.94% and 98.33%, respectively, proving the superiority of the CChan-Vese model in extracting features.
[0033] 5) The present invention improves the effect through transfer learning: the recognition accuracy of the model trained using the transfer learning strategy is improved by 0.84% compared with the model not using transfer learning, which confirms the effectiveness of transfer learning and can help the model better understand and classify human activities, especially improve the recognition accuracy of falling behavior. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.
[0035] Figure 1 It is a schematic diagram of the process of the present invention;
[0036] Figure 2This is the time-frequency image processing flow of the present invention;
[0037] Figure 3 It is a binary image processed by the median filter of the present invention;
[0038] Figure 4 It is a schematic diagram of the test confusion matrix of the present invention integrating binary image features;
[0039] Figure 5 It is a binary image generated after processing the fall data;
[0040] Figure 6 is the cross entropy loss value curve of the training set and the test set;
[0041] Figure 7 is the recognition rate curve of the training set and the test set;
[0042] Figure 8 Schematic diagram of recognition rate for time-frequency graph and binary graph test;
[0043] Fig. 9 Schematic diagram of network recognition rate for each classification. DETAILED DESCRIPTION
[0044] The method is further described in detail below with reference to the accompanying drawings and specific implementation methods.
[0045] like Figure 1 As shown, the present invention discloses a millimeter wave radar fall detection method based on an improved Chan_Vese model, comprising:
[0046] 1) Obtain the radar echo data of the target through the radar equipment and convert it into the time-frequency image of the radar echo;
[0047] 2) After obtaining the time-frequency image of the radar echo, the improved Chan_Vese model is used to extract the rough contour of the target area. The improved Chan_Vese model introduces the Canny edge detection algorithm to automatically extract the initialization contour;
[0048] 3) After the rough outline is extracted, a median filter is used for smoothing to obtain a binary image after time-frequency image preprocessing;
[0049] 4) Classification network: Divide the preprocessed images into training set and test set. Send the training set to the dual attention mechanism network and use the transfer learning method to complete the network training;
[0050] 5) Fall detection: Send the test data into the trained network to obtain the human body activity type.
[0051] The radar echo data is rearranged and represented as a two-dimensional matrix S(m,n), where m=0,1,2…M-1, n=0,1,2…N-1, M refers to the number of sampling points of the radar in one frequency modulation cycle, and N refers to the total number of frequency modulation cycles of the radar. Each column of the two-dimensional matrix S(m,n) is transformed by FFT to obtain the distance distribution information. A Butterworth high-pass filter with an order of 4 and a normalized cutoff frequency of 0.008 is used for clutter processing. The distance distribution information after clutter processing is accumulated over time into a distance-time spectrogram. The two-dimensional matrix of the distance-time spectrogram is summed along the slow time dimension to obtain the amplitude information of the one-dimensional time series signal after normalization, and the time series diagram in the time domain is obtained. The time series diagram in the time domain is converted into a time-frequency diagram in the time-frequency domain for further feature extraction. The time series diagram represents the amplitude change of the human body echo energy over time. However, the amplitude of each human activity has non-stationary changes, and it is difficult to identify different human activities only by the time and amplitude of the time series diagram. The time-frequency diagram obtained by STFT and SPWVD is selected as the image for feature extraction. The complete flow chart for generating STFT time-frequency diagram and SPWVD time-frequency diagram.
[0052] When a person walks or falls, the movement trajectory and posture changes of their limbs will form obvious edges in the image. These edge features are key to identifying different behavior patterns. However, due to the presence of strong echoes from the torso, these edge feature information is often weak and difficult to extract and analyze directly. The CChan_Vese model preprocesses the time-frequency image. By applying it to the processing of millimeter-wave radar signals, it can effectively suppress the strong echoes from the human torso, thereby highlighting the edge feature information in the micro-Doppler signal.
[0053] The processing steps of radar echo signal are as follows. First, the echo signal of the target is obtained by the radar device and converted into a time-frequency image. After obtaining the time-frequency image of the radar echo, the Cchan_Vese model (improved Chan_Vese model) is used to extract the rough contour of the target area. After the rough contour extraction is completed, in order to further reduce the noise and refine the contour, the median filter is used for smoothing. Finally, the binary image after the preprocessing of the time-frequency image is obtained. In the human fall detection system, the time-frequency image of the radar mainly reflects the relationship between the change of Doppler frequency shift over time during human activity, while the binary image can fully retain these main features and greatly simplify the image representation. In the binary image, the target area is presented in the form of continuous white pixel blocks, while the background is presented as black pixel blocks. This simplified representation method not only reduces noise and interference, making the edges and contours clearer, but also significantly reduces the size of the input data, thereby greatly reducing the computational cost.
[0054] The CChan_Vese model inherits the core idea of the Chan_Vese model. By optimizing the energy function, the original time-frequency graph is divided into different regions, thereby highlighting the edge features and weakening the influence of strong echoes of the trunk. After being processed by the CChan_Vese model, the edge and contour information in the image is significantly enhanced. In order to further improve the quality of the image, the median filter is used to smooth the processed image. The median filter can not only effectively remove outliers, but also retain the key feature information in the image, thereby obtaining a smooth and clear binary time-frequency feature map, that is, a binary map.
[0055] The Chan-Vese model algorithm can distinguish the human body from the background based on the grayscale, color or other feature information of the image, and then extract the human body's contour. The algorithm flow is:
[0056] 1. Initialize the contour. Define an initialization contour C, which is a closed interval.
[0057] 2. Energy function. Define an energy function E(C), which is the combination of internal energy E(C1) and external energy E(C2), as shown in equations (1.1), (1.2) and (1.3):
[0058]
[0059] E(C)=E(C1)+E(C2)(1.3)
[0060] Where C is a closed interval on the image, in(C) represents the closed area inside the curve, out(C) represents the area outside the curve, C1 represents the grayscale average inside the curve on the image, C2 represents the grayscale average outside the curve on the image, and I0(m,n) represents the grayscale value of point (m,n) on the image.
[0061] 3. Iterative optimization. The position of the contour C is iteratively updated by minimizing the energy function. The energy minimization formula is:
[0062] E(C)=μLength(C)+νArea(C)+E1(C)+E2(C)(1.4)
[0063] In the formula, Length(C) represents the length of contour C, Area(C) represents the area of contour C, μ is used to control the weight of the contour length in the energy function, and ν is used to control the weight of the area inside the contour in the function. is the level set function constructed based on the closed contour curve. is a signed distance function with positive inside and negative outside, curve C and level set function The relationship between is:
[0064]
[0065] In the formula, Ω represents the time-frequency image, and the level set is used to divide the region into two inner and outer regions. The Chan-Vese model is numerically solved using the level set method, that is, C in the energy is converted into a level set function. The form of expression. And define the Heaviside function H and Dirac function δ, the formulas are as follows (1.6) and (1.7):
[0066]
[0067] The Chan_Vese model is expressed as formula (1.8) through formula (1.6) and (1.7):
[0068]
[0069] In the formula, λ1 and λ2 are the weight coefficients of each energy term, C1 and C2 are regarded as constants in the energy function, which can be expressed as follows (1.9) and (1.10):
[0070]
[0071] By using the variational method for formula (1.8) and introducing the time variable t, we can obtain the gradient descent flow of the Chan_Seve model, that is, the level set form evolution equation of the Chan_Vese model as shown in formula (1.11):
[0072]
[0073] In actual calculations, the Heaviside function and the Dirac function are regularized; expressed as formulas (1.12) and (1.13):
[0074]
[0075] The smaller the value of ξ, the closer it is to the ideal step function. Then, the evolution equation of the level set function is discretized using the equally divided grid and the finite difference implicit scheme. Let h be the spatial step length, Δt be the time step length, and l be the number of moments. Then, (xi,yj)=(ih,jh) is a point in the equally divided grid. Let is the level set function The discretization approximation of is expressed and defined as formula (1.14):
[0076]
[0077] According to the current level set function value First, the grayscale averages of the inner and outer regions of the zero level set closed curve are calculated respectively. and Then calculate according to formula (1.15)
[0078] This discretized equation is a linear system that finds the optimal contour bounding box through iteration.
[0079] ④ Set the stop condition, and stop when the iteration reaches the preset number of iterations. The final contour C is a binary image, with 1 inside the contour and 0 outside.
[0080] (2) Canny edge detection algorithm
[0081] The Canny edge detection algorithm plays an important role in the field of edge detection due to its advantages such as high accuracy, low error rate, simplicity and parameter adjustability. Its core idea is to find the position with the largest gradient change in the image, thereby determining the edge position. The edge detection process is as follows:
[0082] ①Convert color images to grayscale images. The Canny algorithm is often used to process grayscale images, and the time-frequency images obtained by radar data through time-frequency technology are color images, which must be converted to grayscale before processing. The formula used for grayscale conversion is as follows:
[0083] Gray=0.299R+0.587G+0.114B(0.1)
[0084] ② Gaussian filtering. Mainly used to reduce the impact of noise on edge detection.
[0085] ③ Calculate the gradient magnitude and direction. Use the template of the Sobel operator to calculate the horizontal difference G x and vertical difference G y , then the formulas for the gradient magnitude and direction are as follows (4.17) and (4.18):
[0086]
[0087] The gradient angle θ ranges from [-π,π].
[0088] ④ Non-maximum suppression. After obtaining the magnitude and direction of the gradient, traverse the pixels in the image and remove non-edge points. Non-maximum suppression can refine the edges.
[0089] ⑤Dual threshold edge connection. The Canny edge detection algorithm uses a dual threshold mechanism to accurately define edge pixels. This strategy significantly improves the accuracy and robustness of edge detection. Specifically, for pixels whose gradient values are lower than the set low threshold, the algorithm will suppress them and assign them a value of 0, thereby effectively eliminating non-edge information. For pixels whose gradient values are between the low threshold and the high threshold, they will be marked as weak edge points for further processing. Only when the gradient value of a pixel exceeds the high threshold will it be clearly classified as a strong edge point. After obtaining these preliminary edge detection results, the algorithm will perform further processing. It will determine whether each weak edge point (i.e., virtual edge) is connected to a strong edge point. If the virtual edge is connected to the strong edge, then the virtual edge point will be processed as a true edge point; otherwise, it will be suppressed to avoid interference from non-edge information.
[0090] (3) Cchan_Vese model algorithm
[0091] The Chan-Vese model is a region-based image segmentation method. However, the model requires a manual setting of a contour in the initialization stage. This process is easily affected by the operator's subjective consciousness and experience, resulting in inconsistency in the segmentation results. In addition, the position of the initial contour has a great influence on the convergence and results of the algorithm. Improper selection may cause the algorithm to fail to converge correctly. Therefore, in order to solve the limitations of manually selecting the initial contour, a Chan-Vese model is proposed. Based on the traditional model, the Canny edge detection algorithm is introduced to automatically extract the initialization contour. The specific steps for automatically extracting the initialization contour are as follows:
[0092] ① Use Canny edge detection to detect the edges of the original time-frequency image and output the edge detection results.
[0093] ② Further process the results of Canny edge detection. Traverse all edge points and analyze their distribution and continuity. Based on the distribution of edge points, create a polygon that surrounds all edge features of interest. This polygon fits the edge as closely as possible while ensuring that no important edge information is missed.
[0094] ③Convert the polygon constructed in the previous step into the initialization contour format required by the Chan-Vese algorithm. Based on this initialization contour and image information, the Chan-Vese algorithm will optimize the position and shape of the contour through iterative calculation to best segment different areas in the image.
[0095] exist Figure 2 In (a), the time-frequency image of the falling behavior after STFT processing is clearly presented, which contains rich time-frequency feature information. Figure 2(b) shows the result of edge extraction of the time-frequency image using the Canny edge detection algorithm. The generated edge feature map effectively highlights the contours and key dynamic information of the falling behavior. Figure 2 In (c), based on the edge feature map of the Canny detection algorithm, the Chan-Vese algorithm is further applied to extract the binary image containing time-frequency features.
[0096] Median filtering is an effective method for processing digital images and sequences, mainly used to eliminate outliers in data. It reduces the impact of outliers in images or sequences by replacing each pixel value with the median of the pixel values in its neighborhood, making the data smoother and clearer. This filtering method is widely used in the field of image processing because it can better preserve image edges and details while removing noise. The specific operation process is as follows: First, define a moving window of odd size, and align the center point of the window with the target pixel in the image; then, read the grayscale values of all pixels in the window and sort them in order from small to large or from large to small; then, find the grayscale value of the pixel in the middle position from the sorted grayscale sequence, and finally assign this median grayscale value to the pixel at the center of the window in the original image.
[0097] The binary image generated by the CChan_Vese model does have many abnormal points on the edge of the contour, which interfere with the accurate extraction of the fall behavior characteristics. In order to eliminate these abnormal points, the median filter is used to further process the binary image generated by the CChan_Vese model. Figure 3 As shown in Figure 1, the binary image processed by the median filter not only effectively removes the outliers, but also makes the contours in the binary image clearer and more accurate. By comparing the images before and after processing, it can be clearly seen that the median filter significantly improves the overall quality of the image while protecting the edge information.
[0098] Radar data is small batch data, and insufficient data sets lead to overfitting and poor generalization of network models. To this end, transfer learning methods are used to enhance model performance. In the field of image classification, the ImageNet dataset is an ideal choice for transfer learning due to its large scale and rich categories. It contains more than 1.2 million images covering 1,000 different categories. Although there are obvious differences in the nature of ImageNet's image data and FMCW radar datasets, the underlying features of images, such as color, texture, and edges, are highly universal.
[0099] A network-based transfer learning method was adopted. Network-based transfer learning refers to reusing and integrating a part of the pre-trained neural network, including the network structure and connection parameters, into the neural network in the target domain as the initial part of the model. By migrating the knowledge of the source domain network, this method can achieve good performance on the target task faster, especially when there is less data in the target domain. By migrating the network parameters pre-trained on the ImageNet dataset to the model for FMCW radar data, the common features learned by the pre-trained model can be effectively utilized. This transfer learning strategy based on pre-trained models demonstrates its flexibility and effectiveness when dealing with datasets with essential differences. Through careful adjustment and fine-tuning, the pre-trained network can adapt to new task requirements and further improve the performance of human fall detection.
[0100] Human fall detection framework based on transfer learning. First, a human fall dataset is constructed through time-frequency analysis and time-frequency image preprocessing. Then, a model training method of transfer learning is introduced. The dual attention model is pre-trained using the ImageNet dataset. After the pre-training process is completed, the parameters of the model are saved and all network layers except the fully connected layer and the network input are frozen to retain the learned feature representations. Subsequently, this pre-trained model is transferred to a human fall dataset specifically for FMCW radar signals and used as the initial parameter setting for the human fall detection task. Finally, after the trained features are processed by the fully connected layer, the SoftMax classifier is used to output the classification results, where the classification results include walking, sitting down, standing up, picking up things, drinking water, and falling.
[0101] The "IWR1642+DCA1000" radar combination solution of Texas Instruments is used as the core measurement tool. The experiment uses MMWAVE-STUDIO to set radar working parameters, send acquisition instructions, and perform a series of operations such as data saving. In the parameter design, a high-frequency carrier frequency (77GHz) is selected to obtain a higher resolution to meet the needs of subtle movements such as falls of the elderly. The selection of bandwidth (3996MHz) ensures that the system has good distance resolution and can accurately distinguish targets with similar distances, thereby accurately detecting fall events. In addition, the high frequency modulation slope (66.6MHz / us) helps to capture the speed changes during the fall process, and the 5MHz ADC sampling rate ensures accurate signal acquisition. In the selection of the number of chirps and one frame time, the requirements of speed resolution and data update frequency are taken into account to ensure timely response to fall events. Finally, a sufficient number of frames provides ample observation time windows, which is helpful for the analysis and judgment of individual behavior patterns. In summary, the rationality of these parameter designs ensures that the millimeter-wave radar-based elderly fall detection system has efficient, accurate and reliable performance in practical applications. At the same time, in order to fully capture radar data, the experiments were conducted in two different environments, one is a well-lit corridor and the other is a relatively dim dormitory. In order to provide sufficient detection range, the installation height of the radar device is set at 1m.
[0102] In the experiment of this radar system, a total of 6 participants (including 5 men and 1 woman) participated in the radar echo data collection of a series of different actions. For each specific action, each participant performed 40 repetitions of action collection, each lasting 5 seconds. Through this process, a total of 1440 sets of data on human behavior were collected. All data collection was completed within the detection area of the radar system, covering 6 different human actions. The description of each action is as follows:
[0103] (1) Walking. The subject walks in a normal gait, approaching the radar antenna and moving away from the radar antenna to simulate walking movements at different distances and in different ways.
[0104] (2) Sitting down. The subject moves from a standing position facing the radar antenna to a chair, simulating a sitting action.
[0105] (3) Standing up: The subject stands up normally from a sitting position, accompanied by small limb movements and adjustments of the center of gravity.
[0106] (4) Picking up things. The subject simulates the normal action of bending over to pick up an empty bottle or a ball of paper on the ground.
[0107] (5) Drinking water. The subject simulates the action of drinking water, including the whole process of lifting the bottle to drink water and then putting his hand down.
[0108] (6) Falling. When collecting fall data, the subject falls from different directions within the radar detection area, including facing the radar, facing away from the radar, from the left side of the radar, and from the right side of the radar.
[0109] 1) Construction of human fall dataset: Generate STFT time-frequency diagram and SPWVD time-frequency diagram.
[0110] (1) Generation of STFT and SPWVD time-frequency diagrams
[0111] STFT analyzes the time-frequency characteristics of the signal by dividing the signal into several short time segments and then performing Fourier transform on each time segment. The formula of STFT is shown in formula (4.19).
[0112]
[0113] In the formula, w(t) is the window function, X(t,f) is the generated micro-Doppler feature, and τ is the time shift parameter, which determines the moving position of each window on the time axis. The Hamming window used by the window function has a length of 200 samples. The data overlap rate corresponding to each movement of the Hamming window is 95% to minimize the impact of spectrum leakage. Then the modulus is taken to obtain the STFT time-frequency diagram. The STFT time-frequency diagram can reveal the subtle changes in human body movements by showing the changes in speed over time.
[0114] The SPWVD algorithm distributes the energy of the signal in time and frequency to generate an SPWVD time-frequency diagram. SPWVD overcomes the limitation of STFT that it cannot simultaneously meet high time and frequency resolution, while reducing the interference of cross terms in the Wigner transform. The SPWVD time-frequency diagram can more clearly show the energy distribution of the signal at different times and frequencies. The formula of the SPWVD algorithm is shown in formula (4.20).
[0115]
[0116] In the formula, is a time window function with a length of 128 samples, is the frequency window function with a length of 32 samples. Both the time window function and the frequency window function are Hanning windows, where X(t,f) is the generated SPWVD time-frequency diagram.
[0117] (2) Generation of STFT binary images and SPWVD binary images
[0118] When constructing the data set, the radar signal feature extraction process is followed. First, the data is analyzed in time and frequency to obtain the STFT time-frequency diagram and SPWVD time-frequency diagram. These two types of time-frequency diagrams effectively reveal the time-frequency characteristics of human activities. Then, the CChan_Vese model algorithm is used to preprocess the time-frequency diagram. After this step, the STFT time-frequency diagram is converted into a clearer STFT binary diagram. The binary diagram not only simplifies the image information, but also highlights the key features of human activities.
[0119] (3) Experimental results analysis of Chan-Vese and Chan-Vese models
[0120] In the experiment, the data generated by the traditional Chan-Vese model and the data generated by the Chan-Vese model were sent to the classification network for training respectively. In order to ensure the full utilization of data and the improvement of model performance, the two data sets were fused and input into the classification network for training. Specifically, the STFT binary image is obtained by STFT transformation, while the SPWVD binary image is generated based on SPWVD transformation. In the fusion process, the channel splicing method is adopted, that is, the two images are merged according to the channel dimension, so as to make full use of their respective advantages in feature extraction. In addition, in order to improve the generalization ability of the model, the dual attention mechanism model pre-trained on the ImageNet dataset is used as the fall model. Finally, the classification result of the model, that is, the type of human activity, is obtained.
[0121] Figure 4 The confusion matrix of the Chan-Vese model and CChan-Vese model in the test phase is shown. Comparing the recognition rates of the two models, it can be seen that the recognition rate of the traditional Chan-Vese model is 96.39%, while the improved CChan-Vese model reaches 98.89%, an increase of about 2.50%. A detailed analysis of the recognition of each behavior category shows that the Chan-Vese model performs relatively poorly in identifying fall behaviors, with the lowest recognition rate of only 0.92. Some fall data are misclassified as walking and sitting, which leads to false positives in practical applications. In contrast, the CChan-Vese model achieves a 100% recognition rate in identifying fall behaviors, showing a significant advantage. For other behavior categories, such as walking, sitting, and picking up things, the recognition rates of the two models remain consistent, showing high accuracy. In addition, the CChan-Vese model is also slightly better in identifying standing up and drinking water behaviors, with recognition rates 0.05 and 0.03 higher than the Chan-Vese model, respectively. In summary, the CChan-Vese model algorithm shows significant advantages in human fall detection when processing radar data and generating corresponding data sets.
[0122] Due to the obvious difference in falling behaviors, the falling data set was analyzed. The binary images obtained after processing some falling data are as follows Figure 5 As shown in the figure, it can be clearly observed that the traditional Chan-Vese model has some problems in processing these data. Specifically, the contours of some time-frequency graphs fail to converge accurately to the edge of human behavior, resulting in inaccurate edge information extraction. In addition, under normal circumstances, the inside of the contour should be white, and the background outside the contour should be black, but the traditional method has some data errors when extracting the contour. This is mainly due to improper operation when manually selecting features, which makes the algorithm perform poorly when processing complex or special image data.
[0123] In contrast, the CChan-Vese model shows significant advantages in solving these problems. It effectively solves the problem that the manually selected initial contour cannot converge correctly, making contour extraction more accurate and reliable. At the same time, the CChan-Vese model also improves the efficiency of the overall algorithm, making it more efficient and faster when processing large-scale data sets.
[0124] Figure 6 It clearly shows the changing trend of the cross entropy loss value on the training set and test set when using the CChan-Vese model. During the training process of the model, the value of the loss function gradually decreases, which shows that the model is continuously improving its performance through the optimization process and reducing the difference between the predicted value and the actual label. As time goes by, the loss value gradually stabilizes, indicating that the model has fitted the training data well and has a certain generalization ability. At the same time, Figure 7 The figure shows how the recognition rate on the training and test data sets changes with the number of iterations when using the CChan-Vese model. As can be seen from the figure, as the number of iterations increases, the accuracy of the model gradually increases, which means that the model's ability to classify different categories of human behavior is constantly improving. After about 40 iterations, the recognition rate reached a relatively stable level, indicating that the model has fully learned the characteristics of the data. Combining these two figures, it can be concluded that the use of the CChan-Vese model shows good performance in the training and testing process, not only the loss value gradually decreases, but also the recognition rate continues to improve. This fully proves the effectiveness and reliability of the CChan-Vese model in the task of human behavior recognition.
[0125] (2) Analysis of experimental results of different data sets and classification networks
[0126] In order to verify the advantages of the CChan-Vese model in extracting binary time-frequency features, this experiment sent the time-frequency graph dataset generated based on STFT and SPWVD and the binary graph dataset generated by the CChan-Vese model into the dual attention network for training. Figure 8 The performance analysis results of different data sets under the model are intuitively displayed. In the time-frequency graph dataset, the recognition rate of the STFT method is 95.86%, while the SPWVD method reaches 97.24%. This result shows that there are indeed performance differences between different time-frequency transformation methods in the task of human behavior recognition, and the SPWVD method has a higher recognition rate than the STFT method. However, when observing the performance of the binary graph dataset generated by the CChan-Vese model, it is found that its recognition rate has been significantly improved. In the binary graph dataset, the recognition rate of the STFT method is increased to 96.94%, while the SPWVD method reaches 98.33%. Compared with the original time-frequency graph dataset, the recognition rates are increased by 1.08% and 1.09%, respectively. This significant improvement proves that the binary graph generated by the CChan-Vese model has great advantages in the task of human fall detection.
[0127] In addition, in order to further evaluate the recognition performance of the proposed method, the fused features are sent to the Alex-Net and VGG series networks after transfer learning for training. Fig. 9 The performance indicators of fusion features in various classification networks are clearly shown. As can be seen from the figure, although the recognition rates of fusion features in different networks vary, the overall performance is excellent. Specifically, the recognition rate in the Alex-Net network is about 97.22%, and in the VGG series networks, as the network depth increases, the recognition rate also shows a certain upward trend. Among them, the recognition rate in the VGG13 network is 97.56%, the recognition rate in the VGG16 network is 97.84%, and the recognition rate in the VGG19 network is 97.62%. However, it is most worth mentioning that when the fusion features are sent to the dual attention network for training, the recognition rate reaches the highest 98.89%. Based on the above data, it can be concluded that the use of the CChan-Vese model algorithm to process the time-frequency graph into a binary graph has shown good recognition effects in various classification networks.
[0128] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A millimeter wave radar fall detection method based on an improved Chan_Vese model is characterized by: include: 1) Obtain the radar echo data of the target through the radar equipment and convert it into the time-frequency image of the radar echo; 2) After obtaining the time-frequency image of the radar echo, the improved Chan_Vese model is used to extract the rough contour of the target area. The improved Chan_Vese model introduces the Canny edge detection algorithm to automatically extract the initialization contour; 3) After the rough outline is extracted, a median filter is used for smoothing to obtain a binary image after time-frequency image preprocessing; 4) Classification network: Divide the preprocessed images into training set and test set. Send the training set to the dual attention mechanism network and use the transfer learning method to complete the network training; 5) Fall detection: Send the test data into the trained network to obtain the human body activity type.
2. The millimeter wave radar fall detection method based on the improved Chan-Vese model according to claim 1 is characterized in that: In step 2), the steps of automatically extracting the initialization contour are as follows: A. Use Canny edge detection to detect the edge of the time-frequency image of the radar echo and output the edge detection result; B. Further process the results of Canny edge detection, traverse all edge points, analyze their distribution and continuity, and create a polygon that encloses all edge features of interest based on the distribution of edge points; C. Convert the polygon constructed in the previous step into the initialization contour format required by the Chan-Vese algorithm. The Chan-Vese algorithm will optimize the position and shape of the contour through iterative calculation based on this initialization contour and image information to accurately segment different areas in the image.
3. The millimeter wave radar fall detection method based on the improved Chan-Vese model according to claim 2 is characterized in that: The process of Canny edge detection is: a. Convert the time-frequency color image of radar echo into a grayscale image. The formula used for grayscale conversion is: Gray=0.299R+0.587G+0.114B Among them: Gray represents the pixel value of the grayscale image, R, G, and B represent the values of the red, green, and blue color channels of the corresponding pixels in the original image respectively; b. Perform Gaussian filtering on the grayscale image; c. Calculate the gradient magnitude and direction, and use the template of the Sobel operator to calculate the horizontal difference G x and vertical difference G y , then the formula for the gradient magnitude G and direction is as follows: The gradient angle θ ranges from [-π,π]; d. Non-maximum suppression: After obtaining the magnitude and direction of the gradient, traverse the pixels in the image and remove non-edge points. Non-maximum suppression can refine the edges. e. Double-threshold edge connection. The Canny edge detection algorithm uses a double-threshold mechanism to accurately define edge pixels. For pixels whose gradient values are lower than the set low threshold, the algorithm will suppress them and assign them a value of 0, thereby effectively eliminating non-edge information; and for pixels whose gradient values are between the low threshold and the high threshold, they will be marked as weak edge points for further processing; only when the gradient value of the pixel exceeds the high threshold, it will be clearly classified as a strong edge point.
4. The millimeter wave radar fall detection method based on the improved Chan-Vese model according to claim 3 is characterized in that: After obtaining the preliminary edge detection results through Canny edge detection, the Canny edge detection algorithm will determine whether each weak edge point is connected to a strong edge point. If the virtual edge is connected to the strong edge, then the virtual edge point will be processed as a true edge point; otherwise, it will be suppressed to avoid interference from non-edge information.
5. The millimeter wave radar fall detection method based on the improved Chan-Vese model according to claim 4 is characterized in that: The radar echo data is rearranged and represented as a two-dimensional matrix S(m,n), where m=0,1,2…M-, n=0,1,2…N-1, M refers to the number of sampling points of the radar in one frequency modulation cycle, and N refers to the total number of frequency modulation cycles of the radar. Each column of the two-dimensional matrix S(m,n) is transformed by FFT to obtain the distance distribution information. A Butterworth high-pass filter with an order of 4 and a normalized cutoff frequency of 0.008 is used for clutter processing. The distance distribution information after clutter processing is accumulated over time into a distance-time spectrogram. The two-dimensional matrix of the distance-time spectrogram is summed along the slow time dimension to obtain the amplitude information of the one-dimensional time series signal after normalization, and the time series diagram in the time domain is obtained. The time series diagram in the time domain is converted into a time-frequency diagram in the time-frequency domain for further feature extraction.
6. The millimeter wave radar fall detection method based on the improved Chan-Vese model according to claim 5 is characterized in that: Perform STFT and SPWVD time-frequency analysis on the target's radar echo data to obtain the corresponding time-frequency image.
7. The millimeter wave radar fall detection method based on the improved Chan-Vese model according to claim 6 is characterized in that: The smoothing process of the median filter includes: first, defining a moving window of odd size and aligning the center point of the window with the target pixel in the image; then, reading the grayscale values of all pixels in the window and sorting them from small to large or from large to small; then, finding the grayscale value of the pixel in the middle position from the sorted grayscale sequence; finally, assigning this median grayscale value to the pixel at the center position of the window in the original image. By repeating this process for the entire image, that is, moving the window to cover every pixel of the image, the median filtering of the entire image is completed.
8. The millimeter wave radar fall detection method based on the improved Chan-Vese model according to claim 7 is characterized in that: The model training method of transfer learning is introduced. The dual attention model is pre-trained using the ImageNet dataset. After the pre-training process is completed, the model parameters are saved and all network layers except the fully connected layer and the network input are frozen to retain the learned feature representations. Subsequently, this pre-trained model was transferred to the human fall dataset and used as the initial parameter setting for the human fall detection task. Finally, the trained features were processed by the fully connected layer and the SoftMax classifier was used to output the classification results, which included walking, sitting down, standing up, picking up things, drinking water and falling down.
Citation Information
Cited By
Attitude estimation method for ship in infrared image
CN119762583A
A method for estimating the pose of a ship in an infrared image
CN119762583B
Radar detection method and system based on multi-threshold filtering and space-time cross validation
CN120559604A
