Millimeter wave radar fall detection method and system based on convolutional neural network

Abstract

The invention discloses a millimeter wave radar fall detection method and system based on a convolutional neural network, particularly relates to the technical field of radar signal processing, and is used for solving the problem of fall misjudgment caused by point cloud adhesion in a multi-target coexistence scene. Micro-motion time-frequency features of target point cloud clusters are extracted, micro-motion feature cross-correlation and Doppler phase random walk diffusion coefficients are synchronously calculated for spatial adjacent targets, and independent organism targets are judged according to the conditions that the cross-correlation is lower than a separation threshold value and the difference of the diffusion coefficients exceeds a physiological difference threshold value; analyzing a point cloud space density entropy change rate of each independent target in a falling time window, performing weight reduction processing on the point cloud when the change rate exceeds a critical value, and inputting spatial-temporal characteristics after weight reduction into a convolutional neural network to identify a falling action; and high-precision tumble detection in a strong interference environment is realized.

Description

Technical Field

[0001] This invention relates to the field of radar signal processing technology, and in particular to a fall detection method and system for millimeter-wave radar based on convolutional neural networks. Background Technology

[0002] Millimeter-wave radar technology can be applied to human behavior monitoring, especially for non-contact fall detection in privacy-sensitive areas such as toilets and bathrooms. By transmitting frequency-modulated continuous waves and receiving the reflected signals from the target, three-dimensional point cloud data representing the spatial position of the human body can be generated. Existing technologies use convolutional neural networks to perform posture recognition on the point cloud sequence to determine fall events. This method avoids the risk of privacy leakage from cameras and is not affected by lighting or water mist environments. It is gradually replacing wearable devices in smart elderly care monitoring scenarios.

[0003] However, when there are multiple dynamic targets in the monitoring area (such as in rescue scenarios where caregivers intervene), existing methods face fundamental limitations: the micro-motion signals of different targets (such as the sudden movement of a fall victim and the breathing fluctuations of a rescuer) are coupled in the time and frequency domains, causing the spatial features of the point cloud to stick together. Due to the physical resolution limitations, millimeter-wave radar cannot effectively separate the point clouds of multiple targets coexisting at close range at the sub-meter level. When convolutional neural networks extract features from the sticky point clouds, they have difficulty distinguishing between the fall action and the interference micro-motions, resulting in misjudgment of behavior. This significantly reduces the reliability of the system in real-world scenarios that require multi-target collaborative monitoring. Summary of the Invention

[0004] This invention addresses the technical problems existing in the prior art by providing a fall detection method and system based on convolutional neural networks using millimeter-wave radar.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: This invention provides the following technical solution: Fall detection methods based on convolutional neural networks for millimeter-wave radar include: S1. Acquire the raw echo signal from the millimeter-wave radar and generate a three-dimensional point cloud sequence through frequency-modulated continuous wave demodulation. S2. Perform multi-target clustering segmentation on the 3D point cloud sequence to separate independent target point cloud clusters, and extract the micro-motion time-frequency features of each target point cloud cluster; S3. For adjacent target point cloud clusters with a spatial distance less than a set threshold, simultaneously calculate the cross-correlation of their micro-motion time-frequency characteristics and the random walk diffusion coefficient of the Doppler phase signal. S4. If the cross-correlation is lower than the separation threshold and the difference in random walk diffusion coefficient exceeds the physiological difference threshold, then they are determined to be targets from different organisms. S5. By analyzing the point cloud spatial density distribution of each independent biological target within the fall determination time window, the point cloud spatial density entropy change rate is generated. S6. When the rate of change of the spatial density entropy of the point cloud exceeds the critical value, the point cloud data of the current independent biological target is downweighted, and the spatiotemporal features of the downweighted point cloud are input into the convolutional neural network for fall action recognition.

[0006] Furthermore, the raw echo signal from the millimeter-wave radar is acquired, and a three-dimensional point cloud sequence is generated through frequency-modulated continuous wave demodulation, including: The raw echo signal from the millimeter-wave radar is mixed to generate an intermediate frequency signal; Range spectrum is generated by performing a range-dimensional fast Fourier transform on the intermediate frequency signal; Target range cells are extracted from the range spectrum using constant false alarm rate detection. The Doppler-dimensional fast Fourier transform is performed on the signal within the target range cell to generate the range-Doppler matrix; Target scattering points are extracted based on the amplitude peak of the range-Doppler matrix; The distance, Doppler velocity, and azimuth measurements of the target scattering point are converted into three-dimensional spatial coordinates to form a three-dimensional point cloud sequence.

[0007] Furthermore, multi-target clustering is performed on the 3D point cloud sequence to separate independent target point cloud clusters, and the micro-motion time-frequency features of each target point cloud cluster are extracted, including: Calculate the Euclidean distance between points based on the spatial location of points in a 3D point cloud sequence; Density clustering of the 3D point cloud sequence is performed based on the Euclidean distance between points to generate an initial set of point cloud clusters. Independent target point cloud clusters are formed by associating the initial point cloud cluster set with the target centroid displacement between consecutive frames; The timing signal of each independent target point cloud cluster is bandpass filtered to separate the micro-motion signal; The micro-motion signal is transformed by time and frequency to generate micro-motion time and frequency characteristics that include respiratory fluctuation frequency components and limb acceleration frequency band components.

[0008] Furthermore, for adjacent target point cloud clusters with a spatial distance less than a set threshold, the cross-correlation of their micro-motion time-frequency characteristics and the random walk diffusion coefficient of their Doppler phase signals are calculated simultaneously, including: Calculate the Euclidean distance between the centroids of adjacent target point cloud clusters as the spatial distance, and filter out adjacent target point cloud clusters whose spatial distance is less than a set threshold. Obtain the in-phase and quadrature components in the original echo signals corresponding to the filtered adjacent target point cloud clusters; Calculate the Doppler phase signal of each adjacent target point cloud cluster based on in-phase and quadrature components; The Doppler phase signal is time-differenced to generate a phase difference sequence; The variance of the phase difference sequence is calculated as the random walk diffusion coefficient; Extract the time-domain waveforms of the respiratory fluctuation frequency component and the limb acceleration frequency band component from the micro-motion time-frequency features of adjacent target point cloud clusters; The cross-correlation coefficient between the time-domain waveform of the respiratory fluctuation frequency component and the time-domain waveform of the limb acceleration frequency band component is calculated as the cross-correlation coefficient.

[0009] Furthermore, the cross-correlation coefficient between the time-domain waveforms of the respiratory fluctuation frequency component and the time-domain waveforms of the limb acceleration frequency band component is calculated as the cross-correlation coefficient through the following steps: Extract discrete sampling point sequences of the time-domain waveforms of respiratory fluctuation frequency components and limb acceleration frequency band components; Calculate the covariance between the discrete sampling point sequence of the time-domain waveform of the respiratory fluctuation frequency component and the discrete sampling point sequence of the time-domain waveform of the limb acceleration frequency band component; Calculate the standard deviation of the discrete sampling point sequence of the time-domain waveform of the respiratory fluctuation frequency component and the standard deviation of the discrete sampling point sequence of the time-domain waveform of the limb acceleration frequency band component; Divide the covariance by the product of the standard deviation of the discrete sampling point sequence of the time-domain waveform of the respiratory fluctuation frequency component and the standard deviation of the discrete sampling point sequence of the time-domain waveform of the limb acceleration frequency band component, and generate the cross-correlation coefficient as the cross-correlation.

[0010] Furthermore, if the cross-correlation is below the separation threshold and the difference in random walk diffusion coefficients exceeds the physiological difference threshold, then they are determined to be targets from different organisms, including: Calculate the difference between the cross-correlation of adjacent target point cloud clusters and the separation threshold; Calculate the absolute difference in the random walk diffusion coefficients of adjacent target point cloud clusters; When the difference between the cross-correlation and the separation threshold is less than zero and the absolute difference of the random walk diffusion coefficient is greater than the physiological difference threshold, the determination results of different biological targets are generated. The determination result is associated with the corresponding adjacent target point cloud cluster.

[0011] Furthermore, by analyzing the point cloud spatial density distribution of each independent biological target within the fall determination time window, the rate of change of point cloud spatial density entropy is generated, including: Obtain the fall detection time window and divide the fall detection time window into multiple consecutive time sub-intervals; For point cloud data of independent biological targets within each time sub-interval, calculate the point cloud density distribution histogram in three-dimensional space; The proportion of the number of points in each spatial unit to the total number of points in the point cloud is calculated based on the point cloud density distribution histogram. The point cloud spatial density entropy value of the current time sub-interval is calculated using the Shannon entropy formula based on the ratio value. Calculate the difference between the spatial density entropy values ​​of the point cloud in adjacent time intervals; The rate of change of point cloud spatial density entropy is generated by dividing the average of the difference values ​​by the length of the time sub-interval.

[0012] Furthermore, when the rate of change of the spatial density entropy of the point cloud exceeds a critical value, the point cloud data of the current independent biological target is downweighted, and the spatiotemporal features of the downweighted point cloud are input into a convolutional neural network for fall action recognition, including: Compare the relationship between the rate of change of point cloud spatial density entropy and the critical value; When the rate of change of the spatial density entropy of the point cloud exceeds the critical value, the number of points in each spatial unit of the point cloud data of the current independent biological target is multiplied by the weighting factor. Generate spatiotemporal features of point clouds containing three-dimensional spatial coordinates and timestamps based on the reduced-weight point cloud data; The spatiotemporal features of point clouds are input into a convolutional neural network for fall recognition.

[0013] Furthermore, the point cloud spatiotemporal features are input into a convolutional neural network for fall recognition, which is achieved through the following steps: The three-dimensional spatial coordinates in the spatiotemporal features of point clouds are mapped to a two-dimensional planar mesh to generate a spatial distribution matrix; The spatial distribution matrix is ​​superimposed with the timestamp sequence to form a three-dimensional spatiotemporal tensor; Local motion pattern features of spatiotemporal tensors are extracted using the convolutional layers of a convolutional neural network. Local motion pattern features are downsampled using pooling layers of a convolutional neural network; The fully connected layer of a convolutional neural network outputs the probability of a fall action classification.

[0014] On the other hand, the present invention provides a millimeter-wave radar fall detection system based on a convolutional neural network, comprising: The point cloud generation module is used to acquire the raw echo signal of the millimeter-wave radar and generate a three-dimensional point cloud sequence through frequency-modulated continuous wave demodulation. The feature extraction module is used to perform multi-target clustering segmentation on the 3D point cloud sequence to separate independent target point cloud clusters and extract the micro-motion time-frequency features of each target point cloud cluster. The time-frequency analysis module is used to simultaneously calculate the cross-correlation of the micro-motion time-frequency characteristics and the random walk diffusion coefficient of the Doppler phase signal for adjacent target point cloud clusters with a spatial distance of less than a set threshold. The target discrimination module is used to determine different biological targets if the cross-correlation is lower than the separation threshold and the difference in random walk diffusion coefficient exceeds the physiological difference threshold. The entropy analysis module is used to generate the rate of change of point cloud spatial density entropy by analyzing the point cloud spatial density distribution of each independent biological target within the fall determination time window. The fall detection module is used to reduce the weight of the point cloud data of the current independent biological target when the rate of change of the spatial density entropy value of the point cloud exceeds the critical value, and then input the spatiotemporal features of the reduced point cloud into the convolutional neural network for fall action recognition.

[0015] The beneficial effects of this invention are: 1. By decoupling the physiological characteristics of multiple targets, the reliability of fall detection in scenarios with multiple people coexisting is significantly improved. Addressing the point cloud adhesion problem caused by insufficient physical resolution of millimeter-wave radar, the intrinsic differences in micro-motion signals of biological organisms are utilized: by simultaneously calculating the cross-correlation of the time-frequency characteristics of micro-motions of adjacent target point cloud clusters and the Doppler phase random walk diffusion coefficient, the independent respiratory rhythms and limb tremor specificities of different organisms are accurately captured. When the cross-correlation is below the separation threshold and the difference in diffusion coefficient exceeds the physiological difference threshold, it effectively distinguishes independent targets coexisting at close range at sub-meter level, overcoming the bottleneck of misjudgment caused by signal coupling in traditional methods, and fundamentally improving the system's anti-interference capability in multi-target collaborative monitoring scenarios.

[0016] 2. Enhance the spatiotemporal feature representation of fall actions through point cloud spatial density entropy change analysis. Based on the point cloud distribution characteristics of independent biological targets within the fall determination time window, the rate of posture disorder in the human body instability process is quantified by the rate of change of spatial density entropy value. This overcomes the learning bias of traditional convolutional neural networks on the features of adherent point clouds. When the rate of change of entropy value exceeds the critical value, point cloud weight reduction processing is implemented to suppress the influence of sudden interference movements on feature extraction. This allows the convolutional neural network to focus on recognizing the unique instability trajectory of falls. The spatiotemporal dynamic features of radar point clouds are deeply integrated with deep learning models, maintaining fall recognition accuracy even in strong interference environments, and providing highly reliable non-contact monitoring for privacy-sensitive areas. Attached Figure Description

[0017] Figure 1 This is a flowchart of the millimeter-wave radar fall detection method based on convolutional neural networks according to the present invention; Figure 2 This is a schematic diagram of the millimeter-wave radar fall detection system based on a convolutional neural network according to the present invention. Detailed Implementation

[0018] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0019] Example 1: Figure 1 This invention presents a millimeter-wave radar fall detection method based on convolutional neural networks, comprising: S1. Acquire the raw echo signal from the millimeter-wave radar and generate a three-dimensional point cloud sequence through frequency-modulated continuous wave demodulation. S2. Perform multi-target clustering segmentation on the 3D point cloud sequence to separate independent target point cloud clusters, and extract the micro-motion time-frequency features of each target point cloud cluster; S3. For adjacent target point cloud clusters with a spatial distance less than a set threshold, simultaneously calculate the cross-correlation of their micro-motion time-frequency characteristics and the random walk diffusion coefficient of the Doppler phase signal. S4. If the cross-correlation is lower than the separation threshold and the difference in random walk diffusion coefficient exceeds the physiological difference threshold, then they are determined to be targets from different organisms. S5. By analyzing the point cloud spatial density distribution of each independent biological target within the fall determination time window, the point cloud spatial density entropy change rate is generated. S6. When the rate of change of the spatial density entropy of the point cloud exceeds the critical value, the point cloud data of the current independent biological target is downweighted, and the spatiotemporal features of the downweighted point cloud are input into the convolutional neural network for fall action recognition.

[0020] After acquiring the raw echo signal from the millimeter-wave radar, the specific implementation process for generating a three-dimensional point cloud sequence through frequency-modulated continuous wave demodulation is as follows: The original echo signal from the millimeter-wave radar is mixed to generate an intermediate frequency (IF) signal. Mixing involves multiplying the received original echo signal with the transmitted frequency-modulated continuous wave signal, and then filtering out high-frequency components using a low-pass filter to obtain the difference frequency signal as the IF signal. The frequency of this IF signal is linearly related to the target distance. The frequency range of the local oscillator signal used in the mixing process is set according to the radar's operating frequency band. For example, the center frequency is selected within the 76 GHz to 81 GHz band. The cutoff frequency of the low-pass filter is determined based on the difference frequency value corresponding to the radar's maximum detection range. For example, for a radar system with a detection range of 200 meters, the cutoff frequency of the low-pass filter is set to 1 MHz.

[0021] The intermediate frequency signal is subjected to a range-dimensional fast Fourier transform to generate a range spectrum. The number of sampling points for the range-dimensional fast Fourier transform is set according to the maximum detection range of the radar. For example, a 2048-point Fourier transform corresponds to a range of 200 meters. The specific formula for calculating the number of sampling points is that the number of sampling points equals the maximum detection range divided by the range resolution. The range resolution is determined by the signal bandwidth. When the signal bandwidth is 4 GHz, the range resolution is approximately 3.75 cm. The horizontal axis of the range spectrum represents the range cell. The actual distance value corresponding to each range cell is the range cell number multiplied by the range resolution. The peak position of the range spectrum amplitude corresponds to the target distance value.

[0022] Target range cells are extracted from the range spectrum using constant false alarm rate (CFAR) detection. CFAR detection employs a cell average selection method, which specifically includes: selecting 10 range cells before and after the range cell to be detected as a reference window; excluding protected cells, calculating the average power value of all range cells within the reference window; multiplying the average power value by a calibration coefficient to obtain the detection threshold. The calibration coefficient is obtained by looking up a table based on the preset false alarm probability. For example, when the preset false alarm probability is 0.000001, the calibration coefficient is set to 2.5. When the power value of the range cell to be detected exceeds the detection threshold, it is determined to be a target range cell. The number of protected cells is set according to the radar range resolution, for example, 2 range cells before and after the radar are used as protected cells.

[0023] The range-Doppler matrix is ​​generated by performing a Doppler-Fourier Transform (DFFT) on the signal within the target range cell. The DFFT is performed on 128 consecutive radar cycles. The number of transformation points determines the velocity resolution; a 256-point transformation corresponds to a velocity resolution of 0.2 meters per second. The specific velocity resolution is calculated by dividing the speed of light by twice the radar center frequency and multiplying by the coherent accumulation time. The coherent accumulation time is equal to the radar cycle multiplied by the number of transformation points. The rows of the range-Doppler matrix represent range cells, the columns represent Doppler velocity cells, and the matrix element values ​​are signal amplitude values.

[0024] The target scattering point is extracted based on the amplitude peak of the range-Doppler matrix. The amplitude peak extraction is achieved by searching for local maxima. Specifically, a 3×3 sliding window is set in the range-Doppler matrix. When the amplitude value of the central cell of the window is greater than that of the eight surrounding adjacent cells and exceeds the noise floor by 10 dB, it is determined to be a valid target scattering point. The noise floor is determined by the statistical average of the targetless region. The targetless region is selected as the 10% region with the smallest amplitude value in the range-Doppler matrix. The noise floor is calculated as the arithmetic mean of the amplitude values ​​of all cells in this region.

[0025] The range, Doppler velocity, and azimuth measurements of the target scattering points are converted into three-dimensional spatial coordinates to form a three-dimensional point cloud sequence. The specific conversion method is as follows: the range value is calculated using the peak position of the range-dimensional Fast Fourier Transform (FFT), with the formula being the range cell number multiplied by the range resolution; the Doppler velocity value is calculated using the peak position of the Doppler cell number multiplied by the velocity resolution; the azimuth measurement is obtained by calculating the phase difference of the signal received by the array antenna, specifically using beamforming. The three-dimensional spatial coordinate conversion formulas are: the x-axis coordinate equals the range value multiplied by the azimuth cosine, the y-axis coordinate equals the range value multiplied by the azimuth sine, and the z-axis coordinate is obtained by multiplying the Doppler velocity value by the time integral. The time integral is performed using the trapezoidal integration method within each radar cycle. Each radar scan cycle generates one frame of a three-dimensional point cloud sequence containing all target scattering points. The time interval between the three-dimensional point cloud sequences is equal to the radar scan cycle, for example, 30 milliseconds.

[0026] The specific implementation process of performing multi-target clustering segmentation on a 3D point cloud sequence to separate independent target point cloud clusters and extract micro-motion time-frequency features is as follows: The Euclidean distance between points is calculated based on the spatial location of points in a 3D point cloud sequence. The 3D point cloud sequence is generated by radar scanning cycles. Each scanning cycle contains the 3D spatial coordinates of multiple target scattering points. The Euclidean distance between points is calculated using a distance formula in a 3D Cartesian coordinate system, specifically the square root of the sum of the square differences of the coordinate values ​​of two points. The input 3D spatial coordinate values ​​include x-axis, y-axis, and z-axis coordinate values. The calculation range of the Euclidean distance between points is limited to the point cloud data within the same radar scanning cycle. Point cloud data from different scanning cycles are not used for distance calculation. The coordinate values ​​in the Euclidean distance calculation formula are in meters, and the calculation result is also in meters.

[0027] The initial point cloud cluster set is generated by density clustering of the 3D point cloud sequence based on the Euclidean distance between points. The density clustering adopts a clustering algorithm based on the neighborhood radius and the minimum number of points threshold. The neighborhood radius is set according to the radar range resolution, specifically 4 times the radar range resolution. For example, when the radar range resolution is 3.75 cm, the neighborhood radius is set to 15 cm. The minimum number of points threshold is set according to the minimum size of the target, specifically the minimum number of scattering points required to cover the minimum size of the target. For example, the minimum number of points in the point cloud for a human target is set to 5 points. The density clustering process includes: traversing each point in the 3D point cloud sequence, searching for neighboring points within the neighborhood radius with that point as the center, and forming an initial point cloud cluster when the number of neighboring points reaches the minimum number of points threshold. When determining the ownership of boundary points, the principle of prioritizing core points is adopted. When the number of points in the neighborhood of a point is less than the minimum number of points threshold and does not belong to any existing cluster, it is deleted as a noise point. The final output is an initial point cloud cluster set containing multiple initial point cloud clusters.

[0028] Independent target point cloud clusters are formed by associating the target centroid displacement of the initial point cloud clusters with consecutive inter-frame target centroid displacements. A consecutive frame refers to multiple radar scan cycles that are consecutive in time. The target centroid displacement calculation includes: calculating the geometric center of each initial point cloud cluster as the target centroid coordinates. The target centroid coordinates are obtained by averaging the coordinates of all points in the point cloud cluster. The specific calculation formula is that the average x-axis coordinate is equal to the sum of the x-axis coordinates of all points in the point cloud cluster divided by the number of points in the point cloud cluster. The average y-axis coordinate and the average z-axis coordinate are calculated using the same method. The target centroid displacement between consecutive frames is calculated by the difference in the target centroid coordinates between adjacent scan cycles. The displacement association threshold is set according to the target's maximum motion speed. The specific calculation formula is that the displacement association threshold is equal to the target's maximum motion speed multiplied by the radar scan cycle. For example, when the maximum motion speed of a human target is 5 meters per second and the radar scan cycle is 30 milliseconds, the displacement association threshold is set to 0.15 meters. When the target centroid displacement between consecutive frames is less than the displacement association threshold, it is determined to be the same target point cloud cluster, thus forming independent target point cloud clusters with temporal continuity.

[0029] For each independent target point cloud cluster, the time-series signal is bandpass filtered to separate the micro-motion signal. The time-series signal refers to the change in motion characteristics of the independent target point cloud cluster in a continuous scanning cycle. The motion characteristics include the height change of the target centroid in the z-axis direction. The bandpass filtering is implemented using a second-order Butterworth filter. The filter design parameters include the lower limit and upper limit of the passband frequency. The lower limit of the passband frequency is set to 0.1 Hz based on the lowest frequency of respiratory motion, and the upper limit of the passband frequency is set to 10 Hz based on the highest frequency of limb motion. The filter is implemented by converting the analog filter to a digital filter using the bilinear transform method. The filtering process is performed on the time-series signal for 30 consecutive scanning cycles, and the separated micro-motion signal is output. The micro-motion signal contains motion components in the frequency range of 0.1 Hz to 10 Hz.

[0030] The micro-motion signal is subjected to time-frequency transformation to generate micro-motion time-frequency features containing respiratory fluctuation frequency components and limb acceleration frequency band components. The time-frequency transformation is implemented using the short-time Fourier transform method, which specifically includes: dividing the micro-motion signal into data segments of fixed length, the length of which is set according to the frequency resolution requirements, such as 128 sampling points; adjacent data segments overlap by 50% of the sampling points, such as 64 sampling points; applying a Hanning window function to each data segment to suppress spectral leakage, with the window function length equal to the data segment length; performing a fast Fourier transform on the windowed data segments, with the number of transform points set according to the frequency accuracy requirements, such as 256 points; the transform result forms a time-frequency spectrum; the energy distribution of the respiratory fluctuation frequency component in the time-frequency spectrum is extracted from the 0.1 Hz to 0.3 Hz frequency band, and the energy distribution of the limb acceleration frequency band in the 1 Hz to 10 Hz frequency band is extracted from the time-frequency spectrum; the energy distribution is obtained by calculating the sum of squares of the spectral amplitudes within the corresponding frequency bands; finally, micro-motion time-frequency features containing time, frequency, and energy dimensions are generated.

[0031] The specific implementation process for simultaneously calculating the cross-correlation of the micro-motion time-frequency characteristics and the random walk diffusion coefficient of the Doppler phase signal for adjacent target point cloud clusters with a spatial distance of less than a set threshold is as follows: The Euclidean distance between the centroids of adjacent target point cloud clusters is calculated as the spatial distance. The centroid coordinates of the target point cloud cluster are obtained by averaging the coordinates of all points within the cluster. The specific calculation formula is that the centroid x-axis coordinate value equals the sum of the x-axis coordinate values ​​of all points within the cluster divided by the number of points within the cluster. The y-axis and z-axis coordinate values ​​are calculated using the same method. The Euclidean distance is calculated by taking the square root of the sum of the square differences between the two centroid coordinates. The threshold is set based on the minimum distance between human targets, specifically the statistical value of the average shoulder width of an adult human body, for example, 50 centimeters. Adjacent target point cloud clusters with a spatial distance less than the set threshold are filtered out. The filtering operation is performed within the same radar scan cycle. When the number of target point cloud clusters exceeds two, distance calculation and filtering are performed on all adjacent point cloud cluster pairs.

[0032] The in-phase and quadrature components of the original echo signals corresponding to the filtered adjacent target point cloud clusters are obtained. The original echo signals come from the baseband data of the millimeter-wave radar receiving channel. The in-phase and quadrature components are obtained by demodulating the intermediate frequency signal by the radar receiver. The demodulation process uses a quadrature mixer to decompose the intermediate frequency signal into two signals with a 90-degree phase difference. The in-phase and quadrature components are stored in the radar signal processing unit in the form of digital signals. The original echo signal corresponding to each target point cloud cluster is extracted by locating the radar range cell and Doppler cell index where the target scattering point is located. The range cell and Doppler cell index come from the range-Doppler matrix generated in step S1.

[0033] The Doppler phase signals of adjacent target point cloud clusters are calculated based on the in-phase and quadrature components. The Doppler phase signals are obtained by the arctangent operation of the in-phase and quadrature components. The specific calculation formula is that the phase value is equal to the arctangent function value of the quadrature component value divided by the in-phase component value. The phase signal ranges from negative π radians to positive π radians. When the in-phase component value is zero, the phase value is set to positive π / 2 radians or negative π / 2 radians according to the sign of the quadrature component. The phase unwrapping process uses a continuous detection method to eliminate phase jumps. Specifically, when the phase difference between adjacent scanning cycles exceeds π radians, the current phase value is increased or decreased by an integer multiple of 2π radians to make the phase continuous. The Doppler phase signal of each target point cloud cluster is represented by the average phase of all scattering points within that cloud cluster.

[0034] The Doppler phase signal is time-differenced to generate a phase difference sequence. The time difference processing is performed on a continuous time series. Specifically, the phase value of the current scan cycle is subtracted from the phase value of the previous scan cycle. The length of the phase difference sequence is one scan cycle shorter than the original phase sequence. The unit of the phase difference value is radians. When the phase difference value exceeds π radians, phase wrap-around correction is performed. The correction method is to subtract 2π radians if the phase difference value is greater than π radians, and to add 2π radians if it is less than negative π radians. The corrected phase difference value is kept within the range of negative π radians to positive π radians.

[0035] The variance of the phase difference sequence is calculated as the random walk diffusion coefficient. The variance is calculated as the average of the sum of squared deviations of each element of the phase difference sequence from the arithmetic mean of the sequence. The specific calculation steps include: first, calculating the arithmetic mean of the phase difference sequence, which is equal to the sum of all elements of the sequence divided by the sequence length; then, calculating the deviation of each phase difference value from the arithmetic mean; summing the squared deviations; finally, dividing by the length of the phase difference sequence minus 1 to obtain the unbiased estimated variance. The unit of the variance is squared radians. When the length of the phase difference sequence is less than 2, the variance is set to zero.

[0036] The time-domain waveforms of the respiratory fluctuation frequency component and the limb acceleration frequency band component in the micro-motion time-frequency features of adjacent target point cloud clusters are extracted. The micro-motion time-frequency features are derived from the output of step S2. The time-domain waveform of the respiratory fluctuation frequency component corresponds to the time-domain signal reconstruction of the 0.1 Hz to 0.3 Hz frequency band, and the time-domain waveform of the limb acceleration frequency band component corresponds to the time-domain signal reconstruction of the 1 Hz to 10 Hz frequency band. The time-domain waveform reconstruction is achieved by inverse Fourier transform of a bandpass filter. Specifically, inverse Fourier transform is performed on the spectral components of the corresponding frequency band in the micro-motion time-frequency features. The sampling rate of the reconstructed time-domain waveform is consistent with the radar scanning period, for example, a 30-millisecond sampling interval. The length of the time-domain waveform is the same as the length of the time-frequency analysis window in step S2.

[0037] The cross-correlation coefficient between the time-domain waveforms of the respiratory fluctuation frequency component and the limb acceleration frequency band component is calculated as the cross-correlation coefficient. This is achieved through the following steps: Discrete sampling point sequences of the respiratory fluctuation frequency component and the limb acceleration frequency band component are extracted. These sequences are of equal length and time-aligned, with time alignment ensured by radar scan cycle timestamps. The covariance between the two sequences is calculated. The covariance is calculated as the average of the products of corresponding sampling points in the two sequences minus the product of the average values ​​of the two sequences. The average value of each sequence is equal to the sum of all elements in the sequence. Divide by the sequence length; calculate the standard deviation of the discrete sampling point sequence of the respiratory fluctuation frequency component time-domain waveform and the standard deviation of the discrete sampling point sequence of the limb acceleration frequency band component time-domain waveform. The standard deviation is calculated by taking the square root of the average of the sum of squares of the deviations of each sequence sampling point from its arithmetic mean, and dividing the sum of squares of deviations by the sequence length minus 1 and taking the square root; divide the covariance by the product of the standard deviation of the discrete sampling point sequence of the respiratory fluctuation frequency component time-domain waveform and the standard deviation of the discrete sampling point sequence of the limb acceleration frequency band component time-domain waveform to generate a cross-correlation coefficient as the cross-correlation coefficient. The cross-correlation coefficient ranges from -1 to +1. When the standard deviation is zero, the cross-correlation coefficient is set to zero.

[0038] If the cross-correlation is below the separation threshold and the difference in random walk diffusion coefficients exceeds the physiological difference threshold, then the specific implementation process for determining different biological targets is as follows: The difference between the cross-correlation of adjacent target point clusters and the separation threshold is calculated. The cross-correlation comes from the cross-correlation coefficient calculated in step S3, and the cross-correlation coefficient ranges from -1 to +1. The separation threshold is set according to the statistical law of the independence of human physiological motion characteristics. The specific setting method includes collecting multiple sets of motion data samples of different human targets in the same environment. Each set of data samples contains motion characteristics of at least 100 consecutive scanning cycles. For each data sample, the cross-correlation coefficient of the time-domain waveform of the respiratory fluctuation frequency component and the time-domain waveform of the limb acceleration frequency band component of the same target is calculated. The distribution range of the cross-correlation values ​​of all data samples is statistically analyzed, and 90% of the lower limit of the distribution range is taken as the separation threshold. For example, when the minimum cross-correlation coefficient is found to be 0.3, the separation threshold is set to 0.27. The difference is calculated by subtracting the value of the separation threshold from the value of the cross-correlation. The calculation result is retained to four decimal places.

[0039] Calculate the absolute difference of the random walk diffusion coefficients of adjacent target point cloud clusters. The random walk diffusion coefficients are derived from the variance calculated in step S3, and the unit is radian squares. The absolute difference is calculated as the absolute value of the difference between the random walk diffusion coefficients of the two target point cloud clusters. The calculation process retains four decimal places of precision. If the random walk diffusion coefficient of either of the two target point cloud clusters is missing, the difference calculation of that point cloud cluster pair is skipped. Missing conditions include insufficient phase difference sequence length or the invalid data flag being set.

[0040] When the difference between the cross-correlation and the separation threshold is less than zero and the absolute difference of the random walk diffusion coefficient is greater than the physiological difference threshold, a judgment result for different biological targets is generated. The physiological difference threshold is set according to the individual difference range of human physiological tremor characteristics. The specific setting method includes measuring the variance of physiological tremor in different parts of the same body in a static state, collecting measurement data from at least 20 different individuals, with each set of data containing measurement values ​​from at least 5 body parts, calculating the maximum fluctuation range of the variance values ​​of multiple parts for each individual, and taking 120% of the upper limit of the fluctuation range of all individuals as the physiological difference threshold. For example, when the upper limit of the maximum fluctuation range is 0.5 radians squared, the physiological difference threshold is set to 0.6 radians squared. The judgment condition simultaneously satisfies two inequalities: the result of cross-correlation minus the separation threshold is less than zero, and the absolute difference of the random walk diffusion coefficient is greater than the physiological difference threshold. The judgment result is a Boolean logic value of true or false. When the two conditions are not met, the judgment result is false.

[0041] The determination result is associated with the corresponding adjacent target point cloud clusters. The association operation includes: creating a unique identifier for each pair of adjacent target point cloud clusters. The identifier is generated by connecting the numbers of the two point cloud clusters in ascending order of numerical value with an underscore. For example, the identifier for point cloud clusters numbered 3 and 5 is 3_5. The determination result is bound to the identifier and stored in a dynamic data structure. The data structure includes an identifier field, a boolean value field for the determination result, and a timestamp field. The timestamp is synchronized with the start time of the radar scan cycle and is in millisecond-level Unix timestamp format. The determination result is also written to the attribute field of the point cloud cluster. The attribute field is named "Biosome Independence Identifier" and is of integer data type. When the determination result is true, the attribute value is set to 1, indicating different biological targets. When the determination result is false, the attribute value is set to 0, indicating that they may be the same biological target. The attribute update operation is completed within 3 scan cycles after the determination result is generated.

[0042] The specific implementation process for generating the rate of change of point cloud spatial density entropy by analyzing the point cloud spatial density spatial density change of each independent biological target within the fall determination time window is as follows: A fall detection time window is obtained and divided into multiple consecutive time sub-intervals. The fall detection time window is set according to the typical duration of a human fall process, specifically the complete time span from the start to the end of the fall. The time window length is determined based on human kinematics research data, which shows that the average duration of a human fall process is 1.2 to 1.8 seconds. Therefore, the time window length is set to 1.5 seconds. The start time of the time window is triggered by a target acceleration mutation event, which detects an acceleration value exceeding 5 m / s² in the limb acceleration frequency band component of step S2. When dividing the time into sub-intervals, the length of each sub-interval is determined based on the requirement of continuous human movement. The length of each time sub-interval is fixed at 0.2 seconds, which is less than the minimum response time of human posture change of 0.25 seconds. When the total duration of the time window of 1.5 seconds is not divisible by 0.2 seconds, the length of the last sub-interval is adaptively adjusted. The number of time sub-intervals is equal to the total duration of the time window divided by the length of the sub-interval and then rounded up. For example, 1.5 / 0.2=7.5, which is rounded up to get 8 time sub-intervals. The length of the first 7 sub-intervals is 0.2 seconds, and the length of the last sub-interval is 0.1 seconds.

[0043] For point cloud data of independent biological targets within each time sub-interval, a histogram of point cloud density distribution in three-dimensional space is calculated. The three-dimensional space adopts a rectangular coordinate system, and the spatial range is dynamically set according to the target point cloud distribution. Specifically, it is a cube space centered on the target's centroid. The target's centroid coordinates are obtained from the calculation results of step S2. The side length of the cube is twice the statistical value of the target's height. The target's height is recorded based on the height value when standing upright in historical point cloud data. For example, if the detected target's upright height is 1.8 meters, the side length of the cube is set to 3.6 meters. The spatial unit size is set according to the point cloud density resolution. The spatial unit size is set to a 0.2-meter cube, which is smaller than the thickness of the human torso by 0.25 meters. When calculating the histogram, all units in the space are traversed. The point cloud count of each unit includes all point cloud data within the cube of that unit. The mapping relationship between point cloud coordinates and spatial units is achieved by dividing the coordinate value by the unit size and then rounding down. Specifically, the x-axis coordinate of the unit index is equal to the x-axis coordinate value of the point cloud divided by 0.2 meters and then rounded down. The y-axis and z-axis coordinates are calculated using the same method.

[0044] The proportion of the number of points in each spatial unit to the total number of points is calculated based on the point cloud density distribution histogram. The specific calculation method is as follows: the total number of target point clouds in the current time sub-interval is counted. The total number of points is equal to the sum of the counts of all spatial units in the histogram. The proportion of each spatial unit is equal to the number of points in that unit divided by the total number of points. When the total number of points is zero, the proportion is set to zero. The result of the proportion calculation is retained to six decimal places. After the proportion calculation is completed, normalization verification is performed. The sum of the proportions of all spatial units should be 1, and the allowable error range is ±0.000001.

[0045] The spatial density entropy value of the point cloud in the current time sub-interval is calculated using the Shannon entropy formula based on the proportional values. The Shannon entropy formula is implemented as follows: iterate through the proportional values ​​of all spatial units, calculate the natural logarithm of each non-zero proportional value, multiply it by the product of the proportional values, and use a mathematical function library to calculate the natural logarithm. Sum all the product results and take the negative value. The entropy value is calculated using the negative summation result. When the proportional values ​​of all spatial units are zero, the entropy value is set to zero. The entropy value calculation retains four decimal places of precision. The total number of spatial units is determined by the ratio of the cube's spatial dimensions to the unit dimensions. For example, when a 3.6-meter side length space is divided into 0.2-meter unit dimensions, the number of units in the x-axis direction is 3.6 / 0.2 = 18, and the same applies to the y-axis and z-axis directions. The total number of spatial units is 18 × 18 × 18 = 5832.

[0046] The difference between the point cloud spatial density entropy values ​​of adjacent time sub-intervals is calculated. The difference calculation is performed sequentially on the time series, specifically by subtracting the entropy value of the previous time sub-interval from the entropy value of the next time sub-interval. The length of the difference sequence is one less than the number of time sub-intervals. When the length of the entropy value sequence is less than two, the difference value is set to zero. The unit of the difference value is the same as that of the entropy value, which is a dimensionless quantity. The calculation result is retained to three decimal places with precision. After the difference value calculation is completed, a range check is performed. The entropy difference range for normal human movement is between -0.5 and +0.5.

[0047] The rate of change of point cloud spatial density entropy is generated by dividing the average of the difference values ​​by the length of the time sub-interval. The average value is calculated as the algebraic sum of all difference values ​​divided by the length of the difference sequence. When the length of the difference sequence is zero, the average value is set to zero. The length of the time sub-interval is a fixed value of 0.2 seconds. The rate of change of point cloud spatial density entropy is equal to the average value divided by 0.2 seconds. When the length of the difference sequence is zero, the rate of change is set to zero. The unit of the rate of change is per second. The calculation result is retained to two decimal places. The final output is used as a quantitative index to characterize the rate of disorder in point cloud distribution. This index is used to detect the degree of instability of human posture during a fall.

[0048] When the rate of change of the spatial density entropy of the point cloud exceeds a critical value, the point cloud data of the current independent biological target is downweighted, and the spatiotemporal features of the downweighted point cloud are input into a convolutional neural network for fall action recognition. The specific implementation process is as follows: The relationship between the rate of change of point cloud spatial density entropy and the critical value is compared. The rate of change of point cloud spatial density entropy is derived from the calculation result of step S5, and the unit is per second. The critical value is set according to the typical entropy change rate of posture instability during a human fall. The specific setting method includes collecting point cloud data of multiple sets of human fall action samples and normal activity samples. Each set of samples contains at least 50 fall events and 100 normal activity events. The rate of change of point cloud spatial density entropy is calculated for each event. The distribution range of the entropy change rate of fall events and the distribution range of the entropy change rate of normal activities are statistically analyzed. The statistical value of the point where the two distribution ranges separate is taken as the critical value. For example, when the minimum entropy change rate of fall events is 4.2 per second and the maximum entropy change rate of normal activities is 4.0 per second, the critical value is set to 4.1 per second. The comparison operation uses a floating-point exact comparison, and the comparison result generates a Boolean logic value of true or false.

[0049] When the rate of change of the spatial density entropy of the point cloud exceeds a critical value, the number of points in each spatial unit of the point cloud data of the current independent biological target is multiplied by a weighting factor. The weighting factor is set according to the degree to which the rate of change of the entropy exceeds the critical value. The specific calculation formula is that the weighting factor equals 1 divided by 1 plus the exponential function value. The base of the exponential function is the natural constant, and the exponent is the difference between the rate of change of the entropy and the critical value multiplied by a scaling factor. The scaling factor is set to 0.5 according to the reliability requirements of the point cloud data. For example, when the rate of change of the entropy is 6 per second and the critical value is 5 per second, the weighting factor is about 0.37. The weighting operation is performed independently in each unit of the three-dimensional spatial grid. The spatial unit index is consistent with the unit index of the point cloud density distribution histogram in step S5. The number of points after weighting is retained as an integer. When the number of points is less than 1 after calculation, it is set to zero. The maximum value of the weighting factor is limited to 1 and the minimum value is limited to 0.01.

[0050] Spatiotemporal features of the point cloud, including 3D spatial coordinates and timestamps, are generated based on the reduced-weight point cloud data. The spatiotemporal feature data structure includes fields: spatial cell index, average x-axis coordinate, average y-axis coordinate, average z-axis coordinate, and timestamp sequence. The spatial cell index comes from the cell index used in the reduction operation. The average coordinate is calculated as the arithmetic mean of all point cloud coordinates within the spatial cell. When the number of point clouds in the spatial cell is zero, the average coordinate is set as the geometric center coordinate of the cell. The timestamp sequence records the timestamps of the spatial cell in the continuous scanning cycle. The timestamp format is synchronized with the radar scanning cycle and uses millisecond-level Unix timestamps. The feature generation operation is performed on the reduced-weight point cloud data. One feature record is generated for each spatial cell, and the feature records are arranged in ascending order of timestamp.

[0051] The spatiotemporal features of point clouds are input into a convolutional neural network for fall recognition. This is achieved through the following steps: The three-dimensional spatial coordinates of the point cloud spatiotemporal features are mapped onto a two-dimensional planar grid to generate a spatial distribution matrix. The two-dimensional planar grid is based on a horizontal plane, and the grid size is set to 0.2 meters by 0.2 meters based on the spatial unit size. The row index of the spatial distribution matrix is ​​equal to the y-axis coordinate divided by 0.2 meters and rounded down, and the column index is equal to the x-axis coordinate divided by 0.2 meters and rounded down. The matrix element value is the sum of the number of point clouds in all spatial units within that grid unit; when a grid unit has no point clouds, the element value is set to zero. The spatial distribution matrix is ​​superimposed with a timestamp sequence to form a three-dimensional spatiotemporal tensor. The timestamp sequence is arranged in the order of the scan cycle. The first dimension of the three-dimensional spatiotemporal tensor corresponds to the row index of the spatial distribution matrix, the second dimension corresponds to the column index of the spatial distribution matrix, and the third dimension corresponds to the time dimension index. The tensor element value is the spatial distribution matrix element value at the corresponding time moment. The time dimension length is fixed at 8 consecutive scan cycles. The tensor is then processed through convolutional layers of the convolutional neural network. The local motion pattern features of the spatiotemporal tensor are extracted. The convolutional layer uses a 3D convolutional kernel with a kernel size of 3 rows by 3 columns by 3 time frames. There are 32 convolutional kernels with a stride of 1. The padding mode is identical padding, which means zeros are padded at the edges of the input tensor to make the output size the same as the input size. The local motion pattern features are downsampled through the pooling layer of the convolutional neural network. The pooling layer uses the max pooling method with a pooling window size of 2 rows by 2 columns by 2 time frames and a stride of 2. The pooling operation is performed independently in each dimension. The fully connected layer of the convolutional neural network outputs the fall action classification probability. The fully connected layer contains 128 neurons. The neuron input is a one-dimensional vector of the pooling layer output features flattened. The activation function is the ReLU function, which means the output value is equal to the maximum value of the input value and zero. The output layer contains 2 neurons, corresponding to the fall and non-fall categories respectively. The activation function is the Softmax function, which means the output value is equal to the exponential input value divided by the sum of the exponential input values ​​of all categories. The output value is a probability value between 0 and 1, and the sum of the probability values ​​is 1.

[0052] Example 2: Figure 2 A schematic diagram of the millimeter-wave radar fall detection system based on a convolutional neural network according to the present invention is provided. The millimeter-wave radar fall detection system based on a convolutional neural network includes: The point cloud generation module is used to acquire the raw echo signal of the millimeter-wave radar and generate a three-dimensional point cloud sequence through frequency-modulated continuous wave demodulation. The feature extraction module is used to perform multi-target clustering segmentation on the 3D point cloud sequence to separate independent target point cloud clusters and extract the micro-motion time-frequency features of each target point cloud cluster. The time-frequency analysis module is used to simultaneously calculate the cross-correlation of the micro-motion time-frequency characteristics and the random walk diffusion coefficient of the Doppler phase signal for adjacent target point cloud clusters with a spatial distance of less than a set threshold. The target discrimination module is used to determine different biological targets if the cross-correlation is lower than the separation threshold and the difference in random walk diffusion coefficient exceeds the physiological difference threshold. The entropy analysis module is used to generate the rate of change of point cloud spatial density entropy by analyzing the point cloud spatial density distribution of each independent biological target within the fall determination time window. The fall detection module is used to reduce the weight of the point cloud data of the current independent biological target when the rate of change of the spatial density entropy value of the point cloud exceeds the critical value, and then input the spatiotemporal features of the reduced point cloud into the convolutional neural network for fall action recognition.

[0053] All calculations involved in the embodiments are dimensionless numerical calculations, and the preset parameters and thresholds in the calculations are set by those skilled in the art according to the actual situation.

[0054] It should be noted that this invention can be deployed on the device itself to realize embedded applications, or it can run on a PC or other terminal with a user interface, thereby meeting various hardware environments and usage requirements.

[0055] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wireless or wired transmission; wired transmission methods include optical fiber, twisted pair, coaxial cable, etc.; wireless transmission includes infrared, microwave, etc. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center containing one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0056] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0057] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0058] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0059] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0060] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0061] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0062] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A millimeter-wave radar fall detection method based on convolutional neural networks, characterized in that, include: S1. Acquire the raw echo signal from the millimeter-wave radar and generate a three-dimensional point cloud sequence through frequency-modulated continuous wave demodulation. S2. Perform multi-target clustering segmentation on the 3D point cloud sequence to separate independent target point cloud clusters, and extract the micro-motion time-frequency features of each target point cloud cluster; S3. For adjacent target point cloud clusters with a spatial distance less than a set threshold, simultaneously calculate the cross-correlation of their micro-motion time-frequency characteristics and the random walk diffusion coefficient of the Doppler phase signal. S4. If the cross-correlation is lower than the separation threshold and the difference in random walk diffusion coefficient exceeds the physiological difference threshold, then they are determined to be targets from different organisms. S5. By analyzing the point cloud spatial density distribution of each independent biological target within the fall determination time window, the point cloud spatial density entropy change rate is generated. S6. When the rate of change of the spatial density entropy of the point cloud exceeds the critical value, the point cloud data of the current independent biological target is downweighted, and the spatiotemporal features of the downweighted point cloud are input into the convolutional neural network for fall action recognition.

2. The millimeter-wave radar fall detection method based on convolutional neural networks according to claim 1, characterized in that, The raw echo signal from the millimeter-wave radar is acquired, and a three-dimensional point cloud sequence is generated through frequency-modulated continuous wave demodulation, including: The raw echo signal from the millimeter-wave radar is mixed to generate an intermediate frequency signal; Range spectrum is generated by performing a range-dimensional fast Fourier transform on the intermediate frequency signal; Target range cells are extracted from the range spectrum using constant false alarm rate detection. The Doppler-dimensional fast Fourier transform is performed on the signal within the target range cell to generate the range-Doppler matrix; Target scattering points are extracted based on the amplitude peak of the range-Doppler matrix; The distance, Doppler velocity, and azimuth measurements of the target scattering point are converted into three-dimensional spatial coordinates to form a three-dimensional point cloud sequence.

3. The millimeter-wave radar fall detection method based on convolutional neural networks according to claim 1, characterized in that, Multi-target clustering is used to segment and separate independent target point cloud clusters from the 3D point cloud sequence, and the micro-motion time-frequency features of each target point cloud cluster are extracted, including: Calculate the Euclidean distance between points based on the spatial location of points in a 3D point cloud sequence; Density clustering of the 3D point cloud sequence is performed based on the Euclidean distance between points to generate an initial set of point cloud clusters. Independent target point cloud clusters are formed by associating the initial point cloud cluster set with the target centroid displacement between consecutive frames; The timing signal of each independent target point cloud cluster is bandpass filtered to separate the micro-motion signal; The micro-motion signal is transformed by time and frequency to generate micro-motion time and frequency characteristics that include respiratory fluctuation frequency components and limb acceleration frequency band components.

4. The millimeter-wave radar fall detection method based on convolutional neural networks according to claim 1, characterized in that, For adjacent target point cloud clusters with a spatial distance less than a set threshold, the cross-correlation of their micro-motion time-frequency characteristics and the random walk diffusion coefficient of their Doppler phase signals are calculated simultaneously, including: Calculate the Euclidean distance between the centroids of adjacent target point cloud clusters as the spatial distance, and filter out adjacent target point cloud clusters whose spatial distance is less than a set threshold. Obtain the in-phase and quadrature components in the original echo signals corresponding to the filtered adjacent target point cloud clusters; Calculate the Doppler phase signal of each adjacent target point cloud cluster based on in-phase and quadrature components; The Doppler phase signal is time-differenced to generate a phase difference sequence; The variance of the phase difference sequence is calculated as the random walk diffusion coefficient; Extract the time-domain waveforms of the respiratory fluctuation frequency component and the limb acceleration frequency band component from the micro-motion time-frequency features of adjacent target point cloud clusters; The cross-correlation coefficient between the time-domain waveform of the respiratory fluctuation frequency component and the time-domain waveform of the limb acceleration frequency band component is calculated as the cross-correlation coefficient.

5. The millimeter-wave radar fall detection method based on convolutional neural networks according to claim 4, characterized in that, The cross-correlation coefficient between the time-domain waveforms of the respiratory fluctuation frequency component and the time-domain waveforms of the limb acceleration frequency band component is calculated through the following steps: Extract discrete sampling point sequences of the time-domain waveforms of respiratory fluctuation frequency components and limb acceleration frequency band components; Calculate the covariance between the discrete sampling point sequence of the time-domain waveform of the respiratory fluctuation frequency component and the discrete sampling point sequence of the time-domain waveform of the limb acceleration frequency band component; Calculate the standard deviation of the discrete sampling point sequence of the time-domain waveform of the respiratory fluctuation frequency component and the standard deviation of the discrete sampling point sequence of the time-domain waveform of the limb acceleration frequency band component; Divide the covariance by the product of the standard deviation of the discrete sampling point sequence of the time-domain waveform of the respiratory fluctuation frequency component and the standard deviation of the discrete sampling point sequence of the time-domain waveform of the limb acceleration frequency band component, and generate the cross-correlation coefficient as the cross-correlation.

6. The millimeter-wave radar fall detection method based on convolutional neural networks according to claim 1, characterized in that, If the cross-correlation is below the separation threshold and the difference in random walk diffusion coefficients exceeds the physiological difference threshold, then they are determined to be targets from different organisms, including: Calculate the difference between the cross-correlation of adjacent target point cloud clusters and the separation threshold; Calculate the absolute difference in the random walk diffusion coefficients of adjacent target point cloud clusters; When the difference between the cross-correlation and the separation threshold is less than zero and the absolute difference of the random walk diffusion coefficient is greater than the physiological difference threshold, the determination results of different biological targets are generated. The determination result is associated with the corresponding adjacent target point cloud cluster.

7. The millimeter-wave radar fall detection method based on convolutional neural networks according to claim 1, characterized in that, By analyzing the point cloud spatial density distribution of each independent biological target within the fall determination time window, the rate of change of point cloud spatial density entropy is generated, including: Obtain the fall detection time window and divide the fall detection time window into multiple consecutive time sub-intervals; For point cloud data of independent biological targets within each time sub-interval, calculate the point cloud density distribution histogram in three-dimensional space; The proportion of the number of points in each spatial unit to the total number of points in the point cloud is calculated based on the point cloud density distribution histogram. The point cloud spatial density entropy value of the current time sub-interval is calculated using the Shannon entropy formula based on the ratio value. Calculate the difference between the spatial density entropy values ​​of the point cloud in adjacent time intervals; The rate of change of point cloud spatial density entropy is generated by dividing the average of the difference values ​​by the length of the time sub-interval.

8. The millimeter-wave radar fall detection method based on convolutional neural networks according to claim 1, characterized in that, When the rate of change of the spatial density entropy of the point cloud exceeds a critical value, the point cloud data of the current independent biological target is downweighted, and the spatiotemporal features of the downweighted point cloud are input into a convolutional neural network for fall action recognition, including: Compare the relationship between the rate of change of point cloud spatial density entropy and the critical value; When the rate of change of the spatial density entropy of the point cloud exceeds the critical value, the number of points in each spatial unit of the point cloud data of the current independent biological target is multiplied by the weighting factor. Generate spatiotemporal features of point clouds containing three-dimensional spatial coordinates and timestamps based on the reduced-weight point cloud data; The spatiotemporal features of point clouds are input into a convolutional neural network for fall recognition.

9. The millimeter-wave radar fall detection method based on convolutional neural networks according to claim 8, characterized in that, The process of inputting point cloud spatiotemporal features into a convolutional neural network for fall recognition is achieved through the following steps: The three-dimensional spatial coordinates in the spatiotemporal features of point clouds are mapped to a two-dimensional planar mesh to generate a spatial distribution matrix; The spatial distribution matrix is ​​superimposed with the timestamp sequence to form a three-dimensional spatiotemporal tensor; Local motion pattern features of spatiotemporal tensors are extracted using the convolutional layers of a convolutional neural network. Local motion pattern features are downsampled using pooling layers of a convolutional neural network; The fully connected layer of a convolutional neural network outputs the probability of a fall action classification.

10. A millimeter-wave radar fall detection system based on convolutional neural networks, used to implement the millimeter-wave radar fall detection method based on convolutional neural networks as described in any one of claims 1-9, characterized in that, include: The point cloud generation module is used to acquire the raw echo signal of the millimeter-wave radar and generate a three-dimensional point cloud sequence through frequency-modulated continuous wave demodulation. The feature extraction module is used to perform multi-target clustering segmentation on the 3D point cloud sequence to separate independent target point cloud clusters and extract the micro-motion time-frequency features of each target point cloud cluster. The time-frequency analysis module is used to simultaneously calculate the cross-correlation of the micro-motion time-frequency characteristics and the random walk diffusion coefficient of the Doppler phase signal for adjacent target point cloud clusters with a spatial distance of less than a set threshold. The target discrimination module is used to determine different biological targets if the cross-correlation is lower than the separation threshold and the difference in random walk diffusion coefficient exceeds the physiological difference threshold. The entropy analysis module is used to generate the rate of change of point cloud spatial density entropy by analyzing the point cloud spatial density distribution of each independent biological target within the fall determination time window. The fall detection module is used to reduce the weight of the point cloud data of the current independent biological target when the rate of change of the spatial density entropy value of the point cloud exceeds the critical value, and then input the spatiotemporal features of the reduced point cloud into the convolutional neural network for fall action recognition.

Images