Millimeter wave radar fall detection method and system based on convolutional neural network

Abstract

The application discloses a millimeter wave radar fall detection method and system based on a convolutional neural network, and particularly relates to the technical field of radar signal processing, and is used for solving the fall misjudgment problem caused by point cloud adhesion in a multi-target coexistence scene; by extracting the micro-motion time-frequency features of a target point cloud cluster, the micro-motion feature cross-correlation and the Doppler phase random walk diffusion coefficient are synchronously calculated for spatially adjacent targets, and the independent biological body target is determined according to the cross-correlation being lower than a separation threshold value and the diffusion coefficient difference exceeding a physiological difference threshold value; then, the point cloud spatial density entropy value change rate of each independent target in a fall time window is analyzed, when the change rate exceeds a critical value, the point cloud is subjected to weight reduction processing, and the spatio-temporal features after the weight reduction are input into a convolutional neural network to recognize a fall action; and high-precision fall detection in a strong interference environment is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of radar signal processing, in particular to a millimeter wave radar fall detection method and system based on a convolutional neural network. BACKGROUND

[0002] Millimeter wave radar technology can be applied to human behavior monitoring, especially to non-contact fall detection in privacy-sensitive areas such as toilets and bathrooms; by transmitting frequency-modulated continuous waves and receiving target reflection signals, three-dimensional point cloud data representing the spatial position of the human body can be generated, and the existing technology uses a convolutional neural network to recognize the posture of the point cloud sequence to realize fall event judgment. This method avoids the risk of camera privacy leakage and is not affected by light and water mist environment, and gradually replaces wearable devices in the smart elderly care monitoring scene.

[0003] However, when there are multiple dynamic targets in the monitoring area (such as a caregiver intervening in a rescue scene), the existing method faces a fundamental limitation: the micro-motion signals of different targets (such as the sudden movement of a faller and the breathing fluctuations of a rescuer) are coupled in the time-frequency domain, causing point cloud spatial features to stick together. Due to the physical resolution limit of millimeter wave radar, multiple target point clouds coexisting at sub-meter distances cannot be effectively separated, and when the convolutional neural network extracts features from the stuck point cloud, it is difficult to distinguish between fall actions and interference micro-motions, resulting in behavior misjudgment and significantly reducing system reliability in actual scenes that require multi-target collaborative monitoring. SUMMARY

[0004] The present application provides a millimeter wave radar fall detection method and system based on a convolutional neural network to solve the technical problems in the prior art.

[0005] The technical solution of the present application to solve the above technical problems is as follows:

[0006] The present application provides the following technical solutions:

[0007] The millimeter wave radar fall detection method based on a convolutional neural network comprises:

[0008] S1, obtaining the original echo signal of the millimeter wave radar, generating a three-dimensional point cloud sequence through frequency-modulated continuous wave demodulation;

[0009] S2, performing multi-target clustering segmentation on the three-dimensional point cloud sequence to separate independent target point cloud clusters, and extracting the micro-motion time-frequency features of each target point cloud cluster;

[0010] S3, for adjacent target point cloud clusters with a spatial distance less than a set threshold, synchronously calculating the cross-correlation of their micro-motion time-frequency features and the random walk diffusion coefficient of the Doppler phase signal;

[0011] S4, if the cross-correlation is lower than the separation threshold and the random walk diffusion coefficient difference exceeds the physiological difference threshold, the different biological object target is determined;

[0012] S5, by analyzing the point cloud spatial density distribution of each independent biological object target in the fall determination time window, the point cloud spatial density entropy value change rate is generated;

[0013] S6, when the point cloud spatial density entropy value change rate exceeds the critical value, the point cloud data of the current independent biological object target is processed by weight reduction, and the weight-reduced point cloud space-time feature is input into the convolutional neural network for fall action recognition.

[0014] Further, the original echo signal of the millimeter wave radar is acquired, and a three-dimensional point cloud sequence is generated by frequency modulation continuous wave demodulation, including:

[0015] The original echo signal of the millimeter wave radar is mixed to generate an intermediate frequency signal;

[0016] The intermediate frequency signal is subjected to distance dimension fast Fourier transform to generate a distance spectrum;

[0017] The target distance unit is extracted on the distance spectrum by constant false alarm rate detection;

[0018] The signal in the target distance unit is subjected to Doppler dimension fast Fourier transform to generate a range-Doppler matrix;

[0019] The target scattering point is extracted based on the amplitude peak value of the range-Doppler matrix;

[0020] The distance value, Doppler velocity value and azimuth angle measurement value of the target scattering point are converted into three-dimensional space coordinates to form a three-dimensional point cloud sequence.

[0021] Further, the three-dimensional point cloud sequence is subjected to multi-target clustering and segmentation to separate independent target point cloud clusters, and the micro-motion time-frequency feature of each target point cloud cluster is extracted, including:

[0022] The inter-point Euclidean distance is calculated based on the spatial position of the points in the three-dimensional point cloud sequence;

[0023] The three-dimensional point cloud sequence is subjected to density clustering to generate an initial point cloud cluster set according to the inter-point Euclidean distance;

[0024] The independent target point cloud cluster is formed by associating the initial point cloud cluster set through the target mass center displacement between consecutive frames;

[0025] The micro-motion signal is separated by band-pass filtering the time sequence signal of each independent target point cloud cluster;

[0026] The micro-motion time-frequency feature containing the respiratory fluctuation frequency component and the limb acceleration frequency band component is generated by time-frequency transformation of the micro-motion signal.

[0027] Further, for the adjacent target point cloud clusters with spatial distance less than a set threshold, the cross-correlation of the micro-motion time-frequency features and the random walk diffusion coefficient of the Doppler phase signals are calculated synchronously, including:

[0028] The Euclidean distance between the centroids of the adjacent target point cloud clusters is calculated as the spatial distance, and the adjacent target point cloud clusters with spatial distance less than a set threshold are screened out;

[0029] The in-phase component and the quadrature component in the original echo signal corresponding to the screened adjacent target point cloud clusters are obtained;

[0030] The Doppler phase signals of the adjacent target point cloud clusters are calculated based on the in-phase component and the quadrature component;

[0031] The time difference of the Doppler phase signals is performed to generate a phase difference sequence;

[0032] The variance value of the phase difference sequence is calculated as the random walk diffusion coefficient;

[0033] The time-domain waveforms of the respiratory fluctuation frequency component and the limb acceleration frequency band component in the micro-motion time-frequency features of the adjacent target point cloud clusters are extracted;

[0034] The cross-correlation coefficient of the time-domain waveforms of the respiratory fluctuation frequency component and the limb acceleration frequency band component is calculated as the cross-correlation.

[0035] Further, the cross-correlation coefficient of the time-domain waveforms of the respiratory fluctuation frequency component and the limb acceleration frequency band component is calculated as the cross-correlation by the following steps:

[0036] The discrete sampling point sequences of the time-domain waveforms of the respiratory fluctuation frequency component and the limb acceleration frequency band component are extracted;

[0037] The covariance value of the discrete sampling point sequences of the time-domain waveforms of the respiratory fluctuation frequency component and the limb acceleration frequency band component is calculated;

[0038] The standard deviation value of the discrete sampling point sequence of the time-domain waveform of the respiratory fluctuation frequency component and the standard deviation value of the discrete sampling point sequence of the time-domain waveform of the limb acceleration frequency band component are calculated;

[0039] The covariance value is divided by the product of the standard deviation value of the discrete sampling point sequence of the time-domain waveform of the respiratory fluctuation frequency component and the standard deviation value of the discrete sampling point sequence of the time-domain waveform of the limb acceleration frequency band component to generate the cross-correlation coefficient as the cross-correlation.

[0040] Further, if the cross-correlation is lower than a separation threshold and the difference in the random walk diffusion coefficient exceeds a physiological difference threshold, it is determined that the targets are different biological organisms, including:

[0041] calculate the difference between the cross-correlation of adjacent target point cloud clusters and the separation threshold value;

[0042] calculate the absolute difference of random walk diffusion coefficients of adjacent target point cloud clusters;

[0043] generate a determination result of different biological targets when the difference between the cross-correlation and the separation threshold value is less than zero and the absolute difference of the random walk diffusion coefficients is greater than the physiological difference threshold value;

[0044] associate the determination result to the corresponding adjacent target point cloud cluster.

[0045] Further, by analyzing the point cloud spatial density distribution of each independent biological target within the fall determination time window, a point cloud spatial density entropy change rate is generated, including:

[0046] obtain a fall determination time window and divide the fall determination time window into multiple continuous time subintervals;

[0047] calculate the point cloud density distribution histogram in three-dimensional space for the point cloud data of the independent biological target in each time subinterval;

[0048] calculate the proportion value of the point cloud number of each spatial unit to the total point cloud number based on the point cloud density distribution histogram;

[0049] calculate the point cloud spatial density entropy value of the current time subinterval according to the proportion value through the Shannon entropy formula;

[0050] calculate the difference value of the point cloud spatial density entropy values of adjacent time subintervals;

[0051] divide the average value of the difference value by the length of the time subinterval to generate the point cloud spatial density entropy change rate.

[0052] Further, when the point cloud spatial density entropy change rate exceeds a critical value, the point cloud data of the current independent biological target is down-weighted, and the down-weighted point cloud spatiotemporal feature is input into a convolutional neural network for fall motion recognition, including:

[0053] compare the size relationship between the point cloud spatial density entropy change rate and the critical value;

[0054] when the point cloud spatial density entropy change rate exceeds the critical value, multiply the point cloud number of each spatial unit in the point cloud data of the current independent biological target by a down-weighting coefficient;

[0055] generate a point cloud spatiotemporal feature containing three-dimensional spatial coordinates and a timestamp based on the down-weighted point cloud data;

[0056] input the point cloud spatiotemporal feature into a convolutional neural network for fall motion recognition.

[0057] Further, the point cloud spatiotemporal feature is input into a convolutional neural network to realize fall action recognition through the following steps:

[0058] The three-dimensional space coordinates in the point cloud spatiotemporal feature are mapped to a two-dimensional plane grid to generate a spatial distribution matrix;

[0059] The spatial distribution matrix is superimposed with a timestamp sequence to form a three-dimensional spatiotemporal tensor;

[0060] Local motion pattern features of the spatiotemporal tensor are extracted through a convolutional layer of the convolutional neural network;

[0061] The local motion pattern features are down-sampled through a pooling layer of the convolutional neural network;

[0062] Fall action classification probabilities are output through a fully connected layer of the convolutional neural network.

[0063] In another aspect, the present application provides a millimeter wave radar fall detection system based on a convolutional neural network, comprising:

[0064] A point cloud generation module is configured to acquire original echo signals of a millimeter wave radar, and generate a three-dimensional point cloud sequence through frequency-modulated continuous wave demodulation;

[0065] A feature extraction module is configured to separate independent target point cloud clusters from the three-dimensional point cloud sequence through multi-target clustering segmentation, and extract micro-motion time-frequency features of each target point cloud cluster;

[0066] A time-frequency analysis module is configured to calculate the cross-correlation of the micro-motion time-frequency features and the random walk diffusion coefficient of the Doppler phase signal of adjacent target point cloud clusters with a spatial distance less than a set threshold;

[0067] A target discrimination module is configured to determine that the adjacent target point cloud clusters are different biological targets if the cross-correlation is lower than a separation threshold and the random walk diffusion coefficient difference exceeds a physiological difference threshold;

[0068] An entropy value analysis module is configured to generate a point cloud spatial density entropy value change rate by analyzing the point cloud spatial density distribution of each independent biological target within a fall determination time window;

[0069] A fall recognition module is configured to perform weight reduction processing on the point cloud data of the current independent biological target when the point cloud spatial density entropy value change rate exceeds a critical value, and input the weight-reduced point cloud spatiotemporal feature into a convolutional neural network to perform fall action recognition.

[0070] The present application has the following beneficial effects:

[0071] 1. By multi-target physiological feature decoupling, the reliability of fall detection in multi-person coexistence scenarios is significantly improved. To solve the problem of point cloud adhesion caused by insufficient physical resolution of millimeter wave radar, the intrinsic differences of biological micro-motion signals are utilized: by synchronously calculating the micro-motion time-frequency feature cross-correlation of adjacent target point cloud clusters and the diffusion coefficient of Doppler phase random walk, the independence of different biological respiratory rhythms and the specificity of limb tremor are accurately captured. When the cross-correlation is lower than the separation threshold and the diffusion coefficient difference exceeds the physiological difference threshold, the independent targets in sub-meter close-range coexistence are effectively distinguished, breaking through the misjudgment bottleneck caused by signal coupling in traditional methods, and the system anti-interference ability in multi-target cooperative monitoring scenarios is essentially improved.

[0072] 2. By analyzing the spatial density entropy change of point cloud, the spatiotemporal feature representation ability of fall action is strengthened. Based on the point cloud distribution characteristics of independent biological targets within the fall determination time window, the posture disorder rate of human instability process is quantified by the spatial density entropy value change rate, and the learning bias of traditional convolutional neural network to adhesion point cloud features is overcome. When the entropy value change rate exceeds the critical value, point cloud weight reduction processing is implemented to suppress the influence of sudden interference motion on feature extraction, so that the convolutional neural network focuses on identifying the instability trajectory specific to falls. The spatiotemporal dynamics features of radar point cloud are deeply integrated with deep learning model, and the fall recognition accuracy is still maintained in strong interference environment, providing high-reliable non-contact monitoring for privacy-sensitive areas. BRIEF DESCRIPTION OF DRAWINGS

[0073] Figure 1 The flowchart of the millimeter wave radar fall detection method based on convolutional neural network of the present application;

[0074] Figure 2 The structural schematic diagram of the millimeter wave radar fall detection system based on convolutional neural network of the present application. DETAILED DESCRIPTION

[0075] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0076] Embodiment 1: Figure 1 The millimeter wave radar fall detection method based on convolutional neural network of the present application is given, which includes:

[0077] S1, acquire the original echo signal of the millimeter wave radar, and generate a three-dimensional point cloud sequence through frequency-modulated continuous wave demodulation;

[0078] S2, multi-target clustering segmentation is performed on the three-dimensional point cloud sequence to separate independent target point cloud clusters, and the micro-motion time-frequency features of each target point cloud cluster are extracted;

[0079] S3, for adjacent target point cloud clusters with a spatial distance less than a set threshold, the cross-correlation of their micro-motion time-frequency features and the random walk diffusion coefficient of the Doppler phase signal are calculated synchronously;

[0080] S4, if the cross-correlation is lower than a separation threshold and the difference in the random walk diffusion coefficient exceeds a physiological difference threshold, it is determined that the targets are different biological bodies;

[0081] S5, by analyzing the point cloud spatial density distribution of each independent biological body target within a fall determination time window, a point cloud spatial density entropy value change rate is generated;

[0082] S6, when the point cloud spatial density entropy value change rate exceeds a critical value, the point cloud data of the current independent biological body target is down-weighted, and the down-weighted point cloud space-time features are input into a convolutional neural network for fall motion recognition.

[0083] After obtaining the original echo signal of the millimeter wave radar, the specific implementation process of generating the three-dimensional point cloud sequence through the frequency-modulated continuous wave demodulation is as follows:

[0084] The original echo signal of the millimeter wave radar is mixed to generate an intermediate frequency signal. The mixing process refers to multiplying the received original echo signal with the transmitted frequency-modulated continuous wave signal, then filtering out the high-frequency components through a low-pass filter to obtain the difference frequency signal as the intermediate frequency signal. The frequency of the intermediate frequency signal is linearly related to the target distance. The frequency range of the local oscillator signal used in the mixing process is set according to the radar operating frequency band, for example, the center frequency is selected within the 76 gigahertz to 81 gigahertz frequency band. The cutoff frequency of the low-pass filter is determined according to the difference frequency value corresponding to the maximum detection distance of the radar, for example, for a radar system with a 200-meter detection distance, the cutoff frequency of the low-pass filter is set to 1 megahertz.

[0085] The intermediate frequency signal is subjected to distance dimension fast Fourier transform to generate a range profile. The number of sampling points of the distance dimension fast Fourier transform is set according to the maximum detection distance of the radar, for example, 2048-point Fourier transform corresponds to a 200-meter range. The specific point number calculation formula is that the number of sampling points is equal to the maximum detection distance divided by the distance resolution. The distance resolution is determined by the signal bandwidth, and when the signal bandwidth is 4 gigahertz, the distance resolution is about 3.75 centimeters. The horizontal axis of the range profile represents the distance unit, and the actual distance value corresponding to each distance unit is the distance unit number multiplied by the distance resolution. The amplitude peak position of the range profile corresponds to the target distance value.

[0086] The target distance unit is extracted by constant false alarm rate detection on the distance spectrum, and the constant false alarm rate detection is realized by cell average selection method, specifically including: taking the distance unit to be detected as the center, selecting 10 distance units before and after as the reference window, calculating the average power value of all distance units in the reference window after excluding the protection unit, multiplying the average power value by the calibration coefficient as the detection threshold, and the calibration coefficient is obtained by table lookup according to the preset false alarm probability, for example, when the preset false alarm probability is 0.000001, the calibration coefficient takes the value of 2.5, when the power value of the distance unit to be detected exceeds the detection threshold, it is determined as the target distance unit, and the number of protection units is set according to the radar distance resolution, for example, the 2 distance units before and after are taken as the protection unit.

[0087] The signal in the target distance unit is subjected to Doppler frequency fast Fourier transform to generate a distance-Doppler matrix, the Doppler frequency fast Fourier transform is performed on the continuous 128 radar period signals, the transform point number determines the speed resolution, the 256-point transform corresponds to a speed resolution of 0.2 meters per second, and the specific speed resolution calculation formula is the speed of light divided by twice the radar center frequency multiplied by the coherent accumulation time, and the coherent accumulation time is equal to the radar period multiplied by the transform point number, the row of the distance-Doppler matrix represents the distance unit, and the column represents the Doppler speed unit, and the matrix element value is the signal amplitude value.

[0088] The target scattering point is extracted based on the amplitude peak value of the distance-Doppler matrix, and the amplitude peak value extraction is realized by local maximum value search, specifically including: setting a 3*3 sliding window in the distance-Doppler matrix, when the amplitude value of the window center unit is greater than the surrounding 8 adjacent units and exceeds the noise floor by 10 decibels, it is determined as an effective target scattering point, and the noise floor is determined by the statistical average value of the non-target region, the non-target region selects the 10% region with the smallest amplitude value in the distance-Doppler matrix, and the noise floor is calculated as the arithmetic average value of the amplitude values of all units in the region.

[0089] The distance value, Doppler velocity value and azimuth measurement value of the target scattering point are converted into three-dimensional space coordinates to form a three-dimensional point cloud sequence, and the specific conversion method is as follows: the distance value is calculated by the peak position of the distance dimension fast Fourier transform, and the calculation formula is distance unit number multiplied by distance resolution; the Doppler velocity value is calculated by the peak position of the Doppler dimension fast Fourier transform, and the calculation formula is Doppler unit number multiplied by velocity resolution; the azimuth measurement value is obtained by calculating the phase difference of the array antenna received signal, and the specific method is to calculate the azimuth by using beam forming; the three-dimensional space coordinate conversion formula is: the x-axis coordinate is equal to the distance value multiplied by the cosine value of the azimuth, the y-axis coordinate is equal to the distance value multiplied by the sine value of the azimuth, and the z-axis coordinate is obtained by multiplying the Doppler velocity value by time integration, and the time integration is performed in each radar period by using the trapezoidal integration method; one frame of three-dimensional point cloud sequence containing all target scattering points is generated in each radar scanning period, and the time interval of the three-dimensional point cloud sequence is equal to the radar scanning period, for example, 30 milliseconds.

[0090] The specific implementation process of the three-dimensional point cloud sequence is as follows:

[0091] The spatial position of the point in the three-dimensional point cloud sequence is calculated based on the spatial position of the point, and the three-dimensional point cloud sequence is generated by the radar scanning period, and each scanning period contains the three-dimensional space coordinates of multiple target scattering points; the point-to-point Euclidean distance is calculated by using the distance formula in the three-dimensional space rectangular coordinate system, specifically, the sum of the square differences of the two point coordinate values is square rooted; the input three-dimensional space coordinate values include x-axis coordinate values, y-axis coordinate values and z-axis coordinate values; the calculation range of the point-to-point Euclidean distance is limited in the point cloud data in the same radar scanning period, and the point cloud data in different scanning periods is not calculated; the coordinate values in the Euclidean distance calculation formula are in meters, and the calculation result is also in meters.

[0092] The three-dimensional point cloud sequence is density clustered according to the point-to-point Euclidean distance to generate an initial point cloud cluster set, and the density clustering adopts a clustering algorithm based on a neighborhood radius and a minimum point threshold; the neighborhood radius is set according to the radar distance resolution, specifically, 4 times the radar distance resolution, for example, when the radar distance resolution is 3.75 cm, the neighborhood radius is set to 15 cm; the minimum point threshold is set according to the minimum size of the target, specifically, the minimum number of scattering points required to cover the minimum size of the target, for example, the minimum number of points of the human body target is set to 5 points; the density clustering process includes: traversing each point in the three-dimensional point cloud sequence, searching for adjacent points within the neighborhood radius centering on the point, and forming an initial point cloud cluster when the number of adjacent points reaches the minimum point threshold; the core point priority attribution principle is adopted when attributing the boundary points, and the point is deleted as a noise point when the number of points in the neighborhood is less than the minimum point threshold and does not belong to any existing cluster; finally, an initial point cloud cluster set containing multiple initial point cloud clusters is output.

[0093] The independent target point cloud cluster is formed by associating the initial point cloud cluster set through the continuous frame target centroid displacement, the continuous frame refers to a plurality of radar scanning periods that are continuous in time, the target centroid displacement calculation includes: calculating the geometric center of each initial point cloud cluster as the target centroid coordinate, the target centroid coordinate is obtained by averaging the coordinates of all points in the point cloud cluster, the specific calculation formula is that the x-axis coordinate average value is equal to the sum of the x-axis coordinate values of all points in the point cloud cluster divided by the number of points in the point cloud cluster, the y-axis coordinate average value and the z-axis coordinate average value are calculated in the same way, the target centroid displacement between continuous frames is calculated by the difference between the target centroid coordinates of adjacent scanning periods, the displacement association threshold is set according to the maximum target motion speed, the specific calculation formula is that the displacement association threshold is equal to the maximum target motion speed multiplied by the radar scanning period, for example, when the maximum motion speed of the human body target is 5 meters per second and the radar scanning period is 30 milliseconds, the displacement association threshold is set to 0.15 meters, when the target centroid displacement between continuous frames is less than the displacement association threshold, it is determined to be the same target point cloud cluster, and finally the independent target point cloud cluster with time sequence continuity is formed.

[0094] The time sequence signal of each independent target point cloud cluster is band-pass filtered to separate the micro-motion signal, the time sequence signal refers to the motion characteristic change of the independent target point cloud cluster in the continuous scanning period, the motion characteristic includes the height change value of the target centroid in the z-axis direction, the band-pass filtering is realized by using a second-order Butterworth filter, the filter design parameters include the lower limit value of the passband frequency and the upper limit value of the passband frequency, the lower limit value of the passband frequency is set to 0.1 Hz according to the lowest frequency of the respiratory motion, and the upper limit value of the passband frequency is set to 10 Hz according to the highest frequency of the limb motion, the filter is realized by using a bilinear transformation method to convert an analog filter into a digital filter, the filtering process is performed on the time sequence signal of the continuous 30 scanning periods, and the separated micro-motion signal is output, the micro-motion signal contains the motion component in the frequency range of 0.1 Hz to 10 Hz.

[0095] The micro-motion signal is subjected to time-frequency transformation to generate a micro-motion time-frequency feature containing a respiratory fluctuation frequency component and a limb acceleration frequency band component. The time-frequency transformation is implemented by using a short-time Fourier transform method, which specifically includes: dividing the micro-motion signal into data segments of a fixed length, the length of the data segment being set according to the frequency resolution requirement, for example, 128 sampling points, adjacent data segments overlapping by 50% of the sampling points, for example, 64 sampling points, applying a Hanning window function to each data segment to suppress spectral leakage, the window function length being equal to the data segment length, performing fast Fourier transform on the windowed data segment, the number of transform points being set according to the frequency accuracy requirement, for example, 256 points, the transform result forming a time-frequency spectrum, the respiratory fluctuation frequency component extracting the energy distribution of the 0.1-0.3 Hz frequency band in the time-frequency spectrum, and the limb acceleration frequency band component extracting the energy distribution of the 1-10 Hz frequency band, the energy distribution being obtained by calculating the sum of squares of the spectral amplitudes in the corresponding frequency band, and finally generating a micro-motion time-frequency feature containing time dimension, frequency dimension and energy dimension.

[0096] The specific implementation process of synchronously calculating the cross-correlation of the micro-motion time-frequency feature and the random walk diffusion coefficient of the Doppler phase signal for the adjacent target point cloud clusters with a spatial distance less than a set threshold value is as follows:

[0097] The Euclidean distance between the centers of adjacent target point cloud clusters is calculated as the spatial distance, the coordinates of the center of the target point cloud cluster are obtained by averaging the coordinates of all points in the point cloud cluster, and the specific calculation formula is that the x-axis coordinate value of the center is equal to the sum of the x-axis coordinate values of all points in the point cloud cluster divided by the number of points in the point cloud cluster, the y-axis coordinate value and the z-axis coordinate value are calculated in the same way, and the Euclidean distance calculation formula is the square root of the sum of the square differences of the two center coordinates. The threshold value is set according to the minimum distance of the human body target, specifically the statistical value of the average shoulder width of an adult, for example, 50 cm. The adjacent target point cloud clusters with a spatial distance less than the set threshold value are selected, and the selection operation is performed within the same radar scanning period. When the number of target point cloud clusters exceeds 2, the distance calculation and selection are performed for all adjacent point cloud cluster pairs.

[0098] The in-phase component and the quadrature component in the original echo signal corresponding to the selected adjacent target point cloud clusters are obtained, the original echo signal is obtained from the baseband data of the millimeter wave radar receiving channel, the in-phase component and the quadrature component are obtained by demodulating the intermediate frequency signal of the radar receiver, the demodulation process uses a quadrature mixer to decompose the intermediate frequency signal into two signals with a phase difference of 90 degrees, the in-phase component and the quadrature component are stored in the radar signal processing unit in the form of digital signals, and the original echo signal corresponding to each target point cloud cluster is extracted by positioning the radar range unit and the Doppler unit index of the target scattering point, the range unit and the Doppler unit index are obtained from the range-Doppler matrix generated in step S1.

[0099] The Doppler phase signals of adjacent target point cloud clusters are calculated based on in-phase components and quadrature components, and the Doppler phase signals are obtained through arctangent operation of the in-phase components and the quadrature components, and the specific calculation formula is that the phase value is equal to the arctangent function value of the quadrature component value divided by the in-phase component value, and the value range of the phase signal is negative pi radian to positive pi radian, and when the in-phase component value is zero, the phase value is set to positive pi / 2 radian or negative pi / 2 radian according to the quadrature component sign, and the phase unwrapping processing adopts a continuity detection method to eliminate phase jumps, and specifically, when the phase difference value of adjacent scanning periods exceeds pi radian, the current phase value is increased or decreased by an integer multiple of 2pi radian to make the phase continuous, and the Doppler phase signal of each target point cloud cluster is represented by the average phase value of all scattering points in the point cloud cluster.

[0100] The Doppler phase signals are subjected to time difference processing to generate a phase difference sequence, and the time difference processing is performed on continuous time sequences, and specifically, the phase value of the current scanning period is subtracted from the phase value of the previous scanning period, and the length of the phase difference sequence is one scanning period less than that of the original phase sequence, and the unit of the phase difference value is radian, and when the phase difference value exceeds pi radian, phase wraparound correction is performed, and the correction method is that if the phase difference value is greater than pi radian, 2pi radian is subtracted, and if it is less than negative pi radian, 2pi radian is added, and the corrected phase difference value is kept within the range of negative pi radian to positive pi radian.

[0101] The variance value of the phase difference sequence is calculated as a random walk diffusion coefficient, and the variance value calculation formula is the average of the square sum of the deviations of each element of the phase difference sequence from the arithmetic mean of the sequence, and the specific calculation steps include: first, calculating the arithmetic mean of the phase difference sequence, which is equal to the sum of all elements of the sequence divided by the length of the sequence; then, calculating the deviation of each phase difference value from the arithmetic mean; summing the squared deviations; and finally, dividing by the length of the phase difference sequence minus 1 to obtain an unbiased estimated variance value, and the unit of the variance value is radian square, and when the length of the phase difference sequence is less than 2, the variance value is set to zero.

[0102] The time domain waveforms of the respiratory fluctuation frequency component and the limb acceleration frequency band component in the micro-motion time-frequency feature of the adjacent target point cloud cluster are extracted, and the micro-motion time-frequency feature comes from the output of step S2, the time domain waveform of the respiratory fluctuation frequency component corresponds to the time domain signal reconstruction of the 0.1-0.3 Hz frequency band, and the time domain waveform of the limb acceleration frequency band component corresponds to the time domain signal reconstruction of the 1-10 Hz frequency band, and the time domain waveform reconstruction is realized through inverse Fourier transform of a band-pass filter, and specifically, inverse Fourier transform is performed on the frequency spectrum components corresponding to the frequency band in the micro-motion time-frequency feature, the sampling rate of the reconstructed time domain waveform is consistent with the radar scanning period, for example, 30 ms sampling interval, and the length of the time domain waveform is the same as the time-frequency analysis window length in step S2.

[0103] The cross-correlation coefficient of the respiratory fluctuation frequency component time domain waveform and the limb acceleration frequency band component time domain waveform is calculated as the cross-correlation, which is realized by the following steps: extracting the discrete sampling point sequence of the respiratory fluctuation frequency component time domain waveform and the limb acceleration frequency band component time domain waveform, the length of the discrete sampling point sequence is the same and time-aligned, and the time alignment is ensured by the radar scanning period timestamp; calculating the covariance value of the respiratory fluctuation frequency component time domain waveform discrete sampling point sequence and the limb acceleration frequency band component time domain waveform discrete sampling point sequence, the covariance value calculation formula is the average value of the product of the corresponding sampling points of the two sequences minus the product of the average values of the two sequences, and the sequence average value is equal to the sum of all elements of the sequence divided by the sequence length; calculating the standard deviation value of the respiratory fluctuation frequency component time domain waveform discrete sampling point sequence and the standard deviation value of the limb acceleration frequency band component time domain waveform discrete sampling point sequence, the standard deviation value calculation formula is the square root of the average value of the square sum of the deviations of the sampling points of each sequence from their arithmetic mean, and the square root of the square sum divided by the sequence length minus 1; the covariance value is divided by the product of the standard deviation value of the respiratory fluctuation frequency component time domain waveform discrete sampling point sequence and the standard deviation value of the limb acceleration frequency band component time domain waveform discrete sampling point sequence to generate the cross-correlation coefficient as the cross-correlation, the value range of the cross-correlation coefficient is -1 to 1, and the cross-correlation coefficient is set to zero when the standard deviation value is zero.

[0104] If the cross-correlation is lower than the separation threshold and the random walk diffusion coefficient difference exceeds the physiological difference threshold, it is determined that the specific implementation process of the different biological body targets is as follows:

[0105] The difference between the cross-correlation of the adjacent target point cloud clusters and the separation threshold is calculated, the cross-correlation is obtained from the cross-correlation coefficient calculated in S3 step, the value range of the cross-correlation coefficient is -1 to 1, the separation threshold is set according to the statistical law of the independence of human physiological motion characteristics, and the specific setting method includes collecting multiple groups of motion data samples of different human targets in the same environment, each data sample contains at least 100 continuous scanning period motion characteristics, calculating the cross-correlation coefficient of the same body target respiratory fluctuation frequency component time domain waveform and the limb acceleration frequency band component time domain waveform for each data sample, and the value distribution range of the cross-correlation coefficient of all data samples is calculated, and the lower limit value of the distribution range is taken as the separation threshold, for example, when the minimum value of the cross-correlation coefficient is 0.3, the separation threshold is set to 0.27, the difference is calculated by subtracting the value of the separation threshold from the value of the cross-correlation, and the calculation result is kept to four decimal places.

[0106] Calculate the absolute difference of random walk diffusion coefficient of adjacent target point cloud clusters, the random walk diffusion coefficient is from the variance value calculated in S3 step, the unit is radian square, the absolute difference is calculated as the absolute value of the difference of the random walk diffusion coefficient values of the two target point cloud clusters, the calculation process retains four decimal places, when the random walk diffusion coefficient of any one of the two target point cloud clusters is missing, skip the difference calculation of the point cloud cluster pair, the missing conditions include insufficient phase difference sequence length or the data invalid flag is set.

[0107] When the difference between the cross-correlation and the separation threshold is less than zero and the absolute difference of the random walk diffusion coefficient is greater than the physiological difference threshold, generate a determination result of different biological targets, the physiological difference threshold is set according to the individual difference range of human physiological jitter characteristics, the specific setting method includes measuring the physiological jitter variance value of different parts of the same body in a stationary state, collecting at least 20 groups of measurement data of different individuals, each group of data contains at least 5 body part measurement values, calculating the maximum fluctuation range of the multi-part variance value of each individual, taking 120% of the upper limit value of all individual fluctuation ranges as the physiological difference threshold, for example, when the maximum fluctuation range upper limit is 0.5 radian square, the physiological difference threshold is set to 0.6 radian square, the two inequalities are satisfied simultaneously: the result of the cross-correlation minus the separation threshold is less than zero, and the absolute difference of the random walk diffusion coefficient is greater than the physiological difference threshold, the determination result is a Boolean logic value true or false, when the two conditions are not satisfied simultaneously, the determination result is false.

[0108] Associate the determination result to the corresponding adjacent target point cloud cluster, the association operation includes: creating a unique identifier for each adjacent target point cloud cluster pair, the identifier is generated by connecting the two point cloud cluster numbers in ascending order of numerical value with an underscore, for example, the identifier of point cloud cluster numbers 3 and 5 is 3_5, the determination result is stored in a dynamic data structure bound to the identifier, the data structure includes an identifier field, a determination result Boolean value field and a timestamp field, the timestamp is synchronized with the start time of the radar scan cycle, the timestamp format is millisecond Unix timestamp, the determination result is also written to the attribute field of the point cloud cluster, the attribute field name is biological independence identification, the field data type is integer, when the determination result is true, the attribute value is set to 1, indicating different biological targets, when the determination result is false, the attribute value is set to 0, indicating the same biological target, the attribute update operation is completed within 3 scan cycles after the determination result is generated.

[0109] By analyzing the point cloud spatial density distribution of each independent biological target within the fall determination time window, the specific implementation process of the point cloud spatial density entropy value change rate is as follows:

[0110] A fall determination time window is obtained and divided into a plurality of continuous time sub-intervals. The fall determination time window is set according to the typical duration of the human body falling process, specifically the complete time span from the beginning to the end of the falling process. The length of the time window is determined according to the human body kinematics research data, which shows that the average duration of the human body falling process is 1.2-1.8 seconds, so the length of the time window is set to 1.5 seconds. The starting time of the time window is triggered by a target acceleration mutation event, which is detected in the limb acceleration frequency band component in S2. The event detects an acceleration value exceeding 5 m / s2. When dividing the time sub-intervals, the length of each sub-interval is determined according to the human body motion continuity requirement. The length of each time sub-interval is fixed at 0.2 seconds, which is less than the minimum response time of 0.25 seconds for human body posture changes. When the total length of the time window, 1.5 seconds, cannot be divided by 0.2 seconds, the length of the last sub-interval is adjusted adaptively. The number of time sub-intervals is equal to the total length of the time window divided by the length of the sub-interval, rounded up. For example, 1.5 / 0.2=7.5, rounded up to 8 time sub-intervals, of which the first 7 sub-intervals are 0.2 seconds long and the last sub-interval is 0.1 seconds long.

[0111] For the point cloud data of each independent biological target in each time sub-interval, a point cloud density distribution histogram in three-dimensional space is calculated. The three-dimensional space uses a rectangular coordinate system, and the space range is dynamically set according to the target point cloud distribution. Specifically, a cube space centered on the target centroid is used, and the target centroid coordinates come from the calculation results of S2. The side length of the cube is twice the statistical value of the target height. The target height is recorded according to the height value in the history point cloud data when the target is standing upright. For example, if the detected target standing height is 1.8 meters, the cube side length is set to 3.6 meters. The space unit size is set according to the point cloud density resolution, and the space unit size is set to 0.2 meters cubed, which is less than the human body torso thickness of 0.25 meters. When calculating the histogram, all units in the space are traversed. The number of point clouds in each unit includes all point cloud data in the unit cube. The mapping relationship between point cloud coordinates and space units is achieved by dividing the coordinate value by the unit size and rounding down. Specifically, the unit index x-axis coordinate is equal to the point cloud x-axis coordinate value divided by 0.2 meters and rounded down. The y-axis and z-axis coordinates are calculated in the same way.

[0112] Based on the point cloud density distribution histogram, the proportion of the point cloud number of each space unit to the total point cloud number is calculated. The proportion value is calculated as follows: the total number of target point clouds in the current time sub-interval is counted, and the total number of point clouds is equal to the sum of the count values of all space units in the histogram. The proportion value of each space unit is equal to the number of point clouds in the unit divided by the total number of point clouds. When the total number of point clouds is zero, the proportion value is set to zero. The proportion value calculation result is kept to six decimal places. After the proportion value calculation is completed, normalization verification is performed. The sum of the proportion values of all space units should be 1, and the allowed error range is ±0.000001.

[0113] The point cloud spatial density entropy value of the current time sub-interval is calculated according to the proportion value by a Shannon entropy formula, and the Shannon entropy formula is specifically implemented as follows: all the proportion values of the spatial units are traversed, the natural logarithm of each non-zero proportion value is calculated, and the product of the natural logarithm of the proportion value and the proportion value is calculated, the natural logarithm of the proportion value is calculated by a mathematical function library, the sum of all the product results is calculated, and the negative value is taken, the entropy value calculation formula is the negative sum result, when all the proportion values of the spatial units are zero, the entropy value is set to zero, the entropy value calculation is retained to four decimal places, and the total number of the spatial units is determined according to the ratio of the cubic spatial size to the unit size, for example, when a space with a side length of 3.6 meters is divided into units with a size of 0.2 meters, the number of units in the x-axis direction = 3.6 / 0.2 = 18, the number of units in the y-axis direction and the z-axis direction are also 18, and the total number of the spatial units = 18*18*18 = 5832.

[0114] The difference value of the point cloud spatial density entropy value of the adjacent time sub-interval is calculated, and the difference value calculation is sequentially performed on the time sequence, specifically, the entropy value of the next time sub-interval is subtracted from the entropy value of the previous time sub-interval, the difference sequence length is one less than the number of time sub-intervals, when the entropy value sequence length is less than two, the difference value is set to zero, the difference value unit is the same as the entropy value, which is a dimensionless quantity, the calculation result is retained to three decimal places, and after the difference value calculation is completed, the range check is performed, and the entropy difference range of normal human motion is between negative 0.5 and positive 0.5.

[0115] The average value of the difference value is divided by the length of the time sub-interval to generate the point cloud spatial density entropy value change rate, the average value is calculated as the algebraic sum of all the difference values divided by the difference sequence length, when the difference sequence length is zero, the average value is set to zero, the time sub-interval length adopts a fixed value of 0.2 seconds, the point cloud spatial density entropy value change rate is equal to the average value divided by 0.2 seconds, when the difference sequence length is zero, the change rate is set to zero, the change rate unit is per second, the calculation result is retained to two decimal places, and finally output as a quantitative index representing the point cloud distribution disorder rate, which is used to detect the instability degree of the human posture in the falling process.

[0116] When the point cloud spatial density entropy value change rate exceeds the critical value, the point cloud data of the current independent biological target is subjected to weight reduction processing, and the weight-reduced point cloud space-time feature is input into a convolutional neural network for specific implementation process of falling action recognition as follows:

[0117] The size relationship between the point cloud spatial density entropy value change rate and the critical value is compared. The point cloud spatial density entropy value change rate is from the calculation result of S5 step, and the unit is per second. The critical value is set according to the typical entropy change rate of posture instability in the human body falling process. The specific setting method includes collecting point cloud data of multiple groups of human body falling action samples and normal activity samples. Each group of samples contains at least 50 falling events and 100 normal activity events. The point cloud spatial density entropy value change rate of each event is calculated. The entropy value change rate distribution range of the falling event and the entropy value change rate distribution range of the normal activity are counted. The statistical value of the separation point of the two distribution ranges is taken as the critical value. For example, when the minimum entropy value change rate of the falling event is 4.2 per second and the maximum entropy value change rate of the normal activity is 4.0 per second, the critical value is set to 4.1 per second. The size comparison operation uses floating point accurate comparison. The comparison result generates a Boolean logic value true or false.

[0118] When the point cloud spatial density entropy value change rate exceeds the critical value, the point cloud quantity of each space unit in the point cloud data of the current independent biological target is multiplied by the weight reduction coefficient. The weight reduction coefficient is set according to the degree of exceeding the critical value of the entropy value change rate. The specific calculation formula is that the weight reduction coefficient is equal to 1 divided by 1 plus the exponential function value. The base of the exponential function is the natural constant. The exponent is the difference between the entropy value change rate and the critical value multiplied by the proportion factor. The proportion factor is set to 0.5 according to the reliability requirement of the point cloud data. For example, when the entropy value change rate is 6 per second and the critical value is 5 per second, the weight reduction coefficient is about 0.37. The weight reduction operation is independently executed in each unit of the three-dimensional space grid. The space unit index is consistent with the point cloud density distribution histogram unit index of S5 step. The point cloud quantity after weight reduction retains the integer part. When the point cloud quantity calculated is less than 1, it is set to 0. The maximum value of the weight reduction coefficient is limited to 1 and the minimum value is limited to 0.01.

[0119] Based on the point cloud data after weight reduction, point cloud space-time features containing three-dimensional space coordinates and time stamps are generated. The point cloud space-time feature data structure includes fields: space unit index, x-axis coordinate average value, y-axis coordinate average value, z-axis coordinate average value, and time stamp sequence. The space unit index comes from the unit index used in the weight reduction operation. The coordinate average value is calculated as the arithmetic average of all point cloud coordinates in the space unit. When the point cloud quantity in the space unit is zero, the coordinate average value is set to the geometric center coordinate of the unit. The time stamp sequence records the time stamps of the space unit in the continuous scanning period. The time stamp format uses millisecond-level Unix time stamp synchronized with the radar scanning period. The feature generation operation is performed on the point cloud data after weight reduction. One feature record is generated for each space unit. The feature records are arranged in ascending order of time stamp.

[0120] The point cloud space-time feature is input into a convolutional neural network for fall action recognition, which is realized by the following steps: mapping the three-dimensional space coordinates in the point cloud space-time feature to a two-dimensional plane grid to generate a space distribution matrix, the two-dimensional plane grid takes the horizontal plane as the reference, the grid size is set to 0.2 meters by 0.2 meters according to the space unit size, the row index of the space distribution matrix is equal to the y-axis coordinate divided by 0.2 meters and then rounded down, the column index is equal to the x-axis coordinate divided by 0.2 meters and then rounded down, and the matrix element value is the sum of the point cloud quantities of all space units in the grid unit; the space distribution matrix and the timestamp sequence are superimposed to form a three-dimensional space-time tensor, the timestamp sequence is arranged in order according to the scanning period, the first dimension of the three-dimensional space-time tensor corresponds to the row index of the space distribution matrix, the second dimension corresponds to the column index of the space distribution matrix, and the third dimension corresponds to the time dimension index, and the tensor element value is the space distribution matrix element value at the corresponding time; the local motion pattern features of the space-time tensor are extracted through the convolution layer of the convolutional neural network, the convolution layer uses a three-dimensional convolution kernel, the convolution kernel size is 3 rows by 3 columns by 3 time frames, the number of convolution kernels is 32, the convolution step is 1, and the padding mode is the same padding, that is, zero padding is performed on the edges of the input tensor to make the output size the same as the input size; the local motion pattern features are down-sampled through the pooling layer of the convolutional neural network, the pooling layer adopts the maximum pooling method, the pooling window size is 2 rows by 2 columns by 2 time frames, the step is 2, and the pooling operation is independently performed in each dimension; the fall action classification probability is output through the fully connected layer of the convolutional neural network, the fully connected layer contains 128 neurons, the neuron input is a one-dimensional vector after the pooling layer output feature is flattened, the activation function is the ReLU function, that is, the output value is equal to the maximum value of the input value and zero, and the output layer contains 2 neurons corresponding to the fall and non-fall categories respectively, and the activation function is the Softmax function, that is, the output value is equal to the exponential input value divided by the sum of the exponential input values of all categories, the output value is a probability value between 0 and 1, and the sum of the probability values is 1.

[0121] Embodiment 2 Figure 2 The structure diagram of the millimeter wave radar fall detection system based on the convolutional neural network is given, and the millimeter wave radar fall detection system based on the convolutional neural network comprises:

[0122] A point cloud generation module is configured to acquire original echo signals of a millimeter wave radar, and generate a three-dimensional point cloud sequence through frequency-modulated continuous wave demodulation;

[0123] A feature extraction module is configured to separate independent target point cloud clusters from the three-dimensional point cloud sequence through multi-target clustering segmentation, and extract micro-motion time-frequency features of each target point cloud cluster;

[0124] A time-frequency analysis module is configured to calculate, for adjacent target point cloud clusters with a spatial distance less than a set threshold, a cross-correlation of micro-motion time-frequency characteristics and a random walk diffusion coefficient of a Doppler phase signal;

[0125] A target discrimination module is configured to determine that the targets are different biological targets if the cross-correlation is lower than a separation threshold and the random walk diffusion coefficient difference exceeds a physiological difference threshold.

[0126] An entropy value analysis module is configured to generate a point cloud spatial density entropy value change rate by analyzing the point cloud spatial density distribution of each independent biological target within a fall determination time window.

[0127] A fall recognition module is configured to perform weight reduction processing on the point cloud data of the current independent biological target when the point cloud spatial density entropy value change rate exceeds a threshold value, and input the weight-reduced point cloud space-time characteristics into a convolutional neural network for fall motion recognition.

[0128] The calculations involved in the embodiments are all dimensionless numerical calculations, and the preset parameters and threshold values in the calculations are set by a person skilled in the art according to actual conditions.

[0129] It should be noted that the application can be deployed on a device itself to realize embedded applications, or run on a PC terminal or other terminal with a user interface, thereby meeting various hardware environments and use requirements.

[0130] The above embodiments can be realized in whole or in part by software, hardware, firmware or any other combination. When realized by software, the above embodiments can be realized in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server or data center to another through wireless or wired transmission. The wired transmission includes optical fiber, twisted pair, coaxial cable, etc. The wireless transmission includes infrared, microwave, etc. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. containing one or more available medium collections. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state disk.

[0131] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system, device and module described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0132] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed modules can be indirect coupling or communication connection through some interfaces, devices or modules, and can be electrical, mechanical or other forms.

[0133] The modules described as separate components can or can not be physically separated, and the components shown as modules can or can not be physical modules, which can be located in one place or distributed on a plurality of network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0134] In addition, the functional modules in each embodiment of the present application can be integrated in one processing module, or each module can exist physically, or two or more modules can be integrated in one module.

[0135] If the functions are realized in the form of software function modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art or the part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions for making a computer device (which can be a personal computer, a server or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and various program codes that can store program codes.

[0136] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0137] Finally, the above merely provides the preferred embodiments of the present application, but is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A millimeter-wave radar fall detection method based on convolutional neural networks, characterized in that, include: S1. Acquire the raw echo signal from the millimeter-wave radar and generate a three-dimensional point cloud sequence through frequency-modulated continuous wave demodulation. S2. Perform multi-target clustering segmentation on the 3D point cloud sequence to separate independent target point cloud clusters, and extract the micro-motion time-frequency features of each target point cloud cluster; S3. For adjacent target point cloud clusters with a spatial distance less than a set threshold, simultaneously calculate the cross-correlation of their micro-motion time-frequency characteristics and the random walk diffusion coefficient of their Doppler phase signals, including: Calculate the Euclidean distance between the centroids of adjacent target point cloud clusters as the spatial distance, and filter out adjacent target point cloud clusters whose spatial distance is less than a set threshold. Obtain the in-phase and quadrature components in the original echo signals corresponding to the filtered adjacent target point cloud clusters; Calculate the Doppler phase signal of each adjacent target point cloud cluster based on in-phase and quadrature components; The Doppler phase signal is time-differenced to generate a phase difference sequence; The variance of the phase difference sequence is calculated as the random walk diffusion coefficient; Extract the time-domain waveforms of the respiratory fluctuation frequency component and the limb acceleration frequency band component from the micro-motion time-frequency features of adjacent target point cloud clusters; The cross-correlation coefficient between the time-domain waveforms of the respiratory fluctuation frequency component and the time-domain waveforms of the limb acceleration frequency band component is calculated through the following steps: Extract discrete sampling point sequences of the time-domain waveforms of respiratory fluctuation frequency components and limb acceleration frequency band components; Calculate the covariance between the discrete sampling point sequence of the time-domain waveform of the respiratory fluctuation frequency component and the discrete sampling point sequence of the time-domain waveform of the limb acceleration frequency band component; Calculate the standard deviation of the discrete sampling point sequence of the time-domain waveform of the respiratory fluctuation frequency component and the standard deviation of the discrete sampling point sequence of the time-domain waveform of the limb acceleration frequency band component; Divide the covariance value by the product of the standard deviation of the discrete sampling point sequence of the time-domain waveform of the respiratory fluctuation frequency component and the standard deviation of the discrete sampling point sequence of the time-domain waveform of the limb acceleration frequency band component, and generate the cross-correlation coefficient as the cross-correlation. S4. If the cross-correlation is lower than the separation threshold and the difference in random walk diffusion coefficient exceeds the physiological difference threshold, then they are determined to be targets from different organisms. S5. By analyzing the point cloud spatial density distribution of each independent biological target within the fall determination time window, the point cloud spatial density entropy change rate is generated. S6. When the rate of change of the spatial density entropy of the point cloud exceeds the critical value, the point cloud data of the current independent biological target is downweighted, and the spatiotemporal features of the downweighted point cloud are input into the convolutional neural network for fall action recognition.

2. The millimeter-wave radar fall detection method based on convolutional neural networks according to claim 1, characterized in that, The raw echo signal from the millimeter-wave radar is acquired, and a three-dimensional point cloud sequence is generated through frequency-modulated continuous wave demodulation, including: The raw echo signal from the millimeter-wave radar is mixed to generate an intermediate frequency signal; Range spectrum is generated by performing a range-dimensional fast Fourier transform on the intermediate frequency signal; Target range cells are extracted from the range spectrum using constant false alarm rate detection. The Doppler-dimensional fast Fourier transform is performed on the signal within the target range cell to generate the range-Doppler matrix; Target scattering points are extracted based on the amplitude peak of the range-Doppler matrix; The distance, Doppler velocity, and azimuth measurements of the target scattering point are converted into three-dimensional spatial coordinates to form a three-dimensional point cloud sequence.

3. The millimeter-wave radar fall detection method based on convolutional neural networks according to claim 1, characterized in that, Multi-target clustering is used to segment and separate independent target point cloud clusters from the 3D point cloud sequence, and the micro-motion time-frequency features of each target point cloud cluster are extracted, including: Calculate the Euclidean distance between points based on the spatial location of points in a 3D point cloud sequence; Density clustering of the 3D point cloud sequence is performed based on the Euclidean distance between points to generate an initial set of point cloud clusters. Independent target point cloud clusters are formed by associating the initial point cloud cluster set with the target centroid displacement between consecutive frames; The timing signal of each independent target point cloud cluster is bandpass filtered to separate the micro-motion signal; The micro-motion signal is transformed by time and frequency to generate micro-motion time and frequency characteristics that include respiratory fluctuation frequency components and limb acceleration frequency band components.

4. The millimeter-wave radar fall detection method based on convolutional neural networks according to claim 1, characterized in that, If the cross-correlation is below the separation threshold and the difference in random walk diffusion coefficients exceeds the physiological difference threshold, then they are determined to be targets from different organisms, including: Calculate the difference between the cross-correlation of adjacent target point cloud clusters and the separation threshold; Calculate the absolute difference in the random walk diffusion coefficients of adjacent target point cloud clusters; When the difference between the cross-correlation and the separation threshold is less than zero and the absolute difference of the random walk diffusion coefficient is greater than the physiological difference threshold, the determination results of different biological targets are generated. The determination result is associated with the corresponding adjacent target point cloud cluster.

5. The millimeter-wave radar fall detection method based on convolutional neural networks according to claim 1, characterized in that, By analyzing the point cloud spatial density distribution of each independent biological target within the fall determination time window, the rate of change of point cloud spatial density entropy is generated, including: Obtain the fall detection time window and divide the fall detection time window into multiple consecutive time sub-intervals; For point cloud data of independent biological targets within each time sub-interval, calculate the point cloud density distribution histogram in three-dimensional space; The proportion of the number of points in each spatial unit to the total number of points in the point cloud is calculated based on the point cloud density distribution histogram. The point cloud spatial density entropy value of the current time sub-interval is calculated using the Shannon entropy formula based on the ratio value. Calculate the difference between the spatial density entropy values ​​of the point cloud in adjacent time intervals; The rate of change of point cloud spatial density entropy is generated by dividing the average of the difference values ​​by the length of the time sub-interval.

6. The millimeter-wave radar fall detection method based on convolutional neural networks according to claim 1, characterized in that, When the rate of change of the spatial density entropy of the point cloud exceeds a critical value, the point cloud data of the current independent biological target is downweighted, and the spatiotemporal features of the downweighted point cloud are input into a convolutional neural network for fall action recognition, including: Compare the relationship between the rate of change of point cloud spatial density entropy and the critical value; When the rate of change of the spatial density entropy of the point cloud exceeds the critical value, the number of points in each spatial unit of the point cloud data of the current independent biological target is multiplied by the weighting factor. Generate spatiotemporal features of point clouds containing three-dimensional spatial coordinates and timestamps based on the reduced-weight point cloud data; The spatiotemporal features of point clouds are input into a convolutional neural network for fall recognition.

7. The millimeter-wave radar fall detection method based on convolutional neural networks according to claim 6, characterized in that, The process of inputting point cloud spatiotemporal features into a convolutional neural network for fall recognition is achieved through the following steps: The three-dimensional spatial coordinates in the spatiotemporal features of point clouds are mapped to a two-dimensional planar mesh to generate a spatial distribution matrix; The spatial distribution matrix is ​​superimposed with the timestamp sequence to form a three-dimensional spatiotemporal tensor; Local motion pattern features of spatiotemporal tensors are extracted using the convolutional layers of a convolutional neural network. Local motion pattern features are downsampled using pooling layers of a convolutional neural network; The fully connected layer of a convolutional neural network outputs the probability of a fall action classification.

8. A millimeter-wave radar fall detection system based on convolutional neural networks, used to implement the millimeter-wave radar fall detection method based on convolutional neural networks as described in any one of claims 1-7, characterized in that, include: The point cloud generation module is used to acquire the raw echo signal of the millimeter-wave radar and generate a three-dimensional point cloud sequence through frequency-modulated continuous wave demodulation. The feature extraction module is used to perform multi-target clustering segmentation on the 3D point cloud sequence to separate independent target point cloud clusters and extract the micro-motion time-frequency features of each target point cloud cluster. The time-frequency analysis module is used to simultaneously calculate the cross-correlation of the micro-motion time-frequency characteristics and the random walk diffusion coefficient of the Doppler phase signal for adjacent target point cloud clusters with a spatial distance of less than a set threshold. The target discrimination module is used to determine different biological targets if the cross-correlation is lower than the separation threshold and the difference in random walk diffusion coefficient exceeds the physiological difference threshold. The entropy analysis module is used to generate the rate of change of point cloud spatial density entropy by analyzing the point cloud spatial density distribution of each independent biological target within the fall determination time window. The fall detection module is used to reduce the weight of the point cloud data of the current independent biological target when the rate of change of the spatial density entropy value of the point cloud exceeds the critical value, and then input the spatiotemporal features of the reduced point cloud into the convolutional neural network for fall action recognition.

Images