Personnel tumble monitoring method and system based on millimeter wave radar and label fusion

Abstract

The invention relates to the technical field of millimeter-wave radar detection, in particular to a millimeter-wave radar and tag fusion-based personnel falling monitoring method and system, and the method comprises the steps: distributing a unique frequency channel for a tag of each patient, and embedding the basic information of the patient; radar echo signals of the patient wearing the tag are collected through a millimeter wave radar; separating the radar echo signal according to the frequency channel; constructing a time Doppler diagram, a distance angle diagram and a height trajectory diagram of each label, and inputting the time Doppler diagram, the distance angle diagram and the height trajectory diagram into a tumble classification model to obtain a tumble category of each label; when it is monitored that the patient falls down, the falling type and basic information of the patient are sent to medical staff. The tumble monitoring accuracy is improved, whether tumble is conscious tumble or unconscious tumble is further judged according to the tumble condition, medical staff can take correct medical protection measures in time according to the tumble type, and harm caused by tumble to a patient is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of millimeter-wave radar detection, and particularly to a method and system for monitoring personnel falling based on the fusion of millimeter-wave radar and tags. Background Art

[0002] Falling is one of the common safety incidents in hospitals, which may lead to a series of serious consequences, including fractures, skin lacerations, bleeding, and craniocerebral injuries, and may also cause complications in some cases. Especially for elderly patients and patients with limited mobility, they may lose consciousness after falling and are unable to call for help effectively, increasing the risk of death. Therefore, it is of great significance to detect and prevent patients from falling in real time in hospitals.

[0003] Using millimeter-wave radar for fall detection has the advantages of non-contact, good privacy, high precision, and the ability to achieve all-weather detection. In hospital wards, especially at night, traditional monitoring means such as video surveillance or manual patrol have many limitations, do not protect the privacy of patients, and cannot detect patients' falls in time, while millimeter-wave radar can make up for this deficiency.

[0004] However, the use of millimeter-wave radar for fall monitoring in hospital scenarios still has deficiencies. For example, it is not easy to identify the identity of the fallen patients. Identity recognition can help associate the medical records, health conditions, etc. of the fallen patients and provide corresponding detection or treatment plans in a timely manner. Millimeter-wave radar can simultaneously detect the distance and minute changes of targets, and can also distinguish multiple targets at different distances according to the different frequencies of the reflected beat frequency signals. However, when the distances of multiple targets are very close, the reflected signals may have a problem of phase overlap, making it very difficult to distinguish between targets and reducing the accuracy of fall monitoring results. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the problem that when the distances of multiple targets are very close in the prior art, the reflected signals of the millimeter-wave radar may have a phase overlap problem, resulting in great difficulty in distinguishing between targets and reducing the accuracy of fall monitoring results.

[0006] To solve the above technical problem, the present invention provides a method for monitoring personnel falling based on the fusion of millimeter-wave radar and tags, including:

[0007] Assign a unique frequency channel to the tag of each patient and embed the basic information of the patient;

[0008] Use the millimeter-wave radar to collect the radar echo signals of the patients wearing tags;

[0009] Separate the radar echo signals according to the frequency channels to obtain the signals of each tag;

[0010] Perform a Fourier transform on the signal of each tag in the fast time dimension to obtain the radial distance of the tag; perform a second Fourier transform on the signal after the first Fourier transform in the slow time dimension to obtain the radial velocity of the tag; use the MUSIC algorithm on the signal after the first Fourier transform to obtain the azimuth angle and elevation angle of the tag; calculate the height of the tag according to the elevation angle and radial distance of the tag;

[0011] Construct a time Doppler map for each tag with time and the radial velocity of each tag; construct a distance angle map for each tag with the radial distance and azimuth angle of each tag; construct a height trajectory map for each tag with time and the height of each tag;

[0012] Input the time Doppler map, distance angle map and height trajectory map of each tag into the fall classification model to obtain the fall category of each tag; the fall category includes no fall, conscious fall and unconscious fall;

[0013] When the fall category of the monitored tag is a conscious fall or an unconscious fall, send the fall category of the tag and the basic information of the patient to the medical staff.

[0014] Preferably, assign a unique frequency band to the tag of each patient, including: using frequency division multiplexing technology to assign a unique frequency channel to the tag of each patient, and setting a guard interval between the frequency channels of each tag; the tag uses an RFID passive tag.

[0015] Preferably, after using a millimeter-wave radar to collect the radar echo signal of the patient wearing the tag, use the background difference method to filter out the static clutter of the radar echo signal, including:

[0016] Collect the radar signal as the background signal B(t) in a target-free environment, and then collect the radar echo signal S(t) in real time. The formula is:

[0017] D(t) = |S(t) - B(t)|

[0018] Among them, D(t) represents the radar echo signal after filtering out the static clutter, and t represents the moment;

[0019] Use the multi-frame averaging method to calculate the background signal B(t). The formula is:

[0020]

[0021] Among them, B (i) (t) represents the i-th frame background signal, and K represents the total number of frames of the collected background signals.

[0022] Preferably, the signal of each tag is subjected to a Fourier transform in the fast time dimension to obtain the radial distance of the tag; the signal after the first Fourier transform is subjected to a second Fourier transform in the slow time dimension to obtain the radial velocity of the tag, including:

[0023] The signal of each tag is subjected to a Fourier transform in the fast time dimension to obtain the spectral peak position of the signal of the tag; the radial distance of the tag is calculated according to the spectral peak position, and the formula is:

[0024]

[0025] where r t is the radial distance of the tag at time t, c is the speed of light, f IF is the spectral peak position, and k is the signal frequency modulation rate;

[0026] The signal after the first Fourier transform is subjected to a second Fourier transform in the slow time dimension to obtain the Doppler frequency of the tag's velocity; the radial velocity of the tag is calculated according to the Doppler frequency, and the formula is:

[0027]

[0028] where v r represents the radial velocity of the tag at time t, f d represents the Doppler frequency, and λ represents the wavelength.

[0029] Preferably, the MUSIC algorithm is used for the signal after the first Fourier transform to obtain the azimuth angle and elevation angle of the tag, including:

[0030] Obtain the medical history information of the patient's basic information in the tag, and dynamically adjust the fast time window according to the medical history information;

[0031] Select a signal matrix from the signal after the first Fourier transform with the fast time window;

[0032] Calculate the adaptive weighted covariance matrix of the signal matrix according to the medical history information;

[0033] Perform eigenvalue decomposition on the adaptive weighted covariance matrix to obtain the noise subspace and the signal subspace;

[0034] Construct a MUSIC spectral function according to the noise subspace;

[0035] Calculate the azimuth angle and elevation angle of the tag using the MUSIC spectral function.

[0036] Preferably, obtain the medical history information of the patient's basic information in the tag, and dynamically adjust the fast time window according to the medical history information, and the formula is:

[0037] M s= M0×(1 + αC)

[0038] Wherein, M s represents the size of the fast time window, M0 represents the size of the initial fast time window, α represents the dynamic adjustment coefficient, and C represents the patient's disease level in the medical history information;

[0039] Select a signal matrix from the signal after a single Fourier transform with the fast time window;

[0040] Calculate the adaptive weighted covariance matrix of the signal matrix according to the medical history information. The formula is:

[0041]

[0042] Wherein, R x is the adaptive weighted covariance matrix, x i is the signal at the i-th fast time point in the signal matrix, represents the conjugate transpose of x i and ω i is the weighting factor of the signal at the i-th fast time point in the signal matrix;

[0043] ω i is expressed as:

[0044]

[0045] Wherein, γ(C) = e βC represents the weight function of the patient's disease level in the medical history information, C represents the patient's disease level in the medical history information, and β represents the adjustment coefficient; ||x i || 2 represents the energy of the signal x i at the i-th fast time point in the signal matrix, ||x k || 2 represents the energy of the signal x j at the j-th fast time point in the signal matrix;

[0046] Perform eigenvalue decomposition on the adaptive weighted covariance matrix to obtain the noise subspace and the signal subspace. The formula is:

[0047]

[0048] Wherein, R′ x represents the adaptive weighted covariance matrix after eigenvalue decomposition, U s and U n represent the signal subspace and the noise subspace respectively, represents the conjugate transpose of U s and represents the conjugate transpose of U n Λs Denotes a diagonal matrix with signal eigenvalues, Λ n Denotes a diagonal matrix.

[0049] Preferably, the height of the tag is calculated according to the pitch angle and radial distance of the tag, and the formula is:

[0050]

[0051] Where, H t Denotes the height of the tag at time t, H0 denotes the height of the millimeter-wave radar, r t Denotes the radial distance of the tag at time t, θ t The pitch angle of the tag at time t, δ t Denotes the adjustment factor at time t;

[0052] The formula for the adjustment factor at time t is:

[0053]

[0054] Where, δ0 denotes the base coefficient, SNR max Denotes the maximum signal-to-noise ratio, and SNR(t) denotes the signal-to-noise ratio at time t.

[0055] Preferably, the fall classification model includes three convolutional modules, a long short-term memory network, and a fully connected layer; each convolutional module includes two convolutional layers; the long short-term memory network includes two stacked long short-term memory layers, where the first layer of the long short-term memory layer includes 100 long short-term memory units, and the second layer of the long short-term memory layer includes 50 long short-term memory units.

[0056] Preferably, the time Doppler map, the range-angle map, and the height trajectory map are respectively input into the corresponding convolutional modules to obtain the time Doppler feature map, the range feature map, and the height feature map, and then the time Doppler feature map, the range feature map, and the height feature Figure 1 Are input into the long short-term memory network to obtain the fusion feature, and the fall category is output through the fully connected layer.

[0057] The present invention also provides a personnel fall monitoring system based on the fusion of millimeter-wave radar and tags, including:

[0058] A tag design module for assigning a unique frequency band to the tag of each patient and embedding the basic information of the patient;

[0059] A data acquisition module for using a millimeter-wave radar to acquire the radar echo signal of a patient wearing a tag;

[0060] A tag classification module for separating the radar echo signal according to the frequency band to obtain the signal of each tag;

[0061] A data processing module, which is used to perform a Fourier transform on the signal of each tag in the fast time dimension to obtain the radial distance of the tag; perform a second Fourier transform on the signal after the first Fourier transform in the slow time dimension to obtain the radial velocity of the tag; use the MUSIC algorithm on the signal after the first Fourier transform to obtain the azimuth angle and elevation angle of the tag; calculate the height of the tag according to the elevation angle and radial distance of the tag;

[0062] Construct a time Doppler map for each tag with time and the radial velocity of each tag; construct a distance angle map for each tag with the radial distance and azimuth angle of each tag; construct a height trajectory map for each tag with time and the height of each tag;

[0063] A fall classification module, which is used to input the time Doppler map, distance angle map and height trajectory map of each tag into a fall classification model to obtain the fall category of each tag; the fall category includes no fall, conscious fall and unconscious fall;

[0064] A monitoring module, which is used to send the fall category of the tag and the basic information of the patient to the medical staff when the fall category of the monitored tag is a conscious fall or an unconscious fall.

[0065] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0066] The method for monitoring personnel falls based on the fusion of millimeter-wave radar and tags according to the present invention assigns a unique frequency channel to the tags of each patient. In the case where multiple patients are relatively close, the identity and location of each patient can be accurately tracked, and corresponding fall monitoring judgments can be made; after collecting and separating the signals of each tag through a millimeter-wave radar, since there are obvious differences in the distance change, speed change and height change of the target movement between conscious falls and unconscious falls, the present invention obtains the distance, speed, azimuth angle and height information of the patient corresponding to each tag through data processing, and constructs a time Doppler map, a distance angle map and a height trajectory map of the patient corresponding to each tag, and inputs them into a fall classification model that fuses a convolutional neural network and a long short-term memory network to judge whether the patient corresponding to the tag has fallen and whether it is a conscious fall or an unconscious fall. When it is detected that the patient corresponding to the tag has fallen, the fall type of the fallen person and the basic information embedded in the tag are sent to the medical staff. The present invention not only improves the accuracy of fall monitoring, but also further judges whether the fall is a conscious fall or an unconscious fall, enabling medical staff to take correct medical protection measures in a timely manner according to the fall type, and reducing the harm caused by the fall to the patient. Description of the Drawings

[0067] To make the content of the present invention easier to be clearly understood, the following further details the present invention according to specific embodiments of the present invention in conjunction with the accompanying drawings, where:

[0068] Figure 1 is a flowchart of a method for monitoring personnel falling based on the fusion of millimeter-wave radar and tags of the present invention;

[0069] Figure 2 is a schematic diagram of tag separation, where Figure 2 in (1) is the linear frequency modulation pulse signal emitted by the millimeter-wave radar, Figure 2 in (2) is the frequency channel of the first tag, Figure 2 in (3) is the frequency channel of the second tag, Figure 2 in (4) is the frequency channel of the third tag;

[0070] Figure 3 is the structure diagram of the fall classification model;

[0071] Figure 4 is the structure diagram of a personnel fall monitoring system based on the fusion of millimeter-wave radar and tags. Specific Embodiments

[0072] The following further illustrates the present invention in conjunction with the accompanying drawings and specific embodiments, so that those skilled in the art can better understand the present invention and be able to implement it, but the exemplified embodiments are not intended to limit the present invention.

[0073] Embodiment 1

[0074] Referring to Figure 1 as shown, the present invention provides a method for monitoring personnel falling based on the fusion of millimeter-wave radar and tags, including:

[0075] S1: Assign a unique frequency channel to the tag of each patient and embed the basic information of the patient.

[0076] This embodiment uses RFID passive tags.

[0077] RFID passive tags do not have built-in batteries and can obtain energy through the radio waves emitted by the millimeter-wave radar to make the chip work and return the data stored in the tag. Combining the tags with the millimeter-wave radar to monitor the fall status of patients can not only distinguish the fall status of multiple targets at relatively close distances, but also achieve stable tracking of multiple targets in a dynamic scenario while reducing the algorithm complexity.

[0078] Due to the different passband characteristics of different tags, in this embodiment, the Frequency Division Multiplexing (FDM) technology is utilized to allocate a unique frequency channel for each patient's tag, ensuring that there is no frequency overlap when each tag reflects signals, thereby avoiding mutual interference between signals. The millimeter-wave radar emits signals. When different tags receive the signals, they reflect the signals back, and each tag will reflect different frequency signals. The millimeter-wave radar receives the echo signals, which contain the frequency information of each tag. Then, by using the FDM technology to process these signals, the echo signals can be distinguished, realizing independent monitoring of multiple targets and improving the measurement accuracy and robustness.

[0079] Meanwhile, the basic information of the patient, including name, gender, medical history, etc., is embedded in each tag. When the patient falls, an alarm can be sent to the relevant doctors and nurses in a timely manner based on the patient's basic information.

[0080] S2: Use the millimeter-wave radar to collect the radar echo signals of the patients wearing tags.

[0081] Preferably, the millimeter-wave radar is placed in the middle position of the wall to ensure full coverage and efficient monitoring of the patients, and the RFID passive tag is embedded in the back of the hospital gown.

[0082] Preferably, in this embodiment, a millimeter-wave radar above 77 GHz is used to emit microwave signals to the living body.

[0083] After the radar echo signals are collected, preprocessing is performed on them, including:

[0084] The background difference method is used to filter out the static clutter of the radar echo signals. The radar signal is collected as the background signal B(t) in a target-free environment, and then the radar echo signal S(t) is collected in real time. The formula is:

[0085] D(t) = |S(t) - B(t)|

[0086] where D(t) represents the radar echo signal after filtering out the static clutter, and t represents the time.

[0087] In order to improve the stability of the background signal, the multi-frame averaging method is used to calculate the background signal B(t), including:

[0088] Collect K frames of background signals B (i) (t) (i = 1, 2,..., K) in a target-free environment, and calculate the average background signal. The formula is:

[0089]

[0090] where B (i)(t) represents the background signal of the i-th frame, and K represents the total number of frames of the collected background signals.

[0091] The multi-frame averaging method can effectively reduce the noise in the background signal. After removing the static clutter, it can better analyze the patient's fall situation.

[0092] S3: Separate the radar echo signals according to the frequency channels to obtain the signals of each tag.

[0093] Refer to Figure 2 as shown, Figure 2 in (1) is the linear frequency modulation pulse signal emitted by the millimeter-wave radar, Figure 2 in (2) is the frequency channel of the first tag, Figure 2 in (3) is the frequency channel of the second tag, Figure 2 in (4) is the frequency channel of the third tag. Among them, the first tag works in f1~f2, the second tag works in f3~f4, the third tag works in f5~f6, and a guard interval is set between the three frequency bands to further reduce crosstalk. The chirp times corresponding to each tag are t1~t2, t3~t4, and t5~t6 respectively. Capture the signals reflected by all tags through the millimeter-wave radar, separate them using a band-pass filter according to the frequency channels of the tags, extract the signals of the first tag, the second tag, and the third tag respectively, and then demodulate the signals of each tag to extract the signals carried by them.

[0094] S4: Construct the time-Doppler diagram, range-angle diagram, and height trajectory diagram of each tag.

[0095] Falls in reality can be divided into conscious falls and unconscious falls. A conscious fall means that the patient completely loses balance and falls to the ground at a high speed consciously; an unconscious fall means that the patient loses consciousness and slowly slides or collapses to the ground. General detections can detect conscious falls, but may miss unconscious falls, which may bring serious consequences. Therefore, by distinguishing the two types of falls, medical staff can also determine the cause of the fall and take correct measures in a timely manner.

[0096] In a conscious fall, the speed first accelerates rapidly, and after the target lands, the speed quickly decelerates. The exercise intensity of a conscious fall is the strongest. In an unconscious fall, the speed is slow and non-linearly accelerating, and after the target lands, the speed also quickly slows down. The intensity of an unconscious fall is moderate. At the same time, during conscious and unconscious falls, the height changes are significantly different. The height change trend of a conscious fall is relatively rapid compared to that of an unconscious fall.

[0097] The peak speed of a falling motion is often higher than that of most non - falling motions. The intensity of non - falling motions is weaker than that of falling motions. Non - falling motions usually stop at a greater distance from the floor compared to falling motions, and falling motions generally end up on the ground, thus distinguishing between falling motions and non - falling motions.

[0098] Based on the characteristics of non - falling motions, conscious falls, and unconscious falls, the present invention uses the time - Doppler diagram, distance - angle diagram, and height - trajectory diagram of each tag to distinguish the fall categories. The specific data - processing process is as follows:

[0099] Perform a Fourier transform on the signal of each tag in the fast - time dimension to obtain the radial distance of the tag; perform a second - order Fourier transform on the signal after the first - order Fourier transform in the slow - time dimension to obtain the radial velocity of the tag; use the MUSIC algorithm on the signal after the first - order Fourier transform to obtain the azimuth angle and elevation angle of the tag; calculate the height of the tag based on the elevation angle and radial distance of the tag.

[0100] The data after radar demodulation is usually stored in a three - dimensional matrix (m, n, k), where m is the fast - time dimension, n is the slow - time dimension, and k is the antenna dimension, that is, different receiving channels. The fast - time dimension and the slow - time dimension are two important dimensions in radar signal processing, corresponding to distance information and Doppler information respectively. The fast - time samples correspond to the number of ADC samples of the receiving antenna within a chirp. The slow - time samples correspond to the number of chirps received by a single antenna within a frame (a frame consists of multiple chirps).

[0101] Construct a matrix with the fast - time dimension and the slow - time dimension, with a size of N t ×M t ,N t is the number of slow - time samples, and M t is the number of fast - time samples. Perform a first - order Fourier transform on the signal in the fast - time dimension to transform the signal from the time domain to the frequency domain and obtain the radial distance of the tag.

[0102] When the tag is stationary, the intermediate - frequency signal is:

[0103]

[0104] where k is the signal chirp rate, r t is the radial distance, c is the speed of light, and t is the time.

[0105] Perform a first - order Fourier transform on the signal of each tag in the fast - time dimension to obtain the spectral peak position of the signal of the tag:

[0106]

[0107] Calculate the radial distance of the tag according to the spectral peak position, and the formula is:

[0108]

[0109] where r t is the radial distance at time t of the tag, c is the speed of light, and f IF is the spectral peak position, and k is the signal chirp rate.

[0110] When the tag moves, the spectral peak position becomes:

[0111]

[0112] where f c is the signal carrier frequency, and v r is the radial velocity of the target.

[0113] The signal frequency contains both range and velocity information and cannot be directly obtained by a single Fourier transform. When the tag moves, the Doppler shift causes the frequency of the echo signal to shift. Therefore, when calculating the radial velocity of the tag, the Doppler shift needs to be added to the intermediate-frequency signal. Let the signal sampling frequency be T s , the number of samples per pulse be N, and M pulses be received. The intermediate-frequency signal becomes:

[0114]

[0115] where n = 0, 1, 2,..., N - 1 represents the sampling sequence of a single pulse; m = 0, 1, 2,..., M - 1 represents the pulse sequence.

[0116] The phase part of the signal contains velocity information. Performing a second Fourier transform on the signal after the first Fourier transform in the slow-time dimension can obtain the Doppler frequency of the target velocity:

[0117]

[0118] Therefore, the radial velocity of the tag is:

[0119]

[0120] where v r represents the radial velocity of the tag at time t, f d represents the Doppler frequency, and λ represents the wavelength.

[0121] Different patients may have different motion patterns and health conditions, and the fall patterns of different patients vary greatly. A fixed window may lead to false detections or missed detections. By adjusting the size of the fast time window according to the patient's medical history information, signals can be processed more precisely in a noisy environment, and signal capture can be optimized according to the patient's motion pattern, making it more targeted. For patients with severe conditions, using a longer fast time window can improve the signal-to-noise ratio and reduce noise interference. For patients whose conditions do not affect their movement, using a short fast time window can better capture the rapidly changing motion characteristics.

[0122] Therefore, the present invention dynamically adjusts the fast time window according to the patient's medical history recorded by the tag. The medical history information recorded in the tag includes the patient's disease level C. In this embodiment, 4 disease levels are set, 1 for healthy, 2 for mild disease, 3 for moderate disease, and 4 for severe disease.

[0123] Obtain the medical history information of the patient's basic information in the tag, and dynamically adjust the fast time window according to the medical history information. The formula is:

[0124] M s = M0×(1 + αC)

[0125] Where, M s represents the size of the fast time window, M0 represents the initial size of the fast time window, α represents the dynamic adjustment coefficient, and C represents the patient's disease level in the medical history information. The greater the disease level, the more severe the condition, and the larger the window, which can improve the signal smoothness and reduce the noise ratio.

[0126] Select a signal matrix from the signal after a single Fourier transform with the fast time window

[0127] Calculate the adaptive weighted covariance matrix of the signal matrix according to the medical history information. The formula is:

[0128]

[0129] Where, R x is the adaptive weighted covariance matrix, x i is the signal at the i-th fast time point in the signal matrix, represents the conjugate transpose of x i , and ω i is the weighting factor of the signal at the i-th fast time point in the signal matrix.

[0130] ω i is expressed as:

[0131]

[0132] Where, γ(C) = eβC A weight function representing the disease severity level of a patient in the medical history information, C represents the disease severity level of the patient in the medical history information, and β represents an adjustment coefficient; ||x i || 2 Represents the energy of the signal x at the i-th fast time point in the signal matrix, ||x i || k || 2 Represents the energy of the signal x at the j-th fast time point in the signal matrix. j

[0133] Capturing the correlation and noise characteristics of the signal through a time window and performing eigen-decomposition on the adaptive weighted covariance matrix is beneficial for separating the target signal from the noise to obtain the noise subspace and the signal subspace. The formula is:

[0134]

[0135] Among them, R′ x Represents the adaptive weighted covariance matrix after eigen-decomposition, U s And U n Represent the signal subspace and the noise subspace respectively, Represents the conjugate transpose of U s Represents the conjugate transpose of U n Λ s Represents a diagonal matrix with signal eigenvalues, Λ n Represents a diagonal matrix.

[0136] Construct the MUSIC spectral function based on the noise subspace. Taking the elevation angle as an example, the constructed MUSIC spectral function of the elevation angle is: [[ID=4s]]

[0137]

[0138] Among them, Is the steering vector, Is The conjugate transpose of; when The direction is consistent with the signal source direction, Will cause the spectral function to have a peak, and this peak corresponds to the elevation angle

[0139] Calculate the elevation angle of this tag using the MUSIC spectral function of the elevation angle.

[0140] Similarly, by constructing the MUSIC spectral function of the azimuth angle, the azimuth angle of this tag can be calculated. Among them, the received antenna data arranged horizontally is used to measure the azimuth angle, and the received antenna data in the vertical direction is used to measure the elevation angle.

[0141] ​​Combined with trigonometric functions, calculate the height of the tag according to the pitch angle and radial distance of the tag. The formula is:

[0142]

[0143] Among them, H t represents the height of the tag at time t, H0 represents the height of the millimeter-wave radar, r t represents the radial distance of the tag at time t, θ t is the pitch angle of the tag at time t, and δ t represents the adjustment factor at time t, which can adjust the flexibility of the trigonometric function and avoid overfitting when the signal is weak and the noise is large.

[0144] The formula for the adjustment factor at time t is:

[0145]

[0146] Among them, δ0 represents the base coefficient, SNR max represents the maximum signal-to-noise ratio, and SNR(t) represents the signal-to-noise ratio at time t.

[0147] Construct a time Doppler map for each tag based on time and the radial velocity of each tag; construct a distance-angle map for each tag based on the radial distance and azimuth angle of each tag; construct a height trajectory map for each tag based on time and the height of each tag.

[0148] S5: Input the time Doppler map, distance-angle map, and height trajectory map of each tag into the fall classification model to obtain the fall category of each tag.

[0149] Convolutional neural network (CNN) is a feedforward neural network composed of an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer, which can learn effective features from raw data. Long short-term memory network (LSTM) is a time recurrent network. Compared with traditional recurrent neural networks, it solves the long-term dependence problem and avoids the problem of gradient disappearance. The CNN+LSTM neural network combines CNN and LSTM, utilizes the advantages of CNN in spatial feature extraction and the advantages of LSTM in time series modeling. The CNN+LSTM model can automatically extract features and learn time series patterns for classification, and effectively identify and classify human fall situations.

[0150] Refer to Figure 3As shown in the figure, the fall classification model includes three convolutional modules, a long short-term memory network, and a fully connected layer. Each convolutional module includes two convolutional layers, and each convolutional layer uses the RELU activation function for normalization and correction to enhance the non-linear expression ability. Finally, the features are mapped to low-dimensional feature vectors through the fully connected layer, and "0" padding is used to ensure that information at the image edge is not lost. The long short-term memory network includes two stacked long short-term memory layers. The first long short-term memory layer includes 100 long short-term memory units, and the second long short-term memory layer includes 50 long short-term memory units. The long short-term memory layer can capture the dynamic changes of the human body posture. The reduction of long short-term memory units between the two long short-term memory layers can effectively reduce the overfitting of the model.

[0151] The processed time Doppler map, distance-angle map, and height trajectory map of the unified size are respectively input into the corresponding convolutional modules to process the spatial features of each frame of the extracted data, obtaining a time Doppler feature map, a distance feature map, and a height feature map; then the time Doppler feature map, the distance feature map, and the height feature map are input into the long short-term memory network together according to the frame sequence to obtain a fusion feature. The fusion feature undergoes feature mapping through the fully connected layer and combines with the Softmax activation function to classify the fall situation and output the fall category. The fall category includes no fall, conscious fall, and unconscious fall.

[0152] S6: When the fall category of the monitoring label is a conscious fall or an unconscious fall, an alarm is issued, and the fall category of the label and the basic information of the patient are sent to the medical staff.

[0153] In summary, for the method for monitoring personnel fall based on the fusion of millimeter-wave radar and label of the present invention, a unique frequency channel is assigned to the label of each patient. In the case where multiple patients are relatively close to each other, the identity and position of each patient can be accurately tracked to perform corresponding fall monitoring and judgment; the signals of each label are collected and separated by the millimeter-wave radar, and the distance, speed, azimuth angle, and height information of the patient corresponding to each label are obtained through data processing. Based on this, a time Doppler map, a distance-angle map, and a height trajectory map of the patient corresponding to each label are constructed and input into the fall classification model that fuses a convolutional neural network and a long short-term memory network to determine whether the patient corresponding to the label has fallen and whether it is a conscious fall or an unconscious fall. When it is detected that the patient corresponding to the label has fallen, the fall type of the fallen person and the basic information of the embedded label are sent to the medical staff. The present invention not only improves the accuracy of fall monitoring, but also further determines whether the fall situation is a conscious fall or an unconscious fall, enabling medical staff to take correct medical protection measures in a timely manner according to the fall type and reducing the harm caused by the fall to the patient.

[0154] Embodiment 2

[0155] Reference Figure 4 As shown, based on the method for monitoring human falls by fusing millimeter-wave radar and tags described in the first embodiment, this embodiment provides a system for monitoring human falls by fusing millimeter-wave radar and tags, including:

[0156] A tag design module, which is used to assign a unique frequency band to the tag of each patient and embed the basic information of the patient;

[0157] A data acquisition module, which is used to collect the radar echo signals of the patients wearing tags by using millimeter-wave radar;

[0158] A tag classification module, which is used to separate the radar echo signals according to the frequency band to obtain the signals of each tag;

[0159] A data processing module, which is used to perform a Fourier transform on the signals of each tag in the fast time dimension to obtain the radial distance of the tag; perform a second Fourier transform on the signals after the first Fourier transform in the slow time dimension to obtain the radial velocity of the tag; use the MUSIC algorithm for the signals after the first Fourier transform to obtain the azimuth angle and elevation angle of the tag; calculate the height of the tag according to the elevation angle and radial distance of the tag;

[0160] Construct a time-Doppler diagram for each tag with time and the radial velocity of each tag; construct a distance-angle diagram for each tag with the radial distance and azimuth angle of each tag; construct a height trajectory diagram for each tag with time and the height of each tag;

[0161] A fall classification module, which is used to input the time-Doppler diagram, distance-angle diagram, and height trajectory diagram of each tag into a fall classification model to obtain the fall category of each tag; the fall category includes no fall, conscious fall, and unconscious fall;

[0162] A monitoring module, which is used to send the fall category of the tag and the basic information of the patient to the medical staff when the fall category of the monitored tag is a conscious fall or an unconscious fall.

[0163] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0164] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices produce means for implementing the functions specified in the flow Figure 1 one or more flows and / or blocks Figure 1 or means for implementing the functions specified in a block or multiple blocks.

[0165] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including instruction means for implementing the functions specified in the flow Figure 1 one or more flows and / or blocks Figure 1 or a block or multiple blocks.

[0166] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operational steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the flow Figure 1 one or more flows and / or blocks Figure 1 or a block or multiple blocks.

[0167] Obviously, the above embodiments are merely examples for clear illustration and are not limitations on the implementation manners. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation manners here. And the obvious changes or modifications derived therefrom are still within the protection scope of the present invention.

Claims

1. A method for monitoring human falls based on the fusion of millimeter-wave radar and tags, characterized in that, Including: Assign a unique frequency channel to the tag of each patient and embed the basic information of the patient; Use a millimeter-wave radar to collect the radar echo signal of the patient wearing the tag; Separate the radar echo signal according to the frequency channel to obtain the signal of each tag; Perform a Fourier transform on the signal of each tag in the fast time dimension to obtain the radial distance of the tag; perform a second Fourier transform on the signal after the first Fourier transform in the slow time dimension to obtain the radial velocity of the tag; use the MUSIC algorithm on the signal after the first Fourier transform to obtain the azimuth angle and elevation angle of the tag; calculate the height of the tag according to the elevation angle and radial distance of the tag; Construct a time-Doppler map for each tag with time and the radial velocity of each tag; construct a distance-angle map for each tag with the radial distance and azimuth angle of each tag; construct a height trajectory map for each tag with time and the height of each tag; Input the time-Doppler map, distance-angle map, and height trajectory map of each tag into a fall classification model to obtain the fall category of each tag; the fall category includes no fall, conscious fall, and unconscious fall; When the fall category of the monitored tag is a conscious fall or an unconscious fall, send the fall category of the tag and the basic information of the patient to the medical staff.

2. The method for monitoring human falls based on the fusion of millimeter-wave radar and tags according to claim 1, characterized in that, Assign a unique frequency band to the tag of each patient, including: using frequency division multiplexing technology to assign a unique frequency channel to the tag of each patient and setting a guard interval between the frequency channels of each tag; the tag uses an RFID passive tag.

3. The method for monitoring human falls based on the fusion of millimeter-wave radar and tags according to claim 1, characterized in that, After using a millimeter-wave radar to collect the radar echo signal of the patient wearing the tag, use background difference method to filter out the static clutter of the radar echo signal, including: Collect the radar signal as the background signal B(t) in a target-free environment, and then collect the radar echo signal S(t) in real time. The formula is: D(t) = |S(t) - B(t)| Where D(t) represents the radar echo signal after filtering out the static clutter, and t represents the time; Use the multi-frame averaging method to calculate the background signal B(t). The formula is: Among them, B (i) (t) represents the background signal of the i-th frame, and K represents the total number of frames of the collected background signals.

4. The personnel fall monitoring method based on the fusion of millimeter-wave radar and tag according to claim 1, characterized in that Perform a Fourier transform on the signal of each tag in the fast time dimension to obtain the radial distance of the tag; perform a second Fourier transform on the signal after the first Fourier transform in the slow time dimension to obtain the radial velocity of the tag, including: Perform a Fourier transform on the signal of each tag in the fast time dimension to obtain the spectral peak position of the signal of the tag; calculate the radial distance of the tag according to the spectral peak position. The formula is: Among them, r t is the radial distance at time t of the label, c is the speed of light, f IF is the spectral peak position, and k is the signal frequency modulation rate; Perform a second Fourier transform on the signal after the first Fourier transform in the slow time dimension to obtain the Doppler frequency of the tag's velocity; calculate the radial velocity of the tag according to the Doppler frequency. The formula is: Among them, v r represents the radial velocity at time t of the tag, and f d represents the Doppler frequency, and λ represents the wavelength.

5. A method for monitoring personnel falls based on the fusion of millimeter-wave radar and tags according to claim 1, characterized in that, Use the MUSIC algorithm on the signal after the first Fourier transform to obtain the azimuth angle and elevation angle of the tag, including: Obtain the medical history information of the basic information of the patient in the tag, and dynamically adjust the fast time window according to the medical history information; Select a signal matrix from the signal after the first Fourier transform with the fast time window; Calculate the adaptive weighted covariance matrix of the signal matrix according to the medical history information; Perform eigen-decomposition on the adaptive weighted covariance matrix to obtain the noise subspace and the signal subspace; Construct the MUSIC spectral function based on the noise subspace; Use the MUSIC spectral function to calculate the azimuth angle and elevation angle of the tag.

6. The method for monitoring human falls based on the fusion of millimeter-wave radar and tags according to claim 5, characterized in that, Obtain the medical history information of the patient's basic information in the tag, and dynamically adjust the fast time window according to the medical history information. The formula is: M s = M0 × (1 + αC) Among them, M s represents the size of the fast time window, M0 represents the size of the initial fast time window, α represents the dynamic adjustment coefficient, and C represents the patient's disease level in the medical history information; Select the signal matrix from the signal after one Fourier transform with the fast time window; Calculate the adaptive weighted covariance matrix of the signal matrix according to the medical history information. The formula is: Among them, R x is an adaptive weighted covariance matrix, x i is the signal at the i-th fast time point in the signal matrix, denotes the conjugate transpose of x i , ω i is the weighting factor of the signal at the i-th fast time point in the signal matrix; ω i is expressed as: where γ(C) = e βC is a weight function representing the disease severity level of the patient in the medical history information, C represents the disease severity level of the patient in the medical history information, and β represents an adjustment coefficient; ||x i || 2 represents the energy of the signal x i at the i-th fast time point in the signal matrix, ||x k || 2 represents the energy of the signal x j at the j-th fast time point in the signal matrix; Perform eigen-decomposition on the adaptive weighted covariance matrix to obtain the noise subspace and the signal subspace, The formula is: Among them, R′ x represents the adaptive weighted covariance matrix after eigenvalue decomposition, U s and U n represent the signal subspace and the noise subspace respectively, represents the conjugate transpose of U s , represents the conjugate transpose of U n , Λ s represents a diagonal matrix with signal eigenvalues, Λ n represents a diagonal matrix.

7. A method for monitoring human falls based on the fusion of millimeter-wave radar and tags according to claim 1, characterized in that, Calculate the height of the tag according to the elevation angle and radial distance of the tag. The formula is: Among them, H t represents the height at time t of the tag, H0 represents the height of the millimeter-wave radar, r t represents the radial distance at time t of the tag, θ t is the pitch angle at time t of the tag, δ t represents the adjustment factor at time t; The formula for the adjustment factor at time t is: Among them, δ0 represents the base coefficient, SNR max represents the maximum signal-to-noise ratio, and SNR(t) represents the signal-to-noise ratio at time t.

8. A method for monitoring human falls based on the fusion of millimeter-wave radar and tags according to claim 1, characterized in that, The fall classification model includes three convolutional modules, a long short-term memory network, and a fully connected layer; each convolutional module includes two convolutional layers; the long short-term memory network includes two stacked long short-term memory layers, where the first long short-term memory layer includes 100 long short-term memory units, and the second long short-term memory layer includes 50 long short-term memory units.

9. A method for monitoring human falls based on the fusion of millimeter-wave radar and tags according to claim 8, characterized in that, Input the time Doppler map, distance angle map, and height trajectory map into the corresponding convolutional modules respectively to obtain the time Doppler feature map, distance feature map, and height feature map, and then input the time Doppler feature map, distance feature map, and height feature map into the long short-term memory network together to obtain the fused feature, The fused feature passes through the fully connected layer to output the fall category.

10. A personnel fall monitoring system based on the fusion of millimeter-wave radar and tags, characterized in that, Including: A tag design module for assigning a unique frequency band to the tag of each patient and embedding the basic information of the patient; A data acquisition module for using a millimeter-wave radar to collect the radar echo signal of the patient wearing the tag; A tag classification module for separating the radar echo signal according to the frequency band to obtain the signal of each tag; A data processing module for performing one Fourier transform on the signal of each tag in the fast time dimension, to obtain the radial distance of the tag; perform a second Fourier transform on the signal after the first Fourier transform in the slow time dimension to obtain the radial velocity of the tag; use the MUSIC algorithm for the signal after the first Fourier transform to obtain the azimuth angle and elevation angle of the tag; calculate the height of the tag according to the elevation angle and radial distance of the tag; Construct the time Doppler map of each tag with time and the radial velocity of each tag; construct the distance angle map of each tag with the radial distance and azimuth angle of each tag; construct the height trajectory map of each tag with time and the height of each tag; A fall classification module for inputting the time Doppler map, distance angle map, and height trajectory map of each tag into the fall classification model to obtain the fall category of each tag; the fall category includes no fall, conscious fall, and unconscious fall; A monitoring module for sending the fall category of the tag and the basic information of the patient to the medical staff when the fall category of the monitored tag is a conscious fall or an unconscious fall.

Images