Millimeter wave radar vital sign monitoring method and system based on adaptive thoracic cavity positioning

Abstract

The invention discloses a millimeter wave radar vital sign monitoring method and a millimeter wave radar vital sign monitoring system based on self-adaptive thoracic cavity positioning, and the method comprises the steps: firstly, collecting a sleep posture image of a target human body; operations such as image preprocessing, attitude skeleton key point estimation, SVM attitude classification and mechanical arm control are carried out to enable the radar to directly face the chest and back of a target human body, then radar echo data are acquired, direct current clutter suppression, constant false alarm detection and pulse compression are carried out to realize data preprocessing, and then the phase at a target range gate is extracted; and sending the vital sign data to a filter to obtain target vital sign information, and finally realizing apnea detection and recording of the target human body according to the vital sign data. According to the method, the posture of the user can be accurately recognized, the mechanical arm can be controlled, and the accuracy and generalization of the millimeter wave radar in the sleep monitoring field are remarkably improved through the method.

Description

Technical Field

[0001] The present invention relates to the field of millimeter wave radar health monitoring, and in particular to a millimeter wave radar vital sign monitoring method and system based on adaptive chest positioning. Background Art

[0002] Vital sign signal detection has been a research hotspot in recent years. Traditional contact-based life detection instruments such as electrocardiograms and smart bracelets are not suitable for certain special scenarios, such as detecting the vital sign information of burn patients and people trapped in disasters. In addition, in the scenario of sleep monitoring, patients with sleep disorders also need to wear a large number of instruments such as electrocardiograms and respiratory detection instruments, which greatly affects their comfort.

[0003] Millimeter-wave radar can measure the distance and speed information of the target, has high sensitivity and anti-interference ability, and has broad application prospects in the field of non-contact vital signs detection. In addition, radar-based sensing technology is less affected by environmental conditions such as light, temperature, and humidity, and there is no risk of privacy leakage. At the same time, it does not need to directly contact the object being measured, which can free the object being measured from the constraints of electronic instruments and equipment. It is an ideal monitoring method. However, when traditional millimeter-wave radar vital signs monitoring is applied to sleep monitoring scenarios, it is easily affected by the patient's sleeping position changes and cannot continuously and effectively monitor the patient. This is also a major obstacle to the promotion of millimeter-wave radar sleep monitoring. Therefore, a combined method based on adaptive chest positioning came into being.

[0004] In radar detection, compared with traditional microwave frequency radar, millimeter wave radar has the natural properties of small wavelength and large bandwidth, which makes it easier to achieve system miniaturization design and high-precision measurement of targets. At the same time, it has the common advantages of radar, such as all-weather and all-day environmental adaptability, non-contact detection, and non-privacy infringement.

[0005] With the help of the good detection performance of millimeter-wave radar, this method focuses on the demand for non-contact vital signs monitoring in the field of smart health care, conducts research on monitoring technology based on millimeter-wave radar, breaks through the key technologies of human vital signs monitoring in various sleep scenarios, and meets the urgent needs of application fields such as smart medical care, smart elderly care, and smart home. Summary of the invention

[0006] In view of this, the purpose of the present invention is to provide a millimeter-wave radar vital signs monitoring method based on adaptive chest positioning. The system utilizes millimeter-wave radar combined with a robotic arm to achieve real-time monitoring of vital signs in a sleeping scene.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] The millimeter wave radar vital sign monitoring method based on adaptive chest positioning provided by the present invention comprises the following steps:

[0009] S1: Collect human sleeping posture images;

[0010] S2: Estimate the human body posture and obtain the skeleton point estimation data;

[0011] S3: Classify the human skeleton points to obtain posture classification results, and control the robotic arm carrying the radar to move to the best monitoring position corresponding to the current sleeping posture;

[0012] S4: Acquire human body radar echo data;

[0013] S5: Process the radar echo data and extract the phase of the echo data;

[0014] S6: Process the phase of the echo data, analyze and obtain the target human body's breathing and heartbeat signals, and determine the number of apnea times based on the breathing and heartbeat data.

[0015] Further, the method for estimating the sleeping posture in S2 is performed according to the following steps:

[0016] The openpose model is used for pose estimation. First, the entire image is used as the input of the network to predict a set of two-dimensional confidence maps S of body part positions and a set of two-dimensional vector fields L of part affinity fields (PAFs). Then, the confidence map and PAF are parsed through greedy reasoning, and finally the whole is assembled to output the key points and skeleton map of the human body target.

[0017] Further, the method for classifying postures in S3 is performed according to the following steps:

[0018] The estimated skeleton data is cleaned to remove the key points of the human foot skeleton, and then the skeleton key points are sent to the trained SVM classifier to output the posture classification results.

[0019] Further, the method for controlling the robot arm in S3 is performed according to the following steps:

[0020] After obtaining the posture classification results, the host computer connects to the robotic arm through the local area network and outputs the classification results to the Raspberry Pi on the robotic arm end. After receiving the signal, the Raspberry Pi controls the robotic arm carrying the radar to rotate to the corresponding preset position according to the classification results.

[0021] Further, the method for extracting the phase in S5 is performed according to the following steps:

[0022] S51: Acquire radar echo data, and process the radar data into a data cube of sampling point dimension*chirp dimension*antenna dimension;

[0023] S52: performing fast Fourier transform on the ADC original complex signal of the data cube in the distance dimension to obtain a spectrum in the distance dimension, and performing one-dimensional CFAR processing and DC clutter filtering on the distance dimension spectrum to suppress interference signals;

[0024] S53: dynamically search for the maximum value in the distance spectrum at intervals of 5 seconds within the human target distance range to obtain the distance gate corresponding to the target human body;

[0025] S54: extracting the target phase information once in a single frame at the range gate, then performing phase unwrapping, and then removing the impulse signal to obtain a phase signal corresponding to the target human body.

[0026] Further: the method for analyzing the respiratory heartbeat signal in S6 is performed according to the following steps:

[0027] S61: performing phase difference calculation between adjacent frames on the unfolded phase signal to obtain a phase difference signal;

[0028] S62: Since there is a direct connection between the differential signal and the chest rise and fall signal, the phase differential signal is sent to a filter to obtain a breathing and heartbeat signal;

[0029] S63: performing frequency spectrum estimation on the breathing and heartbeat signals respectively to obtain the breathing and heartbeat frequencies;

[0030] S64: The breathing and heartbeat data are saved, and the energy of the breathing signal is monitored in real time. When the energy of the breathing signal is lower than the set threshold for 10 seconds, it is determined as a respiratory arrest.

[0031] Further, the relationship between the differential signal and the chest rise and fall signal comprises the following steps:

[0032] The phase difference signal and the distance change caused by the chest rise and fall have a relationship as indicated by the following formula:

[0033]

[0034] Among them, Δφ represents the phase difference between two adjacent frames, ΔR represents the distance change from the radar caused by the chest rise and fall, and λ c Indicates the wavelength of the RF signal emitted by the radar.

[0035] Further, the impulse signal removal is specifically performed according to the following steps:

[0036] The signal strength value of the current phase point is subtracted from the signal strength values ​​of the two adjacent phase points, and compared with the set threshold value. If it exceeds the threshold, the signal strength value of the current phase point is reset to the average of the signal strengths of the adjacent phase points.

[0037] Further, the SVM classifier training is specifically performed according to the following steps:

[0038] After the training parameters of the SVM classifier are configured, the obtained sleeping posture skeleton data is normalized and sent to the SVM classifier for training to obtain dedicated classifiers for eight sleeping postures.

[0039] The millimeter wave radar vital signs monitoring system based on adaptive chest positioning provided by the present invention includes a sleep posture monitoring module, a vital signs monitoring module, a sleep posture classification module, a mechanical arm control module, and a vital signs monitoring module:

[0040] The sleeping posture monitoring module is used to collect the sleeping posture image of the target;

[0041] The vital signs monitoring module is used to obtain the target echo data and organize it into a radar cube, monitor the target's breathing and heartbeat status through the radar echo data, and warn and record the occurrence of apnea;

[0042] The sleeping posture classification module is used to estimate the sleeping posture skeleton and classify the sleeping posture;

[0043] The robotic arm control module is used to control the robotic arm according to the sleep posture classification result.

[0044] The beneficial effects of the present invention are:

[0045] The present invention provides a millimeter wave radar vital sign monitoring method based on adaptive chest positioning. This method obtains infrared images of human sleeping postures, integrates the openpose model and the SVM classification method, and achieves high-accuracy classification of human sleeping postures after collecting a proprietary sleeping posture data set. At the same time, the mechanical arm is combined to dynamically adjust the radar position according to the sleeping posture. For radar echo signals, the human target vital sign signal is monitored through constant false alarm detection, pulse compression, DC clutter suppression and signal filtering algorithms, and counting monitoring of apnea is achieved based on the extracted data. This method can accurately identify a variety of human sleeping postures, ensure the user's sleeping comfort, and is not affected by the environment. It improves the accuracy and generalization of vital sign monitoring under sleep conditions and enriches the unique function of monitoring apnea during sleep.

[0046] This method can be used for single-person sleep vital sign monitoring. It uses an infrared camera to analyze the human body's sleeping posture without contact, and combines it with a robotic arm to achieve adaptive chest positioning under various sleeping postures, thereby using millimeter waves to monitor vital signs. It can effectively record the user's sleeping posture and improve the accuracy and generalization of vital sign monitoring. Users can check the vital sign status and number of apnea times during sleep and perform self-monitoring of sleep health.

[0047] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to make the purpose, technical solution and beneficial effects of the present invention clearer, the present invention provides the following drawings for illustration:

[0049] Figure 1 This is an architecture diagram of the adaptive chest positioning vital signs monitoring system.

[0050] Figure 2 Implement a flowchart for your program.

[0051] Figure 3 Monitor the GUI interface for the computer.

[0052] Figure 4 This is a diagram of the classification of sleeping postures.

[0053] Figure 5 Implement a flowchart for SVM classification. DETAILED DESCRIPTION

[0054] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.

[0055] In practical application, the radar and network in the method of the present invention can be set up in the following manner. In this preferred embodiment, a millimeter wave radar sensor with a 60GHz frequency band model AWR6843AOP launched by TI will be used to send and receive signals. This radar has a built-in processor unit Arm Cortex-R4F and a digital signal processing unit C674x DSP. Figure 1 As shown, the millimeter wave radar vital signs monitoring system based on adaptive chest positioning provided in this embodiment includes a host computer, a vital signs monitoring module, a sleep posture monitoring module, a robotic arm control module, a user interaction-computer interface, a user interaction-WeChat applet module, and a cloud server;

[0056] The sleeping posture monitoring module collects the sleeping posture image of the target by acquiring images;

[0057] The vital signs monitoring module is used to obtain the radar echo data of the target and organize it into a radar cube; monitor the breathing and heartbeat status of the target through the radar echo data, and warn and record the occurrence of apnea;

[0058] The host computer is provided with a sleep posture classification module and a mechanical arm control module;

[0059] The sleeping posture classification module is used to estimate the sleeping posture skeleton and classify the sleeping posture;

[0060] The robotic arm control module is used to control the robotic arm according to the sleep posture classification result;

[0061] The user interaction-computer interface is used to control the radar, monitor the sleep posture, breathing and heartbeat status of the experimental target in real time, and the number of sleep pauses;

[0062] The user interaction-WeChat applet module is used to remotely view the breathing and heart rate of the current experimental target and monitor the number of apnea during the monitoring period;

[0063] The cloud server is used to upload and store radar monitoring data in real time, and route and forward the data for the WeChat applet module to call, so as to realize remote monitoring;

[0064] like Figure 2 As shown, this embodiment describes in detail a method for monitoring vital signs of a millimeter-wave radar based on adaptive chest positioning. When the user starts monitoring, the data received by the radar echo is transmitted to the upper computer and processed by the hardware algorithm. The information is then uploaded to the cloud through WIFI, and managed and monitored on the cloud device; or transmitted back to the computer through the serial port for visual display; vital sign analysis and posture processing are performed on the computer; the vital sign analysis determines whether it is in a safe state or issues a warning through apnea judgment, and records the number of pauses; the posture processing obtains posture data through the turned-on camera and performs posture classification and posture output; the posture classification is performed through OpenPose reasoning and SVM classification; the WIFI adopts the MQTT protocol;

[0065] After the cloud data is acquired, the vital sign data can be displayed and the data can be comprehensively analyzed, and sleep quality scores and health suggestions can be given through comprehensive data analysis; the user acquires the cloud data by launching the mini program;

[0066] The method provided in this embodiment specifically includes the following steps:

[0067] Step 1: Collect eight types of target human sleeping posture images and train the SVM classifier:

[0068] The infrared camera is used to collect multiple images of the target human body in different sleeping postures to form a data set. The specific sleeping postures considered in this method are as follows: Figure 4 The captured image is sent to the openpose model, and the target human body posture skeleton key point data is output. The skeleton key point data is cleaned, and the skeleton key points and foot key points corresponding to abnormal sleeping postures are removed to ensure the validity of the data. After that, the SVM classifier parameters are set. The specific settings of the parameters are as follows:

[0069] Regularization parameter C: 200;

[0070] Tolerance: 0.0001;

[0071] Maximum number of iterations maxIter: 10000;

[0072] RBF kernel function parameter theta: 20.

[0073] After the parameters are set, the skeleton key points are input into the SVM classifier for training. After meeting the iteration requirements, the Figure 4 Dedicated classifiers for the eight sleeping positions shown.

[0074] Step 2: Collect the target human body sleeping posture image in real time, obtain the real-time posture classification result and control the position of the robotic arm:

[0075] The image captured by the infrared camera is transmitted to the host computer in real time for estimation and reasoning of key points of the sleeping posture skeleton, and the current target human posture is judged in real time based on the reasoning results. The SVM classification process is as follows: Figure 5 As shown in the figure, the image is sent to the model for real-time inference to obtain the key points of the human skeleton, and then the key points are sent to the SVM classifier optimized and trained by the SMO algorithm. The SVM classifier uses a one-versus-one (OVO) strategy to combine multiple binary classifiers to form a multi-classifier. The key point data in the classifier outputs the corresponding judgment category through the internal decision function. At the same time, the robot arm is connected through the local area network, and the posture classification results are output to the Raspberry Pi on the side of the robot arm in real time. After receiving the signal, the Raspberry Pi controls the robot arm to carry the radar and rotate to the corresponding preset position according to the classification result, so that the radar is facing the chest or back of the target human body.

[0076] The eight sleeping positions in this embodiment include supine, prone, left side, upper left, lower left, right side, upper right, and lower right. In supine, the robot arm moves to the top of the chest; in prone, the robot arm moves to the top of the back; in left side, the robot arm moves to the front of the chest; in upper left, the robot arm moves to the front of the chest; in lower left, the robot arm moves to the front of the back; in right side, the robot arm moves to the front of the chest; in upper right, the robot arm moves to the front of the chest; in lower right, the robot arm moves to the front of the back.

[0077] Step 3: Data reception and preprocessing process:

[0078] Receive millimeter-wave radar echo data, process the echo data into a data cube of sampling point dimension*chirp dimension*antenna dimension, perform fast Fourier transform on the ADC original complex signal of the data cube in the distance dimension (sampling point dimension), obtain the spectrum of the distance dimension, and perform one-dimensional CFAR processing and DC clutter filtering on the distance dimension spectrum to suppress interference signals. The one-dimensional CFAR processing is to determine the threshold value of the unit to be evaluated based on the reference unit level value and false alarm rate, and compare it with the actual level value. If the actual level value exceeds the threshold value, it is considered that there is a potential target in the current unit and it is retained in the candidate point sequence; the DC clutter suppression operation is to average the distance dimension data, and subtract the calculated mean from the complex data of each point to suppress the influence of DC clutter.

[0079] Step 4: Echo signal phase information extraction:

[0080] After preprocessing the echo data, the maximum value in the distance-dimensional spectrum is dynamically searched at intervals of 5 seconds within the distance range where the human target is located to obtain the range gate corresponding to the target human body; the target phase information is extracted once in a single frame at the range gate, and then phase unwrapping is performed. Multiple frames of phase data are connected to form a dynamic waveform of phase change, and the impulse signal of the phase waveform is removed to obtain the phase signal corresponding to the target human body.

[0081] Step 5: Extraction of vital signs and determination of sleep apnea:

[0082] Performing phase difference calculation between adjacent frames on the unfolded phase signal to obtain a phase difference signal; since there is a direct relationship between the difference signal and the chest rise and fall signal, the relationship between the difference signal and the chest rise and fall signal includes the following steps:

[0083] The phase difference signal and the distance change caused by the chest rise and fall have a relationship as indicated by the following formula:

[0084]

[0085] Among them, Δφ represents the phase difference between two adjacent frames, ΔR represents the distance change from the radar caused by the chest rise and fall, and λ c Indicates the wavelength of the RF signal emitted by the radar.

[0086] The phase difference signal is then sent to a pre-designed filter, where the passband of the respiratory signal is 0.1-0.5Hz, and the passband of the heart signal is 0.8-2Hz. After passing through the filter, the respiratory and heartbeat signals are obtained; the spectrum of the respiratory and heartbeat signals is then estimated to obtain the respiratory and heartbeat frequencies; the respiratory and heartbeat data are saved, and the energy of the respiratory signal is monitored in real time. When the respiratory signal energy is lower than the set threshold for 10 seconds, it is determined as a respiratory arrest, and the number of respiratory arrests is recorded. Finally, the respiratory and heartbeat waveforms, frequency values, and the number of respiratory arrests are displayed on the computer GUI interface, as shown below. Figure 3 shown.

[0087] exist Figure 3 The CFG_PORT and DATA_PORT in the upper left corner of the interface respectively display the selected configuration port and data port. The corresponding Refresh button on the right is used to refresh the current serial port content, the Open button is used to open the serial port, the green icon button is used to start the radar to start monitoring, and the red icon button is used to stop the radar to pause monitoring; the upper part of the lower left corner of the interface displays the real-time breathing signal waveform, and the lower part displays the real-time heartbeat signal waveform; the upper left part of the right side of the interface displays the real-time posture classification results, and the corresponding posture and Figure 4 As shown in the figure, the real-time heart rate and breathing rate are displayed from top to bottom in the upper right part respectively; the word Camera at the bottom right of the interface corresponds to the currently selected camera, the green icon on the right is used to turn on the currently selected camera, and the red icon is used to turn off the camera. The real-time monitoring image of the camera is displayed below the word Camera, and the word OSA at the bottom of the image corresponds to the number of respiratory pauses that occurred in the experimental target during the monitoring period.

[0088] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A millimeter wave radar vital signs monitoring method based on adaptive chest positioning, characterized in that: The following steps are involved: S1: Collect human sleeping posture images; S2: Estimate the human body posture and obtain the skeleton point estimation data; S3: Classify the human skeleton points to obtain posture classification results, and control the robotic arm carrying the radar to move to the best monitoring position corresponding to the current sleeping posture; S4: Acquire human body radar echo data; S5: Process the radar echo data and extract the phase of the echo data; S6: Process the phase of the echo data, analyze and obtain the target human body's breathing and heartbeat signals, and determine the number of apnea times based on the breathing and heartbeat data.

2. The method for monitoring vital signs using millimeter wave radar based on adaptive chest positioning according to claim 1, characterized in that: The method for estimating the posture in step S2 is performed according to the following steps: After collecting human sleeping posture images, the images are sent to the openpose model for real-time reasoning to output the key points of human posture skeletons.

3. The method for monitoring vital signs using millimeter wave radar based on adaptive chest positioning according to claim 1, characterized in that: The method for classifying postures in step S3 is performed according to the following steps: The estimated skeleton data is cleaned to remove the key points of the human foot skeleton, and then the skeleton key points are sent to the trained SVM classifier to output the posture classification results.

4. The method for monitoring vital signs using millimeter wave radar based on adaptive chest positioning according to claim 1, characterized in that: The method for controlling the robot arm in step S3 is performed according to the following steps: After obtaining the posture classification results, the host computer connects to the robotic arm through the local area network and outputs the classification results to the Raspberry Pi on the robotic arm end. After receiving the signal, the Raspberry Pi controls the robotic arm carrying the radar to rotate to the corresponding preset position according to the classification results.

5. The method for monitoring vital signs using millimeter wave radar based on adaptive chest positioning according to claim 1, characterized in that: The method for extracting the phase in step S5 is performed according to the following steps: S51: Acquire radar echo data, and process the radar data into a data cube of sampling point dimension*chirp dimension*antenna dimension; S52: performing fast Fourier transform on the ADC original complex signal of the data cube in the distance dimension to obtain a spectrum in the distance dimension, and performing one-dimensional CFAR processing and DC clutter filtering on the distance dimension spectrum to suppress interference signals; S53: dynamically search for the maximum value in the distance spectrum at intervals of 5 seconds within the human target distance range to obtain the distance gate corresponding to the target human body; S54: extracting the target phase information once in a single frame at the range gate, then performing phase unwrapping, and then removing the impulse signal to obtain a phase signal corresponding to the target human body.

6. The method for monitoring vital signs using millimeter wave radar based on adaptive chest positioning according to claim 1, characterized in that: The method for analyzing the respiratory heartbeat signal in step S6 is performed according to the following steps: S61: performing phase difference calculation between adjacent frames on the unfolded phase signal to obtain a phase difference signal; S 62: According to the relationship between the differential signal and the chest rise and fall signal, the phase differential signal is sent to the filter to obtain the breathing and heartbeat signals; S63: performing frequency spectrum estimation on the breathing and heartbeat signals respectively to obtain the breathing and heartbeat frequencies; S64: The breathing and heartbeat data are saved, and the energy of the breathing signal is monitored in real time. When the energy of the breathing signal is lower than the set threshold for 10 seconds, it is determined as a respiratory arrest.

7. The method for monitoring vital signs using millimeter wave radar based on adaptive chest positioning according to claim 6, characterized in that: The relationship between the differential signal and the chest rise and fall signal comprises the following steps: The phase difference signal and the distance change caused by the chest rise and fall have a relationship as indicated by the following formula: Among them, represents the phase difference between two adjacent frames, represents the change in distance from the radar caused by the rise and fall of the chest, and represents the wavelength of the radio frequency signal emitted by the radar.

8. The method for monitoring vital signs using millimeter wave radar based on adaptive chest positioning according to claim 5, characterized in that: The impulse signal removal is specifically performed according to the following steps: The signal strength value of the current phase point is subtracted from the signal strength values ​​of the two adjacent phase points, and compared with the set threshold value. If it exceeds the threshold, the signal strength value of the current phase point is reset to the average of the signal strengths of the adjacent phase points.

9. The method for monitoring vital signs using millimeter wave radar based on adaptive chest positioning according to claim 3, characterized in that: The SVM classifier training is specifically performed according to the following steps: After the training parameters of the SVM classifier are configured, the obtained sleeping posture skeleton data is normalized and sent to the SVM classifier for training to obtain dedicated classifiers for eight sleeping postures.

10. The millimeter wave radar vital signs monitoring system based on adaptive chest positioning is characterized by: Including sleep posture monitoring module, vital signs monitoring module, sleep posture classification module, robotic arm control module, vital signs monitoring module: The sleeping posture monitoring module is used to collect the sleeping posture image of the target; The vital signs monitoring module is used to obtain the target echo data and organize it into a radar cube, monitor the target's breathing and heartbeat status through the radar echo data, and warn and record the occurrence of apnea; The sleeping posture classification module is used to estimate the sleeping posture skeleton and classify the sleeping posture; The robotic arm control module is used to control the robotic arm according to the sleep posture classification result.

Images