Non-contact intelligent man-machine interaction medical monitoring system based on millimeter wave radar technology

Abstract

The invention relates to a non-contact intelligent man-machine interaction medical monitoring system based on a millimeter wave radar technology. The non-contact intelligent man-machine interaction medical monitoring system comprises a health index detection module, a voice module and a tracing and obstacle avoidance module. According to the non-contact intelligent man-machine interaction medical monitoring system based on the millimeter wave radar technology, the robot technology, the centimeter wave technology, the electroencephalogram control technology and the like are combined with an informatization platform; the system can help nursing workers or family members to deliver medicines, classify the medicines, monitor the physical condition of a patient in real time, perform timely early warning on emergency conditions and the like, relieve the pressure of medical care work and family nursing, analyze facial expressions and emotions of a user, perform corresponding psychological counseling and improve the nursing efficiency. The method is specifically applied to patients suffering from depression, Alzheimer's disease and the like, provides social and entertainment services for the patients, and helps the patients to improve social ability, reduce aloneness and improve social participation.

Description

Technical Field

[0001] The present invention relates to the technical field of medical robots, and particularly to a non-contact intelligent human-machine interaction medical monitoring system based on millimeter-wave radar technology. Background Art

[0002] According to the definition of the International Federation of Robotics, robots can be divided into industrial robots and service robots. In recent years, with the improvement of people's living quality, a large number of semi-autonomous or fully autonomous robots have emerged in the market to meet the public demand, providing daily life services for people. Representative works include the Pepper robot of SoftBank in Japan and the Baidu robot Xiaodu, etc. With the development of China's aging process, the home service robots have also shown explosive growth and played an important role in accompanying the elderly, home security, home cleaning, etc.

[0003] The research on domestic home companion robots started later than that in foreign countries, but still made good progress in recent years. For example, the epidemic prevention companion robot developed by Ecovacs for the current epidemic can query the medical records of patients and retrieve inspection reports without contact. The Dazhi robot developed by Shandong University can achieve automatic patrol, abnormal alarm, voice chat, and video call functions. The V5-Angel robot developed by Tammy Intelligence has autonomous movement and autonomous obstacle avoidance functions, and can provide medical care, remote monitoring, delivery of items, interactive entertainment and other services for patients.

[0004] The above-mentioned are all cutting-edge products in the same type of robot field. However, through comprehensive sample analysis, it can be seen that most domestic robots of the same type only have functions such as daily communication, medical record query, medication reminder, video chat, etc., with relatively single functions, and are lacking in aspects such as in-depth emotional communication, unable to achieve emotional analysis and psychological counseling, and also unable to relieve the loneliness of elderly users. Therefore, a non-contact intelligent human-machine interaction medical monitoring system based on millimeter-wave radar technology is proposed to solve the above problems. Summary of the Invention

[0005] Aiming at the deficiencies of the prior art, the present invention provides a non-contact intelligent human-machine interaction medical monitoring system based on millimeter-wave radar technology, which has the advantages of complete functions, and solves the problems that most domestic robots of the same type only have functions such as daily communication, medical record query, medication reminder, video chat, etc., with relatively single functions, and are lacking in aspects such as in-depth emotional communication, unable to achieve emotional analysis and psychological counseling, and also unable to relieve the loneliness of elderly users.

[0006] To achieve the above object, the present invention provides the following technical solution: A non-contact intelligent human-machine interaction medical monitoring system based on millimeter-wave radar technology, including a health index detection module, a voice module, and a tracing and obstacle avoidance module;

[0007] Among them, the health index detection module is used for: real-time monitoring and collecting the physical health indexes of the elderly;

[0008] The voice module is used for: directional retrieval of the user's voice signal and recognition and semantic understanding of voice commands;

[0009] The trace tracking and obstacle avoidance module is used for: enabling the robot to identify obstacles during movement and bypass the obstacles when the distance is less than the set range.

[0010] Furthermore, the health index detection module includes a temperature measurement unit, an electrocardiogram measurement unit, and a respiratory body movement monitoring unit.

[0011] Furthermore, the temperature measurement unit adopts the non-contact temperature measurement technology of MLX90614. MLX90614 is an infrared thermometer for non-contact temperature measurement, and MLX90614 integrates a low-noise amplifier, a 17-bit ADC, and a DSP unit.

[0012] Furthermore, the electrocardiogram measurement unit is the core amplification system of the signal acquisition part, and the operational amplifier of the AD623 chip plays an amplification role.

[0013] Furthermore, the core of the respiratory body movement monitoring unit is a respiratory sensor. The respiratory sensor captures the signal of the patient's respiratory movement. The respiratory body movement monitoring unit adopts Bluetooth module and WiFi module technologies to send the physical signs data of the elderly to the mobile phones of medical staff and family members.

[0014] Furthermore, the respiratory body movement monitoring unit includes a built-in storage card. The built-in storage card records the long-term respiratory data of the elderly. The respiratory body movement monitoring unit is a wearable device, including but not limited to chest straps, patches, or other forms.

[0015] Furthermore, the voice module is equipped with a six-microphone circular microphone array. The array adopts a planar distribution structure, including six microphones to achieve 360-degree equivalent sound pickup, and the wake-up resolution is 1 degree.

[0016] Furthermore, the audio types and wake-up angles obtained by the microphone are defined as follows:

[0017] Noise reduction audio: Sampling rate 16khz, 16bit, one channel;

[0018] Original audio: Sampling rate 16khz, 32bit, eight channels, where channels 1-6 correspond to 6 microphones, and channels 7-8 are reference signals;

[0019] Wake-up angle: Starting from microphone No. 0, it is 0-359 in clockwise order.

[0020] Furthermore, the path tracking and obstacle avoidance module is equipped with a single-line TOF short-range mechanical lidar to assist the robot in measuring the specific distance from obstacles and achieving further avoidance. The path tracking and obstacle avoidance module uses the TOF ranging principle to measure the relative distance between the object and the sensor by measuring the time difference between the emission and return of the modulated laser.

[0021] Furthermore, the laser emitter emits modulated pulsed laser, and the internal timer starts to calculate the time from time T1. When the laser irradiates the target object, part of the energy returns. When the radar receives the returned laser signal, the internal timer stops timing at time t2, and the following formula can be obtained:

[0022] D = C * (T2 - T1) / 2, where D is the distance and C is the speed of light.

[0023] Compared with the prior art, the technical solution of this application has the following beneficial effects:

[0024] This non-contact intelligent human-computer interaction medical monitoring system based on millimeter-wave radar technology combines robot technology, centimeter-wave technology, electroencephalogram control technology, etc. with an information platform, with a millisecond-level sampling frequency, more accurate and effective data, passive pressure sensing, no contact throughout the process, safe and radiation-free, continuous monitoring of vital signs, grasping of health trends, establishment of an individual health model by AI big data, and prediction of disease risks.

[0025] It can help nurses or family members deliver medicines, classify medicines, monitor the patient's physical condition in real time, give early warnings of critical conditions, etc., greatly reducing the pressure of medical care work and family care. It can further be transformed into a companion robot, further developing the entertainment and interaction functions of the robot, strengthening facial recognition so as to analyze the user's facial expressions and emotions, and providing corresponding psychological counseling. Specifically, it is applied to patients with depression, Alzheimer's disease, etc., providing them with social and entertainment services, helping them improve their social skills, reduce loneliness, and increase social participation. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is the system block diagram of the present invention;

[0027] Figure 2 It is the schematic diagram of the temperature measurement unit of the present invention;

[0028] Figure 3 It is the schematic diagram of the electrocardiogram measurement unit of the present invention;

[0029] Figure 4 It is the hardware structure diagram of the microphone array of the present invention;

[0030] Figure 5This is a schematic diagram of the ranging principle of the lidar of the present invention. Specific Embodiments

[0031] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0032] Please refer to Figures 1-5 , the non-contact intelligent human-computer interaction medical monitoring system based on millimeter-wave radar technology in this embodiment includes a health index detection module, a voice module, and a tracing and obstacle avoidance module;

[0033] Among them, the health index detection module is used for: real-time monitoring and collecting the physical health indexes of the elderly;

[0034] The voice module is used for: directional retrieval of the user's voice signal and recognition and semantic understanding of voice commands;

[0035] The tracing and obstacle avoidance module is used for: enabling the robot to recognize obstacles during travel and bypass the obstacles when the distance is less than the set range.

[0036] The health index detection module includes a temperature measurement unit, an electrocardiogram measurement unit, and a respiratory body movement monitoring unit.

[0037] The temperature measurement unit adopts the non-contact temperature measurement technology of MLX90614. MLX90614 is an infrared thermometer for non-contact temperature measurement, and MLX90614 integrates a low-noise amplifier, a 17-bit ADC, and a DSP unit.

[0038] This infrared thermometer can provide the measured temperature (resolution of 0.02 °C) within the entire temperature range through digital SMBus output. When in use, the digital output can be configured as pulse width modulation (PWM). Under standard conditions, the 10-bit PWM is configured to continuously transmit the measured temperature between -20 and 120 °C with a resolution of 0.14 °C. Therefore, through this technology, we can achieve very accurate temperature and display the temperature.

[0039] The electrocardiogram measurement unit is the core amplification system of the signal acquisition part, and the operational amplifier of the AD623 chip plays an amplification role.

[0040] Among them, inputs 1, 2, and 3 input signals from the left upper limb (LA), right upper limb (RA), and left lower limb (LF) of the human body respectively through single-lead electrode wires and electrode patches, and each input terminal is capacitively grounded to play a low-pass filtering role.

[0041] The core of the respiratory and body movement monitoring unit is the respiratory sensor, which captures the signals of the patient's respiratory movement. The respiratory and body movement monitoring unit uses Bluetooth module and WiFi module technologies to send the vital signs data of the elderly to the mobile phones of medical staff and family members.

[0042] The respiratory and body movement monitoring unit includes a built-in storage card, which records the long-term respiratory data of the elderly. The respiratory and body movement monitoring unit is a wearable device, including but not limited to chest straps, patches or other forms.

[0043] The determination principle of blood oxygen measurement includes two parts: spectrophotometry and photoplethysmography. Spectrophotometry uses red light with a wavelength of 660nm and infrared light with a wavelength of 940nm. According to the fact that oxyhemoglobin (HbO2) has less absorption of 660nm red light and more absorption of 940nm infrared light, while hemoglobin (Hb) is the opposite, the ratio of the absorption of infrared light to the absorption of red light is determined by spectrophotometry to determine the oxygenation degree of hemoglobin.

[0044] Another important principle of pulse oximetry is that there must be blood pulsation. When the peripheral tissue is irradiated with light, the attenuation degree of the detected transmitted light energy is related to the cardiac cycle (when the heart contracts, the peripheral blood volume is the largest, the light absorption is also the largest, and the detected light energy is the smallest, and vice versa when the heart relaxes). The change in light absorption reflects the change in blood volume.

[0045] The expression of oxygen saturation is: oxygen saturation % = oxyhemoglobin / (oxyhemoglobin + deoxyhemoglobin) X 100%.

[0046] There are various data interfaces on the back panel of the robot, such as I2C, UART, USB, which can be externally connected to a variety of medical monitoring instruments and obtain the corresponding monitoring values in real time through the serial port data processing and parsing algorithm. At the same time, the conditional algorithm and voice broadcast can be used to announce the numerical content and whether it exceeds the standard, etc.

[0047] The voice module is equipped with a six-microphone circular microphone array. The array adopts a planar distribution structure, including six microphones to achieve 360-degree equivalent sound pickup, and the wake-up resolution is 1 degree.

[0048] The audio types and wake-up angles obtained by the microphone are defined as follows:

[0049] Noise reduction audio: sampling rate 16khz, 16bit, one channel;

[0050] Original audio: sampling rate 16khz, 32bit, eight channels, where channels 1-6 correspond to 6 microphones, and channels 7-8 are reference signals;

[0051] Wake-up angle: Starting from the 0th microphone, it is 0 - 359 in a clockwise direction.

[0052] Using the LD3320 module capable of acquiring sound + ASR speech recognition algorithm, the robot can be equipped with the ability to recognize wake-up words and other custom commands. The basic logic is that the robot is first activated by a preset wake-up word to enter the awakened state, and only in the awakened state can it continue to recognize other voice commands, such as "forward", "move right", etc. After the robot recognizes a voice command, it will, through a conditional filtering algorithm, be associated with the corresponding limb movement settings. Additionally, combined with a sound source sensor and sound source calculation and processing, the robot can quickly determine the source of the sound, thus better assisting in the execution of actions.

[0053] A small camera is installed on the robot's head, which can obtain real-time image information through Python's CV2 and Camera libraries. Then, through the OpenCV image processing algorithm plus the Café image analysis framework, basic video recognition functions can be achieved, such as the recognition of preset objects of different colors and the recognition of face shapes. When a corresponding color object is recognized, the robot will use the associated external sound module to announce the recognition content by voice. When a face is recognized, the robot will turn to the recognized target and perform a greeting action.

[0054] The trace-following and obstacle-avoidance module is equipped with a single-line TOF short-range mechanical lidar to assist the robot in measuring the specific distance from obstacles and achieving further avoidance. The trace-following and obstacle-avoidance module uses the TOF ranging principle to measure the relative distance between an object and the sensor by measuring the time difference between the emission and return of modulated laser light.

[0055] The laser emitter emits modulated pulsed laser light, and the internal timer starts counting time from time T1. When the laser light irradiates the target object, part of the energy returns. When the radar receives the returned laser signal, the internal timer stops counting time at time t2, and the following formula is obtained:

[0056] D = C * (T2 - T1) / 2, where D is the distance and C is the speed of light.

[0057] Installing a lidar module on the robot can receive real-time distance information returned by the radar, and through a conditional processing algorithm, the robot can recognize obstacles during the forward movement and bypass the obstacles when the distance is less than the set range.

[0058] During actual use, the robot receives voice signals, moves to the user, and interacts with the user through voice. During the conversation, the robot can perform corresponding tasks through the user's voice semantics, such as bringing the built-in home medical testing equipment to the user, measuring the user's physical indicators, and alarming when the detected values are abnormal. It delivers items through specific commands, reminds the user to take medicine, exercise and avoid sitting for a long time according to the pre-set time, finds, sorts and takes medicine to the user at the custom time, and can have simple voice chats with the user, conduct weather forecasts, tell time, remind memos and schedule planning. In the future, it is expected to add the function of issuing commands through brain wave signals and gestures, so that users can convey command information to the robot through thoughts or simple gestures.

[0059] The robot is equipped with a laser radar device, which can emit a detection signal (laser beam) when placed in a home environment. It then compares the received signal reflected from the target (target echo) with the transmitted signal. After appropriate processing, it obtains the obstacle target distance, direction, shape and other parameters to avoid obstacles. After radar detection, the software algorithm draws a scene map, performs route planning, and realizes the positioning and tracking of the user's position. When the user issues a voice command, different tones, different timbres, or different descriptions of the same semantics (such as "Tell me the current time" and "What time is it now") are used. The robot's built-in microphone sequence can effectively collect sound signals and perform deeper semantic recognition, and can eliminate the interference of noise and echo to a certain extent. The premise of delivering and sorting items is graphic recognition. The camera of the robot device will collect image information within its visual range in real time, and hand it over to the mainboard processor for algorithm analysis to accurately identify and calculate the position and distance of the target object. After analyzing the framework, the robotic arm is used to grab it. At the mechanical level, the computer issues instructions to the servo according to the algorithm to control the robot and the robotic arm to operate.

[0060] Compared with the prior art, the technical solution of this application has the following beneficial effects:

[0061] The contactless intelligent human-computer interactive medical monitoring system based on millimeter-wave radar technology combines robotics technology, centimeter-wave technology, and electroencephalogram control technology with an information platform. It has a millisecond sampling frequency, more accurate and effective data, passive pressure sensing, non-contact throughout the process, safe and radiation-free, continuous monitoring of vital signs, and grasp of health trends. AI big data establishes personal health models and predicts disease risks.

[0062] It can assist caregivers or family members in delivering medications, classifying medications, monitoring the patient's physical condition in real time, and promptly warning of critical situations, greatly reducing the pressure of medical care work and home care. It can also be transformed into a companion robot, further developing the entertainment and interaction functions of the robot, strengthening facial recognition so that it can analyze the user's facial expressions and emotions and provide corresponding psychological counseling. Specifically, it can be applied to patients with depression, Alzheimer's disease, etc., providing them with social and entertainment services, helping them improve their social skills, reduce loneliness, and increase social participation.

[0063] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0064] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention.

Claims

1. A contactless intelligent human-machine interaction medical monitoring system based on millimeter-wave radar technology, characterized in that, It includes a health index detection module, a voice module, and a path tracking and obstacle avoidance module; Among them, the health index detection module is used to: monitor and collect the physical health indexes of the elderly in real time; The voice module is used to: perform directional retrieval of the user's voice signal, as well as recognize and semantically understand voice commands; The path tracking and obstacle avoidance module is used to: enable the robot to recognize obstacles during movement and bypass the obstacles when the distance is less than the set range.

2. The non-contact intelligent human-computer interaction medical monitoring system based on millimeter-wave radar technology according to claim 1, wherein The health index detection module includes a temperature measurement unit, an electrocardiogram measurement unit, and a respiratory body movement monitoring unit.

3. The contactless intelligent human-computer interaction medical monitoring system based on millimeter-wave radar technology according to claim 2, characterized in that The temperature measurement unit adopts the non-contact temperature measurement technology of MLX90614. MLX90614 is an infrared thermometer for non-contact temperature measurement, and MLX90614 integrates a low-noise amplifier, a 17-bit ADC, and a DSP unit.

4. The non-contact intelligent human-machine interaction medical monitoring system based on millimeter-wave radar technology according to claim 2, wherein The electrocardiogram measurement unit is the core amplification system of the signal acquisition part, and the operational amplifier of the AD623 chip plays an amplification role.

5. The non-contact intelligent human-computer interaction medical monitoring system based on millimeter-wave radar technology according to claim 2, wherein The core of the respiratory body movement monitoring unit is a respiratory sensor. The respiratory sensor captures the signal of the patient's respiratory movement. The respiratory body movement monitoring unit adopts Bluetooth module and WiFi module technologies to send the physical sign data of the elderly to the mobile phones of medical staff and family members.

6. The non-contact intelligent human-computer interaction medical monitoring system based on millimeter-wave radar technology according to claim 2, wherein The respiratory body movement monitoring unit includes a built-in storage card. The built-in storage card records the long-term respiratory data of the elderly. The respiratory body movement monitoring unit is a wearable device, including but not limited to chest straps, patches, or other forms.

7. The non-contact intelligent human-computer interaction medical monitoring system based on millimeter-wave radar technology according to claim 1, wherein The voice module is equipped with a six-microphone circular microphone array. The array adopts a planar distribution structure, including six microphones to achieve 360-degree equivalent sound pickup, and the wake-up resolution is 1 degree.

8. The contactless intelligent human-machine interaction medical monitoring system based on millimeter-wave radar technology according to claim 7, characterized in that, The audio types and wake-up angles obtained by the microphone are defined as follows: Noise-reduced audio: Sampling rate 16khz, 16bit, one channel; Original audio: Sampling rate 16khz, 32bit, eight channels. Among them, channels 1-6 correspond to 6 microphones, and channels 7-8 are reference signals; Wake-up angle: Starting from microphone No. 0, it is 0-359 in clockwise order.

9. The non-contact intelligent human-machine interaction medical monitoring system based on millimeter-wave radar technology according to claim 1, characterized in that, The path tracking and obstacle avoidance module is equipped with a single-line TOF short-range mechanical lidar to assist the robot in measuring the specific distance from the obstacle and achieving further avoidance. The path tracking and obstacle avoidance module adopts the TOF ranging principle and measures the relative distance between the object and the sensor by measuring the time difference between the emission and return of the modulated laser.

10. The non-contact intelligent human-computer interaction medical monitoring system based on millimeter-wave radar technology according to claim 9, characterized in that, The laser transmitter emits a modulated pulsed laser, and the internal timer starts to calculate the time from time T1. When the laser irradiates the target object, part of the energy returns. When the radar receives the returned laser signal, the internal timer stops timing at time t2, and the following formula is obtained: D = C*(T2 - T1) / 2, where D is the distance and C is the speed of light.

Images