Intelligent ward patrol robot

Abstract

The invention discloses an intelligent ward patrol robot, and relates to the technical field of intelligent medical robots, the intelligent ward patrol robot comprises a robot body, and the robot body is internally provided with a multi-mode sensing unit, an edge calculation unit, a federal learning unit and a decision execution unit. According to the multi-mode sensing unit, a polarization type 3D camera, a 60 GHz millimeter wave radar and an environment sensor array are integrated in a robot body. By installing the multi-mode sensing unit, the non-contact emotion recognition function is achieved, in combination with contact detection of the intelligent bracelet on the signs of the patient, the bottleneck of non-contact emotion and sign cooperative monitoring is broken through, the emotion sensing ability of the patient is improved, and the medical service quality is improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent medical robot technology, specifically to an intelligent ward inspection robot. Background Technology

[0002] With the continuous development of the Third Technological Revolution, medical conditions have greatly improved. Ward rounds, as one of the main means of monitoring patients, play an important role in their recovery. Traditional ward rounds rely on manual labor, which leads to problems such as manpower shortages, blind spots in nighttime rounds, and delayed detection of abnormal signs. Most monitoring equipment is single-point and lacks the ability to dynamically perceive the patient's emotional state. Vital signs, environmental parameters, and patient behavior data are not analyzed in a spatiotemporal correlation, making it difficult to achieve comprehensive early warning.

[0003] Patent document CN114446416B discloses an intelligent ward round and consultation robot. The above patent realizes efficient and objective intelligent consultation and proposes an intelligent ward round and consultation robot that conducts fixed-point patrols, intelligent consultations, and finally scores through video, audio, and questionnaires.

[0004] The aforementioned patent automatically identifies patient video and voice information through facial recognition, completing the intelligent automatic collection of data during ward rounds and consultations. Through artificial intelligence analysis and comparison of time analysis of different ward round paths, it completes the intelligent evaluation and online setting of ward round paths.

[0005] Therefore, this application proposes an intelligent ward round robot that can perform both physiological and psychological testing on patients. Summary of the Invention

[0006] The purpose of this invention is to provide an intelligent ward inspection robot to solve the technical problem of data silos mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: an intelligent ward inspection robot, comprising a robot body, wherein a multimodal perception unit, an edge computing unit, a federated learning unit, and a decision execution unit are installed inside the robot body; The robot body has a human-computer interaction interface installed on its front, which is a touch screen. A speaker is installed on the upper outer wall of the robot body, and a power interface is opened on the lower outer wall of the robot body. The power interface is for charging the battery installed inside the robot body. Air inlets and air outlets are opened on the top and bottom of the robot body, respectively. The bottom of the robot body is equipped with omnidirectional wheels driven by motors to move the robot.

[0008] Preferably, the multimodal sensing unit includes components integrated into a smart bracelet and the robot body. The smart bracelet has a built-in RFID tag for real-time monitoring of the patient's heart rate, blood oxygen, and body movement frequency. The robot integrates a polarized 3D camera, a 60 GHz millimeter-wave radar, and an environmental sensor array. The polarized 3D camera is used to capture micro-expressions to eliminate interference from glare in the ward. The 60 GHz millimeter-wave radar is used to detect the respiratory rhythm of bedridden patients by penetrating the quilt. The environmental sensor array includes temperature sensors, humidity sensors, carbon dioxide concentration sensors, light intensity sensors, and air pressure sensors to collect environmental data outside the robot.

[0009] Preferably, the edge computing unit is equipped with a processor, which can analyze the data collected by the multimodal perception unit and perform AR face recognition through a polarized 3D camera. The edge computing unit is equipped with an emotion recognition module and a spatiotemporal feature fusion unit, and a spatiotemporal correlation database is installed inside the robot body. The emotion recognition module adopts a lightweight design and MobileNet-3D architecture, and its inference speed reaches 55 FPS after TensorRT quantization; The spatiotemporal feature fusion unit aligns the spatiotemporal windows of vital sign data and emotion data, and achieves weighted fusion of multi-source data through Kalman filtering and dynamic time warping algorithm; The spatiotemporal correlation database is interconnected with the edge computing unit. The spatiotemporal correlation database is used to store timestamps and spatial coordinate labels of vital signs-emotions-environment data.

[0010] Preferably, the federated learning unit constructs a differential privacy federated learning framework, with the smart bracelet and robot as edge nodes respectively. The emotion recognition module is trained locally through the differential privacy federated learning framework, and the original data does not leave the local area, thus protecting personal information. The federated learning unit includes a model update mechanism that automatically triggers local model retraining when the nighttime light intensity is less than 50 lux or the environmental noise characteristics change by more than 20%, maintaining the emotion recognition accuracy greater than 95%.

[0011] Preferably, the decision execution unit includes a dynamic path planning unit, a clinical decision engine, a voice interaction module, and an ultraviolet pulse disinfection unit; The dynamic path planning unit enables autonomous navigation of the ward based on the built-in UWB positioning module and SLAM mapping module; The algorithm for dynamic path planning is defined as follows: Inspection Priority = alpha risk value + beta abnormality value + gamma inappropriate environment The coefficients α, β, and γ are optimized through training on historical data, and: Risk score = |Real-time heart rate - Baseline heart rate| / Baseline heart rate × 100%; Emotional anomaly score = Microexpression anxiety score × Respiratory disorder index; Environmental inadequacy = Deviation of carbon dioxide concentration × Light inadequacy coefficient; The dynamic pathway planning unit can also automatically adjust the frequency of rounds based on the patient's risk level (postoperative / high-risk): High-risk patients: Check on them every 15 minutes; Postoperative patients: Check on them every 30 minutes; For ordinary patients: check on them once every 60 minutes; The dynamic path planning algorithm is the core algorithm of this invention. It is defined as follows: patrol priority equals the sum of α multiplied by the vital sign risk value, β multiplied by the emotional abnormality value, and γ multiplied by the environmental inadequacy factor. The coefficients α, β, and γ are optimized through training with historical data. The vital sign risk value is obtained by multiplying the absolute value of the difference between real-time heart rate and baseline heart rate by the baseline heart rate, then multiplying by 100%. The emotional abnormality value is the product of the micro-expression anxiety score and the respiratory disturbance index. Environmental inadequacy is calculated by multiplying the carbon dioxide concentration deviation value by the light discomfort coefficient.

[0012] The clinical decision engine constructs an interpretable risk decision tree, dynamically adjusts the warning threshold based on the patient's medical history, and connects to the hospital's HIS system. When "shortness of breath + anxiety" is detected, it automatically associates with medication records, distinguishes between postoperative pain and pulmonary embolism treatment procedures, and outputs treatment suggestions, analgesics, or emergency calls. The ultraviolet pulse disinfection unit has a built-in excimer lamp. When the robot detects that a patient has left the ward for more than three minutes, it turns on the excimer lamp to perform ultraviolet pulse disinfection on the ward. After the disinfection is completed, the disinfection log is uploaded to the hospital system.

[0013] Preferably, the emotion recognition module includes: a MobileNet-3D architecture, an FFT frequency domain analysis module, an audio emotion analysis and compensation module, and an MLP classifier; The spatiotemporal features of micro-expressions are extracted using the MobileNet-3D architecture. The respiratory waveform measured by 60 GHz millimeter-wave radar is analyzed in the FFT frequency domain analysis module, and the 0.1-0.5 Hz anxiety feature frequency band is extracted. When the face is occluded, the audio emotion analysis compensation module is activated, which reduces the blind spot by 60%. The micro-expression, respiratory frequency domain features and audio data are fused and input into the MLP classifier to output the anxiety or pain level.

[0014] Preferably, the robot body is equipped with a central control unit, which is connected to an edge computing unit. The central control unit controls the motor, decision execution unit, wristband disinfection chamber and speaker connected to the central control unit based on the data analyzed by the edge computing unit.

[0015] Preferably, the robot body is equipped with a power consumption control unit and a negative pressure ward adaptation module, which is a hardware-optimized design; The robot's surface is covered with a flexible perovskite solar film. Guided by the power consumption control unit, the robot can automatically charge itself while patrolling near the ward window, increasing its battery life by 40%. After the robot enters the negative pressure ward, when the air pressure sensor in the environmental sensor array detects an abnormal external air pressure, the negative pressure ward adaptation module controls the robot to close the air inlet and outlet to adapt to the environment of the negative pressure ward.

[0016] Preferably, the robot body has a wristband disinfection chamber on the front. When the patient is discharged, the robot automatically retrieves the wristband into the wristband disinfection chamber. The outer wall of the wristband disinfection chamber is equipped with a guide rail, which can move horizontally under the control of the central control unit. The inner walls around the wristband disinfection chamber are equipped with ultraviolet lamps for disinfecting the smart wristband and preventing cross-infection of wearable devices.

[0017] Preferably, a lidar is installed in the opening on the outer wall of the robot body, and an avoidance module that communicates with the lidar is installed inside the robot body. The avoidance module controls the robot to actively avoid the patient and maintain a safe distance of more than 1.5 meters through the lidar.

[0018] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention achieves non-contact emotion recognition by installing a multimodal sensing unit. Combined with the contact detection of patient vital signs by a smart bracelet, it breaks through the bottleneck of non-contact collaborative monitoring of emotions and vital signs, improves the ability to perceive patients' emotions, and is conducive to improving the quality of medical services. 2. This invention, by installing a lightweight edge computing unit, realizes the function of spatiotemporal correlation of multi-source data of patient vital signs, emotions and environment, promotes the transformation from passive monitoring to active intervention, speeds up response, shortens the time of abnormal events, and improves the monitoring efficiency of hospitalized patients; 3. This invention, by installing a decision execution unit, realizes the function of adaptive clinical decision-making, constructs a dynamic risk prediction decision tree, improves the level of intelligence in clinical decision-making, and meets the needs of personalized patient care. 4. This invention, by installing an ultraviolet pulse disinfection unit and a wristband disinfection chamber, achieves the function of infection control in hospitals. Non-contact disinfection and automatic robotic disinfection reduce the risk of cross-infection and improve the hospital's safety protection capabilities. Attached Figure Description

[0019] Figure 1 This is a front view structural diagram of the present invention; Figure 2 This is a schematic diagram of the front part of the present invention; Figure 3 This is a schematic diagram of the decision execution unit structure of the present invention; Figure 4 This is a schematic diagram of the multimodal sensing unit structure of the present invention; Figure 5 This is a schematic diagram of the edge computing unit structure of the present invention; Figure 6 This is a schematic diagram of the emotion recognition module structure of the present invention; Figure 7 This is a schematic diagram of the wristband disinfection chamber structure of the present invention; Figure 8 This is a schematic diagram of the system architecture of the present invention.

[0020] In the diagram: 1. Robot body; 2. Human-machine interface; 3. Air inlet; 4. Multimodal perception unit; 5. Ultraviolet pulse disinfection unit; 6. Wristband disinfection chamber; 7. Speaker; 8. Power interface; 9. Casters; 10. Federated learning unit; 11. Alternate obstacle avoidance module; 12. Edge computing unit; 13. Central control unit; 14. Decision execution unit; 15. Motor; 16. Battery; 17. Spatiotemporal relational database; 18. LiDAR; 19. Solar film; 20. Power consumption control unit; 21. Air outlet; 22. Dynamic path planning unit; 23. Clinical decision engine; 24. Negative pressure ward adaptation module; 25. UWB positioning module; 26. Voice interaction module; 27. SLAM mapping module; 28. Polarized 3D camera; 29. ​​Environmental sensor array; 30, 60 GHz millimeter-wave radar; 31. Emotion recognition module; 32. Spatiotemporal feature fusion unit; 33. Processor; 34. MobileNet-3D architecture; 35. FFT frequency domain analysis module; 36. Audio emotion analysis and compensation module; 37. MLP classifier; 38. Guide rail; 39. Ultraviolet lamp; 40. Excimer lamp. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Example 1: Please refer to Figure 1 , Figure 2 , Figure 4 and Figure 5 A smart ward inspection robot includes a robot body 1, which is equipped with a multimodal perception unit 4, an edge computing unit 12, a federated learning unit 10 and a decision execution unit 14. The multimodal sensing unit 4 includes components integrated within a smart bracelet and the robot body 1. The smart bracelet has a built-in RFID tag for real-time monitoring of the patient's heart rate, blood oxygen, and body movement frequency. The robot body 1 integrates: a polarized 3D camera 28, a 60 GHz millimeter-wave radar 30, and an environmental sensor array 29. The polarized 3D camera 28 is used to capture micro-expressions to eliminate interference from glare in the ward. The 60 GHz millimeter-wave radar 30 is used to detect the respiratory rhythm of bedridden patients through the quilt. The environmental sensor array 29 includes a temperature sensor, a humidity sensor, a carbon dioxide concentration sensor, a light intensity sensor, and a barometric pressure sensor, which are used to collect environmental data outside the robot body 1. The edge computing unit 12 is equipped with a processor 33, which can analyze the data collected by the multimodal perception unit 4 and perform AR face recognition through the polarized 3D camera 28. The edge computing unit 12 is equipped with an emotion recognition module 31 and a spatiotemporal feature fusion unit 32. The robot body 1 is equipped with a spatiotemporal correlation database 17. The emotion recognition module 31 includes: a MobileNet-3D architecture 34, an FFT frequency domain analysis module 35, an audio emotion analysis and compensation module 36, and an MLP classifier 37; The spatiotemporal features of micro-expressions are extracted using the MobileNet-3D architecture 34. The respiratory waveform measured by the 60GHz millimeter-wave radar 30 is analyzed in the FFT frequency domain analysis module 35, and the 0.1-0.5 Hz anxiety feature frequency band is extracted. When the face is occluded, the audio emotion analysis compensation module 36 is activated, which reduces the blind spot by 60%. The micro-expression, respiratory frequency domain features and audio data are fused and input into the MLP classifier 37 to output the anxiety or pain level. The Clinical Decision Engine 23 constructs an interpretable risk decision tree and dynamically adjusts the warning threshold based on the patient's medical history. The Clinical Decision Engine 23 connects to the hospital's HIS system. When it detects "shortness of breath + anxiety", it automatically associates with medication records, distinguishes between postoperative pain and pulmonary embolism treatment procedures, and outputs treatment suggestions, analgesics, or emergency calls. Furthermore, for nighttime monitoring of orthopedic knee replacement patients (with a history of diabetes), traditional manual rounds cannot detect the combined physiological and psychological abnormalities caused by pain in real time. The intelligent ward rounds robot uses a polarized 3D camera 28 (Intel RealSense D455) for AR facial recognition and binds a magnetic wristband ID upon patient admission, synchronizing surgical records and basic disease information from the HIS system. During nighttime rounds, the wristband detects a heart rate that is consistently greater than 110 beats / min, triggering a yellow alert. The robot then activates a 60 GHz millimeter-wave radar 30 (Infineon BGT60TR13C) to monitor a respiratory rate of 28 breaths / min. The polarized 3D camera 28 captures micro-expressions such as frowning and drooping corners of the mouth (at a frequency of 0.5Hz), and the environmental sensor array 29 measures a carbon dioxide concentration of 1200 ppm, exceeding the threshold of 1000 ppm. Edge computing unit 12 (Jetson AGX Orin) executes the spatiotemporal feature fusion algorithm: def spatio_temporal_fusion(vital_signs, emotion_data, env_data): # Time alignment: performing a sliding average with a 5-second window. aligned_data = KalmanFilter(vital_signs) + DynamicTimeWarping(emotion_data) # Spatial weighting: Assigning different confidence levels based on patient-robot distance (UWB positioning) weight = 1 / (1 + distance^2) # The closer the data is, the higher its weight. return aligned_data * weight The spatiotemporal feature fusion algorithm first performs Kalman filtering on the vital signs data in a 5-second window and dynamic time warping on the emotional data to achieve time alignment. Then, it assigns different confidence levels to the data based on the distance between the patient and the robot (based on UWB positioning) (the closer the distance, the higher the weight), and finally obtains the weighted fused aligned data.

[0023] Interpretable risk decision tree linked to HIS records: The patient did not use analgesics 6 hours after surgery, and the system comprehensively judged it as "postoperative pain risk"; The robot simultaneously pushes medication recommendations to the nurses' station, plays soothing music to calm patients, and automatically turns on the room ventilation system to reduce carbon dioxide concentration.

[0024] Example 2: Please refer to Figure 1 , Figure 2 , Figure 3 and Figure 7 A smart ward inspection robot includes a robot body 1, which is equipped with a multimodal perception unit 4, an edge computing unit 12, a federated learning unit 10 and a decision execution unit 14. The multimodal sensing unit 4 includes components integrated within a smart bracelet and the robot body 1. The smart bracelet has a built-in RFID tag for real-time monitoring of the patient's heart rate, blood oxygen, and body movement frequency. The UV pulse disinfection unit 5 has a built-in excimer lamp 40. When the robot detects that a patient has left the ward for more than three minutes, it turns on the excimer lamp 40 to perform UV pulse disinfection on the ward. After the disinfection is completed, the disinfection log is uploaded to the hospital system. The robot body 1 is equipped with a negative pressure ward adaptation module 24. When the robot enters the negative pressure ward, the air pressure sensor in the environmental sensor array 29 detects an abnormal external air pressure. The negative pressure ward adaptation module 24 controls the robot to close the air inlet 3 and the air outlet 21 to adapt to the environment of the negative pressure ward. The robot body 1 has a wristband disinfection chamber 6 on its front. The robot automatically retrieves the wristband into the wristband disinfection chamber 6. Ultraviolet lamps 39 are installed on the inner walls around the wristband disinfection chamber 6 to disinfect the smart wristband and prevent cross-infection of wearable devices. The robot body 1 has an opening on its outer wall to install a lidar 18. Inside the robot body 1 is an avoidance module 11 that is connected to the lidar 18. The avoidance module 11 controls the robot to maintain a safe distance of more than 1.5 meters from the patient through the lidar 18. Furthermore, in the monitoring of MRSA-infected patients (severe pneumonia) in negative pressure isolation wards, manual rounds increase the risk of cross-infection and make it impossible to monitor the negative pressure environment in real time. The robot automatically loads a disposable nano-silver antibacterial shell (PLA plus nano-silver material) and achieves sub-meter-level positioning through LiDAR 18, SLAM mapping module 27 and UWB positioning module 25 to ensure a safe distance of ≥1.5 meters from the patient. The air pressure sensor detects a value of -18 Pa (normal range -5 to -30 Pa). When the air pressure is abnormal, an alarm is immediately triggered, and the negative pressure ward adaptation module 24 controls the air inlet 3 and air outlet 21 to close. When the patient leaves for a CT scan, the robot activates a 265 nm excimer lamp 40 to disinfect the ward, and the wristband disinfection chamber 6 collects and disinfects the wristbands to reduce the rate of nosocomial infection (third-party testing showed a sterilization rate of ≥99.2% against MRSA). The patient has a history of pulmonary embolism. The system detected shortness of breath and decreased blood oxygen levels, triggering a red alert. The robot simultaneously called the rescue team.

[0025] Example 3: Please refer to Figure 1 , Figure 2 , Figure 4 and Figure 8 A smart ward inspection robot includes a robot body 1, which is equipped with a multimodal perception unit 4, an edge computing unit 12, a federated learning unit 10 and a decision execution unit 14. The multimodal sensing unit 4 includes components integrated within a smart bracelet and the robot body 1. The smart bracelet has a built-in RFID tag for real-time monitoring of the patient's heart rate, blood oxygen, and body movement frequency. The robot body 1 integrates: a polarized 3D camera 28, a 60 GHz millimeter-wave radar 30, and an environmental sensor array 29. The polarized 3D camera 28 is used to capture micro-expressions to eliminate interference from glare in the ward. The 60 GHz millimeter-wave radar 30 is used to detect the respiratory rhythm of bedridden patients through the quilt. The environmental sensor array 29 includes a temperature sensor, a humidity sensor, a carbon dioxide concentration sensor, a light intensity sensor, and a barometric pressure sensor, which are used to collect environmental data outside the robot body 1. The emotion recognition module 31 includes: MobileNet-3D architecture 34, FFT frequency domain analysis module 35, audio emotion analysis and compensation module 36, and MLP classifier 37; The spatiotemporal features of micro-expressions are extracted using the MobileNet-3D architecture 34. The respiratory waveform measured by the 60GHz millimeter-wave radar 30 is analyzed in the FFT frequency domain analysis module 35, and the 0.1-0.5 Hz anxiety feature frequency band is extracted. When the face is occluded, the audio emotion analysis compensation module 36 is activated, which reduces the blind spot by 60%. The micro-expression, respiratory frequency domain features and audio data are fused and input into the MLP classifier 37 to output the anxiety or pain level. The robot body 1 is equipped with a central control unit 13, which is connected to the edge computing unit 12. The central control unit 13 controls the motor 15, decision execution unit 14, wristband disinfection chamber 6 and speaker 7 connected to the central control unit 13 based on the data analyzed by the edge computing unit 12. Furthermore, in the monitoring of Alzheimer's patients in the neurology ward, the risk of falls increases due to wandering at night. Manual rounds lack personalized psychological intervention. When a patient approaches the ward exit, the RFID tag in the smart wristband triggers the UWB fence at the ward exit. The robot autonomously navigates to the door to make a voice dissuasion and simultaneously notifies the nurse station, reducing the number of patients wandering at night. When the intelligent ward patrol robot detects that a patient has been sitting for a long time through the multimodal perception unit 4, and the emotion recognition module 31 analyzes and shows that the blank expression lasts for more than 5 minutes, the robot reads the patient's place of origin and age from the HIS system to generate a playlist, and plays personalized nostalgic music through the speaker 7 with dynamic volume adjustment and autonomous adjustment according to the ambient noise. When a patient is about to fall, the 60 GHz millimeter-wave radar 30 of the intelligent ward patrol robot detects a sudden change in the patient's position, immediately pops up the bed rail guard, and simultaneously notifies the nurses' station to check if the patient is injured.

[0026] Example 4: Please refer to Figure 2 , Figure 3 , Figure 4 , Figure 6 and Figure 8 A smart ward inspection robot includes a robot body 1, which is equipped with a multimodal perception unit 4, an edge computing unit 12, a federated learning unit 10 and a decision execution unit 14. The robot body 1 integrates: a polarized 3D camera 28, a 60 GHz millimeter-wave radar 30, and an environmental sensor array 29. The polarized 3D camera 28 is used to capture micro-expressions to eliminate interference from glare in the ward. The 60 GHz millimeter-wave radar 30 is used to detect the respiratory rhythm of bedridden patients through the quilt. The environmental sensor array 29 includes a temperature sensor, a humidity sensor, a carbon dioxide concentration sensor, a light intensity sensor, and a barometric pressure sensor, which are used to collect environmental data outside the robot body 1. The emotion recognition module 31 includes: MobileNet-3D architecture 34, FFT frequency domain analysis module 35, audio emotion analysis and compensation module 36, and MLP classifier 37; The emotion recognition algorithm of this invention extracts the spatiotemporal features of micro-expressions through the MobileNet-3D architecture 34, performs FFT frequency domain analysis on the breathing waveform measured by the 60 GHz millimeter-wave radar 30 in the FFT frequency domain analysis module 35, and extracts the 0.1-0.5 Hz anxiety feature frequency band. When the face is occluded, the audio emotion analysis compensation module 36 is activated, reducing the recognition blind zone by 60%. The micro-expression, breathing frequency domain features and audio data are fused and input into the MLP classifier 37 to output the anxiety or pain level. Furthermore, during rounds of oncology chemotherapy patients (whose faces are obscured due to lying on their side), traditional visual emotion recognition is limited by position. When the polarized 3D camera 28 of the intelligent ward rounds robot can only capture 30% of the face (due to lying on its side and obscuring the face), the robot activates a 60 GHz millimeter-wave radar 30 to penetrate the blanket and extract the breathing waveform (frequency 0.35 Hz, fluctuation amplitude ±2 cm). The robot also activates an audio emotion analysis and compensation module 36 to record the fundamental frequency of groans at 280 Hz (the frequency band of pain characteristics), thereby improving the accuracy of emotion recognition in obscured scenes. Inference is performed through federated learning unit 10, and edge nodes (robot and wristband) exchange model parameters through differential privacy federated learning framework. The lightweight MobileNet-3D architecture 34 fuses multi-source data, and the clinical decision engine 23 associates with HIS records. If the patient's last painkiller use time exceeds 4 hours, the robot pushes the suggestion of "morphine extended release tablet 30 mg" to the nurse station, which greatly improves the efficiency of ward rounds and reduces the workload of medical staff.

[0027] Example 5: Please refer to the figure. An intelligent ward inspection robot includes a robot body 1. The robot body 1 is equipped with a multimodal perception unit 4, an edge computing unit 12, a federated learning unit 10, and a decision execution unit 14. The robot body 1 integrates a polarized 3D camera 28, a 60 GHz millimeter-wave radar 30, and an environmental sensor array 29. The environmental sensor array 29 includes a temperature sensor, a humidity sensor, a carbon dioxide concentration sensor, a light intensity sensor, and a barometric pressure sensor, which are used to collect environmental data outside the robot body 1. The robot body 1 is equipped with a power consumption control unit 20, which is a hardware-optimized design. The robot surface is covered with a flexible perovskite solar film 19. Under the guidance of the power consumption control unit 20, the robot can automatically charge itself when patrolling by the window of the ward, increasing the battery life by 40%. The dynamic path planning unit 22 realizes autonomous navigation of the ward based on the built-in UWB positioning module 25 and SLAM mapping module 27; Furthermore, in remote hospital wards, which operate continuously for 48 hours, traditional patrol robots have a battery life of less than 8 hours, and frequent charging interrupts monitoring. In contrast, the intelligent ward patrol robot of this invention integrates a flexible perovskite solar film 19 (with a conversion efficiency of 23%) on its shell. The power consumption control unit 20 guides the robot to charge autonomously while patrolling the ward bedside through the light intensity sensor of the environmental sensor array 29. The SLAM mapping module 27 marks the sunlit area. The robot features dynamic power consumption control. In patrol mode, all sensors are activated to monitor high-risk patients. When a patient is in a stable sleep, the robot enters standby mode, with only the 60 GHz millimeter-wave radar 30 operating. The power consumption control unit 20 extends the continuous working time, increasing the number of beds a single robot can cover from 30 to 50, thus reducing maintenance costs.

[0028] Working principle: The robot scans the ward's 3D space using LiDAR 18, marking the locations of furnishings. Patient wristbands are bound to the robot with sub-meter-level tags, interfacing with the HIS system to automatically download patients' electronic medical records, linking wristband IDs to risk levels. A polarized 3D camera 28 is calibrated for focal length, a 60 GHz millimeter-wave radar 30 performs respiratory examination sensitivity testing, an ultraviolet pulse disinfection unit 5 detects power, a dynamic path planning unit 22 generates a risk map based on risk levels, a multimodal perception unit 4 collects multi-source data, and a spatiotemporal feature fusion unit 32 performs spatiotemporal alignment processing. An edge computing unit 12 constructs an interpretable risk decision tree for real-time computation using a lightweight emotion recognition module 31 and a clinical decision engine 23. A federated learning unit 10 locally trains the emotion recognition module 3 using a differential privacy federated learning framework to protect personal information. A decision execution unit 14 performs tiered intervention, calibrates the camera's white balance using a standard color chart, detects the intensity of the ultraviolet pulse disinfection unit 5, aggregates edge node data for global updates of the federated model, and charges via a solar panel 19 during rounds. When the battery is low, it automatically returns to the charging compartment and recharges via a power interface 8.

[0029] The dynamic path planning algorithm is the core algorithm, defined as follows: patrol priority equals the sum of α multiplied by the vital sign risk value, β multiplied by the emotional abnormality value, and γ multiplied by the environmental inadequacy factor. The coefficients α, β, and γ are optimized through training with historical data. The vital sign risk value is obtained by multiplying the absolute value of the difference between real-time heart rate and baseline heart rate by the baseline heart rate, then multiplying by 100%. The emotional abnormality value is the product of the micro-expression anxiety score and the respiratory disturbance index. Environmental inadequacy is calculated by multiplying the carbon dioxide concentration deviation value by the light discomfort coefficient.

[0030] The spatiotemporal feature fusion algorithm is used to process multi-source data. First, Kalman filtering is performed on the vital signs data with a 5-second window, and dynamic time warping is performed on the emotional data to achieve time alignment. Then, different data confidence levels are assigned according to the distance between the patient and the robot (based on UWB positioning) (the closer the distance, the higher the weight), and finally, the weighted fused aligned data is obtained.

[0031] In the emotion recognition algorithm, the emotion recognition module uses the MobileNet-3D architecture to extract the spatiotemporal features of micro-expressions. It combines the FFT frequency domain analysis module to analyze the breathing waveform measured by 60 GHz millimeter-wave radar and extract the 0.1-0.5 Hz anxiety feature frequency band. When the face is occluded, the audio emotion analysis compensation module is activated. Finally, the multi-source data is fused and the anxiety or pain level is output through the MLP classifier.

[0032] The federated learning algorithm constructs a differential privacy federated learning framework, which uses smart bracelets and robots as edge nodes to jointly train the emotion recognition module locally. When the nighttime light intensity is less than 50 lux or the environmental noise characteristics change by more than 20%, the local model is automatically retrained to maintain the emotion recognition accuracy of more than 95%.

[0033] Beneficial effects on clinical medicine: The multimodal sensing unit of this invention achieves coordinated monitoring of vital signs and emotions by using a smart bracelet for contact monitoring of heart rate, blood oxygen, and other vital signs, and integrating a polarized 3D camera and a 60GHz millimeter-wave radar into the robot body for non-contact capture of micro-expressions and respiratory rhythm. This combination overcomes the limitations of traditional single monitoring methods. For example, in nighttime monitoring of a patient (with a history of diabetes) after orthopedic knee replacement surgery, the robot detects a heart rate consistently greater than 110 beats / min via the smart bracelet, while the 60GHz millimeter-wave radar monitors a respiratory rate of 28 breaths / min, and the polarized 3D camera captures micro-expressions such as frowning. This comprehensive assessment identifies a complex abnormality caused by postoperative pain, triggering timely warnings and medication recommendations. This avoids potential omissions associated with traditional manual inspections, overcomes the bottleneck of synergistic monitoring between non-contact and contact methods, and improves the accuracy of patient status perception.

[0034] The emotion recognition module (MobileNet-3D architecture, inference speed up to 55FPS) of the edge computing unit of this invention, combined with the spatiotemporal feature fusion unit (fusion of multi-source data through algorithms such as Kalman filtering) and the clinical decision engine (interfacing with the hospital HIS system to construct a risk decision tree), can dynamically adjust the warning threshold and output targeted treatment suggestions. For example, when a patient is detected to have "shortness of breath + anxiety," the clinical decision engine will automatically link the medication records, distinguish between postoperative pain and pulmonary embolism, and output "analgesic administration" or "emergency call" suggestions respectively. If an MRSA-infected patient in a negative pressure isolation ward experiences shortness of breath and decreased blood oxygen, the robot immediately triggers a red warning and calls the rescue team, buying time for critical care treatment, realizing dynamic risk assessment and precise intervention, and optimizing the clinical decision-making process.

[0035] The dynamic path planning unit of this invention's decision-making execution unit is based on a "patrol priority algorithm" (comprehensive vital sign risk value, abnormal emotional value, and environmental inadequacy), and automatically adjusts the patrol frequency (15 minutes / 30 minutes / 60 minutes) according to the patient's risk level (high-risk / post-operative / normal), ensuring that resources are tilted towards high-risk patients. For example, in the monitoring of Alzheimer's patients in the neurology department, when a patient approaches the ward exit, the robot uses UWB positioning to quickly navigate to the door, make a voice dissuasion, and notify the nurse station, reducing the risk of falls caused by wandering at night. At the same time, for patients who sit for a long time and have a blank expression for 5 minutes, personalized nostalgic music is played for psychological intervention, demonstrating precise and personalized monitoring capabilities. Intelligent path planning and patrol frequency adjustment improve monitoring efficiency.

[0036] The robot's ultraviolet pulse disinfection unit (automatically disinfects 3 minutes after the patient leaves the ward) and wristband disinfection chamber (disinfects with ultraviolet light after wristband retrieval) achieve non-contact disinfection, reducing direct contact between medical staff and contaminants. For example, when a patient leaves a negative pressure isolation ward for a CT scan, the robot activates a 265nm excimer lamp to disinfect the ward, and the wristband disinfection chamber simultaneously disinfects the wristband. Tests show a MRSA sterilization rate of ≥99.2%, significantly reducing the risk of nosocomial infection; strengthening infection control and reducing the risk of cross-infection.

[0037] This invention features a negative pressure ward adaptation module (which closes the air inlet and outlet when abnormal air pressure is detected) and a power consumption control unit (flexible perovskite solar film, increasing battery life by 40%), enabling the robot to operate stably in complex environments (negative pressure wards) and resource-limited areas (such as remote hospitals). For example, in remote hospitals, the robot can autonomously charge via the solar film, combined with dynamic power consumption control (only activating the millimeter-wave radar when the patient is in stable sleep), significantly extending continuous operating time and increasing the number of beds covered by a single robot from 30 to 50, alleviating the problem of strained primary healthcare resources. It adapts to special environments and resource-constrained scenarios, improving medical coverage capabilities.

[0038] The differential privacy federated learning framework constructed by the federated learning unit of this invention uses smart bracelets and robots as edge nodes to jointly train the emotion recognition model locally, ensuring that the original data does not leave the local area, and balancing model optimization and privacy protection. For example, when optimizing the emotion recognition model in a multi-center collaborative manner, patient data from each hospital does not need to be uploaded to the central server. Training is completed only by exchanging model parameters, avoiding the risk of medical privacy leakage; protecting patient privacy and complying with medical data security regulations.

[0039] The meanings of technical terms and potentially ambiguous terms in the description of this invention HIS (Hospital Information System) is the core system used by hospitals to manage patient electronic medical records, medication records, and treatment processes. This invention proposes a clinical decision engine that interfaces with the HIS system to link patient medical history and medication records, assist in differentiating between postoperative pain and conditions such as pulmonary embolism, and output precise treatment recommendations.

[0040] MRSA: Methicillin-resistant Staphylococcus aureus, a pathogen resistant to multiple antibiotics, easily causing cross-infection in hospitals. The disinfection function of the robot in this invention achieves a sterilization rate of ≥99.2%, demonstrating its infection control capabilities.

[0041] Postoperative pain: Acute pain caused by surgical trauma is often accompanied by physiological reactions such as increased heart rate and anxiety.

[0042] Pulmonary embolism: an acute condition in which blood vessels in the lungs are blocked by a thrombus, manifested as shortness of breath, chest pain, etc. In this invention, the robot distinguishes between the two by combining "shortness of breath + anxiety" with medication records from the HIS system, thus avoiding misdiagnosis.

[0043] Alzheimer's disease: a progressive neurodegenerative disease in which patients may experience memory impairment, nighttime wandering, and other behaviors. In this invention, the robot monitors the patient's activities (such as approaching the ward exit) and emotions (such as a blank expression) to provide safety protection (such as verbal dissuasion) and personalized intervention (such as playing nostalgic music).

[0044] SLAM mapping module: Simultaneous Localization and Mapping (SLAM) is a technology that enables robots to autonomously create maps and locate their own positions in unknown ward environments. It is used to mark furnishings and sunlit areas and supports autonomous navigation.

[0045] UWB Positioning Module: Ultra-Wideband positioning technology achieves sub-meter level high-precision positioning through an extremely wide frequency bandwidth. It is used to track the real-time position of the robot and the patient (wristband) to ensure a safe distance (≥1.5 meters) and path planning.

[0046] Dynamic path planning algorithm: The robot calculates the patrol priority based on the patient's risk level and real-time data. The formula is: Patrol priority = α·signal risk value + β·emotional abnormality value + γ·environmental inadequacy. The coefficients are optimized through historical data and the patrol frequency is automatically adjusted according to the patient type (high-risk / post-operative / normal) (15 / 30 / 60 minutes / time).

[0047] Contact module: Smart bracelet (with built-in RFID tag) to monitor heart rate, blood oxygen, and body movement frequency.

[0048] Non-contact modules: polarized 3D camera (eliminates reflections and captures micro-expressions), 60 GHz millimeter-wave radar (penetrates blankets to detect breathing rhythm), and environmental sensor array (environmental data such as temperature, humidity, and carbon dioxide).

[0049] Polarized 3D camera: Utilizing the polarization properties of light to filter out interference from ward lights, glass reflections, etc., it accurately captures patients' facial micro-expressions (such as frowning and drooping corners of the mouth), providing visual data for emotion recognition.

[0050] 60 GHz millimeter-wave radar: A radar operating in the 60 GHz frequency band with strong penetration (it can penetrate cotton blankets and clothing) for non-contact detection of the respiratory rhythm (frequency and amplitude) of bedridden patients, especially suitable for nighttime or obstructed scenarios.

[0051] Environmental sensor array: A combination module integrating temperature, humidity, carbon dioxide concentration, light intensity, and air pressure sensors, used to monitor the ward environment (e.g., excessively high carbon dioxide concentration indicates insufficient ventilation), and to provide data for robot autonomous charging (light guidance) and adaptation to negative pressure wards (air pressure detection).

[0052] Edge computing unit: A computing module deployed locally on the robot, which analyzes multimodal data in real time through a processor (without uploading to the cloud) to achieve low-latency response (such as AR face recognition and emotion recognition), ensuring real-time performance and privacy in medical scenarios.

[0053] Federated learning unit: Multiple edge nodes (robots, smart bracelets) train models locally, exchanging only parameters without uploading the original data, achieving collaborative training with "data not leaving the local machine".

[0054] Differential privacy: Protects patient privacy by adding noise to prevent backtracking of individual data.

[0055] MobileNet-3D architecture: A lightweight 3D convolutional neural network used to process micro-expression dynamic features (video sequences). After TensorRT quantization, the inference speed reaches 55 FPS (frames per second), meeting the requirements of real-time emotion recognition.

[0056] FFT Frequency Domain Analysis Module: Based on Fast Fourier Transform, it transforms the breathing waveform from the time domain (time variation) to the frequency domain (frequency characteristics), extracting anxiety-related frequency bands of 0.1-0.5 Hz to assist in emotion assessment.

[0057] Kalman filtering: processes sensor noise and optimizes data accuracy; Dynamic time warping: Aligning the timelines of data from different sources (such as the time difference between heart rate and micro-expressions) to achieve weighted fusion of multi-source data. This invention improves the consistency of "vital signs-emotion-environment" data.

[0058] MLP classifier: Multi-Layer Perceptron, which integrates data such as micro-expressions, respiratory frequency domain, and audio to output the level of anxiety or pain.

[0059] Ultraviolet pulse disinfection unit (excimer lamp): Built-in 265 nm excimer lamp, which automatically turns on 3 minutes after the patient leaves the ward to perform ultraviolet pulse disinfection on the ward. The disinfection log is uploaded to the hospital system, which is efficient and traceable.

[0060] Wristband disinfection chamber: After the smart wristband is recycled, the chamber is disinfected by built-in ultraviolet lamps to avoid cross-infection caused by the reuse of wristbands. It is especially suitable for patients with infectious diseases.

[0061] Flexible perovskite solar film: High-efficiency solar cell material (conversion efficiency 23%), covering the robot surface, can autonomously charge by window-side light under the guidance of the power consumption control unit, increasing battery life by 40%, suitable for long-term operation in remote hospitals.

[0062] Clinical Decision Engine (Interpretable Risk Decision Tree): A rule-based model that dynamically adjusts warning thresholds based on patient history (e.g., lower heart rate warning thresholds for diabetic patients) and outputs interpretable treatment recommendations (e.g., analgesic administration, emergency call), which are consistent with clinical logic.

[0063] Edge nodes: refer to local devices (smart bracelets, robots) in federated learning.

[0064] Spatiotemporal window: The range of data collection in time (such as one minute) and space (such as coordinates of a ward) used to align multi-source data.

[0065] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. An intelligent ward patrol robot, characterized by: It includes a robot body (1), which is equipped with a multimodal perception unit (4), an edge computing unit (12), a federated learning unit (10) and a decision execution unit (14). The robot body (1) has a human-computer interaction interface (2) installed on the front. The human-computer interaction interface (2) is a touch screen. A speaker (7) is installed on the upper outer wall of the robot body (1). A power interface (8) is opened on the lower outer wall of the robot body (1). The power interface (8) is used to charge the battery (16) installed inside the robot body (1). An air inlet (3) and an air outlet (21) are opened at the top and bottom of the robot body (1), respectively. A universal wheel (9) driven by a motor (15) is installed at the bottom of the robot body (1) to move the robot.

2. The intelligent ward patrol robot according to claim 1, characterized in that: The multimodal sensing unit (4) includes components integrated within the smart bracelet and the robot body (1). The smart bracelet has a built-in RFID tag for real-time monitoring of the patient's heart rate, blood oxygen and body movement frequency. The robot body (1) integrates a polarized 3D camera (28), a 60 GHz millimeter-wave radar (30), and an environmental sensor array (29). The polarized 3D camera (28) is used to capture micro-expressions to eliminate the interference of ward reflections. The 60 GHz millimeter-wave radar (30) is used to detect the respiratory rhythm of bedridden patients through the quilt. The environmental sensor array (29) includes a temperature sensor, a humidity sensor, a carbon dioxide concentration sensor, a light intensity sensor, and a barometric pressure sensor, which are used to collect environmental data outside the robot body (1).

3. The intelligent ward patrol robot according to claim 1, characterized in that: The edge computing unit (12) is equipped with a processor (33). The processor (33) can analyze the data collected by the multimodal perception unit (4) and can also perform AR face recognition through a polarized 3D camera (28). The edge computing unit (12) is equipped with an emotion recognition module (31) and a spatiotemporal feature fusion unit (32). The robot body (1) is equipped with a spatiotemporal correlation database (17). The emotion recognition module (31) adopts a lightweight design and MobileNet-3D architecture (34), and its inference speed reaches 55 FPS after TensorRT quantization; The spatiotemporal feature fusion unit (32) aligns the spatiotemporal windows of vital sign data and emotion data, and achieves weighted fusion of multi-source data through Kalman filtering and dynamic time warping algorithm; The spatiotemporal correlation database (17) is interconnected with the edge computing unit (12). The spatiotemporal correlation database (17) is used to store the timestamps and spatial coordinate labels of vital signs-emotions-environment data.

4. The intelligent ward patrol robot according to claim 1, characterized in that: The federated learning unit (10) constructs a differential privacy federated learning framework, with the smart bracelet and robot as edge nodes respectively. The emotion recognition module (31) is trained locally through the differential privacy federated learning framework, and the original data does not leave the local area, thus protecting personal information. The federated learning unit (10) includes a model update mechanism that automatically triggers local model retraining when the nighttime light intensity is less than 50 lux or the environmental noise characteristics change by more than 20%, maintaining the emotion recognition accuracy greater than 95%.

5. The intelligent ward patrol robot according to claim 1, characterized in that: The decision execution unit (14) includes a dynamic path planning unit (22), a clinical decision engine (23), a voice interaction module (26), and an ultraviolet pulse disinfection unit (5). The dynamic path planning unit (22) realizes autonomous navigation of the ward based on the built-in UWB positioning module (25) and SLAM mapping module (27); The algorithm for dynamic path planning is defined as follows: Inspection Priority = alpha risk value + beta abnormality value + gamma inappropriate environment The coefficients α, β, and γ are optimized through training on historical data, and: Risk score = |Real-time heart rate - Baseline heart rate| / Baseline heart rate × 100%; Emotional anomaly score = Microexpression anxiety score × Respiratory disorder index; Environmental inadequacy = Deviation of carbon dioxide concentration × Light inadequacy coefficient; The dynamic path planning unit (22) can also automatically adjust the frequency of rounds based on the patient's risk level (postoperative / high-risk): High-risk patients: Check on them every 15 minutes; Postoperative patients: Check on them every 30 minutes; For ordinary patients: check on them once every 60 minutes; The clinical decision engine (23) constructs an interpretable risk decision tree and dynamically adjusts the warning threshold based on the patient's medical history. The clinical decision engine (23) connects to the hospital's HIS system. When "rapid breathing + anxiety" is detected, it automatically associates with medication records, distinguishes between postoperative pain and pulmonary embolism treatment procedures, outputs treatment suggestions, analgesics, or emergency calls. The ultraviolet pulse disinfection unit (5) has a built-in excimer lamp (40). When the robot detects that the patient has left the ward for more than three minutes, it turns on the excimer lamp (40) to perform ultraviolet pulse disinfection on the ward. After the disinfection is completed, the disinfection log is uploaded to the hospital system.

6. The intelligent ward rounds robot according to claim 3, characterized in that: The emotion recognition module (31) includes: MobileNet-3D architecture (34), FFT frequency domain analysis module (35), audio emotion analysis compensation module (36) and MLP classifier (37). The spatiotemporal features of micro-expressions are extracted using the MobileNet-3D architecture (34). The respiratory waveform measured by the 60GHz millimeter-wave radar (30) is analyzed in the FFT frequency domain analysis module (35), and the 0.1-0.5 Hz anxiety feature frequency band is extracted. When the face is occluded, the audio emotion analysis compensation module (36) is activated, and the blind spot is reduced by 60%. The micro-expression, respiratory frequency domain features and audio data are fused and input into the MLP classifier (37) to output the anxiety or pain level.

7. The intelligent ward patrol robot according to claim 1, characterized in that: The robot body (1) is equipped with a central control unit (13), which is connected to the edge computing unit (12). The central control unit (13) controls the motor (15), decision execution unit (14), wristband disinfection chamber (6) and speaker (7) connected to the central control unit (13) based on the data analyzed by the edge computing unit (12).

8. The intelligent ward round robot according to claim 1, characterized in that: The robot body (1) is equipped with a power consumption control unit (20) and a negative pressure ward adaptation module (24), which are optimized for hardware design. The robot's surface is covered with a flexible perovskite solar film (19). Guided by the power consumption control unit (20), the robot charges itself while patrolling the ward window, increasing its battery life by 40%. After the robot enters the negative pressure ward, when the air pressure sensor in the environmental sensor array (29) detects an abnormal external air pressure, the negative pressure ward adaptation module (24) controls the robot to close the air inlet (3) and air outlet (21) to adapt to the environment of the negative pressure ward.

9. The intelligent ward round robot according to claim 5, characterized in that: The robot body (1) has a wristband disinfection chamber (6) on the front. When the patient is discharged, the robot automatically retrieves the wristband to the wristband disinfection chamber (6). The outer wall of the wristband disinfection chamber (6) is equipped with a guide rail (38), which can move horizontally under the control of the central control unit (13). The inner walls around the wristband disinfection chamber (6) are equipped with ultraviolet lamps (39) for disinfecting the smart wristband and preventing cross-infection of wearable devices.

10. The intelligent ward round robot according to claim 1, characterized in that: The robot body (1) has an opening on its outer wall to install a lidar (18). Inside the robot body (1) is an avoidance module (11) that communicates with the lidar (18). The avoidance module (11) controls the robot to actively avoid the patient and maintain a safe distance of more than 1.5 meters through the lidar (18).

Images