Patient full-dimension intelligent nursing inspection system and method based on virtual reality technology

Abstract

The invention belongs to the technical field of intelligent nursing, and relates to a full-dimension intelligent nursing inspection system and method for a patient based on a virtual reality technology, and the system comprises a plurality of different types of sensors, virtual reality intelligent glasses, a mobile calculation unit, and a server. The sensor is arranged on an inspection site and is used for acquiring images and data of the inspection site; the virtual reality intelligent glasses are used for establishing a three-dimensional simulation display scene according to the image and data obtained by the sensor; the mobile calculation unit is arranged on the virtual reality intelligent glasses and is used for carrying out single-mode data calculation through an artificial intelligence model according to the image and data obtained by the sensor; and the server is used for fusing the calculation result of the single-mode data through a multi-mode analysis model to generate a full-dimensional intelligent nursing inspection result. The system can realize objectification and standardization of nursing evaluation, establish continuous and preventive monitoring, and achieve full-dimensional comprehensive evaluation.

Description

Technical Field

[0001] This invention relates to a patient intelligent nursing inspection system and method based on virtual reality technology, belonging to the field of intelligent nursing technology. Background Technology

[0002] Intelligent nursing rounds are a nursing model that integrates IoT, AI, and big data technologies. Through real-time monitoring, intelligent early warning, and automated tasks, it transforms the nursing approach from "passive response" to "proactive intervention." It utilizes sensors and wearable devices to collect patient health data, combines this with AI algorithms to analyze risks and optimize nursing plans, and leverages large language models to support semantic understanding and task decomposition, thereby improving nursing efficiency and patient safety. Currently, this model is widely used in hospitals, nursing homes, and communities, significantly improving the quality of care and reducing medical errors.

[0003] Existing intelligent nursing rounds methods suffer from low efficiency due to their heavy reliance on nurses' personal experience and manual operation, strong subjectivity in assessment, blind spots in monitoring, insufficient data collaboration, and difficulty in assessing special patient groups. Specifically, the data collection process requires frequent switching between equipment and recording systems, and manual transcription of vital signs, estimation of infusion volume, and description of skin condition, which is not only time-consuming and fraught with errors due to human factors; assessment of catheter patency, pain level (especially for patients who cannot communicate), pressure ulcer staging, and edema degree determination rely on nurses' experience, lacking objective and quantifiable standards, resulting in poor consistency; process monitoring struggles to cover the entire infusion and transfusion process, key details such as patient pain expressions and early skin changes, and abnormal situations rely on passive discovery, leading to delayed responses and significant safety risks; information silos are severe, with vital sign data, pain records, skin condition, and treatment information usually stored in isolation, making it impossible to reveal potential clinical problems through multimodal correlation analysis (such as the correlation between increased heart rate and local redness and swelling indicating infection risk), and lacking multimodal correlation analysis capabilities; in addition, it is difficult to accurately and continuously assess the pain perception, comfort, and physiological needs of special groups such as sedated patients, dementia patients, and infants. Summary of the Invention

[0004] To address the aforementioned problems, the purpose of this invention is to provide a comprehensive intelligent nursing inspection system, method, and readable medium for patients based on virtual reality technology. This system enables the objectification and standardization of nursing assessments, establishes continuous and preventative monitoring, and achieves a comprehensive, multi-dimensional evaluation.

[0005] To achieve the above objectives, the present invention proposes the following technical solution: a patient comprehensive intelligent nursing inspection system based on virtual reality technology, comprising: a multimodal sensor array, virtual reality smart glasses, a mobile computing unit, and a server; the multimodal sensor array includes several different types of sensors, which are deployed at the inspection site to acquire images and data of the inspection site; the virtual reality smart glasses are used to establish a three-dimensional real-scene fusion display interface based on the images and data acquired by the sensors; the mobile computing unit is integrated into the virtual reality smart glasses and is used to perform real-time single-modal data analysis based on the images and data acquired by the sensors using a lightweight artificial intelligence model; the server is used to generate a comprehensive inspection report and risk warning by fusing the results of the single-modal data analysis through the multimodal analysis model.

[0006] Furthermore, the multimodal sensor array includes a high-definition RGB camera, an infrared thermal imager, a multispectral sensor, an inertial measurement unit (IMU), a microphone, and a 3D depth sensor; the virtual reality smart glasses support voice commands and gesture recognition, allowing users to initiate rounds, confirm assessment results, and enter nurse notes via voice or gesture.

[0007] Furthermore, the images and data collected during the on-site inspection include: patient heart rate, blood pressure, blood oxygen, respiration, and body temperature data; catheter location, patency, fixation, and skin around the outlet; pump rate, infused volume, and drug name; pain-related facial movements to generate a quantitative pain index; identification of pressure sores, abrasions, and lacerations, and automatic staging based on color, depth, and tissue type; assessment of subcutaneous tissue health status through multispectral or thermal imaging; automatic identification and assessment of the healing status of surgical incisions, stomas, and drainage tube outlets; identification of patient position; and inspection of bed rails and restraint straps.

[0008] Furthermore, the mobile computing unit includes a risk comprehensive scoring module, which is used to compare the data with the safety thresholds of various indicators and discover the relationship between the data; perform a risk comprehensive score based on the comparison results and the relationship between the data; determine the risk level based on the risk comprehensive score; and issue an alarm based on the risk level.

[0009] Furthermore, the risk levels include immediate treatment, key attention, and routine attention. Immediate treatment includes complete catheter dislodgement, severe transfusion reaction, and extremely abnormal vital signs, marked in red. Key attention includes high pain index, stage II or higher pressure ulcers, and large deviations in infusion rate, marked in yellow. Routine attention includes stage I pressure ulcers, mild skin abnormalities, and environmental warnings, marked in blue.

[0010] Furthermore, the multimodal analysis model includes a refined skin lesion classification model and a multimodal data fusion model, which are used for big data storage, in-depth analysis, trend prediction, electronic medical record integration and system management.

[0011] Furthermore, the server generates improvement suggestions based on the results of the comprehensive risk score; it tracks high-risk items in the comprehensive risk score, performs automatic image comparison during subsequent rounds, quantifies the comparison results, and encrypts and stores all time-series data, image snapshots, assessment results, and operation logs to form a dynamic digital twin of patient care. By integrating patient time-series data, assessment snapshots, and operation records, a dynamically updated holographic health record is generated.

[0012] This invention also discloses a method for comprehensive intelligent patient care inspection based on virtual reality technology, used in any of the aforementioned comprehensive intelligent patient care inspection systems based on virtual reality technology, comprising the following steps: initiating inspection by voice or gesture, matching sensors and devices with a 3D map; reading data from sensors and devices; comparing the data with safety thresholds for various indicators and identifying relationships between the data; performing a comprehensive risk score based on the comparison results and the relationships between the data; generating a structured electronic record from the results of the comprehensive risk score and synchronizing it to a server.

[0013] Furthermore, the data from the sensors and devices includes: vital sign recognition, locking the monitor screen, and real-time extraction of heart rate, blood pressure, blood oxygen, respiration, and body temperature; identification of catheter types, assessment of catheter location, patency, fixation, and skin around the exit point; identification of pump rate, infused volume, and drug name; identification of pain-related facial movements and generation of a quantitative pain index; identification of pressure sores, abrasions, and lacerations, and automatic staging based on color, depth, and tissue type; assessment of subcutaneous tissue health status through multispectral or thermal imaging; automatic identification and assessment of the healing status of surgical incisions, stomas, and drainage tube exit points; identification of patient position; and inspection of bed rails and restraint straps.

[0014] Furthermore, based on the results of the comprehensive risk score, improvement suggestions are generated, and nurses can quickly confirm or correct these suggestions via voice or gestures. High-risk items in the comprehensive risk score are tracked, and automatic image comparison is performed during subsequent rounds. The comparison results are quantified, and all time-series data, image snapshots, assessment results, and operation logs are encrypted and stored to form a dynamic digital twin of patient care.

[0015] The technical solution of the present invention has at least the following technical effects or advantages: The solution of the present invention realizes full-dimensional automated inspection, which can simultaneously complete multi-dimensional detection of vital signs, catheter safety, skin damage, pain assessment, environmental safety and other aspects in a single inspection through multi-modal sensors and AI visual analysis, thereby achieving a leap in nursing inspection efficiency, reducing the time for a single comprehensive inspection from 15-20 minutes to 3-5 minutes, freeing up to 70% of nursing record time, and improving clinical response speed.

[0016] This invention significantly improves the consistency of nursing assessments through quantitative evaluation, reducing quality fluctuations caused by subjective differences.

[0017] This invention, through a continuous monitoring and early warning system, transforms the discovery of adverse reactions from reactive to proactive, reducing the incidence of potential adverse events such as catheter-related infections, pressure sores, falls, and medication errors by more than 20%.

[0018] This invention provides nurses and doctors with an unprecedented panoramic view through multi-dimensional data correlation analysis, supporting more accurate clinical decision-making. The resulting digital twin nursing records integrate time-series data, image snapshots, assessment results, and operation logs to form traceable and analyzable holistic patient health records, which serve as a valuable teaching case library and clinical research database.

[0019] This invention achieves early risk identification through multimodal correlation analysis, such as heart rate + facial expression + body position, and supports graded early warning and intelligent intervention suggestions. More frequent and accurate assessments mean more timely intervention, which directly improves patient comfort and satisfaction. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of a patient omnidimensional intelligent nursing inspection system based on virtual reality technology in one embodiment of the present invention; Figure 2 This is a flowchart of a method for comprehensive intelligent nursing inspection of patients based on virtual reality technology in one embodiment of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention is described in detail through specific embodiments. However, it should be understood that the specific embodiments are provided only for a better understanding of the present invention and should not be construed as limiting the present invention. In the description of the present invention, it should be understood that the terminology used is for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0022] To address the shortcomings of existing technologies, such as the lack of objective quantitative criteria for assessment, reliance on passive discovery of anomalies leading to delayed responses and significant safety risks, and severe information silos (where vital signs data, pain records, skin condition, and treatment information are isolated from each other), this invention proposes a comprehensive intelligent patient care inspection system, method, and readable medium based on virtual reality technology. The system includes: several different types of sensors, virtual reality smart glasses, a mobile computing unit, and a server. Sensors are deployed at the inspection site to acquire images and data. The virtual reality smart glasses are used to create a three-dimensional simulation display scene based on the images and data acquired by the sensors. The mobile computing unit, deployed on the virtual reality smart glasses, is used to perform single-modal data calculations based on the images and data acquired by the sensors using an artificial intelligence model. The server is used to fuse the calculation results of the single-modal data through a multimodal analysis model to generate comprehensive intelligent nursing inspection results. Through wearable smart devices, this system empowers nurses to automate, intelligently, and comprehensively assess and record patient vital signs, treatment safety, comfort, and overall condition within a natural workflow. The invention will be described in detail below with reference to the accompanying drawings.

[0023] Example 1 This embodiment discloses a comprehensive intelligent patient care monitoring system based on virtual reality technology, such as... Figure 1 As shown, it adopts an edge-cloud collaborative architecture, integrating virtual reality (VR) / augmented reality (AR), computer vision, multimodal sensing, and artificial intelligence (AI) technologies to achieve a closed loop of perception-evaluation-decision-recording. Specifically, it includes: a multimodal sensor array, virtual reality smart glasses, a mobile computing unit, and a server. A multimodal sensor array, comprising several different types of sensors, is deployed at the inspection site to acquire images and data. The array includes a high-definition RGB camera, an infrared thermal imager, a multispectral sensor, an inertial measurement unit (IMU), a microphone, and a 3D depth sensor. In this embodiment, the system uses multispectral and thermal imaging fusion to exceed the limits of the naked eye, enabling early identification of subcutaneous hematoma, tissue oxygenation, and occult inflammation. All raw images are anonymized on the device (e.g., real-time face blurring), and only the analyzed structured feature data is uploaded to the cloud.

[0024] Images and data collected during site visits include: patient heart rate, blood pressure, blood oxygen saturation, respiration, and body temperature; catheter location, patency, fixation, and skin around the exit point; pump rate, infused volume, and medication name; pain-related facial movements, generating a quantified pain index; identification of pressure sores, abrasions, and lacerations, and automatic staging based on color, depth, and tissue type; assessment of subcutaneous tissue health using multispectral or thermal imaging; automatic identification and assessment of the healing status of surgical incisions, stomas, and drainage tube exit points; identification of patient position; and inspection of bed rails and restraints.

[0025] Virtual reality (VR) smart glasses establish a 3D reality fusion display interface through 3D depth sensors and a display module. The VR smart glasses support voice commands and gesture recognition, allowing nurses to initiate rounds, confirm assessment results, and enter nurse notes via voice or gesture. The VR smart glasses worn by nurses serve as the first-person perspective perception and interaction terminal. The VR smart glasses utilize lightweight AR glasses similar to Microsoft HoloLens 2 or Magic Leap 2, integrating the aforementioned sensors. A visual perception layer for establishing the 3D simulated display scene is constructed: OpenCV and ARKit / ARCore are used for spatial positioning and image acquisition.

[0026] The mobile computing unit, integrated into the virtual reality smart glasses, is used for real-time single-modal data analysis based on images and data acquired by sensors, using a lightweight artificial intelligence model. Specifically, it runs the lightweight AI model, which is a highly efficient AI model operating in environments with limited computing resources. By reducing the number of parameters, optimizing the architecture, and using quantization techniques, it can achieve powerful inference capabilities on ordinary hardware, making it suitable for scenarios such as mobile devices, embedded systems, and edge computing. For example, it can be a dedicated processor or a smartphone, enabling real-time data preprocessing, key feature extraction, and timely alerts. For instance, the YOLO model can be used for duct detection, and MobileNet for facial expression classification. In this embodiment, the mobile computing unit uses a Qualcomm Snapdragon XR2 or equivalent platform to provide sufficient AI computing power.

[0027] The mobile computing unit includes: an object detection module, an image segmentation module, a classification and regression module, a multimodal fusion module, and a business logic and interaction layer.

[0028] The target detection module can use YOLOv7 to locate and detect catheters, pump devices, and skin damage areas in real time. For example, taking skin damage as an example, a photo or video of the skin can be input into the YOLOv7 model. Through feature extraction, the target skin can be distinguished from the background, and the target skin can be compared with normal skin and various types of damaged skin to determine its damage type.

[0029] The image segmentation module can use U-Net or DeepLabv3+ to accurately delineate skin lesions and duct outlines. In this embodiment, the U-Net model is used for segmentation and staging of the skin lesion area. That is, the boundary points of the skin lesion are determined by the U-Net model, and the boundary points are connected to obtain the boundary line of the skin lesion, thereby segmenting the skin lesion area.

[0030] Classification and regression can employ ResNet and Vision Transformer for facial expression classification, pressure ulcer staging, and vital sign digit recognition. For example, this embodiment uses the Vision Transformer model for facial expression pain grading detection, determining whether the patient exhibits expressions such as frowning or clenching their teeth, which are likely to occur when in pain.

[0031] Multimodal fusion can employ neural networks with an attention mechanism to fuse sequential data from visual, physiological, and speech data.

[0032] The business logic and interaction layer is used to manage task flows, record generation, and alert rules.

[0033] The mobile computing unit includes a risk comprehensive scoring module, which compares data with the safety thresholds of various indicators and discovers the relationships between data; based on the comparison results and the relationships between data, a risk comprehensive score is performed, and an alarm is issued based on the risk comprehensive score and the risk level.

[0034] Risk levels are categorized into immediate treatment, critical monitoring, and routine monitoring. Immediate treatment includes complete catheter dislodgement, severe transfusion reactions, and extremely abnormal vital signs, marked in red. Critical monitoring includes high pain levels, stage II or higher pressure ulcers, and significant deviations in infusion rates, marked in yellow. Routine monitoring includes stage I pressure ulcers, mild skin abnormalities, and environmental warnings, marked in blue.

[0035] The server, which can be an internal hospital server or a private cloud, is used to generate comprehensive inspection reports and risk warnings by fusing the results of single-modal data analysis through a multimodal analysis model. The multimodal analysis model on the server is more complex than the AI ​​model in the mobile computing unit, responsible for big data storage, deep analysis, trend prediction, electronic medical record integration, and system management, generating comprehensive processing results and recommendations. This multimodal analysis model can be a refined skin lesion classification model, a multimodal data fusion model, etc., but is not limited to these. The server-side multimodal analysis model system learns the patient's normal vital sign fluctuations and basic skin color, thus becoming more sensitive to abnormal changes. It supports cloud updates, continuously integrating new AI assessment models, such as future identification of delirium and signs of malnutrition. Based on the results of the comprehensive risk assessment, the server generates improvement suggestions; it tracks high-risk items in the comprehensive risk assessment, performs automatic image comparison during subsequent rounds, quantifies the comparison results, and encrypts and stores all time-series data, image snapshots, assessment results, and operation logs to form a dynamic digital twin of patient care. By integrating patient time-series data, assessment snapshots, and operation records, a dynamically updated holographic health record is generated.

[0036] In this embodiment, the server deploys AI microservices based on Kubernetes containerization and integrates with HIS and EMR through the HL7 / FHIR standard. The multimodal analysis model includes a refined skin lesion classification model and a multimodal data fusion model, used for big data storage, deep analysis, trend prediction, electronic medical record integration, and system management.

[0037] In this embodiment, the nursing rounds system is connected to the hospital's HIS (Hospital Information System) / EMR (Electronic Medical Record System) system through the HL7 / FHIR (Fast Healthcare Interoperability Resource) protocol, enabling one-click synchronization of assessment results, intelligent verification of medical orders, and risk warning push.

[0038] In this embodiment, the nursing rounds system can free nurses' hands and reduce their cognitive load: through automated data collection and recording, nurses are freed from tedious manual labor, enabling objective and standardized assessment of nursing care. AI visual analysis provides quantitative indicators, reducing subjective differences. Key indicators are monitored 24 / 7 to provide early warnings of abnormalities. A single rounds simultaneously complete multi-dimensional assessments of physiological, treatment, subjective feelings, and environmental factors. A digital twin patient profile is constructed: integrating multi-source time-series data to form a traceable and analyzable holographic digital medical record.

[0039] Example 2 Based on the same inventive concept, this embodiment discloses a method for comprehensive intelligent patient care monitoring based on virtual reality technology, applicable to any of the aforementioned comprehensive intelligent patient care monitoring systems based on virtual reality technology, such as... Figure 2 As shown, it includes the following steps: S1 can be activated by voice or gesture to start the inspection and match sensors, devices and 3D maps; Nurses wearing the device enter the ward and initiate rounds via voice or gesture. Based on the patient schedule, a patient list is pushed to the nurse. The nurse observes the patient or bed and uses visual recognition to match it with a pre-built 3D map of the ward to confirm the patient's identity. Subsequently, the nurse is guided to perform a systematic panoramic scan, rapidly building or updating a visual map of the patient's bedside environment using SLAM (Simultaneous Localization and Mapping) technology.

[0040] S2 reads data from multimodal sensor arrays and devices.

[0041] Within the nurse's natural line of sight during rounds, the system processes data from multiple sensors and devices in parallel.

[0042] The data from sensors and devices includes: Vital signs recognition locks the monitor screen and extracts values ​​such as heart rate, blood pressure, blood oxygen, respiration, and body temperature in real time.

[0043] Identify the type of catheter, such as infusion tubing, drainage tubing, and urinary catheter. Assess catheter position, checking for dislodgement or displacement. Assess catheter patency, noting any kinks, air bubbles, or deposits. Check catheter fixation, ensuring the dressing is clean and secure, and inspecting the skin around the exit site.

[0044] Intelligent monitoring of the catheter infusion process: such as identifying the rate of the infusion pump or blood transfusion pump, the amount infused, and the name of the medication; visual analysis of the tubing for leaks, whether the drip chamber level is normal, and whether there are clots in the infusion tubing. Automatic verification of blood bag label information against doctor's orders.

[0045] Pain assessment aids: Through facial expression analysis, pain-related facial movements are identified, such as frowning, eye closing, and deepening of the nasolabial folds, generating a quantitative pain index. This is correlated with concurrent vital signs (such as increased heart rate variability) to improve the reliability of the assessment.

[0046] Panoramic Skin and Wound Assessment: Automatically locates and assesses exposed skin areas in panoramic scan images, identifies pressure ulcers, abrasions, and lacerations, and automatically stages them based on color, depth, and tissue type, such as stage I-IV pressure ulcers. It measures lesion area and redness / swelling extent, detects edema depth via 3D vision, and assesses subcutaneous tissue health using multispectral or thermal imaging, including the risk of deep tissue damage and areas of inflammation.

[0047] Special site tracking: Automatically identifies and assesses the healing status of surgical incisions, stomas, and drainage tube exits.

[0048] Overall condition and environmental perception: Identify the patient's position, such as whether the patient is supine or semi-recumbent, and assess the patient's ability to move independently.

[0049] Safety and environmental inspection: Inspect the condition of the bed rails and restraints, and identify environmental risks such as slippery floors and debris.

[0050] S3 compares the data with the safety thresholds of each indicator and discovers the relationships between the data.

[0051] S4 conducts a comprehensive risk score based on the comparison results and the relationships between the data.

[0052] The data from the previous step, spanning multiple dimensions, is input into the multimodal clinical inference engine for single-dimensional threshold comparison and multi-dimensional correlation analysis. Single-dimensional threshold comparison compares the data with the safety thresholds of each indicator, while multi-dimensional correlation analysis reveals relationships between the data. For example: Increased respiratory rate + orthopneic posture + lower extremity edema → suggests "risk of worsening heart failure".

[0053] Painful expression + protective posture of limbs + local redness, swelling and heat of the skin → suggests "local infection or acute injury".

[0054] Comprehensive risk score: Automatically calculates and updates risk scores such as fall / bed fall risk score and stress injury risk score.

[0055] S5 generates a structured electronic record of the risk comprehensive scoring results and synchronizes it to the server.

[0056] Based on the comprehensive risk assessment results, a time-stamped, structured electronic record is compiled and synchronized to the hospital information system (HIS) with a single click. The comprehensive risk assessment results include numerical values, classifications, and image annotations.

[0057] Based on the comprehensive risk assessment results, information is pushed out according to risk level. Risk levels include immediate treatment, key attention, and routine attention. Immediate treatment includes complete catheter dislodgement, severe transfusion reactions, and extremely abnormal vital signs, marked in red; key attention includes high pain index, stage II or higher pressure ulcers, and large deviations in infusion rate, marked in yellow; routine attention includes stage I pressure ulcers, mild skin abnormalities, and environmental warnings, marked in blue.

[0058] Improvement suggestions are generated based on the results of the comprehensive risk score, and nurses can quickly confirm or correct the improvement suggestions through voice or gestures; For items with high risk in the comprehensive risk score, such as pressure sores and surgical incisions, we will track them and perform automatic image comparison during subsequent inspections. We will quantify the comparison results and display changes, such as a 30% reduction in the area of ​​redness and swelling compared to 3 days ago.

[0059] All time-series data, image snapshots, evaluation results, and operation logs are encrypted and stored to form a dynamic digital twin of patient care, supporting historical review, efficacy evaluation, and scientific research analysis.

[0060] Example 3 Based on the same inventive concept, the usage method of the system in Example 1 is explained and verified, taking the morning check-up handover in the Intensive Care Unit (ICU) as an example.

[0061] Nurse A, wearing glasses, enters and looks at the patient. The system confirms the patient's identity, displaying the message: "Patient X, Day 2 post-surgery."

[0062] My gaze swept across the monitor: the system OCR showed: "HR 102, BP 145 / 90, SpO2 96%", and indicated "heart rate increased by 15% from last night's baseline".

[0063] Visual inspection of the internal jugular vein catheter: the catheter is highlighted as a green virtual line, and the voice prompt says: "The central venous catheter is well secured and there is no leakage."

[0064] Visual inspection of the analgesia pump: OCR showed: "Sufentanil, rate 0.5 mcg / kg / h", system assessment: "Analgesia pump is running, parameters meet doctor's orders".

[0065] Facial expression analysis: The pain index was calculated in real time to be 4 / 10 (moderate). The system prompts: "The patient has moderate pain, evaluation is recommended."

[0066] The nurse performed a rapid full-body skin scan: the system marked a "2x3cm Stage I pressure ulcer (does not fade upon pressure)" on the sacrum and coccyx, and marked "mild pitting edema" on the right ankle.

[0067] Environmental check: The system confirms that the bed rails are up.

[0068] Comprehensive report generation: The system generates a voice summary and pushes the structured record to the electronic medical record: "Patient X morning rounds: vital signs are stable but heart rate is slightly fast, catheter is safe, analgesia is in progress, there is moderate pain and risk of stage I pressure ulcers, skin tracking task has been created." Nurse B handover: The patient's "digital twin" can be accessed to visually compare all time-series data and images.

[0069] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0070] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0071] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0072] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific embodiments of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention. The above content is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the protection scope of the claims.

Claims

1. A comprehensive intelligent patient care monitoring system based on virtual reality technology, characterized in that, include: Multimodal sensor arrays, virtual reality smart glasses, mobile computing units, and servers; The multimodal sensor array includes several different types of sensors, which are deployed at the inspection site to acquire images and data of the inspection site; The virtual reality smart glasses are used to establish a three-dimensional real-scene fusion display interface based on the images and data obtained by the sensors. The mobile computing unit, integrated into the virtual reality smart glasses, is used to perform real-time single-modal data analysis based on the images and data obtained by the sensors using a lightweight artificial intelligence model. The server is used to generate a comprehensive inspection report and risk warning by fusing the results of the single-modal data analysis through a multimodal analysis model.

2. The patient omnidimensional intelligent nursing inspection system based on virtual reality technology as described in claim 1, characterized in that, The multimodal sensor array includes a high-definition RGB camera, an infrared thermal imager, a multispectral sensor, an inertial measurement unit (IMU), a microphone, and a 3D depth sensor; the virtual reality smart glasses support voice commands and gesture recognition, allowing users to initiate rounds, confirm assessment results, and enter nurse notes via voice or gesture.

3. The patient omnidimensional intelligent nursing inspection system based on virtual reality technology as described in claim 2, characterized in that, The images and data from the inspection site include: The patient's heart rate, blood pressure, blood oxygen, respiration, and body temperature data; The location, patency, fixation, and skin around the catheter exit point; Pump rate, infused volume, and drug name; Pain-related facial movements are used to generate a quantitative pain index. It identifies pressure sores, abrasions, and lacerations, and automatically stages them based on color, depth, and tissue type. Assess subcutaneous tissue health using multispectral or thermal imaging; Automatically identify and assess the healing status of surgical incisions, stomas, and drainage tube exits; Identify patient position; Check the bed rails and restraints.

4. The patient omnidimensional intelligent nursing inspection system based on virtual reality technology as described in claim 3, characterized in that, The mobile computing unit includes a risk comprehensive scoring module, which is used to compare the data with the safety thresholds of various indicators and discover the relationships between the data. Based on the comparison results and the relationships between the data, a comprehensive risk score is calculated, and an alarm is issued according to the risk level determined by the comprehensive risk score.

5. The patient omnidimensional intelligent nursing inspection system based on virtual reality technology as described in claim 4, characterized in that, The risk levels are categorized into immediate treatment, critical attention, and routine attention. Immediate treatment includes complete catheter dislodgement, severe transfusion reactions, and extremely abnormal vital signs, marked in red. Critical attention includes high pain index, stage II or higher pressure ulcers, and large deviations in infusion rate, marked in yellow. Routine attention includes stage I pressure ulcers, mild skin abnormalities, and environmental warnings, marked in blue.

6. The patient omnidimensional intelligent nursing inspection system based on virtual reality technology as described in any one of claims 1-5, characterized in that, The multimodal analysis model includes a refined skin lesion classification model and a multimodal data fusion model, which are used for big data storage, in-depth analysis, trend prediction, electronic medical record integration and system management.

7. The patient omnidimensional intelligent nursing inspection system based on virtual reality technology as described in any one of claims 1-5, characterized in that, The server generates improvement suggestions based on the comprehensive risk score. High-risk items in the comprehensive risk score are tracked, and automatic image comparison is performed during subsequent rounds. The comparison results are quantified, and all time-series data, image snapshots, assessment results, and operation logs are encrypted and stored to form a dynamic digital twin of patient care. By integrating patient time-series data, assessment snapshots, and operation records, a dynamically updated holographic health record is generated.

8. A method for comprehensive intelligent patient care inspection based on virtual reality technology, used in the comprehensive intelligent patient care inspection system based on virtual reality technology as described in any one of claims 1-7, characterized in that, Includes the following steps: The inspection can be initiated by voice or gesture, and the sensors and devices will be matched with the 3D map. Read data from multimodal sensor arrays and devices; The data is compared with the safety thresholds of each indicator, and the relationships between the data are discovered. A comprehensive risk score is calculated based on the comparison results and the relationships between the data. The results of the comprehensive risk assessment are generated into a structured electronic record and synchronized to the server.

9. The method for comprehensive intelligent patient care monitoring based on virtual reality technology as described in claim 8, characterized in that, The data from the sensors and devices includes: Vital signs recognition, locking the monitor screen, and extracting heart rate, blood pressure, blood oxygen, respiration and body temperature in real time; Identify the type of catheter and assess its location, patency, fixation, and the skin around the exit point. Identify the pump rate, infused volume, and drug name; Identify pain-related facial movements and generate a quantitative pain index; It can identify pressure sores, abrasions, and lacerations, and automatically stage them based on color, depth, and tissue type. Assess subcutaneous tissue health using multispectral or thermal imaging; Automatically identify and assess the healing status of surgical incisions, stomas, and drainage tube exits; Identify patient position; Check the bed rails and restraints.

10. The method for comprehensive intelligent patient care monitoring based on virtual reality technology as described in claim 8, characterized in that, Based on the results of the comprehensive risk score, improvement suggestions are generated. Nurses can quickly confirm or correct these suggestions via voice or gesture. High-risk items in the comprehensive risk score are tracked, and automatic image comparison is performed during subsequent rounds. The comparison results are quantified, and all time-series data, image snapshots, assessment results, and operation logs are encrypted and stored to form a dynamic digital twin of patient care.

Images