Intelligent nursing assisting robot system based on multi-modal perception and nursing assisting method

Abstract

The invention relates to the technical field of medical instruments, and provides an intelligent nursing assisting robot system and a nursing assisting method based on multi-modal perception, and the method comprises the steps: obtaining the identity information of a patient, scanning a two-dimensional code on a wrist strap of the patient, and collecting the identity information of the patient through an identity recognition module; physiological status data of the patient are obtained, wherein the physiological status data comprise lingual surface images, posture characteristics and vital sign parameters such as body temperature, oxyhemoglobin saturation and psychology; the identity information and the physiological state data are uploaded to a cloud server, and the cloud server is in butt joint with a hospital system and obtains diagnosis results, medical advice information, inspection reports and image inspection arrangements of the patients. According to the system, the identity of a patient is automatically recognized through the two-dimensional code, automatic collection of multi-modal data such as vital signs, tongue pictures and postures is achieved in combination with the sensing device and the visual recognition device, the workload of manual recording and information checking of medical staff is effectively reduced, and intelligentization and high efficiency of the nursing process are achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical devices, in particular to an intelligent nursing assistance robot system based on multi-modal perception and a nursing assistance method. BACKGROUND

[0002] Hospitalization service is a comprehensive medical service provided for patients who need inpatient treatment or continuous illness monitoring. During this process, patients receive around-the-clock medical care, including diagnostic examination, treatment implementation, nursing intervention, and close supervision by medical staff. The length of hospitalization usually depends on the severity of the illness and the complexity of the treatment plan. However, the traditional hospitalization care mode faces many challenges in actual operation, seriously affecting medical efficiency, service quality, and patient experience.

[0003] Firstly, the smooth flow of medical information is a prominent pain point in the current hospitalization process. Doctors, medical staff, and different functional departments in the hospital often have inefficient communication mechanisms or fragmented information systems, leading to information gaps and even treatment delays and medical errors. For example, key information such as examination arrangement and medical order changes cannot be timely synchronized to the nursing end, affecting the accuracy of execution.

[0004] Secondly, the heavy workload of nursing restricts the improvement of service quality. Medical staff, especially nurses, spend a lot of time dealing with repetitive administrative tasks, such as manually recording vital signs, filling out nursing documents, and checking examination procedures. Such high-load transactional work not only occupies time for direct patient care, but also may affect their work state and job satisfaction.

[0005] Thirdly, the problem of insufficient patient experience and compliance is increasingly prominent. The traditional care mode is task-oriented, lacking patient-centered service design. Health education content is often monotonous (such as oral explanation or paper materials), lacking personalization and interactivity, leading to patient understanding difficulties, memory confusion, and thus reducing treatment compliance and affecting rehabilitation effectiveness.

[0006] In terms of technology, existing nursing systems generally rely on single-mode, fragmented data collection methods. For example, body temperature, blood oxygen, heart rate, and other vital signs are still measured by independent devices at different times and manually entered, making it difficult to achieve continuous, dynamic health monitoring. At the same time, patient identification still relies on barcode scanning or manual checking, with low efficiency and error-prone defects. More importantly, multi-dimensional health data lacks fusion analysis capability, making it impossible to achieve a closed loop from "data collection" to "intelligent decision-making", limiting the generation of personalized nursing plans.

[0007] With the development of artificial intelligence, Internet of Things and big data technology, multi-modal perception and intelligent auxiliary system gradually become an important direction to improve the quality of medical services. By integrating visual, sensory, voice and other perception modalities, the system can comprehensively obtain the physiological, psychological and behavioral state of the patient. For example, tongue image recognition technology based on convolutional neural network (CNN) can realize automatic identification of traditional Chinese medicine constitution; visual recognition combined with NRS-2002 model can be used for intelligent assessment of nutritional risk; knowledge graph technology can realize intelligent matching of "disease-examination" items and generate personalized diagnosis and treatment path.

[0008] In addition, the progress of human-computer interaction technology makes it possible to use new types of propaganda and education methods such as voice broadcast and projection visualization, which helps to improve the intuitiveness of information transmission and patient participation. The docking of cloud platform and hospital information system (HIS, EMR, LIS, RIS) also provides technical support for realizing data interconnection, process tracing and remote management.

[0009] However, the existing inpatient care system has obvious shortcomings in information collaboration, work efficiency, patient experience and data intelligence. Although multi-modal perception and artificial intelligence technology shows broad application prospects, how to realize intelligent nursing auxiliary system for traditional Chinese medicine constitution identification, nutritional risk assessment, personalized propaganda and education, examination plan generation and abnormal warning is still a technical problem to be solved in this field. Therefore, it is urgent to provide an intelligent nursing auxiliary method and system based on multi-modal perception to solve the above problems. SUMMARY

[0010] The present application provides an intelligent nursing auxiliary robot system and a nursing auxiliary method based on multi-modal perception, aiming to solve the problems in the background art. The specific implementation is as follows: The intelligent nursing auxiliary method based on multi-modal perception includes obtaining the identity information of the patient, scanning the two-dimensional code on the patient's wristband, and collecting the identity information of the patient through the identity recognition module; Obtain the physiological state data of the patient, the physiological state data including tongue image, body posture feature, and vital sign parameters such as body temperature, blood oxygen saturation and psychology; Upload the identity information and physiological state data to the cloud server, wherein the cloud server interfaces with the hospital system to obtain the patient's diagnosis result, medical order information, test report and imaging examination arrangement; Construct a knowledge graph in the central processing module, which contains the mapping relationship between disease codes and routine examination items, and according to the obtained diagnosis result, match the corresponding examination items in the knowledge graph, and generate the next day's personalized examination plan; Push the personalized examination time arrangement and health education content to the patient through the human-computer interaction module; When the system monitors that the patient does not complete the examination within the predetermined time, the event is automatically recorded as examination leakage, and the pre-warning information is pushed to the mobile terminal of the responsible medical staff through the wireless communication unit, and the patient is marked as a high-priority inspection object. Based on the historical test result trend graph of the patient and the preset chronic disease risk prediction model, the reexamination suggestion is generated by the central processing module.

[0011] As a further scheme of the present application, the temperature, oxygen saturation and heart rate are non-contact collected by the sensing device, and the physiological parameters are output as continuous health monitoring data.

[0012] The present application also provides an intelligent nursing auxiliary robot system based on multi-modal perception, comprising an identity recognition module for scanning a patient wristband to obtain identity information. A multi-modal perception module comprising a high-definition camera, a visual recognition device and a sensing device for collecting tongue surface images, body features and vital signs. A central processing module integrating a knowledge graph and a chronic disease risk prediction model for generating examination plans and reexamination suggestions. A human-computer interaction module for pushing personalized information to patients and receiving feedback. A cloud server connected to a hospital information system to realize data intercommunication and long-term storage of evaluation structures and health education records.

[0013] As a further scheme of the present application, the identity recognition module comprises a two-dimensional code / barcode scanner arranged on a mechanical arm structure of the robot.

[0014] As a further scheme of the present application, the mechanical arm structure is rotatably connected to the robot through a rotating shaft, and a driving device is arranged inside the robot, which is connected with the rotating shaft to drive the rotatably connected mechanical arm structure to rotate.

[0015] As a further scheme of the present application, the central processing module is arranged inside the robot host, which comprises an embedded processor and an AI inference engine to support edge real-time calculation.

[0016] As a further scheme of the present application, the human-computer interaction module comprises a multi-point touch display screen, a voice broadcast device, a microphone and a projector, the multi-point touch display screen is arranged on the front body of the robot, and the voice broadcast device, the microphone and the projector are arranged on the head of the robot.

[0017] As a further scheme of the present application, a guide plate is arranged on the robot, which extends along the length direction of the robot, and the touch display screen is movably arranged on the guide plate.

[0018] As a further scheme of the present application, the autonomous navigation wheel set and the laser radar are arranged on the robot and electrically connected with the central processing module.

[0019] The present application also provides a computer readable storage medium, which stores a computer program, when the program is executed by a processor, to implement the intelligent nursing assistance method.

[0020] Due to the above technical solutions, the present application has the following beneficial technical effects: 1. The present application automatically identifies the patient's identity through a two-dimensional code, and realizes the automatic collection of multi-modal data such as vital signs, tongue appearance, and body posture by combining a sensing device and a visual recognition device, effectively reducing the work burden of manual recording and checking information by medical staff, reducing human operation errors, improving the real-time and reliability of data collection, and realizing the intelligentization and high efficiency of the nursing process; 2. The present application integrates traditional Chinese constitution identification, nutrition risk assessment, chronic disease prediction model and knowledge graph, and the system can comprehensively analyze the multi-dimensional health data of the patient, dynamically generate personalized examination plan, review suggestion and health education content, break through the template mode of traditional nursing, and improve the scientificity and pertinence of nursing intervention; 3. The present application provides situational and easy-to-understand health guidance and examination reminders to the patient through multi-modal human-computer interaction modes such as voice broadcast and projection visualization, and establishes an examination omission early warning mechanism to improve the timeliness of nursing response. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 The structure block diagram of the intelligent nursing assistance robot system based on multi-modal perception in the embodiment of the present application is shown in the figure. Figure 2 The structure schematic diagram of the intelligent nursing assistance robot in the embodiment of the present application is shown in the figure.

[0022] Explanation of reference signs: 101, robot head, 102, projector, 103, multi-point touch display screen, 104, guide plate, 105, rotating shaft, 106, mechanical arm structure, 107, scanner, 108, robot mouth, 109, base. DETAILED DESCRIPTION

[0023] The specific embodiment of the present application will be described below in conjunction with the drawings and examples: It should be noted that the structures, proportions, sizes, etc. illustrated in the accompanying drawings of this specification are only used to complement the content disclosed in the specification, so that those skilled in the art can understand and read them, and are not intended to limit the conditions under which the present invention can be implemented. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0024] Furthermore, the terms such as "upper," "lower," "left," "right," "middle," and "one" used in this specification are merely for clarity of description and are not intended to limit the scope of the invention. Any changes or adjustments to their relative relationships, without substantially altering the technical content, should also be considered within the scope of the invention.

[0025] Example 1, combined with Figure 1 , Figure 2 As shown, this invention provides an intelligent nursing assistance robot system based on multimodal perception. The system integrates multi-source sensing, intelligent analysis, and human-computer interaction technologies to achieve inpatient identification, health status assessment, personalized nursing plan generation, and closed-loop management, thereby improving the intelligence level of nursing services and patient experience. This intelligent nursing assistance robot system includes an identification module, a multimodal perception module, a central processing module, a human-computer interaction module, and a cloud server; wherein, The identification module is used to scan the patient's wristband to obtain identification information. Specifically, the identification module includes a QR code / barcode scanner 107, which is mounted on a robotic arm structure 106 on one side of the robot. This robotic arm structure 106 is mounted on the robot body via a pivot 105 and is driven by a built-in motor, causing it to rotate around the central axis of the pivot 105, thus allowing for flexible adjustment of the scanning angle. By driving the robotic arm structure 106 with the motor, the shooting angle and distance can be flexibly adjusted to accommodate patients of different heights and positions, significantly improving the success rate and accuracy of data acquisition. When a patient approaches the robot, the robotic arm structure 106 automatically adjusts its position under the action of the motor, aligning the QR code / barcode scanner 107 with the medical wristband on the patient's wrist, and completing the scanning task. If the wristband is equipped with an RFID tag, the information within the tag is read using an RFID reader. Next, the system compares and matches the obtained patient identification with the hospital information system. After confirming that the service recipient is correct, it continues to execute the subsequent nursing or examination plan.

[0026] The multi-modal perception module is used to collect the tongue surface image, body posture features, vital signs and other data of the patient. The multi-modal perception module includes a high-definition camera, a visual recognition device, a sensing device, and a touch screen input interface. The high-definition camera is arranged on the robot head 101 and is used to collect the tongue surface image. The high-definition camera has an automatic focusing and ring light function, thereby ensuring the definition of the collected image. The visual recognition device integrates an infrared / deep camera (such as an RGB-D camera) and is used to capture the facial contour and trunk shape, identify malnutrition signs such as muscle atrophy, eye socket depression or edema, and the like. The sensing device is a non-contact sensor, which includes an infrared body temperature sensor and a millimeter wave radar unit, and can non-invasively collect body temperature, blood oxygen saturation, heart rate and respiratory rate. The touch screen input interface is used to present a pain assessment scale (such as an NRS score), receive manual input of the patient and record subjective feelings. The data collected by the multi-modal perception module, including the tongue surface image, body posture features, body temperature, blood oxygen, heart rate and pain score, are pre-processed (the pre-processing includes denoising, standardization and feature extraction) locally, and then transmitted to the central processing module, combined with the patient electronic medical record information (the electronic medical record information is obtained from a cloud server), for comprehensive analysis and intelligent decision-making.

[0027] In the present embodiment, the visual recognition device can identify signs such as muscle atrophy, eye socket depression, facial edema or lower limb edema through three-dimensional modeling and feature extraction algorithms. In combination with laboratory test data, a nutritional risk screening model (such as NRS-2002) is called to realize intelligent assessment of malnutrition risk.

[0028] In the present embodiment, the infrared body temperature sensor integrated in the sensing device is used to measure the temperature of the forehead or temple at a distance, thereby realizing rapid temperature screening. The millimeter wave radar module detects the chest micro-motion based on the Doppler effect, non-invasively obtains vital sign parameters such as respiratory rate, heart rate and blood oxygen saturation (SpO2), and avoids the discomfort caused by traditional patch-type sensors.

[0029] The central processing module includes an SoC chip embedded processor and an AI inference engine, supports edge-side real-time computing, and guarantees system response speed and off-network availability. The central processing module is arranged in the robot host, and includes traditional Chinese medicine syndrome evaluation, pain evaluation, nutrition evaluation, and automatic measurement of vital signs. Specifically, the nursing assistance robot is configured with a high-definition camera installed at the end of the mechanical arm structure 106, which can flexibly adjust the angle to take pictures of the patient's tongue fur and complexion, automatically preprocess the collected images (such as denoising and contrast enhancement) through the image processing algorithm built-in the system, and then transmit the images to the AI syndrome model on the cloud or local server. The AI syndrome model is based on a deep convolutional neural network (CNN) and trained through a large number of labeled data sets, which can identify various traditional Chinese medicine constitution types (such as qi deficiency constitution and internal damp-heat retention), and the model output result includes not only constitution classification, but also detailed health guidance suggestions such as diet suggestions, exercise programs, and living and nursing care. The above evaluation results and suggestions are all uploaded to the hospital's electronic medical record system of traditional Chinese medicine in real time, so as to be checked by doctors and further adjust the treatment plan.

[0030] The nursing assistance robot provides various pain evaluation tools, including visual analog scale (VAS), numerical rating scale (NRS), and Wong-Baker Faces. After the patient selects the score that best fits his / her own situation, the system automatically records the evaluation time, score value, and pain site. If the pain score reaches or exceeds 4 points, the system immediately triggers a response, pops up a painkiller use propaganda video, guides the patient to use the medicine correctly, sends a "high pain risk" warning information to the medical staff station, prompts the medical staff to pay attention, and suggests the doctor to adjust the analgesic scheme according to the current evaluation result. The above pain evaluation data is written into the electronic temperature sheet and nursing record sheet in real time, so as to ensure that the medical team can timely grasp the patient's pain state and take corresponding measures.

[0031] The high-definition camera of the nursing assistance robot can preliminarily judge the patient's body characteristics (such as emaciation, edema, and obesity), and automatically import the key indicators such as serum albumin, prealbumin, and hemoglobin in the LIS system through wireless network. The system calls the nutrition risk screening model (such as NRS-2002), combines visual recognition and laboratory data for comprehensive analysis, and automatically generates a nutrition score. According to the score result, the system generates personalized diet suggestions and pushes them to the patient through the display screen or mobile terminal. At the same time, the above data is synchronized to the nutrition consultation system, so as to facilitate the nutritionist to check and develop a more detailed nutrition intervention plan.

[0032] The mechanical arm of the nursing auxiliary robot is integrated with a sensing device, which includes an infrared temperature sensor for measuring forehead or wrist temperature, a photoelectric blood oxygen probe that can be clamped on a finger or worn as a wristband device, and a millimeter wave heart rate radar that can penetrate clothing to measure heart rate. The measurement can be automatically completed without the active cooperation of the patient, and the results are uploaded to the nursing system in real time to update the temperature sheet. If an abnormal value is detected, the system will immediately issue an alarm and send a notification to medical staff. In addition, the system supports continuous monitoring mode, which is particularly suitable for postoperative patients or other situations that require frequent monitoring of vital signs. For example, automatic measurement every 15 minutes ensures timely detection of potential problems.

[0033] The human-computer interaction module includes a multi-touch display screen 103, a voice broadcast device, a microphone, and a projector 102. The multi-touch display screen 103 is arranged on the front body of the robot, the voice broadcast device is arranged adjacent to the head portion 108 of the robot, and the microphone and the projector 102 are both arranged on the head 101 of the robot. Based on the patient ID, the medical order information in the HIS system and the test results in the LIS system are automatically retrieved, and the nursing auxiliary robot system generates health education content related to the current condition of the patient based on the patient's traditional Chinese medicine constitution, nutrition score, and examination arrangement. The education content includes dietary taboos, body position guidance, and psychological counseling, and the education content is output to the patient through voice and projection. In addition, the multi-touch display screen 103 receives patient feedback and records the education completion status, which is uploaded to the nursing management system.

[0034] In this embodiment, the robot corresponding to the multi-touch display screen 103 is provided with a guide plate 104 extending along the length direction of the robot and fixed to the main body of the robot. The multi-touch display screen 103 is installed on the guide plate 104 through a sliding connector (not shown in the figure) and can slide up and down along the length direction of the guide plate 104, thereby realizing flexible adjustment of the height of the display screen. Through the height adjustment of the multi-touch display screen 103, the patient can clearly view the examination plan, health education content, and complete feedback operations in the lying, semi-lying, or sitting state, thereby enhancing the usability and user experience of the system.

[0035] Specifically, the guide plate 104 is a strip-shaped guide rail structure, which is provided with a sliding groove or a guide rail track, and the multi-touch display screen 103 is provided with a sliding block or a sliding bracket matched therewith, which constitute a sliding pair to ensure smooth movement of the display screen in the vertical direction. Further, the multi-touch display screen 103 maintains electrical connection with the central processing module during the sliding process, and the continuous transmission of signals and power is realized through a slip ring type wire structure, thereby ensuring that the human-computer interaction function is not affected by position adjustment.

[0036] In another embodiment, the sliding connection structure is also equipped with a damping adjustment device or an electric push rod driving mechanism, so that the patient can manually push and pull the control screen to adjust the height of the screen, thereby adapting to the viewing needs of patients of different heights or in different sitting and lying positions.

[0037] The cloud server is deployed in the hospital intranet and interfaces with HIS, EMR, LIS, RIS and other information systems to obtain patient diagnosis results, medical orders, test reports and imaging arrangements. In addition, the cloud server stores evaluation records, propaganda and education logs and early warning information for a long time, supports nursing quality traceability and scientific research analysis.

[0038] In the embodiment, the intelligent nursing auxiliary robot system is equipped with an autonomous navigation wheel set and a laser radar on the robot, for realizing autonomous movement and accurate positioning of the robot in a ward environment.

[0039] Specifically, the autonomous navigation wheel set is arranged on the base 109 of the robot and includes two driving wheels and at least one driven universal wheel, constituting an omnidirectional mobile chassis structure. The driving wheels are controlled by built-in motors and support differential steering, which can complete steering, obstacle avoidance and accurate positioning in a narrow space. The laser radar is installed on the head 101 of the robot and is used to scan the surrounding environment in real time to obtain obstacle distance and contour information. In combination with a preset ward electronic map, the robot plans a path and dynamically avoids obstacles through a SLAM (simultaneous localization and mapping) algorithm, and automatically navigates to the bedside of a target patient.

[0040] In actual operation, the central processing module calls the navigation module to generate an optimal path according to a timing inspection or emergency call response scheduling instruction in a nursing task. During the movement of the robot along the planned path, the laser radar continuously senses the front obstacles, and once a dynamic or static obstacle is detected, the movement direction is immediately adjusted or the operation is paused, thereby ensuring the safety and reliability of the movement process.

[0041] In actual application, the robot operates according to the following process: the medical staff starts the robot, the system self-checks and connects to the network, the robot moves to the bedside of a target patient through the autonomous navigation module, the two-dimensional code / bar code scanner 107 automatically scans the wristband to confirm the patient's identity, the multi-modal sensing module is started to collect tongue image, body posture, vital signs and pain score, then the collected data is uploaded to the central processing module, individualized examination plan and health education content are generated in combination with the cloud server medical record information, and are presented to the patient through voice broadcast or wall projection. In addition, the human-computer interaction module pushes information to the patient and records feedback, if the patient does not complete the examination on time, the system automatically triggers an early warning to notify the responsible medical staff, and all data is synchronized to the cloud server to complete a complete nursing auxiliary process.

[0042] Embodiment 2, in combination with Figure 1 , Figure 2As shown, the present application provides an intelligent nursing assistance method based on multi-modal perception, which is performed by an intelligent nursing assistance robot deployed in a ward environment, comprising the following steps: Step one: through the identity recognition module carried by the robot, scan the RFID tag or two-dimensional code on the medical wristband worn by the patient. Specifically, the two-dimensional code / bar code scanner 107 is arranged on the robot mechanical arm structure 106, the mechanical arm is connected to the robot main body through the rotating shaft 105, and is driven to rotate by the built-in motor, so that the scanner 107 is automatically aligned with the wristband. The system quickly reads the unique identity such as the hospitalization number and matches with the hospital information system to ensure that the service object is accurate and correct.

[0043] Step two: collect the physiological state data of the patient through the multi-modal perception module, including tongue surface image, body posture feature, vital sign parameter, and psychological state data collection. Specifically, the high-definition camera arranged on the mechanical arm structure 106 is used for collection, which supports automatic focusing and light compensation to ensure clear image; the visual recognition device integrated with infrared / deep camera is used to capture the patient's face contour and trunk shape, and identify signs such as muscle atrophy, eye socket depression or edema; the sensor device is used to collect parameters such as body temperature, blood oxygen saturation, and heart rate, wherein the infrared body temperature sensor realizes long-distance temperature measurement, the millimeter wave radar unit non-inductive monitors the respiratory rate and blood oxygen saturation, and the NRS pain rating scale is presented through the human-computer interaction module, and the subjective feeling is input by the patient on the touch screen to record the pain level.

[0044] Step three: upload the collected identity information and physiological state data to the central processing module inside the robot, and at the same time, establish connection with the cloud server through Wi-Fi or 5G wireless communication mode. The cloud server interfaces with the HIS (hospital information system), EMR (electronic medical record system), LIS (laboratory system) and RIS (image system) of the hospital, and obtains the diagnosis result, medical order information, laboratory test report and image examination arrangement of the patient in real time, to form a complete clinical data view.

[0045] Step four: build a knowledge graph in the central processing module, which takes ICD disease coding as a node, and regular examination items (such as blood routine, electrocardiogram, CT, etc.) as an associated edge, to establish a mapping relationship between diseases and examinations. When the system obtains the current diagnosis result of the patient, the recommended examination items are matched in the knowledge graph, and the next day's personalized examination plan is generated combining with the medical order time window, including examination type, appointment time, place and matters needing attention.

[0046] Step five: push personalized examination time arrangement and health education content to the patient through the human-computer interaction module. The education content is dynamically generated by the central processing module according to the physiological state data (such as TCM constitution, nutrition score) and the current examination item of the patient, and specifically includes: dietary taboos, pre-examination body position guidance, psychological counseling suggestions, and the content is presented to the patient through a multi-touch display screen, a voice broadcast unit or a projector 102, supporting two-way interaction and feedback recording.

[0047] Step six: the system continuously monitors whether the patient completes each examination within the scheduled time. When it is detected that a certain examination is not performed on time, the event is automatically marked as "examination missing", and the warning information is pushed to the mobile terminal of the responsible medical staff through the wireless communication unit, and the patient is marked as "high priority inspection object" in the nursing board, reminding manual intervention. The mobile terminal can be a mobile phone APP.

[0048] Step seven: based on the trend chart of the patient's historical test results (such as blood glucose, blood pressure, and kidney function indicators), the central processing module calls the preset chronic disease risk prediction model (such as a time series model based on LSTM), analyzes the disease progression trend, and generates review suggestions and early intervention prompts. For example: predicting the risk of HbA1c exceeding the standard in the next 3 months for a diabetic patient, and suggesting to review glycosylated hemoglobin in advance and adjust the drug regimen.

[0049] Embodiment 3, the present application also provides a computer readable storage medium, the storage medium stores a computer program, when the program is executed by a processor, to realize an intelligent nursing auxiliary method.

[0050] It should be noted that the computer readable storage medium of the present embodiment can be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer readable storage medium can include, but are not limited to: electrical connection with one or more conductive wires, portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

[0051] In this embodiment, the computer readable storage medium can be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. In this embodiment, the computer readable signal medium can include a computer readable program that is communicated, propagated, or transported from one place to another, for example, in a baseband or as part of a carrier wave, whether or not modulated. The computer readable signal medium can take a variety of forms, including but not limited to, electro-magnetic, optical, or any suitable combination thereof. The computer readable signal medium can be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable program code contained in the computer readable storage medium can be transmitted by any suitable medium, including but not limited to, wire, cable, RF, etc., or any suitable combination of the foregoing.

[0052] The computer readable storage medium described above can be written in any one or more programming languages, including object-oriented programming languages, such as Java, Python, C++, and conventional procedural programming languages, such as C language or similar programming languages. The program can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, through the Internet using an Internet service provider).

[0053] The present application realizes the acquisition of patient identity through RFID and two-dimensional code scanning technology in the inpatient care scene, and combines high-definition cameras, depth vision and sensing devices to realize multi-modal acquisition of physiological and psychological state data such as tongue image, body state, vital signs and subjective pain score. Based on the AI inference engine, a closed-loop analysis from data perception to intelligent decision-making is realized. In addition, the robot is integrated with a multi-point touch display and a voice broadcast device, automatically patrols to the target bedside according to the ward map, pushes personalized examination plans, review suggestions and health education content to patients, thereby improving the efficiency and accuracy of nursing services, and enhancing patient participation and treatment compliance.

[0054] Many other changes and modifications can be made to the application without departing from the spirit and scope of the application. It should be understood that the application is not limited to a particular embodiment, and the scope of the application is defined by the appended claims.

Claims

1. An intelligent nursing assistance method based on multimodal perception, characterized in that, This includes obtaining the patient's identity information by scanning the QR code on the patient's wristband and collecting the patient's identity information through the identity recognition module; Acquire the patient's physiological status data, which includes tongue images, body posture characteristics, and vital signs parameters such as body temperature, blood oxygen saturation, and psychological state. The identity information and physiological status data are uploaded to a cloud server, which is connected to the hospital system to obtain the patient's diagnosis results, medical orders, test reports and imaging examination arrangements. A knowledge graph is built in the central processing module. This knowledge graph contains the mapping relationship between disease codes and routine examination items. Based on the obtained diagnosis results, the corresponding examination items are matched in the knowledge graph, and a personalized examination plan for the next day is generated. Personalized examination schedules and health education content are pushed to patients through the human-computer interaction module; When the system detects that a patient has not completed the examination within the scheduled time, it automatically records the event as an examination omission and pushes the warning information to the mobile terminal of the responsible medical staff through the wireless communication unit, marking the patient as a high-priority inspection target; Based on the patient's historical test result trend chart and the pre-built chronic disease risk prediction model, follow-up recommendations are generated through the central processing module.

2. The intelligent nursing assistance method based on multimodal perception according to claim 1, characterized in that, Body temperature, blood oxygen saturation, and heart rate are collected non-contactly using sensors, and these physiological parameters are output as continuous health monitoring data.

3. An intelligent nursing assistive robot system based on multimodal perception, characterized in that, Includes an identity recognition module for scanning the patient's wristband to obtain identity information; The multimodal perception module includes a high-definition camera, a visual recognition device, and a sensing device, used to collect images of the tongue surface, body posture characteristics, and vital signs. The central processing module integrates knowledge graphs and chronic disease risk prediction models to generate examination plans and follow-up recommendations. The human-computer interaction module is used to push personalized information to patients and receive feedback; The cloud server interfaces with the hospital's information system to enable data exchange and long-term storage of assessment structures and educational records.

4. The intelligent nursing assistive robot system based on multimodal perception according to claim 3, characterized in that, The identity recognition module includes a QR code / barcode scanner, which is mounted on the robot's robotic arm structure.

5. The intelligent nursing assistive robot system based on multimodal perception according to claim 4, characterized in that, The robotic arm structure is rotatably connected to the robot via a rotating shaft. The robot is equipped with a drive device, which is connected to the rotating shaft to drive the rotatably connected robotic arm structure to rotate.

6. The intelligent nursing assistive robot system based on multimodal perception according to claim 3, characterized in that, The central processing module is located inside the robot's main unit and includes an embedded processor and an AI inference engine to support real-time computing at the edge.

7. The intelligent nursing assistive robot system based on multimodal perception according to claim 3, characterized in that, The human-computer interaction module includes a multi-touch display screen, a voice broadcasting device, a microphone, and a projector. The multi-touch display screen is located at the front of the robot, while the voice broadcasting device, microphone, and projector are all located at the head of the robot.

8. The intelligent nursing assistive robot system based on multimodal perception according to claim 7, characterized in that, The robot is equipped with a guide plate that extends along the length of the robot, and the touch screen is movably mounted on the guide plate.

9. The intelligent nursing assistive robot system based on multimodal perception according to claim 3, characterized in that, It also includes an autonomous navigation wheel assembly and a lidar, which are mounted on the robot and electrically connected to the central processing module.

10. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method as described in any one of claims 1 and 2.

Images