Multi-mode interactive surgical ward intelligent prompting system and method

Abstract

The invention relates to the technical field of surgical ward intelligent prompting, and discloses a multi-modal interaction surgical ward intelligent prompting system and method. The system comprises a patient state sensing module, a multi-mode trigger module, a prompt mode adaptation module, a time sequence regulation and control module, a central control unit and the like. The patient state sensing module obtains patient identity and physiological data and divides data streams; the multi-mode trigger module divides a priority queue and predicts a behavior track; the prompt mode adaptation module generates a prompt mode set according to preset matrix matching; the time sequence regulation and control module calls operation node parameters to determine an execution time sequence benchmark; and the central control unit performs multi-channel verification to generate a final interaction instruction. The system achieves multi-mode interaction, can meet the requirements of different patients, is closely combined with the operation process, improves the nursing quality and medical efficiency of a surgical ward, guarantees the safety of the patients, and improves the medical experience.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent prompts for surgical wards, and in particular to a multimodal interactive intelligent prompt system and method for surgical wards. Background Art

[0002] In modern surgical ward management, efficient and accurate patient care and information interaction are crucial, but traditional ward reminder methods have many problems and are unable to meet the growing medical needs.

[0003] From the perspective of patient information acquisition and management, most surgical wards currently rely on manual, timed collection of patient physiological data. This method is not only inefficient but also prone to human error. Medical staff need to record basic vital signs such as the patient's temperature, blood pressure, and heart rate multiple times at different times. Under the busy work pace, key data may be missed or recorded incorrectly. Moreover, manual collection cannot achieve real-time, continuous monitoring. For patients with rapidly changing conditions, it is difficult to detect subtle physiological fluctuations in a timely manner, which may delay the optimal treatment opportunity. At the same time, different medical staff have different methods and standards for recording data, resulting in a lack of consistency and coherence in patient data, which is not conducive to comprehensive analysis of the condition.

[0004] In terms of information interaction and prompts, traditional prompt systems have a single function. Generally, information is conveyed only through simple voice broadcasts or visual displays, which cannot meet the diverse needs of different patients. For patients with visual impairments, visual prompts are basically ineffective; and for patients with hearing impairments, voice broadcasts are also ineffective. For pediatric patients, conventional prompt methods may be too monotonous and lack interest, making it difficult to attract their attention, resulting in low cooperation. In addition, when multiple devices work together, traditional systems lack effective overall management, and the prompt time and content between different devices are prone to conflicts. Medical staff may receive multiple contradictory prompt messages at the same time, affecting work efficiency and decision-making accuracy.

[0005] From the perspective of coordination with the surgical process, surgical operations involve multiple complex links, and different surgical stages require different care and prompts for patients. Existing prompt systems are often not closely integrated with the surgical process and cannot provide patients with accurate prompts based on the real-time progress of the operation. For example, during the preoperative preparation stage, patients cannot be accurately informed of the specific fasting and water abstinence times and precautions; during the postoperative recovery stage, prompts cannot be adjusted in a timely manner according to the patient's recovery status, such as the time and intensity of rehabilitation training. This results in a lack of clear guidance for patients during pre- and postoperative care, increases patients' anxiety, and is not conducive to the smooth progress of the operation and the patient's recovery. Summary of the Invention

[0006] The purpose of the present invention is to provide a multimodal interactive surgical ward intelligent prompt system and method to solve the problems raised in the above background technology.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a multimodal interactive surgical ward intelligent prompt system, the system comprising:

[0008] A load configuration module, configured to configure dynamic load parameters according to the type of target charging device and generate corresponding load simulation rules;

[0009] a charging parameter acquisition module, configured to obtain an operating parameter set of the charging device from a charging database in real time according to the load simulation rule, and use all parameters in the operating parameter set as detection dimension parameters of the charging device;

[0010] a load fluctuation calculation module, configured to generate a load fluctuation curve of the charging device through a dynamic simulation algorithm based on a load association relationship preset in the charging database, and use the load fluctuation curve as a detection dimension parameter of the charging device;

[0011] an abnormality identification module, configured to calculate an abnormal fluctuation index of the charging device using a preset fluctuation analysis method, and use the abnormal fluctuation index as a detection dimension parameter of the charging device;

[0012] A central processing unit is used to transmit the charging device information to the load fluctuation calculation module and the anomaly identification module, send the load simulation rules to the charging parameter acquisition module, and fuse all the detection dimension parameters to generate the status evaluation parameters of the charging device.

[0013] Preferably, the calculating of the abnormal fluctuation index of the charging device by a preset fluctuation analysis method includes:

[0014] The charging device and its associated preset reference device are used as dynamic analysis nodes, and parameter change patterns of the dynamic analysis nodes over multiple load cycles are obtained based on historical operating data;

[0015] Based on the parameter change pattern, according to the fluctuation range of each of the historical operation data in the corresponding dynamic analysis node, calculate the initial abnormal index of each of the dynamic analysis nodes and the initial deviation of each of the historical operation data;

[0016] Based on the initial abnormality index and the initial deviation, a final abnormality index of each dynamic analysis node and a final deviation of each historical operation data are jointly optimized through a bidirectional recursive model to obtain a final abnormality index of the dynamic analysis node corresponding to the charging device as its abnormal fluctuation index;

[0017] Set the target analysis node to be any dynamic analysis node, and the target historical data to be any historical operation data. Based on the parameter change pattern, calculate the ratio of the independent fluctuation amount of the target analysis node in the target historical data to the independent total fluctuation amount in all historical data, as well as the ratio of the global fluctuation amount of all analysis nodes in the target historical data to the global total fluctuation amount in all historical data. If the independent fluctuation amount ratio exceeds the global fluctuation amount ratio, it is determined that the target analysis node has abnormal fluctuation characteristics in the target historical data.

[0018] Preferably, the load simulation rules include a basic load type group and a variable load type group; the basic load type group contains multiple basic load modes and a fixed adjustment coefficient corresponding to each basic load mode; the variable load type group contains multiple variable load modes and a dynamic adjustment coefficient corresponding to each variable load mode.

[0019] Preferably, the step of acquiring the operating parameter set of the charging device in real time from the charging database according to the load simulation rule includes:

[0020] The charging parameter acquisition module extracts the structured operating data of the charging device from the charging database, extracts corresponding basic parameters based on each basic load mode in the basic load type group, extracts corresponding variable parameters based on each variable load mode in the variable load type group, and merges all the basic parameters and variable parameters into the operating parameter set.

[0021] Preferably, when the load association relationship is a direct load association, it is determined whether the charging device is directly connected to the load node. If a direct connection relationship exists, the load fluctuation curve is a linear superposition of the fluctuation contribution values ​​of all associated load nodes; otherwise, the load fluctuation curve is generated based on the basic load value of the charging device itself.

[0022] When the load association relationship is indirect load association, the load fluctuation curve is a dynamic weighted result of fluctuation contribution values ​​of all associated nodes obtained by the charging device through multi-level load links.

[0023] Preferably, it further comprises a fault tracing module connected to the central processing unit;

[0024] The fault tracing module is used to set a target fault type, obtain a feature matching degree between the charging device and the target fault type through a preset association matching method, and use the feature matching degree as a detection dimension parameter of the charging device.

[0025] Preferably, the fault tracing module obtains the feature matching degree by a preset correlation matching method, including:

[0026] Extracting a historical fault waveform set of the charging device from a fault model library, extracting a standard waveform set of the target fault type, and calculating a time domain coincidence degree between the historical fault waveform set and the standard waveform set as a first matching degree;

[0027] Comparing the parameter change trend of the charging device during the load cycle with the standard change trend of the target fault type, and obtaining a trend similarity between the two as a second matching degree;

[0028] The first matching degree, the second matching degree, or a combination thereof is calculated as the feature matching degree.

[0029] Preferably, the central processing unit generates the state evaluation parameters by linear weighting or nonlinear mapping for all the detection dimension parameters.

[0030] Preferably, it further comprises a storage module connected to the central processing unit, wherein the storage module is used to record the operating parameter set, load fluctuation curve, abnormal fluctuation index and status assessment parameters.

[0031] Preferably, the present invention further includes a method for intelligent detection of a charging device based on dynamic load simulation, which is applied to any of the above-mentioned intelligent detection systems for charging devices based on dynamic load simulation, and the method includes the following steps:

[0032] Step 1: Configure dynamic load parameters through the load configuration module according to the type of target charging device and generate corresponding load simulation rules;

[0033] Step 2: Based on the load simulation rule generated in step 1, a charging parameter acquisition module is used to obtain the operating parameter set of the charging device from the charging database in real time, and all parameters in the operating parameter set are used as detection dimension parameters of the charging device;

[0034] Step 3: Based on the load association relationship preset in the charging database, a load fluctuation calculation module uses a dynamic simulation algorithm to generate a load fluctuation curve of the charging device, and uses the load fluctuation curve as a detection dimension parameter of the charging device;

[0035] Step 4: Calculate an abnormal fluctuation index of the charging device using a preset fluctuation analysis method through an abnormality identification module, and use the abnormal fluctuation index as a detection dimension parameter of the charging device;

[0036] Step 5: Use the central processing unit to transmit the charging device information to the load fluctuation calculation module and the anomaly identification module, send the load simulation rules to the charging parameter acquisition module, and fuse all the detection dimension parameters obtained in steps 2-4 to generate the status evaluation parameters of the charging device.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] In terms of patient information management and monitoring, the system's patient status perception module can obtain patient identity information and physiological status data in real time, and intelligently divide it into basic vital sign data streams and interactive demand data streams. This function realizes the automation and refinement of data collection. Compared with manual collection, it not only greatly improves the accuracy and completeness of the data, but also can timely capture subtle changes in the patient's physiological state through continuous monitoring. Medical staff can obtain comprehensive and reliable patient data at any time, providing a strong basis for disease diagnosis and treatment plan adjustment. For example, for patients in the critical recovery period after surgery, the system continuously monitors their vital signs. Once abnormal fluctuations occur, it can immediately issue an early warning, allowing medical staff to take quick measures to effectively reduce the patient's health risks.

[0039] Multimodal interaction is a major highlight of this system. The multimodal trigger module and prompt mode adaptation module work together to provide a rich variety of prompt modes through a preset barrier-free interaction matrix. For visually impaired patients, the system can provide prompts through tactile feedback or Braille auxiliary symbols. For pediatric patients, child-friendly symbols are used to provide more interesting and attractive prompts, such as cartoon visual displays or cheerful voice prompts, to improve patient cooperation. This personalized, comprehensive interactive method ensures that every patient receives prompt information accurately, greatly improving their medical experience and making them feel more at ease and comfortable during treatment.

[0040] The system is closely integrated with the surgical process, demonstrating a high degree of intelligence. The timing control module calls the surgical node parameters in the preset timeline database based on the prompt mode set to establish a precise execution timing benchmark for the patient. Before the operation, the system can accurately inform the patient of various preparation matters and time nodes in advance according to the surgical schedule, such as fasting and water abstinence time accurate to the minute, as well as preoperative examination items that need to be completed, etc., to help patients make adequate preparations and reduce preoperative anxiety. After the operation, according to the patient's recovery progress, the time, intensity and method of rehabilitation training are timely prompted to promote the recovery of the patient's physical functions. This not only improves the success rate of the operation, but also speeds up the patient's recovery and reduces the probability of complications.

[0041] The device collaboration module and central control unit play a key role in multi-device collaboration and management. The device collaboration module selects available device groups based on the final interaction instructions and generates an interaction execution queue. By loading the ward spatial topology model and energy balancing algorithm, it achieves rational allocation and efficient utilization of device resources. The central control unit uses a double-check mode to ensure the accuracy and timeliness of prompt information, effectively resolving response delays and data conflicts between multiple devices. Medical staff can receive prompt information clearly and accurately, avoiding work errors caused by information confusion, improving work efficiency and medical service quality.

[0042] Furthermore, the cross-device communication interface synchronizes data between the system and the hospital's information management system, facilitating smoother information flow throughout the surgical ward. Data sharing and interaction between modules are more convenient, enabling medical staff to gain a holistic understanding of the patient's condition and providing strong support for developing personalized treatment and care plans. Overall, this patented technology possesses significant value in improving medical efficiency, enhancing the patient experience, and ensuring medical safety, bringing a brand-new solution to the intelligent management of surgical wards. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a working principle diagram of the multimodal interactive surgical ward intelligent prompt system of the present invention;

[0044] Figure 2 A diagram showing the working principle of the multimodal trigger module for prioritizing the basic vital sign data stream;

[0045] Figure 3 A working diagram of the patient status perception module dividing data streams based on the biometric recognition algorithm;

[0046] Figure 4 Generate a working principle diagram of the interactive execution queue for the device collaboration module;

[0047] Figure 5 This is the working principle diagram of the central control unit multi-channel verification. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0049] See also Figure 1-Figure 5The present invention provides a technical solution: a multimodal interactive surgical ward intelligent prompt system. The system includes a patient status sensing module, a multimodal triggering module, a prompt mode adaptation module, a timing control module, and a central control unit. The specific implementation steps are as follows:

[0050] The patient status perception module is used to obtain the patient's identity information and physiological status data, and divide the data into basic vital signs data stream and interactive demand data stream. In actual application scenarios, this module can realize data collection through a variety of devices. For example, physiological monitoring equipment at the bedside, such as electrocardiogram monitors and blood pressure meters, can be used to obtain the patient's heart rate, blood pressure, blood oxygen saturation and other basic physiological signs data. These data constitute part of the basic vital signs data stream; at the same time, wearable devices such as smart bracelets are used to collect information such as the patient's activity steps and sleep status, which are also included in the basic vital signs data stream. For the interactive demand data stream, the patient's special needs, such as whether they have hearing impairments, visual impairments, whether they are pediatric patients, and other information, can be obtained through the electronic questionnaire filled out by the patient when they are admitted to the hospital.

[0051] The multimodal trigger module is used to perform dynamic priority queue division on the basic vital sign data stream to generate a trigger instruction set, and to perform behavioral trajectory prediction on the interactive demand data stream to generate an interactive adaptation strategy. This module will continuously receive the basic vital sign data stream from the patient status perception module, process the continuous monitoring signals therein, divide it into a static vital sign group and a dynamic behavior group, and then perform action intention analysis on the dynamic behavior group based on a preset behavioral trajectory prediction algorithm to generate a trigger feature set. At the same time, multi-dimensional feature extraction is performed on the identity recognition data in the basic vital sign data stream, a patient status portrait is constructed and an interactive weight coefficient is generated, and finally the trigger feature set is associated with the interactive weight coefficient in a time series to generate a trigger instruction set. For the interactive demand data stream, a machine learning algorithm is used to analyze the patient's previous behavioral data, predict its future behavioral trajectory, and then generate an interactive adaptation strategy.

[0052] The prompt mode adaptation module is used to map the trigger instruction set and the interaction adaptation strategy to the corresponding prompt mode set after logical matching based on the preset barrier-free interaction matrix. The preset barrier-free interaction matrix includes a basic interaction set and an enhanced interaction set. The basic interaction set covers voice broadcast identifiers, visual display identifiers and tactile feedback identifiers. The enhanced interaction set includes Braille auxiliary identifiers and child adaptation identifiers, and each identifier corresponds to an independent interaction control channel. After receiving the trigger instruction set and the interaction adaptation strategy, the module will search for the corresponding identifier in the barrier-free interaction matrix according to the preset matching rules, and then determine the corresponding prompt mode set. For example, if the trigger instruction set indicates that the patient's blood pressure is abnormally high, and the interaction adaptation strategy shows that the patient has normal hearing but poor vision, then the prompt mode adaptation module will give priority to the prompt mode corresponding to the tactile feedback identifier and the voice broadcast identifier.

[0053] The timing control module is used to call the surgical node parameters in the preset timeline database based on the prompt pattern set, and use the surgical node parameters as the execution timing benchmark for the current patient. The preset timeline database stores the standard time nodes and related parameters for different types of surgeries at various stages. The timing control module will retrieve the corresponding surgical node parameters from the database based on the type of surgery involved in the prompt pattern set. These parameters include preoperative preparation time, surgery start time, key surgical step time, etc. Taking a heart bypass surgery as an example, the timing control module will determine that the patient is undergoing a heart bypass surgery based on the prompt pattern set, and then retrieve the time nodes for various inspections and preparations required to complete the preoperative preparation stage of the surgery from the timeline database, and use these time nodes as the timing benchmark for the current patient to be prompted to execute at this stage.

[0054] The central control unit is used to perform multi-channel verification on the prompt mode set and the execution timing benchmark to generate the final interaction instruction. The central control unit will adopt a dual verification mode of signal integrity verification mechanism and timing conflict detection mechanism. The signal integrity verification mechanism is used to confirm whether the data transmitted from each module is accurate and complete during the acquisition process, and whether there is any data loss or error. For example, check whether the physiological data collected by the patient status perception module has abnormal fluctuations or unreasonable data points. The timing conflict detection mechanism is used to solve the response delay problem between multiple devices and ensure that different prompt devices can perform prompt actions in the correct order and time intervals. For example, when there are voice prompt devices and visual prompt devices at the same time, in order to avoid the situation where the voice prompt has ended but the visual prompt has not started, the final interaction instruction is generated by verifying the prompt mode set and the execution timing benchmark.

[0055] The present invention will be further described below in conjunction with Examples 1 to 5:

[0056] Example 1:

[0057] After receiving the basic vital sign data stream from the patient status perception module, the multimodal trigger module first classifies and processes the continuous monitoring signals. Relatively stable and slowly changing signals, such as body temperature and resting heart rate, are divided into the static vital sign group; while those signals that reflect the patient's real-time activity status, such as motion signals collected by accelerometers and respiratory rate change signals, are classified into the dynamic behavior group. For the dynamic behavior group, a preset behavior trajectory prediction algorithm is used to analyze the action intention. This algorithm is based on a deep learning model. By learning from a large amount of historical behavior data, it can identify different action patterns, such as whether the patient is turning over, sitting up, or walking, thereby generating a trigger feature set.

[0058] Simultaneously, the multimodal trigger module extracts multidimensional features from the identification data in the basic vital sign data stream. These features include information such as the patient's age, gender, medical history, and allergy history. For example, age, as patients' physical functions and responses to illness vary with age, is a crucial dimension in constructing a patient status profile. These multidimensional features are integrated and a data mining algorithm is used to construct a patient status profile. Weights are assigned to each feature based on its importance within the profile, generating interaction weight coefficients. Finally, the trigger feature set and interaction weight coefficients are chronologically correlated. For example, at a certain moment, if the trigger feature set indicates that the patient intends to stand up, and the interaction weight coefficient indicates that this action may pose a risk due to the patient's advanced age and history of cardiovascular disease, a corresponding trigger instruction set will be generated, prompting medical staff to pay attention to the patient's attempt to stand up.

[0059] In the hospital's surgical ward, a 65-year-old patient, Mr. Li, was admitted for heart disease and about to undergo coronary bypass surgery. The multimodal trigger module began dynamically prioritizing Mr. Li's basic vital sign data stream.

[0060] The patient status sensing module continuously provides a stream of basic vital sign data. The multimodal trigger module divides these continuous monitoring signals into static vital sign and dynamic behavior groups. Mr. Li's relatively stable body temperature and resting heart rate are classified as static vital sign groups. Meanwhile, his accelerometer-collected motion signals, respiratory rate changes, and other signals reflecting real-time activity status are classified as dynamic behavior groups.

[0061] For the dynamic behavior group, the multimodal trigger module uses a pre-set behavior trajectory prediction algorithm to analyze motion intent. For example, if Mr. Li's accelerometer detects a series of signal changes, the algorithm analyzes these signals and, based on extensive experience with similar signals, determines that Mr. Li may be attempting to sit up. Because the algorithm finds a high degree of similarity between the current acceleration signal pattern and previously recorded acceleration signal patterns of numerous patients sitting up, it generates a trigger feature set, indicating that Mr. Li intends to sit up.

[0062] The multimodal triggering module extracts multidimensional features from the identification data in Mr. Li's basic vital sign data stream. This includes information such as Mr. Li's age, gender, history of heart disease, and history of drug allergies. At 65, his physical function has declined, and his cardiovascular system is relatively fragile. These multidimensional features are integrated and a specialized data mining algorithm is used to construct a patient profile of Mr. Li. During this construction process, Mr. Li's history of heart disease is given a relatively high weight in the profile. Based on this profile, each feature is assigned a corresponding weight, generating an interaction weight coefficient.

[0063] The multimodal trigger module performs a time-series correlation between the trigger feature set and the interaction weight coefficient. Mr. Li intends to sit up, and his interaction weight coefficient indicates that due to his age and heart disease, sitting up may place an additional burden on his heart, posing a certain risk. Based on this, the multimodal trigger module generates a trigger instruction set. This instruction set may instruct the system to send an alert to the medical staff's terminal device, informing them that Mr. Li is attempting to sit up and asking them to monitor his physical condition and ensure his safety during the sitting process, to avoid heart problems caused by sudden sitting up.

[0064] Example 2:

[0065] The preset barrier-free interaction matrix is ​​divided into a basic interaction set and an enhanced interaction set. The voice broadcast identifier in the basic interaction set corresponds to a voice prompt device, such as a smart speaker in the ward. When the system generates a corresponding prompt instruction, the smart speaker will convey information to the patient in the form of voice, such as reminding the patient to take medicine on time, or that an examination is about to take place. The visual display identifier is associated with the display screen in the ward, including a small display screen next to the bed and a large screen in the public area of ​​the ward, which is used to display text, images and other prompt information, such as a visual display of the surgical process, a brief description of the examination results, etc. Tactile feedback identifiers can be achieved through a vibration device installed on the armrest or mattress of the bed. When the patient needs to receive prompts but it is inconvenient to use voice or visual methods, such as when the patient is resting and other patients in the ward are also resting, the vibration device can convey different prompt information through different vibration patterns.

[0066] Enhanced interactive Braille signage within the interactive set features Braille signage at key locations within the ward, such as bedside tables and bathroom entrances, for visually impaired patients. These signage, connected to the system, updates the Braille content in real time based on prompts. Child-friendly signage incorporates more engaging prompts tailored to the characteristics of pediatric patients, such as cartoon sound effects and animated images. The cross-device communication interface serves as a data bridge within the entire system. It synchronizes data between the patient status perception module, multimodal triggering module, prompt mode adaptation module, and timing control module, respectively, and the hospital information management system. The patient status perception module retrieves surgical scheduling data packets from the hospital information management system in real time via the cross-device communication interface. By matching the protocol fields of the data packets with the patient number in the basic vital sign data stream, it extracts basic timing parameters, including the scheduled surgery time and various preoperative preparation time points. At the same time, according to the patient type code in the interaction requirement data stream, the extended fields of the data packet are traversed to parse special interaction requirements. For example, some patients may be allergic to specific drugs and need to be specially reminded to medical staff before surgery. These basic timing parameters and special interaction requirements are calibrated according to the time axis offset of the surgical node and written into the basic instruction queue and enhanced instruction cache respectively, providing accurate data support for subsequent prompt operations.

[0067] The surgical ward of a large general hospital houses patients with diverse conditions and needs. Take, for example, Mr. Wang, a 70-year-old hearing-impaired patient admitted for a leg fracture; and Mengmeng, a 6-year-old child hospitalized for appendicitis. The ward is equipped with an intelligent notification system based on the present invention, whose pre-configured barrier-free interaction matrix and cross-device communication interface play a key role.

[0068] In the basic interaction set of the preset barrier-free interaction matrix, the visual display logo corresponds to the display screen in the ward. There is a small display screen next to each bed in the ward, and there is also a large screen in the public area. When the system receives a reminder instruction that Uncle Wang needs to take medicine on time, the small display screen will display "Uncle Wang, it is time to take medicine now, please take your medicine on time" in a large font, accompanied by a pill icon, so that Uncle Wang can intuitively understand the content of the prompt. For Mengmeng, a colorful animated image will be displayed on the display screen, such as a cute little rabbit holding a medicine bottle and saying: "Mengmeng, it's time to take medicine. You will get better faster after taking the medicine!"

[0069] Tactile feedback is provided by a vibration device mounted on the bed's armrest. When Mr. Wang is about to undergo an examination, and the ward is quiet and unsuitable for voice prompts, the vibration device will vibrate in a specific pattern. For example, a short vibration, a one-second pause, and another short vibration will indicate that the examination is about to begin, reminding Mr. Wang to prepare. For Mengmeng, the vibration pattern might be designed to be more interesting, such as three consecutive slight vibrations to imitate the feeling of a bunny hopping to attract her attention.

[0070] Braille assistive signage within the enhanced interactive collection plays a vital role in the care of visually impaired patients like Mr. Wang. A Braille sign is placed at Mr. Wang's bedside. When the system generates instructions regarding care arrangements, such as "The nurse will be coming to change your dressing," the sign connects to the system and updates the corresponding Braille content in real time, allowing Mr. Wang to easily read the information and understand the upcoming arrangements.

[0071] For children like Mengmeng, child-friendly identification comes into play. In Mengmeng's ward, when she needs to be reminded to perform simple rehabilitation activities, the system plays cheerful cartoon sound effects, such as birdsong or relaxing music, while displaying fun animations on the screen to encourage her to participate.

[0072] The cross-device communication interface plays a key role in data transmission within the entire system. The patient status perception module uses this interface to retrieve the surgical scheduling data packet from the hospital information management system. For example, if Mengmeng is about to undergo appendectomy surgery, the patient status perception module matches the patient number in Mengmeng's basic vital signs data stream with the protocol field in the surgical scheduling data packet. It successfully extracts basic timing parameters, such as the scheduled surgery time of 10:00 AM and the need for preoperative preparations to be completed by 9:00 AM. Furthermore, based on the patient type code (child patient) in Mengmeng's interaction needs data stream, the module traverses the extended fields of the data packet and parses out her special interaction needs, including a fear of injections and the need for special comfort.

[0073] The system calibrates these basic timing parameters and special interaction requirements according to the timeline offset of the surgical node and writes them into the basic command queue and enhanced command cache, respectively. Before 9:00 a.m., the instructions in the basic command queue will trigger corresponding prompts, such as displaying "Mengmeng, the nurse will be here to make some small preparations in a while. It won't hurt, so be good" on Mengmeng's ward screen. The instructions in the enhanced command cache will prompt medical staff to prepare a small toy to comfort Mengmeng in advance, which will be used during the injection. This ensures smooth surgical preparations and provides patients with more intimate and personalized care.

[0074] Example 3:

[0075] This embodiment primarily describes how prompt pattern sets are generated when the preset barrier-free interaction matrix uses different models. When the preset barrier-free interaction matrix uses a static matching model, the prompt pattern set is a discrete mapping set of the linear superposition results of the trigger instruction set and the interaction adaptation strategy. After receiving the trigger instruction set and the interaction adaptation strategy, the system linearly superimposes the two. For example, if the trigger instruction set contains information about a patient's abnormal blood pressure, and the interaction adaptation strategy indicates that the patient has good hearing, then these two pieces of information are comprehensively considered and linearly superimposed according to preset rules. Then, based on the mapping relationships pre-set in the static matching model, the superimposed results are mapped to the corresponding prompt pattern set. These mapping relationships are determined during the system development phase based on extensive clinical experience and user needs. For example, the above situation may be mapped to a voice announcement stating "Your blood pressure is abnormal. Please remain quiet. Medical personnel will be here soon" and a warning icon highlighting the abnormal blood pressure on the visual display.

[0076] When the preset accessibility interaction matrix uses a dynamic adaptation model, the prompt pattern set is a continuous sequence of patterns that uses an incremental learning algorithm to optimize the correlation between the trigger instruction set and the interaction adaptation strategy in real time. This incremental learning algorithm continuously adjusts the correlation between the two as the system receives new trigger instruction set and interaction adaptation strategy data. For example, during a certain period of time, the system may discover that when a patient is asleep and has an abnormal heart rate, the existing prompt pattern (such as a direct voice announcement of the abnormal heart rate) may cause significant startle to the patient, hindering their recovery. Using the incremental learning algorithm, the system adjusts the prompt pattern, first waking the patient with gentle tactile feedback, then providing a voice prompt. The content and tone of the voice prompt are also optimized to be gentler and more acceptable to the patient. In this way, over time and with the accumulation of data, the prompt pattern set will be continuously optimized to better meet the needs of different patients and various complex clinical scenarios.

[0077] In the surgical ward, there are patients Ms. Zhang and Mr. Li. The implementation of this embodiment will be described in detail below using them as an example.

[0078] Ms. Zhang was admitted to the hospital for gallstones. During her postoperative recovery, the system monitored her various physiological data in real time. At a certain point, the patient status perception module acquired Ms. Zhang's basic vital sign data stream, including blood pressure data outside the normal range. This was recognized by the multimodal trigger module, which generated a trigger instruction set indicating that Ms. Zhang's blood pressure was abnormal. Simultaneously, based on the information Ms. Zhang provided upon admission and analysis of her past behavioral data, the interactive adaptation strategy determined that she had normal hearing and good visual perception.

[0079] When the preset barrier-free interaction matrix uses a static matching model, the system triggers the instruction set and interaction adaptation strategy for processing. First, the abnormal blood pressure trigger instruction is linearly superimposed with the interaction adaptation strategy suitable for voice and visual prompts. According to the pre-set mapping rules, the superposition result is mapped to the corresponding prompt pattern set. At this time, the smart speaker in the ward issues a voice prompt: "Ms. Zhang, your blood pressure is a bit high. Please remain quiet and avoid strenuous activities. Medical staff will be here to check on you soon." Simultaneously, the display screen next to the bed displays "Abnormal blood pressure, please pay attention" in eye-catching red font, accompanied by a comparison chart of the normal blood pressure range and Ms. Zhang's current blood pressure data, allowing her to intuitively understand her condition. The prompt pattern under this static matching model is generated based on established rules, and the same prompt action is consistently executed in similar situations.

[0080] Consider Mr. Li, who was hospitalized for gastric surgery. During his recovery, the system detected abnormal heart rate fluctuations, triggering an instruction set to generate an abnormal heart rate alert. The interactive adaptation strategy revealed that Mr. Li had poor vision but normal hearing.

[0081] The situation is different when the preset accessibility interaction matrix adopts a dynamic adaptation model. An incremental learning algorithm comes into play, optimizing the correlation between the trigger instruction set and the interaction adaptation strategy in real time based on the system's previously accumulated data and the current situation. In the past, when encountering patients like Mr. Li with an abnormal heart rate and poor vision, directly issuing loud voice prompts might frighten the patient and cause their heart rate to fluctuate further. Based on this experience, the algorithm adjusts the prompt mode. First, the smart speaker speaks in a relatively soft and slow voice: "Mr. Li, don't worry. Your heart rate is fluctuating. We will address it immediately." Simultaneously, a vibration device next to Mr. Li's bed vibrates gently and rhythmically to remind him, preventing the sudden voice startle and increased psychological burden. As time goes by and data accumulates, if the system finds that Mr. Li has an abnormal heart rate while sleeping at night, the incremental learning algorithm may further optimize the prompt mode. For example, it may first wake Mr. Li up with a very slight vibration, and then gently inform him of the situation, so as to minimize the interference to Mr. Li and ensure that he knows his condition in a timely manner. This dynamic adaptation model allows the prompt mode to better adapt to various complex scenarios and the personalized needs of patients.

[0082] Example 4:

[0083] The device collaboration module is connected to the ward terminal device database via a cross-device communication interface. When the central control unit generates the final interaction instruction, the device collaboration module will filter the available device group from the ward terminal device database based on the device operation requirements in the instruction. The ward terminal device database stores detailed information on all devices in the ward, including device type, device status (whether it is available, whether it is under maintenance, etc.), device location, etc. For example, if the final interaction instruction requires voice prompts and visual prompts, the device collaboration module will search the database for available voice prompt devices (such as smart speakers) and visual prompt devices (such as display screens) to form an available device group.

[0084] When generating the interaction execution queue, the device collaboration module first loads the ward's spatial topology model. This model details the layout of each area within the ward and the spatial location nodes of devices, such as the smart speaker installed above the left side of the bed and the display screen on the wall directly in front of the bed. An energy-balancing algorithm is then used to calculate the optimal response path from each device's current state to the target interaction mode. This energy-balancing algorithm comprehensively considers factors such as device energy consumption and response speed. For example, if a smart speaker with a low battery level is used, the algorithm may prioritize other speakers with sufficient battery and faster response speed for voice prompts. Furthermore, resources are allocated to available device groups based on device priority, which can be determined based on factors such as device importance and stability. Finally, the optimal response path and resource allocation are integrated into the spatial topology model to generate a three-dimensional interaction execution queue. This queue not only considers the device operation sequence but also their spatial relationships, making prompt operations more efficient and orderly, and avoiding conflicts and interference between devices.

[0085] In a modern surgical ward, Mr. Zhao, a patient, had just undergone a complex abdominal surgery and was in the postoperative observation phase. At this point, the device collaboration module within the ward's multimodal interactive surgical ward intelligent notification system began to play a key role.

[0086] Based on Mr. Zhao's condition and current state, the central control unit generated a final interactive instruction. This instruction required both voice and visual prompts, informing Mr. Zhao that he would need to cooperate with the nurse to examine his wound and reminding him to remain relaxed. After receiving this instruction, the device collaboration module accessed the ward terminal device database through the cross-device communication interface.

[0087] The ward terminal device database records in detail the information of all devices in the ward, including the type of device, current operating status, and specific location in the ward. The device collaboration module begins to screen the available device groups based on the device operation requirements in the instructions. In terms of voice prompt devices, the database shows that there are two smart speakers available in the ward, one located above the left side of the bed and the other in the corner of the ward; in terms of visual prompt devices, the small display screen next to the bed and the large screen in the public area of ​​the ward are in normal operation. After screening, the device collaboration module determined the available device group consisting of the smart speaker above the left side and the small display screen next to the bed, because these two devices are closest to Mr. Zhao and can ensure that he receives prompt information more clearly and promptly.

[0088] The device coordination module loads the ward's spatial topology model. This model acts like a digital map of the ward, precisely depicting the layout of each area and the spatial locations of all devices. The locations of the smart speaker and small display are clearly marked in the model. Based on an energy-balancing algorithm, the device coordination module calculates the optimal response path from each device's current state to the target interaction mode. For example, if the smart speaker's volume is currently low, it needs to be adjusted to clearly provide Mr. Zhao with prompts. The energy-balancing algorithm considers factors such as the power consumption of adjusting the volume and the time required to reach the target volume. Furthermore, the small display next to Mr. Zhao's bed is closer to him and more directly and effectively displays prompts, so it receives a higher priority. Based on device priorities, the device coordination module allocates resources among available device groups, prioritizing the small display to display prompts promptly and accurately, while also rationally allocating resources to the smart speaker to ensure the quality of voice prompts.

[0089] Finally, the device collaboration module integrates the optimal response path and resource allocation into the spatial topology model, generating a three-dimensional interactive execution queue. This queue specifies that the small display screen will first display a text message stating, "Mr. Zhao, a nurse will now examine your wound. Please remain relaxed." Simultaneously, the smart speaker plays a voice prompt with the same message at a moderate volume. This approach not only determines the device operation sequence but also fully considers factors such as the spatial location of the devices and energy consumption. This makes prompt operations more efficient and orderly, avoids conflicts and interference between devices, and provides Mr. Zhao with a positive prompt experience, helping him better cooperate with medical staff.

[0090] Example 5:

[0091] The instruction distribution module is used to convert the final interaction instruction generated by the central control unit into a device control code, and send the device control code to the designated ward terminal device through the cross-device communication interface to activate the prompt program. When the instruction distribution module receives the final interaction instruction, it will convert the instruction into the corresponding device control code according to different device types and communication protocols. For example, for smart speakers, the encoding format corresponding to a specific Bluetooth communication protocol may be used; for display screens, the display instruction code under the network communication protocol is used. The converted device control code is sent out through the cross-device communication interface. The cross-device communication interface will accurately send the device control code to the designated ward terminal device based on the target device information in the instruction. During the sending process, data verification will be performed to ensure the accuracy and completeness of the code.

[0092] When the ward terminal device receives the device control code, it parses the code and identifies the corresponding prompt operation instructions. For example, after receiving the code, the smart speaker will provide a voice prompt based on the voice content and voice broadcast parameters such as volume and speaking speed in the code; after receiving the code, the display screen will display the corresponding text, images, and other prompt information on the screen according to the display content and display format requirements in the code. The work of the instruction distribution module completes the system's complete process from generating the final interactive command to executing the prompt operation on the ward terminal device, ensuring that patients can receive various prompt information in a timely and accurate manner.

[0093] In a hospital's surgical ward, Ms. Chen, a patient, had just completed breast surgery and was recovering from surgery. The ward was equipped with a multimodal interactive surgical ward intelligent reminder system. Based on Ms. Chen's recovery progress and real-time monitoring data, the central control unit generated a final interactive instruction, reminding her to perform postoperative rehabilitation exercises on schedule and instructing her on the key techniques. This instruction was then transmitted to a command distribution module connected to the central control unit.

[0094] After receiving the final interaction command, the command distribution module must first convert it into a device control code. Assume that a custom communication protocol is used between the command distribution module and the ward terminal devices (such as the smart display and smart speaker next to the bed). This protocol specifies the format of the device control code as: [Device Type Identifier][Command Content Identifier][Check Code]. Here, the "Device Type Identifier" is used to distinguish different terminal devices, such as "01" for a smart display and "02" for a smart speaker; the "Command Content Identifier" encodes the specific prompt content, with different numeric combinations corresponding to different prompts; and the "Check Code" is used to verify the accuracy of the code during transmission. It is generated using a specific algorithm. Assuming a simple parity check algorithm, the formula is: Check Code = (Device Type Identifier + Command Content Identifier) ​​% 2, where "%" represents the remainder operation, the "Check Code" is the calculation result, used to verify the correctness of the data transmission, and the "Device Type Identifier" and "Command Content Identifier" are the numerical values ​​of the corresponding parts of the command.

[0095] The instruction distribution module converts the instruction prompting Ms. Chen to perform rehabilitation exercises into a device control code for the smart display: Assuming the device type identifier for the smart display is "01," the instruction content identifier for the rehabilitation exercise prompt is encoded internally as "100101." Using the checksum formula above, (01 + 100101) % 2 = 0, the checksum is "0," resulting in the device control code "011001010." The device control code for the smart speaker is generated similarly.

[0096] After generating the device control codes, the command distribution module sends these codes to the designated ward terminal devices through the cross-device communication interface. During the transmission process, the communication interface packages and transmits the codes, while using a specific transmission protocol to ensure reliable data transmission.

[0097] After receiving the device control code, the smart display and smart speaker next to the bedside parsed the code. The smart display recognized the device type identifier "01" and determined it was a command for itself. It then parsed the command content identifier "100101" and converted it into specific display content, displaying the message "Ms. Chen, it's time to practice postoperative rehabilitation exercises. The key points are as follows:..." on the screen, accompanied by images demonstrating the movements. The smart speaker, recognizing its own device type identifier "02," parsed the command content and converted it into a voice signal, playing the corresponding prompt. This allowed Ms. Chen to receive both visual and auditory prompts, allowing her to better cooperate with postoperative rehabilitation exercises.

[0098] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0099] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A multimodal interactive surgical ward intelligent prompt system, characterized by: include: A patient status perception module is used to obtain patient identity information and physiological status data, and divide the data into a basic vital sign data stream and an interaction demand data stream; A multimodal trigger module, configured to perform dynamic priority queue division on the basic vital sign data stream to generate a trigger instruction set, and perform behavior trajectory prediction on the interaction demand data stream to generate an interaction adaptation strategy; A prompt mode adaptation module is used to perform logical matching between the trigger instruction set and the interaction adaptation strategy and then map them to a corresponding prompt mode set according to a preset barrier-free interaction matrix; A timing control module is used to call the surgery node parameters in the preset timeline database based on the prompt mode set, and use the surgery node parameters as the execution timing benchmark for the current patient; The central control unit is used to perform multi-channel verification on the prompt mode set and the execution timing benchmark to generate a final interactive instruction.

2. The multimodal interactive surgical ward intelligent prompt system according to claim 1, characterized in that: The multimodal trigger module dynamically prioritizes the basic vital sign data stream, including: Dividing the continuous monitoring signals in the basic vital sign data stream into a static vital sign group and a dynamic behavior group, and performing action intention analysis on the dynamic behavior group based on a preset behavior trajectory prediction algorithm to generate a trigger feature set; Performing multi-dimensional feature extraction on the identity recognition data in the basic vital sign data stream, constructing a patient status portrait and generating an interaction weight coefficient; The trigger feature set is associated with the interaction weight coefficient in time series to generate the trigger instruction set.

3. The multimodal interactive surgical ward intelligent prompt system according to claim 1, characterized in that: The preset barrier-free interaction matrix includes a basic interaction set and an enhanced interaction set; the basic interaction set includes a voice broadcast identifier, a visual display identifier, and a tactile feedback identifier; the enhanced interaction set includes a Braille auxiliary identifier and a child adaptation identifier, and each identifier corresponds to an independent interaction control channel.

4. The multimodal interactive surgical ward intelligent prompt system according to claim 3, characterized in that: The system further includes a cross-device communication interface, which is used to synchronize data between the patient status perception module, the multimodal triggering module, the prompt mode adaptation module, and the timing control module and a hospital information management system; The patient status perception module divides the data stream based on the biometric recognition algorithm, including: Obtaining surgery scheduling data packets from the hospital information management system in real time through the cross-device communication interface, and matching the protocol fields of the data packets with the patient numbers in the basic vital sign data stream to extract basic timing parameters; Traversing the extension field of the data packet according to the patient type code in the interaction requirement data stream to parse special interaction requirements; The basic timing parameters and special interaction requirements are calibrated for time axis offset according to the surgical nodes and then written into the basic instruction queue and the enhanced instruction cache respectively.

5. The multimodal interactive surgical ward intelligent prompt system according to claim 1, characterized in that: When the preset barrier-free interaction matrix adopts a static matching model, the prompt mode set is a discrete mapping set of linear superposition results of the trigger instruction set and the interaction adaptation strategy; When the preset barrier-free interaction matrix adopts a dynamic adaptation model, the prompt pattern set is a continuous pattern sequence that optimizes the correlation between the trigger instruction set and the interaction adaptation strategy in real time through an incremental learning algorithm.

6. The multimodal interactive surgical ward intelligent prompt system according to claim 1, characterized in that: It also includes a device collaboration module connected to the central control unit, and the device collaboration module is connected to the ward terminal device database through the cross-device communication interface; The device collaboration module is used to screen available device groups from the ward terminal device database according to the device operation requirements in the final interaction instruction, and generate an interaction execution queue to optimize the prompt response process.

7. The multimodal interactive surgical ward intelligent prompt system according to claim 6, characterized in that: The device collaboration module generates an interactive execution queue including: Loading a ward spatial topology model, and locating a spatial location node of each device in the available device group in the model; Calculating an optimal response path from the current state of each device to the target interaction mode based on an energy consumption balancing algorithm, and allocating resources to the available device group according to device priority; The optimal response path and the resource allocation are integrated into the spatial topology model to generate a three-dimensional interactive execution queue.

8. The multimodal interactive surgical ward intelligent prompt system according to claim 1, characterized in that: When the central control unit performs multi-channel verification on the prompt mode set and the execution timing benchmark, a dual verification mode of a signal integrity verification mechanism and a timing conflict detection mechanism is adopted. The signal integrity verification mechanism is used to confirm the validity of data collection, and the timing conflict detection mechanism is used to solve the response delay problem between multiple devices.

9. The multimodal interactive surgical ward intelligent prompt system according to claim 1, characterized in that: It also includes an instruction distribution module connected to the central control unit, which is used to convert the final interaction instruction into a device control code and send the device control code to the designated ward terminal device through the cross-device communication interface to activate the prompt program.

10. A multimodal interactive surgical ward intelligent prompt method, characterized in that: The following steps are involved: Step 1: Acquire patient identity information and physiological status data, and divide the data into basic vital sign data stream and interaction demand data stream; Step 2: performing dynamic priority queue division on the basic vital sign data stream to generate a trigger instruction set, and performing behavior trajectory prediction on the interaction demand data stream to generate an interaction adaptation strategy; Step 3: According to a preset barrier-free interaction matrix, the trigger instruction set and the interaction adaptation strategy are logically matched and mapped to a corresponding prompt mode set; Step 4: calling the surgical node parameters in the preset timeline database based on the prompt pattern set, and using the surgical node parameters as the execution timing benchmark for the current patient; Step 5: Perform multi-channel verification on the prompt mode set and the execution timing benchmark to generate a final interactive instruction.

Images