A method and system for preventing adverse events from nursing administration

Abstract

This invention relates to the field of adverse nursing care technology and discloses a method and system for preventing adverse drug events. The method includes constructing a spatial digital model based on 3D point cloud within a ward, dividing the ward into public and private areas. Multimodal authentication is used to determine in real time whether authorized medical personnel are located within the private area. When at least one medical personnel is confirmed to be within the private area, a camera is activated to capture images of the private area, obtaining image data and continuous physiological signals from the patient. The image data and physiological signals are then subjected to multimodal fusion analysis. This invention significantly improves the accuracy of adverse event identification and early warning capabilities through multimodal fusion analysis, reducing false alarms and missed alarms. Furthermore, by leveraging edge intelligence and automated emergency dispatch, it significantly shortens rescue response time.

Description

Technical Field

[0001] This invention relates to the field of prevention of adverse nursing events, specifically a method and system for preventing adverse events related to medication administration. Background Technology

[0002] Adverse nursing events refer to events that result in patient death, prolonged hospital stay, or disability upon discharge due to medical and nursing care practices. They are divided into preventable and unpreventable categories. Common types include falls during hospitalization, medication errors, and damage to medical devices, involving specific situations such as medication errors, falls from beds, and tube dislodgement. The causes of these events are often related to factors such as lax implementation of verification procedures, errors in medical order execution, and insufficient patient assessment. It is necessary to reduce the incidence of these events by improving core nursing systems and strengthening operational standards. Medical institutions have generally established proactive reporting systems and adopted tiered handling and root cause analysis mechanisms to improve patient safety.

[0003] However, existing methods and systems for preventing adverse drug events only focus on nurses' nursing behaviors and do not fully consider the patient's condition. For example, if a patient with a chest drainage tube becomes agitated at night and tries to remove the tube, nurses will not monitor the patient constantly, which may lead to an accident. Furthermore, after an accident, especially at night when manpower is scarce, incorrect rescue actions are more likely to occur, thus delaying the patient's rescue time. Summary of the Invention

[0004] This invention provides a method and system for preventing adverse events related to medication administration, which has the beneficial effects of early warning and correct guidance.

[0005] This invention provides the following technical solution: a method and system for preventing adverse events related to medication administration, comprising the following steps:

[0006] A spatial digital model was constructed in the ward based on 3D point cloud, and public and private areas were divided accordingly.

[0007] Multimodal authentication is used to determine in real time whether authorized medical personnel are located within the privacy area;

[0008] When it is confirmed that at least one medical staff member is located within the privacy area, the camera equipment is activated to capture images of the privacy area and obtain image data.

[0009] Acquire continuous physiological signals from the patient;

[0010] The image data and the physiological signals are subjected to multimodal fusion analysis. A spatiotemporal alignment mechanism is used to associate visual events with physiological mutations, and a graph attention network is used to determine whether there is a risk of adverse events.

[0011] If the risk score exceeds a preset threshold, the structured adverse event knowledge base is invoked, and combined with patient information, an emergency response instruction is generated that includes personnel scheduling, medication dispensing, and operational guidelines.

[0012] As an optional embodiment of the method and system for preventing adverse drug events during medication as described in this invention, the method of multimodal fusion analysis includes:

[0013] Perform intelligent verification of drug appearance on the image data;

[0014] OCR technology is used to identify the name, batch number, and expiration date of medications in the ward.

[0015] The liquid region of the drug is extracted by image segmentation, and its color value, transparency and fill rate are calculated;

[0016] The identification results are compared with the corresponding parameters in the pharmacy's standard drug digital archives in multiple dimensions.

[0017] If any parameter deviation exceeds the preset threshold, a drug verification alarm will be triggered, requiring the nurse to perform a manual review.

[0018] The manual verification process includes checking the odor, shaking to observe the sedimentation state, and scanning the original outbound code.

[0019] As an optional solution to the method and system for preventing adverse drug events in nursing care according to the present invention, the emergency response instruction includes locating the qualified and nearest doctor and nurse from the scheduling information according to the type of adverse event;

[0020] The system calculates the optimal route using Bluetooth beacons and indoor positioning, and sends directional voice alerts and navigation guidance to doctors and nurses.

[0021] The system simultaneously controls the illumination of indicator lights along the ward route and sends location information for the retrieval of necessary medications and emergency equipment to nurses.

[0022] As an alternative to the method and system for preventing adverse drug events during medication as described in this invention, it further includes:

[0023] Once it is confirmed that medical staff have entered the privacy area, a countdown will begin.

[0024] During the countdown, a pose estimation algorithm is continuously used to detect whether nursing-related actions occur;

[0025] If no nursing-related actions are detected by the end of the countdown, the capture of images in the privacy area will be automatically turned off, the event log will be recorded as a suspected accidental entry, and an audit alert will be sent to the head nurse's terminal.

[0026] As an optional solution to the method and system for preventing adverse drug events described in this invention, it further includes establishing a digital drug file and placing the drug in a standard light source box when it is first put into storage.

[0027] Capture images from multiple angles and simultaneously acquire near-infrared spectral data;

[0028] The standard Lab value of liquid medicines was determined by a Lab color analyzer, and typical degradation characteristics were manually labeled, including turbidity, precipitation, stratification, and color deepening.

[0029] The above near-infrared spectral data and feature data, together with drug ID, dosage form, packaging material, and storage temperature and humidity requirements, are structured and stored to form a digital twin of the drug;

[0030] When a new batch of drugs enters the warehouse, its appearance parameters are automatically compared with those of historical batches. If the deviation exceeds the limit, a drug quality inspection review is triggered.

[0031] As an alternative to the method and system for preventing adverse drug events in nursing care as described in this invention, when a nurse is detected entering the bedside area with a drug in hand, an overall image of the drug is captured, and the drug information on the label is identified using a PaddleOCR model.

[0032] The U-Net++ segmentation network was used to extract the bottle body region and liquid surface contour from the image, and the fill rate was calculated.

[0033] ;

[0034] In the CIELAB color space, the average Lab value of the liquid surface area is calculated and compared with the standard color vector stored in the pharmacy's standard drug digital archive. Color difference calculation:

[0035] ;

[0036] in, , and The color value of the standard medicine. , and The measured color values ​​of medications in the ward;

[0037] An anomaly is identified and a level 2 verification alarm is triggered when at least one of the following conditions occurs:

[0038] The OCR detected a drug name that did not match the doctor's prescription.

[0039] The fill rate is lower than the standard outbound range;

[0040] The color difference exceeds the set threshold.

[0041] An image gradient entropy exceeding the turbidity threshold indicates that the liquid is opaque.

[0042] When a Level 2 verification alarm is triggered, the medication administration process is forcibly suspended, and a review guide is displayed on the nurse's terminal to re-examine the medicine bottle.

[0043] As an alternative to the method and system for preventing adverse drug events during medication as described in this invention, the physiological signals include heart rate, respiratory rate, blood oxygen saturation, and non-invasive blood pressure.

[0044] The heart rate and blood oxygen saturation are based on dual-wavelength photoplethysmography pulse wave signals. After eliminating motion artifacts through adaptive filtering, the instantaneous heart rate is calculated using a peak detection algorithm, and the blood oxygen saturation is calculated using a ratio method.

[0045] The respiratory rate is extracted by the baseline drift period of the PPG signal or the change in thoracic impedance of the integrated impedance respiratory sensor.

[0046] The non-invasive blood pressure measurement is based on pulse wave conduction time, i.e. the time delay between the ECG R wave and the peripheral PPG main wave. Combined with the patient's age, height, and historical calibration parameters, the systolic and diastolic blood pressure are dynamically estimated through a linear regression model.

[0047] The present invention also provides a system for a method of preventing adverse events associated with medication administration, comprising:

[0048] The camera module is used to collect 3D point cloud data to construct a digital model of the ward space, and to define public areas and private areas in the model;

[0049] A multimodal authentication module is used to fuse UWB 3D coordinate positioning results with lightweight authentication results. The model's visual identity recognition results are cross-validated with the authorized list in the hospital's human resources system to determine in real time whether there are medical staff with nursing privileges located in the privacy area;

[0050] After receiving the authorization signal from the multimodal authentication module, the camera device enables image acquisition of the privacy area;

[0051] Semantic segmentation unit based on The model performs pixel-level occlusion on the video stream and applies Gaussian blur to the patient's face and torso even when monitoring is enabled, leaving only the areas related to nursing procedures clear.

[0052] A multi-parameter physiological signal acquisition module is used to continuously output non-invasive blood pressure data such as heart rate, respiratory rate, blood oxygen saturation, and pulse wave conduction time estimation.

[0053] The multimodal fusion analysis module includes:

[0054] The visual event detection submodule is used to extract timestamps of drug administration operations and facial flushing events through pose estimation, facial expression recognition, and target detection algorithms.

[0055] The physiological mutation detection submodule is used to identify sudden changes in heart rate. Rapid decline, respiratory depression, and hypotensive events;

[0056] The graph attention network reasoning unit constructs a heterogeneous graph with the patient's current state, historical medical history and environmental context as nodes, and outputs the risk probability of three types of adverse events: falls, drug allergies and cardiac arrest precursors.

[0057] The drug verification module automatically captures images, identifies drug text information, segments the liquid surface area, and calculates the fill rate and its relationship to the CIELAB color space when it detects a drug entering the bedside area. Color difference is detected and compared with the pre-stored digital twin of the drug; if the deviation exceeds the limit, a level 2 verification alarm is triggered.

[0058] When the risk probability output by the multimodal fusion analysis engine exceeds the threshold, the structured adverse event knowledge base is invoked to generate an emergency response instruction that includes the target medical staff location, optimal route planning, AR navigation guidance, control instructions for LED indicator lights along the way, and a list of emergency supplies to be retrieved, and then pushed to the medical staff via Bluetooth.

[0059] The present invention has the following beneficial effects:

[0060] 1. The method and system for preventing adverse drug events through medication significantly improve the accuracy of adverse event identification and early warning capabilities through multimodal fusion analysis, reduce false alarms and missed alarms, and greatly shorten rescue response time by leveraging edge intelligence and automated emergency dispatch.

[0061] 2. This method and system for preventing adverse drug events through multidimensional intelligent verification significantly reduces the risk of medication errors caused by similar drug appearance, label wear, liquid deterioration, or human negligence. By combining objective quantitative indicators with subjective manual review, it not only improves the scientific rigor and consistency of verification but also preserves the final judgment of clinical personnel. All comparison processes are completed at the edge, ensuring the security and real-time processing of drug information.

[0062] 3. This method and system for preventing adverse drug events by constructing a high-fidelity digital twin of the drug and using standard light sources and instrumented measurements ensures the accuracy and repeatability of the data. It also enhances the ability to identify slightly deteriorated or counterfeit drugs by using multimodal data fusion, and the automated batch consistency verification significantly improves the efficiency of pharmacy warehousing. It proactively intercepts potential medication risks and provides a source guarantee for safe clinical medication use. Attached Figure Description

[0063] Figure 1 This is a schematic diagram of the overall operation process of the present invention.

[0064] Figure 2 This is a schematic diagram of the drug appearance verification process of the present invention.

[0065] Figure 3 This is a schematic diagram of the process for constructing and verifying the digital twin of the drug according to the present invention.

[0066] Figure 4 This is a schematic diagram of the physiological signal acquisition and processing flow of the present invention.

[0067] Figure 5 This is a schematic diagram of the emergency response command generation and execution process of the present invention.

[0068] Figure 6 This is a schematic diagram of the privacy area image acquisition control process of the present invention. Detailed Implementation

[0069] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0070] Example 1

[0071] Please see Figures 1-6 One method for preventing adverse events related to medication administration includes the following steps:

[0072] A spatial digital model was constructed in the ward based on 3D point cloud, and public and private areas were divided accordingly.

[0073] Multimodal authentication is used to determine in real time whether authorized medical personnel are located within a privacy zone;

[0074] Once it is confirmed that at least one medical staff member is located within the privacy area, the camera equipment is activated to capture images of the privacy area and obtain image data.

[0075] Acquire continuous physiological signals from the patient;

[0076] Multimodal fusion analysis of image data and physiological signals is performed, a spatiotemporal alignment mechanism is used to associate visual events with physiological mutations, and a graph attention network is used to determine whether there is a risk of adverse events.

[0077] If the risk score exceeds a preset threshold, the structured adverse event knowledge base is invoked, and combined with patient information, an emergency response instruction is generated that includes personnel scheduling, medication dispensing, and operational guidelines.

[0078] Specifically, firstly, depth cameras or lidar devices are deployed in the ward to collect environmental point cloud data in real time, and a high-precision spatial digital model is built based on this 3D point cloud. On this model, public areas (such as the doorway and visible corridor areas) and private areas (at least including a monitoring range with a radius of 1.2 meters centered on the patient's bed) are automatically divided to ensure accurate identification and management of sensitive spaces.

[0079] Secondly, a multimodal authentication mechanism is used to determine in real time whether authorized medical personnel are located within the privacy area. This verification integrates the three-dimensional coordinate information provided by the UWB / BLE dual-mode positioning beacon with the visual identity results output by a lightweight pedestrian re-identification algorithm running on an RGB-D camera, and cross-compares it with the nursing qualification list in the hospital's human resources department to achieve highly reliable presence confirmation.

[0080] When at least one healthcare worker with the current patient care authority is confirmed to be present within the privacy area, a high-definition camera is automatically activated to capture images of the privacy area, obtaining multispectral image data including visible light and near-infrared light. If no authorized personnel are present, the privacy area is immediately dynamically masked or the recording is stopped, protecting patient privacy from the outset. Simultaneously, wearable physiological sensors attached to the patient's body continuously collect key physiological signals such as heart rate, respiratory rate, blood oxygen saturation, and non-invasive blood pressure, forming a high-temporal-resolution vital signs stream. The acquired image data and physiological signals are then input into a multimodal fusion analysis engine deployed on the ward's local edge computing gateway. This engine first detects visual events, such as medication administration and facial flushing, and physiological abrupt events, such as sudden drops in heart rate. Rapid decline, respiratory depression, and then through a spatiotemporal alignment mechanism within a ±5-second time window, the co-occurring visual and physiological abnormalities are associated as a composite event;

[0081] Furthermore, based on the graph attention network model, combined with contextual information such as the patient's medical history, current medication, and nursing level, personalized risk scores are performed for high-risk adverse nursing events such as falls, drug allergies, and cardiac arrest precursors.

[0082] Finally, when the risk score of any event exceeds the preset threshold, the structured adverse event knowledge base is automatically invoked. Combined with individualized data such as allergy history and diagnostic information in the patient's electronic medical record, an integrated emergency response command is generated, which includes dispatching qualified medical personnel nearby, pushing the optimal route navigation, indicating the location of emergency medicines and equipment, and step-by-step operation guidance. The command is then executed simultaneously through wearable terminals, ward broadcasts, and smart lighting.

[0083] Multimodal fusion analysis significantly improves the accuracy of adverse event identification and early warning capabilities, reduces false alarms and missed alarms, and greatly shortens rescue response time by leveraging edge intelligence and automated emergency dispatch.

[0084] Example 2

[0085] This embodiment is an improvement upon embodiment 1. For details, please refer to [link / reference]. Figures 1-6 Modal fusion analysis methods include:

[0086] Perform intelligent verification of drug appearance using image data;

[0087] OCR technology is used to identify the name, batch number, and expiration date of medications in the ward.

[0088] The liquid region of the drug is extracted by image segmentation, and its color value, transparency and fill rate are calculated;

[0089] The identification results are compared with the corresponding parameters in the pharmacy's standard drug digital archives from multiple dimensions.

[0090] If any parameter deviation exceeds the preset threshold, a drug verification alarm will be triggered, requiring the nurse to perform a manual review.

[0091] The manual verification process includes checking the odor, shaking to observe the sedimentation state, and scanning the original outbound code.

[0092] Multimodal fusion analysis methods include performing intelligent drug appearance verification on image data. Specifically, when a nurse is detected carrying medication into the patient bed area, the drug verification process is automatically triggered.

[0093] First, a high-resolution camera is used to capture an overall image of the drug, and optical character recognition technology, especially deep learning-based text detection and recognition models such as PaddleOCR, is used to accurately extract key text information such as the drug name, specifications, batch number, and expiration date from the drug label.

[0094] Secondly, image semantic segmentation algorithms, such as the U-Net++ network, are used to perform pixel-level analysis on the medicine bottle in the image, accurately separating the liquid area. Based on this, multiple physical feature parameters are calculated, including the average color value in the CIELAB color space, the transparency evaluated based on the image gradient entropy or light scattering model, and the fill rate determined by the ratio of the liquid level height to the total height of the bottle.

[0095] Subsequently, the identification and calculation results were compared with a pre-constructed digital archive of standard pharmacy drugs across multiple dimensions. This digital archive, established when the drug is first received, contains structured data such as standard color vectors, typical fill ranges, normal transparency benchmarks, and packaging characteristics. The comparison dimensions covered drug name consistency, batch number validity, and color deviation (as shown in the original text). (Color difference quantification), whether the fill rate is abnormally low, and whether the liquid is cloudy or opaque, etc. If the deviation of any parameter exceeds the preset threshold, such as the OCR-recognized drug name not matching the current prescription, or the fill rate being less than 90% and not a reusable medicine bottle, etc. If the color difference is greater than 5.0 or the transparency index exceeds the normal range, a medication risk is identified, and a medication verification alarm is immediately triggered. At this time, the medication administration process is forcibly paused, and a secondary review prompt appears on the nurse's terminal, requiring manual review.

[0096] The manual verification process includes smelling the medicine to determine if it is rancid or has an off-odor, gently shaking the bottle to observe for abnormal sediment, flocculent matter or stratification, and scanning the original outbound QR code on the bottom of the bottle or outer packaging to verify its origin and batch authenticity.

[0097] Compared to traditional manual visual verification, this embodiment significantly reduces the risk of medication errors due to similar drug appearance, label wear, liquid deterioration, or human negligence through multi-dimensional intelligent verification. By combining objective quantitative indicators with subjective manual review, it improves the scientific rigor and consistency of verification while preserving the final judgment of clinicians. Furthermore, all comparison processes are completed at the edge, ensuring the security and real-time processing of drug information.

[0098] Example 3

[0099] This embodiment is an improvement upon embodiment 2. For details, please refer to [link / reference]. Figures 1-6 Emergency response instructions include locating the nearest qualified doctor or nurse from the scheduling information based on the type of adverse event;

[0100] The system calculates the optimal route using Bluetooth beacons and indoor positioning, and sends directional voice alerts and navigation guidance to doctors and nurses.

[0101] The system simultaneously controls the illumination of indicator lights along the ward route and sends location information for the retrieval of necessary medications and emergency equipment to nurses.

[0102] Based on the adverse event type determined by the multimodal fusion analysis engine (such as falls, drug allergies, and cardiac arrest precursors), the system automatically matches preset emergency response protocols and retrieves currently on-duty medical staff with the corresponding response qualifications from the hospital's real-time scheduling information.

[0103] Based on this, combined with indoor positioning data (obtained through Bluetooth 5.0 beacons and UWB base stations deployed in corridors and wards), the physical distance and travel time from the current location of each candidate medical staff member to the target ward are calculated, and the doctor and nurse who are closest and respond the fastest are selected as priority dispatch targets.

[0104] Subsequently, based on the hospital's building information model or high-precision indoor map, path planning algorithms, such as Dijkstra's or A* algorithms, are used to dynamically generate the optimal travel route for the dispatched personnel. Directional voice alerts (e.g., "Patient in bed 3 is suspected of having an allergic reaction; please proceed immediately") are sent via their smart bracelets, Bluetooth headsets, or mobile terminals, along with visual navigation guidance overlaid on camera footage or AR glasses, ensuring rapid and accurate arrival. Simultaneously, the IoT control platform is linked to automatically illuminate embedded LED indicators or wall-mounted guide lights along the route from the medical staff's current location to the target ward, forming a prominent light-guided path. Furthermore, based on the event type, a structured emergency resource list is pushed to the nurse's terminal, clearly indicating the cabinet number of the required medications (such as adrenaline and dexamethasone), the specific location of the defibrillator or oxygen cylinder, and key points for retrieving them.

[0105] This transforms the traditional passive response model that relies on manual calling and memory-based object retrieval into an intelligent, automated, and precise scheduling system. Through multi-terminal linkage, such as voice, vision, lighting, and information push, the system effectively reduces the cognitive load and decision-making delay of medical staff in emergency situations, thereby improving the success rate of rescue.

[0106] Example 4

[0107] This embodiment is an improvement upon embodiment 3. For details, please refer to [link / reference]. Figures 1-6 This also includes establishing digital files for drugs and placing them in a standard light source box when they first enter the warehouse;

[0108] Capture images from multiple angles and simultaneously acquire near-infrared spectral data;

[0109] The standard Lab value of liquid medicines was determined by a Lab color analyzer, and typical degradation characteristics were manually marked, including turbidity, precipitation, layering, and color deepening.

[0110] The above near-infrared spectral data and feature data, together with drug ID, dosage form, packaging material, and storage temperature and humidity requirements, are structured and stored to form a digital twin of the drug;

[0111] When a new batch of drugs enters the warehouse, its appearance parameters are automatically compared with those of historical batches. If the deviation exceeds the limit, a drug quality inspection review is triggered.

[0112] When a nurse is detected entering the bedside area with medication in hand, an image of the entire medication is captured, and the medication information on the label is identified using the PaddleOCR model.

[0113] The U-Net++ segmentation network was used to extract the bottle body region and liquid surface contour from the image, and the fill rate was calculated.

[0114] ;

[0115] In the CIELAB color space, the average Lab value of the liquid surface area is calculated and compared with the standard color vector stored in the pharmacy's standard drug digital archive. Color difference calculation:

[0116] ;

[0117] in, , and The color value of the standard medicine. , and The measured color values ​​of medications in the ward;

[0118] An anomaly is identified and a level 2 verification alarm is triggered when at least one of the following conditions occurs:

[0119] The OCR detected a drug name that did not match the doctor's prescription.

[0120] The fill rate is lower than the standard outbound range;

[0121] The color difference exceeds the set threshold.

[0122] An image gradient entropy exceeding the turbidity threshold indicates that the liquid is opaque.

[0123] When a Level 2 verification alarm is triggered, the medication administration process is forcibly suspended, and a review guide is displayed on the nurse's terminal to re-examine the medicine bottle.

[0124] Specifically, when the medicine is first put into storage, it is placed in a standard light source box that meets international standards, such as CIED65 daylight simulation, to ensure constant lighting conditions, illuminance of 500 lux and color temperature of 6500K, so as to eliminate the interference of ambient light on the judgment of color and appearance.

[0125] Subsequently, images of the drug were captured from multiple angles, including front, side, and top views, using a camera. Simultaneously, a near-infrared spectrometer was activated to collect reflectance or transmission spectral data in the 700-2500nm wavelength range, used to characterize the intrinsic features of the drug's components and physical state. At the same time, a calibrated Lab color analyzer was used to measure the liquid surface area of ​​the liquid drug, obtaining its standard Lab value in the CIELAB color space as a color reference. Based on visual observation and historical experience, professional pharmacists or quality inspectors manually labeled the drug with typical degradation or deterioration characteristics, including liquid turbidity, visible sediment, oil-water separation or stratification, and abnormal states such as color deepening or shift. This multi-source data, including multi-angle images, near-infrared spectral curves, standard Lab values, degradation labeling, and basic drug attribute information (such as drug ID, generic name, trade name, dosage form, specifications, packaging material, expiration date, storage temperature and humidity requirements, and supplier information), was uniformly structured and stored in the hospital's main drug database, forming a version-manageable, traceable, and comparable digital twin of the drug.

[0126] When a new batch of the same drug is received into the warehouse, the corresponding digital twin of the drug is automatically used as a comparison benchmark, and its newly collected appearance parameters (such as color) are compared. Automated consistency checks are performed on parameters such as fill consistency, transparency, and near-infrared spectral similarity. If the deviation of any key parameter exceeds the preset tolerance (e.g., fill consistency, transparency, near-infrared spectral similarity), the consistency check will be performed. If the value is >3.0 or the spectral correlation coefficient is <0.95, it is considered an abnormal appearance, and the drug quality inspection and review process is immediately triggered. The pharmacist is notified to conduct a manual re-inspection, and the clinical dispensing of this batch of drugs is suspended until its safety and compliance are confirmed.

[0127] By constructing a high-fidelity digital twin of pharmaceuticals and utilizing standard light sources and instrumented measurements, the accuracy and repeatability of archival data are ensured. The fusion of multimodal data enhances the ability to identify subtly deteriorated or counterfeit drugs. Automated batch consistency verification significantly improves pharmacy warehousing efficiency, proactively intercepts potential medication risks, and provides source assurance for safe clinical medication use.

[0128] Example 5

[0129] This embodiment is an improvement upon embodiment 4. For details, please refer to [link / reference]. Figures 1-6 Physiological signals include heart rate, respiratory rate, blood oxygen saturation, and non-invasive blood pressure;

[0130] Among them, heart rate and blood oxygen saturation are based on dual-wavelength photoplethysmography pulse wave signals. After eliminating motion artifacts through adaptive filtering, instantaneous heart rate is calculated using a peak detection algorithm, and blood oxygen saturation is calculated using the ratio method.

[0131] Respiratory rate is extracted by baseline drift period of PPG signal or thoracic impedance change of integrated impedance respiratory sensor.

[0132] Non-invasive blood pressure is based on pulse wave conduction time, i.e. the time delay between the ECG R wave and the peripheral PPG main wave. Combined with the patient's age, height and historical calibration parameters, systolic and diastolic blood pressure are dynamically estimated through a linear regression model.

[0133] Among them, heart rate and blood oxygen saturation were obtained based on dual-wavelength photoplethysmography (PPG) signals:

[0134] Simultaneously emitting light sources of 660nm (red light) and 940nm (infrared light), the system receives transmitted or reflected light intensity signals. The raw PPG signal first undergoes an adaptive filtering algorithm (such as LMS or Kalman filtering) to suppress artifact interference caused by patient limb movement. Subsequently, a peak detection algorithm (such as based on derivative zero-crossing or template matching) is used on the filtered PPG waveform to identify the pulse cycle, thereby calculating the instantaneous heart rate. Simultaneously, based on the ratio of the AC to DC components of the red and infrared light channels, the blood oxygen saturation is calculated using the ratio method, with the formula: ;

[0135] in, and The coefficients are those determined through clinical calibration. For two wavelengths The ratio of the values ​​reflects the oxygenation status. This represents the estimated percentage of blood oxygen saturation.

[0136] Respiratory rate is extracted using two complementary methods: first, analyzing the slow periodic drift of the PPG signal baseline (i.e., baseline modulation caused by respiratory sinus arrhythmia) and extracting its dominant period using Fourier transform or autocorrelation methods; second, when the device integrates impedance respiratory sensing, directly acquiring a more stable respiratory waveform by measuring the minute impedance changes caused by the chest wall during respiration, and calculating the respiratory rate from it. Impedance signals are preferred; if unavailable, analysis reverts to PPG baseline analysis to improve robustness. Non-invasive blood pressure is indirectly estimated using pulse wave transit time (PWTT): simultaneously acquiring ECG and peripheral PPG signals, accurately detecting the time interval between the R wave peak in the ECG and the rising edge of the PPG main wave as the PWTT; this time interval is negatively correlated with arterial blood pressure. Based on this, combined with the patient's age... ,height Based on gender and historical invasive / non-invasive blood pressure calibration records, the current systolic blood pressure (SBP) and diastolic blood pressure (DBP) are dynamically estimated using a pre-trained linear regression model (or lightweight neural network). A typical model format is as follows:

[0137] ;

[0138] Among them, the regression coefficient Blood pressure is updated online based on individual historical data to improve long-term monitoring accuracy. The blood pressure is updated at least every 2 minutes to meet clinical monitoring needs.

[0139] By integrating multi-source signals and adaptive algorithms, common clinical challenges such as motion interference are effectively overcome. All signal processing is completed on the wearable device or edge gateway, ensuring data real-time performance and privacy. The acquired high-quality physiological stream provides a reliable data foundation for subsequent multimodal fusion analysis (such as early warning of drug allergies and identification of cardiac arrest precursors), significantly improving the ability to detect adverse nursing events in the early stage. Thus, when adverse drug administration occurs, human feedback can trigger timely rescue.

[0140] Example 6

[0141] A system for applying a method for preventing adverse events related to medication administration includes:

[0142] The camera module is used to collect 3D point cloud data to construct a digital model of the ward space and define public and private areas in the model.

[0143] A multimodal authentication module is used to fuse UWB 3D coordinate positioning results with lightweight authentication methods. The model's visual identity recognition results are cross-validated with the authorized list in the hospital's human resources system to determine in real time whether there are medical staff with nursing privileges located in the privacy area;

[0144] After receiving the authorization signal from the multimodal authentication module, the camera device enables image capture of the privacy area;

[0145] Semantic segmentation unit based on The model performs pixel-level occlusion on the video stream and applies Gaussian blur to the patient's face and torso even when monitoring is enabled, leaving only the areas related to nursing procedures clear.

[0146] A multi-parameter physiological signal acquisition module is used to continuously output non-invasive blood pressure data such as heart rate, respiratory rate, blood oxygen saturation, and pulse wave conduction time estimation.

[0147] The multimodal fusion analysis module includes:

[0148] The visual event detection submodule is used to extract timestamps of drug administration operations and facial flushing events through pose estimation, facial expression recognition, and target detection algorithms.

[0149] The physiological mutation detection submodule is used to identify sudden changes in heart rate, rapid decrease in SpO2, respiratory depression, and hypotension events.

[0150] The graph attention network reasoning unit constructs a heterogeneous graph with the patient's current state, historical medical history and environmental context as nodes, and outputs the risk probability of three types of adverse events: falls, drug allergies and cardiac arrest precursors.

[0151] The drug verification module automatically captures images, identifies drug text information, segments the liquid surface area, calculates the fill rate and ΔE color difference in the CIELAB color space, and compares it with the pre-stored digital twin of the drug. If the deviation exceeds the limit, a secondary verification alarm is triggered.

[0152] When the risk probability output by the multimodal fusion analysis engine exceeds the threshold, the structured adverse event knowledge base is invoked to generate an emergency response instruction that includes the target medical staff location, optimal route planning, AR navigation guidance, control instructions for LED indicator lights along the way, and a list of emergency supplies to be retrieved, and then pushed to the medical staff via Bluetooth.

[0153] This specification also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform the multiple steps described in the above embodiments. If the constituent modules of the above-described electronic device are implemented as software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium.

[0154] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0155] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method of preventing an adverse event of a care administration, characterized by, Includes the following steps: A spatial digital model was constructed in the ward based on 3D point cloud, and public and private areas were divided accordingly. Multimodal authentication is used to determine in real time whether authorized medical personnel are located within the privacy area; When it is confirmed that at least one medical staff member is located within the privacy area, the camera equipment is activated to capture images of the privacy area and obtain image data. Acquire continuous physiological signals from the patient; The image data and the physiological signals are subjected to multimodal fusion analysis. A spatiotemporal alignment mechanism is used to associate visual events with physiological mutations, and a graph attention network is used to determine whether there is a risk of adverse events. If the risk score exceeds a preset threshold, the structured adverse event knowledge base is invoked, and combined with patient information, an emergency response instruction is generated that includes personnel scheduling, medication dispensing, and operational guidelines.

2. The method of preventing adverse events from the administration of care according to claim 1, characterized in that, The method for multimodal fusion analysis includes: Perform intelligent verification of drug appearance on the image data; OCR technology is used to identify the name, batch number, and expiration date of medications in the ward. The liquid region of the drug is extracted by image segmentation, and its color value, transparency and fill rate are calculated; The identification results are compared with the corresponding parameters in the pharmacy's standard drug digital archives in multiple dimensions. If any parameter deviation exceeds the preset threshold, a drug verification alarm will be triggered, requiring the nurse to perform a manual review. The manual verification process includes checking the odor, shaking to observe the sedimentation state, and scanning the original outbound code.

3. The method of preventing adverse events from the administration of care according to claim 2, characterized in that: The emergency response instructions include locating the nearest qualified doctor or nurse from the scheduling information based on the type of adverse event; The system calculates the optimal route using Bluetooth beacons and indoor positioning, and sends directional voice alerts and navigation guidance to doctors and nurses. The system simultaneously controls the illumination of indicator lights along the ward route and sends location information for the retrieval of necessary medications and emergency equipment to nurses.

4. The method of preventing adverse events from the administration of care according to claim 3, characterized in that, Also includes: Once it is confirmed that medical staff have entered the privacy area, a countdown will begin. During the countdown, a pose estimation algorithm is continuously used to detect whether nursing-related actions occur; If no nursing-related actions are detected by the end of the countdown, the capture of images in the privacy area will be automatically turned off, the event log will be recorded as a suspected accidental entry, and an audit alert will be sent to the head nurse's terminal.

5. The method of preventing adverse events from the administration of care according to claim 1, wherein, This also includes establishing digital files for drugs and placing them in a standard light source box when they first enter the warehouse; Capture images from multiple angles and simultaneously acquire near-infrared spectral data; The standard Lab value of liquid medicines was determined by a Lab color analyzer, and typical degradation characteristics were manually labeled, including turbidity, precipitation, stratification, and color deepening. The above near-infrared spectral data and feature data, together with drug ID, dosage form, packaging material, and storage temperature and humidity requirements, are structured and stored to form a digital twin of the drug; When a new batch of drugs enters the warehouse, its appearance parameters are automatically compared with those of historical batches. If the deviation exceeds the limit, a drug quality inspection review is triggered.

6. The method of preventing adverse events from the administration of care according to claim 2, wherein: When a nurse is detected entering the bedside area with medication in hand, an image of the entire medication is captured, and the medication information on the label is identified using the PaddleOCR model. The U-Net++ segmentation network was used to extract the bottle body region and liquid surface contour from the image, and the fill rate was calculated. ； In the CIELAB color space, the average Lab value of the liquid surface area is calculated, and the ΔE color difference is calculated by comparing it with the standard color vector stored in the pharmacy's standard drug digital archive: ； wherein, , and are the color values of the standard drug, , and are the measured color values of the ward-site drug; An anomaly is identified and a level 2 verification alarm is triggered when at least one of the following conditions occurs: The OCR detected a drug name that did not match the doctor's prescription. The fill rate is lower than the standard outbound range; a color difference greater than a set threshold; An image gradient entropy exceeding the turbidity threshold indicates that the liquid is opaque. When a Level 2 verification alarm is triggered, the medication administration process is forcibly suspended, and a review guide is displayed on the nurse's terminal to re-examine the medicine bottle.

7. The method of preventing adverse events from the administration of care according to claim 6, characterized in that: The physiological signals include heart rate, respiratory rate, blood oxygen saturation, and non-invasive blood pressure; The heart rate and blood oxygen saturation are based on dual-wavelength photoplethysmography pulse wave signals. After eliminating motion artifacts through adaptive filtering, the instantaneous heart rate is calculated using a peak detection algorithm, and the blood oxygen saturation is calculated using a ratio method. The respiratory rate is extracted by the baseline drift period of the PPG signal or the change in thoracic impedance of the integrated impedance respiratory sensor. The non-invasive blood pressure measurement is based on pulse wave conduction time, i.e. the time delay between the ECG R wave and the peripheral PPG main wave. Combined with the patient's age, height, and historical calibration parameters, the systolic and diastolic blood pressure are dynamically estimated through a linear regression model.

8. The system for preventing adverse events from a care administration according to any one of claims 1 to 7, characterized in that, include: The camera module is used to collect 3D point cloud data to construct a digital model of the ward space, and to define public areas and private areas in the model; A multi-modal identity verification module is used to fuse the UWB three-dimensional coordinate positioning result and the visual identity recognition result based on a lightweight Model, and cross-verify with the authorized list in the hospital human resource system to determine in real time whether a medical staff with nursing authority is located in the privacy area. After receiving the authorization signal from the multimodal authentication module, the camera device enables image acquisition of the privacy area; The semantic segmentation unit is based on The model performs pixel-level masking on the video stream, still applying Gaussian blur to the patient's face and torso area in the monitoring enabled state, only keeping the area related to nursing operations clear. A multi-parameter physiological signal acquisition module is used to continuously output non-invasive blood pressure data such as heart rate, respiratory rate, blood oxygen saturation, and pulse wave conduction time estimation. The multimodal fusion analysis module includes: The visual event detection submodule is used to extract timestamps of drug administration operations and facial flushing events through pose estimation, facial expression recognition, and target detection algorithms. a physiological mutation detection sub-module for identifying heart rate crashes, rapid drops, respiratory depression, and hypotension events; The graph attention network reasoning unit constructs a heterogeneous graph with the patient's current state, historical medical history and environmental context as nodes, and outputs the risk probability of three types of adverse events: falls, drug allergies and cardiac arrest precursors. The drug verification module automatically captures images, identifies drug text information, segments the liquid surface area, and calculates the fill rate and its relationship to the CIELAB color space when it detects a drug entering the bedside area. Color difference is detected and compared with the pre-stored digital twin of the drug; if the deviation exceeds the limit, a level 2 verification alarm is triggered. When the risk probability output by the multimodal fusion analysis engine exceeds the threshold, the structured adverse event knowledge base is invoked to generate an emergency response instruction that includes the target medical staff location, optimal route planning, AR navigation guidance, control instructions for LED indicator lights along the way, and a list of emergency supplies to be retrieved, and then pushed to the medical staff via Bluetooth.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

10. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

Images