Input and output monitoring and early warning system and method based on multi-modal data fusion

Abstract

The invention discloses an input and output monitoring and early warning system and method based on multi-modal data fusion, and the system comprises an input and output data collection module which is used for collecting input data and output data of a patient; the local processing module is used for storing the collected input data and output data in local equipment of the patient and preprocessing the input data and the output data to obtain categories, water content and confidence of the input and output of the patient; the data processing module is used for carrying out secondary processing on the data of which the confidence is lower than a preset confidence threshold to obtain water content correction results of the input and output of the patient, and generating a patient input and output report; and the early warning module is used for obtaining the patient input and output report and sending out early warning to the patient and medical care. According to the invention, the comprehensiveness, accuracy and patient compliance of data acquisition can be improved, so that accurate input and output prediction and personalized analysis are realized, a more scientific basis is provided for clinical treatment, and the treatment effect is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical health information, in particular to an outflow and inflow monitoring and early warning system and method based on multi-modal data fusion. BACKGROUND

[0002] The liquid outflow and inflow refers to the total amount of liquid taken in and discharged out of the body by various routes within 24 hours, and its monitoring can reflect the body fluid metabolism balance state of the patient. The liquid outflow and inflow statistics is a very basic but extremely important nursing link, which is directly related to the life safety and rehabilitation process of the patient. Its core significance lies in maintaining the balance of water, electrolyte and nutrition in the patient's body, and providing key basis for the doctor's diagnosis and treatment. Through accurate recording of the inflow (including drinking water, food water content, infusion volume, etc.) and outflow (including urine volume, vomit, drainage volume, etc.), the clinical can assist in diagnosing changes in severe diseases such as shock, renal failure and heart failure.

[0003] With the development of medical informatization, the traditional paper record or simple electronic form record method is also gradually developing towards a more efficient automatic acquisition and analysis system, such as an electronic medical record system cooperating with intelligent sensors or an online outflow and inflow statistics system. However, the above-mentioned method of recording the patient's outflow and inflow by using electronic medical records and intelligent sensors can only be implemented in the hospital, and has not been extended to the post-discharge stage, the diagnosis and treatment closed loop has not been formed, the cooperation between doctors, nurses and patients is insufficient, and the system covers a small number of people. The online outflow and inflow statistics system also needs the cooperation of patients and medical staff, and has strong network dependence, which leads to the unavailability of offline mode when the network is disconnected. SUMMARY

[0004] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present application is to provide an outflow and inflow monitoring and early warning system and method based on multi-modal data fusion, which can improve the comprehensiveness, accuracy and patient compliance of data acquisition, and the multi-modal data cooperative acquisition can more comprehensively reflect the patient's physical condition, and the non-invasive acquisition method reduces the patient's discomfort, thereby realizing accurate outflow and inflow prediction and personalized analysis, providing a more scientific basis for clinical treatment, and helping to improve the treatment effect.

[0005] One of the technical solutions adopted by the present application is to provide an outflow and inflow monitoring and early warning system based on multi-modal data fusion, which comprises: An outflow and inflow data acquisition module for acquiring the inflow data and outflow data of the patient; A local processing module for storing the acquired inflow data and outflow data in the local device of the patient, and using the first data processing model deployed in the local device to preprocess the inflow data and outflow data to obtain the category, water content and confidence of the patient's inflow and outflow; The data processing module is configured to obtain a pre-processing result of a first data processing model, perform secondary processing on data with a confidence level lower than a preset confidence threshold by using a second data processing model, obtain a water content correction result of patient intake and output, and generate a patient intake and output report; The early warning module is configured to obtain the patient intake and output report, and issue an early warning to the patient and medical staff according to a preset early warning rule.

[0006] Further, the intake and output data acquisition module comprises the following sub-modules: The intake acquisition sub-module is configured to acquire image and weight and / or voice data of patient intake by using an intake data acquisition device; The explicit output acquisition sub-module is configured to acquire image and weight and / or voice data of patient explicit output by using an output data acquisition device; The implicit output acquisition sub-module is configured to acquire an implicit output detection index provided by the patient or medical staff.

[0007] Further, the explicit output is an output that can be directly measured; the implicit output is an output that cannot be directly measured, and each type of the implicit output corresponds to an implicit output detection index that can affect the implicit output data; the implicit output detection index comprises a patient physical index and an environmental index.

[0008] Further, the local processing module comprises the following sub-modules: The data storage sub-module is configured to acquire intake data and output data acquired by the intake and output data acquisition module, and store the data in a patient local device; The water content database sub-module is configured to store a food water content comparison table; The data preprocessing sub-module is configured to classify patient intake and output by using a first data processing model, and preliminarily calculate water content and corresponding confidence levels of the intake and output; The data synchronization sub-module is configured to synchronize the acquired intake data and output data, together with the categories, water content and confidence levels of the intake and output obtained by pre-processing, to the data processing module when the patient local device is in a network connection state.

[0009] Further, the first data processing model is a lightweight classification calculation model that can be deployed on a mobile device; The first data processing model can identify the types of patient intake and output according to images of patient intake and images of patient explicit output; The first data processing model can also calculate water content in patient intake and explicit output by using a water content calculation algorithm according to the weights of the intake and explicit output; The first data processing model is further capable of calculating the patient's hidden output water content according to the hidden output detection index.

[0010] Further, the data processing module comprises the following sub-modules: The confidence degree judgment sub-module is configured to compare the confidence degrees of the patient's intake and output obtained in the preprocessing with a preset confidence degree threshold, and take the intake and output original collection data corresponding to the confidence degrees less than the preset confidence degree threshold as secondary processing data. The data processing sub-module is configured to extract the secondary processing data and correct the categories and water contents of the secondary processing data by using a second data processing model. The report generation sub-module is configured to generate a structured intake and output report according to the data processing results of the intake and output of each patient.

[0011] Further, the second data processing model is a neural network model deployed in the cloud; and the accuracy of the second data processing model in identifying the categories and calculating the water contents of the patient's intake and output is higher than that of the first data processing model.

[0012] Further, the early warning module comprises the following sub-modules: The patient classification sub-module is configured to classify patients into newborns, infants, toddlers, preschoolers, school-age children, teenagers and adults according to patient medical record data.

[0013] The hierarchical early warning sub-module is configured to compare the intake and output report of the patient with a preset early warning rule according to the classification to which the patient belongs, and if the early warning rule is triggered, the patient and medical staff are warned, and / or basic nursing intervention measures are generated and sent to the patient and medical staff; and the preset early warning rule can be adjusted according to actual needs.

[0014] Further, the preset early warning rule includes that, for adult patients, when liquid positive balance > 20ml / kg / 24h, and / or urine volume < 0.5ml / kg / h for 6h, and / or central venous pressure > 12mmHg, and / or weight gain > 2% / 24h, and / or blood sodium change > 5mmol / L / 24h, the alert rule is triggered, and the patient and medical staff are alerted; when liquid positive balance > 40ml / kg / 24h, and / or urine volume < 0.3ml / kg / h for 6h, and / or central venous pressure > 15mmHg, and / or weight gain > 5% / 24h, and / or blood sodium change > 10mmol / L / 24h, the dangerous rule is triggered, and basic nursing intervention measures are generated and sent to the patient and medical staff; for newborns, the alert rule is triggered when urine volume < 1ml / kg / h and / or liquid positive balance > 10ml / kg / 24h; for infants, the alert rule is triggered when urine volume < 1ml / kg / h and / or liquid positive balance > 15ml / kg / 24h; for young children, the alert rule is triggered when urine volume < 0.8ml / kg / h and / or liquid positive balance > 20ml / kg / 24h; for preschoolers, the alert rule is triggered when urine volume < 0.7ml / kg / h and / or liquid positive balance > 25ml / kg / 24h; for school-age children, the alert rule is triggered when urine volume < 0.6ml / kg / h and / or liquid positive balance > 30ml / kg / 24h; for adolescents, the alert rule is triggered when urine volume < 0.5ml / kg / h and / or liquid positive balance > 35ml / kg / 24h, and the patient and medical staff are alerted.

[0015] Another technical solution adopted by the present application is a dynamic monitoring and early warning method for intake and output amount based on multi-modal data fusion, which comprises the following steps: S1: collecting intake and output data of a patient; S2: storing the collected intake and output data in a local device of the patient, and using a first data processing model deployed in the local device to pre-process the intake and output data to obtain the categories, water content and confidence of the patient's intake and output; S3: obtaining the pre-processing result of the first data processing model, using a second data processing model to perform secondary processing on data with a confidence lower than a preset confidence threshold, obtaining a water content correction result of the patient's intake and output, and generating a patient intake and output report; S4: obtaining the patient intake and output report, and issuing an early warning to the patient and medical staff according to a preset early warning rule.

[0016] The present invention provides a multimodal data fusion-based intake and output monitoring and early warning system and method, which has at least the following beneficial effects: 1. When collecting patient output data, the present invention directly collects image, weight and / or voice data for visible output that can be directly collected, while for latent water loss that cannot be directly detected, it collects external indicators that can affect latent water loss and are easy to detect, and uses these external indicators to calculate latent water loss through an algorithm model, thereby improving patient cooperation; 2. By using a local processing module and a cloud-based data processing module in cooperation, the first processing module, as a distributed processing unit of the cloud-based data processing module, can preprocess the collected intake and output data offline, and then synchronize them to the cloud-based data processing module after the network is accessible, thereby not only improving the efficiency of data processing and reducing the computational load of the cloud-based data processing module, but also extending the statistics of intake and output to outside the hospital. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a structural block diagram of an embodiment of an inflow / outflow monitoring and early warning system based on multimodal data fusion according to the present invention.

[0018] Figure 2 This is a structural block diagram of the input / output data acquisition module of the present invention.

[0019] Figure 3 This is a structural block diagram of the local processing module of the present invention.

[0020] Figure 4 This is a structural block diagram of the data processing module of the present invention.

[0021] Figure 5 This is a structural block diagram of the early warning module of the present invention.

[0022] Figure 6 This is a flowchart of one embodiment of a dynamic monitoring and early warning method for inflow and outflow based on multimodal data fusion according to the present invention. Detailed Implementation

[0023] The invention will now be further described with reference to the accompanying drawings.

[0024] Please see Figure 1 This is a structural block diagram of an embodiment of an inflow / outflow monitoring and early warning system based on multimodal data fusion according to the present invention. The system includes: The intake and output data acquisition module 100 is used to collect the patient's intake and output data.

[0025] Referring to Figure 2 , the structural block diagram of the intake and output data acquisition module 100, in some embodiments, the intake and output data acquisition module 100 comprises: The intake data acquisition submodule 110 is used to acquire the image and weight of the food taken by the patient and / or voice data by using the intake data acquisition device; since the intake such as food can be directly observed and the water content thereof can be converted, the food taken by the patient can be classified by using the image of the food, and the water content of the intake food can be converted by using the weight of the food and cooperating with the food water content table, or the patient can directly input the type and weight of the intake food by voice. The weight of the food described above can be monitored in real time by the intake data acquisition device such as a smart weighing container, and the image of the food can be obtained by directly taking a picture by the patient mobile terminal. Therefore, it is very convenient to use the intake data acquisition module 100 to acquire the intake data, and the patient only needs to take a picture or input by voice, which can greatly improve the cooperation degree of the patient for the intake data acquisition, and also provides a more accurate data basis for the out-of-hospital detection of the intake and output data.

[0026] The explicit output data acquisition submodule 120 is used to acquire the image and weight of the explicit output of the patient and / or voice data by using the output data acquisition device.

[0027] The implicit output data acquisition submodule 130 is used to acquire the implicit output detection index provided by the patient or medical staff. The output data and the intake data are in the clinical intake and output statistics, and the output often contains excretion, vomiting, drainage, wound exudation, respiratory evaporation and sweat and other different forms and different degrees of data, and these data except excretion and vomit are difficult to be directly detected outside the hospital, so the output data of the patient is classified into the explicit output such as excretion and vomiting which can be directly detected in the embodiment, and the same way as described above for the intake data acquisition is adopted, such as using a smart toilet and a vomit bag to collect the explicit output, and for the implicit output, the external index such as the body index and the environment index of the patient can be used for indirect conversion, which avoids the patient from performing complicated operations such as sweat weighing.

[0028] As the classification of the above-mentioned outflow data, the explicit outflow is an outflow that can be directly measured; the implicit outflow is an outflow that cannot be directly measured, and specifically, the explicit outflow can mainly include: urine discharge, digestive tract excretion (defecation, vomiting), skin explicit sweating, medical operation drainage, bleeding, etc.; the implicit outflow mainly includes: skin implicit evaporation, respiratory tract evaporation, special physiological and pathological states, for example: burn wound evaporation, increased transdermal water loss in large-area dermatitis / eczema, increased basal consumption in hypermetabolic state, etc., each type of implicit outflow corresponds to an implicit outflow detection index that can affect the implicit outflow data; the implicit outflow detection index includes patient physical indicators and environmental indicators. Taking sweat as an example, for the detection of sweat in the outflow data, the conventional method needs to detect the weight of the sweat-absorbing pad to convert the weight of the sweat, and in the case of out-of-hospital, it is difficult for the patient to completely weigh the sweat, so collecting external indicators of the patient, such as implementing core temperature, environmental temperature and humidity, heart rate, etc. The indicators can be monitored without the patient's awareness, which can avoid the patient from cheating and reporting false data due to the tediousness of the conventional operation, thereby improving the data collection efficiency and to a certain extent, improving the accuracy of the original data.

[0029] The local processing module 200 is configured to store the collected inflow data and outflow data in the patient local device, and utilize the first data processing model deployed in the local device to pre-process the inflow data and outflow data, to obtain the category, water content and confidence of the patient inflow and outflow. The collected inflow data and outflow data exist in the form of multiple modalities such as images, texts and voices, and the local processing module 200 can utilize the first data processing model deployed in the patient local device such as a mobile phone to pre-process the inflow data and outflow data, to obtain the category, water content and confidence of the patient inflow and outflow.

[0030] Please refer to Figure 3 That is, the structure block diagram of the local processing module 200 of the present application, in some embodiments, the local processing module 200 of the present application comprises: The data storage submodule 210 is configured to obtain the inflow data and outflow data collected by the inflow and outflow data collection module, and store them in the patient local device; The water content database submodule 220 is used to store a food water content comparison table. The water content database submodule 220 stores a variety of food water content comparison tables, which are used to compare the intake data and the food water content comparison table called from the water content database submodule 220 after the patient's intake data is collected, so as to calculate the patient's intake after the food intake. In addition, the general food water content comparison table only contains common food, and some local food is not added to the food water content comparison table, therefore, the medical staff can gradually improve the water content comparison table by measuring the water content of new food and manually filling new data into the water content database, so that the water content comparison table can correspond to more food.

[0031] The data preprocessing submodule 230 is used to classify the patient's intake and output by using a first data processing model, and preliminarily calculate the water content of the intake and output and the corresponding confidence. The data preprocessing submodule 230.

[0032] The data synchronization submodule 240 is used to synchronize the collected intake data and output data, as well as the categories, water content and confidence of the preprocessed intake and output, to the data processing module when the patient's local device is in a network connection state. When the patient's local device is in an offline state without network connection, it can still collect the patient's intake data and output data, and when the patient's local device is in a network connection state, the collected intake data and output data and the preprocessed results can be synchronized to the cloud through the data synchronization submodule 240. It should be noted that the synchronization of the data in the present scheme can also be set to a time period, such as every four hours to attempt synchronization, and if the network is not connected, the synchronization will be attempted again after half an hour.

[0033] It should be noted that the first data processing model described above is a lightweight model suitable for patient local devices (such as mobile phones). The first data processing model is a lightweight classification calculation model that can be deployed on a mobile device, such as an efficientnet model.

[0034] The first data processing model can identify the types of patient intake and output according to the images of patient intake and patient dominant output. The mobile efficientnet model described above can complete the image recognition and classification task.

[0035] The first data processing model can also calculate the water content in the intake and the explicit output of the patient according to the weight of the intake and the explicit output and using a water content calculation algorithm; after obtaining the intake food type and food water content comparison table of the patient, the weight of the food can be converted into water content, and the explicit output can also be calculated in a conversion manner.

[0036] The first data processing model can also calculate the water content in the implicit output of the patient according to the implicit output detection index. The calculation of the implicit output is different from the calculation of the intake and the explicit output, and the calculation depends on the relationship between the collected implicit output detection index and the implicit water loss, and this relationship can be fitted according to the historical data of the patient, so as to obtain a fitting model of the implicit output with respect to the implicit output detection index. According to the fitting model, the easily detected implicit output detection index can be input, and the implicit output of the patient can be output.

[0037] The data processing module 300 is used for obtaining the preprocessing result of the first data processing model, performing secondary processing on the data with a confidence lower than a preset confidence threshold by using a second data processing model, obtaining the water content correction result of the intake and the output of the patient, and generating the intake and output report of the patient. The data processing module 300 is a cloud data processing subject. After the distributed local processing module completes the preprocessing of the collected intake data and output data and filters out most of the “simple, clear and no need to be confirmed again” classification data, only the data with a confidence lower than a preset confidence threshold needs to be processed again.

[0038] The second data processing model can be a neural network model deployed in the cloud, such as a CNN neural network model. The second data processing model has higher accuracy in the category identification and water content calculation of the intake and the output of the patient than the first data processing model.

[0039] Please refer to Figure 4 That is, the structure block diagram of the data processing module 300 of the application, the data processing module 300 comprises: The confidence judgment submodule 310 is used for comparing the confidence of the intake and the output of the patient obtained in the preprocessing with a preset confidence threshold, and taking the original collection data of the intake and the output corresponding to the confidence lower than the preset confidence threshold as the secondary processing data. The above-mentioned confidence is actually equivalent to the accuracy judgment of the processing result of the first data processing model after preprocessing the intake data and the output data. The confidence judgment submodule 310 can compare the confidence of the preprocessing result with a preset confidence threshold, so as to screen the intake data and the output data with low preprocessing accuracy, so as to perform secondary processing on these data in the second data processing model.

[0040] The data processing submodule 320 is configured to extract the secondary processing data and correct the category and water content of the secondary processing data by using a second data processing model.

[0041] The report generation submodule 330 is configured to generate a structured intake and output report according to the data processing results of the intake and output of each patient. After obtaining the final data processing results of the intake and output, the report generation submodule 330 can generate a structured intake and output report according to the patient label and other associated data of the patient, such as medical record data, and the intake and output results of the patient.

[0042] The early warning module 400 is configured to obtain the intake and output report of the patient and issue an early warning to the patient and medical staff according to a preset early warning rule.

[0043] Please refer to Figure 5 , namely the structural block diagram of the early warning module 400 of the present application, the early warning module 400 comprises: The patient classification submodule 410 is configured to classify the patient into a chronic stage or an acute stage or a heart failure patient according to the medical record data of the patient. Since different types of patients have different tolerances to changes in intake and output, in order to distinguish in the early warning, the patient classification submodule 410 can classify the heart failure patients into the chronic stage, the acute stage, wherein the chronic stage is generally more stable, and has a higher tolerance to changes in intake and output, and the acute stage and the heart failure patient are next. Or classified by age into newborn, infant, toddler, preschool, school age, teenager and adult.

[0044] The hierarchical early warning submodule 420 is configured to compare the intake and output report of the patient with a preset early warning rule according to the classification to which the patient belongs, and if the early warning rule is triggered, an alert is issued to the patient and medical staff, and / or basic nursing intervention measures are generated and sent to the patient and medical staff. The preset early warning rule can be adjusted according to actual needs, for example, the early warning rule can be replaced by some data as new evidence appears, and the updated data can be replaced by system maintenance personnel.

[0045] The pre-warning rules can include, for an adult patient, the adult fluid balance management pre-warning table shown in Table 1 below: when the fluid balance is >20ml / kg / 24h, and / or the urine output is <0.5ml / kg / h for 6h, and / or the central venous pressure is >12mmHg, and / or the body weight increases by >2% / 24h, and / or the blood sodium changes by >5mmol / L / 24h, the warning rule is triggered, and an alert is sent to the patient and the medical staff; when the fluid balance is >40ml / kg / 24h, and / or the urine output is <0.3ml / kg / h for 6h, and / or the central venous pressure is >15mmHg, and / or the body weight increases by >5% / 24h, and / or the blood sodium changes by >10mmol / L / 24h, the danger rule is triggered, and basic nursing interventions are generated and sent to the patient and the medical staff. Table 1. Adult fluid balance management pre-warning table

[0046] The pre-warning rules can include, for an adult patient, the adult fluid balance management pre-warning table shown in Table 1 below: when the fluid balance is >20ml / kg / 24h, and / or the urine output is <0.5ml / kg / h for 6h, and / or the central venous pressure is >12mmHg, and / or the body weight increases by >2% / 24h, and / or the blood sodium changes by >5mmol / L / 24h, the warning rule is triggered, and an alert is sent to the patient and the medical staff; when the fluid balance is >40ml / kg / 24h, and / or the urine output is <0.3ml / kg / h for 6h, and / or the central venous pressure is >15mmHg, and / or the body weight increases by >5% / 24h, and / or the blood sodium changes by >10mmol / L / 24h, the danger rule is triggered, and basic nursing interventions are generated and sent to the patient and the medical staff.

[0047] Table 2. Child fluid balance management pre-warning table

[0048] In addition, for heart failure patients among them, the early warning core of the acute phase is to evaluate the early response of diuretics, and the early warning core of the stable phase is to monitor the long-term self-monitoring through body weight changes, and the monitoring frequency can be according to the acute phase: hourly urine volume, electrolyte every 6 hours, daily body weight monitoring. The hourly urine volume reaches the early warning value, the 6-hour electrolyte is abnormal, and the daily weight gain is greater than 1kg, which triggers the early warning rule, wherein the adult urine volume is calculated after using a sufficient loop diuretic, and the continuously monitored hourly urine volume fails to reach 100mL / hour, which can also trigger the early warning rule. And the stable phase can be monitored according to the rules of: body weight twice a week, electrolyte every 2 weeks, and the early warning rule is triggered when the electrolyte is abnormal, or the body weight increases by more than 2.0kg within 3 days, or the body weight increases by more than 1.0-1.5kg within 24 hours (1 day).

[0049] In addition, after obtaining the in-out quantity report of the user, the in-out quantity report can be fed back to the nutritionist, so that the diet plan can be adjusted.

[0050] The in-out quantity monitoring and early warning system and method based on multi-modal data fusion provided by the application can improve the comprehensiveness, accuracy and patient compliance of data acquisition, and the multi-modal data cooperative acquisition can more comprehensively reflect the patient's physical condition. The non-invasive acquisition method reduces the discomfort of the patient. At the same time, the algorithm is dynamically adjusted: the early warning threshold can be automatically adjusted according to the real-time state of the patient, the predictive analysis: based on the early data change, the liquid state in the future 12-24 hours is predicted, the multi-specialty integration: the standards of cardiology, nephrology, critical care medicine and other multidisciplinary standards are integrated, the man-machine interaction optimization: the clinician can conveniently adjust the parameter weight, and finally the precise in-out quantity prediction and personalized analysis can be realized, which provides a more scientific basis for clinical treatment, and helps to improve the treatment effect. In addition, a complete diagnosis and treatment closed loop can be formed, the cooperation of doctors, nurses and patients is strengthened, the patient management efficiency and quality are improved, and the diversified clinical needs are met.

[0051] On the other hand, referring to Figure 6 The application further provides a dynamic in-out quantity monitoring and early warning method based on multi-modal data fusion, and the implementation of the method is based on the above-mentioned in-out quantity monitoring and early warning system based on multi-modal data fusion. The method comprises the following steps: S1: collecting in-out quantity data of a patient; S2: storing the collected in-out quantity data in a local device of the patient, and using a first data processing model deployed in the local device to pre-process the in-out quantity data, to obtain the category, water content and confidence of the in-out quantity of the patient; S3: Obtain the pre-processing result of the first data processing model, use the second data processing model to perform secondary processing on the data with a confidence lower than a preset confidence threshold, obtain a water content correction result of the patient's intake and output, and generate a patient intake and output report; S4: Obtain the patient intake and output report, and issue a warning to the patient and medical staff according to a preset warning rule.

[0052] The above only expresses the preferred embodiments of the present application, which are described in detail, but cannot be understood as a limitation on the scope of the patent. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the protection scope of the present application patent should be subject to the appended claims.

Claims

1. A system for monitoring and early warning of intake and output based on multi-modal data fusion, comprising: an intake and output data acquisition module for acquiring intake and output data of a patient; a local processing module for storing the acquired intake and output data in a local device of the patient, and pre-processing the intake and output data using a first data processing model deployed in the local device to obtain the category, water content and confidence of the intake and output of the patient; a data processing module for obtaining the pre-processing result of the first data processing model, performing secondary processing on data with a confidence lower than a pre-set confidence threshold using a second data processing model, obtaining a corrected result of the water content of the intake and output of the patient, and generating an intake and output report of the patient; an early warning module for obtaining the intake and output report of the patient, and issuing an early warning to the patient and medical staff according to a pre-set early warning rule.

2. The multi-modal data fusion based monitoring and early warning system for entry and exit volume according to claim 1, wherein, The intake and output data acquisition module comprises the following sub-modules: an intake acquisition sub-module for acquiring images and weight and / or voice data of the patient's intake using an intake data acquisition device; an explicit output acquisition sub-module for acquiring images and weight and / or voice data of the patient's explicit output using an output data acquisition device; an implicit output acquisition sub-module for acquiring implicit output detection indicators provided by the patient or medical staff. 3.The multi-modal data fusion based monitoring and early warning system for the number of entries and exits according to claim 2, wherein, The explicit output is an output that can be directly measured; the implicit output is an output that cannot be directly measured, and each type of implicit output corresponds to an implicit output detection indicator that can affect the implicit output data; the implicit output detection indicator includes patient physical indicators and environmental indicators.

4. The multi-modal data fusion based monitoring and early warning system for entry and exit volume according to claim 3, wherein, The local processing module comprises the following sub-modules: a data storage sub-module for obtaining the intake and output data acquired by the intake and output data acquisition module and storing them in the local device of the patient; a water content database sub-module for storing a food water content comparison table; a data preprocessing sub-module for classifying the intake and output of the patient using the first data processing model, and preliminarily calculating the water content and corresponding confidence of the intake and output; a data synchronization sub-module for synchronizing the acquired intake and output data, as well as the categories, water contents and confidences of the intake and output obtained by preprocessing, to the data processing module when the local device of the patient is in a network connection state.

5. The multi-modal data fusion based monitoring and early warning system for monitoring the access amount according to claim 4, wherein, The first data processing model is a lightweight classification calculation model that can be deployed on a mobile device; The first data processing model can identify the types of intake and output of the patient according to the images of the patient's intake and the images of the patient's explicit output; The first data processing model can also calculate the water content in the intake and explicit output of the patient using a water content calculation algorithm according to the weight of the intake and explicit output; The first data processing model can also calculate the water content of the patient's implicit output according to the implicit output detection indicators. 6.The multi-modal data fusion based monitoring and early warning system for the quantity of people entering and leaving according to claim 1, wherein, The data processing module comprises the following sub-modules: a confidence judgment sub-module for comparing the confidence of the intake and output of the patient obtained in the preprocessing with a pre-set confidence threshold, and taking the original acquisition data of the intake and output corresponding to the confidence lower than the pre-set confidence threshold as secondary processing data; The data processing submodule is configured to extract secondary processing data and correct the category and water content of the secondary processing data by using a second data processing model. The report generation submodule is configured to generate a structured intake and output report according to the data processing results of the intake and output data corresponding to each patient.

7. The multi-modal data fusion based monitoring and early warning system for monitoring the inflow and outflow according to claim 6, wherein, The second data processing model is a neural network model deployed in the cloud, and the accuracy of the category identification and water content calculation of the intake and output data of the patient by the second data processing model is higher than that of the first data processing model. 8.The multi-modal data fusion based monitoring and early warning system for access amount according to claim 1, wherein, The warning module comprises the following submodules: The patient classification submodule is configured to classify patients into newborns, infants, toddlers, preschoolers, school-age children, adolescents and adults according to patient medical record data. The hierarchical warning submodule is configured to compare the intake and output report of the patient with a preset warning rule according to the classification of the patient, and if the warning rule is triggered, the patient and medical staff are warned, and / or basic nursing intervention measures are generated and sent to the patient and medical staff; the preset warning rule can be adjusted according to actual needs.

9. The multi-modal data fusion based monitoring and early warning system for monitoring the inflow and outflow according to claim 8, wherein, The preset warning rule comprises: for adult patients, when the liquid positive balance is greater than 20ml / kg / 24h, and / or the urine output is less than 0.5ml / kg / h for 6h, and / or the central venous pressure is greater than 12mmHg, and / or the body weight increases by more than 2% / 24h, and / or the change of blood sodium is greater than 5mmol / L / 24h, the warning rule is triggered, and the patient and medical staff are warned; when the liquid positive balance is greater than 40ml / kg / 24h, and / or the urine output is less than 0.3ml / kg / h for 6h, and / or the central venous pressure is greater than 15mmHg, and / or the body weight increases by more than 5% / 24h, and / or the change of blood sodium is greater than 10mmol / L / 24h, the danger rule is triggered, and basic nursing intervention measures are generated and sent to the patient and medical staff; for newborns, the warning rule is triggered when the urine output is less than 1ml / kg / h and / or the liquid positive balance is greater than 10ml / kg / 24h; for infants, the warning rule is triggered when the urine output is less than 1ml / kg / h and / or the liquid positive balance is greater than 15ml / kg / 24h; for toddlers, the warning rule is triggered when the urine output is less than 0.8ml / kg / h and / or the liquid positive balance is greater than 20ml / kg / 24h; for preschoolers, the warning rule is triggered when the urine output is less than 0.7ml / kg / h and / or the liquid positive balance is greater than 25ml / kg / 24h; for school-age children, the warning rule is triggered when the urine output is less than 0.6ml / kg / h and / or the liquid positive balance is greater than 30ml / kg / 24h; for adolescents, the warning rule is triggered when the urine output is less than 0.5ml / kg / h and / or the liquid positive balance is greater than 35ml / kg / 24h, and the patient and medical staff are warned.

10. A dynamic monitoring and early warning method for the number of entries and exits based on multi-modal data fusion, characterized in that, The method comprises the following steps: S1: collecting intake data and output data of a patient; S2: storing the collected intake data and output data in a local device of the patient, and preprocessing the intake data and output data by using a first data processing model deployed in the local device to obtain the category, water content and confidence of the intake and output of the patient. S3: Obtain the preprocessing result of the first data processing model, use the second data processing model to perform secondary processing on the data with a confidence lower than a preset confidence threshold, obtain the water content correction result of the patient's intake and output, and generate a patient intake and output report; S4: Obtain the patient intake and output report, and issue a warning to the patient and medical staff according to a preset warning rule.

Images