Method, device and system for integrated processing of vital sign information based on artificial intelligence

Abstract

The present specification relates to the field of data processing technology, and provides a method, device and system for integrated processing of vital signs information based on artificial intelligence, the method comprising: obtaining multiple vital signs information of a target object in real time; converting the vital signs information into structured vital signs information; inputting the structured vital signs information into a pre-trained first machine learning model for analysis to obtain the vital signs change trend of the target object; and outputting auxiliary warning information according to the vital signs change trend. The embodiments of the present specification can improve the timeliness and accuracy of identifying the vital signs change trend of the target object.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device and system for integrating and processing vital sign information based on artificial intelligence. Background Art

[0002] At present, the existing integrated processing method of vital sign information is to aggregate the vital sign information output by various devices such as vital sign monitoring, support equipment, and video acquisition cameras through direct or computer room connection to achieve centralized monitoring (for example, aggregate to a central matrix all-in-one machine); or aggregate the pure data information of similar devices for centralized monitoring (such as a central monitoring workstation or infusion pump workstation, etc.); for relevant staff to view, and the relevant staff will manually determine whether to intervene or respond based on the results. However, the existing integrated processing scheme for vital sign information does not have data analysis capabilities, and it is difficult to accurately, timely, and efficiently identify the trend of changes in the vital signs of the target object by integrating multiple vital sign information. Summary of the invention

[0003] The purpose of the embodiments of this specification is to provide a method, device and system for integrated processing of vital sign information based on artificial intelligence to improve the timeliness and accuracy of identifying the trend of changes in the vital signs of a target object.

[0004] To achieve the above objectives, on the one hand, the embodiments of this specification provide a method for integrating and processing vital sign information based on artificial intelligence, including:

[0005] Obtain multiple vital signs information of the target object in real time;

[0006] converting the vital sign information into structured vital sign information;

[0007] Inputting the structured vital sign information into a pre-trained first machine learning model for analysis to obtain a vital sign change trend of the target object;

[0008] Auxiliary warning information is output according to the changing trend of the vital signs.

[0009] In the embodiment of this specification, the vital sign information includes a digitized image; and the converting the vital sign information into structured vital sign information includes:

[0010] Extracting regional image features from digitized images;

[0011] Using a preset classification algorithm to identify the region type of each of the region image features;

[0012] Determining the data type of each sub-region in each of the regional image features according to the region type;

[0013] According to the data type, each sub-region feature in each of the regional image features is converted into structured vital sign information.

[0014] In the embodiment of the present specification, the input of the first machine learning model also includes clinical constraint parameters; the trend of changes in vital signs of the target object includes the probability of abnormal vital signs.

[0015] In the embodiment of this specification, the outputting of auxiliary warning information according to the trend of the vital signs change includes:

[0016] Outputting the target object's vital sign change trend and a dialog box through a human-computer interface; the dialog box includes a first control for indicating approval and a second control for indicating disapproval;

[0017] When an operation on the first control is received, determining whether the probability of abnormal vital signs exceeds a preset probability threshold;

[0018] If the probability of the vital sign being abnormal exceeds a preset probability threshold, clinical auxiliary warning information is output to the target device.

[0019] In the embodiment of this specification, the outputting of auxiliary warning information according to the trend of the vital signs change also includes:

[0020] When an operation is received for the second control, the clinical constraint parameters are adjusted and the analysis is re-performed using the first machine learning model.

[0021] In the embodiment of this specification, the method further includes:

[0022] Acquire the monitoring images of the working scene of the target object in real time;

[0023] Converting the operation scene monitoring image into structured operation scene monitoring data;

[0024] Inputting the structured operation scene monitoring data into a pre-trained second machine learning model for analysis to obtain an operation monitoring result for the target object;

[0025] Auxiliary warning information is output according to the operation monitoring result.

[0026] On the other hand, an embodiment of the present specification further provides a vital sign information integrated processing device, including a memory, a processor, and a computer program stored in the memory, wherein when the computer program is run by the processor, the instructions of the above method are executed.

[0027] On the other hand, the embodiment of this specification also provides a vital sign information integration processing system based on artificial intelligence, including:

[0028] A variety of vital signs monitoring equipment, used to collect the target object's vital signs information in real time;

[0029] Vital sign information integrated processing equipment, which is used for:

[0030] Obtain multiple vital signs information of the target object in real time;

[0031] converting the vital sign information into structured vital sign information;

[0032] Inputting the structured vital sign information into a pre-trained first machine learning model for analysis to obtain a vital sign change trend of the target object;

[0033] Auxiliary warning information is output according to the changing trend of the vital signs.

[0034] In the embodiments of the present specification, the vital signs monitoring equipment includes a first type of equipment and a second type of equipment; the vital signs information collected by the first type of equipment is non-digital vital signs information; and the vital signs information collected by the second type of equipment is digital vital signs information.

[0035] It can be seen from the technical solutions provided in the above embodiments of this specification that the embodiments of this specification can obtain the target object's vital sign change trend by analyzing and processing the structured vital signs information using a pre-trained machine learning model, and then determine whether to output auxiliary warning information based on the vital signs change trend, that is, realize the automatic integration and automatic analysis and processing of vital signs information, thereby improving the timeliness of identifying the target object's vital sign change trend. Moreover, compared with the traditional technology, which manually determines whether to intervene or respond based on the results viewed, the embodiments of this specification analyze, process and warn the structured vital signs information based on a pre-trained machine learning model, which can avoid human errors and is more accurate and reliable, thereby also improving the accuracy of identifying the target object's vital sign change trend. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art description. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:

[0037] Figure 1A structural block diagram of a vital sign information integration processing system based on artificial intelligence in some embodiments of this specification is shown;

[0038] Figure 2 A flowchart showing a method for integrating and processing vital sign information based on artificial intelligence in some embodiments of this specification is shown;

[0039] Figure 3 A flowchart of converting digital vital sign information into structured vital sign information in some embodiments of this specification is shown;

[0040] Figure 4 A schematic diagram of a region image in an exemplary embodiment of the present specification is shown;

[0041] Figure 5 Shows Figure 4 A schematic diagram of a sub-region image in the region image shown;

[0042] Figure 6 A schematic diagram of the training process of a machine learning model in some embodiments of this specification is shown;

[0043] Figure 7 A flowchart showing a method for integrating and processing vital sign information based on artificial intelligence in other embodiments of this specification is shown;

[0044] Figure 8 A schematic diagram showing a surgical operation scene monitoring image in an exemplary embodiment of the present specification is shown;

[0045] Fig. 9 The structure block diagram of the vital sign information integration processing device in some embodiments of this specification is shown.

[0046] [Description of Reference Numerals]

[0047] 1. Category I equipment;

[0048] 2. Category II equipment;

[0049] 3. Video conversion equipment;

[0050] 4. Data exchanger;

[0051] 5. Vital signs information integration and processing equipment;

[0052] 6. Data server;

[0053] 7. Interactive terminal;

[0054] 8. Display screen;

[0055] 9. Guidance terminal;

[0056] 10. EAI data platform;

[0057] 11. Upgrade Manager;

[0058] 902. Vital signs information integration and processing equipment;

[0059] 904, processor;

[0060] 906. Memory;

[0061] 908, driving mechanism;

[0062] 910, input / output interface;

[0063] 912. Input device;

[0064] 914. Output device;

[0065] 916. Presentation equipment;

[0066] 918. Graphical user interface;

[0067] 920, network interface;

[0068] 922, communication link;

[0069] 924. Communication bus. DETAILED DESCRIPTION

[0070] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0071] In view of the fact that the existing vital signs information integration scheme does not have data analysis capabilities and it is difficult to integrate multiple vital signs information to accurately, timely and efficiently identify the vital signs change trend of the target object, the embodiments of this specification provide an improved vital signs information integration processing scheme. It should be pointed out that the vital signs information in the embodiments of this specification can be the vital signs information of the target object (such as a person or an animal) (typical vital signs information can include heart rate, pulse, blood pressure, respiration, blood oxygen, etc.). Moreover, the vital signs information integration processing scheme in the embodiments of this specification can not only integrate and process multiple vital signs information, but also integrate and process multiple non-vital signs information (such as monitoring of operation scenes such as surgery). Therefore, the embodiments of this specification can be applied to scenes such as intensive care units (ICU), specialist examination rooms (such as endoscope chambers, etc.), operating rooms, wards, and nursing homes.

[0072] Figure 1 An artificial intelligence-based vital sign information integration and processing system in some embodiments of the present specification is shown, and the system may include: multiple first-category devices 1, multiple second-category devices 2, video conversion devices 3, data exchangers 4, vital sign information integration and processing devices 5, data servers 6, interactive terminals 7, display screens 8, and guidance terminals 9, etc.

[0073] The first type of equipment 1 and the second type of equipment 2 are both vital signs monitoring equipment used for life support, monitoring, etc. In one embodiment, the vital signs monitoring equipment may include single-function vital signs monitoring equipment, such as a ventilator, an electrocardiograph, a blood pressure monitor, etc. In another embodiment, the vital signs monitoring equipment may also include multifunctional vital signs monitoring equipment, such as a vital signs monitor (an instrument with functions such as non-invasive blood pressure (NIBP), pulse rate, mean arterial pressure (MAP), blood oxygen saturation (SpO2), and temperature monitoring). Among them, the first type of equipment 1 can collect the non-digital vital signs information (i.e., analog vital signs information) of the target object in real time, but does not have a digital communication interface, and it is difficult to communicate digitally with the outside. The second type of equipment 2 can collect the digital vital signs information (i.e., digital vital signs information) of the target object in real time, and it has a digital communication interface and can communicate digitally with the outside.

[0074] For the convenience of viewing, the above-mentioned terminal devices for life support, monitoring, etc. are generally equipped with display screens; in other words, these terminal devices can output video signals, for example, the first type of device 1 can output analog video signals, and the second type of device 2 can output digital video signals. However, since the formats of the analog video signals output by different first type devices 1 may be different, it is not conducive to subsequent processing; the video conversion device 3 can be used to convert the non-digital video signals of different formats collected by the first type of device 1 into digital video signals of the same format. Obviously, the video conversion device 3 in the embodiment of this specification also has the function of data aggregation, that is, the video conversion device 3 can communicate with multiple first type devices 1 and process the video signals output by them.

[0075] Although the digital vital signs information output by different second-category devices 2 is all digital vital signs information, the data formats or data structures of these digital vital signs information may be different. In order to facilitate the analysis and processing of the pre-trained machine learning model, it is necessary to standardize and structure the digital vital signs information output by different second-category devices 2. Therefore, the data exchanger 4 can be used to standardize and structure the digital vital signs information output by each second-category device 2, and provide it to the data server 6, which writes it into the database.

[0076] The vital sign information integration and processing device 5 can be configured to: obtain multiple vital sign information of the target object from the video conversion device 3 in real time; convert the vital sign information into structured vital sign information; input the structured vital sign information into the pre-trained first machine learning model for analysis to obtain the vital sign change trend of the target object; and output auxiliary warning information according to the vital sign change trend. In this way, by using the pre-trained machine learning model to analyze and process the structured vital sign information, the vital sign change trend of the target object can be obtained, and then determine whether to output auxiliary warning information according to the vital sign change trend, that is, the automatic integration and automatic analysis and processing of the vital sign information are realized, thereby improving the timeliness of identifying the vital sign change trend of the target object. Moreover, compared with the traditional technology of manually judging whether to intervene or respond to the results viewed, the embodiment of this specification analyzes and processes the structured vital sign information and warns based on the pre-trained machine learning model, which can avoid human errors and is more accurate and reliable, thereby also improving the accuracy of identifying the vital sign change trend of the target object.

[0077] The interactive terminal 7 is mainly used to realize the interaction between devices or human-computer interaction. Depending on the application scenario, the interactive terminal 7 can be different devices. For example, in the ICU scenario, the interactive terminal 7 can be a client of relevant personnel (such as doctors or nurses), so that the vital signs information integration processing device 5 can send the integrated processing results to the relevant personnel for confirmation (only the integrated processing results recognized or confirmed by the relevant personnel can be used to assist in early warning judgment); in some embodiments, the user end can be a self-service terminal device, a mobile terminal (i.e., a smart phone), a display, a desktop computer, a tablet computer, a laptop computer, a digital assistant or a smart wearable device, etc. Among them, the smart wearable device can include a smart bracelet, a smart watch, a smart glasses or a smart helmet, etc. Of course, the user end is not limited to the above-mentioned electronic device with a certain entity, and it can also be an application running in the above-mentioned electronic device. In the surgical operation monitoring scenario, the interactive terminal 7 can be a tracking camera, through which the real-time picture of the surgical operation site can be collected and provided to the vital signs information integration processing device 5 for processing; of course, as needed, the vital signs information integration processing device 5 can also control the tracking camera to adjust the tracking range through instructions.

[0078] The display screen 8 may be a centralized display screen, which may be used to visually display the processing results of the vital sign information integrated processing device 5, and may also be used to display auxiliary warning information. And as required, the display screen 8 may also support display screen switching.

[0079] The guidance terminal 9 is configured with guidance software, which is used to generate parameterized clinical rules and provide them to the data server, which is written into the database by the data server. Parameterized clinical rules are clinical constraint parameters; when the machine learning model is trained, they are used as training inputs to realize the training of the machine learning model under the supervision of clinical rules, so as to obtain a more accurate machine learning model. In practical applications, they are used as inputs of pre-trained machine learning models to obtain more accurate integrated analysis and processing results. Among them, clinical rules are guiding clinical rules summarized based on clinical papers, clinical guidelines and / or clinical consensus. For example, in an exemplary embodiment, a clinical rule may be: the normal range of blood oxygen concentration is greater than 90%. In individual scenarios, there may be no corresponding clinical rules, so parameterized clinical rules can be given randomly, so that by training machine learning models, they can also be used for the research and discovery of clinical rules in corresponding scenarios.

[0080] Please continue to refer to Figure 1As shown, the vital sign information integration processing system based on artificial intelligence can also include an EAI (Enterprise Application Integration) data platform 10. Among them, the EAI data platform 10 can be used for various heterogeneous information systems to extract and integrate information, and provide the obtained data to the data server 6, which is written into the database by the data server 6 for model training. In one embodiment, the heterogeneous information system can include, for example, EMR (Electronic Medical Record), HIS (Hospital Information System), PACS (Picture Archiving and Communication Systems), LIS (Laboratory Information Management System), etc. The basic information (such as name, age, gender, bed, etc.) and medical records of the target object can be obtained from EMR or HIS, and the diagnostic data (such as medical images, test data, etc.) of the target object can be obtained from PACS or LIS.

[0081] Please continue to refer to Figure 1 As shown, the vital sign information integrated processing system based on artificial intelligence may further include an upgrade manager 11. The upgrade manager 11 may be used to uniformly manage online updates of multiple second-category devices 2 to reduce update costs and operation and maintenance pressure.

[0082] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0083] The embodiment of this specification also provides a method for integrating and processing vital sign information based on artificial intelligence, which can be applied to the vital sign information integration and processing device side of the above system. Figure 2 As shown, in some embodiments, the method for integrating and processing vital sign information based on artificial intelligence may include the following steps:

[0084] S201. Acquire various vital signs information of a target object in real time.

[0085] S202: Convert the vital sign information into structured vital sign information.

[0086] S203: Input the structured vital sign information into a pre-trained first machine learning model for analysis to obtain a vital sign change trend of the target object.

[0087] S204: Output auxiliary warning information according to the changing trend of the vital signs.

[0088] In the embodiments of the present specification, by using a pre-trained machine learning model to analyze and process structured vital signs information, the target object's vital signs changing trend can be obtained, and then whether to output auxiliary warning information is determined based on the vital signs changing trend, thereby realizing automatic integration and automatic analysis and processing of vital signs information, thereby improving the timeliness of identifying the target object's vital signs changing trend. Moreover, compared with traditional technologies that manually determine whether to intervene or respond based on the results viewed, the embodiments of the present specification analyze, process and warn structured vital signs information based on a pre-trained machine learning model, which can avoid human errors and is more accurate and reliable, thereby also improving the accuracy of identifying the target object's vital signs changing trend.

[0089] In some embodiments, real-time acquisition of multiple vital signs information of the target object may refer to: acquiring multiple vital signs information of the target object from the video conversion device and the data exchanger in real time. Those skilled in the art may understand that in a special case, there may be a situation where all the multiple vital signs monitoring devices are first-class devices, then real-time acquisition of multiple vital signs information of the target object may refer to: acquiring multiple vital signs information of the target object from the video conversion device in real time; in another special case, there may also be a situation where all the multiple vital signs monitoring devices are second-class devices, then real-time acquisition of multiple vital signs information of the target object may refer to: acquiring multiple vital signs information of the target object from the data exchanger in real time.

[0090] refer to Figure 3 As shown, in some embodiments, for a digitized image in a variety of vital sign information, converting the vital sign information into structured vital sign information may include the following steps:

[0091] S301, extracting regional image features from the digitized image.

[0092] The regional image may be a digitized image or a composite image formed by stitching together multiple digitized images. The size parameters of the regional image may be set as needed, and the size parameters are adjustable. In some embodiments, any suitable image feature extraction method (such as a directional gradient histogram method, etc.) may be used to extract regional image features.

[0093] S302: Identify the region type of each of the regional image features using a preset classification algorithm.

[0094] Identifying the regional type of each regional image feature is to identify what kind of image of the vital sign monitoring device each regional image feature is outputted. For example, in an exemplary embodiment, the regional type of an identified regional image feature may be a ventilator type; in another exemplary embodiment, the regional type of an identified regional image feature may be an electrocardiograph type. In some embodiments, a decision tree algorithm may be used to identify the regional type of each regional image feature. Obviously, this is only an exemplary illustration, and in other embodiments, any other suitable classification algorithm may be used as needed, and the specification does not limit this to a single one.

[0095] S303: Determine the data type of each sub-region in each of the regional image features according to the region type.

[0096] In some cases, an image output by a vital signs monitoring device may include multiple display sub-areas (referred to as sub-areas), each of which can display different parameters, which can belong to the same or different data types. Therefore, based on the pre-trained machine learning model, the sub-areas contained in each regional image can be identified, and the data type of each sub-area can be identified. In the embodiments of this specification, the data type can be a waveform type, a text type, a digital type, etc. For example, in Figure 4 The area image of the ventilator type shown includes multiple sub-areas, the main sub-area displays some waveform parameters, and other sub-areas display numeric parameters or text parameters. Figure 5 Take the sub-region shown as an example (the sub-region is Figure 4 A sub-region of the area image shown), through identification it can be confirmed that the data type of the sub-region is a digital type.

[0097] S304: Convert each sub-region feature in each of the regional image features into structured vital sign information according to the data type.

[0098] For sub-region features that are of digital or text type, the corresponding digital parameters or text parameters can be directly written into the database table to be converted into structured vital signs information; for graphic parameters such as waveforms, the graphic parameters can be first converted into corresponding strings, and then the strings can be written into the database table to be converted into structured vital signs information.

[0099] In the embodiments of the present specification, the input of the first machine learning model may also include clinical constraint parameters, and the clinical constraint parameters are also structured data; therefore, after the structured vital signs information and clinical constraint parameters are input into the pre-trained first machine learning model, the pre-trained first machine learning model can analyze the structured vital signs information under the constraints of the clinical constraint parameters, thereby obtaining a more accurate trend of changes in the vital signs of the target object.

[0100] In some embodiments, the initial first machine learning model can be any suitable machine learning model, which is not limited in this specification and can be selected according to actual needs. In some embodiments, the pre-trained first machine learning model can be based on Figure 6 The training process shown is obtained by training, wherein the data set required for training can be retrieved from the database through the data server. Figure 6 In the process shown, the video output screen of the medical device is collected at a set frequency, which may be data pre-collected and stored in a database. Figure 6 In the process shown, the purpose of comparison and judgment is mainly to confirm whether the output result of the currently trained machine learning model is consistent with the actual situation; if it is consistent with the actual situation, other images in the training data set can be used to continue training; if it is not consistent with the actual situation, the regional image parameters (such as the size of the regional image) can be adjusted, and the regional image features can be re-extracted to retrain. This is done until each image in the training data set is correctly analyzed, or the output result of the currently trained machine learning model reaches the set evaluation index value.

[0101] The trend of changes in vital signs may be a trend of changes in one or more vital signs over time, which may be specifically determined by a pre-trained first machine learning model according to input. It should be noted that the trend of changes in vital signs is correlated with the input of a variety of structured vital sign information, that is, the comprehensive input of a variety of structured vital sign information may be used to analyze one or more trends in changes in vital signs. For example, in an exemplary embodiment, when the target subject's breathing rate increases, blood pressure decreases, and heart rate increases, it can be determined that the target subject has a trend of low blood oxygen.

[0102] In some embodiments, when the target object's vital sign change trend exceeds a normal range (the normal range can be set based on clinical constraint parameters), the target object's vital sign change trend can also include a vital sign abnormality probability. In this case, outputting auxiliary warning information according to the vital sign change trend can include the following steps:

[0103] (1) Outputting the vital sign change trend of the target object and a dialog box through a human-computer interface; the dialog box includes a first control for indicating approval and a second control for indicating disapproval, for confirmation by relevant personnel.

[0104] (2) When an operation on the first control is received, determine whether the probability of the vital sign abnormality exceeds a preset probability threshold.

[0105] For example, when the relevant person clicks the first control, the operation on the first control is received, indicating that the relevant person recognizes the trend of the vital sign change (including recognizing the abnormal probability of the vital sign). At this time, it can be determined whether the abnormal probability of the vital sign exceeds the preset probability threshold.

[0106] (3) If the probability of abnormal vital signs exceeds a preset probability threshold, clinical auxiliary warning information is output to the target device. The target device may be any terminal device (such as a mobile phone, desktop computer, monitor, etc.) that allows relevant personnel to perceive the abnormal vital signs in a timely manner, so that relevant personnel can make corresponding decisions accordingly.

[0107] In an embodiment of the present specification, the output of auxiliary warning information based on the trend of changes in vital signs may also include: when an operation on the second control is received, it indicates that the relevant personnel do not recognize the trend of changes in vital signs (including not recognizing the probability of abnormal vital signs); at this time, the clinical constraint parameters can be adjusted and re-analyzed using the first machine learning model.

[0108] refer to Figure 7 As shown, in other embodiments, the method for integrating and processing vital sign information based on artificial intelligence may further include the following steps:

[0109] S701. Acquire a monitoring image of an operation scene of a target object in real time.

[0110] S702: Convert the operation scene monitoring image into structured operation scene monitoring data.

[0111] S703. Input the structured operation scene monitoring data into a pre-trained second machine learning model for analysis to obtain an operation monitoring result for the target object.

[0112] S704: Output auxiliary warning information according to the operation monitoring result.

[0113] For example, in addition to monitoring of vital signs, operations such as surgery also require operation scene monitoring. In the embodiments of this specification, by analyzing and processing the operation scene monitoring images using a pre-trained machine learning model, the operation monitoring results for the target object can be obtained, and then it is determined whether to output auxiliary warning information based on the operation monitoring results, thereby realizing automatic monitoring of the operation scene, thereby improving the timeliness of operation scene monitoring. Compared with traditional manual monitoring of operation scenes, the embodiments of this specification automatically monitor the operation scene based on a pre-trained machine learning model, and perform analysis, processing and warning, which can also avoid human errors and is more accurate and reliable. Therefore, it is more suitable for monitoring of surgical fields, panoramas and other scenes in operation processes (such as actual surgical operations and surgical simulation operations) in scenarios such as operating rooms and specialist examination rooms. Among them, the panorama may refer to the panorama of the operation process in the operating room and specialist examination room, including the target object's body movements, facial expressions, and chest rise and fall, as well as the operation scenes of relevant personnel such as surgeons (such as Figure 8 shown) etc.

[0114] In some embodiments, real-time acquisition of the operation scene monitoring image for the target object may refer to: real-time acquisition of the operation scene monitoring image for the target object from multiple tracking cameras. Therefore, under the operation scene monitoring, the vital signs information integration and processing system based on artificial intelligence may also include multiple tracking cameras, which may monitor the operation scene for the target object throughout the process autonomously or under the control of the vital signs information integration and processing device.

[0115] In some embodiments, the converting of the operation scene monitoring image into structured operation scene monitoring data may include the following steps:

[0116] (1) Extract regional image features from the operation scene monitoring image.

[0117] The work scene monitoring image is a digitized image, and the regional image can be a work scene monitoring image or a composite image formed by stitching together multiple work scene monitoring images. The regional image can set the size parameters of the regional image as needed, and the size parameters are adjustable. In some embodiments, any suitable image feature extraction method (such as the directional gradient histogram method, etc.) can be used to extract regional image features.

[0118] (2) Using a preset classification algorithm to identify the region type of each of the region image features.

[0119] Since the shooting ranges and tracking targets (or objects of interest) of different tracking cameras may be different, identifying the regional type of each regional image feature is to identify which tracking camera captured the image of each regional image feature. In some embodiments, a decision tree algorithm may be used to identify the regional type of each regional image feature. Obviously, this is only an exemplary example. In other embodiments, any other suitable classification algorithm may be used as needed, and this specification does not limit this to a single one.

[0120] (3) Determine the type of the object of interest of each sub-region in each of the regional image features according to the region type.

[0121] In some cases, an image output by a tracking camera may include multiple display sub-areas (referred to as sub-areas), each of which can display a different object of interest. Therefore, based on a pre-trained machine learning model, the sub-areas contained in each regional image can be identified, and the objects of interest in each sub-area can be identified. In the embodiments of this specification, the object of interest refers to an object that needs to be monitored (such as the surgeon's movements, surgical instruments, the surgical site of the target object, etc.).

[0122] (4) Convert each sub-region feature in each of the regional image features into structured operation scene monitoring data.

[0123] Since the object of interest is image data, the image data can be first converted into a corresponding character string, and then the character string is written into a database table, so that it can be converted into structured operation scene monitoring data.

[0124] In an embodiment of the present specification, the input of the second machine learning model may also include clinical constraint parameters, and the clinical constraint parameters are also structured data; therefore, the structured work scene monitoring data and the clinical constraint parameters are input into the pre-trained second machine learning model, so that the pre-trained second machine learning model can analyze the structured work scene monitoring data under the constraints of the clinical constraint parameters, thereby obtaining more accurate work monitoring results for the target object.

[0125] In some embodiments, the initial second machine learning model can be any suitable machine learning model, which is not limited in this specification and can be selected according to actual needs. The pre-trained second machine learning model is similar to the training process of the first machine learning model described above, which will not be repeated here.

[0126] The operation monitoring results may include whether the operator's operation is in compliance with regulations. For example, whether the preoperative preparations are complete, whether the intraoperative operation is correct and standardized (for example, whether the surgical instrument is taken incorrectly, whether the surgical instrument is taken back after use, etc.). The operation monitoring results may also include whether the target object's body movements, facial expressions, etc. are normal, to assist in monitoring the target object's vital signs.

[0127] In some embodiments, when it is confirmed that the operator's operation is not in compliance with regulations based on the operation monitoring results, corresponding auxiliary warning information can be generated on the on-site display device, and the auxiliary warning information can be output through voice or other means at the same time to provide relevant staff with timely response and processing. For example, when the operator takes the wrong surgical instrument, the operator can be reminded to change the surgical instrument in a timely manner through voice warning and text warning.

[0128] Although the process flows described above include multiple operations that occur in a particular order, it should be clearly understood that the processes may include more or fewer operations, which may be performed sequentially or in parallel (eg, using parallel processors or a multi-threaded environment).

[0129] The embodiments of this specification also provide a vital sign information integration processing device. Fig. 9 As shown, in some embodiments of the present specification, the vital sign information integration processing device 902 may include one or more processors 904, such as one or more central processing units (CPUs) or graphics processing units (GPUs), and each processing unit may implement one or more hardware threads. The vital sign information integration processing device 902 may also include any memory 906, which is used to store any kind of information such as code, settings, data, etc. In a specific embodiment, the computer program on the memory 906 and can be run on the processor 904, when the computer program is run by the processor 904, the instructions of the vital sign information integration processing method based on artificial intelligence described in any of the above embodiments can be executed. Non-limitingly, for example, the memory 906 may include any one or more combinations of the following: any type of RAM, any type of ROM, flash memory device, hard disk, optical disk, etc. More generally, any memory can use any technology to store information. Further, any memory can provide volatile or non-volatile retention of information. Further, any memory can represent a fixed or removable component of the vital sign information integration processing device 902. In one case, when the processor 904 executes the associated instructions stored in any memory or combination of memories, the vital sign information integrated processing device 902 can perform any operation of the associated instructions. The vital sign information integrated processing device 902 also includes one or more drive mechanisms 908 for interacting with any memory, such as a hard disk drive mechanism, an optical disk drive mechanism, etc.

[0130] The vital sign information integrated processing device 902 may also include an input / output interface 910 (I / O) for receiving various inputs (via input device 912) and for providing various outputs (via output device 914). A specific output mechanism may include a presentation device 916 and an associated graphical user interface 918 (GUI). In other embodiments, the input / output interface 910 (I / O), input device 912, and output device 914 may not be included, and the device may only serve as a vital sign information integrated processing device in a network. The vital sign information integrated processing device 902 may also include one or more network interfaces 920 for exchanging data with other devices via one or more communication links 922. One or more communication buses 924 couple the components described above together.

[0131] The communication link 922 may be implemented in any manner, for example, through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication link 922 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc. governed by any protocol or combination of protocols.

[0132] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products of some embodiments of the present specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processor to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processor generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0133] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processor to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processor so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0135] In a typical configuration, the vital sign information integration and processing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0136] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0137] Computer-readable media include permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by the vital signs information integration and processing device. As defined in this specification, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carriers.

[0138] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, the embodiments of this specification may take the form of complete hardware embodiments, complete software embodiments or embodiments combining software and hardware. Moreover, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0139] The present specification embodiments may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present specification embodiments may also be practiced in distributed computing environments where tasks are performed by remote processors connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0140] It should also be understood that in the embodiments of this specification, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0141] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. In the description of this specification, the description of the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of this specification. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, in the absence of contradiction, a person skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0142] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for integrating and processing vital sign information based on artificial intelligence, characterized in that: include: Obtain multiple vital signs information of the target object in real time; converting the vital sign information into structured vital sign information; Inputting the structured vital sign information and clinical constraint parameters into a pre-trained first machine learning model for analysis to obtain a vital sign change trend of the target object; the vital sign change trend of the target object includes a probability of vital sign abnormality; The first machine learning model is also used to obtain clinical constraint parameters of the target scenario; Output auxiliary warning information according to the trend of the vital signs change, including: output through the human-machine interface The target object's vital sign change trend and dialog box; the dialog box includes a first control for indicating approval and a second control for indicating disapproval; when receiving an operation of the relevant personnel on the first control, determining whether the probability of abnormal vital signs exceeds a preset probability threshold; if the probability of abnormal vital signs exceeds the preset probability threshold, outputting clinical auxiliary warning information to the target device; When an operation of the relevant personnel on the second control is received, the clinical constraint parameters are adjusted and re-analyzed using the first machine learning model.

2. The method for integrating and processing vital sign information based on artificial intelligence according to claim 1, characterized in that: The vital sign information includes a digitized image; the converting the vital sign information into structured vital sign information includes: Extracting regional image features from digitized images; Using a preset classification algorithm to identify the region type of each of the region image features; Determining the data type of each sub-region in each of the regional image features according to the region type; According to the data type, each sub-region feature in each of the regional image features is converted into structured vital sign information.

3. The method for integrating and processing vital sign information based on artificial intelligence according to claim 1, characterized in that: Also includes: Acquire the monitoring images of the working scene of the target object in real time; Converting the operation scene monitoring image into structured operation scene monitoring data; Inputting the structured operation scene monitoring data into a pre-trained second machine learning model for analysis to obtain an operation monitoring result for the target object; Auxiliary warning information is output according to the operation monitoring result.

4. A vital sign information integrated processing device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that: When the computer program is executed by the processor, the computer program executes the instructions of the method according to any one of claims 1 to 3.

5. A vital sign information integration processing system based on artificial intelligence, characterized in that: include: A variety of vital signs monitoring equipment, used to collect the target object's vital signs information in real time; Vital sign information integrated processing equipment, which is used for: Obtain multiple vital signs information of the target object in real time; converting the vital sign information into structured vital sign information; Inputting the structured vital sign information and clinical constraint parameters into a pre-trained first machine learning model for analysis to obtain a vital sign change trend of the target subject; The target object's vital sign change trend includes the probability of vital sign abnormality; Outputting auxiliary warning information according to the trend of changes in the vital signs includes: outputting the trend of changes in the vital signs of the target object and a dialog box through a human-computer interface; the dialog box includes a first control for indicating approval and a second control for indicating disapproval; when receiving an operation of the first control by a relevant person, determining whether the probability of abnormal vital signs exceeds a preset probability threshold; if the probability of abnormal vital signs exceeds the preset probability threshold, outputting clinical auxiliary warning information to the target device; When an operation of the relevant personnel on the second control is received, the clinical constraint parameters are adjusted and re-analyzed using the first machine learning model.

6. The artificial intelligence-based vital sign information integrated processing system according to claim 5, characterized in that: The vital sign monitoring equipment includes a first type of equipment and a second type of equipment; the vital sign information collected by the first type of equipment is non-digital vital sign information; the vital sign information collected by the second type of equipment is digital vital sign information.

7. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor of a vital sign information integration and processing device, the computer program executes instructions of the method according to any one of claims 1 to 3.

Images