Operating room visual data calling method

Abstract

The invention provides an operating room visual data calling method, and belongs to the technical field of operating room visual data calling, and the method comprises the steps: firstly taking monitoring equipment, image equipment, an electronic medical record system and an environment detector as data sources, configuring an interface to obtain data in real time, and adopting a unified timestamp to ensure data synchronization; cleaning and feature extraction are carried out on the collected vital sign data, and weighted fusion features are carried out based on an expert scoring method to form a data set; the data set is divided into a training set and a test set, time sequence prediction is carried out based on a deep learning model, fusion features of the next moment are predicted through the model according to the data monitored in real time, and the fusion features are compared with a preset threshold value to judge the vital sign state. And finally, the AR technology is utilized to superpose real-time monitoring data in an operation view, voice instructions are processed through a voice recognition API and a natural language processing model, efficient data calling and task execution are achieved, and therefore the safety and efficiency of the operation process are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of visual data calling in operating rooms, and in particular to a visual data calling method in an operating room. Background Art

[0002] The operating room visual data calling technology uses advanced sensors, cameras and data integration platforms to collect and process various key data during the operation, such as patient vital signs, operation progress and equipment status, in real time, and presents this data intuitively to the surgical team through high-resolution displays or augmented reality devices, thereby improving the accuracy and safety of the operation and optimizing team collaboration and decision-making efficiency.

[0003] Deficiencies of existing technologies:

[0004] The background technology of visual data call in the operating room usually involves data integration and monitoring of various medical devices and systems, such as monitoring equipment, imaging equipment, electronic medical record systems and environmental sensors. However, the existing technology usually has the problem of scattered data sources and difficulty in unified integration. The data update is not real-time enough, resulting in information lag, which in turn affects the timeliness and accuracy of decision-making. In addition, the existing system responds slowly to abnormal situations, lacks automated alarm functions, and cannot respond quickly in emergencies. The processing and execution efficiency of voice commands also needs to be improved, resulting in medical staff wasting time and energy in data call and operation, affecting the overall efficiency and safety of the operating room. These shortcomings limit the potential of data-driven decision-making in the medical environment and also affect the safety and treatment effects of patients. Therefore, improvements and innovations to these shortcomings are particularly important.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0006] The purpose of the present invention is to provide a method for calling visualization data in an operating room to solve the problems raised in the above-mentioned background technology.

[0007] To achieve the above object, the present invention provides a method for calling operating room visualization data, which specifically comprises the following steps:

[0008] Step 1: Consider the monitoring equipment, imaging equipment, electronic medical record system and environmental detectors in the operating room as data sources, configure interfaces for each data source, collect data from the data source in real time, and apply unified timestamps to mark the data to keep the data synchronized;

[0009] Step 2: Clean the data collected by the monitoring equipment, extract features from the vital signs data collected by the monitoring equipment, assign weights to each data based on the expert scoring method, perform weighted fusion on all feature data, obtain fusion features, and store the fusion features as a data set based on timestamps;

[0010] Step 3: Divide the data set into a training set and a test set with a division ratio of , based on deep learning model for time series prediction, input The fusion features of timestamps are integrated into the data fusion model. The fusion features of timestamps are used as labels to build a data fusion model;

[0011] Step 4: Obtain the data collected by the monitoring device in real time, obtain the real-time fusion features and input them into the model, compare the fusion features predicted at the next moment with the preset fusion feature threshold, and determine the status of the vital sign data at the current moment;

[0012] Step 5: Send real-time monitoring data through Technology superimposed on the surgical view, based on voice recognition , process the voice signal of the call task and convert it into text, extract keywords through the natural language processing model and perform data call tasks.

[0013] Furthermore, the patient's vital signs data, including heart rate, blood pressure, blood oxygen saturation, respiratory rate, body temperature and carbon dioxide partial pressure, are obtained through monitoring equipment, and the patient's intraoperative ultrasound, The optical machine and endoscope, the electronic medical record system obtains the patient's basic information, medical history records and laboratory results. The basic information includes name, gender, age, height and weight. The medical history records include previous medical history, surgical history and drug allergy history. The laboratory results include blood routine and urine routine results. The environmental detector obtains the temperature, humidity, air quality, air pressure and light intensity in the operating room;

[0014] Taking each patient as a unit, all the data are stored as a sub-database according to different data sources, and all sub-databases are integrated to build an operating room visualization database.

[0015] Furthermore, the specific logic for obtaining the fusion features of vital signs data is as follows:

[0016] The patient's heart rate, blood pressure, blood oxygen saturation, respiratory rate, body temperature and carbon dioxide partial pressure data were cleaned using The method performs outlier detection on each data. When , the data point is considered as an outlier and deleted;

[0017] The cleaned vital signs data is subjected to feature extraction, and a recurrent neural network is used for feature extraction. A weight is assigned to each data based on the expert scoring method to obtain the timestamp fusion feature. The formula used is:

[0018]

[0019] in, for The fusion feature of timestamp, is the number of data, for Timestamp The weight of the data, for Timestamp individual data;

[0020] The calculated fusion features are integrated into a data set based on the timestamp.

[0021] Furthermore, the specific logic for building the data fusion model is as follows:

[0022] The long short-term memory network model is used to build a data fusion model, and the previous The fusion features of timestamps, The fusion features of the timestamp are used as labels to perform time series prediction, train the data fusion model, and select the mean square error as the loss function. When the mean square error value is Within the range, it indicates that the data fusion model training is completed, and the test is performed by inputting the test set into the trained data fusion model.

[0023] Furthermore, the specific logic for determining whether the current vital sign data is abnormal is as follows:

[0024] The electrocardiogram, blood pressure, blood oxygen saturation, respiratory rate, body temperature and carbon dioxide partial pressure data of the current patient are obtained, the fusion features at the current moment are calculated, and input into the data fusion model to obtain the predicted fusion features at the next moment, and the fusion features predicted by the model are compared with the preset fusion feature threshold. If the fusion features predicted by the model are greater than the preset fusion feature threshold, an alarm is automatically issued, all vital signs data at the current moment are displayed, and abnormal vital signs data are marked. If the fusion features predicted by the model are lower than the preset fusion feature threshold, the vital signs data continue to be monitored.

[0025] Furthermore, the specific logic for executing the calling task is:

[0026] Monitor each vital sign in real time. If it exceeds the normal range, the device will automatically highlight the relevant data and use different colors to represent different value ranges. Capture the voice instructions of the task in real time. When a voice instruction is issued to call the task, the voice signal is processed and converted into text. The model extracts text keywords, first finds the sub-database corresponding to the patient in the operating room visualization database based on the name keyword, then finds the data in the corresponding data source in the sub-database based on the calling task instruction, and sends the data to the display terminal.

[0027] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0028] This operating room visualization data calling method improves the accuracy and timeliness of vital sign monitoring by integrating multiple data sources in real time, and uses deep learning models for time series prediction, making the judgment of patient status more accurate and identifying potential risks in advance. The application of technology enables monitoring data to be intuitively superimposed on the surgical view, enhancing the decision-making support capabilities of doctors. At the same time, through voice recognition and natural language processing, the data call process is simplified, the convenience and efficiency of operation are improved, thus significantly improving safety and medical quality during surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the overall method flow of the present invention. DETAILED DESCRIPTION

[0030] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0031] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0032] Example:

[0033] See also Figure 1 The present invention provides a method for calling visualization data in an operating room, and the specific steps include:

[0034] Step 1: Consider the monitoring equipment, imaging equipment, electronic medical record system and environmental detectors in the operating room as data sources, configure interfaces for each data source, collect data from the data source in real time, and apply unified timestamps to mark the data to keep the data synchronized;

[0035] In this embodiment, the patient's vital signs data, including heart rate, blood pressure, blood oxygen saturation, respiratory rate, body temperature and carbon dioxide partial pressure, are obtained through monitoring equipment, and the imaging equipment obtains the patient's intraoperative ultrasound, The optical machine and endoscope, the electronic medical record system obtains the patient's basic information, medical history records and laboratory results. The basic information includes name, gender, age, height and weight. The medical history records include previous medical history, surgical history and drug allergy history. The laboratory results include blood routine and urine routine results. The environmental detector obtains the temperature, humidity, air quality, air pressure and light intensity in the operating room;

[0036] Taking each patient as a unit, all the data are stored as a sub-database according to different data sources, and all sub-databases are integrated to build an operating room visualization database.

[0037] The purpose of integrating and storing the data collected by various monitoring devices, imaging equipment, electronic medical record systems and environmental detectors in the operating room into a visual database is to achieve all-round real-time monitoring and management of the surgical process. The data of each patient comes from a wide range of sources, covering everything from vital signs to environmental conditions, and these data are heterogeneous and complex. Storing these data in sub-databases according to different data sources and integrating them to build an operating room visual database can help improve the organization and accessibility of the data.

[0038] Step 2: Clean the data collected by the monitoring equipment, extract features from the vital signs data collected by the monitoring equipment, assign weights to each data based on the expert scoring method, perform weighted fusion on all feature data, obtain fusion features, and store the fusion features as a data set based on timestamps;

[0039] In this embodiment, the specific logic for obtaining the fusion features of vital sign data is as follows:

[0040] The patient's heart rate, blood pressure, blood oxygen saturation, respiratory rate, body temperature and carbon dioxide partial pressure data were cleaned using The method performs outlier detection on each data. When , the data point is considered as an outlier and deleted;

[0041] The cleaned vital signs data is subjected to feature extraction, and a recurrent neural network is used for feature extraction. A weight is assigned to each data based on the expert scoring method to obtain the timestamp fusion feature. The formula used is:

[0042]

[0043] in, for The fusion feature of timestamp, is the number of data, for Timestamp The weight of the data, for Timestamp individual data;

[0044] The calculated fusion features are integrated into a data set based on the timestamp.

[0045] Raw vital sign data may contain noise, errors, and outliers that, if not processed, can interfere with subsequent analysis and model performance. The method detects and removes outliers to ensure that only reliable data is used for analysis. Vital sign data usually contains a lot of redundant information. Through feature extraction, we can focus on the key features that affect the patient's health status. This refinement process helps to improve the accuracy and efficiency of the model. Using recurrent neural networks for feature extraction can capture complex patterns and time dependencies in time series data, thereby improving the understanding and prediction of patient status. Assigning weights to features based on expert scoring can effectively integrate medical expertise, so that the model not only relies on data, but also considers the experience and insights of experts. Weight assignment helps to highlight the influence of important features in the final decision and enhance the reliability and effectiveness of the model in practical applications. Through weighted fusion, features from different vital signs can be integrated to form a comprehensive view of the patient's health status. This comprehensive analysis can provide more accurate health assessments. In a monitoring environment, vital sign data changes dynamically. Real-time feature fusion can quickly reflect the patient's health changes and help doctors make decisions and interventions in a timely manner. Storing fused feature data according to timestamps not only facilitates subsequent analysis, but also helps the medical team quickly query and compare data from different periods to form a more comprehensive patient monitoring record.

[0046] Step 3: Divide the data set into a training set and a test set with a division ratio of , based on deep learning model for time series prediction, input The fusion features of timestamps are integrated into the data fusion model. The fusion features of timestamps are used as labels to build a data fusion model;

[0047] In this embodiment, the specific logic for constructing the data fusion model is:

[0048] The long short-term memory network model is used to build a data fusion model, and the previous The fusion features of timestamps, The fusion features of the timestamp are used as labels to perform time series prediction, train the data fusion model, and select the mean square error as the loss function. When the mean square error value is Within the range, it indicates that the data fusion model training is completed, and the test is performed by inputting the test set into the trained data fusion model.

[0049] In the early stages of deep learning model training, the data may not be sufficient or the model has not yet converged, and the weights obtained by automatic learning based on the data may be unstable or inaccurate. At this time, the weights assigned based on the expert scoring method can provide a reasonable initial weight setting, so that the model starts learning from a better initial state, thereby improving the model convergence speed and the accuracy of the initial prediction. The expert scoring method is highly flexible and can be adjusted according to specific application scenarios, individual differences of patients, or the needs of specific surgical procedures. The situation of each patient may be different, and experts can dynamically adjust the weights of different data sources according to the specific situation, making the model more personalized and customized, and better able to adapt to different clinical environments.

[0050] The long short-term memory network is a neural network model suitable for processing time series data and can effectively capture long-term dependencies. In this method, the long short-term memory network model is used to predict the future state of vital signs. By inputting the data collected by the monitoring device into the long short-term memory network model and using the fusion features of the corresponding timestamps as labels for training, the model can learn the temporal relationship between the data and the mutual influence between the data sources. The mean square error is used as the loss function because the mean square error can effectively measure the gap between the predicted value and the actual value, thereby guiding the optimization of the model. When the mean square error value is When it is within the range, it means that the prediction error of the model has dropped to a reasonable range, indicating that the model has been basically trained and has good generalization ability.

[0051] Step 4: Obtain the data collected by the monitoring device in real time, obtain the real-time fusion features and input them into the model, compare the fusion features predicted at the next moment with the preset fusion feature threshold, and determine the status of the vital sign data at the current moment;

[0052] In this embodiment, the specific logic for determining whether the current vital sign data is abnormal is:

[0053] The formula for obtaining the fusion feature threshold is:

[0054]

[0055] in, is the fusion feature threshold, is the mean of historical fusion features, is the standard deviation of historical fusion feature data, is the safety factor;

[0056] The electrocardiogram, blood pressure, blood oxygen saturation, respiratory rate, body temperature and carbon dioxide partial pressure data of the current patient are obtained, the fusion features at the current moment are calculated, and input into the data fusion model to obtain the predicted fusion features at the next moment, and the fusion features predicted by the model are compared with the preset fusion feature threshold. If the fusion features predicted by the model are greater than the preset fusion feature threshold, an alarm is automatically issued, all vital signs data at the current moment are displayed, and abnormal vital signs data are marked. If the fusion features predicted by the model are lower than the preset fusion feature threshold, the vital signs data continue to be monitored.

[0057] Usually the safety factor It can be 2 or 3. Physiological data will naturally fluctuate within the normal range. Setting an appropriate safety factor can avoid alarms due to normal fluctuations, thereby reducing unnecessary interference and anxiety. If the threshold is too close to normal data, it may not be monitored in time when the patient's condition deteriorates. By setting the safety factor, the threshold will be more relaxed, allowing the device to capture abnormal trends more sensitively. The physiological parameters of each patient may vary due to factors such as age, gender, and underlying diseases. Dynamic thresholds combined with historical data can better adapt to the characteristics of different individuals. In statistics, many physiological data follow a normal distribution. According to the principle of standard deviation, set or Usually covers about or This means that most physiological data under normal circumstances will be within this range.

[0058] By collecting the patient's vital signs data in real time, the system can continuously monitor the patient's health status. This continuous monitoring is an important means to ensure timely detection of changes in the patient's status. When the fusion feature data predicted by the model is lower than the preset threshold, the system will continue to input the vital signs data of the next moment for monitoring. This feedback mechanism ensures that the system always keeps an eye on the patient's status and makes adjustments when necessary. Even in a relatively safe state, the system will not stop monitoring to ensure that potential risks can be detected in a timely manner at any time. By continuously updating the input data, the model can adapt to changes in the patient's status, learn and adjust incrementally, and provide the latest information for clinical decision-making. This dynamic adaptability is an important feature of the design of modern medical monitoring systems, which helps to respond to changes in patient conditions in a timely manner.

[0059] Step 5: Send real-time monitoring data through Technology superimposed on the surgical view, based on voice recognition , process the voice signal of the call task and convert it into text, extract keywords through the natural language processing model and perform data call tasks.

[0060] In this embodiment, the specific logic for executing the calling task is:

[0061] Monitor each vital sign in real time. If it exceeds the normal range, the device will automatically highlight the relevant data and use different colors to represent different value ranges. Capture the voice instructions of the task in real time. When a voice instruction is issued to call the task, the voice signal is processed and converted into text. The model extracts text keywords, first finds the sub-database corresponding to the patient in the operating room visualization database based on the name keyword, then finds the data in the corresponding data source in the sub-database based on the calling task instruction, and sends the data to the display terminal.

[0062] The normal range of heart rate is 60-100 beats / minute, the borderline range is 50-60 beats / minute and 100-110 beats / minute, the abnormal range is below 50 beats / minute and above 110 beats / minute, and the normal range of blood pressure is systolic pressure. , diastolic blood pressure , the boundary range is systolic blood pressure , diastolic pressure , the abnormal range is systolic blood pressure , diastolic blood pressure The normal range of blood oxygen saturation is , the boundary range is , the abnormal range is below The normal range of respiratory rate is 12-20 times / minute, the borderline range is 10-12 times / minute and 20-24 times / minute, the abnormal range is less than 10 times / minute or more than 24 times / minute, and the normal range of body temperature is , the boundary range is and , the abnormal range is below or higher The normal range of partial pressure of carbon dioxide is , the boundary range is and , the abnormal range is below or higher .

[0063] During the monitoring process, if the data is within the normal range, it is indicated in green; if it is within the boundary range, it is indicated in yellow; and if it is within the abnormal range, it is indicated in red. During the operation, the doctor needs to respond quickly to the patient's physiological state. Superimposing real-time monitoring data on the surgical view allows the doctor to see the patient's vital signs at a glance and make judgments and decisions in a timely manner. Automatically highlighting vital signs that are out of the normal range can quickly attract the doctor's attention and reduce medical risks caused by missing or delayed information. By using voice recognition , which can capture and convert voice commands in the operating room in real time, converting voice signals into text. Speech recognition technology relies on trained models that can recognize different speech patterns and vocabulary to accurately convert voice information into text information. During surgery, time is of the essence and the surgical team may need to obtain certain information immediately. Speech recognition technology can capture instructions quickly and seamlessly, reducing the time delay of manual operations. The converted text is analyzed using a natural language processing model to extract keywords and important information. The natural language processing model can understand the semantic structure of the sentence, so as to accurately identify the core content of the task instruction, such as the patient's name, the task or data type that needs to be called. The system can handle multiple voice requests, supporting doctors and other team members to quickly switch between different information needs during complex surgeries without delays or conflicts.

[0064] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0065] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. Those skilled in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0066] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0067] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A method for calling operating room visualization data, characterized in that: The specific steps include: Step 1: Consider the monitoring equipment, imaging equipment, electronic medical record system and environmental detectors in the operating room as data sources, configure interfaces for each data source, collect data from the data source in real time, and apply unified timestamps to mark the data to keep the data synchronized; Step 2: Clean the data collected by the monitoring equipment, extract features from the vital signs data collected by the monitoring equipment, assign weights to each data based on the expert scoring method, perform weighted fusion on all feature data, obtain fusion features, and store the fusion features as a data set based on timestamps; Step 3: Divide the data set into a training set and a test set with a division ratio of , based on deep learning model for time series prediction, input The fusion features of timestamps are integrated into the data fusion model. The fusion features of timestamps are used as labels to build a data fusion model; Step 4: Obtain the data collected by the monitoring device in real time, obtain the real-time fusion features and input them into the model, compare the fusion features predicted at the next moment with the preset fusion feature threshold, and determine the status of the vital sign data at the current moment; Step 5: Send real-time monitoring data through Technology superimposed on the surgical view, based on voice recognition , process the voice signal of the call task and convert it into text, extract keywords through the natural language processing model and perform data call tasks.

2. The method for calling operating room visualization data according to claim 1, characterized in that: The patient's vital signs data, including heart rate, blood pressure, blood oxygen saturation, respiratory rate, body temperature and carbon dioxide partial pressure, are obtained through monitoring equipment, and the patient's intraoperative ultrasound, The optical machine and endoscope, the electronic medical record system obtains the patient's basic information, medical history records and laboratory results. The basic information includes name, gender, age, height and weight. The medical history records include previous medical history, surgical history and drug allergy history. The laboratory results include blood routine and urine routine results. The environmental detector obtains the temperature, humidity, air quality, air pressure and light intensity in the operating room; Taking each patient as a unit, all the data are stored as a sub-database according to different data sources, and all sub-databases are integrated to build an operating room visualization database.

3. The method for calling operating room visualization data according to claim 2, characterized in that: The specific logic for obtaining the fusion features of vital signs data is as follows: The patient's heart rate, blood pressure, blood oxygen saturation, respiratory rate, body temperature and carbon dioxide partial pressure data were cleaned using The method performs outlier detection on each data. When , the data point is considered as an outlier and deleted; The cleaned vital signs data is subjected to feature extraction, and a recurrent neural network is used for feature extraction. A weight is assigned to each data based on the expert scoring method to obtain the timestamp fusion feature. The formula used is: ； in, for The fusion feature of timestamp, is the number of data, for Timestamp The weight of the data, for Timestamp individual data; The calculated fusion features are integrated into a data set based on the timestamp.

4. The method for calling operating room visualization data according to claim 3, characterized in that: The specific logic for building the data fusion model is: The long short-term memory network model is used to build a data fusion model, and the previous The fusion features of timestamps, The fusion features of the timestamp are used as labels to perform time series prediction, train the data fusion model, and select the mean square error as the loss function. When the mean square error value is Within the range, it indicates that the data fusion model training is completed, and the test is performed by inputting the test set into the trained data fusion model.

5. The method for calling operating room visualization data according to claim 4, characterized in that: The specific logic for judging whether the current vital signs data is abnormal is: The electrocardiogram, blood pressure, blood oxygen saturation, respiratory rate, body temperature and carbon dioxide partial pressure data of the current patient are obtained, the fusion features at the current moment are calculated, and input into the data fusion model to obtain the predicted fusion features at the next moment, and the fusion features predicted by the model are compared with the preset fusion feature threshold. If the fusion features predicted by the model are greater than the preset fusion feature threshold, an alarm is automatically issued, all vital signs data at the current moment are displayed, and abnormal vital signs data are marked. If the fusion features predicted by the model are lower than the preset fusion feature threshold, the vital signs data continue to be monitored.

6. The method for calling operating room visualization data according to claim 5, characterized in that: The specific logic for executing the calling task is: Monitor each vital sign in real time. If it exceeds the normal range, the device will automatically highlight the relevant data and use different colors to represent different value ranges. Capture the voice instructions of the task in real time. When a voice instruction is issued to call the task, the voice signal is processed and converted into text. The model extracts text keywords, first finds the sub-database corresponding to the patient in the operating room visualization database based on the name keyword, then finds the data in the corresponding data source in the sub-database based on the calling task instruction, and sends the data to the display terminal.

Images