Dynamic monitoring and recording system for digital operating rooms

Abstract

This application provides a dynamic monitoring and recording system for a digital operating room, comprising: a surgical recording module for real-time surgical recording based on intraoperative sensing data; a dynamic monitoring module for real-time anomaly monitoring of the operating room environment, medical staff behavior, and patient intraoperative status based on intraoperative sensing data, and obtaining real-time monitoring results; and an anomaly response module for corresponding anomaly responses based on the anomaly level of the real-time monitoring results.

Description

Technical Field

[0001] This application belongs to the field of digital operating rooms, and particularly relates to a dynamic monitoring and recording system for digital operating rooms. Background Technology

[0002] A digital integrated operating room is a medical project that integrates purification engineering and digital information technology. By integrating medical imaging, equipment control and communication technologies, it can realize functions such as real-time sharing of surgical images, centralized control of equipment, information integration and remote teaching and consultation.

[0003] In related digital operating room solutions, a separate integrated control terminal is often required to control all medical equipment in the operating room in a unified manner, which makes the operating room renovation cost high. Moreover, the improvement of digital operating rooms compared with traditional operating rooms is mainly reflected in unified control and content presentation. The overall level of intelligence is not high, and it is difficult to meet the personalized needs of doctors and patients. Summary of the Invention

[0004] This application provides a dynamic monitoring and recording system for a digital operating room, which can record surgery, monitor the operation dynamically, and respond to abnormalities during the operation, thereby improving surgical safety and recording efficiency.

[0005] In a first aspect, embodiments of this application provide a dynamic monitoring and recording system for a digital operating room, comprising: The surgical recording module is used to record the surgical procedure in real time based on intraoperative sensory data. The dynamic monitoring module is used to monitor the operating room environment, medical staff behavior, and patient status in real time based on intraoperative perception data, and obtain real-time monitoring results. The anomaly response module is used to respond to anomalies according to the anomaly level of the real-time monitoring results.

[0006] Optionally, the surgical record module is specifically used for: Surgical records are generated in real time based on intraoperative sensory data, resulting in real-time surgical record updates. These results are then used to update surgical progress, surgical scheduling plans, provide real-time surgical operation suggestions, and offer real-time surgical operation risk alerts. The intraoperative sensing data includes patient physiological data, real-time surgical video data, and surgical instrument tracking data.

[0007] Optionally, the surgical record module is specifically used for: Intraoperative sensory data is input into a pre-trained intelligent surgical collaborative decision-making model to obtain a real-time surgical record output by the intelligent surgical collaborative decision-making model; wherein, The intelligent surgical collaborative decision-making model includes a multimodal perception data input module, a feature fusion module, a surgical record generation module, a surgical progress and surgical scheduling prediction module, and a surgical operation suggestion and risk warning module.

[0008] Optionally, the surgical record module is specifically used for: Based on the real-time audio and video data and patient physiological data recorded during the operation, a surgical record is generated in real time. Based on the surgical records and preoperative surgical plan, a surgical debriefing report containing suggestions for surgical improvement is generated.

[0009] Optionally, the dynamic monitoring module is specifically used for: Monitoring of medical staff's behavior, operational procedures, and status; Real-time monitoring of patients' physiological parameters, monitoring of key surgical points, and monitoring of anesthesia recovery; Air quality monitoring, temperature and humidity monitoring, and equipment operation status monitoring are conducted in the operating room environment.

[0010] Optionally, the exception response module is specifically used for: When the real-time monitoring results are abnormal, an anomaly description is generated; Intraoperative sensing data, patient visit data, and abnormal description information are input into a pre-trained surgical abnormality response model to obtain the abnormality response scheme output by the surgical abnormality response model.

[0011] Optionally, the intelligent operating room further includes a surgical robot system, an information integration system, an intelligent collaborative decision-making system, an intelligent content distribution system, an intelligent integrated control system, and a 5G remote system; wherein, Surgical robot systems are used to assist in surgical operations according to the pre-operative surgical plan and serve as the control center for digital and intelligent surgery to realize corresponding intelligent functions. The information integration system serves as the data hub of the digital operating room, enabling data flow and business collaboration from different business systems and smart terminals. The intelligent collaborative decision-making system is used to generate surgical plans based on patient medical data through intelligent preoperative planning assistance; to provide real-time intraoperative prompts based on the preoperative surgical plan; to generate surgical operation suggestions in real time during the operation through a pre-trained intelligent surgical collaborative decision-making model, and to update the surgical plan in real time based on the response results of the surgical operation suggestions; and to perform surgical effect evaluation, personalized rehabilitation plan generation, and postoperative rehabilitation assessment. The intelligent content distribution system is used to aggregate content through wired and / or wireless interfaces and share intraoperative perception data to the surgical robot system; record and intelligently analyze intraoperative perception data to generate personalized recommended content that matches each intelligent terminal associated with the digital operating room; and use the surgical robot system as the content distribution center to distribute personalized recommended content to each intelligent terminal associated with the digital operating room for interactive presentation of intraoperative content. The intelligent integrated control system is used to comprehensively control various medical devices in the digital operating room, with the surgical robot system as the control center; and to perform operation scheduling and management of the digital operating room to improve its utilization efficiency. The 5G remote system is used for live teaching demonstrations and automatically generating surgical teaching videos based on the live broadcast content; controlling surgical robots according to received remote surgical instructions to achieve remote surgical intervention; and conducting remote multidisciplinary consultations, intraoperative medical data retrieval, and remote collaboration for pre-hospital emergency care.

[0012] Optionally, the intelligent collaborative decision-making system is specifically used for: Based on the surgical plan information included in the surgical plan, real-time intraoperative prompts are provided when the surgery reaches the target operation. The surgical plan is generated based on a pre-trained personalized surgical plan improvement model. The input of the personalized surgical plan improvement model includes the initial surgical plan provided by the doctor and the anonymized patient medical data. The output modules of the personalized surgical plan improvement model include a surgical plan matching degree assessment module, a surgical plan improvement suggestion module, and an operation guidance and risk warning module.

[0013] Optionally, the intelligent collaborative decision-making system is specifically used for: Intraoperative perception data and preoperative surgical plans are input into a pre-trained intelligent surgical collaborative decision-making model to obtain real-time surgical operation suggestions output by the intelligent surgical collaborative decision-making model. The intraoperative sensing data includes patient physiological data, real-time surgical video data, and surgical instrument tracking data.

[0014] Optionally, the intelligent content distribution system is specifically used for: Intraoperative perception data, patient visit data, content receiver identification, and corresponding smart device type are input into the personalized recommendation content generation model to obtain personalized recommendation content generated by the model for each content receiver; wherein... The personalized recommendation content generation model includes a multi-dimensional feature extraction module, a feature fusion and recommendation logic engine, and a content generation and display enhancement module.

[0015] Secondly, embodiments of this application provide a dynamic monitoring and recording method for a digital operating room, wherein the intelligent method is used to implement the functions of the dynamic monitoring and recording system for a digital operating room as described in any embodiment of the first aspect.

[0016] Thirdly, embodiments of this application provide an electronic device, which includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the functions of a dynamic monitoring and recording system for a digital operating room.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement functions for a dynamic monitoring and recording system in a digital operating room.

[0018] The dynamic monitoring and recording system, method, device, and computer-readable storage medium for digital operating rooms in this application embodiment provide three functions: surgical recording, dynamic monitoring, and anomaly response. It can record the surgery in real time based on intraoperative perception data, and perform real-time anomaly monitoring of the operating room environment, medical staff behavior, and patient's intraoperative status based on intraoperative perception data to obtain real-time monitoring results. Thus, it can perform corresponding anomaly responses based on the anomaly level of the real-time monitoring results, thereby improving surgical safety and surgical recording efficiency. Attached Figure Description

[0019] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the architecture of a dynamic monitoring and recording system for a digital operating room provided in one embodiment of this application; Figure 2 This is an overall architecture diagram of the intelligent surgical collaborative decision-making model used in a dynamic monitoring and recording system for a digital operating room, provided in one embodiment of this application. Figure 3 This is an overall architecture diagram of the surgical abnormality response model used in a dynamic monitoring and recording system for a digital operating room, provided in one embodiment of this application. Figure 4 This is a schematic diagram of the architecture of a digital operating room where a dynamic monitoring and recording system for a digital operating room is located, according to one embodiment of this application. Figure 5 This is a schematic diagram of the architecture of a dynamic monitoring and recording system for a digital operating room provided in one embodiment of this application, in which a surgical robot serves as the content distribution and control center. Figure 6 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0021] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0022] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0023] To address the problems of existing technologies, embodiments of this application provide a dynamic monitoring and recording system for a digitalized operating room. The dynamic monitoring and recording system for a digitalized operating room provided in this application embodiment will be described below. Figure 1 This is a schematic diagram of the architecture of a dynamic monitoring and recording system for a digital operating room according to an embodiment of this application. The dynamic monitoring and recording system for a digital operating room includes a surgical recording module, a dynamic monitoring module, and an anomaly response module; wherein, The surgical recording module is used to record the surgical procedure in real time based on intraoperative sensory data. The dynamic monitoring module is used to monitor the operating room environment, medical staff behavior, and patient status in real time based on intraoperative perception data, and obtain real-time monitoring results. The anomaly response module is used to respond to anomalies according to the anomaly level of the real-time monitoring results.

[0024] In some embodiments, the surgical recording module is specifically used for: Surgical records are generated in real time based on intraoperative sensory data, resulting in real-time surgical record updates. These results are then used to update surgical progress, surgical scheduling plans, provide real-time surgical operation suggestions, and offer real-time surgical operation risk alerts. The intraoperative sensing data includes patient physiological data, real-time surgical video data, and surgical instrument tracking data.

[0025] In some embodiments, the surgical recording module is specifically used for: Intraoperative sensory data is input into a pre-trained intelligent surgical collaborative decision-making model to obtain a real-time surgical record output by the intelligent surgical collaborative decision-making model; wherein, The intelligent surgical collaborative decision-making model includes a multimodal perception data input module, a feature fusion module, a surgical record generation module, a surgical progress and surgical scheduling prediction module, and a surgical operation suggestion and risk warning module.

[0026] Figure 2 This is an overall architecture diagram of the intelligent surgical collaborative decision-making model used in a dynamic monitoring and recording system for a digital operating room, provided in one embodiment of this application.

[0027] The intraoperative perception data includes patient physiological data, real-time surgical audio and video data, and surgical instrument tracking data. The intelligent surgical collaborative decision-making model is a generative model capable of processing real-time serialized data. The input of the intelligent surgical collaborative decision-making model includes serialized intraoperative perception data and prompt information. The prompt information may include intraoperative interaction requirements (such as confirming the current surgical progress, estimating the remaining surgical time, etc.).

[0028] The output of the intelligent surgical collaborative decision-making model includes surgical records, real-time intraoperative prompts, intraoperative operation suggestions, surgical progress estimates, and surgical scheduling; among which... The model architecture of the intraoperative real-time alert module can be a CNN+BiLSTM+Attention mechanism. The number of CNN convolutional kernels can be 32 to extract local features of vital signs; the kernel size can be 3*3 to optimize feature extraction; the number of BiLSTM hidden units can be set to 64 to handle temporal data; the number of Attention heads can be set to 8 to optimize multi-dimensional feature fusion; the loss function can be set to a weighted cross-entropy loss function to focus on identifying high-risk events (high-risk surgical procedures, etc.); the optimizer can be set to AdamW to improve the convergence speed during training; and the learning rate can be set to 0.001 to control the learning speed.

[0029] The model architecture of the intraoperative operation suggestion module can be DQN + expert knowledge base. It generates initial surgical suggestions based on DQN and filters and sorts them according to the expert knowledge base to output high-value surgical operation suggestions. The state space dimension can be set to 50, which includes surgical stage, vital signs, risk level, etc.; the action space size can be set to 100, which is used to define the number of possible operation suggestions; the discount factor can be set to 0.95 to encourage future reward weights; the learning rate can be set to 0.001 to control the learning speed; the experience replay buffer size can be set to 100,000 to store historical experience; the batch size can be set to 64; and the target network update frequency can be set to 100 to balance training effect and training efficiency.

[0030] The surgical progress prediction module can be built using an LSTM + Attention mechanism. This mechanism can predict the remaining time based on historical data, adjust the prediction according to the current risk level, and generate a confidence interval. The LSTM hidden units can be set to 128 for temporal modeling; the Attention heads can be set to 4 for feature fusion; the sliding window size can be dynamically adjusted according to the surgical type, for example, it can be set to 30 minutes; the prediction step size can be set to 5 minutes; and the loss function can be set to MAE to evaluate prediction accuracy. The surgical record module can be built on a multimodal pre-trained generative model, which can generate surgical records in multimedia format based on the input intraoperative perception data; the surgical scheduling module can predict the remaining surgical time based on the estimated current surgical progress and update the subsequent surgical schedule based on the remaining surgical time.

[0031] In some embodiments, the surgical recording module is specifically used for: Based on the real-time audio and video data and patient physiological data recorded during the operation, a surgical record is generated in real time. Based on the surgical records and preoperative surgical plan, a surgical debriefing report containing suggestions for surgical improvement is generated.

[0032] In some embodiments, the dynamic monitoring module is specifically used for: Monitoring of medical staff's behavior, operational procedures, and status; Real-time monitoring of patients' physiological parameters, monitoring of key surgical points, and monitoring of anesthesia recovery; Air quality monitoring, temperature and humidity monitoring, and equipment operation status monitoring are conducted in the operating room environment.

[0033] In some embodiments, the exception response module is specifically used for: When the real-time monitoring results are abnormal, an anomaly description is generated; Intraoperative sensing data, patient visit data, and abnormal description information are input into a pre-trained surgical abnormality response model to obtain the abnormality response scheme output by the surgical abnormality response model.

[0034] Figure 3 This is an overall architecture diagram of the surgical abnormality response model used in a dynamic monitoring and recording system for a digital operating room, provided in one embodiment of this application.

[0035] The patient's medical data includes historical diagnostic records, medical imaging data, historical medical test data, historical surgical data, historical medication data, and patient attribute information. The patient attribute information may include age, gender, weight, height, BMI, occupation, history of underlying diseases, and allergies. Historical diagnostic records may include diagnostic descriptions, such as "left knee arthritis." Medical imaging data may include knee X-rays, MRI scans, CT scans, etc. Historical medical test data may include complete blood count data, blood biochemistry data, and urinalysis results. Historical surgical records may include the type of surgery, surgery time, postoperative rehabilitation records, and postoperative complication records. Historical medication records may include the name, dosage, frequency, and time of medication use.

[0036] The surgical anomaly response model includes a feature extraction module, an anomaly classification module, and a response scheme generation module.

[0037] The feature extraction module includes a data cleaning and integration module, a multi-dimensional feature extraction module, and a multi-dimensional feature fusion module. The data cleaning and integration module includes a data cleaning branch and a data integration branch. In the data cleaning branch, wavelet transform can be used to eliminate noise, mean interpolation algorithm can be used to fill missing data, and interquartile range method can be used to remove outliers. In the data integration branch, data from different sources can be associated with the same timestamp, and time series alignment algorithm can be used to ensure the time consistency of multi-source data. A time window sliding algorithm is used to construct a historical data window of a preset number of minutes (e.g., 30 minutes). The multi-dimensional feature fusion module can perform weighted fusion of time-related features (to reflect the trend of vital signs), spatially related features (to reflect abnormal conditions in the surgical field), and semantically related signs (to reflect abnormal features described by medical staff) obtained from multi-dimensional feature extraction.

[0038] The multidimensional feature extraction module includes an intraoperative perception data encoder, a patient visit data encoder, and an abnormal description information encoder. The intraoperative perception data encoder includes a temporal feature extraction layer, a statistical feature extraction layer, an anomaly detection feature extraction layer, and a multi-scale feature extraction layer. The temporal feature extraction layer uses a bidirectional LSTM network with an input dimension of 12, a hidden layer size of 64, bidirectional output, and an output dimension of 128. The statistical feature extraction layer employs global statistical pooling to calculate statistical data such as mean, variance, maximum, and minimum values, with an output dimension of 48. The anomaly detection feature extraction layer uses an Isolation Forest anomaly detector with 100 trees and a maximum depth of 10 for anomaly detection. The multi-scale feature extraction layer uses Daubechies wavelet transform with a decomposition layer of 4 to extract features at different scales. The medical data encoder includes a text feature extraction layer, an image feature extraction layer, and a structured data feature extraction layer. The text feature extraction layer is configured as a fine-tuned model based on BERT-base, which can extract text features from the input case-related text. The image feature extraction layer is configured as a ResNet-50 pre-trained model, which can extract image features from the input medical images. The anomaly description information encoder includes a text feature extraction layer, an image feature extraction layer, and a structured data feature extraction layer. The text feature extraction layer is configured as a fine-tuned model based on BERT-base, which can extract text features from the input anomaly description text. The image feature extraction layer is configured as a ResNet-50 pre-trained model, which can extract image features from the input anomaly medical images.

[0039] The anomaly classification stage includes anomaly detection, anomaly classification, and a potential anomaly matching engine based on medical knowledge graphs and rule bases. The anomaly detection process includes anomaly detection based on historical data and anomaly detection based on real-time data, so as to perform sufficient anomaly detection based on historical and real-time data and avoid false detections. In the anomaly detection process based on historical data, Isolation Forest can be used for preliminary anomaly detection, and then LSTM-Autoencoder is used for real-time anomaly detection. The library makes a comprehensive judgment based on the detection results of preliminary and real-time anomaly detection.

[0040] When performing anomaly classification, the Transformer model and the XGBoost model can be used. The Transformer model is used for high-precision classification, while the XGBoost model is used as an auxiliary classifier to handle classification tasks with sparse features. The final classification result can be a combination of the classification results of the Transformer model and the XGBoost model.

[0041] During the process of potential anomaly matching, the potential anomaly matching engine can perform matching based on the pre-built knowledge graph and medical knowledge rules, according to the abnormal feature vector output after multi-dimensional feature fusion, and output the current potential anomaly to enrich the anomaly classification results and avoid missed detection.

[0042] The response scheme generation module includes a response scheme matching module, a personalized fine-tuning module, and a priority sorting module. It can match response schemes based on the input abnormality classification results and the supplementary potential abnormality classification results to obtain the preset response scheme that best matches the current abnormality. It can also fine-tune the scheme based on the patient's medical data to obtain the abnormal response scheme, and output it after priority sorting.

[0043] Figure 4 This is a schematic diagram of the architecture of a digital operating room, in which a dynamic monitoring and recording system for a digital operating room, provided in one embodiment of this application, is located. In some embodiments, the intelligent operating room further includes a surgical robot system, an information integration system, an intelligent collaborative decision-making system, an intelligent content distribution system, an intelligent integrated control system, and a 5G remote system; wherein, Surgical robot systems are used to assist in surgical operations according to the pre-operative surgical plan and serve as the control center for digital and intelligent surgery to realize corresponding intelligent functions. The information integration system serves as the data hub of the digital operating room, enabling data flow and business collaboration from different business systems and smart terminals. The intelligent collaborative decision-making system is used to generate surgical plans based on patient medical data through intelligent preoperative planning assistance; to provide real-time intraoperative prompts based on the preoperative surgical plan; to generate surgical operation suggestions in real time during the operation through a pre-trained intelligent surgical collaborative decision-making model, and to update the surgical plan in real time based on the response results of the surgical operation suggestions; and to perform surgical effect evaluation, personalized rehabilitation plan generation, and postoperative rehabilitation assessment. The intelligent content distribution system is used to aggregate content through wired and / or wireless interfaces and share intraoperative perception data to the surgical robot system; record and intelligently analyze intraoperative perception data to generate personalized recommended content that matches each intelligent terminal associated with the digital operating room; and use the surgical robot system as the content distribution center to distribute personalized recommended content to each intelligent terminal associated with the digital operating room for interactive presentation of intraoperative content. The intelligent integrated control system is used to comprehensively control various medical devices in the digital operating room, with the surgical robot system as the control center; and to perform operation scheduling and management of the digital operating room to improve its utilization efficiency. The 5G remote system is used for live teaching demonstrations and automatically generating surgical teaching videos based on the live broadcast content; controlling surgical robots according to received remote surgical instructions to achieve remote surgical intervention; and conducting remote multidisciplinary consultations, intraoperative medical data retrieval, and remote collaboration for pre-hospital emergency care.

[0044] The business system can be a hospital-based business system such as an imaging business system (the data can be CT data, MRI data, etc.), and the smart terminal can be a smart terminal associated with the digital operating room, such as a smart terminal used by doctors, scrub nurses, anesthesiologists, and other personnel.

[0045] In some embodiments, the aforementioned systems used in the digital operating room can be deployed in the control equipment used by the surgical robot, so that the control equipment of the surgical robot can not only control the surgical robot, but can also serve as the central hub of the entire digital operating room for comprehensive control. Thus, when the operating room is undergoing digital transformation, the digital transformation of the operating room can be completed by introducing the surgical robot system and making corresponding adaptations, which greatly reduces the cost of digital transformation of the operating room and improves the transformation efficiency and the operating room's usage effect.

[0046] In some embodiments, the intelligent collaborative decision-making system is specifically used for: Based on the surgical plan information included in the surgical plan, real-time intraoperative prompts are provided when the surgery reaches the target operation. The surgical plan is generated based on a pre-trained personalized surgical plan improvement model. The input of the personalized surgical plan improvement model includes the initial surgical plan provided by the doctor and the anonymized patient medical data. The output modules of the personalized surgical plan improvement model include a surgical plan matching degree assessment module, a surgical plan improvement suggestion module, and an operation guidance and risk warning module.

[0047] In some embodiments, the intelligent collaborative decision-making system is specifically used for: Intraoperative perception data and preoperative surgical plans are input into a pre-trained intelligent surgical collaborative decision-making model to obtain real-time surgical operation suggestions output by the intelligent surgical collaborative decision-making model. The intraoperative sensing data includes patient physiological data, real-time surgical video data, and surgical instrument tracking data.

[0048] In some embodiments, the intelligent content distribution system is specifically used for: Intraoperative perception data, patient visit data, content receiver identification, and corresponding smart device type are input into the personalized recommendation content generation model to obtain personalized recommendation content generated by the model for each content receiver; wherein... The personalized recommendation content generation model includes a multi-dimensional feature extraction module, a feature fusion and recommendation logic engine, and a content generation and display enhancement module.

[0049] The patient's medical data includes historical diagnostic records, medical imaging data, historical medical test data, historical surgical data, historical medication data, and patient attribute information. The patient attribute information may include age, gender, weight, height, BMI, occupation, history of underlying diseases, and allergies. Historical diagnostic records may include diagnostic descriptions, such as "left knee arthritis." Medical imaging data may include knee X-rays, MRI scans, CT scans, etc. Historical medical test data may include complete blood count data, blood biochemistry data, and urinalysis results. Historical surgical records may include the type of surgery, surgery time, postoperative rehabilitation records, and postoperative complication records. Historical medication records may include the name, dosage, frequency, and time of medication use.

[0050] The multidimensional feature extraction module includes an intraoperative data encoder, a patient data encoder, and a receiving end smart device encoder. The intraoperative data encoder includes a temporal feature extraction layer, a statistical feature extraction layer, an anomaly detection feature extraction layer, and a multi-scale feature extraction layer. The temporal feature extraction layer uses a bidirectional LSTM network with an input dimension of 12, a hidden layer size of 64, bidirectional output, and an output dimension of 128. The statistical feature extraction layer uses a global statistical pooling design to calculate statistical data such as mean, variance, maximum, and minimum values, with an output dimension of 48. The anomaly detection feature extraction layer uses an Isolation Forest anomaly detector with 100 trees and a maximum depth of 10 for anomaly detection. The multi-scale feature extraction layer uses Daubechies wavelet transform with a decomposition layer of 4 to extract features at different scales. The medical data encoder includes a text feature extraction layer and an image feature extraction layer; the text feature extraction layer is configured as a fine-tuned model based on BERT-base, which can extract text features from the input case-related text; the image feature extraction layer is configured as a ResNet-50 pre-trained model, which can extract image features from the input medical images. The receiver-side smart device encoder includes a content receiver identity feature extraction branch and a smart terminal type feature extraction branch. The content receiver identity feature extraction branch includes an identity encoding layer, an experience feature extraction layer, and a professional domain feature extraction layer. The identity encoding layer is an embedding layer, and the experience feature extraction layer is a fully connected network with an input dimension that can be set to 2 (identity, years of work experience). The professional domain feature extraction layer is also a fully connected network with an input dimension that can be set to 3 (identity, professional domain, professional title). The smart terminal type feature extraction branch includes a terminal type encoding layer, a display capability feature encoding layer, and an interaction method feature encoding layer. The terminal type encoding layer is an embedding layer, and the display capability feature encoding layer is a fully connected network with an input dimension that is set to 3 (resolution, screen size, refresh rate). The interaction method feature encoding layer is a fully connected network with an input dimension that is set to 3 (touch, voice, gesture interaction).

[0051] The feature fusion and recommendation logic engine includes a cross-modal feature alignment module, a recommendation content planner, and a content recommendation engine based on knowledge graphs and rule bases; The cross-modal feature alignment module includes a multimodal alignment network and an adaptive feature mapping layer. The multimodal alignment network uses contrastive learning and includes three projection heads (for intraoperative perception data, patient visit data, and receiving smart devices, respectively). The adaptive feature mapping layer is based on Transformer-based adaptive mapping, which can dynamically adjust the weights of different modal features. The contrastive learning loss uses the InfoNCE loss function.

[0052] The content recommendation engine based on knowledge graphs and rule bases outputs a list of recommended content based on the rule base and graph neural networks. The medical knowledge graph contains entities such as diseases, symptoms, surgical methods, physiological data, drugs, and medical devices, as well as their relationships. The rule base contains rules such as surgical guidelines, medical consensus, and expert recommendations. The graph neural network uses a graph attention network (GAT) to process the medical knowledge graph. The recommended content planner is used to output the recommended content corresponding to each content receiver based on the fused features obtained after aligning the input cross-modal features and the recommended content list. Its architecture can be set to TransformerDecoder with 4 layers, 512 hidden layers, and a learning rate of 1e-4 to avoid overfitting.

[0053] The content generation and display enhancement module includes a content generator, a recommended content optimizer, and a display effect enhancer. The content generator includes a text generation module, an image generation module, and an audio generation module. The text generation module generates corresponding text content based on the text content generation requirements in the input recommended content, and its architecture can be a pre-trained generative model. The image generation module generates corresponding image content based on the image content generation requirements in the input recommended content, and its architecture can be a pre-trained generative model with image generation capabilities, such as StyleGAN. The audio generation module generates corresponding audio content based on the audio content generation requirements in the input recommended content, and its architecture can be a pre-trained generative model.

[0054] The architecture of the recommended content optimizer is a combination of the DQN algorithm and the Transformer's self-attention mechanism, which optimizes the content presentation and highlights important content. The DQN algorithm is used to implement reinforcement learning to optimize the content presentation, while the Transformer's self-attention mechanism highlights important content.

[0055] The display enhancement module includes an adaptive display module, a display optimization module, and an XR enhancement module. The adaptive display module is used to adjust the layout content and format based on the terminal resolution and a pre-set layout template for different terminal types. The display optimization module is used to optimize the display effect using image processing techniques (such as sharpening, contrast adjustment, etc.). The XR enhancement module is used to optimize the 3D model accuracy, interaction method, and annotation position for XR devices to improve the presentation effect under XR devices.

[0056] Figure 5This is a schematic diagram of the architecture of a dynamic monitoring and recording system for a digital operating room provided in one embodiment of this application, in which a surgical robot serves as the content distribution and control center.

[0057] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown.

[0058] The electronic device may include a processor 601 and a memory 602 storing computer program instructions.

[0059] Specifically, the processor 601 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0060] Memory 602 may include mass storage for data or instructions. For example, and not limitingly, memory 602 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where suitable, memory 602 may include removable or non-removable (or fixed) media. Where suitable, memory 602 may be internal or external to an electronic device. In a particular embodiment, memory 602 may be a non-volatile solid-state memory.

[0061] In one embodiment, memory 602 may be read-only memory (ROM). In one embodiment, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0062] The processor 601 reads and executes computer program instructions stored in the memory 602 to implement the functions of the dynamic monitoring and recording system for the digital operating room described in any of the above embodiments.

[0063] In one example, the electronic device may also include a communication interface 603 and a bus 610. For example, Figure 6 As shown, the processor 601, memory 602, and communication interface 603 are connected through bus 610 and complete communication with each other.

[0064] The communication interface 603 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0065] Bus 610 includes hardware, software, or both, that couples components of an electronic device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 610 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0066] Alternatively, embodiments of this application may be implemented using a computer-readable storage medium. This computer-readable storage medium stores computer program instructions; when executed by a processor, these computer program instructions implement the functions described in any of the above embodiments for the dynamic monitoring and recording system of a digital operating room.

[0067] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0068] The functional modules shown in the above-described block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0069] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0070] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0071] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A dynamic monitoring and recording system for a digital operating room, characterized in that, include: The surgical recording module is used to record the surgical procedure in real time based on intraoperative sensory data. The dynamic monitoring module is used to monitor the operating room environment, medical staff behavior, and patient status in real time based on intraoperative perception data, and obtain real-time monitoring results. The anomaly response module is used to respond to anomalies according to the anomaly level of the real-time monitoring results.

2. The dynamic monitoring and recording system for a digital operating room according to claim 1, characterized in that, The surgical record module is specifically used for: Surgical records are generated in real time based on intraoperative sensory data, resulting in real-time surgical record updates. These results are then used to update surgical progress, surgical scheduling plans, provide real-time surgical operation suggestions, and offer real-time surgical operation risk alerts. The intraoperative sensing data includes patient physiological data, real-time surgical video data, and surgical instrument tracking data.

3. The dynamic monitoring and recording system for a digital operating room according to claim 2, characterized in that, The surgical record module is specifically used for: Intraoperative sensory data is input into a pre-trained intelligent surgical collaborative decision-making model to obtain a real-time surgical record output by the intelligent surgical collaborative decision-making model; wherein, The intelligent surgical collaborative decision-making model includes a multimodal perception data input module, a feature fusion module, a surgical record generation module, a surgical progress and surgical scheduling prediction module, and a surgical operation suggestion and risk warning module.

4. The dynamic monitoring and recording system for a digital operating room according to claim 1, characterized in that, The surgical record module is specifically used for: Based on the real-time audio and video data and patient physiological data recorded during the operation, a surgical record is generated in real time. Based on the surgical records and preoperative surgical plan, a surgical debriefing report containing suggestions for surgical improvement is generated.

5. The dynamic monitoring and recording system for a digital operating room according to claim 1, characterized in that, The dynamic monitoring module is specifically used for: Monitoring of medical staff's behavior, operational procedures, and status; Real-time monitoring of patients' physiological parameters, monitoring of key surgical points, and monitoring of anesthesia recovery; Air quality monitoring, temperature and humidity monitoring, and equipment operation status monitoring are conducted in the operating room environment.

6. The dynamic monitoring and recording system for a digital operating room according to claim 5, characterized in that, The exception response module is specifically used for: When the real-time monitoring results are abnormal, an anomaly description is generated; Intraoperative sensing data, patient visit data, and abnormal description information are input into a pre-trained surgical abnormality response model to obtain the abnormality response scheme output by the surgical abnormality response model.

7. A dynamic monitoring and recording system for a digital operating room, characterized in that, The intelligent digital operating room also includes a surgical robot system, an information integration system, an intelligent collaborative decision-making system, an intelligent content distribution system, an intelligent comprehensive control system, and a 5G remote system; among which, Surgical robot systems are used to assist in surgical operations according to the pre-operative surgical plan and serve as the control center for digital and intelligent surgery to realize corresponding intelligent functions. The information integration system serves as the data hub of the digital operating room, enabling data flow and business collaboration from different business systems and smart terminals. The intelligent collaborative decision-making system is used to generate surgical plans based on patient medical data through intelligent preoperative planning assistance; to provide real-time intraoperative prompts based on the preoperative surgical plan; to generate surgical operation suggestions in real time during the operation through a pre-trained intelligent surgical collaborative decision-making model, and to update the surgical plan in real time based on the response results of the surgical operation suggestions; and to perform surgical effect evaluation, personalized rehabilitation plan generation, and postoperative rehabilitation assessment. The intelligent content distribution system is used to aggregate content through wired and / or wireless interfaces and share intraoperative perception data to the surgical robot system; record and intelligently analyze intraoperative perception data to generate personalized recommended content that matches each intelligent terminal associated with the digital operating room; and use the surgical robot system as the content distribution center to distribute personalized recommended content to each intelligent terminal associated with the digital operating room for interactive presentation of intraoperative content. The intelligent integrated control system is used to comprehensively control various medical devices in the digital operating room, with the surgical robot system as the control center; and to perform operation scheduling and management of the digital operating room to improve its utilization efficiency. The 5G remote system is used for live teaching demonstrations and automatically generating surgical teaching videos based on the live broadcast content; controlling surgical robots according to received remote surgical instructions to achieve remote surgical intervention; and conducting remote multidisciplinary consultations, intraoperative medical data retrieval, and remote collaboration for pre-hospital emergency care.

8. The dynamic monitoring and recording system for a digital operating room according to claim 1, characterized in that, The intelligent collaborative decision-making system is specifically used for: Based on the surgical plan information included in the surgical plan, real-time intraoperative prompts are provided when the surgery reaches the target operation. The surgical plan is generated based on a pre-trained personalized surgical plan improvement model. The input of the personalized surgical plan improvement model includes the initial surgical plan provided by the doctor and the anonymized patient medical data. The output modules of the personalized surgical plan improvement model include a surgical plan matching degree assessment module, a surgical plan improvement suggestion module, and an operation guidance and risk warning module.

9. The dynamic monitoring and recording system for a digital operating room according to claim 7 or 8, characterized in that, The intelligent collaborative decision-making system is specifically used for: Intraoperative perception data and preoperative surgical plans are input into a pre-trained intelligent surgical collaborative decision-making model to obtain real-time surgical operation suggestions output by the intelligent surgical collaborative decision-making model. The intraoperative sensing data includes patient physiological data, real-time surgical video data, and surgical instrument tracking data.

10. The dynamic monitoring and recording system for a digital operating room according to claim 1, characterized in that, The intelligent content distribution system is specifically used for: Intraoperative perception data, patient visit data, content receiver identification, and corresponding smart device type are input into the personalized recommendation content generation model to obtain personalized recommendation content generated by the model for each content receiver; wherein... The personalized recommendation content generation model includes a multi-dimensional feature extraction module, a feature fusion and recommendation logic engine, and a content generation and display enhancement module.

Images