Intelligent integrated control system for digitalized operating room

Abstract

The application provides an intelligent comprehensive control system for a digital intelligent operating room, comprising: a device comprehensive control module, configured to take a surgical robot system as a control center and comprehensively control various medical devices in the digital intelligent operating room; an operation scheduling management module, configured to take the surgical robot system as the control center and perform operation scheduling management on the digital intelligent operating room, so as to improve the use efficiency of the digital intelligent operating room; an intelligent data and decision hub module, configured to take the surgical robot system as the control center and perform surgical whole-process data recording, intraoperative intelligent decision assistance and intraoperative data intelligent analysis; and an intelligent operation guarantee module, configured to take the surgical robot system as the control center and perform surgical consumable management, device energy consumption management and operating room safety and permission management.

Description

Technical Field

[0001] This application belongs to the field of digital operating rooms, and particularly relates to an intelligent integrated control system for digital operating rooms. Background Technology

[0002] A digital integrated operating room is a medical project that integrates purification engineering and digital information technology. By integrating medical imaging, equipment control and communication technologies, it can realize functions such as real-time sharing of surgical images, centralized control of equipment, information integration and remote teaching and consultation.

[0003] In related digital operating room solutions, a separate integrated control terminal is often required to control all medical equipment in the operating room in a unified manner, which makes the operating room renovation cost high. Moreover, the improvement of digital operating rooms compared with traditional operating rooms is mainly reflected in unified control and content presentation. The overall level of intelligence is not high, and it is difficult to meet the personalized needs of doctors and patients. Summary of the Invention

[0004] This application provides an intelligent integrated control system for a digital operating room, which constructs a control and operation scheduling center for the digital operating room with a surgical robot system as the core. This eliminates the need to introduce a new control center during the digital transformation of traditional operating rooms, thereby reducing the cost of digital transformation of traditional operating rooms.

[0005] In a first aspect, embodiments of this application provide an intelligent integrated control system for a digital operating room, comprising: The equipment integrated control module is used to comprehensively control various medical devices in the digital operating room, with the surgical robot system as the control center; The operation scheduling and management module is used to manage the operation of the digital operating room with the surgical robot system as the control center, so as to improve the utilization efficiency of the digital operating room. The intelligent data and decision-making center module is used to record data throughout the entire surgical process, provide intelligent decision support during surgery, and perform intelligent analysis of intraoperative data, with the surgical robot system as the control center. The intelligent operation and support module is used to manage surgical consumables, equipment energy consumption, and operating room safety and access control, with the surgical robot system as the control center.

[0006] Optionally, the integrated control module of the equipment is specifically used for: The operating interface of the surgical robot system displays a comprehensive operating room control page, and the system controls various medical devices within the operating room based on interactive operations performed on this page; among them, The operating room integrated control page includes control areas for each medical device in the operating room.

[0007] Optionally, the integrated control module of the equipment is specifically used for: Using the surgical robot system as the integrated control center, communication connections are established according to the communication methods matched to the medical system types of each medical device, and centralized control is performed in the corresponding medical system area on the operating room's integrated control page; among which, The types of medical systems include anesthesia systems, medical imaging systems, surgical recording systems, and operating table control systems.

[0008] Optionally, the integrated control module of the equipment is specifically used for: For medical devices temporarily connected to the surgical robot system, temporary control of the devices is achieved through network communication and screen mirroring.

[0009] Optionally, the intelligent integrated control module is specifically used for: The operating interface of the surgical robot system displays a comprehensive operating room operation management page, and the operation scheduling and management of the operating room are performed based on the interactive operations on this page; among them, The operating room integrated operation management page contains the current operating room's surgery schedule.

[0010] Optionally, the intelligent integrated control module is specifically used for: In response to meeting the conditions for updating the surgical scheduling plan, the system determines the current surgical progress and updates the surgical scheduling plan in real time based on intraoperative sensing data, preoperative surgical plans and scheduling plans, and a pre-trained intelligent surgical scheduling model; among which, The conditions for updating the surgical schedule include entering the target surgical phase, performing the target surgical procedure, and receiving a surgical schedule update instruction.

[0011] Optionally, the intelligent system used in the digital operating room also includes a surgical robot system, an information integration system, an intelligent collaborative decision-making system, an intelligent content distribution system, a 5G remote system, and a dynamic monitoring and recording system; wherein, Surgical robot systems are used to assist in surgical operations according to the pre-operative surgical plan and serve as the control center for digital and intelligent surgery to realize corresponding intelligent functions. The information integration system serves as the data hub of the digital operating room, enabling data flow and business collaboration from different business systems and smart terminals. The intelligent collaborative decision-making system is used to generate surgical plans based on patient medical data through intelligent preoperative planning assistance; to provide real-time intraoperative prompts based on the preoperative surgical plan; to generate surgical operation suggestions in real time during the operation through a pre-trained intelligent surgical collaborative decision-making model, and to update the surgical plan in real time based on the response results of the surgical operation suggestions; and to perform surgical effect evaluation, personalized rehabilitation plan generation, and postoperative rehabilitation assessment. The intelligent content distribution system is used to aggregate content through wired and / or wireless interfaces and share intraoperative perception data to the surgical robot system; record and intelligently analyze intraoperative perception data to generate personalized recommended content that matches each intelligent terminal associated with the digital operating room; and use the surgical robot system as the content distribution center to distribute personalized recommended content to each intelligent terminal associated with the digital operating room for interactive presentation of intraoperative content. The 5G remote system is used for live teaching demonstrations and automatically generating surgical teaching videos based on the live broadcast content; controlling surgical robots according to received remote surgical instructions to achieve remote surgical intervention; and conducting remote multidisciplinary consultations, intraoperative medical data retrieval, and remote pre-hospital emergency collaboration. The dynamic monitoring and recording system is used to record intraoperative events and monitor the operating room environment, medical staff behavior, and patient status in real time to obtain real-time monitoring results; and to respond to abnormalities according to the level of abnormality in the real-time monitoring results.

[0012] Optionally, the intelligent collaborative decision-making system is specifically used for: Based on the surgical plan information included in the surgical plan, real-time intraoperative prompts are provided when the surgery reaches the target operation. The surgical plan is generated based on a pre-trained personalized surgical plan improvement model. The input of the personalized surgical plan improvement model includes the initial surgical plan provided by the doctor and the anonymized patient medical data. The output modules of the personalized surgical plan improvement model include a surgical plan matching degree assessment module, a surgical plan improvement suggestion module, and an operation guidance and risk warning module.

[0013] Optionally, the intelligent collaborative decision-making system is specifically used for: Intraoperative perception data and preoperative surgical plans are input into a pre-trained intelligent surgical collaborative decision-making model to obtain real-time surgical operation suggestions output by the intelligent surgical collaborative decision-making model. Intraoperative sensing data includes patient physiological data, real-time surgical audio and video data, and surgical instrument tracking data.

[0014] Optionally, the intelligent content distribution system is specifically used for: The system records and intelligently analyzes intraoperative sensory data sent from the anesthesia system, medical imaging system, surgical recording system, and operating table control system to generate personalized recommendations that match the various intelligent terminals associated with the digitalized operating room; among which, Intraoperative sensing data includes patient intraoperative physiological data, surgical record data, equipment operating status data, surgical progress data, and operating room environment data.

[0015] Secondly, embodiments of this application provide an intelligent integrated control method for a digital operating room, the intelligent method being used to implement the functions in the intelligent integrated control system for a digital operating room as described in any embodiment of the first aspect.

[0016] Thirdly, embodiments of this application provide an electronic device, which includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the functions of the intelligent integrated control system for the digital operating room.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement functions in an intelligent integrated control system for a digital operating room.

[0018] The intelligent integrated control system, method, device, and computer-readable storage medium for digital operating rooms in this application embodiment provide a control and operation scheduling center for a digital operating room with a surgical robot system at its core. This eliminates the need to introduce a new control center during the digital transformation of traditional operating rooms, reducing the cost of such transformations. Furthermore, doctors can directly perform comprehensive control of medical equipment, operation scheduling management, full-process surgical data recording, intraoperative intelligent decision support, intraoperative data intelligent analysis, surgical consumable management, equipment energy consumption management, and operating room security and access control on the surgical robot system. Compared to traditional integrated digital operating rooms, the intelligent integrated control system offers a richer intelligent experience and can meet the personalized needs of doctors and patients. Attached Figure Description

[0019] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1This is a schematic diagram of the architecture of an intelligent integrated control system for a digital operating room provided in one embodiment of this application; Figure 2 This is a schematic diagram of the integrated control page of the operating room used in the intelligent integrated control system for the digital operating room provided in one embodiment of this application; Figure 3 This is a schematic diagram of the integrated operation management page of the intelligent integrated control system for a digital operating room provided in one embodiment of this application; Figure 4 This is an overall architecture diagram of the intelligent surgical scheduling planning model used in the intelligent integrated control system for a digital operating room provided in one embodiment of this application; Figure 5 This is a schematic diagram of the architecture of the intelligent integrated control system for the intelligent operating room provided in one embodiment of this application; Figure 6 This is a schematic diagram of the architecture of an intelligent integrated control system for a digital operating room provided in one embodiment of this application, in which a surgical robot serves as the content distribution and control center. Figure 7 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0021] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0022] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0023] To address the problems of existing technologies, embodiments of this application provide an intelligent integrated control system for a digitalized operating room. The intelligent integrated control system for a digitalized operating room provided in this application embodiment will be described below. Figure 1 This is a schematic diagram of the architecture of an intelligent integrated control system for a digital operating room according to an embodiment of this application. The intelligent integrated control system for a digital operating room includes an equipment integrated control module, an operation scheduling and management module, an intelligent data and decision-making center module, and an intelligent operation support module; wherein, The equipment integrated control module is used to comprehensively control various medical devices in the digital operating room, with the surgical robot system as the control center; The operation scheduling and management module is used to manage the operation of the digital operating room with the surgical robot system as the control center, so as to improve the utilization efficiency of the digital operating room. The intelligent data and decision-making center module is used to record data throughout the entire surgical process, provide intelligent decision support during surgery, and perform intelligent analysis of intraoperative data, with the surgical robot system as the control center. The intelligent operation and support module is used to manage surgical consumables, equipment energy consumption, and operating room safety and access control, with the surgical robot system as the control center.

[0024] In some embodiments, the device integrated control module is specifically used for: The operating interface of the surgical robot system displays a comprehensive operating room control page, and the system controls various medical devices within the operating room based on interactive operations performed on this page; among them, The operating room integrated control page includes control areas for each medical device in the operating room.

[0025] Figure 2 This is a schematic diagram of the integrated control page of the operating room used in the intelligent integrated control system for the digital operating room provided in one embodiment of this application.

[0026] In some embodiments, the device integrated control module is specifically used for: Using the surgical robot system as the integrated control center, communication connections are established according to the communication methods matched to the medical system types of each medical device, and centralized control is performed in the corresponding medical system area on the operating room's integrated control page; among which, The types of medical systems include anesthesia systems, medical imaging systems, surgical recording systems, and operating table control systems.

[0027] In some embodiments, the device integrated control module is specifically used for: For medical devices temporarily connected to the surgical robot system, temporary control of the devices is achieved through network communication and screen mirroring.

[0028] In some embodiments, the intelligent integrated control module is specifically used for: The operating interface of the surgical robot system displays a comprehensive operating room operation management page, and the operation scheduling and management of the operating room are performed based on the interactive operations on this page; among them, The operating room integrated operation management page contains the current operating room's surgery schedule.

[0029] Figure 3 This is a schematic diagram of the integrated operation management page of the intelligent integrated control system for a digital operating room provided in one embodiment of this application.

[0030] In some embodiments, the intelligent integrated control module is specifically used for: In response to meeting the conditions for updating the surgical scheduling plan, the system determines the current surgical progress and updates the surgical scheduling plan in real time based on intraoperative sensing data, preoperative surgical plans and scheduling plans, and a pre-trained intelligent surgical scheduling model; among which, The conditions for updating the surgical schedule include entering the target surgical phase, performing the target surgical procedure, and receiving a surgical schedule update instruction.

[0031] Figure 4 This is an overall architecture diagram of the intelligent surgical scheduling planning model used in the intelligent integrated control system for a digital operating room, provided in one embodiment of this application.

[0032] The intelligent surgical scheduling planning model includes a multimodal feature extraction module, a current surgical progress and remaining time prediction module, and a remaining surgical dynamic scheduling module.

[0033] The patient's medical data includes historical diagnostic records, medical imaging data, historical medical test data, historical surgical data, historical medication data, and patient attribute information. The patient attribute information may include age, gender, weight, height, BMI, occupation, history of underlying diseases, and allergies. Historical diagnostic records may include diagnostic descriptions, such as "left knee arthritis." Medical imaging data may include knee X-rays, MRI scans, CT scans, etc. Historical medical test data may include complete blood count data, blood biochemistry data, and urinalysis results. Historical surgical records may include the type of surgery, surgery time, postoperative rehabilitation records, and postoperative complication records. Historical medication records may include the name, dosage, frequency, and time of medication use.

[0034] The multimodal and feature extraction module includes an intraoperative data encoder, a patient visit data encoder, and a surgical scheduling scheme encoder.

[0035] The intraoperative data encoder includes a temporal feature extraction layer, a statistical feature extraction layer, an anomaly detection feature extraction layer, and a multi-scale feature extraction layer. The temporal feature extraction layer uses a bidirectional LSTM network with an input dimension of 12, a hidden layer size of 64, bidirectional output, and an output dimension of 128. The statistical feature extraction layer uses a global statistical pooling design to calculate statistical data such as mean, variance, maximum, and minimum values, with an output dimension of 48. The anomaly detection feature extraction layer uses an Isolation Forest anomaly detector with 100 trees and a maximum depth of 10 to detect data anomalies. The multi-scale feature extraction layer uses Daubechies wavelet transform with a decomposition layer of 4 to extract features at different scales.

[0036] The medical data encoder includes a text feature extraction layer, an image feature extraction layer, and a structured data feature extraction layer. The text feature extraction layer is configured as a fine-tuned model based on BERT-base, which can extract text features from the input case-related text. The image feature extraction layer is configured as a ResNet-50 pre-trained model, which can extract image features from the input medical images.

[0037] The surgical scheduling scheme encoder is based on a graph convolutional network (GCN). The initial surgical scheduling scheme is represented as a directed graph, where nodes = surgeries and edges = dependencies. The GCN can contain two layers (with 128 hidden layers). The first layer takes into account node features (surgery type encoding, doctor ID encoding, initial surgery start time, current operating room ID, etc.) and edge features (doctor surgery interval, etc.) and outputs 128 dimensions. The second layer aggregates neighbor information and outputs 128 dimensions. Finally, it is mapped to 256 dimensions through an MLP.

[0038] The current surgical progress and remaining time prediction module includes a cross-modal feature alignment module, an end time prediction model, and a personalized parameter matching engine based on medical knowledge graphs and rule bases.

[0039] The architecture of the cross-modal feature alignment module is a cross-modal attention mechanism, which uses multi-head self-attention and contrastive learning loss InfoNCE to maximize the feature similarity of the same surgery. The number of attention heads can be set to 8, the contrastive loss temperature can be set to 0.07, the alignment loss weight can be 0.7, and the sampling strategy can be set to triple sampling.

[0040] The personalized parameter matching engine based on a medical knowledge graph and rule base includes a knowledge graph and a rule base. The knowledge graph can be embedded into an open-source medical knowledge graph using the TransR model. Nodes include diseases, surgeries, and doctor skills, while edges include treatment relationships. The embedding dimension is set to 128. The rule base contains matching relationships corresponding to personalized parameter factors, including time adjustment factors, priority correction factors, and risk level factors. The rule threshold can be set to 0.7 (i.e., when the similarity is greater than 0.7, the corresponding personalized parameter factor can be selected). The personalized parameter matching engine can match personalized parameter factors that match the current surgical situation based on the fusion features obtained after aligning the input cross-modal features. By concatenating these features, a personalized parameter vector can be obtained.

[0041] The end time prediction model is based on a Transformer architecture. The input data consists of fused features and personalized parameter vectors obtained after cross-modal feature alignment. It includes a 6-layer encoder (256 dimensions, 8 attention heads), a 512-dimensional hidden layer in the feedforward layer, a linear layer as the output layer, and the remaining time as the output data. The loss function is Huber Loss.

[0042] The remaining surgery dynamic scheduling module includes a constraint condition module, a priority sorting rule module, and a surgery dynamic scheduling algorithm. The constraints include hard constraints and soft constraints. Hard constraints include doctor availability, equipment availability, and priority constraints. Soft constraints include minimum waiting time and resource utilization. Each constraint is defined as a function, with the scheduling status as input and the violation score as output. Different constraints can use corresponding weight values ​​to calculate the sum of violation scores. The weight ratio of doctor time: equipment conflict: waiting time: resource utilization can be set to 10:10:1:2.

[0043] The priority ranking rule module can calculate priority scores based on parameters such as surgical urgency, surgical risk, and time adjustment factor, and perform weight calibration based on historical hospital data to improve the accuracy of priority score calculation.

[0044] The surgical dynamic scheduling algorithm can calculate the violation score, priority score, and initial surgical scheduling plan based on constraints. Then, it can use sorting algorithms such as greedy allocation algorithm and local optimization algorithm to perform optimal allocation under the condition of satisfying constraints and priority conditions, so as to update the initial surgical scheduling plan. The optimization objective can be set as minimizing the total violation score + minimizing the total waiting time.

[0045] Figure 5 This is a schematic diagram of the architecture of the intelligent integrated control system for the intelligent operating room provided in one embodiment of this application.

[0046] In some embodiments, the intelligent system used in the digital operating room further includes a surgical robot system, an information integration system, an intelligent collaborative decision-making system, an intelligent content distribution system, a 5G remote system, and a dynamic monitoring and recording system; wherein, Surgical robot systems are used to assist in surgical operations according to the pre-operative surgical plan and serve as the control center for digital and intelligent surgery to realize corresponding intelligent functions. The information integration system serves as the data hub of the digital operating room, enabling data flow and business collaboration from different business systems and smart terminals. The intelligent collaborative decision-making system is used to generate surgical plans based on patient medical data through intelligent preoperative planning assistance; to provide real-time intraoperative prompts based on the preoperative surgical plan; to generate surgical operation suggestions in real time during the operation through a pre-trained intelligent surgical collaborative decision-making model, and to update the surgical plan in real time based on the response results of the surgical operation suggestions; and to perform surgical effect evaluation, personalized rehabilitation plan generation, and postoperative rehabilitation assessment. The intelligent content distribution system is used to aggregate content through wired and / or wireless interfaces and share intraoperative perception data to the surgical robot system; record and intelligently analyze intraoperative perception data to generate personalized recommended content that matches each intelligent terminal associated with the digital operating room; and use the surgical robot system as the content distribution center to distribute personalized recommended content to each intelligent terminal associated with the digital operating room for interactive presentation of intraoperative content. The 5G remote system is used for live teaching demonstrations and automatically generating surgical teaching videos based on the live broadcast content; controlling surgical robots according to received remote surgical instructions to achieve remote surgical intervention; and conducting remote multidisciplinary consultations, intraoperative medical data retrieval, and remote pre-hospital emergency collaboration. The dynamic monitoring and recording system is used to record intraoperative events and monitor the operating room environment, medical staff behavior, and patient status in real time to obtain real-time monitoring results; and to respond to abnormalities according to the level of abnormality in the real-time monitoring results.

[0047] The business system can be a hospital-based business system such as an imaging business system (the data can be CT data, MRI data, etc.), and the smart terminal can be a smart terminal associated with the digital operating room, such as a smart terminal used by doctors, scrub nurses, anesthesiologists, and other personnel.

[0048] In some embodiments, the aforementioned systems used in the digital operating room can be deployed in the control equipment used by the surgical robot, so that the control equipment of the surgical robot can not only control the surgical robot, but can also serve as the central hub of the entire digital operating room for comprehensive control. Thus, when the operating room is undergoing digital transformation, the digital transformation of the operating room can be completed by introducing the surgical robot system and making corresponding adaptations, which greatly reduces the cost of digital transformation of the operating room and improves the transformation efficiency and the operating room's usage effect.

[0049] In some embodiments, the intelligent collaborative decision-making system is specifically used for: Based on the surgical plan information included in the surgical plan, real-time intraoperative prompts are provided when the surgery reaches the target operation. The surgical plan is generated based on a pre-trained personalized surgical plan improvement model. The input of the personalized surgical plan improvement model includes the initial surgical plan provided by the doctor and the anonymized patient medical data. The output modules of the personalized surgical plan improvement model include a surgical plan matching degree assessment module, a surgical plan improvement suggestion module, and an operation guidance and risk warning module.

[0050] In some embodiments, the intelligent collaborative decision-making system is specifically used for: Intraoperative perception data and preoperative surgical plans are input into a pre-trained intelligent surgical collaborative decision-making model to obtain real-time surgical operation suggestions output by the intelligent surgical collaborative decision-making model. Intraoperative sensing data includes patient physiological data, real-time surgical audio and video data, and surgical instrument tracking data.

[0051] In some embodiments, the intelligent content distribution system is specifically used for: The system records and intelligently analyzes intraoperative sensory data sent from the anesthesia system, medical imaging system, surgical recording system, and operating table control system to generate personalized recommendations that match the various intelligent terminals associated with the digitalized operating room; among which, Intraoperative sensing data includes patient intraoperative physiological data, surgical record data, equipment operating status data, surgical progress data, and operating room environment data.

[0052] In some embodiments, a personalized recommendation content generation model can be used for intelligent analysis to generate personalized recommendation content that matches each smart terminal associated with the digital operating room.

[0053] The personalized recommendation content generation model includes a multi-dimensional feature extraction module, a feature fusion and recommendation logic engine, and a content generation and display enhancement module.

[0054] The patient's medical data includes historical diagnostic records, medical imaging data, historical medical test data, historical surgical data, historical medication data, and patient attribute information. The patient attribute information may include age, gender, weight, height, BMI, occupation, history of underlying diseases, and allergies. Historical diagnostic records may include diagnostic descriptions, such as "left knee arthritis." Medical imaging data may include knee X-rays, MRI scans, CT scans, etc. Historical medical test data may include complete blood count data, blood biochemistry data, and urinalysis results. Historical surgical records may include the type of surgery, surgery time, postoperative rehabilitation records, and postoperative complication records. Historical medication records may include the name, dosage, frequency, and time of medication use.

[0055] The multidimensional feature extraction module includes an intraoperative data encoder, a patient data encoder, and a receiving end smart device encoder. The intraoperative data encoder includes a temporal feature extraction layer, a statistical feature extraction layer, an anomaly detection feature extraction layer, and a multi-scale feature extraction layer. The temporal feature extraction layer uses a bidirectional LSTM network with an input dimension of 12, a hidden layer size of 64, bidirectional output, and an output dimension of 128. The statistical feature extraction layer uses a global statistical pooling design to calculate statistical data such as mean, variance, maximum, and minimum values, with an output dimension of 48. The anomaly detection feature extraction layer uses an Isolation Forest anomaly detector with 100 trees and a maximum depth of 10 for anomaly detection. The multi-scale feature extraction layer uses Daubechies wavelet transform with a decomposition layer of 4 to extract features at different scales. The medical data encoder includes a text feature extraction layer and an image feature extraction layer; the text feature extraction layer is configured as a fine-tuned model based on BERT-base, which can extract text features from the input case-related text; the image feature extraction layer is configured as a ResNet-50 pre-trained model, which can extract image features from the input medical images. The receiver-side smart device encoder includes a content receiver identity feature extraction branch and a smart terminal type feature extraction branch. The content receiver identity feature extraction branch includes an identity encoding layer, an experience feature extraction layer, and a professional domain feature extraction layer. The identity encoding layer is an embedding layer, and the experience feature extraction layer is a fully connected network with an input dimension that can be set to 2 (identity, years of work experience). The professional domain feature extraction layer is also a fully connected network with an input dimension that can be set to 3 (identity, professional domain, professional title). The smart terminal type feature extraction branch includes a terminal type encoding layer, a display capability feature encoding layer, and an interaction method feature encoding layer. The terminal type encoding layer is an embedding layer, and the display capability feature encoding layer is a fully connected network with an input dimension that is set to 3 (resolution, screen size, refresh rate). The interaction method feature encoding layer is a fully connected network with an input dimension that is set to 3 (touch, voice, gesture interaction).

[0056] The feature fusion and recommendation logic engine includes a cross-modal feature alignment module, a recommendation content planner, and a content recommendation engine based on knowledge graphs and rule bases; The cross-modal feature alignment module includes a multimodal alignment network and an adaptive feature mapping layer. The multimodal alignment network uses contrastive learning and includes three projection heads (for intraoperative perception data, patient visit data, and receiving smart devices, respectively). The adaptive feature mapping layer is based on Transformer-based adaptive mapping, which can dynamically adjust the weights of different modal features. The contrastive learning loss uses the InfoNCE loss function.

[0057] The content recommendation engine based on knowledge graphs and rule bases outputs a list of recommended content based on the rule base and graph neural networks. The medical knowledge graph contains entities such as diseases, symptoms, surgical methods, physiological data, drugs, and medical devices, as well as their relationships. The rule base contains rules such as surgical guidelines, medical consensus, and expert recommendations. The graph neural network uses a graph attention network (GAT) to process the medical knowledge graph. The recommended content planner is used to output the recommended content corresponding to each content receiver based on the fused features obtained after aligning the input cross-modal features and the recommended content list. Its architecture can be set to TransformerDecoder with 4 layers, 512 hidden layers, and a learning rate of 1e-4 to avoid overfitting.

[0058] The content generation and display enhancement module includes a content generator, a recommended content optimizer, and a display effect enhancer. The content generator includes a text generation module, an image generation module, and an audio generation module. The text generation module generates corresponding text content based on the text content generation requirements in the input recommended content, and its architecture can be a pre-trained generative model. The image generation module generates corresponding image content based on the image content generation requirements in the input recommended content, and its architecture can be a pre-trained generative model with image generation capabilities, such as StyleGAN. The audio generation module generates corresponding audio content based on the audio content generation requirements in the input recommended content, and its architecture can be a pre-trained generative model.

[0059] The architecture of the recommended content optimizer is a combination of the DQN algorithm and the Transformer's self-attention mechanism, which optimizes the content presentation and highlights important content. The DQN algorithm is used to implement reinforcement learning to optimize the content presentation, while the Transformer's self-attention mechanism highlights important content.

[0060] The display enhancement module includes an adaptive display module, a display optimization module, and an XR enhancement module. The adaptive display module is used to adjust the layout content and format based on the terminal resolution and a pre-set layout template for different terminal types. The display optimization module is used to optimize the display effect using image processing techniques (such as sharpening, contrast adjustment, etc.). The XR enhancement module is used to optimize the 3D model accuracy, interaction method, and annotation position for XR devices to improve the presentation effect under XR devices.

[0061] Figure 6 This is a schematic diagram of the architecture of an intelligent integrated control system for a digital operating room provided in one embodiment of this application, in which a surgical robot serves as the content distribution and control center.

[0062] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown.

[0063] The electronic device may include a processor 701 and a memory 702 storing computer program instructions.

[0064] Specifically, the processor 701 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0065] Memory 702 may include mass storage for data or instructions. For example, and not limitingly, memory 702 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 702 may include removable or non-removable (or fixed) media. Where appropriate, memory 702 may be internal or external to an electronic device. In a particular embodiment, memory 702 may be a non-volatile solid-state memory.

[0066] In one embodiment, memory 702 may be read-only memory (ROM). In one embodiment, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0067] The processor 701 reads and executes computer program instructions stored in the memory 702 to implement the functions in the intelligent integrated control system for the digital operating room described in any of the above embodiments.

[0068] In one example, the electronic device may also include a communication interface 703 and a bus 710. For example, Figure 7 As shown, the processor 701, memory 702, and communication interface 703 are connected through bus 710 and complete communication with each other.

[0069] The communication interface 703 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0070] Bus 710 includes hardware, software, or both, that couples components of an electronic device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 710 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0071] Alternatively, embodiments of this application may be implemented using a computer-readable storage medium. This computer-readable storage medium stores computer program instructions; when executed by a processor, these computer program instructions implement the functions of the intelligent integrated control system for a digital operating room as described in any of the above embodiments.

[0072] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0073] The functional modules shown in the above-described block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0074] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0075] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0076] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. An intelligent integrated control system for a digital operating room, characterized in that, include: The equipment integrated control module is used to comprehensively control various medical devices in the digital operating room, with the surgical robot system as the control center; The operation scheduling and management module is used to manage the operation of the digital operating room with the surgical robot system as the control center, so as to improve the utilization efficiency of the digital operating room. The intelligent data and decision-making center module is used to record data throughout the entire surgical process, provide intelligent decision support during surgery, and perform intelligent analysis of intraoperative data, with the surgical robot system as the control center. The intelligent operation and support module is used to manage surgical consumables, equipment energy consumption, and operating room safety and access control, with the surgical robot system as the control center.

2. The intelligent integrated control system for a digital operating room according to claim 1, characterized in that, The equipment integrated control module is specifically used for: The operating interface of the surgical robot system displays a comprehensive operating room control page, and the system controls various medical devices within the operating room based on interactive operations performed on this page; among them, The operating room integrated control page includes control areas for each medical device in the operating room.

3. The intelligent integrated control system for a digital operating room according to claim 2, characterized in that, The equipment integrated control module is specifically used for: Using the surgical robot system as the integrated control center, communication connections are established according to the communication methods matched to the medical system types of each medical device, and centralized control is performed in the corresponding medical system area on the operating room's integrated control page; among which, The types of medical systems include anesthesia systems, medical imaging systems, surgical recording systems, and operating table control systems.

4. The intelligent integrated control system for a digital operating room according to claim 2 or 3, characterized in that, The equipment integrated control module is specifically used for: For medical devices temporarily connected to the surgical robot system, temporary control of the devices is achieved through network communication and screen mirroring.

5. The intelligent integrated control system for a digital operating room according to claim 1 or 2, characterized in that, The intelligent integrated control module is specifically used for: The operating interface of the surgical robot system displays a comprehensive operating room operation management page, and the operation scheduling and management of the operating room are performed based on the interactive operations on this page; among them, The operating room integrated operation management page contains the current operating room's surgery schedule.

6. The intelligent integrated control system for a digital operating room according to claim 5, characterized in that, The intelligent integrated control module is specifically used for: In response to meeting the conditions for updating the surgical scheduling plan, the system determines the current surgical progress and updates the surgical scheduling plan in real time based on intraoperative sensing data, preoperative surgical plans and scheduling plans, and a pre-trained intelligent surgical scheduling model; among which, The conditions for updating the surgical schedule include entering the target surgical phase, performing the target surgical procedure, and receiving a surgical schedule update instruction.

7. The intelligent integrated control system for a digital operating room according to claim 1, characterized in that, The intelligent digital operating room also includes a surgical robot system, an information integration system, an intelligent collaborative decision-making system, an intelligent content distribution system, a 5G remote system, and a dynamic monitoring and recording system. Surgical robot systems are used to assist in surgical operations according to the pre-operative surgical plan and serve as the control center for digital and intelligent surgery to realize corresponding intelligent functions. The information integration system serves as the data hub of the digital operating room, enabling data flow and business collaboration from different business systems and smart terminals. The intelligent collaborative decision-making system is used to generate surgical plans based on patient medical data through intelligent preoperative planning assistance; to provide real-time intraoperative prompts based on the preoperative surgical plan; to generate surgical operation suggestions in real time during the operation through a pre-trained intelligent surgical collaborative decision-making model, and to update the surgical plan in real time based on the response results of the surgical operation suggestions; and to perform surgical effect evaluation, personalized rehabilitation plan generation, and postoperative rehabilitation assessment. The intelligent content distribution system is used to aggregate content through wired and / or wireless interfaces and share intraoperative perception data to the surgical robot system; record and intelligently analyze intraoperative perception data to generate personalized recommended content that matches each intelligent terminal associated with the digital operating room; and use the surgical robot system as the content distribution center to distribute personalized recommended content to each intelligent terminal associated with the digital operating room for interactive presentation of intraoperative content. The 5G remote system is used for live teaching demonstrations and automatically generating surgical teaching videos based on the live broadcast content; controlling surgical robots according to received remote surgical instructions to achieve remote surgical intervention; and conducting remote multidisciplinary consultations, intraoperative medical data retrieval, and remote pre-hospital emergency collaboration. The dynamic monitoring and recording system is used to record intraoperative events and monitor the operating room environment, medical staff behavior, and patient status in real time to obtain real-time monitoring results; and to respond to abnormalities according to the level of abnormality in the real-time monitoring results.

8. The intelligent integrated control system for a digital operating room according to claim 7, characterized in that, The intelligent collaborative decision-making system is specifically used for: Based on the surgical plan information included in the surgical plan, real-time intraoperative prompts are provided when the surgery reaches the target operation. The surgical plan is generated based on a pre-trained personalized surgical plan improvement model. The input of the personalized surgical plan improvement model includes the initial surgical plan provided by the doctor and the anonymized patient medical data. The output modules of the personalized surgical plan improvement model include a surgical plan matching degree assessment module, a surgical plan improvement suggestion module, and an operation guidance and risk warning module.

9. The intelligent integrated control system for a digital operating room according to claim 7 or 8, characterized in that, The intelligent collaborative decision-making system is specifically used for: Intraoperative perception data and preoperative surgical plans are input into a pre-trained intelligent surgical collaborative decision-making model to obtain real-time surgical operation suggestions output by the intelligent surgical collaborative decision-making model. Intraoperative sensing data includes patient physiological data, real-time surgical audio and video data, and surgical instrument tracking data.

10. The intelligent integrated control system for a digital operating room according to claim 9, characterized in that, The intelligent content distribution system is specifically used for: The system records and intelligently analyzes intraoperative sensory data sent from the anesthesia system, medical imaging system, surgical recording system, and operating table control system to generate personalized recommendations that match the various intelligent terminals associated with the digitalized operating room; among which, Intraoperative sensing data includes patient intraoperative physiological data, surgical record data, equipment operating status data, surgical progress data, and operating room environment data.

Images