Operating room management system, operating room management method and computer equipment

Abstract

The invention relates to an operating room management system, an operating room management method and computer equipment. The system comprises an image acquisition device and a management device. The image acquisition equipment comprises a panoramic image acquisition device, an operation bed image acquisition device and an operation field image acquisition device; the image acquisition equipment is used for acquiring scene images of the operating room; the scene image comprises an operation behavior image and an operation environment image, and sending the scene image to the management equipment; and the management equipment is used for performing operation behavior management on the scene image to obtain a management result. By adopting the method, the operating room behavior management accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of operating room management, and in particular to an operating room management system, an operating room management method and computer equipment. Background Art

[0002] With the development of medical technology, operating room management technology has emerged. Doctors can not only obtain a large amount of patient-related information in real time and conveniently through operating room management technology, improve the efficiency and success rate of surgery, but also conduct scientific research and surgical observation and teaching based on real-time information in the operating room. In order to optimize operating room management technology, operating room behavior can be managed.

[0003] However, the traditional operating room behavior management method has the problem of inaccurate operating room behavior management. Summary of the invention

[0004] Based on this, it is necessary to provide an operating room management system, an operating room management method and a computer device that can improve the accuracy of operating room behavior management in response to the above-mentioned technical problems.

[0005] In a first aspect, the present application provides an operating room management system, the system comprising an image acquisition device and a management device; the image acquisition device comprises a panoramic image acquisition device, an operating table image acquisition device and an operating field image acquisition device;

[0006] The image acquisition device is used to acquire scene images of the operating room; the scene images include surgical behavior images and surgical environment images, and the scene images are sent to the management device;

[0007] The management device is used to manage the surgical behavior of the scene image and obtain a management result.

[0008] In one of the embodiments, the management device is specifically used to identify surgical behaviors on the scene image, obtain behavior information of each object in the operating room, and manage the behavior information of each object according to preset behavior standards to obtain the management result.

[0009] In one of the embodiments, the management device is specifically used to manage the behavior information of each of the objects according to the preset behavior standard, and when it is determined that the behavior information of the object does not meet the preset behavior standard, output a management result including warning information.

[0010] In one embodiment, the management device is further used to display the behavior information of each of the objects, receive an operation instruction generated by a user terminal based on the behavior information of each of the objects, and perform surgical behavior recognition based on a new scene image captured by the image acquisition device according to the display area indicated by the operation instruction to obtain the behavior information of each object in the operating room;

[0011] The new scene image is a scene image re-captured by the image acquisition device according to the operation instruction sent by the user terminal; the operation instruction is used to instruct the management device to perform analysis based on the display area, and to instruct the image acquisition device to capture the scene image of the display area.

[0012] In one of the embodiments, the management device is further used to construct an operating room process model according to the scene image, and determine each surgical process parameter in the operating room process model.

[0013] In one of the embodiments, the management device is further used to perform statistical analysis based on each surgical process parameter in the operating room process model and each historical surgical process parameter in multiple historical operating room process models to obtain and display statistical results.

[0014] In one of the embodiments, the panoramic image acquisition device is used to acquire surgical environment information in the operating room; the operating table image acquisition device is used to acquire environmental information of the area near the operating table in the operating room; and the surgical field image acquisition device is used to acquire surgical behavior information of personnel in the operating room.

[0015] In one embodiment, the operating room management system also includes: other operating room equipment; the other operating room equipment includes a user terminal, an operating table, surgical instruments, an operating light and an anesthesia cart.

[0016] In a second aspect, the present application further provides an operating room management method, which is applied to an operating room management system in any one of the embodiments of the first aspect above; the method comprises:

[0017] The management device performs surgical behavior management on the scene image of the operating room acquired by the image acquisition device to obtain a management result; the scene image includes a surgical behavior image and a surgical environment image.

[0018] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method in any one of the embodiments of the second aspect are implemented.

[0019] The above-mentioned operating room management system, operating room management method and computer equipment, the operating room management system includes an image acquisition device and a management device; the image acquisition device includes a panoramic image acquisition device, an operating table image acquisition device and an operating field image acquisition device; the image acquisition device is used to acquire scene images of the operating room; the scene images include surgical behavior images and surgical environment images, and the scene images are sent to the management device; the management device is used to perform surgical behavior management on the scene images to obtain management results. Since the image acquisition device includes a panoramic image acquisition device, an operating table image acquisition device and an operating field image acquisition device, the present application can acquire all-round scene images in the operating room through the panoramic image acquisition device, the operating table image acquisition device and the operating field image acquisition device, thereby obtaining relatively complete surgical behavior images and surgical environment images including different perspectives. Therefore, the management device can perform surgical behavior management on relatively complete surgical behavior images and surgical environment images from different perspectives to obtain more accurate management results. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A schematic diagram of the structure of an operating room management system in one embodiment;

[0022] Figure 2 A schematic diagram of the structure of an operating room management system including other equipment in an operating room in one embodiment;

[0023] Figure 3 A schematic diagram of a perioperative workflow in an operating room management system in one embodiment;

[0024] Figure 4 A schematic diagram of a process of operating room management through an operating room management system in one embodiment;

[0025] Figure 5 is a schematic diagram of a visual interface of a user terminal in one embodiment;

[0026] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.

[0029] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0030] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0031] A successful operation is inseparable from the precise cooperation of the medical team during the perioperative period. In order to improve the cooperation and tacit understanding of the medical team, the medical staff of the medical team usually need long-term practice and accumulation and summary of practical operation experience. However, this will increase the training time and money cost of the medical team, and on the other hand, it will reduce the scheduling efficiency of the operating room and bring challenges to the assessment of medical staff. With the development of medical technology, operating room management technology has emerged. Doctors can not only obtain a large amount of patient-related information in real time and conveniently through operating room management technology, improve the efficiency and success rate of surgery; but also conduct scientific research and surgical observation and teaching based on real-time information in the operating room, thereby improving the cooperation and tacit understanding of the medical team. In order to optimize the operating room management technology, the operating room behavior can be managed.

[0032] The traditional operating room behavior management method is to reconstruct the operating room space and surgical process based on a single data source and / or information source, and then manage the operating room behavior according to the reconstruction results. For example, the traditional method can only evaluate the quality of the surgical results from the perspective of the duration of the operation, but in reality, a long operation may be due to an emergency, and if the emergency is successfully resolved with the cooperation of the medical team, the result of the operation should be successful.

[0033] However, since the information generated during the operation is generally diverse and the representativeness of a single data source and / or information source is limited, it is impossible to reconstruct the complete surgical process based on a single piece of information. Therefore, it is impossible to conduct a comprehensive analysis of data from different angles and types based on only one or several specific data indicators. Furthermore, the traditional operating room behavior management method performs operating room behavior management based on the reconstruction results of incomplete surgical processes, which leads to the problem of inaccurate operating room behavior management.

[0034] In one embodiment, an operating room management system is provided, the operating room management system includes an image acquisition device and a management device; the image acquisition device includes a panoramic image acquisition device, an operating table image acquisition device and an operating field image acquisition device;

[0035] An image acquisition device is used to acquire scene images of the operating room; the scene images include surgical behavior images and surgical environment images, and the scene images are sent to a management device;

[0036] The management device is used to manage the surgical behavior of the scene image and obtain the management result.

[0037] In the present application embodiment, Figure 1 As shown, Figure 1 : is a structural diagram of an operating room management system in an embodiment. The operating room management system includes an image acquisition device 11 and a management device 12, and the image acquisition device 11 includes a panoramic image acquisition device 111, an operating table image acquisition device 112, and an operating field image acquisition device 113. The image acquisition device 11 is connected to the management device 12, that is, it can be understood that the panoramic image acquisition device 111, the operating table image acquisition device 112, and the operating field image acquisition device 113 are all connected to the management device 12. The image acquisition device 11 is used to acquire scene images of the operating room and send the scene images to the management device, that is, the scene images of the operating room can be acquired by the panoramic image acquisition device 111, the operating table image acquisition device 112, and the operating field image acquisition device 113, and the scene images can be sent to the management device.

[0038] The management device 12 is used to perform surgical behavior management on the scene image to obtain a management result. Exemplarily, after the management device receives the scene image sent by the image acquisition device, the management device may first use a segmentation model to segment each object in the scene image to obtain a segmentation result of each object. Among them, each object may be any person or object in the operating room. In the embodiment of the present application, corresponding weights may also be set in advance for each object. For example, the weight corresponding to the doctor may be set to the highest, indicating that the doctor's behavior information is the most concerned; the weight corresponding to the patient may be set to the lowest, indicating that the patient is in an anesthetized state during the operation, and therefore, the patient's behavior information is less concerned. Thus, the management device may determine the behavior information of each object in the scene image in order from high to low according to the segmentation results of each object, and perform surgical behavior management based on the behavior information of each object to obtain a management result. Optionally, the management device may compare the behavior information of each object with a preset behavior standard set in advance, and perform surgical behavior management according to the comparison result; or, the management device may also perform surgical behavior management on the behavior information of each object in response to a management instruction generated by a user triggering a preset interface. Of course, the embodiment of the present application does not limit the method of surgical behavior management.

[0039] Among them, the scene image includes a surgical behavior image and a surgical environment image. The surgical behavior image may include but is not limited to the behavioral information of medical staff and patients during the operation. The surgical environment image may include but is not limited to the environmental information of each person and equipment in the operating room, wherein each person in the operating room includes medical staff and patients. In addition, the scene image of the operating room may also include instrument data with ports in the operating room, relevant data of the microphone in the operating room, etc. Based on this, the types of data of the scene image collected by this application can cover images, sounds, patient signs, the operation status of instruments or equipment, etc. Therefore, the sources and types of data collected by this application are relatively rich, which provides the basic data conditions for comprehensive management and analysis of surgical behaviors from multiple angles; and when the number of image acquisition devices and the number of observation positions are increased, the proportion of the time when the field of view is blocked to the total operation time can be reduced, thereby improving the comprehensiveness of the collected data and the accuracy of surgical behavior management and analysis.

[0040] Among them, the management result refers to the result obtained after the surgical behavior management and analysis of the scene image of the operating room. Optionally, the management device 12 may have its own user terminal, so that after the surgical behavior management and analysis of the scene image of the operating room, the management result can be displayed through the built-in user terminal. Alternatively, the management device 12 may not carry a user terminal, so that after the surgical behavior management and analysis of the scene image of the operating room, the management result can be saved in the hospital's storage device or server or cloud; when the management result needs to be displayed, the management device can send the management result to the user terminal connected to the management device in real time or at a fixed time, and display the management result on the user terminal. Among them, the user terminal can be used not only for data visualization display, but also for receiving and / or issuing instructions. Exemplarily, the user terminal may be an independent mobile electronic device, which may include but not be limited to a smart phone, a tablet computer, a smart watch, a remote control, a brain-computer interface, an AR (Augmented Reality) display device, etc.; alternatively, the user terminal may be an electronic device embedded in the operating room, which may include but not be limited to a display, a touch screen, a keyboard, a microphone, a central processing unit, a graphics processing unit, a data storage device, etc.

[0041] In one embodiment, the panoramic image acquisition device 111 is used to acquire surgical environment information in the operating room, that is, the panoramic image acquisition device 111 can observe and record the position information and movement information of each person and equipment in the operating room (including the ceiling of the operating room) from multiple installation locations of the panoramic image acquisition device 111, and form continuous video recording data with time axis marks; the operating table image acquisition device 112 is used to acquire environmental information of the area near the operating table in the operating room, that is, multiple operating table image acquisition devices 112 installed in the center of the cantilever tower and the shadowless lamp can be used to acquire multi-view video data of the operating table field of view; the surgical field image acquisition device 113 is used to acquire surgical behavior information of personnel in the operating room, that is, the surgical field image acquisition device 113 can acquire surgical operation behaviors and instrument usage in the surgical field, etc. The surgical field image acquisition device 113 may include but is not limited to surgical endoscopes, laparoscopy, surgical robots, microscopes and other equipment operated by doctors during surgery. It should be noted that before acquiring the scene image, the time axis marks of the panoramic image acquisition device 111, the operating table image acquisition device 112 and the surgical field image acquisition device 113 need to be synchronized to ensure the accuracy of the time information corresponding to the scene image.

[0042] In the above-mentioned operating room management system, the operating room management system includes an image acquisition device and a management device; the image acquisition device includes a panoramic image acquisition device, an operating table image acquisition device and an operating field image acquisition device; the image acquisition device is used to acquire scene images of the operating room; the scene images include surgical behavior images and surgical environment images, and the scene images are sent to the management device; the management device is used to perform surgical behavior management on the scene images to obtain management results. Since the image acquisition device includes a panoramic image acquisition device, an operating table image acquisition device and an operating field image acquisition device, the present application can acquire all-round scene images in the operating room through the panoramic image acquisition device, the operating table image acquisition device and the operating field image acquisition device, thereby obtaining relatively complete surgical behavior images and surgical environment images from different perspectives. Therefore, the management device can perform surgical behavior management on relatively complete surgical behavior images and surgical environment images from different perspectives to obtain more accurate management results.

[0043] In the above embodiments, a management device is involved, which is used to manage the surgical behavior of the scene image and obtain the management result. The specific functions of the management device are introduced below. In one embodiment, the management device is specifically used to identify the surgical behavior of the scene image, obtain the behavior information of each object in the operating room, and manage the behavior information of each object according to the preset behavior standard to obtain the management result.

[0044] In an embodiment of the present application, when the panoramic image acquisition device detects that the patient's preoperative preparation is completed and pushed into the operating room, the operating room management system will automatically turn on the operating table image acquisition device, the surgical field image acquisition device and the management device, so that the panoramic image acquisition device, the operating table image acquisition device and the surgical field image acquisition device can collect scene images of the operating room from multiple angles and send the scene images to the management device. The management device can use a preset artificial intelligence algorithm to identify surgical behaviors on the scene images, obtain the behavior information of each object in the operating room, and record or mark the real-time spatial position of each object in the operating room. Among them, the objects in the operating room may include but are not limited to patients, surgical equipment, operating tables, equipment trolleys, medical staff in the operating room, etc., and the behavior information of each object may include but is not limited to the location information and / or action information of each object, etc.

[0045] Among them, the preset artificial intelligence algorithm may include but is not limited to CNN (Convolutional Neural Networks) algorithm, LSTM (Long Short-Term Memory) algorithm, etc. It should be noted that in order to improve the accuracy of surgical behavior recognition results, different artificial intelligence algorithms need to be used for surgical behavior recognition for different objects. For example, the CNN algorithm can be used to extract the spatial features in the operating room, and the LSTM algorithm can be used to extract the temporal features of the previous and next frames in the timeline. Of course, the embodiment of the present application does not limit the correspondence between the objects in the operating room and the artificial intelligence algorithm.

[0046] Thus, the management device can determine whether the behavior information of each object in the operating room meets the preset behavior standard based on the preset behavior standard, so as to manage the behavior information of each object and obtain a management result. Optionally, the preset behavior standard can be pre-set according to the execution standard of the hospital or operation; or, the preset behavior standard can also be customized by the medical staff on the user terminal. In this case, the preset behavior standard can be changed during the operation according to the focus of the medical staff. Of course, the embodiment of the present application does not limit the setting method of the preset behavior standard.

[0047] In one of the embodiments, the management device is specifically used to manage the behavior information of each object according to a preset behavior standard, and when it is determined that the behavior information of the object does not meet the preset behavior standard, output a management result including warning information.

[0048] Exemplarily, the management device can determine whether the behavior information of each object in the operating room meets the preset behavior standard based on the preset behavior standard, so as to manage the behavior information of each object and obtain a management result. If it is determined that the behavior information of the object in the operating room meets the preset behavior standard, the management device can output the management result, and the management result can be a scene image with the real-time spatial position of each object in the operating room marked without any other prompts; if it is determined that the behavior information of the object in the operating room does not meet the preset behavior standard, the management device can output the management result, and the management result can include not only the scene image with the real-time spatial position of each object in the operating room marked, but also early warning information. Among them, the early warning information is information used to prompt that the behavior information of the object in the operating room does not meet the preset behavior standard.

[0049] In this embodiment, the management device is specifically used to identify surgical behaviors on scene images, obtain behavioral information of each object in the operating room, and manage the behavioral information of each object according to preset behavioral standards to obtain management results. Since the present application can collect relatively complete scene images including different perspectives, the management device can identify surgical behaviors on relatively complete scene images from different perspectives, obtain relatively accurate behavioral information of each object in the operating room, and thus manage relatively accurate behavioral information of each object according to preset behavioral standards to obtain relatively accurate management results.

[0050] In the above embodiment, a management device is involved, which is specifically used to identify surgical behaviors in scene images, obtain behavior information of each object in the operating room, and manage the behavior information of each object according to preset behavior standards to obtain management results. Another function of the management device is introduced below.

[0051] In one embodiment, the management device is further used to display the behavior information of each object, receive the operation instruction generated by the user terminal based on the behavior information of each object, and perform surgical behavior recognition based on the new scene image acquired by the image acquisition device according to the display area indicated by the operation instruction to obtain the behavior information of each object in the operating room;

[0052] The new scene image is a scene image re-captured by the image acquisition device according to the operation instruction sent by the user terminal; the operation instruction is used to instruct the management device to perform analysis based on the display area, and to instruct the image acquisition device to capture the scene image of the display area.

[0053] In an embodiment of the present application, if the management device does not carry a user terminal, and the operating room management system also includes a user terminal, and the management device is connected to the user terminal, then, after obtaining the behavior information of each object in the operating room, the management device can send the behavior information of each object to the user terminal, and display the behavior information of each object on the user terminal. Thus, the user terminal can trigger the generation of operation instructions based on the behavior information of each object, and send the operation instructions to the management device and the image acquisition device. The image acquisition device can receive the operation instructions generated by the user terminal based on the behavior information of each object, and re-capture new scene images according to the display area indicated by the operation instructions, and send the new scene images to the management device. Furthermore, the management device can receive the operation instructions and the new scene images, and use a preset artificial intelligence algorithm to identify surgical behaviors on the new scene images according to the display area indicated by the operation instructions to obtain the behavior information of each object in the operating room.

[0054] Among them, the display area refers to the area within the operating room defined or set by the operation instruction. The new scene image is a scene image that the image acquisition device re-captures according to the display area indicated by the operation instruction sent by the user terminal. The operation instruction can not only be used to instruct the management device to perform surgical behavior recognition and analysis on the new scene image based on the display area indicated by the operation instruction, but also can be used to instruct the image acquisition device to re-capture the scene image within the display area indicated by the operation instruction. Optionally, the triggering generation process of the operation instruction can be achieved through the key / click operation of the medical staff, or it can also be achieved through the voice recognition operation or gesture recognition operation of the medical staff. Of course, the embodiment of the present application does not limit the triggering generation process of the operation instruction.

[0055] In this embodiment, the user terminal can trigger the generation of operation instructions based on the behavior information of each object during the operation. Thus, the image acquisition device can re-capture new scene images according to the operation instructions of the user terminal, and the management device can re-recognize surgical behavior based on the new scene images captured by the image acquisition device according to the display area indicated by the operation instructions, thereby obtaining more accurate behavior information of each object in the operating room that is updated in real time.

[0056] Optionally, another function of the management device is provided below. In one embodiment, the management device is further used to construct an operating room process model according to the scene image and determine each surgical process parameter in the operating room process model.

[0057] Among them, the operating room process model can be a digital twin model, and the operating room process model is a three-dimensional reconstruction of the entire process of the operation in the operating room. The various surgical process parameters in the operating room process model may include but are not limited to the time and number of times the operating room door is opened during the operation, the number of bleeding times during the operation, the displacement distance and displacement time ratio of medical staff in the operating room, the displacement distance and displacement time ratio of patients in the operating room, the proportion of surgical instrument transfer time, the success rate of surgical instrument transfer, the number of repetitions of the same instructions of the medical team, the relevant parameters of the energy device, the duration of the operation, the discharge time of the postoperative patient or the complications of the postoperative patient, etc. For the number of repetitions of the same instructions of the medical team, preferably, it can be the internal instructions of the medical team collected by voice. The relevant parameters of the energy device may include but are not limited to the firing duration or energy type, etc.

[0058] In the embodiment of the present application, since the management device can save the scene images obtained during and after the operation and the behavior information of each object in the operating room in the hospital's storage device, server or cloud, after the operation, the management device can obtain the scene images corresponding to the operation and the behavior information of each object in the operating room from the hospital information system (HIS) and / or picture archiving and communication system (PACS) in the hospital, and construct an operating room process model based on the scene images and the behavior information of each object in the operating room, and determine each surgical process parameter in the operating room process model.

[0059] In this embodiment, the management device is also used to construct an operating room process model based on the scene image and determine the parameters of each surgical process in the operating room process model. The entire surgical process in the operating room can be reconstructed after surgery, so that the doctor can quantitatively review the entire surgical process at any time after the surgery, and the entire surgical process can also be used as a material for academic exchanges and team self-improvement.

[0060] Optionally, another function of the management device is introduced below. In one embodiment, the management device is also used to perform statistical analysis based on each surgical process parameter in the operating room process model and each historical surgical process parameter in multiple historical operating room process models to obtain and display statistical results.

[0061] In an embodiment of the present application, the management device can also obtain each surgical process parameter in the operating room process model and each historical surgical process parameter in multiple historical operating room process models, and perform statistical analysis based on each surgical process parameter in the operating room process model and each historical surgical process parameter in multiple historical operating room process models, and obtain and display statistical results. Exemplarily, the management device can perform horizontal comparison and statistics on the surgical process parameters of multiple surgeries of the same type, so as to obtain the correlation analysis results between the surgical process parameters of multiple surgeries of the same type. Among them, the type of surgery is uniformly defined by the Hospital Information System (HIS) within the hospital. The historical operating room process model refers to the operating room process model constructed before the current operating room process model, and each historical surgical process parameter refers to the surgical process parameter of the historical operating room process model. The statistical results can be presented in the form of pictures or tables.

[0062] For example, if in the record of the number of repetitions of the same instruction by the medical team, the number of hemostasis operations is large, the operation time is long, and the success rate of the transfer of surgical instruments is low, then the statistical result of the operation is poor, and a chart of comparative statistics can be displayed. Of course, the medical team can customize the surgical process parameters involved in horizontal comparison and statistics according to the actual surgical situation, that is, the embodiment of the present application does not limit the surgical process parameters involved in horizontal comparison and statistics.

[0063] Based on this, as the amount of data of the stored operating room process model (i.e., surgical records) increases, the medical team can select a surgical process parameter of interest, and the operating room management system will statistically analyze the surgical process parameters that the surgical process parameter of interest will affect based on all surgical records. Thus, the medical team can analyze the impact of the surgical operation on the surgical process parameters (including surgical effects) based on the relationship between the surgical operation and the surgical process parameters, so as to help the medical team improve the surgical process. In addition, the management device can also predict the risks and advantages of the surgical operation in the same type of historical surgery based on the real-time identified surgical operation during the operation, so that not only can the medical team be advised during the operation, but the medical team can also screen the surgical operation behaviors that are suitable for their own surgical habits and work requirements based on the risks and advantages of the surgical operation, and establish a surgical core workflow that is more suitable for their own medical team on this basis. Of course, the core surgical workflow can be generated into a workflow file through user personalized settings and stored in a cloud server or a local server. If the workflow file is stored on a cloud server, the medical team can use the workflow file in multiple locations, so that each user can log in to different medical teams and view their personalized workflow files.

[0064] In this embodiment, the management device is also used to perform statistical analysis based on each surgical process parameter in the operating room process model and each historical surgical process parameter in multiple historical operating room process models, and obtain and display statistical results. Therefore, after collecting multiple operating room process models, it is possible to statistically analyze the impact of surgical operations on surgical results based on the surgical process parameters in multiple operating room process models, so that the risks and advantages of the surgical operation can be predicted in real time based on the visually recognized surgical operation during the operation, and provide auxiliary suggestions for the medical team.

[0065] In one embodiment, the operating room management system also includes: other operating room equipment; other operating room equipment includes user terminals, operating tables, surgical instruments, operating lights and anesthesia carts.

[0066] Among them, the operating room management system also includes other equipment in the operating room. Other equipment in the operating room may include electronic equipment and devices in the operating room, such as user terminals, operating tables, surgical instruments, operating lights and other lights, laminar flow purification devices, voice communication devices inside and outside the operating room, electrically controllable operating room doors, air conditioners, equipment power supplies, doors, shadowless lamps, medicine cabinets or anesthesia carts, etc. Surgical instruments may include but are not limited to intraoperative magnetic resonance imaging equipment or energy platforms, etc., and energy platforms may include but are not limited to electric knives, ultrasonic scalpels or staplers, etc. For example, Figure 2 As shown, Figure 2 1 is a schematic diagram of the structure of an operating room management system including other equipment in an operating room in an embodiment. The operating room management system includes an image acquisition device 11, a management device 12, other equipment 13, two user terminals 14, a patient 15, medical staff 161 in the operating room, and medical staff 162 outside the operating room. The image acquisition device 11 includes a panoramic image acquisition device 111, an operating table image acquisition device 112, and an surgical field image acquisition device 113. Among them, the image acquisition device 11, the management device 12, other equipment 13, one user terminal 14, the patient 15, and the medical staff 161 in the operating room are located in the operating room, and the other user terminal 14 and the medical staff 162 outside the operating room are located outside the operating room. Other equipment 13 includes other equipment in the operating room except the user terminal.

[0067] Combination Figure 2 As shown, in the operating room, the image acquisition device 11 can collect and integrate the scene data of the patient 15 and the medical staff 161 in the operating room, and send the scene data / images of the patient 15 and the medical staff 161 in the operating room to the management device 12, and other devices 13 can send the operation data of other devices 13 to the management device 12. Therefore, the management device 12 can manage and analyze the above scene data / images and operation data, obtain management results, and synchronously transmit the management results to the user terminals 14 in and outside the operating room, so that the medical staff 161 in the operating room and the medical staff 162 outside the operating room can view the management results. Afterwards, the medical staff 161 in the operating room and the medical staff 162 outside the operating room can generate operation instructions based on the management results and send the operation instructions to the user terminal 14. Thus, the user terminal 14 can send the operation instructions to the image acquisition device 11, the management device 12, and other devices 13 to instruct the image acquisition device 11 to re-capture new scene images based on the operation instructions, instruct other devices 13 to re-capture new operation data based on the operation instructions, and instruct the management device 12 to perform surgical behavior recognition based on the operation instructions and the new scene images captured by the image acquisition device, so as to obtain the behavior information of each object in the operating room, thereby obtaining new management results.

[0068] In one embodiment, an operating room management method is provided, which is described by taking the method applied to an operating room management system of any of the above embodiments as an example, and includes the following steps:

[0069] The management device performs surgical behavior management on the scene images of the operating room acquired by the image acquisition device to obtain management results; the scene images include surgical behavior images and surgical environment images.

[0070] The scene image includes a surgical behavior image and a surgical environment image. The specific implementation steps and beneficial effects of the surgical behavior management of the scene image of the operating room collected by the image acquisition device through the management device to obtain the management result can refer to the above embodiment, which will not be repeated here.

[0071] Combining all the above embodiments, Figure 3 As shown, Figure 3 The following is a schematic diagram of the workflow of the perioperative period in an operating room management system in one embodiment. The entire perioperative period includes the preoperative stage, the intraoperative stage, and the postoperative stage. Exemplarily, the workflow of the perioperative period in the operating room management system is as follows:

[0072] Combination Figure 3 As shown, in the preoperative stage, before the patient starts the operation, S301, the medical staff 162 outside the operating room can input the patient's operation schedule into the user terminal 14, and send the patient's operation schedule to the acquisition device 11, the management device 12, and other devices 13 through the user terminal 14. At this time, S302, the acquisition device 11 can pre-start the panoramic image acquisition device 111, the operating table image acquisition device 112 and the surgical field image acquisition device 113, and synchronize the time axes of the panoramic image acquisition device 111, the operating table image acquisition device 112 and the surgical field image acquisition device 113 by means of timestamp; S303, the management device 12 can obtain the identification of the patient in the surgical schedule, S304, and according to the identification of the patient in the surgical schedule, obtain the patient's preoperative imaging data and preoperative planning data from the PACS system and HIS system of the hospital where the patient is located, and process the preoperative imaging data and preoperative planning data into a format that can be comprehensively displayed and / or split-screen displayed in the operating room; other devices 13 can start preheating to prepare for the operation; S305, the user terminals 14 inside and outside the operating room can synchronously timestamp the operation instructions and sound signals to facilitate subsequent synchronization with the time axis of the video data; if necessary, S306, the management device 12 can also synchronize the time axes of different data and signals to ensure the synchronization of the time axis in the operating room management system. After the above operations are completed, the operating room management system may enter a standby state, and after the standby state, the operating room management system may only turn on the panoramic image acquisition device.

[0073] Combination Figure 3As shown, S307, when the panoramic image acquisition device detects that the patient's preoperative preparation is complete and is pushed into the operating room, it indicates that the intraoperative stage is being carried out (i.e., the operation start instruction is determined). At this time, the operating room management system will automatically turn on the operating table image acquisition device, the surgical field image acquisition device and the management device, so that, S308, the panoramic image acquisition device, the operating table image acquisition device and the surgical field image acquisition device can collect scene images of the operating room from multiple angles and send the scene images to the management device. Other devices 13 can send the operation data of other devices 13 to the management device 12. S309, the management device can record the start time of the operation, S310, and use a preset artificial intelligence algorithm to identify the surgical behavior of the scene image and the operation data, obtain the behavior information of each object in the operating room, and record or mark the real-time spatial position of each object in the operating room, and send the behavior information of each object in the operating room to the user terminal. Among them, the behavior information of each object in the operating room includes the real-time spatial position of each object in the operating room. S311, the user terminal can display the behavior information of each object in the operating room in real time.

[0074] Combination Figure 3 As shown, thus, S312, the user terminal can trigger the generation of operation instructions based on the behavior information of each object, and send the operation instructions to the image acquisition device, the management device and other devices. S313, the image acquisition device can receive the operation instructions triggered by the user terminal based on the behavior information of each object, S314, and re-collect new scene images according to the display area indicated by the operation instructions, and send the new scene images to the management device; other devices can also receive the operation instructions triggered by the user terminal based on the behavior information of each object, and re-collect new operation data according to the display area indicated by the operation instructions, and send the new operation data to the management device. Then, the management device can receive the operation instructions, the new scene images, and the new operation data, and use the preset artificial intelligence algorithm to identify the surgical behavior of the new scene images and the new operation data according to the display area indicated by the operation instructions, and obtain the behavior information of each object in the operating room. Then, S315, the management device can determine whether the behavior information of each object in the operating room meets the preset behavior standard according to the preset behavior standard, so as to manage the behavior information of each object and obtain the management result.

[0075] In addition, the medical team can pre-customize the definition of the end of the operation. Optionally, the definition of the end of the operation can be at least one of the following ways. The first is a stop signal from the medical staff 161 in the operating room: for example, the doctor's preset gesture signal, voice signal, key command, and delay command corresponding to the stop signal, wherein the key command refers to the command expressed by the medical staff through certain physical and / or virtual media, and the key command may include but is not limited to physical keys or computer keyboard input, touch screen, AR / VR virtual key command, eye movement / blink recognition command, etc. The second is automatic video recognition: exemplarily, when the patient is pushed out of the operating room, the panoramic image acquisition device 111 can capture image information of the opening of the operating room door and the image information of the patient leaving the operating room, and send the image information to the management device 12, so that the management device 12 can recognize the above-mentioned combined image information and automatically define the end of the operation for the patient. The third type is the shutdown and / or sleep signals of other devices: the shutdown and / or sleep signals of other devices 13 may include but are not limited to the lights in the operating room being turned off or asleep, the power of the energy platform being turned off or asleep, the signal that the pressure on the operating bed is greatly reduced, the Bluetooth connection between the doctor's mobile phone or mobile music player and the operating room audio system is disconnected, etc. The management device 12 can automatically define the end of the patient's surgery based on the shutdown and / or sleep signals of one or more other devices 13.

[0076] Combination Figure 3 As shown, S316, after determining that the operation is over, the postoperative stage is carried out. In the postoperative stage, S317, the management device can save the scene images obtained during and after the operation, the behavior information of each object in the operating room, and the management results in the hospital's storage device, server, or cloud, so that, S318, the management device can construct an operating room process model based on the scene images and the behavior information of each object in the operating room, and determine each surgical process parameter in the operating room process model, so as to perform statistical analysis on each surgical process parameter in the operating room process model of the operation, and obtain and display the statistical results of the operation. In addition, the management device can also obtain each surgical process parameter in the operating room process model and each historical surgical process parameter in multiple historical operating room process models, and perform statistical analysis based on each surgical process parameter in the operating room process model and each historical surgical process parameter in multiple historical operating room process models, and obtain and display the statistical results. Afterwards, S319, the saved data can be archived and managed on the user terminal, S320, and the entire surgical process can be reviewed on the user terminal.

[0077] like Figure 4 As shown, Figure 4The figure is a flowchart of operating room management through an operating room management system in one embodiment. When the management device executes the surgical process (i.e., during the operation) at S401, S402, a preset artificial intelligence algorithm can be used to identify surgical behaviors on the scene image and operation data, obtain the behavioral information of each object in the operating room, and record or mark the real-time spatial position of each object in the operating room. After the operation, S403, the management device can construct an operating room process model based on the scene image and the behavioral information of each object in the operating room, and determine each surgical process parameter in the operating room process model, S404, thereby performing statistical analysis on each surgical process parameter in the operating room process model of the operation, and obtaining and displaying the statistical results of the operation. Combined with Figure 4 As shown, exemplarily, surgical process parameters may include behavior types, behavior parameters, etc. Behavior types may include but are not limited to the opening time of the operating room door, the displacement length of each object or the waiting time of the patient, etc. Behavior parameters may include but are not limited to time parameters or displacement parameters, etc. In addition, S405, the management device may also obtain each surgical process parameter in the operating room process model and each historical surgical process parameter in multiple historical operating room process models, and perform statistical analysis based on each surgical process parameter in the operating room process model and each historical surgical process parameter in multiple historical operating room process models to obtain and display the statistical results at S406. The statistical results may be displayed in the form of graphs, tables, etc., and the embodiment of the present application does not limit the display form of the statistical results. Combined with Figure 4 As shown, Figure 4 The chart corresponding to S406 is a schematic diagram of the display format of the statistical results in an exemplary embodiment.

[0078] Based on the above analysis, it can be known that the statistical results in the embodiments of the present application may include information of the following dimensions: 1) spatial information, including displacement information in the operating room; 2) temporal information, including information and changes generated by a certain object on the timeline; 3) analog change information, the changing rules of the same or similar behaviors of the same object in multiple operations; 4) risk deduction information, based on the surgical operations and surgical results in the operating room, statistical analysis is performed to predict the risks and advantages of surgical operations in historical operations of the same type. It should be noted that the above information can be visualized in two stages: intraoperative (real-time) and postoperative (review) to meet the diverse needs of operating room behaviors.

[0079] For example, Figure 5 As shown, Figure 5FIG. 1 is a schematic diagram of a visualization interface of a user terminal in an embodiment. The visualization interface of the user terminal may include but is not limited to a reconstruction result and object display area, a timeline area, a side function menu area, an option area, a custom information display area, and the like. Among them, the reconstruction result and object display area include background and content, the background includes the three-dimensional space reconstruction result of the operating room, the content includes the positioning / position of each object in the operating room, each object includes medical staff and equipment, and the positioning / position of each object in the operating room can carry the name mark of each object; the timeline area is a draggable area, which is used to review the entire reconstructed surgical process after surgery; the side function menu area is used to determine whether to display / hide the positioning / position of each object in the operating room; the option area includes the current time component, interface login option, interface exit option, setting option, and function switching option. The current time component is used to display the current time, the interface login option is used to log in to the current interface, the interface exit option is used to exit the current interface, the setting option is used to set the parameters and / or functions of the current interface, and the function switching option is used to switch between the real-time monitoring function during surgery and the postoperative review function; the custom information display area is used to display information related to the surgery, and the custom information display area can be linked with the timeline.

[0080] Exemplarily, the custom information display area may display time-reviewable information, relevant information of the digital system in the operating room, or relevant information of other application data interfaces. The time-reviewable information may include but is not limited to the patient's digital twin file, similar surgical history comparison, operating room personnel behavior statistics or operating room video surveillance, etc. The operating room video surveillance may include but is not limited to the monitoring data of the panoramic image acquisition device, the operating table image acquisition device and the surgical field image acquisition device; the relevant information of the digital system in the operating room may include but is not limited to a single-axis image of the patient, intraoperative imaging data, patient electrophysiological signals, anesthesia monitoring data, surgical navigation, patient biochemical indicator data or equipment control buttons, etc. The intraoperative imaging data may be image data acquired using MR (magnetic resonance imaging), CT (computer tomography) or DSA (digital subtraction angiography) and other technologies.

[0081] In addition, optionally, the operating room management system in the embodiment of the present application can operate 24 hours a day, or can be turned on and / or off at a scheduled time, or can be actively turned on and / or off by medical staff inside and / or outside the operating room, or a low-power cruise mode can be set when the operating room is not in use, that is, only the panoramic monitoring of the panoramic image acquisition device in the operating room and the control signals collected and recognized by the user terminal are turned on.

[0082] In the embodiment of the present application, an operating room management system is proposed, which can objectively record and analyze the complete surgical process through visual tracking based on multiple audio and video data, and can establish a surgical digital twin model that integrates various data or information, so as to automatically calculate the key indicators in the entire surgical process and intelligently analyze the surgical process. Therefore, the operating room management system can not only facilitate the medical team to objectively, macroscopically and systematically understand all surgical processes and key indicators in the surgical process carried out in the operating room over a period of time, so as to optimize the allocation of operating room resources and improve the operating efficiency of the operating room; but also can provide the medical team with an objective and real review tool based on the key indicators in the surgical process, so as to facilitate the medical team to check for omissions and summarize experience, thereby improving the cooperation mode of the medical team and improving the surgical efficiency of the medical team.

[0083] It should be noted that, first, since the traditional method is only a simple record of the data in the surgical process, and does not conduct a deeper data perspective and fusion analysis of the data in the surgical process, the traditional method cannot intuitively combine the data and the surgical effect for display. The operating room behavior management method in the embodiment of the present application is no longer a simple stacking of data display and trend analysis, but after statistics of the surgical process and action behavior, it is associated with the surgical effect, doctor-patient feedback and the big data learning process of the artificial intelligence algorithm to obtain statistical analysis results. The statistical analysis results can provide auxiliary suggestions for the medical team, thereby improving the efficiency of intraoperative decision-making assistance and the value of postoperative summary learning. Second, since the traditional method can only focus on the intraoperative behavioral data, and does not correlate the surgical process at different periods in the entire perioperative period, the traditional method will miss the details in the surgical process and affect the representativeness of the data in the surgical process. This application includes a variety of data sources and types, and not only considers intraoperative behavioral data, but also preoperative and postoperative data. Therefore, this application can supplement preoperative and postoperative information or data into the digital twin model of the surgery for structured association, so as to obtain more comprehensive perioperative operating room management analysis results. Third, this application can also determine the data that doctors are concerned about through user terminals, so as to collect and analyze data in a differentiated, free, and personalized manner based on the data that doctors are concerned about, thereby helping different medical teams to propose the best surgical process suitable for them.

[0084] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0085] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an operating room management method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0086] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0087] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0088] The management device performs surgical behavior management on the scene images of the operating room acquired by the image acquisition device to obtain management results; the scene images include surgical behavior images and surgical environment images.

[0089] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0090] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0091] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0092] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An operating room management system, characterized in that: The system includes image acquisition equipment and management equipment; the image acquisition equipment includes a panoramic image acquisition device, an operating table image acquisition device and an operating field image acquisition device; The image acquisition device is used to acquire scene images of the operating room; the scene images include surgical behavior images and surgical environment images, and the scene images are sent to the management device; The management device is used to manage the surgical behavior of the scene image and obtain a management result.

2. The system according to claim 1, characterized in that The management device is specifically used to perform surgical behavior recognition on the scene image, obtain behavior information of each object in the operating room, and manage the behavior information of each object according to preset behavior standards to obtain the management result.

3. The system according to claim 2, characterized in that The management device is specifically used to manage the behavior information of each of the objects according to the preset behavior standard, and output a management result including warning information when it is determined that the behavior information of the object does not meet the preset behavior standard.

4. The system according to claim 2, characterized in that The management device is further used to display the behavior information of each of the objects, receive operation instructions generated by the user terminal based on the behavior information of each of the objects, and perform surgical behavior recognition based on the new scene image acquired by the image acquisition device according to the display area indicated by the operation instruction to obtain the behavior information of each object in the operating room; The new scene image is a scene image re-captured by the image acquisition device according to the operation instruction sent by the user terminal; The operation instruction is used to instruct the management device to perform analysis based on the display area, and to instruct the image acquisition device to acquire a scene image of the display area.

5. The system according to any one of claims 2 to 4, characterized in that: The management device is further used to construct an operating room process model according to the scene image, and determine each surgical process parameter in the operating room process model.

6. The system according to claim 5, characterized in that The management device is also used to perform statistical analysis based on each surgical process parameter in the operating room process model and each historical surgical process parameter in multiple historical operating room process models to obtain and display statistical results.

7. The system according to any one of claims 1 to 4, characterized in that: The panoramic image acquisition device is used to acquire the surgical environment information in the operating room; the operating table image acquisition device is used to acquire the environmental information of the area near the operating table in the operating room; and the surgical field image acquisition device is used to acquire the surgical behavior information of the personnel in the operating room.

8. The system according to any one of claims 1 to 3, characterized in that: The operating room management system also includes: other operating room equipment; the other operating room equipment includes a user terminal, an operating table, surgical instruments, an operating light, a laminar flow purification device, a voice communication device, an operating room door, an air conditioner or an anesthesia cart.

9. An operating room management method, characterized in that: The method is applied to the operating room management system according to any one of claims 1 to 8; the method comprises: The management device performs surgical behavior management on the scene image of the operating room acquired by the image acquisition device to obtain a management result; the scene image includes a surgical behavior image and a surgical environment image.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method described in claim 9 are implemented.

Images