Operating room system, image recording method, program, and medical information processing system

Abstract

The present disclosure pertains to a surgical operation room system, an image recording method, a program, and a medical information processing system, which enable proper preservation of records. A setup unit sets up whether or not automatic recording of an image is to be executed for each of multiple input sources which are disposed in a surgical operation room and to which images are inputted. The present disclosure is applicable to a surgical operation room system.

Description

[Technical Field]

[0001] The present disclosure relates to an operating room system, an image recording method, a program, and a medical information processing system, and more particularly to an operating room system, an image recording method, a program, and a medical information processing system that can appropriately keep records. [Background technology]

[0002] Operating rooms and ICUs (Intensive Care Units) are equipped with a variety of medical devices as well as multiple input sources for inputting images, such as MRI (Magnetic Resonance Imaging) devices, CT (Computed Tomography) devices, and ultrasound devices.

[0003] In recent years, operating room systems that integrate and control these input sources and display images have been used to improve the efficiency of surgeries. These operating room systems also require image recording functionality.

[0004] For example, Patent Document 1 discloses a system that transmits an image captured by a surgical loupe in an operating room to an external device for recording. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-104365 Summary of the Invention [Problem to be solved by the invention]

[0006] Typically, the timing to start recording surgical images and the duration of the recording are decided by the surgical staff. However, during surgery, the surgical staff are busy with other tasks and are sometimes unable to record at the appropriate time. On the other hand, recording surgical images from start to finish can result in a huge amount of recording capacity. This is not limited to operating rooms, but can also occur in medical and nursing care settings.

[0007] The present disclosure has been made in light of such circumstances and allows for appropriate records to be kept. [Means for solving the problem]

[0008] An operating room system according to a first aspect of the present disclosure is an operating room system that includes a setting unit that sets whether or not to automatically record images for each of a plurality of input sources that input images and that are provided in the operating room.

[0009] The image recording method of the first aspect of the present disclosure is an image recording method in which an operating room system sets whether or not to perform automatic recording of images for each of multiple input sources that input images and are installed in the operating room.

[0010] The program of the first aspect of the present disclosure is a program for causing a computer to execute a process for setting whether or not to automatically record images for each of multiple input sources that input images and are installed in an operating room.

[0011] A medical information processing system according to a second aspect of the present disclosure is a medical information processing system including a setting unit that sets, for each of a plurality of video sources that output information about a patient, whether or not to automatically record the information.

[0012] In the first aspect of the present disclosure, for each of a plurality of input sources for inputting images provided in an operating room, whether or not to perform automatic recording of the images is set.

[0013] In a second aspect of the present disclosure, for each of a plurality of video sources that output information about a patient, whether or not to automatically record the information is set. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a diagram showing an overview of an operating room system to which the technology disclosed herein can be applied. [Figure 2] FIG. 2 is a block diagram illustrating an example of the functional configuration of the operating room system. [Figure 3] FIG. 10 is a diagram illustrating an example of a setting screen. [Figure 4] 10 is a flowchart illustrating the flow of an automatic recording setting process. [Figure 5] 10 is a flowchart illustrating the flow of an automatic recording setting process. [Figure 6] 10 is a flowchart illustrating the flow of an automatic recording setting process. [Figure 7] 10 is a flowchart illustrating the flow of a recording process for each input source. [Figure 8] 10 is a flowchart illustrating the flow of a recording process for each input source. [Figure 9] 10 is a flowchart illustrating the flow of a recording process for each input source. [Figure 10] FIG. 10 is a diagram illustrating an example of an integrated screen. [Figure 11] FIG. 10 is a diagram illustrating an example of a preview screen. [Figure 12] FIG. 1 illustrates an example of the configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0015] Modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described below in the following order.

[0016] 1. Overview of the Operating Room System 2. Technical background and example functional configuration of operating room system 3. Automatic recording setting process 4. Recording process for each input source 5. Other automatic recording functions 6. Variations 7. Computer configuration example

[0017] <1. Overview of the operating room system> FIG. 1 is a diagram showing an overview of an operating room system 100 to which the technology according to the present disclosure can be applied.

[0018] In the operating room system 100 shown in Fig. 1, a group of devices installed in an operating room are connected to each other so that they can cooperate with each other via an operating room server 107 and an IP switch (SW) 109. The operating room system 100 is configured with an IP (Internet Protocol) network capable of sending and receiving 4K / 8K images, and input / output images and control information for each device are sent and received via the IP network.

[0019] A variety of devices may be installed in an operating room.

[0020] 1 shows a group of various devices 101 for endoscopic surgery, a ceiling camera 187 that captures images of the surgeon's hands, an operating room camera 189 that captures images of the entire operating room, multiple display devices 103A to 103D, a patient bed 183, and lighting 191. Both the ceiling camera 187 and the operating room camera 189 are installed on the ceiling of the operating room. In addition to the endoscope shown in the figure, the group of devices 101 may include various medical devices that acquire images, such as a master-slave endoscopic surgery robot or an X-ray device.

[0021] The display devices 103A to 103C, the device group 101, the ceiling camera 187, and the operating room camera 189 are connected to the IP SW 109 via IP converters 115A to 115F, respectively. Hereinafter, when there is no need to distinguish between the IP converters 115A to 115F, they will simply be referred to as IP converters 115.

[0022] IP converters 115D, 115E, and 115F on the input source side (camera side) that input images convert images from individual medical imaging devices (endoscopes, surgical microscopes, X-ray imaging devices, operating room cameras, pathology imaging devices, etc.) into IP formats and transmit them over the network. IP converters 115A to 115C on the image output side (monitor side) that output images convert images transmitted over the network into a format specific to the monitor and output them. The IP converter 115 on the input source side functions as an encoder, and the IP converter 115 on the image output side functions as a decoder. Input sources include, for example, video sources.

[0023] The IP converter 115 can have various image processing functions. For example, the IP converter 115 can have functions to perform resolution conversion processing according to the output destination, rotation correction and image stabilization of the endoscopic image, object recognition processing, etc.

[0024] These image processing functions may be inherent to the connected medical imaging device, or may be externally upgradeable. The IP converter 115 on the image output side (monitor side) can also perform processing such as combining multiple images (such as PinP processing) and superimposing annotation information.

[0025] The protocol conversion function of the IP converter 115 is a function of converting a received signal into a converted signal that conforms to a communication protocol that can be communicated over a network such as the Internet. Any communication protocol may be set as the communication protocol. Furthermore, the protocol-convertible signal received by the IP converter 115 is a digital signal, such as an image signal or a pixel signal. The IP converter 115 may be incorporated inside a device on the input source side or inside a device on the image output side.

[0026] The device group 101, for example, belongs to an endoscopic surgery system and includes an endoscope and a display device that displays images captured by the endoscope. Meanwhile, the display devices 103A to 103D, the patient bed 183, and the lighting 191 are devices installed in an operating room separate from the endoscopic surgery system. These devices used for surgery or diagnosis are also called medical devices. The operating room server 107 and / or the IP SW 109 coordinate and control the operation of these medical devices. Furthermore, if the operating room includes a surgical robot (surgical master-slave) system or a medical image acquisition device such as an X-ray device, these devices may also be connected to the device group 101.

[0027] The operating room server 107 comprehensively controls the processing related to image display in the medical equipment.

[0028] Among the devices included in the operating room system 100, the device group 101, the ceiling camera 187, and the operating room camera 189 may be devices (hereinafter also referred to as source devices) that have the function of transmitting information to be displayed during surgery (hereinafter also referred to as display information). Also, the display devices 103A to 103D may be devices (hereinafter also referred to as destination devices) that output display information.

[0029] The operating room server 107 has the function of controlling the operation of the source device and the destination device, acquiring display information from the source device, and transmitting the display information to the destination device for display or recording. The display information includes various images captured during surgery and various information related to the surgery (for example, physical information about the patient, past test results, information about the surgical procedure, etc.).

[0030] Specifically, the operating room server 107 may receive, as display information from the device group 101, information about an image of the operating area inside the patient's body cavity captured by an endoscope. The operating room server 107 may also receive, as display information from the ceiling camera 187, information about an image of the surgeon's hands captured by the ceiling camera 187. The operating room server 107 may also receive, as display information from the operating room camera 189, information about an image showing the overall state of the operating room captured by the operating room camera 189. If other devices with imaging capabilities are present in the operating room system 100, the operating room server 107 may also obtain, as display information, information about images captured by the other devices from the other devices.

[0031] The operating room server 107 displays the acquired display information (i.e., images captured during surgery and various pieces of information related to the surgery) on at least one of the display devices 103A to 103D, which are output destination devices. In the example of FIG. 1, the display device 103A is a display device hung from the ceiling of the operating room, and the display device 103B is a display device installed on a wall of the operating room. The display device 103C is a display device installed on a desk in the operating room, and the display device 103D is a mobile device with a display function (for example, a tablet PC (Personal Computer) or a smartphone).

[0032] The IP SW 109 is configured as one of the input / output controllers that controls the input / output of image signals to connected devices. For example, the IP SW 109 controls the input / output of image signals under the control of the operating room server 107. The IP SW 109 controls high-speed transfer of image signals between devices placed on the IP network.

[0033] The operating room system 100 may also include devices external to the operating room. Such devices may include, for example, a server connected to a network established inside or outside the hospital, a PC used by medical staff, or a projector installed in a hospital conference room. When such external devices are located outside the hospital, the operating room server 107 can display information on a display device in another hospital via a video conference system or the like for remote medical care.

[0034] The external server 113 may be, for example, an in-hospital server or a cloud server outside the operating room, and may be used for image analysis, data analysis, etc. In this case, image information from within the operating room is transmitted to the external server 113, and additional information may be generated by the external server 113 through big data analysis or recognition and analysis processing using AI (machine learning), and fed back to a display device within the operating room. At this time, an IP converter 115H connected to an IP SW 109 in the operating room transmits data to the external server 113 and analyzes the image. The transmitted data may be the surgical image itself, such as an endoscopic image, metadata extracted from the image, or data indicating the operating status of connected equipment.

[0035] The operating room system 100 is provided with a centralized operation panel 111. A user can issue instructions to the operating room server 107 regarding input / output control of the IP SW 109 and instructions regarding the operation of connected devices via the centralized operation panel 111. The user can also switch image displays via the centralized operation panel 111. The centralized operation panel 111 is configured by providing a touch panel on the display surface of a display device. The centralized operation panel 111 and the IP SW 109 are connected via an IP converter 115J.

[0036] In the operating room system 100, the IP network may be configured as a wired network, or part or all of the IP network may be configured as a wireless network. For example, an IP converter 115 on the input source side having a wireless communication function may transmit a received image to an IP converter 115 on the image output side via a wireless communication network such as a fifth generation mobile communication system (5G) or a sixth generation mobile communication system (6G).

[0037] <2. Technical background and example of functional configuration of operating room system> (Background of the technology related to the present disclosure) As described above, an operating room is provided with a plurality of input sources for inputting images (generating images and supplying them to other devices), such as various medical devices as well as an MRI device and a CT device.

[0038] In recent years, operating room systems that integrate and control these input sources and display images have been used to improve the efficiency of surgeries. These operating room systems also require image recording functionality.

[0039] Typically, the timing to start recording surgical images and the duration of the recording are determined by the surgical staff, which includes the surgeon, assistants, nurses, clinical engineers, and the like.

[0040] However, during surgery, surgical staff are often busy with other tasks and are unable to record at the appropriate time. Furthermore, in the case of unexpected events such as bleeding, even if recording is started after the event occurs, the desired surgical images cannot be recorded. On the other hand, recording surgical images from start to finish can result in a huge amount of recording capacity.

[0041] Therefore, in the operating room system 100 to which the technology according to the present disclosure is applied, whether or not to automatically record images is set for each of a plurality of input sources. The plurality of input sources include the above-mentioned device group 101, the ceiling camera 187, the operating room camera 189, and an MRI device or CT device that captures MRI images or CT images before surgery.

[0042] The automatic recording mentioned here includes pre-recording, which starts recording of images a predetermined time (for example, 30 seconds) before a recording start command, and trigger recording, which starts recording using the recognition results of image recognition as a trigger.

[0043] Furthermore, the automatic recording may be triggered by an input from an electronic medical device or by an input from a device external to the operating room system 100. The input from the device external to the operating room system 100 may be, for example, a signal transmitted from the integrated system to the operating room system 100 when the surgical staff sets the integrated system to start surgery.

[0044] (Example of functional configuration of operating room system) Fig. 2 is a block diagram showing an example of the functional configuration of an operating room system 100 that enables the setting of whether to automatically record images for each of multiple input sources provided in an operating room. Note that in Fig. 2, the same components as those in Fig. 1 are assigned the same reference numerals.

[0045] In the operating room system 100 of Fig. 2, multiple input sources 211 are each connected to the IP switch 109 via an IP converter 115. The multiple input sources 211 correspond to devices that input images to the IP switch 109, such as the device group 101, ceiling camera 187, operating room camera 189, MRI devices, CT devices, etc.

[0046] An image input from the input source 211 to the IP switch 109 is recorded in the recording unit 231 under the control of the operating room server 107 .

[0047] The recording unit 231 may be a recording device such as a hard disk drive (HDD) or a solid state drive (SSD) built into the operating room server 107, or may be a recording medium connected to the operating room server 107 via a driver. The recording unit 231 may also be a recording device provided outside the operating room and connected to the operating room server 107 via an in-hospital network, or a cloud server outside the hospital.

[0048] The operating room server 107 controls the recording of images input from the input source 211 to the IP switch 109 in the recording unit 231, and sets for each input source 211 whether or not to automatically record images.

[0049] The operating room server 107 is configured to include a setting reception unit 251 , an automatic recording setting unit 252 , and a recording control unit 253 .

[0050] The setting reception unit 251 receives a setting of whether to execute automatic recording (ON / OFF of automatic recording) for each input source 211, based on an operation by the surgical staff on a setting screen described later. Reception information indicating the received setting content is supplied to the automatic recording setting unit 252.

[0051] The automatic recording setting unit 252 sets automatic recording ON / OFF for each input source 211 based on the received information from the setting receiving unit 251. The setting information indicating the set automatic recording ON / OFF is supplied to the recording control unit 253.

[0052] Furthermore, the automatic recording setting unit 252 may set the automatic recording method for each input source 211 in addition to setting the automatic recording ON / OFF for each input source 211. For example, the automatic recording setting unit 252 may set for each input source 211 whether the automatic recording is to be performed as pre-recording, trigger recording triggered by the recognition result of image recognition, or trigger recording triggered by input from an electronic medical device.

[0053] The recording control unit 253 controls recording for each input source 211 based on the setting information from the automatic recording setting unit 252. Specifically, for an input source 211 for which automatic recording is set to ON, the recording control unit 253 starts the automatic recording described above. For an input source 211 for which automatic recording is set to OFF, the automatic recording is not started, and the input source 211 is set to a standby state for manual recording (recording instruction from surgical staff). For an input source 211 for which automatic recording is set to OFF, the input source 211 may be set to record continuously, rather than being set to a standby state for manual recording.

[0054] Images from each input source 211 are recorded in the operating room server 107 (recording unit 231) connected to each input source 211 via the IP converter 115. This allows for large-capacity recording.

[0055] Alternatively, images from each input source 211 may be recorded in each IP converter 115 connected to the corresponding input source 211. This allows recording to continue even if some kind of system failure occurs in the operating room system 100, as long as each IP converter 115 is operating. In this case, the automatic recording setting unit 252 sets automatic recording ON / OFF for the IP converters 115 corresponding to each input source 211.

[0056] <3. Automatic recording setting process> (Example of settings screen) FIG. 3 is a diagram showing an example of a setting screen operated by the surgical staff when accepting the ON / OFF setting of automatic recording.

[0057] The setting screen 300 in FIG. 3 is displayed on the centralized operation panel 111, for example.

[0058] The setting screen 300 has a setting area 310 for accepting settings for image display and recording for each input source 211. In the example of Fig. 3, four setting areas 310 are shown corresponding to the four input sources 211.

[0059] An image display section 321, a record button 322, and a shutter button 323 are provided above the setting area 310. The image display section 321 is an area where images input from the input source 211 are displayed in real time. The record button 322 is a button operated to start and stop recording of the image (moving image) displayed on the image display section 321. The shutter button 323 is a button operated to record the image displayed on the image display section 321 as a still image.

[0060] At the bottom of the setting area 310, there are provided switch buttons 341, 342, and 343 for switching the monitor to which the image displayed on the image display unit 321 is to be output. The switch buttons 341, 342, and 343 correspond to three monitors: monitor 1, monitor 2, and monitor 3. For example, when the switch button 341 is selected, the image displayed on the image display unit 321 is output to monitor 1. The number of switch buttons can be changed depending on the number of monitors provided in the operating room system 100.

[0061] The upper left portion of the setting screen 300 is provided with a recording button 350 and an automatic recording setting button 360. The recording button 350 is a button operated to start and stop recording of sounds in the operating room. The automatic recording setting button 360 is a button operated to set the operating room system 100 to an automatic recording setting mode for setting automatic recording ON / OFF for each input source 211.

[0062] For example, when the operating room system 100 is set to the automatic recording setting mode, a setting button (not shown) for setting ON / OFF of automatic recording for the corresponding input source 211 is displayed in a predetermined area of ​​each setting area 310. The user (surgery staff) can set ON / OFF of automatic recording for each input source 211 by operating the setting button of each setting area 310.

[0063] (Automatic recording setting process flow 1) Here, with reference to the flowchart in Fig. 4, a description will be given of the flow of the automatic recording setting process for setting ON / OFF of automatic recording for each input source 211. The process in Fig. 4 is executed for each input source 211 when the operating room system 100 is set to the automatic recording setting mode.

[0064] In step S11, the setting receiving unit 251 determines whether or not an automatic recording setting for the input source 211 corresponding to the setting area 310 has been received, depending on whether or not the setting button displayed in the setting area 310 has been operated.

[0065] If it is determined in step S11 that the automatic recording setting has been accepted, the process proceeds to step S12, where the automatic recording setting unit 252 sets automatic recording to ON for the input source 211 corresponding to the setting area 310 in which the setting button was operated.

[0066] On the other hand, if it is determined in step S11 that the automatic recording setting has not been accepted, the process proceeds to step S13, where the automatic recording setting unit 252 sets automatic recording to OFF for the input source 211 corresponding to the setting area 310 in which the setting button was operated.

[0067] In this manner, it is possible to set whether or not to automatically record images for each of the multiple input sources 211.

[0068] 4, whether or not to execute automatic recording is set individually for each input source 211 by operating the setting button displayed in the setting area 310. However, without being limited to this, whether or not to execute automatic recording for each input source 211 may be set collectively (automatically).

[0069] (Automatic recording setting process flow 2) 5, an example of an automatic recording setting process for collectively setting ON / OFF of automatic recording for each input source 211 will be described. The process in FIG. 5 starts when the automatic recording setting button 360 is operated to set the operating room system 100 to the automatic recording setting mode.

[0070] In step S31, the setting receiving unit 251 receives the setting of automatic recording for each input source 211 by setting the operating room system 100 to the automatic recording setting mode.

[0071] In step S32, the automatic recording setting unit 252 determines whether or not one of the plurality of input sources 211 is an input source for which automatic recording is to be set.

[0072] Specifically, the automatic recording setting unit 252 determines whether or not the input source 211 is an input source for which automatic recording is to be set, based on the type of the input source 211. For example, the automatic recording setting unit 252 acquires medical device information of the input source 211, and determines whether or not the input source is an input source for which automatic recording is to be set, according to a table set in advance based on the type of electronic medical device. For example, the medical device information may be acquired from metadata of the image of the input source 211, or may be acquired as DICOM (Digital Imaging and Communications in Medicine) information. Furthermore, the medical device information may be connection information acquired by the IP converter 115 connected to the input source 211.

[0073] Furthermore, the automatic recording setting unit 252 may determine whether or not the input source is for which automatic recording is to be set based on preset information in which ON / OFF of automatic recording is set in advance. For example, the preset information is information in which ON / OFF of automatic recording is set for each IP converter 115 to which the input source 211 is connected. That is, ON / OFF of automatic recording for the input source 211 is set according to the IP converter 115 to which the input source 211 is connected. Furthermore, multiple pieces of preset information may be stored in the operating room server 107, and ON / OFF of automatic recording for each input source 211 may be set based on preset information selected by the user (operating staff).

[0074] The preset information may be associated with a surgical procedure or may be information set by a user. Furthermore, recommended preset information may be presented to a user by recognizing the type of each electronic medical device.

[0075] If it is determined in step S32 that the input source 211 is an input source for which automatic recording is to be set based on the type and preset information of the input source 211, the process proceeds to step S33. In step S33, the automatic recording setting unit 252 sets automatic recording to ON for the input source 211.

[0076] On the other hand, if it is determined in step S32 based on the type and preset information of the input source 211 that the input source 211 is not an input source for which automatic recording is to be set, the process proceeds to step S34. In step S34, the automatic recording setting unit 252 sets automatic recording to OFF for the input source 211.

[0077] After step S33 or step S34, the process proceeds to step S35, where the automatic recording setting unit 252 determines whether or not automatic recording has been set to ON / OFF for all input sources 211.

[0078] If it is determined that the automatic recording ON / OFF has not been set for all the input sources 211, the process returns to step S32 and the subsequent processes are repeated. On the other hand, if it is determined that the automatic recording ON / OFF has been set for all the input sources 211, the series of processes ends.

[0079] In this way, it is possible to collectively set whether or not to perform automatic image recording for each of the plurality of input sources 211.

[0080] According to the above process, whether or not to automatically record images can be set for each of multiple input sources, making it possible to appropriately record surgical images for each input source. Specifically, desired surgical images can be recorded without having to record the entire surgical image. This prevents overlooking unexpected events such as bleeding and reduces the recording capacity per surgery.

[0081] (Automatic recording setting process flow 3) Incidentally, in the operating room system 100, when automatic recording is started for one input source 211, it may be preferable that automatic recording is also started for related input sources 211 in conjunction with the start of automatic recording.

[0082] 6, a flow of an automatic recording setting process for setting ON / OFF of automatic recording so that when automatic recording is started for one input source 211, automatic recording is also started for related input sources 211. Similar to the process of FIG. 4, the process of FIG. 6 is executed for each input source 211 when the operating room system 100 is set to the automatic recording setting mode.

[0083] In step S51, the setting reception unit 251 determines whether or not an automatic recording setting for the input source 211 corresponding to the setting area 310 has been received, depending on whether or not the setting button displayed in the setting area 310 has been operated.

[0084] If it is determined in step S51 that the automatic recording setting has been accepted, the process proceeds to step S52, where the automatic recording setting unit 252 sets automatic recording to ON for the input source 211 corresponding to the setting area 310 in which the setting button was operated.

[0085] In step S53, the setting reception unit 251 determines whether or not a setting or selection of an input source that starts automatic recording simultaneously with the input source 211 for which automatic recording is set ON has been received, depending on, for example, whether or not a GUI (not shown) displayed on the setting screen 300 has been operated.

[0086] Here, the input sources for which the setting / selection to simultaneously start automatic recording is accepted may be limited to input sources 211 for which automatic recording has already been set to ON, or may include input sources 211 for which automatic recording has been set to OFF.

[0087] If it is determined in step S53 that setting / selection of input sources for which automatic recording is to be started simultaneously has been accepted, the process proceeds to step S54. In step S54, the automatic recording setting unit 252 sets automatic recording to ON for the input sources 211 for which setting or selection for automatic recording has been accepted simultaneously.

[0088] If it is determined in step S53 that the setting / selection of an input source for simultaneously starting automatic recording has not been accepted, step S54 is skipped.

[0089] On the other hand, if it is determined in step S51 that the automatic recording setting has not been accepted, the process proceeds to step S55, where the automatic recording setting unit 252 sets automatic recording to OFF for the input source 211 corresponding to the setting area 310 in which the setting button was operated.

[0090] In this way, when automatic recording is started for one input source 211, automatic recording can be set to ON / OFF so that automatic recording also starts for related input sources 211 in conjunction with the start of automatic recording.

[0091] According to the above process, when pre-recording is started for an endoscope by a predetermined trigger, pre-recording is also started for other related input sources in conjunction with the endoscope. As a result, when a sudden event such as bleeding occurs, images from the other input sources before the event can be recorded simultaneously with the endoscope images before the event.

[0092] <4. Recording process for each input source> The following describes the flow of the recording process for each input source 211 for which the automatic recording ON / OFF has been set by the above-described automatic recording setting process.

[0093] (Recording process flow for each input source 1) The flow of recording processing for each input source 211 when automatic recording is pre-recording will be described with reference to the flowchart in Fig. 7. The processing in Fig. 7 is executed for each input source 211 for which automatic recording is set to ON / OFF.

[0094] In step S111, the recording control unit 253 determines whether automatic recording is set to ON for the input source 211, for example, based on setting information indicating whether automatic recording is ON / OFF for each input source 211, which is set by the automatic recording setting unit 252.

[0095] If it is determined in step S111 that the input source has automatic recording set to ON, the process proceeds to step S112, where the recording control unit 253 starts recording images of the input source 211 in a buffer. As a result, for example, 30 seconds' worth of frames of the input source 211 are always recorded in a buffer area of ​​a ROM (Read Only Memory) or the recording unit 231 provided in the operating room server 107. 30 seconds' worth of frames may always be recorded in this buffer area by repeating recording and deletion on a frame-by-frame basis, or the group of frames to be recorded may be replaced every 30 seconds.

[0096] In step S113, the recording control unit 253 determines whether or not a recording start instruction has been input for the input source 211. Step S113 is repeated until a recording start instruction is input.

[0097] The recording start instruction here may be triggered by an operation such as pressing a button by the user, or by the arrival of a time point preset by the user. The recording start instruction may also be triggered by the recognition result of image recognition on the image from the input source 211, or by input from an electronic medical device. For example, the recording start instruction may be triggered by image recognition recognizing the insertion of an endoscope into the patient's body, or by detecting that the power of an electric scalpel has been turned on. Furthermore, the recording start instruction may also be triggered by the user's voice.

[0098] If it is determined in step S113 that a recording start instruction has been input for the input source 211, then in step S114 the recording control unit 253 starts pre-recording of the image of the input source 211. This makes it possible to record the image of the input source 211, for example, 30 seconds before the recording start instruction.

[0099] On the other hand, if it is determined in step S111 that the input source is not one for which automatic recording is set to ON, the process ends. That is, since the input source 211 is one for which automatic recording is set to OFF, the input source enters a state in which it waits for manual recording, for example.

[0100] The automatic recording setting unit 252 may set the details of pre-recording in accordance with user operations on the setting screen 300. Specifically, the frame rate of images recorded in the buffer and the buffer recording time (such as the above-mentioned 30 seconds) may be set in accordance with the image quality (resolution) of the images from the input source 211. For example, the buffer recording time may be set to 30 seconds if the images from the input source 211 are 4K images, or 60 seconds if the images from the input source 211 are HD images. Furthermore, it may be set in accordance with user operations whether to prioritize the frame rate or the buffer recording time when recording frames in the buffer.

[0101] Furthermore, the buffer recording time may be changed depending on whether the image for each input source 211 is recorded in the operating room server 107 (recording unit 231) or stored in the corresponding IP converter 115. For example, when the image for the input source 211 is recorded in the corresponding IP converter 115, the buffer recording time may be set to 30 seconds, whereas when the image is recorded in the operating room server 107 (recording unit 231) that is capable of large-capacity recording, the buffer recording time may be set to 60 seconds.

[0102] (Recording process flow for each input source 2) The flow of recording processing for each input source 211 when automatic recording is trigger recording, in which recording is started using the recognition result of image recognition as a trigger, will be described with reference to the flowchart of Fig. 8. The processing of Fig. 8 is also executed for each input source 211 for which automatic recording is set to ON / OFF.

[0103] In step S131, the recording control unit 253 determines whether automatic recording is set to ON for the input source 211 based on the setting information set by the automatic recording setting unit 252 indicating whether automatic recording is ON / OFF for each input source 211.

[0104] If it is determined in step S131 that the input source is one for which automatic recording is set to ON, the process proceeds to step S132, where the recording control unit 253 starts image recognition for the image of the input source 211 in question.

[0105] In step S133, the recording control unit 253 determines whether or not a recognition result that triggers the start of recording has been obtained for the input source 211. Step S133 is repeated until a recognition result that triggers the start of recording is obtained.

[0106] For example, if the input source 211 is an endoscope, the recognition result that triggers the start of recording may be the insertion of the endoscope into the patient's body, the entry of surgical staff into the operating room, or a specific action of the surgical staff. These image recognitions are performed, for example, by a machine learning model using a neural network.

[0107] If it is determined in step S133 that a recognition result that triggers the start of recording has been obtained for the input source 211, the recording control unit 253 starts recording the image of the input source 211 in step S134.

[0108] On the other hand, if it is determined in step S131 that the input source is not one for which automatic recording is set to ON, the process ends. That is, since the input source 211 is one for which automatic recording is set to OFF, the input source enters a state in which it waits for manual recording, for example.

[0109] (Recording process flow for each input source 3) The flow of recording processing for each input source 211 when automatic recording is triggered by input from an electronic medical device will be described with reference to the flowchart in Fig. 9. The processing in Fig. 9 is also executed for each input source 211 for which automatic recording is set to ON / OFF.

[0110] In step S151, the recording control unit 253 determines whether automatic recording is set to ON for the input source 211 based on the setting information set by the automatic recording setting unit 252 indicating whether automatic recording is ON / OFF for each input source 211.

[0111] If it is determined in step S151 that the input source has automatic recording set to ON, the process proceeds to step S152, where the recording control unit 253 determines whether or not an input from an electronic medical device that triggers the start of recording has been obtained for the input source 211. Step S152 is repeated until an input from an electronic medical device that triggers the start of recording is obtained.

[0112] Examples of inputs from electronic medical devices that trigger the start of recording include turning on / off the power of an electric scalpel, opening and closing the door to the operating room (entrance of surgical staff), and wireless communication via RFID from the electronic medical device.

[0113] If it is determined in step S152 that an input from an electronic medical device that triggers the start of recording has been obtained for the input source 211, the recording control unit 253 starts recording the image of the input source 211 in step S153.

[0114] On the other hand, if it is determined in step S151 that the input source is not one for which automatic recording is set to ON, the process ends. That is, since the input source 211 is one for which automatic recording is set to OFF, the input source enters a state in which it waits for manual recording, for example.

[0115] <5. Other automatic recording functions> For input source 211 with automatic recording set to ON, pre-recording may be started in response to a recording start instruction from another location outside the operating room via a network. For example, even if automatic recording is set to ON for input source 211, if an unexpected event occurs, the surgical staff may be busy dealing with the event and may not be able to press the button to start recording within 30 seconds. Therefore, a system may be set up so that staff in a surgical monitoring room outside the operating room can input a recording start instruction when they notice that an unexpected event has occurred.

[0116] In the operating room system 100, the automatic recording setting unit 252 may set whether to automatically record images from each input source 211 depending on the purpose of the recorded images. For example, images from the operating room camera 189 may be constantly recorded for use as evidence in court in the event of a medical accident, and automatic recording of images from other input sources 211 may be set to ON for report creation, patient explanations, and procedure propagation, thereby allowing each image to be recorded with the optimal recording capacity.

[0117] In this case, a black box function may be applied to images recorded for court evidence, which prevents anyone other than the administrator from editing the images. For example, when a surgery start button displayed on the centralized operation panel 111, which is operated at the start of surgery, is pressed, image recording using the black box function may begin.

[0118] As shown in FIG. 10, the centralized operation panel 111 can display an integrated screen that integrates images from a plurality of input sources 211 in the operating room system 100.

[0119] 10, automatic image recording may be set to execute automatic recording of the entire integrated screen or to execute automatic recording of each image of each input source 211 displayed on the integrated screen. When automatic recording of each image of each input source 211 is set, the operating room system 100 acquires information about each input source 211 that is inputting the image displayed on the integrated screen, and based on that information, determines which input source 211 has automatic recording set to ON, and starts automatic recording.

[0120] The integrated screen as shown in FIG. 10 may be displayed in a surgery monitor room located outside the operating room, as described above.

[0121] <6. Variations> (Preview screen) The images of each input source 211 recorded in the operating room system 100 described above can also be reproduced after the operation on a preview screen 500 as shown in FIG.

[0122] The preview screen 500 includes a playback area 510, a playback target selection area 520, a timeline 530, and a group of operation buttons 540.

[0123] The playback area 510 is an area for playing back images and sounds selected in the playback target selection area 520. In the example of Fig. 11, the playback target selection area 520 displays selection buttons 521 to 524 including thumbnails and icons representing images and sounds recorded in the operating room system 100.

[0124] The selection button 521 corresponds to the audio recorded in the operating room system 100. The selection button 521 displays an icon indicating that it is audio, as well as the playback time and recording start time of the audio.

[0125] The selection button 522 corresponds to an image captured by the operating room camera 189 and recorded in the operating room system 100. The selection button 522 displays a thumbnail of the image as well as the playback time and recording start time of the image. In the example of FIG. 11, the selection button 522 is selected, and an image of the entire operating room captured by the operating room camera 189 is played in the playback area 510.

[0126] The selection button 523 corresponds to a still image captured by an endoscope, which is one of the devices 101, and recorded in the operating room system 100. The selection button 523 displays a thumbnail of the still image as well as the recording time (capture time) of the still image.

[0127] The selection button 524 corresponds to the vital signs of the patient during surgery that were recorded in the operating room system 100. The selection button 524 displays a thumbnail of the vital sign as well as the recording start time of the vital sign.

[0128] A timeline 530 provided below the playback area 510 indicates the total playback time of the images and sounds being played in the playback area 510, and a pointer is displayed at the position corresponding to the playback position.

[0129] The operation button group 540 is composed of buttons for accepting various operations on the images and audio being played in the playback area 510, such as stopping or pausing playback, fast forwarding or rewinding, and double-speed playback, as well as muting audio, enlarging or reducing images, and capturing still images.

[0130] On such a preview screen 500, it is also possible to select an image recorded by automatic recording in a playback object selection area 520.

[0131] In this case, the still image at the timing when a recording start instruction (such as a button press) is input during pre-recording is displayed as a thumbnail on the selection button displayed in playback target selection area 520. This allows, for example, a doctor to select an image to be played back based on his or her memory of "this kind of image should have been recorded."

[0132] In this case, the timeline 530 may be displayed so as to distinguish between the image section recorded in the buffer for, for example, 30 seconds during pre-recording and the image section recorded after the recording start command was input, thereby enabling the doctor to know at a glance when he or she started recording.

[0133] It is possible that images recorded automatically may include images that the user does not remember. Therefore, it is preferable that the thumbnails of the selection buttons displayed in the playback target selection area 520 be displayed so that images recorded automatically and images recorded manually can be distinguished.

[0134] (others) The automatic recording described above can reduce the recording capacity per surgery. Furthermore, a predetermined threshold value may be set for the recording capacity in the automatic recording, and when the recording capacity exceeds the threshold value, the automatic recording of images may be stopped. This allows images to be recorded with an appropriate recording capacity depending on the capacity of the recording unit 231, etc.

[0135] Furthermore, images recorded by automatic recording may be assigned a level of importance, and images with lower importance may be deleted preferentially. For example, the lower the importance of an image, the shorter the storage period may be set, and the image may be deleted after the storage period has elapsed, or the deletion timing may be set at predetermined intervals, and images with lower importance may be deleted at each deletion timing.

[0136] In the above-described embodiment, the image output from the medical imaging device is IP converted using an external IP converter, but the function for performing IP conversion may also be built into the medical imaging device.

[0137] Although the above description has been given of an example in which multiple input sources are provided in one operating room, multiple input sources may be provided in multiple operating rooms. Furthermore, multiple input sources may be provided not only in operating rooms but also in ICUs, hospital wards, nursing homes, etc. In this case, the input source may be, for example, a video source that outputs information about a patient. This allows appropriate patient records and nursing care records to be kept.

[0138] <7. Computer configuration example> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware or a general-purpose personal computer.

[0139] FIG. 12 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0140] The operating room server 107 to which the technology according to the present disclosure can be applied is realized by a computer having the configuration shown in FIG.

[0141] A CPU 1001 , a ROM (Read Only Memory) 1002 , and a RAM (Random Access Memory) 1003 are interconnected by a bus 1004 .

[0142] An input / output interface 1005 is also connected to the bus 1004. An input unit 1006 including a keyboard, a mouse, etc., and an output unit 1007 including a display, a speaker, etc. are connected to the input / output interface 1005. Also connected to the input / output interface 1005 are a storage unit 1008 including a hard disk, a nonvolatile memory, etc., a communication unit 1009 including a network interface, etc., and a drive 1010 that drives removable media 1011.

[0143] In a computer configured as above, the CPU 1001 loads a program stored in the storage unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004 and executes the program, thereby performing the above-described series of processes.

[0144] The program executed by the CPU 1001 is installed in the storage unit 1008 by being recorded on a removable medium 1011, or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting.

[0145] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0146] The embodiments of the present disclosure are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure.

[0147] Furthermore, the effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0148] Furthermore, the present disclosure can be configured as follows. (1) a setting unit that is provided in the operating room and that sets whether or not to automatically record the images for each of a plurality of input sources that input images; An operating room system comprising: (2) The automatic recording includes pre-recording, which starts recording the image a predetermined time before a recording start instruction is issued. (1) The operating room system described in (1). (3) The instruction to start recording is triggered by at least one of a user operation or setting, a recognition result of image recognition on the image, and an input from an electronic medical device. (2) The operating room system described in (2). (4) The recording start instruction is an instruction from outside the operating room. (2) The operating room system described in (2). (5) The setting unit further sets the predetermined time. (2) The operating room system described in (2). (6) The setting unit further sets a frame rate of the image to be recorded. An operating room system according to (2) or (3). (7) The automatic recording includes trigger recording, which starts recording in response to a recognition result of image recognition on the image. (1) The operating room system described in (1). (8) The image recognition is performed by a machine learning model. (7) The operating room system according to (7). (9) The automatic recording includes trigger recording, which starts recording when triggered by an input from an electronic medical device. (1) The operating room system described in (1). (10) The input from the electronic medical device includes at least one of power ON / OFF, opening and closing of a door in the operating room, and wireless communication by RFID (Radio Frequency Identifier). (9) The operating room system according to (9). (11) an IP (Internet Protocol) converter connected to each of the input sources; Each of the IP converters records the image for each of the input sources. An operating room system according to any one of (1) to (10). (12) a server connected to each of the input sources via an IP (Internet Protocol) converter; The server records the image for each of the input sources. An operating room system according to any one of (1) to (10). (13) The setting unit sets whether or not to execute the automatic recording of the image for each of the input sources based on the type of the input source. An operating room system according to any one of (1) to (12). (14) The setting unit sets whether or not to execute the automatic recording of the image for each of the input sources based on preset information. An operating room system according to any one of (1) to (12). (15) When one of the input sources starts the automatic recording, the setting unit sets another of the input sources to start the automatic recording at the same time. An operating room system according to any one of (1) to (12). (16) The setting unit sets whether or not to execute the automatic recording of the image for each of the input sources depending on the purpose of the image to be recorded. (1) The operating room system described in (1). (17) a display unit that displays an integrated screen in which a plurality of the images from a plurality of the input sources are integrated; The setting unit sets whether to perform the automatic recording of the integrated screen or the automatic recording of each image. (1) The operating room system described in (1). (18) The input source includes a medical imaging device. An operating room system according to any one of (1) to (17). (19) The operating room system For each of a plurality of input sources for inputting images provided in the operating room, whether or not to automatically record the images is set. Image recording method. (20) On the computer, For each of a plurality of input sources for inputting images provided in the operating room, whether or not to automatically record the images is set. A program for executing a process. (twenty one) A setting section for setting whether or not to automatically record information about a patient for each of a plurality of video sources that output the information about the patient. A medical information processing system comprising: [Explanation of symbols]

[0149] 100 Operating room system, 107 Operating room server, 109 IP switch, 115 IP converter, 211 Input source, 231 Recording unit, 251 Setting reception unit, 252 Automatic recording setting unit, 253 Recording control unit

Claims

1. a setting unit that is provided in the operating room and that sets whether or not to automatically record the images for each of a plurality of input sources that input images; Equipped with the automatic recording includes pre-recording, which starts recording the image a predetermined time before a recording start instruction is issued; The setting unit changes the buffer recording time for the pre-recording in accordance with the recording destination of the image. Operating room system.

2. The instruction to start recording is triggered by at least one of a user operation or setting, a recognition result of image recognition on the image, and an input from an electronic medical device. The operating room system of claim 1 .

3. The recording start instruction is an instruction from outside the operating room. The operating room system of claim 1 .

4. The setting unit further sets the predetermined time. The operating room system of claim 1 .

5. The setting unit further sets a frame rate of the image to be recorded. The operating room system of claim 1 .

6. The automatic recording further includes trigger recording, which starts recording when triggered by a recognition result of image recognition on the image. The operating room system of claim 1 .

7. The image recognition is performed by a machine learning model. The operating room system of claim 6.

8. The automatic recording further includes trigger recording, which starts recording when triggered by an input from an electronic medical device. The operating room system of claim 1 .

9. The input from the electronic medical device includes at least one of power ON / OFF, opening and closing of a door in the operating room, and wireless communication by RFID (Radio Frequency Identifier). The operating room system of claim 8.

10. an IP (Internet Protocol) converter connected to each of the input sources; Each of the IP converters records the image for each of the input sources. The operating room system of claim 1 .

11. a server connected to each of the input sources via an IP (Internet Protocol) converter; The server records the image for each of the input sources. The operating room system of claim 1 .

12. The setting unit sets whether or not to execute the automatic recording of the image for each of the input sources based on the type of the input source. The operating room system of claim 1 .

13. The setting unit sets whether or not to execute the automatic recording of the image for each of the input sources based on preset information. The operating room system of claim 1 .

14. When one of the input sources starts the automatic recording, the setting unit sets another of the input sources to start the automatic recording at the same time. The operating room system of claim 1 .

15. The setting unit sets whether or not to execute the automatic recording of the image for each of the input sources depending on the purpose of the image to be recorded. The operating room system of claim 1 .

16. a display unit that displays an integrated screen in which a plurality of the images from a plurality of the input sources are integrated; The setting unit sets whether to perform the automatic recording of the integrated screen or the automatic recording of each image. The operating room system of claim 1 .

17. The operating room system setting whether to automatically record the images for each of a plurality of input sources provided in the operating room for inputting the images; the automatic recording includes pre-recording, which starts recording the image a predetermined time before a recording start instruction is issued; The buffer recording time for the pre-recording is changed depending on the recording destination of the image. Image recording method.

18. On the computer, setting whether to automatically record the images for each of a plurality of input sources provided in the operating room for inputting the images; the automatic recording includes pre-recording, which starts recording the image a predetermined time before a recording start instruction is issued; The buffer recording time for the pre-recording is changed depending on the recording destination of the image. A program for executing a process.

19. A setting section for setting whether or not to automatically record information about a patient for each of a plurality of video sources that output the information about the patient. Equipped with the automatic recording includes pre-recording, which starts recording the information a predetermined time before a recording start instruction is issued; The setting unit changes the buffer recording time for the pre-recording in accordance with the recording destination of the information. Medical information processing system.

Images