Surgery support system and operation method of the support system

Abstract

To facilitate confirmation of information relating to an event occurring during a surgical operation.SOLUTION: A surgery support system includes an acquisition unit, a detection unit and a generation unit. The acquisition unit acquires medical information of a subject during a surgical operation. The detection unit detects any anomaly on the basis of the acquired medical information of the subject. The generation unit generates anomaly information by associating the time when detecting the anomaly with the medical information acquired at the time.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The embodiments disclosed in the present specification and drawings include a surgical assistance system and and how the assistance system works Regarding the law. [Background technology]

[0002] Conventionally, various surgical support systems used during surgery are known. For example, in laparoscopic surgery, a surgical support system is known that generates a virtual endoscopic image from a computed tomography (CT) image taken before surgery to avoid damaging blood vessels and organs, and displays the virtual endoscopic image in conjunction with the actual endoscopic image during surgery. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2011-036371 A [Patent Document 2] JP 2016-182228 A [Patent Document 3] Special Publication No. 2017-006337 Summary of the Invention [Problem to be solved by the invention]

[0004] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to facilitate confirmation of information regarding events occurring during surgery. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0005] A surgery support system according to an embodiment includes an acquisition unit, a detection unit, and a generation unit. The acquisition unit acquires medical information of a subject during surgery. The detection unit detects an abnormality based on the acquired medical information of the subject. The generation unit generates abnormality information that associates a time at which the abnormality is detected with the medical information acquired at that time. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a surgery support system according to the first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of display control by the control function according to the first embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of display control by the control function according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of display control by the control function according to the first embodiment. [Diagram 5] FIG. 5 is a flowchart for explaining a procedure of processing of the surgery support device according to the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, the embodiments of the surgery support system and the surgery support method will be described in detail with reference to the drawings. Note that the surgery support system and the surgery support method according to the present application are not limited to the embodiments shown below. In addition, the embodiments can be combined with other embodiments or conventional techniques as long as there is no contradiction in the processing content.

[0008] (First embodiment) Fig. 1 is a diagram showing an example of the configuration of a surgery support system 10 according to the first embodiment. Here, in Fig. 1, a surgery support system 10 including a surgery support device that executes surgery support according to the present application is described, but the embodiment is not limited thereto, and the surgery support method described below may be executed by any device in the surgery support system 10.

[0009] For example, as shown in FIG. 1, a surgery support system 10 according to this embodiment includes a medical image diagnostic device 1, an endoscope system 2, a virtualized laparoscopic image system 3, a position sensor 4, and a surgery support device 5. Here, each device and system is communicably connected via a network. In addition, in the first embodiment, a case where laparoscopic surgery is performed as the surgery is described as an example, but the surgery is not limited to this, and the present invention may be applied to other surgeries. In addition, the surgery support system 10 may include a system (e.g., HIS (Hospital Information System)) or device (e.g., image storage device) other than those shown in the figure.

[0010] The medical image diagnostic device 1 captures an image of a subject to collect medical images. Then, the medical image diagnostic device 1 transmits the collected medical images to a virtualized laparoscopic image system 3, a surgery support device 5, or the like. For example, the medical image diagnostic device 1 is an X-ray diagnostic device, an X-ray CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, an ultrasonic diagnostic device, a SPECT (Single Photon Emission Computed Tomography) device, a PET (Positron Emission computed Tomography) device, or the like.

[0011] The medical image diagnostic device 1 collects medical images of a subject undergoing surgery. Specifically, the medical image diagnostic device 1 collects medical images of a site to be the target of surgery before and after surgery. Then, the medical image diagnostic device 1 transmits the collected medical images to a virtualized laparoscopic image system 3, a surgery support device 5, or the like.

[0012] The endoscope system 2 has an endoscope 21, a display 22, a processing circuit 23, and a memory circuit 24. The endoscope 21 includes an insertion section that is inserted into the subject, and an operation section that operates the insertion section. The insertion section is a treatment section that performs treatment on a surgical target site (affected area) in the subject, and an imaging section that images the inside of the subject. The operation section accepts operations by the surgeon on the treatment section and the imaging section.

[0013] The treatment unit is, for example, a forceps, an electric scalpel, a suture instrument, etc. The imaging unit has an imaging element such as a CCD (Charge Coupled Device) type image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, a lens, and a light emitting unit, and captures an image of the affected area irradiated with light from the light emitting unit using the imaging element.

[0014] The display 22 displays the video (endoscopic image) captured by the imaging unit. The processing circuitry 23 is connected to the endoscope 21, the display 22, and the memory circuitry 24, and controls the entire endoscope system. For example, the processing circuitry 23 controls the operation of the treatment unit in the endoscope 21, the collection of endoscopic images by the imaging unit, the display of the endoscopic images by the display 22, the storage of the endoscopic images in the memory circuitry 24, the endoscopic images, etc. The memory circuitry 24 stores the endoscopic images 241 collected by the imaging unit of the endoscope 21.

[0015] For example, a surgeon such as a doctor inserts an endoscope 21 having a treatment section such as forceps or an electric scalpel as an insertion section, or an endoscope 21 having an imaging section as an insertion section, into a subject, and performs treatment on the surgical target area (affected area) within the subject by operating the treatment section while observing the endoscopic images collected by the imaging section and displayed on the display 22.

[0016] The virtualized laparoscopic image system 3 includes a display 31, a processing circuit 32, and a memory circuit 33. The display 31 displays an image generated by the processing circuit 32. Specifically, the display 31 displays a virtualized laparoscopic image generated based on a medical image collected by the medical image diagnostic device 1.

[0017] The processing circuitry 32 is connected to the display 31 and the storage circuitry 33, and controls the entire virtualized laparoscopic image system. Specifically, the processing circuitry 32 controls acquisition of medical images from the medical image diagnostic device 1, display of the virtualized laparoscopic images by the display 22, storage of the virtualized laparoscopic images in the storage circuitry 33, and the like. The processing circuitry 32 also generates a virtualized laparoscopic image using a medical image by executing a generation function 321. For example, the generation function 321 generates a virtualized laparoscopic image based on a three-dimensional CT image collected from the abdomen of a subject before surgery by an X-ray CT device, which is the medical image diagnostic device 1.

[0018] For example, the generating function 321 generates a virtualized laparoscopic image in which the inside of the abdominal cavity is projected from a predetermined line of sight based on information of a site inside the abdominal cavity included in a two-dimensional CT image generated using a three-dimensional CT image. The memory circuitry 33 stores the virtualized laparoscopic image 331 generated by the processing circuitry 32.

[0019] The position sensor 4 has a sensor unit, a magnetic field generating unit, and a signal receiving unit. The sensor unit is, for example, a magnetic sensor, and is disposed at the tip of the insertion portion of the endoscope 21 or inside the subject. The magnetic field generating unit is disposed near the subject, and forms a magnetic field that extends outward from the device itself as a center. The signal receiving unit receives the signal output by the sensor unit.

[0020] The sensor unit detects the three-dimensional magnetic field formed by the magnetic field generating unit. Then, based on the information of the detected three-dimensional magnetic field, the sensor unit calculates position information (coordinates and angle) of the device itself in a space with the magnetic field generating unit as the origin, and transmits the calculated position information of the device itself to the signal receiving unit. For example, position information received from a sensor unit attached to the tip of an insertion unit in the endoscope 21 indicates the position of the tip of the insertion unit in a space with the magnetic field generating unit as the origin. Furthermore, position information received from a sensor unit placed inside the subject (for example, an affected organ) indicates the position of the affected area in a space with the magnetic field generating unit as the origin. The signal receiving unit transmits the position information received from the sensor unit to the virtualized laparoscopic image system 3 and the surgery support device 5.

[0021] Here, the virtualized laparoscopic image system 3 can generate and display a virtualized laparoscopic image linked to an endoscopic image by using the position information acquired by the above-mentioned position sensor 4. In such a case, first, the three-dimensional coordinates of a space having the magnetic field generating unit as the origin are aligned with the three-dimensional coordinates of a three-dimensional medical image used to generate the virtualized laparoscopic image.

[0022] For example, the generating function 321 extracts a position (position of the organ in which the sensor unit is placed) in the three-dimensional medical image corresponding to position information acquired by a sensor unit placed in the subject (for example, an affected organ), and performs registration so that the extracted position is the same as the position acquired by the sensor unit. Here, the generating function 321 performs the above-mentioned registration for position information from sensor units placed at multiple positions in the subject, thereby performing registration between the three-dimensional coordinates of the space with the magnetic field generating unit as the origin and the three-dimensional coordinates in the three-dimensional medical image used to generate the virtualized laparoscopic image.

[0023] The alignment between the three-dimensional coordinates of the space with the magnetic field generating unit as the origin and the three-dimensional coordinates of the three-dimensional medical image used to generate the virtualized laparoscopic image is not limited to the above-mentioned method, and other methods may be used. For example, position information of a part in the subject depicted in an endoscopic image captured by an imaging unit of the endoscope 21 to which the sensor unit is attached may be used.

[0024] In such a case, for example, the generating function 321 calculates the three-dimensional coordinates of a part (such as a characteristic part) depicted in the endoscopic image in a space with the magnetic field generating unit as the origin based on position information from a sensor unit attached to the imaging unit. Then, the generating function 321 extracts a position in the three-dimensional medical image corresponding to the part whose three-dimensional coordinates have been calculated, and performs registration so that the extracted position and the position whose three-dimensional coordinates have been calculated are the same position. Here, the generating function 321 performs the above-mentioned registration for a plurality of positions in the subject, thereby performing registration between the three-dimensional coordinates in the space with the magnetic field generating unit as the origin and the three-dimensional coordinates in the three-dimensional medical image used to generate the virtualized laparoscopic image.

[0025] When the alignment is performed in this manner, the generation function 321 generates a virtualized laparoscopic image linked to the endoscopic image based on the position information of the sensor unit attached to the imaging unit of the endoscope 21. For example, the generation function 321 generates a virtualized laparoscopic image projected inside the three-dimensional CT image (inside the abdominal cavity) with the three-dimensional coordinates acquired by the sensor unit attached to the imaging unit of the endoscope 21 as the viewpoint and the imaging direction derived from the angle of the sensor unit as the projection direction.

[0026] Then, the generation function 321 sequentially generates virtualized laparoscopic images with the viewpoint and projection direction changed according to the change in the position of the imaging unit (change in the three-dimensional coordinates and angle acquired by the sensor unit). By sequentially displaying the virtualized laparoscopic images thus generated, the virtualized laparoscopic images linked to the change in the endoscopic image are displayed.

[0027] Here, the generating function 321 can reflect the change in organ shape during surgery in the virtualized laparoscopic image by using position information acquired by a sensor unit placed inside the subject (for example, an affected organ). For example, the generating function 321 calculates the amount of change each time the position information acquired by the sensor unit placed on the organ changes, and adds the calculated amount of change to the corresponding position in the three-dimensional medical image. The generating function 321 then generates a virtualized laparoscopic image using the medical image after the change, thereby reflecting the change in organ shape during surgery in the virtualized laparoscopic image.

[0028] The surgery support device 5 generates information about abnormalities during surgery based on various types of information during surgery. Specifically, the surgery support device 5 acquires information from a medical imaging device and various medical devices during surgery, and generates information about abnormalities during surgery based on the acquired information. For example, the surgery support device 5 is realized by a computer device such as a workstation, a personal computer, or a tablet terminal.

[0029] For example, the surgery support device 5 includes an input interface 51, a display 52, a memory circuitry 53, and a processing circuitry 54. The surgery support device 5 is connected to a medical image diagnostic device 1, an endoscope system 2, a virtualized laparoscopic image system 3, and a position sensor 4 via a network.

[0030] The input interface 51 accepts input operations of various instructions and various information from a user. Specifically, the input interface 51 is connected to the processing circuit 54, converts the input operation received from the user into an electrical signal, and outputs the electrical signal to the processing circuit 54. For example, the input interface 51 is realized by a trackball, a switch button, a mouse, a keyboard, a touch pad that performs an input operation by touching the operation surface, a touch screen in which a display screen and a touch pad are integrated, a non-contact input interface using an optical sensor, a voice input interface, and the like. Note that in this specification, the input interface 51 is not limited to only those that have physical operation parts such as a mouse and a keyboard. For example, an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs the electrical signal to a control circuit is also included as an example of the input interface 51.

[0031] Display 154 displays various information and data. Specifically, display 154 is connected to processing circuit 155, and displays various information and data output from processing circuit 155. For example, display 154 is realized by a liquid crystal display, a CRT (Cathode Ray Tube) display, an organic EL display, a plasma display, a touch panel, or the like.

[0032] The memory circuitry 53 stores various data and various programs. Specifically, the memory circuitry 53 is connected to the processing circuitry 54 and stores data input from the processing circuitry 54, or reads out stored data and outputs it to the processing circuitry 54. For example, the memory circuitry 53 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, a hard disk, an optical disk, or the like.

[0033] For example, the storage circuitry 53 stores a judgment condition 531 and abnormality information 532. The judgment condition 531 and the abnormality information 532 will be described in detail later.

[0034] The processing circuitry 54 controls the entire surgery support device 5. For example, the processing circuitry 54 performs various processes in response to an input operation received from a user via the input interface 51. For example, the processing circuitry 54 stores data transmitted from another device in the memory circuitry 53. Also, for example, the processing circuitry 54 transmits data read from the memory circuitry 53 to another device by outputting the data. Also, for example, the processing circuitry 54 displays the data read from the memory circuitry 53 on the display 52.

[0035] Here, each processing circuit in the above-mentioned endoscopic system 2, virtualized laparoscopic image system 3, and surgery support device 5 is realized, for example, by a processor. In this case, each of the above-mentioned processing functions is stored in a storage circuit in the form of a program executable by a computer. Then, each processing circuit realizes a function corresponding to each program by reading and executing each program stored in each storage circuit. In other words, each processing circuit has each processing function shown in FIG. 1 in a state where each program is read.

[0036] Each processing circuit may be configured by combining multiple independent processors, and each processor may execute a program to realize each processing function. Each processing function of each processing circuit may be appropriately distributed or integrated into a single or multiple processing circuits. Each processing function of each processing circuit may be realized by a combination of hardware and software such as a circuit. Here, an example in which a program corresponding to each processing function is stored in a single storage circuit has been described, but the embodiment is not limited to this. For example, a configuration in which a program corresponding to each processing function is distributed and stored in multiple storage circuits, and a processing circuit reads out and executes each program from each storage circuit may be used.

[0037] An example of the configuration of the surgery support system 10 according to this embodiment has been described above. For example, the surgery support system 10 according to this embodiment is installed in an operating room of a medical facility such as a hospital or clinic, and supports confirmation of events that occur during surgery performed by a user such as a doctor.

[0038] For example, in laparoscopic surgery, bleeding can occur without the surgeon noticing. For example, when an instrument comes into contact with a blood vessel, when a membrane is peeled off with an electric scalpel and a blood vessel deep inside is damaged, when the force applied to the organ with forceps becomes too strong and a tear occurs in another place where pressure was applied, or when a blood vessel is held with forceps and pulled slightly, a tear occurs in another place that was weak.

[0039] When such bleeding occurs, the bleeding point must be identified and stopped, and the endoscopic surgery must continue, but bleeding impairs the endoscopic field of view, making it difficult to identify the bleeding point. Even if the surgeon rewinds the endoscopic image to check the bleeding point, it may be difficult and time-consuming to identify the bleeding point from the image because the surgeon does not know when and where the operation was performed. Furthermore, although the virtual endoscopic image can grasp the internal organ structure outside the field of view, abnormalities such as bleeding events that occur during surgery are not reflected in the virtual endoscopic image, so it is difficult to identify the bleeding point using the virtual endoscopic image.

[0040] In this way, if it takes time to identify the bleeding site and the bleeding cannot be stopped, the surgery will have to be switched to open surgery and the operation will continue, but this can result in a larger wound and the patient being put in a dangerous condition due to the time it takes to switch to open surgery, which places a heavy burden on the patient.

[0041] Therefore, the surgical support device 5 in the surgical support system 10 of this embodiment is configured to acquire various information during surgery and generate information that corresponds the acquired information with time information, thereby making it easy to check information regarding events that occur during surgery.

[0042] Specifically, the surgery support device 5 continues to acquire information from the medical image diagnostic device 1, the endoscope system 2, and various other medical devices used during surgery, analyzes the acquired information, and stores information determined to be abnormal in association with the acquisition time. In this way, when an event such as bleeding occurs during surgery, the surgery support system 10 can present information determined to be abnormal, making it easy to check information related to events that occur during surgery. The surgery support device 5 having such a configuration will be described in detail below.

[0043] For example, as shown in FIG. 1, in this embodiment, the processing circuitry 54 of the surgery support device 5 executes a control function 541, an analysis function 542, and a generation function 543. Here, the control function 541 is an example of an acquisition unit and a display control unit. The analysis function 542 is an example of a detection unit. The generation function 543 is an example of a generation unit.

[0044] The control function 541 acquires various data (medical information) from other devices connected via the network, and stores the acquired medical information in the memory circuitry 53. For example, the control function 541 acquires medical images collected by the medical image diagnostic device 1, endoscopic images generated by the endoscope system 2, virtualized laparoscopic images generated by the virtualized laparoscopic image system 3, and the like.

[0045] Here, the control function 541 can acquire medical information of the subject before and during surgery. For example, the control function 541 acquires medical images collected before surgery and medical images collected during surgery. The control function 541 also acquires endoscopic images during surgery. The control function 541 also acquires virtualized laparoscopic images generated before surgery and virtualized laparoscopic images generated during surgery.

[0046] Furthermore, the control function 541 can acquire various medical information acquired from the subject during surgery. For example, the control function 541 acquires vital field information acquired from the subject during surgery, position information acquired by the position sensor 4, or various information acquired by various sensors attached to the endoscope 21. Here, examples of the various sensors attached to the endoscope 21 include a MEMS (Micro Electro Mechanical Systems) sensor that acquires pressure information in forceps. When the above-mentioned MEMS sensor is attached to the endoscope 21, the control function 541 can also acquire pressure information acquired by the MEMS sensor.

[0047] Furthermore, the control function 541 causes various medical information to be displayed on the display 52. ​​For example, the control function 541 causes information generated based on the analysis result of the acquired medical information to be displayed on the display 52. ​​Here, the control function 541 can also control so that the generated information is transmitted to another device via a network and displayed on the display of the other device. For example, the control function 541 transmits the generated information to the endoscope system 2 and the virtualized laparoscopic image system 3. The endoscope system 2 and the virtualized laparoscopic image system 3 cause the information received from the surgery support device 5 to be displayed on the display of their own device.

[0048] The analysis function 542 detects an abnormality based on the acquired medical information of the subject. Specifically, the analysis function 542 compares the medical information acquired by the control function 541 with the judgment condition 531 stored in the memory circuitry 53, and detects information that matches the judgment condition among the acquired medical information as an abnormality. For example, the analysis function 542 detects an abnormality during surgery based on the medical image acquired from the medical image diagnostic device 1, the endoscopic image acquired from the endoscope system 2, vital information, and the like.

[0049] Here, the judgment conditions 531 include various conditions corresponding to the acquired medical information. For example, judgment conditions corresponding to the analysis using the endoscopic image include "whether the treatment part touches the blood vessel," "the degree of penetration of the electric scalpel into the tissue," "the time during which the blood vessel is pinched by the forceps," "the time during which the organ is held by the forceps," and "the degree of deformation of the organ."

[0050] For example, "whether or not the treatment part touches the blood vessel" in the judgment condition 531 indicates a condition that classifies in stages whether or not the treatment part of the endoscope 21 touches the blood vessel and further the degree of contact. Note that the degree of contact may be classified, for example, by the amount of movement of the treatment tool. As an example, the degree of contact is classified so that the greater the amount of movement of the treatment tool, the greater the degree of contact.

[0051] The analysis function 542 judges whether or not the treatment part has come into contact with a blood vessel by image analysis of each endoscopic image sequentially acquired from the endoscope system 2. Then, when the analysis function 542 judges that the treatment part has come into contact with a blood vessel, it calculates the amount of movement of the treatment part from the previous endoscopic image in time series, and classifies the degree of contact by comparing the calculated amount of movement with the classification in the judgment condition 531. Here, the analysis function 542 detects, for example, a case where the treatment part has come into contact with a blood vessel as an abnormality, and judges the classified degree of contact as a risk level. For example, the analysis function 542 judges that the risk level is higher as the degree of contact is greater.

[0052] Furthermore, the "degree of penetration of the electric scalpel into tissue" in the judgment condition 531 indicates a condition that classifies the degree of penetration of the electric scalpel in stages. The degree of penetration may be classified, for example, by the movement amount of the electric scalpel. As an example, the classification is performed such that the greater the movement amount of the electric scalpel, the greater the degree of penetration. The analysis function 542 calculates the movement amount of the electric scalpel from the previous endoscope image in time series by image analysis of each endoscope image sequentially acquired from the endoscope system 2, and classifies the degree of penetration by comparing the calculated movement amount with the classification in the judgment condition 531. Here, the analysis function 542 detects a case where a predetermined degree of penetration is exceeded as an abnormality, and judges the classified degree of penetration as a risk level. For example, the analysis function 542 judges that the risk level is higher as the degree of penetration is greater.

[0053] Furthermore, "time during which blood vessel is pinched with forceps" and "time during which organ is held with forceps" in the judgment condition 531 indicate conditions that classify the length of time in stages. The analysis function 542 calculates "time during which blood vessel is pinched with forceps" and "time during which organ is held with forceps" by image analysis of each endoscopic image sequentially acquired from the endoscope system 2, and classifies the calculated time by comparing the calculated time with the classification in the judgment condition 531. Here, the analysis function 542 detects that the calculated time exceeds a predetermined time as an abnormality, and judges the length of the time as a risk level. For example, the analysis function 542 judges that the risk level is higher as the calculated time is longer.

[0054] Moreover, the "degree of organ deformation" in the judgment condition 531 indicates a condition for classifying the degree of organ deformation in stages. The degree of organ deformation may be classified, for example, according to the amount of change in the shape of the organ. For example, the degree of deformation is classified so that the greater the amount of change in the shape of the organ, the greater the degree of deformation. The analysis function 542 calculates the amount of change in the shape of the organ from the previous endoscope image in time series by image analysis of each endoscope image sequentially acquired from the endoscope system 2, and classifies the degree of deformation by comparing the calculated amount of change with the classification in the judgment condition 531. Here, the analysis function 542 detects a case where a predetermined amount of change is exceeded as an abnormality, and judges the classified degree of deformation as a risk level. For example, the analysis function 542 judges that the risk level is higher as the degree of deformation is greater.

[0055] The deformation of an organ occurs, for example, when the organ is held down or pulled by forceps, or when the insertion portion of the endoscope 21 inserted into the body presses against the organ.

[0056] The image analysis described above may be performed by feature detection using AI (Artificial Intelligence). In the above example, the movement amount of the treatment part and the change amount of the organ are calculated by image analysis. However, the embodiment is not limited to this. For example, when a position sensor 4 is used, the position information acquired by the position sensor 4 may be used. As an example, the analysis function 542 calculates the movement amount of the treatment part based on the position information acquired by a sensor unit attached to the tip of the treatment part. Also, for example, the analysis function 542 calculates the deformation amount of the organ based on the position information acquired by a sensor unit placed on the affected organ.

[0057] Next, for example, an example of a judgment condition corresponding to an analysis using a medical image is "presence or absence of blood leakage detected by color Doppler imaging." For example, "presence or absence of blood leakage detected by color Doppler imaging" in judgment condition 531 indicates whether blood leakage has occurred and further indicates a condition that classifies the degree of blood leakage into stages.

[0058] For example, the analysis function 542 determines the presence or absence of blood leakage by image analysis of color Doppler images sequentially acquired from an ultrasound diagnostic device, which is the medical image diagnostic device 1. Then, the analysis function 542 detects a case where it is determined that there is blood leakage as an abnormality, and determines the degree of blood leakage as a risk level according to the determination condition 531. For example, the analysis function 542 determines that the risk level is higher as the degree of blood leakage is greater.

[0059] Furthermore, for example, judgment conditions corresponding to an analysis using vital information include conditions such as "drop in blood pressure," "abnormal electrocardiogram," "ischemia," etc. For example, "drop in blood pressure" in judgment condition 531 indicates whether or not blood pressure has fallen below a predetermined value, and further indicates a condition in which the degree of blood pressure drop is classified into stages.

[0060] For example, the control function 541 acquires blood pressure information of the subject from a blood pressure monitor during surgery. The analysis function 542 judges whether the blood pressure has fallen below a predetermined value based on the blood pressure information sequentially acquired from the blood pressure monitor. The analysis function 542 then detects the case where it is determined that the blood pressure has fallen below the predetermined value as an abnormality, and judges the degree of the drop in blood pressure as the risk level according to the judgment condition 531. For example, the analysis function 542 judges that the risk level is higher as the degree of the drop in blood pressure is greater.

[0061] Also, for example, "abnormal electrocardiogram" in the judgment condition 531 indicates a condition of whether or not the rhythm or waveform in the electrocardiogram has changed beyond a predetermined amount of change, and further indicates a condition of classifying the degree of change into stages.

[0062] For example, the control function 541 acquires an electrocardiogram of the subject from an electrocardiograph during surgery. The analysis function 542 judges whether or not the rhythm or waveform in the electrocardiogram has changed beyond a predetermined amount of change based on the electrocardiograms sequentially acquired from the electrocardiograph. The analysis function 542 detects as an abnormality a case in which the rhythm or waveform in the electrocardiogram has changed beyond a predetermined amount of change, and judges the degree of the amount of change as a risk level according to the judgment condition 531. For example, the analysis function 542 judges that the risk level is higher as the amount of change is larger.

[0063] Furthermore, for example, "ischemia" in the judgment condition 531 indicates whether ischemia has occurred and further indicates a condition that classifies the degree of ischemia into stages. The analysis function 542 judges whether ischemia has occurred based on the electrocardiogram waveforms sequentially acquired from the electrocardiograph. Then, the analysis function 542 detects the occurrence of ischemia as an abnormality, and judges the degree of ischemia as a risk level according to the judgment condition 531. For example, the analysis function 542 judges that the risk level is higher as the degree of ischemia is greater.

[0064] As described above, the analysis function 542 detects an abnormality based on medical information acquired during surgery and determines the risk level of the detected abnormality. Here, the above-mentioned determination conditions are merely examples, and an abnormality may be detected based on other information. For example, when a MEMS sensor is attached to the treatment section and pressure information is acquired by the MEMS sensor, an abnormality may be detected based on the acquired pressure information.

[0065] In such a case, a condition related to the pressure information is stored as the judgment condition 531. For example, "pressure" in the judgment condition 531 indicates whether or not the acquired pressure value exceeds a predetermined value, and further indicates a condition in which the degree of pressure is classified into stages.

[0066] For example, the control function 541 acquires pressure information acquired by a MEMS sensor. The analysis function 542 judges whether or not the pressure exceeds a predetermined value based on the pressure information acquired by the control function 541. Then, the analysis function 542 detects a case where it is judged that the pressure exceeds the predetermined value as an abnormality, and judges the level of the pressure as a risk level according to the judgment condition 531. For example, the analysis function 542 judges that the risk level is higher as the pressure value is larger.

[0067] An example of analysis by the analysis function 542 has been described above. In the above example, a case where an abnormality is detected based on one condition has been described, but the embodiment is not limited to this, and an abnormality may be detected by combining multiple conditions. For example, an abnormality may be detected by combining multiple conditions (e.g., the presence or absence of contact with a blood vessel and the degree of deformation of an organ, etc.) included in an analysis by a single device (e.g., an analysis using an endoscopic image) or an abnormality may be detected by combining conditions of an analysis by multiple devices (e.g., an analysis using an endoscopic image and an analysis using vital information). In addition, various conditions of the judgment condition 531 referred to by the analysis function 542 can be set arbitrarily. For example, they can be set appropriately according to the procedure, the type of the affected organ, etc.

[0068] The generating function 543 generates anomaly information that associates the time when an abnormality is detected with the medical information acquired at that time. Specifically, the generating function 543 generates anomaly information that associates the medical information when an abnormality is detected by the analyzing function 542 with the time when the medical information is acquired. For example, when an abnormality is detected in an analysis based on the "degree of organ deformation", the generating function 543 generates anomaly information that associates an endoscopic image in which an abnormality is detected with the time when the endoscopic image was captured. Note that the time when the endoscopic image was captured is acquired simultaneously when the control function 541 acquires the endoscopic image.

[0069] Here, the generating function 543 may generate abnormality information in which only the medical information in which the abnormality was detected is associated with the time, or may generate abnormality information in which other medical information at the time when the medical information in which the abnormality was detected was acquired is further associated. For example, when vital information is acquired together with an endoscopic image, and an abnormality is detected in an analysis based on the "degree of organ deformation" by the endoscopic image, the generating function 543 can generate abnormality information in which the vital information acquired at the same time is further associated. That is, the generating function 543 generates abnormality information in which the vital information at the time when the endoscopic image in which the abnormality was detected was captured is further associated.

[0070] Furthermore, the generating function 543 can also associate a risk level with the abnormality information that associates the time when the medical information in which the abnormality was detected was acquired with the medical information acquired at that time. For example, the generating function 543 further associates the risk level determined in the analysis based on the "degree of organ deformation" with the abnormality information.

[0071] Furthermore, the generating function 543 can also generate abnormality information associated with position information. Specifically, the generating function 543 generates abnormality information that associates the time when the abnormality is detected, the medical information acquired at that time, and the position information acquired at that time. In such a case, first, the sensor unit of the position sensor 4 is attached to the tip of the insertion part of the endoscope 21, and position information of the tip of the insertion part during surgery is acquired. The control function 541 acquires the position information acquired by the position sensor 4 from the position sensor 4, and stores the acquired position information in the memory circuitry 53 in association with the time when the position information was acquired.

[0072] The generating function 543 obtains from the memory circuitry 53 the position information of the time when the medical information was acquired when the abnormality was detected by the analyzing function 542, and generates abnormality information that associates the time when the medical information at which the abnormality was detected was acquired with the medical information acquired at that time. For example, when an abnormality is detected in the analysis based on the "degree of deformation of an organ", the generating function 543 generates abnormality information that associates the endoscopic image in which the abnormality was detected, the time when the endoscopic image was captured, and the position information at that time.

[0073] The information associated with the abnormality information can be set as appropriate. For example, the generating function 543 associates the time, medical information, analysis results, and position information as the abnormality information according to the information acquired during the operation.

[0074] During surgery, each time an abnormality is detected by the analysis function 542, the generation function 543 generates the abnormality information described above and stores it in the memory circuitry 53. The abnormality information 532 in the memory circuitry 53 is generated and stored by the generation function 543 as described above. Note that the abnormality information 532 may be stored for each surgery and read out and used after the surgery.

[0075] When the abnormality information is generated as described above, the control function 541 causes various pieces of information using the abnormality information to be displayed on the display 52. ​​In addition, the control function 541 can also cause various pieces of information using the abnormality information to be displayed on the display 22 in the endoscope system 2 and the display 31 in the virtualized laparoscopic image system 3.

[0076] An example of information displayed by the control function 541 will now be described.

[0077] For example, the control function 541 displays timeline information that shows abnormality information in chronological order. Fig. 2 is a diagram showing an example of display control by the control function 541 according to the first embodiment. Here, Fig. 2 shows a display of timeline information for an endoscopic image displayed by the endoscopic system 2.

[0078] For example, as shown in the upper diagram of Fig. 2, in the endoscope system 2, a real-time video of the intraperitoneal cavity is displayed on the display 22 based on an endoscopic image acquired during surgery. During this time, the surgery support device 5 acquires various medical information and determines whether or not an abnormality has occurred.

[0079] Here, when the analysis function 542 detects an abnormality and the generation function 543 generates abnormality information, the control function 541 controls the display of timeline information 101 on the endoscopic image based on the generated abnormality information, as shown in the middle diagram of Figure 2.

[0080] Here, the control function 541 displays identifiable information regarding at least one of the degree of the detected abnormality and the detection means, as shown in position 1 of the timeline information 101. For example, the control function 541 displays position 1 of the timeline information in a color corresponding to the risk level of the detected abnormality. Also, for example, the control function 541 displays position 1 of the timeline information in a color corresponding to the detection means (for example, the conditions used for analysis) that detected the abnormality. Here, when the risk level and the detection means are made identifiable, the control function 541 assigns, for example, the color of the outer frame and the inner color of position 1 on the timeline information 101 to the detection means and the risk level. As an example, the control function 541 displays the color of the outer frame indicating position 1 in a color corresponding to the detection means, and displays the color inside the frame indicating position 1 in a color corresponding to the risk level.

[0081] Furthermore, the control function 541 can change the display according to a change in the risk level. As described above, the analysis function 542 detects an abnormality based on medical information sequentially acquired by the control function 541. Therefore, when an abnormality is continuously detected in the medical information sequentially acquired, the abnormality information is continuously generated, and the control function 541 causes identifiable information to be continuously displayed on the timeline information.

[0082] Here, for example, when the degree of deformation of an organ gradually changes, the risk level determined by the analysis function 542 gradually changes. In such a case, the risk level associated with the abnormality information generated by the generation function 543 changes. As a result, for example, the control function 541 changes the color at position 1 in the timeline information 101 and displays it, as shown in the lower part of FIG.

[0083] For example, the control function 541 does not display the timeline information 101 as shown in the upper part of Fig. 2 until anomaly information is generated, and when the anomaly information is generated, automatically displays the timeline information 101 as shown in the middle and lower parts of Fig. 2. Then, the control function 541 can also perform control so that the timeline information is not displayed when anomaly information is no longer generated.

[0084] Also, for example, the control function 541 can display intraoperative medical images collected at the time an abnormality is detected in association with the corresponding time in the timeline information. Fig. 3 is a diagram showing an example of display control by the control function 541 according to the first embodiment. Here, Fig. 3 shows the display of timeline information for an endoscopic image displayed by the endoscopic system 2.

[0085] For example, the control function 541, in response to an operation by the operator, displays the timeline information 101 or displays a thumbnail 102 as shown in Fig. 3. As an example, when the operator executes an operation to display the timeline information 101 during surgery, the control function 541 displays the timeline information 101 based on the abnormality information generated up to that point as shown in the middle part of Fig. 3.

[0086] Furthermore, when the operator specifies position 6 in the timeline information 101, the control function 541 causes a thumbnail 102 of the endoscopic image corresponding to the specified position 6 to be displayed in association with position 6, as shown in the lower part of FIG.

[0087] For example, when an event such as bleeding occurs during surgery, if the operator causes timeline information to be displayed, the control function 541 causes timeline information 101 to be displayed that makes it possible to identify the time point at which the abnormality was detected, as shown in the middle part of Fig. 3. This allows the operator to easily identify the time point at which an event that may be the cause of the event occurred.

[0088] Then, when the operator specifies a position in the timeline information 101, the control function 541 displays a thumbnail 102 of the image acquired at the specified time, as shown in the lower part of Fig. 3. This allows the operator to understand the treatment situation when an abnormality occurs, and to quickly identify the cause of an event such as bleeding.

[0089] Also, for example, the control function 541 displays the position of the medical instrument corresponding to the time when the abnormality was detected in the medical image of the subject. That is, when the abnormality information includes position information, the control function 541 displays the medical image showing the position information acquired at the time when the abnormality was detected based on the position information associated with the abnormality information.

[0090] Fig. 4 is a diagram showing an example of display control by the control function 541 according to the first embodiment. Here, the diagram on the left side of Fig. 4 shows an endoscopic image on which timeline information 101 is displayed. Moreover, the diagram on the right side of Fig. 4 shows a medical image on which position information is displayed.

[0091] For example, as shown in Fig. 4, the control function 541 displays the timeline information 101 on the endoscopic image, and also displays a medical image showing the position information of the medical device at the time when the abnormality information was generated. Here, the control function 541 displays the medical image showing the position information by using the three-dimensional medical image aligned with the three-dimensional coordinates of the space with the magnetic field generating unit in the position sensor 4 as the origin.

[0092] For example, the control function 541 displays position information using a virtualized laparoscopic image based on a three-dimensional medical image aligned with three-dimensional coordinates in a space with the magnetic field generating unit as the origin, or a CT image, as shown in the right side of Fig. 4. As an example, the control function 541 specifies positions in the three-dimensional medical image corresponding to each position information based on the position information corresponding to the times of positions 1 to 6 in the timeline information 101 and the alignment information.

[0093] Then, the control function 541 displays a medical image showing identification information at each specified position. For example, as shown in Fig. 4, the control function 541 displays numbers corresponding to positions 1 to 6 in the timeline information 101 at the specified positions on the CT image. Here, the control function 541 can display each number so that the detected risk level and detection means can be identified, similarly to the timeline information 101. For example, the control function 541 displays the risk level and detection means so that they can be identified by the shape of the outer frame surrounding the numbers, the color inside the shape, or the like.

[0094] Furthermore, the control function 541 can display not only the position information of the medical equipment at the time when the medical information in which an abnormality was detected was acquired, but also the current position of the medical instrument (the tip of the endoscope 21) in the medical image of the subject. That is, the control function 541 identifies a position in the three-dimensional medical image that corresponds to the current position of the medical instrument acquired from the position sensor 4. Then, the control function 541 displays a medical image showing the identification information at the identified position. For example, the control function 541 displays the current position 104 of the medical instrument as shown in the diagram on the right side of FIG. 4.

[0095] Furthermore, the control function 541 can display the thumbnail 103 in response to an operation by the operator, as shown in Fig. 4. As an example, when the operator executes an operation to display the timeline information 101 and an operation to specify a position 6 in the timeline information 101 during surgery, the control function 541 reflects the information of the specified position 6 in the virtualized laparoscopic image, as shown in Fig. 4, and displays the thumbnail 103 of the virtualized laparoscopic image corresponding to the time of the position 6 in association with a number.

[0096] For example, when an event such as bleeding occurs during surgery, the control function 541 displays an image showing the position of the medical instrument at the time when the medical information in which the abnormality was detected was acquired, allowing the operator to easily identify the position where the phenomenon that may have caused the event occurred.

[0097] Then, when the operator specifies a position in the timeline information 101, the control function 541 displays a thumbnail 103 of the image at the specified time, as shown in the lower part of Fig. 3. This allows the operator to understand the treatment situation when an abnormality occurs, and to quickly identify the cause of an event such as bleeding.

[0098] Next, the processing of the surgery support device 5 according to the first embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart for explaining the procedure of the processing of the surgery support device 5 according to the first embodiment. Note that Fig. 5 shows an example in which time, medical information, and location information are associated as abnormality information.

[0099] 5 are steps realized by the processing circuitry 54 reading out a program corresponding to the control function 541 from the storage circuitry 53 and executing the program, and step S103 in Fig. 5 is a step realized by the processing circuitry 54 reading out a program corresponding to the analysis function 542 from the storage circuitry 53 and executing the program, and step S104 to S105 in Fig. 5 are steps realized by the processing circuitry 54 reading out a program corresponding to the generation function 543 from the storage circuitry 53 and executing the program, and step S105 in Fig. 5 is a step realized by the processing circuitry 54 reading out a program corresponding to the generation function 543 from the storage circuitry 53 and executing the program, and step S103 in Fig. 5 is a step realized by the processing circuitry 54 reading out a program corresponding to the generation function 543 from the storage circuitry 53 and executing the program, and step S104 to S105 in Fig. 5 are steps ...

[0100] As shown in Fig. 5, in the surgery support device 5, first, the processing circuitry 54 determines whether or not a procedure has been started (step S101). For example, the processing circuitry 54 determines the start of a procedure based on whether or not an operation for starting processing has been received. Here, when the procedure is started (step S101, YES), the processing circuitry 54 acquires medical information (step S102). Note that the surgery support device 5 is in a standby state until the procedure is started (step S101, NO).

[0101] Then, the processing circuitry 54 analyzes the medical information (step S103) and determines whether or not there is an abnormality (step S104). If there is an abnormality (step S104, YES), the processing circuitry 54 generates abnormality information that associates the medical information, time, and position information at which the abnormality was detected, and stores the abnormality information (step S105). On the other hand, if no abnormality is detected in step S104 (step S104, NO), the processing circuitry 54 proceeds to step S106.

[0102] Then, in step S106, the processing circuitry 54 displays medical information. For example, the processing circuitry 54 displays an endoscopic image or the like. Here, when abnormality information is generated, the processing circuitry 54 displays an endoscopic image or the like including timeline information based on the abnormality information. Then, the processing circuitry 54 determines whether or not a designation operation has been received (step S107).

[0103] Here, if a designation operation is received (step S107, YES), the processing circuitry 54 displays detailed information (e.g., thumbnails) of the designated medical information (step S108) and determines whether the procedure has ended (step S109). On the other hand, if a designation operation is not received (step S107, NO), the processing circuitry 54 determines whether the procedure has ended (step S109).

[0104] In step S109, if the procedure has not ended (step S109, negative), the processing circuitry 54 returns to step S102 and continues acquiring medical information. On the other hand, if the procedure has ended (step S109, positive), the processing circuitry 54 ends the process.

[0105] As described above, according to the first embodiment, the control function 541 acquires medical information of a subject during surgery. The analysis function 542 detects an abnormality based on the acquired medical information of the subject. The generation function 543 generates abnormality information that associates the time when the abnormality was detected with the medical information acquired at that time. Therefore, the surgery support device 5 according to the first embodiment can provide information regarding the time when the abnormality occurred, and makes it possible to easily confirm information regarding an event that occurred during surgery.

[0106] For example, even if bleeding occurs during endoscopic surgery, the time when the abnormality was detected can be displayed to quickly identify the bleeding site. As a result, endoscopic surgery can be continued without having to switch to open surgery, contributing to improving the patient's quality of life after surgery.

[0107] According to the first embodiment, the control function 541 displays timeline information showing abnormality information in chronological order. Therefore, the surgery support device 5 according to the first embodiment makes it possible to provide easily observable information about the time when the abnormality was detected.

[0108] According to the first embodiment, the control function 541 displays intraoperative medical images collected at the time when an abnormality is detected in association with the corresponding time in the timeline information. Therefore, the surgery support device 5 according to the first embodiment can present an image at the time when an abnormality is detected, rather than a real-time image, and can, for example, more quickly identify a bleeding site.

[0109] According to the first embodiment, the control function 541 further acquires position information indicating the position of the medical instrument used during surgery. The generation function 543 generates abnormality information that associates the time when the abnormality is detected, the medical information acquired at that time, and the position information acquired at that time. Therefore, the surgery support device 5 according to the first embodiment can further present position information, making it possible to more quickly identify the bleeding site.

[0110] According to the first embodiment, the control function 541 acquires position information indicating the position of the medical instrument operated in the subject during surgery. Therefore, the surgery support device 5 according to the first embodiment makes it possible to acquire the position information of the medical instrument in the subject.

[0111] Moreover, according to the first embodiment, the control function 541 further acquires a medical image of the subject. The control function 541 displays the position of the medical instrument corresponding to the time when the abnormality was detected in the medical image of the subject. Therefore, the surgery support device 5 according to the first embodiment makes it possible to display position information that is easier to observe.

[0112] According to the first embodiment, the control function 541 displays the current position of the medical instrument in the medical image of the subject. Therefore, the surgery support device 5 according to the first embodiment makes it possible to easily grasp the positional relationship between the current position of the medical instrument and the position at the time when the medical information in which an abnormality is detected is acquired.

[0113] According to the first embodiment, the control function 541 displays identifiable information about at least one of the degree of the detected abnormality and the detection means. Therefore, the surgery support device 5 according to the first embodiment makes it possible to grasp the degree of the abnormality and the detection means at a glance.

[0114] (Other embodiments) Although the first embodiment has been described above, the present invention may be embodied in various different forms other than the above-described first embodiment.

[0115] In the above-described embodiment, the surgery support system 10 includes the surgery support device 5, and the surgery support device 5 executes various processes. However, the embodiment is not limited to this, and various processes of the surgery support method according to the present application may be executed independently by any device in the surgery support system 10, or may be executed in a distributed manner by multiple devices in the surgery support system 10.

[0116] For example, the analysis process for each piece of medical information may be executed by a device that has acquired each piece of medical information. For example, the endoscope system 2 may perform the analysis for the above-mentioned endoscopic image. In such a case, the processing circuit 23 executes the above-mentioned analysis function 542 to detect an abnormality based on the endoscopic image. Also, an ultrasound diagnostic device may detect an abnormality based on an ultrasound image, and a device such as an electrocardiograph may detect an abnormality based on vital information.

[0117] Furthermore, the process of generating abnormality information and various display processes related to the abnormality information may be executed in the virtualized laparoscopic image system 3.

[0118] In the above-described embodiment, the surgical support method according to the present application has been described as being applied to laparoscopic surgery. However, the embodiment is not limited thereto, and the surgical support method according to the present application may be applied to various other surgeries.

[0119] In the above-described embodiment, the acquisition unit, detection unit, generation unit, and display control unit in this specification are respectively realized by the control function, analysis function, generation function, and control function of the processing circuit, but the embodiment is not limited to this. For example, the acquisition unit, detection unit, generation unit, and display control unit in this specification may be realized by only hardware, only software, or a combination of hardware and software, in addition to being realized by the control function, analysis function, generation function, and control function described in the embodiment.

[0120] The term "processor" used in the description of the above-mentioned embodiment means a circuit such as a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)). Here, instead of storing a program in a memory circuit, the program may be directly built into the circuit of the processor. In this case, the processor realizes its function by reading and executing the program built into the circuit. Furthermore, each processor in the present embodiment is not limited to being configured as a single circuit for each processor, and may be configured as a single processor by combining multiple independent circuits to realize its function.

[0121] Here, the program executed by the processor is provided in advance in a ROM (Read Only Memory) or a storage circuit. The program may be provided by being recorded in a computer-readable non-transitory storage medium such as a CD (Compact Disk)-ROM, a FD (Flexible Disk), a CD-R (Recordable), or a DVD (Digital Versatile Disk) in a format that can be installed in these devices or in a format that can be executed. The program may also be provided or distributed by being stored on a computer connected to a network such as the Internet and downloaded via the network. For example, the program is composed of modules including each of the above-mentioned processing functions. As actual hardware, a CPU reads out the program from a storage medium such as a ROM and executes it, so that each module is loaded onto a main storage device and generated on the main storage device.

[0122] In addition, in the above-mentioned embodiment and modified examples, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution or integration of each device is not limited to that shown in the figure, and all or a part of it can be functionally or physically distributed or integrated in any unit according to various loads, usage conditions, etc. Furthermore, all or any part of each processing function performed by each device can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0123] Furthermore, among the processes described in the above-mentioned embodiments and modifications, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified.

[0124] According to at least one of the embodiments described above, it is possible to easily check information related to events that occur during surgery.

[0125] Although some embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0126] 2 Endoscope system 3 Virtualized Laparoscopic Imaging System 4 Position Sensor 5 Surgical support equipment 10. Surgical Support System 23, 32, 54 Processing circuit 541 Control Functions 542 Analysis Function 543 Generation function

Claims

1. an acquisition unit that sequentially acquires medical information of a subject during surgery; A detection unit that detects the presence or absence and the degree of an abnormality based on the acquired medical information; a display control unit that displays information identifying a degree of the abnormality at a position corresponding to the time at which the abnormality was detected in display information indicating a time axis based on a detection result by the detection unit, and displays medical information at which the abnormality was detected at a time corresponding to a specified position in the display information; A surgical support system comprising:

2. The surgery support system according to claim 1 , wherein the display control unit displays an intraoperative medical image collected at the time the abnormality was detected in association with a corresponding position of the display information.

3. The acquisition unit further acquires position information representing a position of a medical instrument used during surgery; The surgical support system according to claim 1 or 2, further comprising a generation unit that generates abnormality information that associates the time at which the abnormality was detected, the medical information acquired at that time, and the location information acquired at that time.

4. The surgery support system according to claim 3 , wherein the acquisition unit acquires position information representing a position of a medical instrument operated inside the subject during surgery.

5. The acquisition unit further acquires a medical image of the subject, The surgery support system according to claim 3 , wherein the display control unit causes a position of the medical instrument corresponding to a time when the abnormality was detected to be displayed in the medical image of the subject.

6. The surgery support system according to claim 5 , wherein the display control unit causes a current position of the medical instrument to be displayed in the medical image of the subject.

7. The surgery support system according to any one of claims 1 to 6, wherein the display control unit causes the detection means that detected the abnormality to be identifiable and displayed.

8. A method for operating a support system that supports a confirmation task of information regarding an event that occurred during surgery, comprising: The support system includes an acquisition unit, a detection unit, and a display control unit, The acquiring unit sequentially acquires medical information of a subject during an operation; a step of detecting the presence or absence and the degree of an abnormality based on the acquired medical information by the detection unit; a step in which the display control unit displays information identifying the degree of the abnormality at a position corresponding to the time at which the abnormality was detected in display information indicating a time axis based on the detection result, and displays medical information at which the abnormality was detected at the time corresponding to the specified position in the display information; A method for operating an assistance system, comprising:

Images