Medical device and control method

Abstract

To make a display state of a target portion included in a medical image and appearance of the target portion observed at the time of surgical treatment similar to each other.SOLUTION: A medical device according to an embodiment includes an acquisition part and a change part. The acquisition part acquires appearance information of a target portion observed during procedure in surgical treatment on a subject. The change part changes a display state of the medical image obtained by imaging the target portion on the basis of the appearance information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] SUMMARY OF THE INVENTION The embodiments disclosed herein and in the drawings relate to medical devices and control methods. [Background technology]

[0002] Conventionally, in surgical treatment, doctors carefully observe medical images before surgery and memorize the planning images used to create a treatment plan before starting the procedure. Therefore, various methods have been proposed for displaying medical images for preoperative observation. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 194174 [Patent Document 2] Japanese Patent Application Laid-Open No. 2015-176775 [Patent Document 3] Japanese Patent Application Laid-Open No. 2005-237973 Summary of the Invention [Problem to be solved by the invention]

[0004] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to make the display state of a target area included in a medical image similar to the appearance of the target area observed during surgical treatment. However, the problem to be solved by the embodiments disclosed in this specification and the drawings is not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0005] A medical device according to an embodiment includes an acquisition unit and a modification unit. The acquisition unit acquires appearance information of a target region observed during a procedure in a surgical treatment on a subject. The modification unit modifies a display state of a medical image of the target region based on the appearance information. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a medical device according to the first embodiment. [Figure 2] FIG. 2 is a flowchart illustrating an example of a processing procedure of the medical device according to the first embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of the change process according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of the change process according to the first embodiment. [Figure 5] FIG. 5 is a diagram showing an example of display of a medical image according to the first embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of the change process according to the first embodiment. [Figure 7] FIG. 7 is a diagram for explaining an example of remote surgery according to the first embodiment. [Figure 8] FIG. 8 is a flowchart illustrating an example of a processing procedure of the medical device according to the second embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of device control according to the second embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of device control according to the second embodiment. [Figure 11] FIG. 11 is a diagram showing an example of a display image according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0007] Hereinafter, embodiments of a medical device and a control method will be described in detail with reference to the drawings. Note that the medical device and control method according to the present application are not limited to the embodiments shown below. Furthermore, the embodiments shown below can be combined with other embodiments or conventional techniques to the extent that no inconsistencies arise in the processing content. Furthermore, in the following description, similar components will be given common reference numerals, and duplicated descriptions will be omitted.

[0008] The medical device and control method according to this embodiment execute various processes to make the display state of the target area included in the medical image resemble the appearance of the target area observed during surgical treatment. Here, the appearance of the target area observed during surgical treatment includes the appearance of the target area itself (actual object) observed during open surgery and the appearance of an endoscopic image observed during endoscopic surgery. Note that what is described below as endoscopic surgery also applies to laparoscopic surgery.

[0009] (First embodiment) Fig. 1 is a block diagram showing an example of the configuration of a medical device according to the first embodiment. For example, as shown in Fig. 1, a medical device 40 is connected via a network to each department system, each device installed in an operating room 1, and each device installed in an operating room 2 so as to be able to communicate with each other. Here, the network includes, for example, an in-hospital LAN (Local Area Network) or WAN (Wide Area Network) installed in a hospital.

[0010] It should be noted that various other devices and systems, such as a medical image diagnostic device, may also be connected to the network shown in Fig. 1. A medical image diagnostic device is a device that captures an image of a subject to generate a medical image, and includes, for example, an X-ray CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, an X-ray diagnostic device, an ultrasound diagnostic device, a SPECT (Single Photon Emission Computed Tomography) device, and a PET (Positron Emission Computed Tomography) device.

[0011] 1 shows only operating room 1 and operating room 2, but additional operating rooms may be provided. In such a case, each operating room is provided with devices that are communicably connected to the medical device 40.

[0012] Each department system 10 includes various systems such as a Hospital Information System (HIS), a Radiology Information System (RIS), a diagnostic report system, a Laboratory Information System (LIS), a rehabilitation department system, a dialysis department system, and a surgery department system. The medical device 40 is connected to each of these systems and exchanges various information with them. For example, the medical device 40 exchanges information such as subject information, examination information, treatment information, information on analysis results, and surgery information (surgery information) with each system included in each department system 10. For example, the medical device 40 acquires information about the operating room to be used, the type of surgery (e.g., open surgery, endoscopic surgery), and the equipment used for the surgery from each department system 10 regarding the subject undergoing surgery.

[0013] The operating room 1 is an operating room used for, for example, open surgery, and is equipped with devices such as a shadowless lamp 21, an input device 22, and a display 23. Although not shown, the operating room 1 is also equipped with an operating table, a camera for photographing the surgery, and the like in addition to the above-mentioned devices.

[0014] The shadowless lamp 21 is a device that illuminates the surgical field so that the surgeon performing the surgery can clearly see the surgical field of the surgical target area. For example, the shadowless lamp 21 has multiple light source units, each of which irradiates the surgical field with light from a different position and direction, thereby preventing shadows from appearing in the surgical field. Note that the shadowless lamp 21 may be configured such that each light source unit is composed of a single light source (light bulb), or each light source unit is composed of multiple light sources. The light source may be an LED (Light Emitting Diode), an incandescent bulb, a xenon tube, an organic EL (Electro Luminescence), or the like.

[0015] For example, the shadowless lamp 21 has a control device (not shown) that is configured to be able to individually control the light emission intensity and wavelength of each light source of the light source unit. The shadowless lamp 21 is also configured to be able to arbitrarily change the distance to the operating table, the angle relative to the operating table, and the position relative to the operating table using a drive mechanism driven by the control device. For example, the shadowless lamp 21 controls the light emission intensity and wavelength of each light source, the distance to the operating table, the angle relative to the operating table, and the position relative to the operating table by operating the control device based on a control signal input by the input device 22. The shadowless lamp 21 is also provided with a grip, which allows an operator to manually change the distance to the operating table, the angle relative to the operating table, and the position relative to the operating table.

[0016] The input device 22 inputs control information for the shadowless lamp 21 to the medical apparatus 40 and inputs control signals to the shadowless lamp 21. Specifically, the input device 22 transmits control information to the medical apparatus 40, including the light emission intensity and wavelength of each light source in the shadowless lamp 21, the distance to the operating table, the angle relative to the operating table, and the position relative to the operating table. The control information is input by an operator, for example, via an input interface (not shown) of the input device 22. The input device 22 also transmits to the shadowless lamp 21 control signals based on the control information input by the operator via the input interface (not shown) and control signals based on the control information received from the medical apparatus 40.

[0017] The display 23 displays various types of information and data. For example, the display 23 displays medical images in accordance with display control by the medical device 40. As an example, the display 23 displays medical images including preoperative images, treatment plan images, and simulation images. Note that a preoperative image is an image of a subject captured before surgery, a treatment plan image is an image showing a treatment plan, and a simulation image is an image obtained by performing various simulations. For example, the display 23 is realized by a liquid crystal display, a CRT (Cathode Ray Tube) display, a touch panel, or the like.

[0018] The operating room 2 is, for example, an operating room used for endoscopic surgery, and is equipped with devices such as an endoscope system 31 and an input device 32. Although not shown, the operating room 2 is equipped with an operating table, a camera for photographing the surgery, and the like in addition to the above-mentioned devices. The operating room 2 may also be equipped with a display different from the display 31b of the endoscope system 31, which will be described later.

[0019] The endoscope system 31 includes an endoscope 31a and a display 31b. Although not shown, the endoscope system 31 further includes a processing circuit that executes various processes and a memory circuit that stores various information. The endoscope 31a includes an insertion section that is inserted into the subject and an operation section that operates the insertion section. The insertion section includes a treatment section that performs treatment on a surgical target site (affected area) within the subject, and an imaging section that images the inside of the subject. The operation section accepts operations by the surgeon on the treatment section and imaging section.

[0020] The treatment unit is, for example, a forceps, an electric scalpel, a suture instrument, etc. The imaging unit has an imaging element such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, a lens, and a light emitting unit, and captures an image of the target site irradiated with light from the light emitting unit using the imaging element.

[0021] The display 31b displays the video (endoscopic image) captured by the imaging unit. A processing circuit (not shown) is connected to the endoscope 31a, the display 31b, and a storage circuit (not shown), and controls the entire endoscopic system 31. For example, the processing circuit controls the operation of the treatment unit in the endoscope 31a, the collection of endoscopic images by the imaging unit, the display of the endoscopic images by the display 31b, and the storage of the endoscopic images in the storage circuit. The storage circuit stores the endoscopic images collected by the imaging unit of the endoscope 31a.

[0022] The endoscope system 31 can also generate a virtual endoscopic image from a medical image (e.g., a CT (Computed Tomography) image) captured before surgery and display the virtual endoscopic image on the display 31b in conjunction with an actual endoscopic image during surgery. The endoscope system 31 can also display a medical image received from the medical device 40 on the display 31b.

[0023] The input device 32 inputs control information of the endoscope system 31 to the medical device 40 and inputs control signals to the endoscope system 31. Specifically, the input device 32 transmits control information for the endoscope 31a in the endoscope system 31 to the medical device 40. In this case, for example, the input device 32 acquires the above-mentioned control information from the endoscope system 31. Furthermore, the input device 32 transmits a control signal based on the control information received from the medical device 40 to the endoscope system 31.

[0024] 1, the medical device 40 includes a communication interface 41, an input interface 42, a display 43, a memory circuitry 44, and a processing circuitry 45. For example, the medical device 40 is realized by a computer device such as a server or a workstation. Here, the medical device 40 according to the first embodiment acquires appearance information of a target area during surgical treatment based on device information of devices installed in each operating room, and changes the display state of a medical image of the target area based on the acquired appearance information.

[0025] Specifically, the medical device 40 according to the first embodiment changes the display state of the medical image when observing a preoperative medical image and when observing a medical image during surgery so that the display state resembles the appearance of the target site during surgical treatment. For example, when observing a preoperative medical image (e.g., when formulating a treatment plan while referring to the medical image or when conducting a surgical simulation), the medical device 40 estimates the appearance of the target site that will be observed during surgery based on surgery information and information about the devices installed in the operating room, and changes the display state of the medical image so that the display state resembles the estimated appearance. Furthermore, when observing a medical image during surgery, the medical device 40 estimates the appearance of the target site observed during surgery based on control information and device information about the device during surgery, and changes the display state of the medical image so that the display state resembles the estimated appearance.

[0026] The communication interface 41 controls the transmission and communication of various data sent and received between the medical device 40 and each device connected via a network. Specifically, the communication interface 41 is connected to the processing circuitry 45, and transmits data received from each device on the network to the processing circuitry 45, or transmits data received from the processing circuitry 45 to each device on the network. For example, the communication interface 41 is realized by a network card, a network adapter, a NIC (Network Interface Controller), etc.

[0027] The input interface 42 accepts input operations of various instructions and information from an operator. Specifically, the input interface 42 is connected to the processing circuit 45 and converts the input operations received from the operator into electrical signals and transmits them to the processing circuit 45. For example, the input interface 42 may be realized by a trackball, switch buttons, a mouse, a keyboard, a touchpad that performs input operations by touching the operation surface, a touchscreen that integrates a display screen and a touchpad, a non-contact input interface using an optical sensor, a voice input interface, etc. Note that, in this specification, the input interface 42 is not limited to interfaces equipped with physical operation components such as a mouse and a keyboard. For example, an electrical signal processing circuit that receives electrical signals corresponding to input operations from an external input device provided separately from the device and transmits these electrical signals to a control circuit is also included as an example of the input interface 42. Note that the input interface is an example of a receiving unit.

[0028] The display 43 displays various types of information and data. Specifically, the display 43 is connected to the processing circuitry 45 and displays various types of information and data (e.g., medical images) received from the processing circuitry 45. For example, the display 43 is realized by a liquid crystal display, a CRT (Cathode Ray Tube) display, a touch panel, or the like.

[0029] The memory circuitry 44 stores various data and programs. Specifically, the memory circuitry 44 is connected to the processing circuitry 45 and stores data received from the processing circuitry 45 or reads and transmits stored data to the processing circuitry 45. For example, the memory circuitry 44 may be implemented using a semiconductor memory element such as a read-only memory (ROM), a random access memory (RAM), or a flash memory, or a hard disk or optical disk. For example, the memory circuitry 44 stores medical images collected from a subject before surgery. As shown in FIG. 1, the memory circuitry 44 also stores device information 441. The device information 441 is information about the device for each operating room. For example, the memory circuitry 44 stores, as the device information 441, information (such as the manufacturer's name, model number, and catalog information) about the shadowless lamp 21 installed in the operating room 1 and information (such as the manufacturer's name, model number, and catalog information) about the endoscope system 31 installed in the operating room 2. The memory circuitry 44 may be implemented using a cloud computer connected to the medical device 40 via a network.

[0030] The processing circuitry 45 controls the entire medical device 40. Specifically, the processing circuitry 45 controls the transmission and reception of information between each device on the network and controls various processes related to surgical treatment. The processing circuitry 45 can also perform various processes in response to input operations via the input interface 42.

[0031] An example of the configuration of the medical device 40 according to this embodiment has been described above. For example, the medical device 40 is installed in a medical facility such as a hospital, and supports various diagnoses and the formulation of treatment plans performed by users such as doctors. The medical device 40 also performs various controls in surgical treatment. The medical device 40 having such a configuration will be described in detail below.

[0032] For example, as shown in Fig. 1, the processing circuitry 45 of the medical device 40 executes a control function 451, an acquisition function 452, a generation function 453, and a change function 454. Here, the control function 451 is an example of a display control unit. The acquisition function 452 is an example of an acquisition unit. The change function 454 is an example of a change unit.

[0033] The control function 451 controls the display of various information on the display 43 and the display 23 installed in the operating room 1. The control function 451 also controls the shadowless lamp 21 by transmitting control information to the input device 22 installed in the operating room 1. The control function 451 also controls the endoscope system 31 by transmitting control information to the input device 32 installed in the operating room 2.

[0034] The acquisition function 452 acquires various information from each department system 10. For example, the acquisition function 452 acquires medical images collected in advance from a subject who will undergo a surgical operation from each department system 10. Here, the acquisition function 452 can acquire, as the above-mentioned medical images, preoperative images (raw data collected by a medical image diagnostic device, image data after reconstructing the raw data, and images obtained by performing various image processing on the image data), treatment plan images, simulation images, etc.

[0035] The acquisition function 452 also acquires appearance information of the target area observed during a procedure in surgical treatment of the subject. Specifically, the acquisition function 452 acquires surgery information of the subject undergoing surgical treatment, and acquires appearance information indicating the appearance of the target area at the time of surgery based on the surgery information. More specifically, the acquisition function 452 acquires information about the operating room in which the surgery on the subject will be performed and information about the equipment used in the surgery from each department system 10. The acquisition function 452 then references the equipment information 441 stored in the memory circuitry 44 to read out equipment information corresponding to the acquired information, and acquires appearance information of the target area at the time of surgery based on the read-out equipment information. The processing performed by the acquisition function 452 will be described in detail later.

[0036] The generation function 453 generates various images based on the medical images acquired by the acquisition function 452. For example, the generation function 453 generates image data by performing known reconstruction processing on raw data. The generation function 453 also generates various display images by performing known image processing on the image data (for example, rendering processing, etc.). The generation function 453 can also generate treatment plan images and simulation images in response to input operations by the operator via the input interface 42.

[0037] The modification function 454 changes the display state of the medical image of the target site based on the appearance information. Specifically, the modification function 454 changes the display state of the medical image acquired by the acquisition function 452 based on the appearance information of the target site acquired by the acquisition function 452. More specifically, the modification function 454 changes the display state of the medical image to resemble the appearance of the target site at the time of surgery. For example, the modification function 454 changes at least one of brightness and color as the display state of the medical image. Here, the modification function 454 may change the display state of the medical image by changing the display conditions of the medical image acquired by the acquisition function 452, or may change the display state of the medical image by having the generation function 453 generate a new medical image using the medical image acquired by the acquisition function 452. The processing by the modification function 454 will be described in detail later.

[0038] The processing circuitry 45 described above is realized by, for example, a processor. In this case, each of the processing functions described above is stored in the storage circuitry 44 in the form of a program executable by a computer. The processing circuitry 45 then reads and executes each program stored in the storage circuitry 44 to realize the function corresponding to each program. In other words, the processing circuitry 45 has each of the processing functions shown in FIG. 1 when each program is read.

[0039] The processing circuitry 45 may be configured by combining multiple independent processors, and each processor may execute a program to realize each processing function. Furthermore, each processing function of the processing circuitry 45 may be realized by being appropriately distributed or integrated among a single or multiple processing circuits. Furthermore, each processing function of the processing circuitry 45 may be realized by a combination of hardware and software, such as circuits. While the example described here is one in which programs corresponding to each processing function are stored in a single storage circuitry 44, the embodiment is not limited to this. For example, programs corresponding to each processing function may be stored in multiple storage circuits in a distributed manner, and the processing circuitry 45 may read and execute each program from each storage circuit. Furthermore, some of the processing functions of the processing circuitry 45 may be realized by a cloud computer connected to the medical device 40 via a network.

[0040] Next, the processing procedure by the medical device 40 according to the first embodiment will be described with reference to Fig. 2, and then each step will be described in detail. Fig. 2 is a flowchart showing an example of the processing procedure by the medical device 40 according to the first embodiment.

[0041] 2, in this embodiment, the acquisition function 452 acquires surgical information of a subject undergoing surgical treatment (step S101), and acquires appearance information of a target region based on device information corresponding to the acquired surgical information (step S102). This process is realized, for example, by the processing circuitry 45 calling up a program corresponding to the acquisition function 452 from the storage circuitry 44 and executing it.

[0042] Next, the change function 454 changes the display state of the medical image based on the appearance information (step S103). Specifically, the change function 454 controls the generation function 453 and / or the control function 451 so as to display a medical image that resembles the appearance of the target site during surgery. Then, the change function 454 determines whether or not a change condition for changing the display state of the medical image is satisfied (step S104). This process is realized, for example, by the processing circuitry 45 calling up and executing programs corresponding to the change function 454, the control function 451, and the generation function 453 from the storage circuitry 44.

[0043] If it is determined in step S104 that the change condition is satisfied (step S104, YES), the process returns to step S102, and the acquisition function 452 acquires the appearance information. On the other hand, if it is determined in step S104 that the change condition is not satisfied (step S104, No), the change function 454 determines whether or not to end the process (step S105). Here, if the process is not to be ended (step S105, No), the change function 454 continues the determination of step S104. On the other hand, if the process is to be ended (step S105, Yes), the medical device 40 ends the process. This process is realized, for example, by the processing circuitry 45 calling up a program corresponding to the change function 454 from the storage circuitry 44 and executing it.

[0044] The following describes in detail each process executed by the medical device 40. Here, the medical device 40 according to the first embodiment executes different processes depending on the type of surgery being performed. Specifically, the medical device 40 executes different processes when laparotomy is performed as the surgical treatment and when endoscopic surgery is performed. Below, the processes during laparotomy and endoscopic surgery will be explained in order.

[0045] (Processing during abdominal surgery) The processing during abdominal surgery will be described below in accordance with the processing of the flowchart explained in FIG.

[0046] (Surgery information acquisition process) As described in step S101 of FIG. 2, the acquisition function 452 acquires surgery information on a subject on whom a surgical treatment is to be performed. Specifically, the acquisition function 452 acquires information such as the type of surgical treatment the subject will undergo, information on the operating room where the surgical treatment will be performed, and information on the equipment used in the surgical treatment. For example, the acquisition function 452 acquires the subject's surgery information from each department system 10 based on the subject information input via the input interface 42. As an example, the acquisition function 452 acquires information that an open surgery will be performed in the operating room 1 as the subject's surgery information. Note that the above-mentioned surgery information may be acquired by being input simultaneously when the subject information is input via the input interface 42.

[0047] (Appearance information acquisition process) 2, the acquisition function 452 acquires appearance information of the target site of surgical treatment based on the device information corresponding to the surgery information acquired in step S101. Specifically, the acquisition function 452 first acquires the device information corresponding to the surgery information from the device information 441 stored in the memory circuitry 44. For example, the acquisition function 452 acquires information (such as the manufacturer name, model number, and catalog information) of the shadowless lamp 21 as the device information corresponding to the operating room 1 acquired as the surgery information.

[0048] Then, the acquisition function 452 acquires appearance information indicating the appearance of the target area during surgery from the acquired device information (information about the shadowless lamp 21). Specifically, the acquisition function 452 acquires appearance information of the target area based on the illumination conditions of the lighting equipment (shadowless lamp 21) used during the procedure in surgical treatment.

[0049] Here, the medical device 40 can use various methods to acquire the appearance information of the target site from the device information. For example, information on the appearance of the target site for each lighting fixture may be acquired in advance and stored in the memory circuitry 44 in association with the device information 441, and the acquisition function 452 may also acquire the associated appearance information when acquiring the device information. In this case, for example, the appearance information may be information on the appearance (the target site after laparotomy and in a state where it is irradiated with light) (such as the brightness and color of the target site) under various irradiation conditions, such as the position and angle of the shadowless lamp 21 relative to the operating table, the light intensity, and the wavelength, for each type of target site (open surgical procedure).

[0050] That is, the acquisition function 452 acquires, from the appearance information associated with the device information, appearance information that corresponds to the target site for surgical treatment and that corresponds to the current irradiation conditions of the shadowless lamp 21. For example, during preoperative treatment planning or simulation, the acquisition function 452 acquires appearance information that corresponds to the irradiation conditions set by default, and acquires corresponding appearance information every time the irradiation conditions are changed (conditions are changed via the input interface 42). That is, the acquisition function 452 acquires appearance information that corresponds to the irradiation conditions suited to the planned procedure, and acquires corresponding appearance information even when the irradiation conditions are changed.

[0051] Furthermore, for example, during surgery, the acquisition function 452 constantly acquires the current illumination conditions of the shadowless lamp 21 and acquires appearance information corresponding to the acquired illumination conditions. That is, the acquisition function 452 acquires appearance information of the target area based on the illumination conditions of the lighting equipment used during the procedure. The acquisition function 452 also acquires appearance information of the target area based on the illumination conditions of the lighting equipment for each progress state of the procedure. That is, the acquisition function 452 acquires appearance information of the target area every time new control information is received from the input device 22.

[0052] The medical device 40 can also use a three-dimensional model to acquire appearance information of the target region from the device information. In this case, for example, a simulation model is generated in advance, in which a three-dimensional model of the operating table, a three-dimensional model of the shadowless lamp 21, and a three-dimensional model of the subject are arranged in a virtual space simulating the operating room 1, and stored in the memory circuitry 44. The acquisition function 452 reads the simulation model and acquires appearance information of the target region by illuminating the three-dimensional model of the shadowless lamp 21 based on the current illumination conditions. The positional relationship between each part of the three-dimensional model of the subject and each part of the medical image can be established by, for example, alignment based on anatomical landmarks. The three-dimensional model of the subject may also be a three-dimensional medical image acquired from the subject.

[0053] When the above-described simulation model is used, the acquisition function 452 can acquire appearance information for each irradiation condition by changing the illumination of the three-dimensional model of the shadowless lamp 21 each time the irradiation condition is changed. For example, during preoperative treatment planning, simulation, or during surgery, the acquisition function 452 can acquire appearance information each time the irradiation condition is changed (new control information is received from the input device 22). Note that the above-described simulation model can also simulate laparotomy of a subject. That is, the acquisition function 452 can acquire appearance information of a target area according to the laparotomy state of the subject.

[0054] (Display state change processing) 2, the change function 454 changes the display state of the medical image based on the appearance information acquired by the acquisition function 452. Specifically, the change function 454 changes the display state of the medical image so that the display state of the target site in the medical image resembles the appearance of the target site indicated in the appearance information.

[0055] 3 is a diagram illustrating an example of the change processing according to the first embodiment. For example, as shown in FIG. 3, the change function 454 changes the display state of the medical image so as to resemble the appearance of the target area when illuminated by the shadowless lamp 21. Here, the illumination conditions of the shadowless lamp 21 include, for example, the illumination angle, illuminance, wavelength, color temperature, number of light source units, distance from the light source units to the target area, and spread of light from the light source units. Then, the acquisition function 452 acquires appearance information (information on the brightness and color of the target area) corresponding to the current illumination conditions.

[0056] The modification function 454 modifies the display state of the medical image of the target region so that the display state resembles the brightness and color of the target region shown in the appearance information. For example, the modification function 454 controls the reconstruction process and rendering process of the medical image so that the bright and dark parts (or shadow parts) of the target region resemble the appearance information. In other words, the modification function 454 controls the generation function 453 to generate a medical image that resembles the appearance information.

[0057] Furthermore, for example, the modification function 454 controls the color of the medical image so that the color of each part of the target region becomes the same as the appearance information. That is, the modification function 454 controls the control function 451 to control the color of each part in the medical image to become the same as the appearance information.

[0058] Here, information about the color of the target area may be acquired by a camera installed in the operating room 1. In such a case, the change function 454 identifies each area in the surgical field from the image acquired by the camera. Here, the identification of the area may be performed by a known method using image analysis, AI (Artificial Intelligence), or the like. The change function 454 acquires information about the color of each identified area from the image, and changes the display state of the medical image by reflecting the acquired color information on the corresponding area in the medical image.

[0059] For example, the change function 454 controls the display states of various medical images such as preoperative images, treatment plan images, and simulation images so that they resemble the appearance of the target site indicated in the appearance information. The control function 451 controls the display of the medical images whose display states have been changed by the change function 454 on the display 43 or the display 23. For example, in the case of medical images observed during the formulation of a preoperative treatment plan or a simulation, the control function 451 causes the display 43 to display the medical images whose display states have been changed by the change function 454. Furthermore, in the case of medical images observed during surgery, the control function 451 causes the display 23 to display the medical images whose display states have been changed by the change function 454.

[0060] (Change determination process) As described in step S104 of FIG. 2, the change function 454 determines whether the change condition has been met. Specifically, the change function 454 determines whether the illumination conditions of the lighting device have changed. For example, during surgery, illumination conditions such as the position, angle, and light emission amount of the shadowless lamp 21 may be changed as the procedure progresses. When the change function 454 detects such a change in illumination conditions, the acquisition function 452 acquires appearance information corresponding to the changed illumination conditions. That is, the acquisition function 452 acquires appearance information of the target area based on the illumination conditions of the lighting device for each progress state of the procedure. Each time appearance information is acquired, the change function 454 changes the display state of the medical image based on the acquired appearance information.

[0061] (Processing end determination process) As described in step S105 of Fig. 2, the change function 454 determines whether to end the process of changing the display state of the medical image. For example, when the display state of a medical image being observed preoperatively is being changed, the change function 454 determines whether to end the process by determining whether the display of the medical image on the display 43 has ended. Also, for example, when the display state of a medical image being observed intraoperatively is being changed, the change function 454 determines whether to end the process by determining whether the display of the medical image on the display 23 has ended.

[0062] In the above-described embodiment, a case has been described in which the irradiation conditions are changed depending on the type of target area and the progress of the procedure. However, the embodiment is not limited to this. For example, the irradiation conditions of a lighting device (e.g., the shadowless lamp 21) may be changed depending on the number and standing positions of the surgeons performing the surgery. For example, when multiple surgeons perform surgery, the irradiation conditions of the shadowless lamp 21 are determined so that each surgeon's standing position does not cast a shadow on the target area. The acquisition function 452 acquires appearance information of the target area based on the determined irradiation conditions. Note that the standing positions of each surgeon are determined in advance, and for example, a three-dimensional model of each surgeon is placed at a corresponding position in the above-described simulation model, and the irradiation conditions for the target area are determined.

[0063] (Processing during endoscopic surgery) Next, the processing during endoscopic surgery will be described in accordance with the processing of the flowchart explained in FIG.

[0064] (Surgery information acquisition process) In the process of endoscopic surgery, the acquisition function 452 acquires, for example, information that endoscopic surgery will be performed in the operating room 2 as surgery information on the subject.

[0065] (Appearance information acquisition process) In processing during endoscopic surgery, the acquisition function 452 acquires information about the endoscopic system 31 (such as the manufacturer's name, model number, and catalog information) as device information corresponding to the operating room 2 acquired as surgery information. Specifically, the acquisition function 452 acquires information about the insertion section (treatment section and imaging section) of the endoscope 31a of the endoscopic system 31. For example, the acquisition function 452 acquires information about the shape of the insertion section of the endoscope 31a, the number and position of the light emitting sections, the amount of light emitted, wavelength, and color, etc.

[0066] The acquisition function 452 then acquires appearance information indicating the appearance of the target site during endoscopic surgery from the acquired device information (information about the insertion part of the endoscope 31a). Here, the acquisition function 452 acquires different appearance information when observing medical images before surgery and when observing medical images during surgery.

[0067] For example, in observing a medical image before surgery, the acquisition function 452 acquires appearance information of the target site based on the irradiation conditions of the light emitting unit of the endoscope 31a. That is, in observing a medical image before surgery, the acquisition function 452 acquires appearance information of the target site based on the light irradiation conditions, similar to during laparotomy.

[0068] For example, as in open surgery, information on the appearance of the target area may be acquired in advance for each endoscope 31a and stored in association with device information 441 in the memory circuitry 44, and the acquisition function 452 may acquire the associated appearance information when acquiring the device information.

[0069] That is, the acquisition function 452 acquires, from the appearance information associated with the device information, appearance information that corresponds to the target site for surgical treatment and that corresponds to the current irradiation conditions of the endoscope 31a. For example, during preoperative treatment planning or simulation, the acquisition function 452 acquires appearance information that corresponds to the irradiation conditions set by default, and acquires corresponding appearance information every time the irradiation conditions are changed (conditions are changed via the input interface 42). That is, the acquisition function 452 acquires appearance information that corresponds to the irradiation conditions according to the planned procedure, and acquires corresponding appearance information even when the irradiation conditions are changed.

[0070] Furthermore, during endoscopic surgery, appearance information can be acquired using a 3D model. In such a case, for example, a simulation model in which a 3D model of the operating table, a 3D model of the endoscope 31a, and a 3D model of the subject are arranged in a virtual space simulating the operating room 2 is generated in advance and stored in the memory circuitry 44. The acquisition function 452 reads the simulation model and acquires appearance information of the target area by illuminating the light emitting unit of the 3D model of the endoscope 31a based on the current irradiation conditions (the position and orientation of the insertion portion within the subject, the luminous intensity, wavelength, and color of the light emitted from the light emitting unit, etc.). Note that the positional relationship between each part in the 3D model of the subject and each part in the medical image can be associated by, for example, alignment based on anatomical landmarks. Furthermore, the 3D model of the subject may be formed using 3D medical images collected from the subject.

[0071] Next, in observing medical images during surgery, the acquisition function 452 acquires an endoscopic image of the target site captured by the endoscope 31a as appearance information. That is, the acquisition function 452 acquires appearance information of the target site in the captured image based on the display conditions of the captured image (endoscopic image) of the target site captured during the surgical treatment procedure. For example, the acquisition function 452 acquires the state of the target site depicted in the endoscopic image (such as the brightness and color of each part) as appearance information.

[0072] Note that the region in the endoscopic image and the region in the medical image can be associated with each other by, for example, alignment using anatomical landmarks, etc. For example, the acquisition function 452 estimates the region depicted in the endoscopic image from the anatomical landmarks included in the endoscopic image, and identifies the corresponding position in the medical image based on the estimated region.

[0073] Furthermore, the correspondence between the region in the endoscopic image and the region in the medical image may be performed using a position sensor. In such a case, the position and orientation of the insertion portion of the endoscope 31a within the subject are acquired by the position sensor, thereby estimating the region depicted in the currently captured endoscopic image, and identifying the corresponding position in the medical image based on the estimated region. For example, the acquisition function 452 estimates the region depicted in the currently captured endoscopic image by setting the position and orientation of the insertion portion of the endoscope 31a in the simulation model to the position and orientation acquired by the position sensor. Then, the acquisition function 452 identifies the corresponding position in the medical image based on the estimated region.

[0074] (Display state change processing) In the process of endoscopic surgery, the change function 454 also changes the display state of the medical image so that the display state of the target site in the medical image resembles the appearance of the target site shown in the appearance information.

[0075] FIG. 4 is a diagram for explaining an example of the modification process according to the first embodiment. Note that FIG. 4 shows an example in which an endoscopic image is acquired as the appearance information. For example, as shown in FIG. 4, the modification function 454 modifies the display state of the medical image of the target region so that the brightness and color of the target region depicted in the endoscopic image are similar. For example, the modification function 454 controls the reconstruction process and rendering process of the medical image so that bright and dark parts (shadow parts) of the target region are similar to those in the endoscopic image. In other words, the modification function 454 controls the generation function 453 to generate a medical image in a state similar to the appearance information.

[0076] Furthermore, for example, the modification function 454 controls the color of the medical image so that the color of each part of the target region becomes the same as the appearance information. That is, the modification function 454 controls the control function 451 to control the color of each part in the medical image to become the same as the appearance information. For example, the modification function 454 changes the display state so that organs with circulating blood are displayed in red and organs that are prone to anemia are displayed in blue.

[0077] For example, the change function 454 controls the display state of various medical images such as preoperative images, treatment plan images, and simulation images so that they resemble the appearance of the target region indicated in the appearance information. The control function 451 controls the medical images whose display state has been changed by the change function 454 to be displayed on the display 43 or the display 31b of the endoscope system 31. For example, in the case of medical images observed during the formulation of a preoperative treatment plan or a simulation, the control function 451 causes the medical images whose display state has been changed by the change function 454 to be displayed on the display 43.

[0078] Furthermore, for example, in the case of a medical image observed during surgery, the control function 451 controls the display 31b to display the medical image whose display state has been changed by the change function 454. That is, the control function 451 transmits the medical image whose display state has been changed to the endoscope system 31, thereby displaying the medical image on the display 31b in the endoscope system 31, as shown in Fig. 5. This allows the doctor to observe the endoscopic image and the medical image side by side during surgery. Note that Fig. 5 is a diagram showing an example of the display of a medical image according to the first embodiment.

[0079] Furthermore, in endoscope systems and systems using surgical support robots, systems that display three-dimensional endoscopic images that allow a target region to be viewed stereoscopically are known. The medical device 40 according to this embodiment can acquire the three-dimensional endoscopic images that allow a stereoscopic view in such systems as appearance information and change the display state of the medical images.

[0080] 6 is a diagram illustrating an example of the change processing according to the first embodiment. As shown in FIG. 6, when displaying a medical image similar to a target region stereoscopically viewed in an endoscopic image, the acquisition function 452 acquires, for example, appearance information including parts that will be in shadow in stereoscopic viewing. The change function 454 changes the display state of the medical image based on the acquired appearance information so as to make the parts that will be in shadow in stereoscopic viewing shadow.

[0081] (Change determination process) In processing during endoscopic surgery, the change function 454 determines whether the conditions of the endoscope 31a have changed, thereby determining whether the change conditions have been met. For example, during surgery, conditions such as the position and angle of the insertion portion of the endoscope 31a and the amount of light emitted from the light emitting portion may change as the procedure progresses. When the change function 454 detects such a change in conditions, the acquisition function 452 acquires appearance information corresponding to the changed conditions.

[0082] (Processing end determination process) In the process during endoscopic surgery, the change function 454 also determines whether to end the process by determining whether the display of the medical image on the display 43 or the display 31b has ended.

[0083] (Variation 1) In the above-described embodiment, a single medical image is displayed, and the display state of the displayed medical image is changed to match the appearance of the target region. However, the embodiment is not limited to this, and multiple medical images may be displayed. In such a case, the display conditions of each medical image are changed so that each medical image resembles the appearance of the target region. Note that the display state of only limited medical images among the multiple medical images may also be changed. For example, the display state of only medical images that are primarily observed by the surgeon may be changed among the multiple medical images.

[0084] The medical device 40 can also display medical images that look different for multiple doctors. For example, when multiple doctors are performing surgery, the appearance of the target area changes depending on their positions relative to the subject (patient). Therefore, the medical device 40 changes the display state of the medical image to match the appearance of the target area observed by each doctor. For example, when displaying medical images to Doctor A, who will stand on the right side of the subject during surgery, and Doctor B, who will stand on the left side, the medical device 40 acquires appearance information of the target area observed from the standing positions of each doctor, and changes the display state of the medical image according to the acquired appearance information.

[0085] In such a case, for example, the acquisition function 452 reads out the above-mentioned simulation model, illuminates the three-dimensional model of the shadowless lamp 21 based on the current illumination conditions, and acquires appearance information of the target area observed from the standing position of Doctor A (orientation of the target area, positional relationship with other organs, brightness and darkness of light, color, etc.) and appearance information of the target area observed from the standing position of Doctor B (orientation of the target area, positional relationship with other organs, brightness and darkness of light, color, etc.).

[0086] The modification function 454 changes the display state of the medical image by switching the appearance information used when the medical image is observed by Doctor A and when the medical image is observed by Doctor B. That is, the modification function 454 changes the display state of the medical image (the orientation of the target region, its positional relationship with other organs, the brightness and darkness of light, color, etc.) for the medical image observed by Doctor A so that it resembles the appearance information of the target region when observed from a position on the right side of the subject. On the other hand, the modification function 454 changes the display state of the medical image (the orientation of the target region, its positional relationship with other organs, the brightness and darkness of light, color, etc.) for the medical image observed by Doctor B so that it resembles the appearance information of the target region when observed from a position on the left side of the subject.

[0087] As described above, the medical device 40 can change the display state of the displayed medical image depending on the surgeon's position during surgery, thereby displaying appropriate medical images for each surgeon. For example, in image training using preoperative planning images, each surgeon can observe medical images that resemble the appearance of the target area they will be observing, allowing for appropriate image training. Furthermore, for medical images observed during surgery, the medical images observed by each surgeon can be displayed in an appropriate display state, allowing for appropriate support during surgery.

[0088] (Variation 2) In the above-described embodiment, a case has been described in which a surgeon performs surgery in operating room 1 or operating room 2. However, the embodiment is not limited to this, and may be a case in which remote surgery is performed on a subject in a remote location, for example. FIG. 7 is a diagram for explaining an example of remote surgery according to the first embodiment. As shown in FIG. 7, for example, in remote surgery, a surgical support robot is installed in a remote location where the subject is located, and a console for operating the surgical support robot is installed in a hospital where the surgeon is located.

[0089] The surgeon operates the robotic arm of the surgical support robot by operating the control unit while observing the endoscopic image displayed on the console display. The robotic arm holds medical instruments such as forceps and an endoscope, and the surgeon operates the control unit to operate these medical instruments and perform surgery.

[0090] In such remote surgery, the medical device 40 can display medical images similar to endoscopic images on the console display. Here, in remote surgery, in addition to displaying medical images similar to endoscopic images, medical images that are easy for the surgeon to observe may also be displayed. That is, when surgical treatment is performed by remote operation, the change function 454 can change the display state of the medical images depending on the remote operator observing the captured images (endoscopic images) of the target area.

[0091] In such a case, the medical device 40 receives and stores in advance information regarding the color and shadow state desired by the surgeon. When remote surgery is performed, the change function 454 reads the information in response to an instruction from the surgeon to change the display state, and controls the change so that the display state of the medical image is changed to the state desired by the surgeon.

[0092] (Variation 3) In the above embodiment, a case where a medical image resembling the actual appearance during surgery is displayed has been described. However, the embodiment is not limited to this, and the medical device 40 may have various display modes for displaying medical images. For example, the medical device 40 has a "realistic display mode," a "warning mode," and an "informed consent mode," and displays medical images according to each mode.

[0093] Here, the "realistic display mode" is a mode for displaying medical images that resemble the actual appearance during surgery, the "attention mode" is a mode for calling attention during surgery, and the "informed consent mode" is a mode for explaining to the subject. For example, the medical device 40 may be configured to switch between the above modes in accordance with a workflow related to surgical treatment.

[0094] For example, in a workflow related to surgical treatment, when explaining the surgical treatment to a subject, the modification function 454 transitions to an "informed consent mode" and changes the display state of the medical image to one that is easy to see. For example, the modification function 454 changes the display state of the medical image so that the target area is easy to see, by emphasizing the boundary of the area in the medical image, adding a shadow to the area, changing the direction of the shadow, or adding a shadow to the surface of the area. In other words, the modification function 454 further changes the display state of the medical image when explaining the surgical treatment to the subject.

[0095] Also, for example, in a workflow related to surgical treatment, when formulating a treatment plan or simulating surgery, or during surgery, the change function 454 transitions to the "real display mode" and executes the above-mentioned processes.

[0096] Furthermore, for example, when the target area is near a large blood vessel or a nerve, the change function 454 transitions to an "attention mode" and further changes the display state of the medical image at the timing of a warning during the procedure. For example, by transitioning to the "attention mode," the change function 454 changes the display state so that when the treatment unit of the endoscope 31a approaches a blood vessel or a nerve during endoscopic surgery, the medical image is displayed in a color that calls attention. In such a case, for example, the change function 454 acquires the position of the treatment unit of the endoscope 31a detected by the position sensor, monitors the distance between the acquired position of the treatment unit and the position of the blood vessel or nerve in the above-mentioned simulation model, and controls to change the display state of the medical image when the distance becomes equal to or less than a threshold.

[0097] (Variation 4) In addition to the display control described above, the medical device 40 can also perform display control according to the elapsed time of the procedure. Specifically, the change function 454 further changes the display state of the medical image according to the elapsed time since the start of the surgical treatment. For example, the change function 454 controls the display to suppress blue light on the display showing the medical image according to the elapsed time since the start of the surgical treatment.

[0098] (Variation 5) In the above-described third modification, a case has been described in which a display calling attention is performed in the "attention mode" according to the distance between the treatment unit of the endoscope 31a and a blood vessel or nerve. However, the embodiment is not limited to this, and the display state of the medical image may be changed according to the distance between the treatment unit and the target site or the orientation of the treatment unit (lens) relative to the target site. For example, the modification function 454 controls the medical image to reflect light so as to represent moisture on the surface of the target site.

[0099] For example, the modification function 454 modifies the display state of the medical image so that the surface of the target site becomes brighter and the intensity of reflected light increases as the distance between the treatment unit and the target site becomes shorter. Also, the modification function 454 modifies the display state of the medical image so that the intensity of reflected light from the target site increases as the orientation of the treatment unit (lens) relative to the surface of the target site becomes closer to perpendicular, and modifies the display state of the medical image so that the intensity of reflected light from the target site decreases as the orientation of the treatment unit (lens) relative to the surface of the target site becomes more oblique.

[0100] (Variation 6) In the above-described embodiment, the appearance information of the target area is acquired based on the device information of the device used in the surgery. However, the embodiment is not limited to this, and the appearance information of the target area may be acquired based on the surgical instrument used in the surgery. In such a case, for example, the acquisition function 452 reads the simulation model described above, illuminates the 3D model of the shadowless lamp 21 based on the current illumination conditions, and acquires the appearance information of the target area when the surgical instrument is placed in the virtual space of the simulation model. Here, the type of surgical instrument and the placement of the surgical instrument in the virtual space can be arbitrarily set by the operator (e.g., a doctor). That is, the operator can select a surgical instrument and place the surgical instrument at any position in the virtual space via the input interface 42. For example, the operator places the surgical instrument in the virtual space according to the doctor's position during surgery.

[0101] The acquisition function 452 reads out information about the surgical instrument set by the operator (information about the shape, size, etc.) from the memory circuitry 44, and acquires appearance information about the target area when the surgical instrument is placed at the set position in the virtual space (information about the shadow cast by the surgical instrument, etc.). The memory circuitry 44 stores in advance information about the surgical instruments used in the surgery (information about the shape and size of each type of surgical instrument).

[0102] The modification function 454 modifies the display state of the medical image so that appearance information based on the surgical instrument is reflected in the medical image. For example, the modification function 454 modifies the display state of the medical image so that shadows cast by the surgical instrument are displayed in the medical image. This makes it possible, for example, when a simulation image of surgery is displayed, to display on the simulation image changes in the appearance of the target site based on the medical instrument used in the simulation, thereby displaying a medical image that more closely resembles the actual appearance during surgery.

[0103] As described above, according to the first embodiment, the acquisition function 452 acquires appearance information of a target site observed during a procedure in a surgical treatment on a subject. The modification function 454 modifies the display state of a medical image of the target site based on the appearance information. Therefore, the medical device 40 according to the first embodiment can display a medical image according to the appearance of the target site, making it possible to make the display state of the target site included in the medical image similar to the appearance of the target site observed during the surgical treatment.

[0104] Furthermore, according to the first embodiment, the acquisition function 452 acquires appearance information of the target area based on the illumination conditions of the lighting equipment used during the surgical procedure. The modification function 454 modifies the display state of the medical image so that the display state of the target area in the medical image resembles the appearance of the target area shown in the appearance information. Therefore, the medical device 40 according to the first embodiment makes it possible to display medical images that resemble the actual appearance of the target area and that do not seem strange to the doctor.

[0105] Furthermore, according to the first embodiment, the acquisition function 452 acquires appearance information of the target site in the captured image based on the display conditions of the captured image of the target site captured during a surgical procedure. The modification function 454 modifies the display state of the medical image so that the display state of the target site in the medical image resembles the appearance of the target site indicated in the appearance information. Therefore, the medical device 40 according to the first embodiment makes it possible to display medical images that resemble the captured image and do not seem strange to the doctor.

[0106] According to the first embodiment, the acquisition function 452 acquires appearance information of the target area based on the illumination conditions of the lighting equipment used during the procedure. Therefore, the medical device 40 according to the first embodiment makes it possible to acquire information on the actual appearance of the target area.

[0107] Furthermore, according to the first embodiment, the acquisition function 452 acquires appearance information of the target region based on the illumination conditions of the lighting device for each progress state of the procedure. The change function 454 changes the display state of the medical image based on the acquired appearance information each time the appearance information is acquired. Therefore, the medical device 40 according to the first embodiment makes it possible to change the display state of the medical image in accordance with changes in the appearance of the target region.

[0108] Furthermore, according to the first embodiment, when surgical treatment is performed by remote operation, the change function 454 changes the display state of the medical image according to the remote operator observing the captured image of the target area. Therefore, the medical device 40 according to the first embodiment allows the doctor performing the remote surgery to display the medical image as desired.

[0109] Furthermore, according to the first embodiment, the change function 454 further changes the display state of the medical image at the timing of calling attention during the procedure. Therefore, the medical device 40 according to the first embodiment can support performing the procedure more safely.

[0110] Furthermore, according to the first embodiment, the change function 454 further changes the display state of the medical image when explaining the surgical treatment to the subject. Therefore, the medical device 40 according to the first embodiment makes it possible to display medical images that are easy for the subject (patient) to view.

[0111] Furthermore, according to the first embodiment, the change function 454 further changes the display state of the medical image depending on the time elapsed since the start of the surgical treatment. Therefore, the medical device 40 according to the first embodiment makes it possible to change the display state of the medical image depending on the condition of the doctor performing the surgery.

[0112] Furthermore, according to the first embodiment, the change function 454 changes at least one of brightness and color as the display state of the medical image. Therefore, the medical device 40 according to the first embodiment makes it possible to appropriately change the display state of the medical image.

[0113] (Second embodiment) In the first embodiment described above, a case where the display conditions of a medical image are changed based on the illumination conditions of a lighting device or the appearance information of a target area corresponding to a captured image (endoscopic image) is described. In the second embodiment, a case where the illumination conditions of a lighting device or the display state of a captured image is changed based on the display conditions of a medical image is described. Note that the medical device 40 according to the second embodiment differs from the first embodiment in the processing by the acquisition function 452 and the processing by the change function 454. These points will be mainly described below.

[0114] The acquisition function 452 according to the second embodiment acquires display conditions for medical images of a target area for surgical treatment on a subject. Specifically, the acquisition function 452 acquires information about the brightness and color of each part of a medical image, such as a preoperative image of a subject undergoing surgical treatment, a treatment plan image, or a simulation image, when the medical image is displayed on a display.

[0115] Here, when the display conditions of the medical image are changed, the acquisition function 452 also acquires the changed display conditions. For example, the input interface 42 accepts a change operation to change the display conditions of the medical image. The acquisition function 452 acquires the display conditions of the medical image after the change operation.

[0116] The modification function 454 according to the second embodiment modifies the appearance of the target site observed during the surgical procedure based on the display conditions of the medical image. Specifically, the modification function 454 modifies the appearance of the target site observed during the surgical procedure so that the appearance of the target site observed during the surgical procedure resembles the display conditions of the medical image.

[0117] Next, the processing procedure by the medical device 40 according to the second embodiment will be described with reference to Fig. 8, and then each step will be described in detail. Fig. 8 is a flowchart showing an example of the processing procedure by the medical device 40 according to the second embodiment.

[0118] 8, in this embodiment, the acquisition function 452 acquires display conditions for medical images collected from a subject undergoing surgical treatment (step S201). This process is realized, for example, by the processing circuitry 45 calling up a program corresponding to the acquisition function 452 from the storage circuitry 44 and executing it.

[0119] Next, the change function 454 determines control information for the device based on the display conditions (step S202), and controls the device based on the control information to change the appearance of the target region (step S203). Specifically, the change function 454 controls the device in the operating room 1 or 2 by causing the input device 22 or 32 to transmit the determined control information. The control information is transmitted by the control function 451. This process is realized, for example, by the processing circuitry 45 calling up and executing a program corresponding to the change function 454 and the control function 451 from the storage circuitry 44.

[0120] Then, the change function 454 determines whether the display state of the medical image has been changed (step S204). This process is realized, for example, by the processing circuitry 45 calling up a program corresponding to the change function 454 from the storage circuitry 44 and executing it.

[0121] If it is determined in step S204 that the display conditions have been changed (step S204, YES), the process returns to step S202, and the change function 454 determines the control information based on the changed display conditions. On the other hand, if it is determined in step S204 that the display conditions have not been changed (step S204, No), the change function 454 determines whether the surgery has ended (step S205). Here, if the surgery has not ended (step S205, No), the change function 454 continues the determination in step S204. On the other hand, if the surgery has ended (step S205, Yes), the medical device 40 ends the processing. This processing is realized, for example, by the processing circuitry 45 calling up a program corresponding to the change function 454 from the storage circuitry 44 and executing it.

[0122] The following describes in detail each process executed by the medical device 40 according to the second embodiment. The medical device 40 according to the second embodiment also executes different processes when an open surgery is performed as the surgical treatment and when an endoscopic surgery is performed. The following describes the processes during the open surgery and the processes during the endoscopic surgery in order.

[0123] (Processing during abdominal surgery) The processing during abdominal surgery will be described below in accordance with the processing of the flowchart explained in FIG.

[0124] (Display condition acquisition process) As described in step S201 of FIG. 8, the acquisition function 452 acquires the display conditions of medical images collected from a subject undergoing surgical treatment. For example, during surgical treatment, the display state of a medical image may be changed in various ways when observing the medical image before surgery or during surgery. For example, the direction, illuminance, wavelength, etc. of light in the displayed medical image may be changed so that the target area depicted in the medical image can be more easily seen. For example, the doctor observing the medical image changes the display conditions of the medical image by performing a change operation via the input interface 42. The acquisition function 452 acquires such display conditions of the medical image every time the display conditions are changed.

[0125] (Control information determination process) As described in step S202 of Fig. 8, the change function 454 determines control information for controlling the device based on the display conditions of the medical image acquired in step S101. Specifically, the change function 454 first acquires device information corresponding to the surgery information from the device information 441 stored in the memory circuitry 44. Note that the surgery information is acquired by the acquisition function 452, as in the first embodiment. For example, the change function 454 acquires information about the shadowless lamp 21 (such as the manufacturer name, model number, and catalog information) as the device information.

[0126] The change function 454 then changes the illumination conditions of the lighting equipment used during the surgical procedure so that the appearance of the target area observed during the surgical procedure resembles the display state of the medical image whose display conditions have been changed by the change operation. That is, the change function 454 determines control information that sets the illumination conditions of the shadowless lamp 21 so that the appearance of the target area resembles the display state of the medical image.

[0127] Fig. 9 is a diagram for explaining an example of device control according to the second embodiment. For example, as shown in Fig. 9, the change function 454 determines the illumination conditions of the shadowless lamp 21 (at least one of the light angle, illuminance, wavelength, color temperature, number, distance, spread, etc.) based on the display conditions of the medical image generated by various reconstruction processes and rendering processes so that the appearance of the target site resembles the display state of the medical image.

[0128] For example, the change function 454 determines the control information using the above-mentioned simulation model (a simulation model in which a three-dimensional model of the operating table, a three-dimensional model of the shadowless lamp 21, and a three-dimensional model of the subject are arranged in a virtual space simulating the operating room 1). In such a case, the change function 454 determines the control information for controlling the actual shadowless lamp 21 by determining the position and orientation of the shadowless lamp 21, the illuminance and wavelength of light emitted from each light source, etc. in the simulation model based on the display conditions of the medical image (the direction, illuminance, wavelength, etc. of light relative to the target area).

[0129] The accuracy of the control information can also be confirmed by simulating the appearance of the target area when irradiated under the irradiation conditions of the shadowless lamp 21 determined as described above using the simulation model and comparing the simulation results with the display state of the medical image. In such a case, the change function 454 may, for example, calculate the similarity between the appearance of the target area obtained by the simulation (such as the brightness and color of each part) and the display state of the medical image (such as the brightness and color of each part), and determine the irradiation conditions as the control information when the similarity exceeds a threshold.

[0130] The change function 454 executes the above-described process each time the acquisition function 452 acquires display conditions to determine control information for controlling the shadowless lamp 21. For example, during preoperative surgical planning, a doctor refers to medical images to adjust the lighting method, thereby setting conditions such as the angle and illuminance so that the target area is not obscured by light reflection. The change function 454 controls the shadowless lamp 21 based on the set conditions. Also, during preoperative surgical planning, a doctor refers to medical images to adjust the size of the surgical field. The change function 454 determines the illumination conditions of the shadowless lamp 21 and controls the shadowless lamp 21 so that light is irradiated onto the set surgical field.

[0131] (Device control processing) 8, the change function 454 changes the illumination conditions of the shadowless lamp 21 by causing the control function 451 to send control information to the input device 22. This changes the appearance of the target area during abdominal surgery to resemble the display state of the medical image. Note that the control of the device may be performed automatically by sending control information to the input device 22 as described above, or may be performed by displaying the control information on the display 23 and having the operator operate the shadowless lamp 21 based on the displayed control information.

[0132] (Change determination process) As described in step S204 of FIG. 8, the change function 454 determines whether the display conditions of the medical image have been changed. For example, when observing a medical image before surgery or during surgery, the display state of the medical image may be changed (for example, the viewing angle or the area to be viewed may be changed). When the change function 454 detects such a change in the display conditions, the acquisition function 452 acquires the changed display conditions. Each time the change function 454 acquires the display conditions, it determines the control information for the shadowless lamp 21.

[0133] (Processing end determination process) 8, the change function 454 determines whether the surgery has ended. For example, the change function 454 determines whether the surgery has ended by determining whether the display of the medical image on the display 23 has ended.

[0134] In the above-described embodiment, the control information is determined according to the display conditions of the medical image. However, the embodiment is not limited to this. For example, the control information may be changed according to the number and standing positions of the surgeons performing the surgery. For example, the control information for the shadowless lamp 21 is determined so that the standing positions of each surgeon do not cast a shadow on the target area. In such a case, the standing positions of each surgeon are determined in advance, and for example, a three-dimensional model of each surgeon is placed at a corresponding position in the above-described simulation model, and the control information for the shadowless lamp 21 is determined.

[0135] (Processing during endoscopic surgery) Next, the processing during endoscopic surgery will be described in accordance with the processing of the flowchart explained in FIG.

[0136] (Display condition acquisition process) In the process of endoscopic surgery, the acquisition function 452 also acquires the display conditions of medical images, similar to the process of open surgery.

[0137] (Control information determination process) In processing during endoscopic surgery, the change function 454 acquires information about the endoscopic system 31 (such as the manufacturer's name, model number, and catalog information) as device information corresponding to the surgery information. Then, the change function 454 changes the display conditions of the captured image so that the appearance of the target area in the captured image captured during the surgical procedure resembles the display state of the medical image whose display conditions have been changed by the change operation. In other words, the change function 454 determines control information that sets the conditions of the endoscopic system 31 so that the display state of the endoscopic image becomes similar to the display state of the medical image.

[0138] Fig. 10 is a diagram for explaining an example of device control according to the second embodiment. For example, as shown in Fig. 10, the change function 454 determines the conditions of the endoscope 31a based on the display conditions of medical images generated by various reconstruction processes and rendering processes so that the display state of the endoscopic image resembles the display state of the medical image. Here, the endoscope 31a is provided with an illumination lens 311, an objective lens 312, a treatment unit channel 313, and a lens cleaning nozzle 314 at its tip, for example, as shown in Fig. 10.

[0139] The illumination lens 311 is a light emitting unit that emits light emitted from an internal light source, and two are provided at the tip of the endoscope 31a, for example, as shown in FIG. 10. The objective lens 312 is a lens for photographing that has an image sensor installed inside. The treatment unit channel 313 is configured to store medical instruments such as forceps inside and to allow the medical instruments to exit during endoscopic surgery. The lens cleaning nozzle sprays cleaning water to clean the illumination lens 311 and the objective lens 312.

[0140] For example, the change function 454 determines the control information using the above-mentioned simulation model (a simulation model in which a three-dimensional model of the operating table, a three-dimensional model of the endoscope 31a, and a three-dimensional model of the subject are arranged in a virtual space simulating the operating room 2). In such a case, the change function 454 determines the control information for controlling the actual endoscope 31a by determining the way in which light is emitted from the illumination lens 311 of the endoscope 31a in the simulation model (e.g., light emission from one of two lenses or light emission from both lenses), the illuminance, wavelength, etc. of the light emitted from each illumination lens 311, based on the display conditions of the medical image (e.g., the direction, illuminance, wavelength, etc. of the light relative to the target area).

[0141] In addition, if the endoscopic system 31 is a system that performs procedures using a surgical support robot, the change function 454 can also determine control information for controlling the direction in which light is emitted and the position of the endoscope 31a relative to the target area.

[0142] The accuracy of the control information can also be confirmed by comparing the display state of the medical image with the endoscopic image captured by the endoscope 31a controlled based on the control information determined as described above. In such a case, the change function 454 may calculate the similarity between the display state of the endoscopic image (such as the brightness and color of each part) and the display state of the medical image (such as the brightness and color of each part), and determine the condition when the similarity exceeds a threshold as the control information.

[0143] The change function 454 executes the above-described processing each time the acquisition function 452 acquires a display condition, and determines control information for controlling the endoscope 31a.

[0144] (Device control processing) In processing during endoscopic surgery, the change function 454 changes the control conditions of the endoscopic system by causing the control function 451 to send control information to the input device 32. As a result, the display state of the endoscopic image is changed to resemble the display state of the medical image. Note that the control of the device may be performed automatically by sending control information to the input device 32 as described above, or may be performed by displaying the control information on the display 31b and having the operator operate the endoscopic system 31 based on the displayed control information.

[0145] (Change determination process) In the process during endoscopic surgery, the change function 454 determines whether the display conditions of the medical image have been changed, as in the case of open surgery.

[0146] (Processing end determination process) In the process during endoscopic surgery, the change function 454 determines whether the surgery has ended by determining whether the display of medical images on the display 31b has ended, for example.

[0147] In the above-described embodiment, a case has been described in which control information is determined according to the display conditions of a medical image. However, the embodiment is not limited to this. For example, when surgery is performed by a surgical assistance robot, the control information may be changed according to the position of the robot arm. For example, the control information for the endoscope 31a is determined based on the position of the robot arm so as not to cast a shadow on the target site. In such a case, the position of the robot arm is acquired, and, for example, a three-dimensional model of the robot arm is placed at a corresponding position in the above-described simulation model, and the control information for the endoscope 31a is determined.

[0148] In the above embodiment, the case where the present invention is applied to surgery in a hospital as shown in Fig. 1 has been described. However, the embodiment is not limited to this, and may be applied to remote surgery as shown in Fig. 7. In such a case, the surgical support robot and / or the endoscope operated by the surgical support robot are controlled so that the endoscopic image displayed on the display of the console operating the surgical support robot at a remote location is displayed in a state similar to the medical image. (Variation 1) In the above embodiment, the display state of an endoscopic image is changed by changing the imaging conditions. However, the embodiment is not limited to this. For example, the display state of an endoscopic image may be changed by changing the display format when the endoscopic image is displayed on a display.

[0149] For example, if there are organs or parts that need to be distinguished, the modification function 454 modifies the display format of the endoscopic image so that they can be displayed distinctly, for example, by emphasizing the boundaries within the endoscopic image, adding shadows to the parts, changing the direction of the shadows, or adding shading to the surfaces of the parts.

[0150] (Variation 2) In the above-described embodiment, the illumination conditions of the shadowless lamp 21 and the control conditions of the endoscope 31a are changed in response to a change in the display state of a medical image. However, the embodiment is not limited to this. For example, the control may be performed at a timing desired by a doctor. In such a case, for example, when the doctor presses a switch button, the change function 454 controls to change the illumination conditions of the shadowless lamp 21 and the control conditions of the endoscope 31a.

[0151] (Variation 3) The medical device 40 according to the second embodiment may also be provided with an "attention mode," as in the first embodiment. In such a case, the change function 454 further changes the display conditions of the captured image at the timing of an attention call during the procedure. For example, by switching to the "attention mode," the change function 454 changes the display state so that the endoscopic image is displayed in a color that calls attention when the treatment unit of the endoscope 31a approaches a blood vessel or nerve during endoscopic surgery. In such a case, for example, the change function 454 acquires the position of the treatment unit of the endoscope 31a detected by the position sensor, monitors the distance between the acquired position of the treatment unit and the position of the blood vessel or nerve in the simulation model, and controls to change the display state of the endoscopic image when the distance becomes equal to or less than a threshold.

[0152] (Variation 4) The medical device 40 according to the second embodiment can also perform control according to the elapsed time of the procedure, as in the first embodiment. Specifically, the change function 454 further changes the illumination conditions of the lighting device according to the elapsed time since the start of surgical treatment. For example, if the shadowless lamp 21 is configured to change the intensity of red, blue, and green colors, the change function 454 can control the color according to the elapsed time of the surgery. For example, the change function 454 controls the blue light emitted from the shadowless lamp 21 to gradually decrease depending on the elapsed time since the start of surgical treatment. Because blue light causes eye fatigue, such control can reduce eye fatigue.

[0153] Furthermore, looking at red for a long period of time causes eye fatigue. Therefore, the change function 454 controls the amount of red light emitted from the shadowless lamp 21 to gradually decrease as the total amount of bleeding during surgery increases. For example, the change function 454 estimates the amount of bleeding during an abdominal surgery based on images from a camera installed in the operating room 1. Then, the change function 454 controls the amount of red light emitted from the shadowless lamp 21 to decrease as the total amount of bleeding increases. This makes it possible to reduce eye fatigue and optical illusions.

[0154] Here, the change function 454 can also control the red light to return to its original intensity if a large amount of bleeding occurs instantaneously, so that the field of view appears red, providing a visual warning that a large amount of bleeding has occurred.

[0155] (Variation 5) In the above embodiment, the illumination conditions of the shadowless lamp 21 and the control conditions of the endoscope 31a are changed in response to changes in the display state of the medical image. However, the medical device 40 can also output information based on the display state of the medical image. Specifically, the medical device 40 can output support information based on a simulation using the display state of the medical image.

[0156] For example, when the lighting conditions of the shadowless lamp 21 are determined based on the display state of the medical image, the medical device 40 can output, as support information, a position where the doctor should stand so that a shadow is not cast. In this case, when the lighting conditions of the shadowless lamp 21 are determined using the simulation model described above, the modification function 454 determines whether a shadow will be cast on the target area when a 3D model of the doctor is placed at each position in the virtual space of the simulation model. That is, the modification function 454 identifies a position in the virtual space where a shadow will not be cast on the target area even when the 3D model of the doctor is placed. The control function 451 displays, as support information, the position in the operating room 1 corresponding to the position identified by the modification function 454 on the display 43 or the display 23. By viewing this information, the doctor can easily identify a position where a shadow will not be cast.

[0157] Furthermore, for example, when determining the control conditions of the endoscope 31a based on the display state of the medical image, the medical device 40 can output the insertion position of the insertion portion as support information. For example, when determining the control conditions of the endoscope 31a using the above-mentioned simulation model, the change function 454 determines, on the three-dimensional model, an insertion position of the insertion portion that will not cast a shadow inside the subject. The control function 451 displays, on the display 43 or the display 23, the position on the three-dimensional model identified by the change function 454 as support information. By looking at this information, the doctor can easily understand the insertion position of the insertion portion that will not cast a shadow.

[0158] As described above, according to the second embodiment, the acquisition function 452 acquires the display conditions of a medical image of a target area for surgical treatment on a subject. The change function 454 changes the appearance of the target area observed during a surgical procedure based on the display conditions of the medical image. Therefore, the medical device 40 according to the second embodiment can display the appearance of the target area according to the display state of the medical image, making it possible to make the display state of the target area included in the medical image similar to the appearance of the target area observed during surgical treatment.

[0159] Furthermore, according to the second embodiment, the input interface 42 accepts a change operation to change the display conditions of the medical image. The acquisition function 452 acquires the display conditions of the medical image after the change operation. The change function 454 changes the irradiation conditions of the lighting equipment used during the surgical procedure so that the appearance of the target site observed during the surgical procedure resembles the display state of the medical image whose display conditions have been changed by the change operation. Therefore, the medical device 40 according to the second embodiment makes it possible to observe the target site with an appearance similar to the display state when observing the medical image.

[0160] Furthermore, according to the second embodiment, the input interface 42 accepts a change operation to change the display conditions of the medical image. The acquisition function 452 acquires the display conditions of the medical image after the change operation. The change function 454 changes the display conditions of the captured image so that the appearance of the target area in the captured image captured during the surgical procedure resembles the display state of the medical image whose display conditions have been changed by the change operation. Therefore, the medical device 40 according to the second embodiment makes it possible to observe the captured image similar to the display state when the medical image was observed.

[0161] Furthermore, according to the second embodiment, the change function 454 further changes the display state of the captured image at the timing of calling attention during the procedure. Therefore, the medical device 40 according to the second embodiment can support performing the procedure more safely.

[0162] Furthermore, according to the second embodiment, the change function 454 further changes the illumination conditions of the lighting equipment depending on the time elapsed since the start of the surgical treatment. Therefore, the medical device 40 according to the second embodiment makes it possible to change the lighting environment depending on the condition of the surgeon performing the surgery.

[0163] According to the second embodiment, the change function 454 changes at least one of the illumination angle, illuminance, wavelength, color temperature, number, distance, and spread as the illumination conditions of the lighting device. Therefore, the medical device 40 according to the second embodiment makes it possible to appropriately change the light emitted from the lighting device.

[0164] (Third embodiment) In the third embodiment, variations in the display of medical images and captured images (e.g., endoscopic images) will be described. The medical device 40 according to the third embodiment differs from the first and second embodiments in the processing performed by the control function 451. This point will be mainly described below.

[0165] The control function 451 according to the third embodiment displays a medical image and a captured image in a superimposed manner. Fig. 11 is a diagram showing an example of a display image according to the third embodiment. For example, as shown in Fig. 11, the control function 451 can display a display image in which a medical image and an endoscopic image are superimposed on the display 43 and the display 31b.

[0166] Here, the control function 451 displays a display image in which the images are superimposed while matching the brightness and darkness conditions, color conditions, etc. That is, as described in the first or second embodiment, the control function 451 displays a display image in which the display states of the medical image and the captured image (endoscopic image) are superimposed together in a similar state.

[0167] Furthermore, the control function 451 can further change the display state of each image so that light reflection in each image is reduced, and can also display a display image in which these images are superimposed.

[0168] The control function 451 can also display a display image in which each image (medical image and photographed image) before the display state is made similar is superimposed. For example, a photographed image including light reflection and a medical image not including light reflection can be superimposed and displayed. This allows, for example, parts of the photographed image that are difficult to see due to light reflection to be confirmed in the medical image.

[0169] As described above, according to the third embodiment, the control function 451 displays a medical image and a captured image in a superimposed manner. Therefore, the medical device 40 according to the third embodiment makes it possible to display a superimposed image that is easy to observe.

[0170] (Other embodiments) In the above-described embodiment, a case has been described in which the display state of a medical image is changed to resemble the appearance of the target site (the brightness and darkness of each part, color, and shadow state). However, the embodiment is not limited to this, and for example, the display state of a medical image may be changed in accordance with the progress of surgery. In such a case, for example, the acquisition function 452 acquires information regarding the progress of an abdominal surgery. As an example, the acquisition function 452 acquires video captured by a camera installed in the operating room 1. Then, the acquisition function 452 analyzes the acquired video to acquire information regarding the progress of the surgery (for example, whether or not the abdominal surgery has been performed).

[0171] When the progress status acquired by the acquisition function 452 is before laparotomy, the change function 454 controls the generation function 453 to generate a medical image showing the body surface by generating a medical image by surface rendering. When the progress status acquired by the acquisition function 452 is after laparotomy, the change function 454 controls the generation function 453 to generate a medical image by volume rendering and change the transparency to generate a medical image in which blood vessels and the like are visible.

[0172] In the above-described embodiment, an example has been described in which the control unit, acquisition unit, generation unit, and change unit in this specification are realized by the control function, acquisition function, generation function, and change function of a processing circuit, respectively, but the embodiment is not limited to this. For example, the control unit, acquisition unit, generation unit, and change unit in this specification may be realized by hardware only, software only, or a combination of hardware and software, in addition to being realized by the control function, acquisition function, generation function, and change function described in the embodiment.

[0173] Furthermore, the term "processor" used in the description of the above-mentioned embodiments refers to circuits such as a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)). Here, instead of storing a program in a memory circuit, the processor may be configured so that the program is directly embedded in the circuit. In this case, the processor realizes its function by reading and executing the program embedded in the circuit. Furthermore, each processor in the present embodiment is not limited to being configured as a single circuit for each processor, but may also be configured as a single processor by combining multiple independent circuits to realize its function.

[0174] Here, the control program executed by the processor is provided in advance in a read-only memory (ROM) or a storage circuit. The control program may be provided by being recorded on a computer-readable, non-transitory storage medium such as a compact disk (CD)-ROM, a flexible disk (FD), a recordable CD-R (CD-R), or a digital versatile disk (DVD) in a format that can be installed or executed by these devices. The control program may also be stored on a computer connected to a network such as the Internet and provided or distributed by being downloaded via the network. For example, the control program may be composed of modules including the above-described processing functions. In actual hardware, a CPU reads and executes the medical image processing program from a storage medium such as a ROM, whereby each module is loaded into a main memory device and generated on the main memory device.

[0175] In addition, in the above-described embodiments and modifications, the components of each device shown in the drawings are functional concepts and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution or integration of each device is not limited to that shown in the drawings, and all or part of the devices can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0176] Furthermore, among the processes described in the above-mentioned embodiments and modifications, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method.In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

[0177] According to at least one of the embodiments described above, the display state of a target site included in a medical image can be made to resemble the appearance of the target site observed during surgical treatment.

[0178] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are intended to be included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0179] 40 Medical equipment 451 Control Functions 452 Acquisition Function 453 Generation function 454 Change Function

Claims

1. an acquisition unit that acquires appearance information of a target site observed during a procedure in a surgical treatment on a subject; a change unit that changes a display state of a medical image of the target region based on the appearance information; A medical device comprising:

2. the acquisition unit acquires appearance information of the target site based on illumination conditions of a lighting device used during a procedure in the surgical treatment; The medical device according to claim 1 , wherein the change unit changes the display state of the medical image so that the display state of the target region in the medical image resembles the appearance of the target region indicated in the appearance information.

3. the acquisition unit acquires appearance information of the target site in a captured image based on a display condition of the captured image of the target site captured during the surgical treatment procedure; The medical device according to claim 1 , wherein the change unit changes the display state of the medical image so that the display state of the target region in the medical image resembles the appearance of the target region indicated in the appearance information.

4. The medical device according to claim 2 , wherein the acquisition unit acquires the appearance information of the target site based on an illumination condition of a lighting device used during the procedure.

5. the acquisition unit acquires appearance information of the target site based on irradiation conditions of the lighting device for each progress state of the procedure; The medical device according to claim 2 , wherein the change unit changes the display state of the medical image based on the acquired appearance information each time the appearance information is acquired.

6. The medical device according to claim 3 , wherein the change unit changes the display state of the medical image in accordance with a remote operator observing the captured image of the target area when the surgical treatment is performed by remote control.

7. The medical device according to claim 1 , wherein the change unit further changes the display state of the medical image at a timing when a warning is issued during the procedure.

8. The medical device according to claim 1 , wherein the change unit further changes a display state of the medical image in explaining the surgical treatment to the subject.

9. The medical device according to claim 1 , wherein the change unit further changes the display state of the medical image in accordance with the elapsed time since the start of the surgical treatment.

10. The medical device according to claim 3 , further comprising a display control unit that displays the medical image and the captured image in a superimposed manner.

11. 11. The medical device according to claim 1, wherein the change unit changes at least one of brightness and color as the display state of the medical image.

12. an acquisition unit that acquires display conditions for a medical image of a target area for surgical treatment on a subject; a change unit that changes the appearance of the target site observed during the surgical procedure based on the display conditions of the medical image; A medical device comprising:

13. a reception unit that receives a change operation to change the display conditions of the medical image; the acquisition unit acquires the display conditions of the medical image after the change operation; 13. The medical device according to claim 12, wherein the change unit changes the illumination conditions of a lighting device used during a procedure in the surgical treatment so that an appearance of the target area observed during the procedure in the surgical treatment resembles a display state of the medical image whose display conditions have been changed by the change operation.

14. a reception unit that receives a change operation to change the display conditions of the medical image; the acquisition unit acquires the display conditions of the medical image after the change operation; The medical device according to claim 12, wherein the change unit changes the display conditions of the captured image so that the appearance of the target area in the captured image taken during the surgical treatment procedure resembles the display state of the medical image whose display conditions have been changed by the change operation.

15. The medical device according to claim 14 , wherein the change unit further changes the display conditions of the captured image at a timing when a warning is issued during the procedure.

16. The medical device according to claim 13 , wherein the change unit further changes the irradiation conditions of the lighting device in accordance with the elapsed time since the start of the surgical treatment.

17. The medical device according to claim 14 , further comprising a display control unit that displays the medical image and the captured image in a superimposed manner.

18. The medical device according to claim 13 , wherein the change unit changes at least one of the illumination conditions of the lighting device, including an angle of illumination, illuminance, wavelength, color temperature, number, distance, and spread.

19. Obtaining appearance information of a target area observed during a surgical procedure for a subject; changing a display state of a medical image of the target region based on the appearance information; A control method comprising:

20. Acquire display conditions for medical images of a target area for surgical treatment of a subject; changing the appearance of the target site observed during the surgical procedure based on the display conditions of the medical image; A control method comprising:

Images