Medical device, medical system, method for operating medical device, and program for operating medical device

Abstract

A medical device according to the present disclosure is provided with: a generation unit that generates a fluorescence image based on fluorescence generated by excitation light that excites a glycosylation end product generated by burning using an energy device; a fluorescence region determination unit that determines, on the basis of the output information of the energy device and the fluorescence image, an off-time fluorescence generation region generated during a period in which the output of the energy device is in an off state; and a control unit that, if it is determined by the fluorescent region determination unit that the fluorescent region is generated when closed, executes notification processing that notifies that the fluorescent region is generated while the output of the energy device is in the closed state.

Description

Technical Field

[0001] The present disclosure relates to a medical device, a medical system, an operating method of the medical device, and an operating program of the medical device that performs image processing on an imaging signal of an image of a subject and outputs the resultant image. Background Art

[0002] Conventionally, there is known a technique in which a surgical endoscope is inserted into a subject, and the operator performs treatment by cauterizing the living tissue using a treatment instrument such as an energy device while observing the treatment portion (see, for example, Patent Document 1).

[0003] Furthermore, when biological tissue is burned, it undergoes thermal denaturation, producing advanced glycation end-products (AGEs), also known as "scorches." These AGEs fluoresce when exposed to light of a specific wavelength. By observing the fluorescence emitted by the AGEs, the surgeon can identify the thermally denatured area at the treatment site.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: International Publication No. 2020 / 174666 Summary of the Invention

[0007] Problems to be solved by the invention

[0008] Even if the treatment device's output is turned off after treatment, the device remains at a high temperature immediately after the treatment, and thus may undergo thermal denaturation when in contact with living tissue. To ensure proper treatment, it is desirable to identify AGEs generated during the time when the output is turned off.

[0009] The present disclosure has been made in view of the above circumstances, and an object of the present disclosure is to provide a medical device, a medical system, an operating method of the medical device, and an operating procedure of the medical device that enable a surgeon to understand thermal denaturation occurring when the output of a treatment instrument is turned off.

[0010] Solutions for solving problems

[0011] To solve the aforementioned problems and achieve the objectives, the medical device of the present disclosure includes: a generating unit that generates a fluorescence image based on fluorescence generated by excitation light, the excitation light being light that excites a substance generated by cauterization using an energy device; a fluorescence region determining unit that determines, based on output information from the energy device and the fluorescence image, an off-time fluorescence region generated while the output of the energy device is in an off state; and a control unit that, when the fluorescence region determining unit determines that the off-time fluorescence region is generated, executes a notification process in which notification is given of the generation of the fluorescence region while the output of the energy device is in an off state.

[0012] In addition, in the medical device according to the present disclosure, in the above-mentioned disclosure, the generating unit generates a first fluorescent image and a second fluorescent image captured later than the first fluorescent image, and the fluorescent area determining unit determines the presence or absence of a new fluorescent area that exists only in the second fluorescent image based on the first fluorescent image and the second fluorescent image. If the new fluorescent area exists, the generating unit determines whether the new fluorescent area corresponds to the fluorescent area generated during shutdown based on the output information of the energy device.

[0013] In the medical apparatus according to the present disclosure, in the above disclosure, the first fluorescent image is captured when the output of the energy device is on, and the second fluorescent image is captured when the output of the energy device is off.

[0014] In addition, the medical device according to the present disclosure further includes an extraction unit that extracts fluorescent areas in the first fluorescent image and the second fluorescent image, and the fluorescent area determination unit compares the fluorescent areas extracted by the extraction unit to determine the presence or absence of the new fluorescent area.

[0015] In the medical device according to the present disclosure described above, the fluorescence is light generated by the excitation of the substance.

[0016] In the medical device according to the present disclosure, in the above disclosure, the substance is an advanced glycation end product produced by thermal denaturation.

[0017] In the medical apparatus according to the present disclosure, in the above disclosure, the fluorescence area determination unit determines the presence or absence of a fluorescence area generated during the shutdown period, with respect to a fluorescence area in a second fluorescence image captured after a predetermined time has elapsed from when the energy device was switched off.

[0018] In the medical device according to the present disclosure, in the above disclosure, the generating unit generates a display image that displays the fluorescence generating region during shutdown and the fluorescence regions other than the fluorescence generating region during shutdown in different formats.

[0019] In the medical apparatus according to the present disclosure, in the above disclosure, the generating unit generates the display image in which the fluorescence region generated during OFF and the fluorescence region other than the fluorescence region generated during OFF are displayed in different formats on the second fluorescence image.

[0020] In addition, in the above-mentioned disclosure of the medical device involved in the present disclosure, the generating unit generates a white light image based on reflected light when white light is irradiated on biological tissue and return light from the biological tissue, and the generating unit generates the display image in which the fluorescence area generated when the off state is used and the fluorescence areas other than the fluorescence area generated when the off state are used are displayed in different styles on the white light image.

[0021] The medical device according to the present disclosure includes: a fluorescence region determination unit that determines, based on output information of an energy device and a fluorescence image based on fluorescence generated by excitation light, an off-time fluorescence region generated while the output of the energy device is in an off state, wherein the excitation light is light that excites a substance generated by ablation using the energy device; and a control unit that, when the fluorescence region determination unit determines that the off-time fluorescence region is generated, executes a notification process in which notification is given of the generation of the fluorescence region while the output of the energy device is in the off state.

[0022] The medical system according to the present disclosure includes: an imaging device for imaging a subject; a light source device capable of irradiating excitation light that excites a substance generated by thermal treatment of living tissue; and a control device, the imaging device being detachably attachable to the control device, the control device being capable of communicating with a control device for controlling an energy device that is a target for cauterization treatment, wherein the control device includes: a generating unit for generating a fluorescence image based on fluorescence generated by the excitation light, the excitation light being light that excites a substance generated by cauterization using the energy device; a fluorescence region determining unit for determining, based on output information from the energy device and the fluorescence image, an off-time fluorescence region generated while the output of the energy device is in an off state; and a control unit for executing, when the fluorescence region determining unit determines that the off-time fluorescence region is the off-time fluorescence region, a notification process for notifying that the fluorescence region has been generated while the output of the energy device is in an off state.

[0023] In addition, the operating method of the medical device involved in the present disclosure is an operating method of the medical device performed by the medical device, and the operating method includes the following steps: a generation step, in which a generation unit generates a fluorescence image based on fluorescence generated by excitation light, the excitation light being light that excites a substance generated by ablation using an energy device; a fluorescence area determination step, in which a fluorescence area determination unit determines, based on output information of the energy device and the fluorescence image, an off-time fluorescence area generated while the output of the energy device is in an off state; and a notification step, in which a control unit notifies that the fluorescence area has been generated while the output of the energy device is in an off state, when the fluorescence area determination unit determines that the off-time fluorescence area is generated.

[0024] In addition, the operating program of the medical device involved in the present disclosure is an operating program of the medical device executed by the medical device, and the operating program causes the following steps to be executed: a generation step of generating a fluorescence image based on fluorescence generated by excitation light, the excitation light being light that excites a substance generated by ablation using an energy device; a fluorescence area determination step of determining, based on output information of the energy device and the fluorescence image, a fluorescence area generated during an off-state, which is generated while the output of the energy device is in an off-state; and a notification step of notifying that the fluorescence area was generated during the off-state when the fluorescence area determination step determines that the fluorescence area was generated during the off-state.

[0025] Effects of the Invention

[0026] According to the present disclosure, there is an effect of enabling a surgeon to understand thermal denaturation occurring when the output of a treatment instrument is turned off. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a diagram showing a schematic configuration of an endoscope system according to the first embodiment.

[0028] Figure 2 This is a diagram showing a schematic configuration of a treatment system to which the endoscope system according to the first embodiment is connected.

[0029] Figure 3 This is a block diagram showing the functional configuration of the main parts of the endoscope system according to the first embodiment.

[0030] Figure 4 This is a diagram schematically showing wavelength characteristics of light emitted by the first light source unit and the second light source unit according to the first embodiment.

[0031] Figure 5 This is a diagram schematically showing the structure of a pixel portion according to Embodiment 1.

[0032] Figure 6 This is a diagram schematically showing the structure of the color filter according to Embodiment 1.

[0033] Figure 7 It is a diagram schematically showing the sensitivity characteristics of each filter.

[0034] Figure 8A Schematically shows the signal value of the R pixel of the image sensor according to the first embodiment.

[0035] Figure 8B Schematically showing the signal values ​​of the G pixels of the image sensor according to the first embodiment.

[0036] Figure 8C This is a diagram schematically showing the signal value of the B pixel of the image sensor according to the first embodiment.

[0037] Figure 9 This is a diagram schematically showing the structure of the cut filter according to the first embodiment.

[0038] Figure 10 This is a diagram schematically showing the transmission characteristics of the cut filter according to the first embodiment.

[0039] Figure 11 This is a diagram schematically showing the observation principle in the normal light observation mode according to the first embodiment.

[0040] Figure 12 This is a diagram schematically showing the observation principle in the heat treatment observation mode according to the first embodiment.

[0041] Figure 13 This is a flowchart for explaining the thermal denaturation region determination process using the endoscope system according to the first embodiment.

[0042] Figure 14 It is a diagram for explaining a fluorescent image in the fluorescence observation mode.

[0043] Figure 15 This is a flowchart for explaining a thermally denatured region determination process using an endoscope system according to a modification of the first embodiment.

[0044] Figure 16 This is a diagram showing a schematic configuration of an endoscope system according to the second embodiment.

[0045] Figure 17 This is a block diagram showing the functional configuration of the main parts of the endoscope system according to the second embodiment.

[0046] Figure 18This is a diagram showing a schematic configuration of a surgical microscope system according to a third embodiment. DETAILED DESCRIPTION

[0047] The following is a method for implementing the present disclosure and the attached Figure 1 Detailed description will be given below. In addition, the present disclosure is not limited to the following embodiments. In addition, the figures referred to in the following description are merely schematic illustrations of shapes, sizes and positional relationships to the extent that the contents of the present disclosure can be understood. That is, the present disclosure is not limited to the shapes, sizes and positional relationships illustrated in the figures. Moreover, in the description of the drawings, the same parts are marked with the same figure numbers for description. In addition, as an example of the endoscope system involved in the present disclosure, an endoscope system including a rigid endoscope and a medical imaging device is described.

[0048] (Implementation Method 1)

[0049] [Structure of the endoscope system]

[0050] Figure 1 This is a diagram showing a schematic configuration of an endoscope system according to the first embodiment. Figure 1 The illustrated endoscope system 1 is a system used in the medical field for observing biological tissue within a subject, such as a living organism. The endoscope system 1 is used when performing surgery or treatment on a subject using a treatment instrument (not shown), such as an energy device capable of thermal treatment. The operator performs surgery or treatment while observing a display device displaying an observation image based on image data captured by a medical imaging device.

[0051] The endoscope system 1 includes an insertion portion 2 , a light source device 3 , a light guide 4 , an endoscopic camera head 5 (medical imaging device), a first transmission cable 6 , a display device 7 , a second transmission cable 8 , a control device 9 , and a third transmission cable 10 .

[0052] The insertion portion 2 is rigid or at least partially flexible and has an elongated shape. The insertion portion 2 is inserted into a subject such as a patient via a cannula. The insertion portion 2 is internally provided with an optical system such as a lens for forming an observation image.

[0053] One end of the light guide 4 is connected to the light source device 3. Under the control of the control device 9, the light source device 3 supplies illumination light to the end of the light guide 4 for irradiation into the subject. The light source device 3 is implemented using the following components: one or more light sources such as LEDs (light-emitting diodes), xenon lamps, and semiconductor laser elements such as LDs (laser diodes); a processor serving as a processing device comprising hardware such as an FPGA (field programmable gate array) and a CPU (central processing unit); and a memory serving as a temporary storage area used by the processor.

[0054] One end of the light guide 4 is detachably connected to the light source device 3 , and the other end is detachably connected to the insertion portion 2 . The light guide 4 guides illumination light supplied from the light source device 3 from one end to the other end, and supplies the illumination light to the insertion portion 2 .

[0055] The endoscope camera head 5 is detachably connected to the eyepiece portion 21 of the insertion portion 2. Under the control of the control device 9, the endoscope camera head 5 receives the observation image formed by the insertion portion 2 and performs photoelectric conversion to generate an imaging signal (RAW data). The imaging signal is output to the control device 9 via the first transmission cable 6.

[0056] One end of the first transmission cable 6 is detachably connected to the control device 9 via a video connector 61, and the other end is detachably connected to the endoscopic camera 5 via a camera connector 62. The first transmission cable 6 transmits the imaging signal output from the endoscopic camera 5 to the control device 9, and transmits setting data and power output from the control device 9 to the endoscopic camera 5. Here, the setting data refers to control signals, synchronization signals, clock signals, and the like for controlling the endoscopic camera 5.

[0057] The display device 7 displays an observation image based on an imaging signal processed by the control device 9 and various information related to the endoscope system 1 under the control of the control device 9. The display device 7 is implemented using a display monitor such as liquid crystal or organic EL (Electro Luminescence).

[0058] One end of the second transmission cable 8 is detachably connected to the display device 7 , and the other end is detachably connected to the control device 9 . The second transmission cable 8 transmits the image signal processed by the control device 9 to the display device 7 .

[0059] The control device 9 is implemented using the following components: a processor (including hardware such as a GPU (Graphics Processing Unit), FPGA, or CPU) as a processing device; and memory as a temporary storage area used by the processor. The control device 9 comprehensively controls the operation of the light source device 3, the endoscopic camera head 5, and the display device 7 via the first transmission cable 6, the second transmission cable 8, and the third transmission cable 10, in accordance with a program stored in the memory. Furthermore, the control device 9 performs various image processing on the imaging signal input via the first transmission cable 6 and outputs it to the second transmission cable 8.

[0060] One end of the third transmission cable 10 is detachably connected to the light source device 3 , and the other end is detachably connected to the control device 9 . The third transmission cable 10 transmits control data from the control device 9 to the light source device 3 .

[0061] [Functional structure of disposal system]

[0062] Next, the configuration of a treatment system 100 connected to the above-described endoscope system 1 will be described. Figure 2 1 is a diagram showing a schematic configuration of a treatment system to which the endoscope system according to the first embodiment is connected. Figure 2 In FIG. 1 , one side along the central axis Ax of the treatment instrument is referred to as the distal end side Ar1 , and the other side is referred to as the proximal end side Ar2 .

[0063] The treatment system 100 treats a target area of ​​biological tissue (hereinafter referred to as the target area) by applying ultrasonic energy and high-frequency energy to the target area. Furthermore, the treatment system of this embodiment can perform treatments such as coagulation to seal the target area, incision, or simultaneous coagulation and incision. Furthermore, the treatment system 100 includes a treatment instrument 110 and a treatment instrument control device 120.

[0064] The treatment instrument 110 is an ultrasonic treatment instrument that treats a target area by applying ultrasonic energy and high-frequency energy to the target area, and corresponds to the surgical device of the present invention. The treatment instrument 110 includes a handpiece 111 and an ultrasonic transducer 112 .

[0065] The handpiece 111 includes a holding housing 113 , a movable handle 114 , a switch 115 , a rotation knob 116 , a tube 117 , a jaw 118 , and a vibration transmitting member 119 .

[0066] The ultrasonic transducer 112 includes a TD (transducer) case 112 a and an ultrasonic vibrator 112 b .

[0067] The TD housing 112 a supports the ultrasonic transducer 112 b and is detachably connected to the holding housing main body 113 a .

[0068] The ultrasonic vibrator 112b generates ultrasonic vibrations under the control of the treatment instrument control device 120. In the present embodiment, the ultrasonic vibrator 112b is composed of a BLT (Bolt-Tightened Langevin Type Vibrator).

[0069] The holding shell 113 constitutes the appearance of the treatment instrument 110 and supports the entire treatment instrument 110. The holding shell 113 includes: a substantially cylindrical holding shell body 113a coaxial with the central axis Ax; and a fixed handle 113b extending from the main body of the holding shell 113 to the Figure 2 It extends below the cervix and is held by the operator.

[0070] The movable handle 114 is opened and closed by an operator such as a surgeon. This opening and closing operation is an operation of opening and closing the jaw 118 relative to an end portion 119 a on the distal end side Ar1 of the vibration transmission member 119 .

[0071] The switch 115 is provided so as to be exposed to the outside from the side surface of the front end Ar1 of the fixed handle 113b. Furthermore, the switch 115 receives treatment operations by an operator, such as a surgeon. This treatment operation involves applying ultrasonic energy or high-frequency energy to the target area. If the switch 115 has multiple buttons, each button is assigned an operation instruction.

[0072] The rotation knob 116 has a generally cylindrical shape coaxial with the central axis Ax and is located on the distal end side Ar1 of the holding housing body 113a. Furthermore, the rotation knob 116 is rotated by an operator, such as a surgeon. This rotational operation causes the rotation knob 116 to rotate relative to the holding housing body 113a about the central axis Ax. Furthermore, rotation of the rotation knob 116 causes the conduit 117, jaws 118, and vibration transmission member 119 to rotate about the central axis Ax.

[0073] The conduit 117 is a cylindrical conduit. A pin (not shown) that rotatably supports the jaw 118 is fixed to the end portion of the conduit 117 on the distal end side Ar1.

[0074] At least a portion of the jaws 118 is made of a conductive material. The jaws 118 open and close relative to the end 119a of the distal end Ar1 of the vibration transmitting member 119 as the operator grips the movable handle 114, thereby gripping the target area.

[0075] The vibration transmission member 119 is made of a conductive material and has a long strip shape extending linearly along the central axis Ax. In addition, the vibration transmission member 119 is inserted into the pipeline 117 in a state where the end 119a of the front end side Ar1 protrudes to the outside. At this time, although the end of the base end side Ar2 in the vibration transmission member 119 is omitted in the specific figure, it is mechanically connected to the ultrasonic transducer 112. That is, the vibration transmission member 12 transmits the ultrasonic vibration generated by the ultrasonic transducer 112 from the end of the base end side Ar2 to the end 119a of the front end side Ar1. In this embodiment, the ultrasonic vibration is a longitudinal vibration that vibrates in the direction along the central axis Ax.

[0076] The treatment instrument control device 120 comprehensively controls the operation of the treatment instrument 110 via the electric cable 130 .

[0077] Specifically, the treatment instrument control device 120 detects a treatment operation of the switch 115 by an operator, such as a surgeon, via the electrical cable 130. Upon detecting this treatment operation, the treatment instrument control device 120 applies ultrasonic energy or high-frequency energy to the target area grasped between the jaws 118 and the end 119a of the distal end Ar1 of the vibration transmission member 119 via the electrical cable 130. In other words, the treatment instrument control device 120 performs treatment on the target area.

[0078] For example, when imparting ultrasonic energy to the target area, the treatment instrument control device 120 supplies driving power to the ultrasonic vibrator 112b via the electrical cable 130. As a result, the ultrasonic vibrator 112b generates longitudinal vibrations (ultrasonic vibrations) that vibrate in the direction along the central axis Ax. Furthermore, the end 119a of the front end side Ar1 of the vibration transmission member 119 vibrates at a desired amplitude due to this longitudinal vibration. Furthermore, ultrasonic vibrations are imparted from the end 119a to the target area held between the jaws 118 and the end 119a. In other words, ultrasonic energy is imparted from the end 119a to the target area.

[0079] Furthermore, for example, when applying high-frequency energy to a target site, the treatment instrument control device 120 supplies high-frequency power between the jaws 118 and the vibration transmission member 119 via the electrical cable 130. This causes a high-frequency current to flow through the target site, which is grasped between the jaws 118 and the end 119a of the distal end Ar1 of the vibration transmission member 119. In other words, high-frequency energy is applied to the target site.

[0080] Furthermore, the treatment instrument control device 120 is communicably connected to the control device 9 , and when the switch 115 is pressed, the treatment instrument control device 120 outputs a signal indicating that the switch has been pressed to the control device 3 .

[0081] [Functional structure of the main parts of the endoscope system]

[0082] Next, the functional configuration of the main parts of the endoscope system 1 will be described. Figure 3 1 is a block diagram showing the functional configuration of the main parts of the endoscope system 1 .

[0083] [Structure of the insertion part]

[0084] First, the structure of the insertion portion 2 will be described. The insertion portion 2 includes an optical system 22 and an illumination optical system 23 .

[0085] The optical system 22 forms an image of the subject by converging reflected light from the subject, return light from the subject, excitation light from the subject, and luminescence emitted by the subject. The optical system 22 is implemented using one or more lenses.

[0086] The illumination optical system 23 irradiates the subject with illumination light supplied from the light guide 4. The illumination optical system 23 is implemented using one or more lenses and the like.

[0087] [Structure of light source device]

[0088] Next, a description will be given of the configuration of the light source device 3 . The light source device 3 includes a condenser lens 30 , a first light source unit 31 , a second light source unit 32 , and a light source control unit 33 .

[0089] The condenser lens 30 condenses the light emitted from the first light source unit 31 and the second light source unit 32 and emits the light toward the light guide 4 .

[0090] Under the control of the light source control unit 33, the first light source unit 31 emits white light (normal light) as visible light, thereby supplying white light to the light guide 4 as illumination light. The first light source unit 31 is constructed using a collimating lens, a white LED lamp, and a driver. Alternatively, the first light source unit 31 can simultaneously emit red, green, and blue LED lamps to supply visible white light. Of course, the first light source unit 31 can also be constructed using a halogen lamp, a xenon lamp, or the like.

[0091] Under the control of the light source control unit 33, the second light source unit 32 emits narrowband light in a wavelength range different from and narrower than that of white light, thereby supplying narrowband light as illumination light to the light guide 4. Here, narrowband light refers to, for example, light in the wavelength range of 400 nm to 430 nm with a center wavelength of 415 nm. The second light source unit 32 is implemented using a collimating lens, a semiconductor laser such as a violet laser diode (LD), and a driver. Furthermore, in the present embodiment, the narrowband light functions as excitation light for exciting advanced glycation end products produced by heat treatment of biological tissue.

[0092] The light source control unit 33 is implemented using a processor (e.g., an FPGA or CPU) as a processing device and a memory (e.g., a temporary storage area) used by the processor. Based on control data input from the control device 9, the light source control unit 33 controls the light emission timing and duration of each of the first light source unit 31 and the second light source unit 32.

[0093] Here, the wavelength characteristics of the light emitted by the first light source unit 31 and the second light source unit 32 will be described. Figure 4 Schematically shows the wavelength characteristics of the light emitted by the first light source unit 31 and the second light source unit 32. Figure 4 In the figure, the horizontal axis represents wavelength (nm) and the vertical axis represents relative intensity. Figure 4 In the middle, curve L WL The curve L represents the wavelength characteristics of the white light emitted by the first light source unit 31. V The wavelength characteristics of the narrowband light (excitation light) emitted by the second light source unit 32 are shown. The second light source unit 32 emits light with a central wavelength (peak wavelength) of 415 nm and a wavelength range of 400 nm to 430 nm. Figure 4 The curve L in WL The wavelength characteristics shown represent the characteristics when a white LED is used as the first light source unit 31 .

[0094] [Structure of endoscope camera]

[0095] return Figure 3 , the structure of the endoscope system 1 is further described.

[0096] Next, the configuration of the endoscopic camera head 5 will be described. The endoscopic camera head 5 includes an optical system 51, a drive unit 52, an imaging element 53, a cut filter 54, an A / D converter 55, a P / S converter 56, an imaging and recording unit 57, and an imaging control unit 58.

[0097] The optical system 51 forms an image of the subject, focused by the optical system 22 of the insertion portion 2, onto the light-receiving surface of the imaging element 53. The optical system 51 is capable of changing the focal length and focus position. The optical system 51 is constructed using a plurality of lenses 511. The optical system 51 changes the focal length and focus position by moving each of the plurality of lenses 511 along the optical axis L1 via the drive unit 52.

[0098] The drive unit 52 moves the multiple lenses 511 of the optical system 51 along the optical axis L1 under the control of the imaging control unit 58. The drive unit 52 is configured using a motor such as a stepping motor, a DC motor, or a voice coil motor, and a transmission mechanism such as gears that transmits the motor's rotation to the optical system 51.

[0099] The imaging element 53 is implemented using a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) image sensor having a plurality of pixels arranged in a two-dimensional matrix. Under the control of the imaging control unit 58, the imaging element 53 receives the subject image (light) formed by the optical system 51 and passed through the cut filter 54, performs photoelectric conversion on the subject image, generates an imaging signal (RAW data), and outputs it to the A / D converter 55. The imaging element 53 includes a pixel unit 531 and a color filter 532.

[0100] Figure 5 : is a diagram schematically showing the structure of the pixel portion 531. The pixel portion 531 is composed of a plurality of pixels P such as photodiodes that store charges according to the amount of light. nm (n, m are integers greater than 1) are arranged in a two-dimensional matrix. The pixel unit 531 is controlled by the imaging control unit 58 to obtain a plurality of pixels P nm The pixel P in the reading area arbitrarily set as the reading object nm The image signal is read as image data and output to the A / D conversion unit 55 .

[0101] Figure 6 : This is a diagram schematically showing the structure of the color filter 532. The color filter 532 is composed of a Bayer array with 2×2 as one unit. The color filter 532 is composed of a filter R that transmits light in the red wavelength range, two filters G that transmit light in the green wavelength range, and a filter B that transmits light in the blue wavelength range. Figure 5 In the figure, the reference numerals for each filter (e.g. G 11 ) and pixel P nm Corresponding means configured at corresponding pixel positions.

[0102] Figure 7 is a diagram schematically showing the sensitivity characteristics of each filter. Figure 7 In the figure, the horizontal axis represents wavelength (nm) and the vertical axis represents transmission characteristics (sensitivity characteristics). Figure 7 In the middle, curve L B Indicates the transmission characteristics of filter B, curve L G Indicates the transmission characteristics of filter G, curve L R Indicates the transmission characteristics of filter R.

[0103] Filter B transmits light in the blue wavelength range (see Figure 7 The curve L B ). In addition, the filter G transmits light in the green wavelength range (see Figure 7 The curve L G ). In addition, the filter R transmits light in the red wavelength range (see Figure 7 The curve L R ). In addition, in the following, the filter R is arranged on the light receiving surface to form a pixel P nm The pixel P is formed by placing the filter G on the light receiving surface. nm The pixel P is formed by placing the filter B on the light receiving surface. nm This will be described as a B pixel.

[0104] According to the imaging element 53 configured in this manner, when receiving the subject image formed by the optical system 51, the imaging element 53 generates color signals (R signal, G signal, and B signal) for each of the R pixel, the G pixel, and the B pixel (see Figures 8A to 8C ).

[0105] return Figure 3 , the structure of the endoscope system 1 is further described.

[0106] The cutoff filter 54 is disposed on the optical axis L1 between the optical system 51 and the imaging element 53. The cutoff filter 54 is provided on the light-receiving surface side (incident surface side) of at least the G pixel of the color filter 532, which is provided with the filter G that transmits the green wavelength range. The cutoff filter 54 blocks light in the wavelength range of the excitation light and transmits light in the wavelength range on the long wavelength side that is longer than the wavelength range of the excitation light.

[0107] Figure 9 Schematically shows the structure of the cut filter 54. Figure 9 As shown, the filter F constituting the cutoff filter 54 11 Configured on filter G 11 (Refer to Figure 6 ) is configured at a position where the filter G 11The light-receiving side directly above the .

[0108] Figure 10 is a diagram schematically showing the transmission characteristics of the cut filter 54. Figure 10 In the figure, the horizontal axis represents wavelength (nm) and the vertical axis represents transmittance. Figure 10 In the middle, curve L F The curve L represents the transmission characteristics of the cutoff filter 54. V Indicates the wavelength characteristics of the excitation light.

[0109] The cutoff filter 54 blocks light in the wavelength range of the excitation light and transmits light in a wavelength range longer than the wavelength range of the excitation light. Specifically, the cutoff filter 54 blocks light in a wavelength range shorter than the wavelength range of the excitation light and transmits light in a wavelength range longer than the wavelength range of the excitation light.

[0110] return Figure 3 , continue to explain the structure of the endoscope camera 5.

[0111] Under the control of the imaging control unit 58, the A / D converter 55 performs A / D conversion processing on the analog imaging signal input from the imaging element 53 and outputs the resulting signal to the P / S converter 56. The A / D converter 55 is implemented using an A / D conversion circuit or the like.

[0112] Under the control of the camera control unit 58, the P / S converter 56 performs parallel / serial conversion on the digital camera signal input from the A / D converter 55, and outputs the parallel / serial converted camera signal to the control device 9 via the first transmission cable 6. The P / S converter 56 is implemented using a P / S conversion circuit, etc. In the first embodiment, an E / O converter that converts the camera signal into an optical signal may be provided in place of the P / S converter 56, and the camera signal may be output to the control device 9 via an optical signal. Alternatively, the camera signal may be transmitted to the control device 9 via wireless communication such as Wi-Fi (Wireless Fidelity) (registered trademark).

[0113] The image recording unit 57 records various information related to the endoscopic camera head 5 (e.g., pixel information of the image sensor 53 and characteristics of the cutoff filter 54). Furthermore, the image recording unit 57 records various setting data and control parameters transmitted from the control device 9 via the first transmission cable 6. The image recording unit 57 is configured using a nonvolatile memory or a volatile memory.

[0114] The imaging control unit 58 controls the operation of the drive unit 52, the imaging element 53, the A / D converter 55, and the P / S converter 56 based on the setting data received from the control device 9 via the first transmission cable 6. The imaging control unit 58 is implemented using the following components: a TG (Timing Generator); a processor as a processing device including hardware such as a CPU; and a memory as a temporary storage area used by the processor.

[0115] [Structure of control device]

[0116] Next, the configuration of the control device 9 will be described.

[0117] The control device 9 includes an S / P conversion unit 91 , an image processing unit 92 , an input unit 93 , a recording unit 94 , and a control unit 95 .

[0118] Under the control of the control unit 95, the S / P converter 91 performs serial / parallel conversion on the image data received from the endoscopic camera 5 via the first transmission cable 6 and outputs the data to the image processing unit 92. Furthermore, if the endoscopic camera 5 outputs the imaging signal via an optical signal, an O / E converter that converts the optical signal into an electrical signal may be provided in place of the S / P converter 91. Furthermore, if the endoscopic camera 5 transmits the imaging signal via wireless communication, a communication module capable of receiving wireless signals may be provided in place of the S / P converter 91.

[0119] Under the control of the control unit 95, the image processing unit 92 performs predetermined image processing on the parallel data imaging signal input from the S / P conversion unit 91 and outputs the resulting image to the display device 7. The predetermined image processing herein includes processing such as demosaicing, white balance, gain adjustment, gamma correction, and format conversion. The image processing unit 92 is implemented using a processor comprising hardware such as a GPU or FPGA as a processing device, and a memory serving as a temporary storage area for the processor. The image processing unit 92 includes a generation unit 921, an extraction unit 922, a fluorescence area determination unit 923, an output state determination unit 924, and an output unit 925.

[0120] The generator 921 generates a first image and a second image, the first image including one or more characteristic regions to be removed by the operator, and the second image including one or more cauterized regions after being cauterized by the energy device (treatment instrument 110). Specifically, the generator 921 generates a white light image as the first image based on an imaging signal generated by capturing reflected light and return light from the living tissue when white light is irradiated. Furthermore, the generator 921 generates a fluorescence image as the second image based on an imaging signal generated by capturing fluorescence generated by excitation light, which is irradiated to excite advanced glycation end products (AGEs) produced by thermal treatment of the living tissue, in a fluorescence observation mode described later. Alternatively, the generator 921 can generate a pseudo-color image including one or more characteristic regions (lesion regions) to be removed by the operator based on an imaging signal generated by capturing reflected light and return light from the living tissue when excitation light is irradiated in the fluorescence observation mode of the endoscope system 1 described later.

[0121] The extraction unit 922 extracts a fluorescence area, which is an area of ​​the fluorescence image, from the fluorescence image generated by the generation unit 921 .

[0122] The fluorescence area determination unit 923 determines whether there is a change in the fluorescence area between fluorescence images captured at different times.

[0123] The output state determination unit 924 determines the output state of the treatment instrument 110 based on the signal received by the control device 3 from the treatment instrument control device 120. Specifically, the output state determination unit 924 determines whether the output of the treatment instrument 110 is in the on state or the off state.

[0124] Based on the determination results of the fluorescence region determination unit 923 and the output state determination unit 924 , the control unit 95 sets the corresponding fluorescence region as a thermal denaturation region (fluorescence region generated when the output of the energy device (treatment instrument 110 ) is turned off).

[0125] The output unit 925 outputs the white light image, the fluorescence image, the determination result of the fluorescence area determination unit 923 , setting information set by the control unit 95 , and the like.

[0126] The input unit 93 receives input of various operations related to the endoscope system 1 and outputs the received operations to the control unit 95. The input unit 93 is configured using a mouse, a foot switch, a keyboard, buttons, switches, a touch panel, and the like.

[0127] The recording unit 94 is implemented using a recording medium such as a volatile memory, a nonvolatile memory, an SSD (Solid State Drive), an HDD (Hard Disk Drive), or a memory card. The recording unit 94 records data including various parameters required for the operation of the endoscope system 1. The recording unit 94 also includes a program recording unit 941 for recording various programs used to operate the endoscope system 1.

[0128] The control unit 95 is implemented using a processor (e.g., a hardware processor such as an FPGA or CPU) as a processing device and a memory (e.g., a temporary storage area) used by the processor. The control unit 95 comprehensively controls the various components that comprise the endoscope system 1. Furthermore, the control unit 95 receives signals from the treatment instrument control device 120 regarding the pressing of the switch 115 (output from the treatment instrument 110).

[0129] [Overview of each observation mode]

[0130] Next, an overview of each observation mode that can be executed by the endoscope system 1 will be described. The normal light observation mode and the fluorescence observation mode will be described in order.

[0131] [Overview of Normal Light Observation Mode]

[0132] First, the normal light observation mode will be described. Figure 11 This is a diagram schematically showing the observation principle in the normal light observation mode.

[0133] Under the control of the control device 9, the light source device 3 emits white light W1 having an intensity distribution shown in the graph G11 toward the biological tissue T1 of the subject by causing the first light source unit 31 to emit light. In this case, a portion of the reflected light and return light (hereinafter referred to as "reflected light WR10, reflected light WG10, and reflected light WB10") reflected by the biological tissue is shielded by the cutoff filter 54, and the remaining portion is incident on the imaging element 53. For example, specifically, the cutoff filter 54 shields the reflected light (reflected light WG10) incident on the G pixel and the reflected light in the wavelength range of the excitation light (excitation light W2 described later). That is, the reflected light and return light based on the irradiation of white light are incident on the filter R and the filter B, and the light in the wavelength range longer than the wavelength range of the excitation light is incident on the filter G. Therefore, the component of the light in the blue wavelength range incident on the pixel becomes smaller than when the cutoff filter 54 is not configured. The light incident on each filter is selectively transmitted according to the filter characteristics shown in graph G12.

[0134] Next, the image processing unit 92 acquires image data (RAW data) from the imaging element 53 of the endoscopic camera head 5 and performs image processing on the signal values ​​of the R pixels, G pixels, and B pixels contained in the acquired image data to generate a white light image. In this case, since the blue component contained in the image data is smaller than in conventional white light observation, the image processing unit 92 performs white balance adjustment processing to adjust the white balance so that the ratio of the red component, green component, and blue component is constant.

[0135] In the normal light observation mode, even when the cut filter 54 is arranged on the light receiving surface side of the G pixel, a natural white light image (observation image) can be observed.

[0136] [Overview of Fluorescence Observation Mode]

[0137] Next, the fluorescence observation mode will be described. Figure 12 This is a diagram schematically showing the observation principle in the fluorescence observation mode.

[0138] In recent years, minimally invasive treatments using endoscopes and laparoscopy have become increasingly common in the medical field. For example, widely performed minimally invasive treatments using endoscopes and laparoscopy include endoscopic submucosal dissection (ESD), laparoscopic endoscopic co-operative gastrectomy (LECS), non-exposed endoscopic wall-inversion surgery (NEWS), and transurethral resection of the bladder tumor (TUR-bt).

[0139] In these minimally invasive treatments, when performing treatment, for example, doctors or other operators use treatment instruments that emit energy such as high-frequency waves, ultrasound, microwaves, etc. to perform heat treatment as a pre-treatment to mark the surgical area, or as a treatment to remove the diseased area, close the incision, or coagulate the incision.

[0140] Furthermore, when amino compounds and reducing sugars are heated, a glycation reaction (Maillard reaction) occurs, in which the amino acids react with the reducing sugars. The end products produced by this Maillard reaction are collectively referred to as advanced glycation end products (AGEs). AGEs are known to contain substances with fluorescent properties. AGEs are known to emit fluorescence with a higher intensity than the autofluorescent substances naturally present in biological tissues. Therefore, the formation of AGEs significantly increases their fluorescence intensity compared to before the formation of AGEs.

[0141] AGEs generated by burning during treatment can be visualized by fluorescence observation, and the fluorescence intensity serves as an indicator of the state of thermal treatment.

[0142] Specifically, the fluorescence observation mode utilizes the fluorescence properties of AGEs generated in living tissues by thermal treatment with an energy device, etc., to visualize the thermally treated area. Therefore, in fluorescence observation mode, excitation light for exciting AGEs, for example, blue narrowband light with a center wavelength of 415 nm, is irradiated from the light source device 3 onto the living tissue. Consequently, in fluorescence observation mode, a thermally treated image (fluorescence image) that captures fluorescence (for example, green light with a wavelength of 490 to 625 nm) generated by AGEs can be observed.

[0143] Specifically, first, the light source device 3, under the control of the control device 9, emits the excitation light W2 (center wavelength 415nm: refer to Figure G13) by making the second light source unit 32 emit light to irradiate the biological tissue T2 (heat treatment area) to which the subject has been subjected to heat treatment by an energy device or the like. In this case, the reflected light (hereinafter referred to as "reflected light WR20, reflected light WG20, reflected light WB20") including at least the components of the excitation light W2 reflected at the biological tissue T2 (heat treatment area) and the return light is shielded by the cutoff filter 54, and a portion of the components on the long wavelength side is incident on the imaging element 53 (refer to Figure G14). In addition, Figure 11 In FIG, the thickness of the arrow represents the intensity of the component (light amount or signal value) of each line.

[0144] More specifically, if Figure 11As shown in graph G2, cutoff filter 54 blocks reflected light WG20 incident on G pixels within a wavelength range that includes the wavelength range of excitation light W2. Furthermore, cutoff filter 54 transmits fluorescence WF1 emitted by AGEs in tissue T2 (heat-treated area) (see graph G14). Consequently, reflected light WG20 is prevented from entering G pixels, while fluorescence WF1 does enter. Since cutoff filter 54 is positioned on the light-receiving side (incident side) of the G pixels, it prevents the reflected light WG20 from excitation light W2 from mixing with fluorescence WF1 and burying the fluorescence component.

[0145] Furthermore, reflected light (reflected light WR20 , WB20 ) and fluorescent light WF1 are incident on the R pixel and the B pixel, respectively.

[0146] The image processing unit 92 then acquires image data (RAW data) from the imaging element 53 of the endoscopic camera head 5 and performs image processing on the signal values ​​of the G and B pixels contained in the acquired image data to generate a fluorescence image. In this case, the signal values ​​of the G pixels include fluorescence information representing the fluorescence image emitted from the heat-treated area. Furthermore, the B pixels include background information representing the surrounding biological tissues of the heat-treated area, which serve as the background of the heat-treated area. The image processing unit 92 performs image processing such as gain control, pixel interpolation, and mucosal enhancement on the signal values ​​of the G and B pixels contained in the image data to generate a fluorescence image. In this gain control process, the image processing unit 92 performs processing to make the gain of the signal value for the G pixels greater than the gain of the signal value for the G pixels during normal light observation, while making the gain of the signal value for the B pixels less than the gain of the signal value for the B pixels during normal light observation. Furthermore, the image processing unit 92 performs processing to make the signal values ​​of the G pixels and the signal values ​​of the B pixels equal (1:1). Alternatively, the image processing unit 92 may generate a pseudo-color image by superimposing color information whose hue is changed according to the fluorescence intensity on the fluorescent image.

[0147] [Treatment using an endoscopic system]

[0148] Next, a treatment using the endoscope system 1 of the present disclosure will be described. In this case, the operator inserts the insertion portion 2 into the subject and causes the light source device 3 to illuminate the area containing the treatment target with white light. The operator confirms the treatment target while observing the observation image displayed on the display device 7.

[0149] The operator then performs treatment on the treatment target of the subject while checking the white light image displayed on the display device 7. For example, the operator inserts an energy device (treatment instrument 110) into the subject via the insertion portion 2 to cauterize and remove the treatment target.

[0150] Afterwards, the operator irradiates the treatment object with excitation light and observes the fluorescent image displayed on the display device 7. The operator determines whether the treatment (e.g., resection) at the treatment position has been completed by observing the fluorescent image displayed on the display device 7. When the operator determines that the treatment is completed, the operator ends the operation. Specifically, the operator determines whether the resection of the treatment object has been completed by observing the fluorescent image displayed on the display device 7 and the burnt area that has been burnt and removed by the treatment instrument 110. At this time, when the operator determines that the resection of the treatment object is not completed, the operator switches the observation mode of the endoscope system 1 while repeating the observation of the white light image based on the irradiation of white light and the observation of the fluorescent image based on the irradiation of excitation light to continue the treatment.

[0151] [Handling of endoscope system]

[0152] Next, the processing executed by the endoscope system 1 will be described. Figure 13 This is a flowchart for explaining a thermally denatured region determination process using an endoscope system according to one embodiment. The thermally denatured region determination process is a process executed in the fluorescence observation mode.

[0153] The control unit 95 generates a first fluorescent image (step S101). At this point, the control unit 95 controls the light source control unit 33 to cause the second light source unit 32 to emit light, irradiating the subject with excitation light. The generator 921 generates the first fluorescent image by acquiring an imaging signal from the imaging element 53 of the endoscopic camera head 5. Thus, the first fluorescent image is acquired. In this case, the output unit 925 may also cause the display device 7 to display the first fluorescent image generated by the generator 921.

[0154] Next, the control unit 95 generates a second fluorescent image (step S102). At this point, the control unit 95 controls the light source control unit 33 to cause the second light source unit 32 to emit light, irradiating the subject with excitation light. The generator 921 generates the second fluorescent image by acquiring an imaging signal from the imaging element 53 of the endoscopic camera head 5. Thus, the second fluorescent image is acquired. In this case, the output unit 925 may also cause the display device 7 to display the second fluorescent image generated by the generator 921.

[0155] The second fluorescent image is a fluorescent image based on image data acquired at a later time than the first fluorescent image. The image data acquisition time (imaging timing) is performed, for example, after a predetermined time has passed since the first fluorescent image was acquired.

[0156] Next, the control unit 95 determines whether there has been a change in the fluorescence area between the first and second fluorescence images (step S103). In this step, the extraction unit 922 extracts the area depicting the fluorescence image (fluorescent area) from each fluorescence image. The extraction unit 922 performs contour extraction based on, for example, brightness values ​​to extract one or more fluorescence areas within the image. The fluorescence area determination unit 923 then determines whether there has been a change in the fluorescence area of ​​the second fluorescence image relative to the extracted fluorescence area of ​​the first fluorescence image. In this process, the fluorescence area determination unit 923 detects a change in the fluorescence area by determining the presence of new fluorescence areas in the second fluorescence image that were not present in the first fluorescence image. If the fluorescence area determination unit 923 determines that there has been no change in the fluorescence area (step S103: No), the control unit 95 terminates the process. On the other hand, if the fluorescence area determination unit 923 determines that there has been a change in the fluorescence area (step S103: Yes), the control unit 95 proceeds to step S104.

[0157] In step S104, the control unit 95 determines whether the output status of the treatment instrument is off. In this step, the output status determination unit 924 determines whether the treatment instrument 110 was in the off or on state when the second fluorescent image was captured. The output status determination unit 924 determines whether the output of the treatment instrument 110 was on or off at the time the second fluorescent image was captured, based on a signal received from the treatment instrument control unit 120 by the control device 3, for example. Furthermore, if information indicating whether the output of the treatment instrument 110 is on or off is assigned to each fluorescent image, the output status determination unit 924 refers to this information to determine whether the output is on or off. If the output status determination unit 924 determines that the output status of the treatment instrument is on, i.e., not off (step S104: No), the control unit 95 terminates the process. On the other hand, if the output status determination unit 924 determines that the output status of the treatment instrument is off (step S104: Yes), the control unit 95 proceeds to step S105.

[0158] In step S105, the control unit 95 sets the thermally denatured region (new fluorescent region) added to the second fluorescent image as the thermally denatured region generated when the output is turned off (fluorescent region generated when the output is turned off). At this time, the control unit 95 sets the fluorescent region that is not present in the first fluorescent image among the fluorescent regions extracted by the extraction unit 922 in the second fluorescent image as the thermally denatured region generated when the output is turned off.

[0159] Here, refer to Figure 14 To illustrate the temporal changes of fluorescence images. Figure 14 This is a diagram for explaining the fluorescence image in the fluorescence observation mode. Figure 14 In the figure, it is shown that at time t10 The output of the treatment instrument 110 is turned on, and at time t 11 This is an example in which the output of the treatment instrument 110 is turned off.

[0160] exist Figure 14 For example, at a time earlier than t 10 At time t1, even if the treatment area is irradiated with excitation light, no AGEs exist and no fluorescence image (reference image PI1) is drawn. Therefore, even if the fluorescence image at this time is displayed on the display device 7, no fluorescence image (reference image PO1) is displayed.

[0161] Then, for example, at a time later than t 10 and earlier than time t 11 At time t2, when excitation light is irradiated onto the treatment area, a fluorescent image FL11 corresponding to AGEs is drawn (reference image PI2). Therefore, when the fluorescent image at this time is displayed on the display device 7, a fluorescent image FL21 (reference image PO2) is displayed. This fluorescent image FL11 (FL21) corresponds to AGEs generated by the energy applied from the treatment instrument 110, for example.

[0162] Furthermore, the fluorescent image FL11 and the fluorescent image FL21 may be displayed with the same hue, or may be displayed with a pseudo color superimposed on the fluorescent image FL21 .

[0163] Then, for example, at a time later than t 11 At time t3, new AGEs are generated by contact of the treatment instrument 110 with its output turned off, resulting in the generation of new AGEs. Fluorescence image FL12 (see image PI3) corresponding to these AGEs is drawn. Therefore, when the fluorescence image at this time is displayed on the display device 7, a fluorescence image FL22 (see image PO3) is displayed. This fluorescence image FL12 (FL21) corresponds to AGEs generated by, for example, residual heat from the treatment instrument 110.

[0164] The newly generated fluorescent image FL12 is detected as a change in the fluorescent area by the fluorescent area determination unit 923 and is set as a thermal denaturation area generated when the output is turned off. The fluorescent image FL22 displayed on the display device 7 is given information indicating that it was generated when the output was turned off, or a color indicating that it was generated when the output was turned off is superimposed. Alternatively, the following display method may be used: only the thermal denaturation area generated after the output is turned off (in the case of the output being turned off) is displayed. Figure 14 The middle is the fluorescent image FL22), which allows the thermal denaturation area to be grasped.

[0165] return Figure 13After setting the thermal denaturation area, the control unit 95 performs a notification process for the thermal denaturation area generated when the output is turned off (step S106). At this time, the control unit 95 displays information on the display device 7 indicating that a thermal denaturation area has been generated when the output of the treatment instrument 110 is turned off. For example, the fluorescent images of the comparison objects are arranged and displayed with text information indicating that a new thermal denaturation area has been generated when the output is turned off, or the fluorescent area of ​​the object on the second fluorescent image is superimposed with text information indicating that it is a thermal denaturation area when the output is turned off or a pseudo color assigned according to when the output is turned on / off. The generation unit 921 generates the above-mentioned display image according to the conditions set for the notification process. In addition, it is also possible to display information corresponding to the fluorescent area on the white light image, or to use sound or light to notify that a thermal denaturation area has been generated when the output is turned off.

[0166] For example, the thermally denatured region determination process may be executed at pre-set time intervals or when a surgeon inputs an instruction to execute the detection process. In this case, the second fluorescent image acquired during the previous process may be used as the first fluorescent image. In this case, the process may be restarted from step S102.

[0167] In the first embodiment described above, when a fluorescent area in a fluorescent image changes, the output state of the treatment device at the time the changed fluorescent image was captured is determined, and the newly generated fluorescent area is set as a fluorescent area corresponding to the thermally denatured area generated when the energy device's output is turned off (a fluorescent area generated when the output is turned off), and the operator is notified of this. According to this first embodiment, the operator can understand the thermal denaturation that occurs when the treatment device's output is turned off.

[0168] (Variation)

[0169] Next, refer to Figure 15 A modified example of Embodiment 1 will now be described. The endoscope system involved in this modified example is the same as the endoscope system 1 involved in the embodiment, and therefore its description will be omitted. Regarding the thermal denaturation region, even after the treatment instrument 110 is removed, the region may expand due to residual heat. This region expanded due to residual heat is often already known to the operator, so it does not need to be set as a new thermal denaturation region. This modified example will describe an example in which this region expanded due to residual heat is excluded from the new thermal denaturation region generated after the output is turned off.

[0170] [Handling of endoscope system]

[0171] Next, the processing executed by the endoscope system according to the modification will be described. Figure 17 This is a flowchart for explaining a thermally denatured region determination process using the endoscope system according to the modification.

[0172] The control unit 95 generates the first fluorescent light image and the second fluorescent light image and detects changes in the fluorescent area (steps S201 to S203 ), similarly to the embodiment.

[0173] If the fluorescence area changes (step S203: Yes), the control unit 95 determines whether the output state of the treatment instrument is off (step S204), similar to step S104. If it is determined that the output of the treatment instrument 110 is off at the time of capturing the second fluorescence image (step S204: Yes), the process proceeds to step S205.

[0174] In step S205, the control unit 95 determines whether a predetermined time has elapsed since the treatment instrument 110 was switched off. Specifically, the control unit 95 determines whether the second fluorescent image was captured at a predetermined time from the moment the switch 115 was pressed, turning the treatment instrument 110 off. If the control unit 95 determines that the predetermined time has not elapsed since the second fluorescent image was captured (step S205: No), the process ends. Conversely, if the control unit 95 determines that the predetermined time has elapsed since the second fluorescent image was captured (step S205: No), the process proceeds to step S206.

[0175] In step S206 , the control unit 95 sets the thermally denatured region added to the second fluorescent image as the thermally denatured region generated when the output is turned off.

[0176] After setting the thermal denaturation region, the control unit 95 performs notification processing for the thermal denaturation region generated when the output is turned off (step S207). At this time, the control unit 95 displays a message on the display device 7 indicating that the thermal denaturation region has occurred when the output of the treatment instrument 110 is turned off, similarly to step S106.

[0177] In the above-described variation, similar to the first embodiment, when the fluorescent area in the fluorescence image changes, the output state of the treatment device at the time the changed fluorescence image was captured is determined. If the output is off, the newly generated fluorescent area is defined as the thermal denaturation area generated when the energy device's output is off, and this is notified to the operator. This variation enables the operator to understand the thermal denaturation that occurs when the treatment device's output is off.

[0178] Furthermore, in this modification, even when the output of the treatment instrument 110 is turned off, any changed fluorescent area is not set as a new thermal denaturation area within a specified time period from the time of the switch-off. Therefore, areas that have expanded due to residual heat immediately after treatment are excluded from the new thermal denaturation areas generated after the output is turned off. According to this modification, thermal denaturation areas that the operator believes are known are excluded from the setting of the thermal denaturation areas after the output is turned off, while thermal denaturation areas that have passed the specified time period become the target thermal denaturation areas for notification. This allows the operator to identify only thermal denaturation areas that the operator is not aware of, enabling efficient treatment.

[0179] Furthermore, in the above-described modified example, a new fluorescent area that has elapsed a predetermined time after the treatment instrument 110 was turned off is used as a target for determination of the thermally denatured area generated when the output was turned off. However, the target for determination may be set based on the expansion rate of the fluorescent area rather than the elapsed time. In this case, a threshold for the area expansion rate and a threshold for the change (difference) in the distance from the center of gravity to the outer edge of the area are set based on the size (expansion pattern) of the fluorescent area expanded by residual heat, and determination of whether or not to select a target for determination is made based on these thresholds.

[0180] (Implementation Method 2)

[0181] Next, Embodiment 2 will be described. While Embodiment 1 described an endoscope system equipped with a rigid endoscope, Embodiment 2 will describe an endoscope system equipped with a flexible endoscope. The endoscope system according to Embodiment 2 will be described below. In Embodiment 2, components identical to those of the endoscope system 1 according to Embodiment 1 are denoted by the same reference numerals, and detailed descriptions thereof will be omitted.

[0182] [Structure of the endoscope system]

[0183] Figure 18 19 is a block diagram showing a functional configuration of a main portion of an endoscope system according to the second embodiment.

[0184] The endoscope system 101 is inserted into a patient or other subject to be examined and images the subject's interior. The display device 7 displays images based on the captured image data. A physician or other operator observes the images displayed on the display device 7 to inspect the presence and condition of abnormal areas such as bleeding sites, tumor sites, and abnormal sites captured as examination targets. Furthermore, the physician or other operator inserts treatment instruments, such as energy devices, into the subject's interior through the treatment instrument channel of the endoscope to perform treatment on the subject. The endoscope system 101 includes an endoscope 102 in addition to the aforementioned light source device 3, display device 7, and control device 9.

[0185] [Structure of an endoscope]

[0186] The configuration of the endoscope 102 will be described. The endoscope 102 generates image data by imaging the inside of a subject, and outputs the generated image data to the control device 9. The endoscope 102 includes an insertion portion 121, an operation portion 122, and a universal cable 123.

[0187] The insertion portion 121 is flexible and elongated. It comprises a distal end portion 124 housing a camera device (described later); a freely bendable bending portion 125 composed of a plurality of bendable pieces; and a long, flexible flexible tubular portion 126 connected to the proximal end of the bending portion 125.

[0188] The front end portion 124 is made of glass fiber, etc. The front end portion 124 includes a light guide 241 that forms a light guide path for light supplied from the light source device 3 , an illumination lens 242 provided at the front end of the light guide 241 , and an imaging device 243 .

[0189] The imaging device 243 includes a light-converging optical system 244 , the imaging element 53 of the first embodiment, a cut filter 54 , an A / D converter 55 , a P / S converter 56 , an imaging and recording unit 57 , and an imaging control unit 58 .

[0190] The universal cable 123 has at least a built-in optical fiber 241 and a bundled cable that brings together one or more cables. The bundled cable is a signal line that sends and receives signals between the endoscope 102 and the light source device 3 and the control device 9, including a signal line for sending and receiving setting data, a signal line for sending and receiving camera images (image data), a signal line for sending and receiving timing signals for driving the camera element 53, etc. The universal cable 123 has a connector portion 127 that can be freely attached and detached from the light source device 3. The coiled coil cable 127a of the connector portion 127 extends, and a connector portion 128 that can be freely attached and detached from the control device 9 is provided at the extended end of the coil cable 127a.

[0191] The endoscope system 101 configured in this manner performs the same processing as that of the endoscope system 1 according to the first embodiment described above.

[0192] In the second embodiment described above, similar to the first embodiment, when a fluorescent area in a fluorescent image changes, the output state of the treatment device at the time the changed fluorescent image was captured is determined. If the output is off, the newly generated fluorescent area is defined as the thermal denaturation area generated when the energy device's output is off, and this is notified to the operator. According to this second embodiment, the operator can understand the thermal denaturation that occurs when the treatment device's output is off.

[0193] (Implementation 3)

[0194] Next, Embodiment 3 will be described. While Embodiments 1 and 2 above describe endoscope systems, Embodiment 3 will describe a case where the system is applied to a surgical microscope system. In Embodiment 3, components identical to those of the endoscope system 1 according to Embodiment 1 are denoted by the same reference numerals, and detailed descriptions thereof will be omitted.

[0195] [Structure of surgical microscope system]

[0196] FIG20 is a diagram schematically illustrating the configuration of a surgical microscope system according to Embodiment 3. The surgical microscope system 300 includes a microscope device 310, which serves as a medical imaging device and captures images for observing a subject, and a display device 7. Alternatively, the display device 7 and the microscope device 310 may be integrated.

[0197] The microscope system 310 includes a microscope unit 312 for magnifying and photographing minute portions of a subject; a support unit 313 connected to the base of the microscope unit 312 and including an arm that rotatably supports the microscope unit 312; and a base unit 314 that rotatably holds the base of the support unit 313 and allows for movement on the floor. The base unit 314 includes a light source device 3 for generating white light, first narrowband light, and second narrowband light, etc., which are emitted from the microscope unit 310 toward the subject; and a control device 9 for controlling the operation of the surgical microscope system 300. The light source device 3 and the control device 9 each have at least the same configuration as that of the first embodiment described above. Specifically, the light source device 3 includes a condenser lens 30, a first light source unit 31, a second light source unit 32, and a light source control unit 33. Furthermore, the control device 9 includes an image / picture converter 91, an image processor 92, an input unit 93, a recording unit 94, and a control unit 95. The base portion 314 may be configured to be fixed to a ceiling, a wall, or the like to support the support portion 313 instead of being movably provided on the floor.

[0198] The microscope section 312 is, for example, cylindrical in shape and has the above-mentioned medical imaging device inside. Specifically, the medical imaging device has the same structure as the endoscopic camera head 5 involved in the above-mentioned embodiment 1. For example, the microscope section 312 includes an optical system 51, a drive section 52, an imaging element 53, a cutoff filter 54, an A / D conversion section 55, a P / S conversion section 56, an imaging recording section 57, and an imaging control section 58. In addition, a switch for receiving input of an operation instruction of the microscope device 310 is provided on the side surface of the microscope section 312. A cover glass (not shown) is provided on the opening surface of the lower end portion of the microscope section 312 to protect the interior.

[0199] In the surgical microscope system 300 thus configured, the operator or other user can operate various switches while holding the microscope unit 312 to move the microscope unit 312, perform zoom operations, or switch the illumination light. Furthermore, the microscope unit 312 is preferably shaped elongated in the observation direction to facilitate changing the field of view while holding the microscope unit. Therefore, the microscope unit 312 may also have a shape other than a cylinder, such as a polygonal prism.

[0200] In the third embodiment described above, in the surgical microscope system 300, similar to the first embodiment, when a fluorescent area in a fluorescent image changes, the output state of the treatment instrument at the time the changed fluorescent image was captured is determined. If the output is off, the newly generated fluorescent area is defined as the thermal denaturation area generated when the energy device's output is off, and this is notified to the operator. According to this third embodiment, the operator can understand the thermal denaturation that occurs when the treatment instrument's output is off.

[0201] (Other embodiments)

[0202] By appropriately combining the multiple components disclosed in the endoscope systems according to the above-mentioned embodiments 1 and 2 of the present disclosure or the surgical microscope system according to the above-mentioned embodiment 3, various inventions can be formed. For example, some components can be deleted from all the components described in the endoscope systems or surgical microscope systems according to the above-mentioned embodiments of the present disclosure. In addition, the components described in the endoscope systems or surgical microscope systems according to the above-mentioned embodiments of the present disclosure can also be appropriately combined. Furthermore, this embodiment can be applied as long as the processing is based on the fluorescence emitted by a substance generated by burning, etc.

[0203] In addition, in the embodiment and the modified example, the processing example is described on the premise that the first fluorescent image and the second fluorescent image are images of the same viewing angle. However, when using images of different viewing angles in which the same subject is captured in part, a known method such as pattern matching is used to match the fluorescent areas (thermal denaturation areas), detect changes in the fluorescent areas, and perform setting processing of the thermal denaturation areas generated after the output is turned off.

[0204] In the endoscope system or surgical microscope system according to the embodiment of the present disclosure, the above-mentioned "unit" can be replaced by "unit", "circuit", etc. For example, the control unit can be replaced by a control unit or a control circuit.

[0205] In the descriptions of the flowcharts in this specification, although expressions such as "first," "afterwards," and "next" are used to clarify the sequential relationship between the processes in the steps, the order of the processes required to implement the present invention is not solely defined by these expressions. In other words, the order of the processes in the flowcharts described in this specification can be changed within a range consistent with the order in which they are performed.

[0206] In addition, the programs executed by the devices involved in embodiments 1 to 3 are provided in the form of file data in an installable or executable form recorded on a computer-readable recording medium such as a CD-ROM, floppy disk (FD), CD-R, DVD (Digital Versatile Disk), USB medium, flash memory, etc.

[0207] Furthermore, the program for causing the apparatuses according to Embodiments 1 to 3 to execute may be stored on a computer connected to a network such as the Internet and provided by downloading via the network. Furthermore, the program for causing the information processing apparatuses according to Embodiments 1 to 3 to execute may be provided or distributed via a network such as the Internet.

[0208] In addition, in embodiments 1 and 2, an example in which the light source device 3 and the control device 9 are different entities is described, but a structure in which the light source device 3 and the control device 9 are integrated can also be provided. In addition, in embodiment 3, an example in which the light source device 3 and the control device 9 are integrated is described, but a structure in which the light source device 3 and the control device 9 are different entities can also be provided.

[0209] While several embodiments of the present application have been described in detail above based on the drawings, these are merely examples and the present invention can be implemented in other ways that are variously modified and improved based on the knowledge of those skilled in the art, represented by the methods described in the columns of this disclosure.

[0210] Industrial applicability

[0211] As described above, the medical device, medical system, medical device operating method, and medical device operating program according to the present invention are useful for enabling the operator to understand the thermal denaturation that occurs when the output of the treatment instrument is turned off.

[0212] Description of Reference Numerals

[0213] 1.101: Endoscope system; 2. Insertion unit; 3. Light source device; 4. Light guide; 5. Endoscope camera head; 6. First transmission cable; 7. Display device; 8. Second transmission cable; 9.9A: Control device; 10. Third transmission cable; 21. Eyepiece unit; 22. Optical system; 23. Illumination optical system; 30. Converging lens; 31. First light source unit; 32. Second light source unit; 33. Light source control unit; 51. Optical system; 52. Drive unit; 53. Image sensor; 54. Cutoff filter; 55. A / D converter; 56. P / S converter; 57. Video recording unit; 58. Video control unit; 61. Video connector; 62. Camera head connector; 91. S / P converter; 92. Image processing unit; 93. Input unit; 94. Recording unit; 95. Control unit; 100. Treatment system; 1 02: Endoscope; 110: Treatment instrument; 115: Switch; 120: Treatment instrument control device; 121: Insertion part; 122: Operation part; 123: Universal cable; 124: Front end part; 125: Bending part; 126: Flexible tube part; 127, 128: Connector part; 127a: Coil cable; 130: Electric cable; 241: Light guide; 242: Illumination lens; 243: Camera device; 244: Optical system; 300: Surgical microscope system; 310: Microscope device; 312: Microscope part; 313: Support part; 314: Base part; 511: Lens; 531: Pixel part; 532: Color filter; 921: Generation part; 922: Extraction part; 923: Fluorescence area determination part; 924: Output status determination part; 925: Output part; 941: Program recording part.

Claims

1. A medical device comprising: a generating unit that generates a fluorescent image based on fluorescence generated by excitation light that excites a substance generated by cauterization using an energy device; a fluorescence region determination unit that determines, based on the output information of the energy device and the fluorescence image, whether a fluorescence region is generated during an off-time period generated while the output of the energy device is in an off state; as well as The control unit, when the fluorescence area determination unit determines that the fluorescence area is generated during the off state, executes a notification process for notifying that the fluorescence area is generated while the output of the energy device is in the off state.

2. The medical device according to claim 1, wherein The generating unit generates a first fluorescent light image and a second fluorescent light image captured later than the first fluorescent light image. The fluorescence area determination unit determines the presence or absence of a new fluorescence area that exists only in the second fluorescence image based on the first fluorescence image and the second fluorescence image, and if the new fluorescence area exists, determines whether the new fluorescence area corresponds to the fluorescence area generated during shutdown based on the output information of the energy device.

3. The medical device according to claim 2, wherein: The first fluorescent image is an image captured when the output of the energy device is in an on state, The second fluorescent image is an image captured when the output of the energy device is in an off state.

4. The medical device according to claim 2, wherein: further comprising an extraction unit configured to extract fluorescent areas in the first fluorescent image and the second fluorescent image, The fluorescence region determination unit compares the fluorescence regions extracted by the extraction unit to determine the presence or absence of the new fluorescence region.

5. The medical device according to claim 1, wherein The fluorescence is light generated when the substance is excited.

6. The medical device according to claim 5, wherein The substances are advanced glycation end products produced by heat denaturation.

7. The medical device according to claim 1, wherein The fluorescence region determination unit determines the presence or absence of a fluorescence region generated during the shutdown period, with respect to a fluorescence region in a second fluorescence image captured after a predetermined time has elapsed from the time when the energy device was switched off.

8. The medical device according to claim 1, wherein The generating unit generates a display image in which the fluorescence generating region during the OFF period and the fluorescence regions other than the fluorescence generating region during the OFF period are displayed in different forms.

9. The medical device according to claim 8, wherein The generating unit generates the display image in which the fluorescence generating region during the OFF period and the fluorescence regions other than the fluorescence generating region during the OFF period are displayed in different patterns on the second fluorescence image.

10. The medical device according to claim 8, wherein The generating unit generates a white light image based on reflected light and return light from the living tissue when white light is irradiated on the living tissue. The generating unit generates the display image in which the fluorescence generating region during OFF and the fluorescence regions other than the fluorescence generating region during OFF are displayed in different patterns on the white-light image.

11. A medical device comprising: A fluorescence region determination unit determines a fluorescence region generated during an off-time period when the output of the energy device is in an off state based on output information of the energy device and a fluorescence image based on fluorescence generated by the excitation light, wherein: The excitation light is light that excites a substance produced by burning with the energy device; as well as The control unit, when the fluorescence area determination unit determines that the fluorescence area is generated during the off state, executes a notification process for notifying that the fluorescence area is generated while the output of the energy device is in the off state.

12. A medical system comprising: an imaging device for capturing an image of the subject; a light source device capable of irradiating excitation light for exciting a substance generated by thermally treating living tissue; as well as A control device, the imaging device being detachably mounted on the control device, and the control device being capable of communicating with a control device for controlling an energy device for a burn treatment target. Wherein, the control device has: a generating unit that generates a fluorescent image based on fluorescence generated by excitation light that excites a substance generated by cauterization using an energy device; a fluorescence region determination unit that determines, based on the output information of the energy device and the fluorescence image, whether a fluorescence region is generated at an off time generated while the output of the energy device is in an off state; and The control unit, when the fluorescence area determination unit determines that the fluorescence area is generated during the off state, executes a notification process for notifying that the fluorescence area is generated while the output of the energy device is in the off state.

13. A method for operating a medical device, performed by the medical device, the method comprising the following steps: a generating step in which a generating unit generates a fluorescent image based on fluorescence generated by excitation light, the excitation light being light that excites a substance generated by cauterization using an energy device; a fluorescence region determination step in which a fluorescence region determination unit determines, based on the output information of the energy device and the fluorescence image, that a fluorescence region is generated during an off-time period when the output of the energy device is in an off state; as well as In the notification step, when the fluorescent area determination unit determines that the fluorescent area is generated during the off state, the control unit notifies that the fluorescent area is generated during the period when the output of the energy device is in the off state.

14. A medical device operating program, executed by the medical device, wherein the operating program causes the following steps to be performed: a generating step of generating a fluorescent image based on fluorescence generated by excitation light, the excitation light being light that excites a substance generated by cauterization using an energy device; a fluorescence region determination step of determining, based on the output information of the energy device and the fluorescence image, a fluorescence region generated during a period when the output of the energy device is in an off state; as well as A notification step of notifying that the fluorescent area is generated while the output of the energy device is in the off state, when the fluorescent area determination step determines that the fluorescent area is generated during the off state.

Images