Medical device, medical system, method for operating medical device, and program for operating medical device

Abstract

A medical apparatus according to the present disclosure is provided with: an output pattern information imparting unit for imparting, on the basis of output information of an energy device, information on an output pattern of the energy device to a fluorescence region in a fluorescence image based on fluorescence generated by excitation light, the excitation light is light for exciting a substance generated by burning using an energy device; and an output unit that outputs information in which the output mode information is imparted to the fluorescent region by the output mode information imparting unit.

Description

Technical Field

[0001] The present disclosure relates to a medical device, a medical system, an operating method of the medical device, and an operating program of the medical device that performs image processing on an imaging signal of an image of a subject and outputs the resultant image. Background Art

[0002] Conventionally, there is known a technique in which a surgical endoscope is inserted into a subject, and the operator performs treatment by cauterizing the living tissue using a treatment instrument such as an energy device while observing the treatment portion (see, for example, Patent Document 1).

[0003] When biological tissue is burned, it undergoes thermal denaturation, producing advanced glycation end-products (AGEs), also known as "scorches." These AGEs fluoresce when exposed to light of a specific wavelength. By observing the fluorescence emitted by the AGEs, the surgeon can identify the thermally denatured area at the treatment site.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: International Publication No. 2020 / 174666 Summary of the Invention

[0007] Problems to be solved by the invention

[0008] Furthermore, AGEs may require different types of treatment. For example, there are AGEs that arise from incision and AGEs that arise from sealing the incision. However, when observed using fluorescence images, these AGEs appear to emit light in the same pattern, making it difficult to distinguish between fluorescent images resulting from incision and luminescent images resulting from sealing.

[0009] The present disclosure has been made in view of the above circumstances, and an object of the present disclosure is to provide a medical device, a medical system, an operating method of a medical device, and an operating procedure of a medical device that enable a surgeon to understand the treatment type of a fluorescent image in a fluorescent image.

[0010] Solutions for solving problems

[0011] To solve the above-mentioned problems and achieve the purpose, the medical device disclosed herein comprises: an output mode information imparting unit that imparts information on the output mode of an energy device to a fluorescent area in a fluorescent image based on fluorescence generated by excitation light, based on output information from the energy device, wherein the excitation light is light that excites a substance generated by ablation using the energy device; and an output unit that outputs information resulting from the imparting of the output mode information to the fluorescent area by the output mode information imparting unit.

[0012] Furthermore, the medical device according to the present disclosure further includes a generating unit that generates the fluorescence image in the above disclosure.

[0013] In the medical device according to the present disclosure, in the above disclosure, the output mode is either an incision mode for incising the treatment object or a sealing mode for coagulating and sealing the treatment object.

[0014] Furthermore, the medical device according to the present disclosure further comprises, in the aforementioned disclosure, an extraction unit for extracting fluorescent areas in a first fluorescent image and a second fluorescent image having mutually different output modes from the energy device; and a fluorescent area determination unit for determining, with respect to the fluorescent areas extracted by the extraction unit, the presence or absence of a new fluorescent area existing only in the second fluorescent image, wherein the output mode information imparting unit imparts information of different output modes to the new fluorescent area and fluorescent areas other than the new fluorescent area.

[0015] In the medical device according to the present disclosure described above, the second fluorescent light image is an image captured later in time than the first fluorescent light image.

[0016] In the medical device according to the present disclosure described above, the fluorescence is light generated by the excitation of the substance.

[0017] In the medical device according to the present disclosure, in the above disclosure, the substance is an advanced glycation end product produced by thermal denaturation.

[0018] In the medical apparatus according to the present disclosure, in the above disclosure, the generating unit generates a display image that is displayed in a different manner for each type of the output mode assigned by the output mode information assigning unit.

[0019] In the medical apparatus according to the present disclosure described above, the fluorescence indicates the trajectory of the distal end of the energy device.

[0020] In addition, in the above-mentioned disclosure of the medical device involved in the present disclosure, the output mode includes a first mode and a second mode different from the first mode, and the generating unit is capable of respectively generating a display image that emphasizes only the fluorescent area to which the information of the first mode is assigned as the output mode and a display image that emphasizes only the fluorescent area to which the information of the second mode is assigned as the output mode.

[0021] In the medical apparatus according to the present disclosure described above, the generating unit generates a display image that is displayed in a fluorescent image in a manner different for each type of the output mode assigned by the output mode information assigning unit.

[0022] In the medical apparatus according to the present disclosure, in the above disclosure, the generating unit generates a display image that is displayed in a white-light image in a manner different for each type of the output mode assigned by the output mode information assigning unit.

[0023] Furthermore, in the medical apparatus according to the present disclosure, in the above disclosure, the output mode information imparting unit imparts information on the output mode of the energy device to a region exceeding a specific fluorescence intensity among the fluorescence regions in the fluorescence image.

[0024] In addition, the medical system according to the present disclosure includes: an imaging device for imaging a subject; a light source device capable of irradiating excitation light that excites a substance generated by heat treatment of living tissue; and a control device electrically connected to the imaging device and capable of communicating with a device control device that controls an energy device that is to be subjected to cauterization treatment, wherein the control device includes: an output mode information imparting unit for imparting information on an output mode of the energy device to a fluorescent area in a fluorescent image based on fluorescence generated by the excitation light, based on output information from the energy device, wherein the excitation light is light that excites a substance generated by cauterization using the energy device; and an output unit for outputting information obtained by imparting the output mode information to the fluorescent area by the output mode information imparting unit.

[0025] In addition, the operating method of the medical device involved in the present disclosure is an operating method of the medical device performed by the medical device, and the operating method includes the following steps: an output mode information assigning step, in which an output mode information assigning unit assigns information on the output mode of the energy device to a fluorescent area in a fluorescent image based on fluorescence generated by excitation light, based on output information of the energy device, wherein the excitation light is light that excites a substance generated by ablation using the energy device; and an output step, in which the output unit outputs information after the output mode information is assigned to the fluorescent area by the output mode information assigning unit.

[0026] In addition, the operating program of the medical device involved in the present disclosure is an operating program of the medical device executed by the medical device, and the operating program causes the following steps to be executed: an output mode information assigning step of assigning information of the output mode of the energy device to a fluorescent area in a fluorescent image based on fluorescence generated by excitation light, based on output information of the energy device, wherein the fluorescence is light that excites a substance generated by ablation using the energy device; and an output step of outputting information after the output mode information is assigned to the fluorescent area in the output mode information assigning step.

[0027] Effects of the Invention

[0028] According to the present disclosure, there is an effect of enabling the operator to grasp the treatment type of the fluorescent image (thermal denaturation area) in the fluorescent image. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a diagram showing a schematic configuration of an endoscope system according to the first embodiment.

[0030] Figure 2 This is a diagram showing a schematic configuration of a treatment system to which the endoscope system according to the first embodiment is connected.

[0031] Figure 3 This is a block diagram showing the functional configuration of the main parts of the endoscope system according to the first embodiment.

[0032] Figure 4 This is a diagram schematically showing wavelength characteristics of light emitted by the first light source unit and the second light source unit according to the first embodiment.

[0033] Figure 5 This is a diagram schematically showing the structure of a pixel portion according to Embodiment 1.

[0034] Figure 6 This is a diagram schematically showing the structure of the color filter according to Embodiment 1.

[0035] Figure 7 It is a diagram schematically showing the sensitivity characteristics of each filter.

[0036] Figure 8A Schematically shows the signal value of the R pixel of the image sensor according to the first embodiment.

[0037] Figure 8B Schematically showing the signal values ​​of the G pixels of the image sensor according to the first embodiment.

[0038] Figure 8C This is a diagram schematically showing the signal value of the B pixel of the image sensor according to the first embodiment.

[0039] Figure 9 This is a diagram schematically showing the structure of the cut filter according to the first embodiment.

[0040] Figure 10 This is a diagram schematically showing the transmission characteristics of the cut filter according to the first embodiment.

[0041] Figure 11 This is a diagram schematically showing the observation principle in the normal light observation mode according to the first embodiment.

[0042] Figure 12 This is a diagram schematically showing the observation principle in the heat treatment observation mode according to the first embodiment.

[0043] Figure 13 This is a flowchart for explaining the thermal denaturation region determination process using the endoscope system according to the first embodiment.

[0044] Figure 14 It is a diagram for explaining a fluorescent image in the fluorescence observation mode.

[0045] Figure 15 This is a diagram showing a schematic configuration of an endoscope system according to the second embodiment.

[0046] Figure 16 This is a block diagram showing the functional configuration of the main parts of the endoscope system according to the second embodiment.

[0047] Figure 17 This is a diagram showing a schematic configuration of a surgical microscope system according to a third embodiment. DETAILED DESCRIPTION

[0048] The following is a method for implementing the present disclosure and the attached Figure 1 Detailed description will be given below. In addition, the present disclosure is not limited to the following embodiments. In addition, the figures referred to in the following description are merely schematic illustrations of shapes, sizes and positional relationships to the extent that the contents of the present disclosure can be understood. That is, the present disclosure is not limited to the shapes, sizes and positional relationships illustrated in the figures. Moreover, in the description of the drawings, the same parts are marked with the same figure numbers for description. In addition, as an example of the endoscope system involved in the present disclosure, an endoscope system including a rigid endoscope and a medical imaging device is described.

[0049] (Implementation 1)

[0050] [Structure of the endoscope system]

[0051] Figure 1 This is a diagram showing a schematic configuration of an endoscope system according to the first embodiment. Figure 1 The illustrated endoscope system 1 is a system used in the medical field for observing biological tissue within a subject, such as a living organism. The endoscope system 1 is used when performing surgery or treatment on the subject using a treatment instrument, such as an energy device capable of thermal treatment. The operator performs surgery or treatment while observing a display device displaying an observation image based on image data captured by a medical imaging device.

[0052] The endoscope system 1 includes an insertion portion 2 , a light source device 3 , a light guide 4 , an endoscopic camera head 5 (medical imaging device), a first transmission cable 6 , a display device 7 , a second transmission cable 8 , a control device 9 , and a third transmission cable 10 .

[0053] The insertion portion 2 is rigid or at least partially flexible and has an elongated shape. The insertion portion 2 is inserted into a subject such as a patient via a cannula. The insertion portion 2 is internally provided with an optical system such as a lens for forming an observation image.

[0054] One end of the light guide 4 is connected to the light source device 3. Under the control of the control device 9, the light source device 3 supplies illumination light to the end of the light guide 4 for irradiation into the subject. The light source device 3 is implemented using the following components: one or more light sources such as LEDs (light-emitting diodes), xenon lamps, and semiconductor laser elements such as LDs (laser diodes); a processor serving as a processing device comprising hardware such as an FPGA (field programmable gate array) and a CPU (central processing unit); and a memory serving as a temporary storage area used by the processor.

[0055] One end of the light guide 4 is detachably connected to the light source device 3 , and the other end is detachably connected to the insertion portion 2 . The light guide 4 guides illumination light supplied from the light source device 3 from one end to the other end, and supplies the illumination light to the insertion portion 2 .

[0056] The endoscopic camera head 5 is detachably connected to the eyepiece portion 21 of the insertion portion 2. Under the control of the control device 9, the endoscopic camera head 5 receives the observation image formed by the insertion portion 2 and performs photoelectric conversion to generate an imaging signal (RAW data). The imaging signal is then output to the control device 9 via the first transmission cable 6.

[0057] One end of the first transmission cable 6 is detachably connected to the control device 9 via a video connector 61, and the other end is detachably connected to the endoscopic camera 5 via a camera connector 62. The first transmission cable 6 transmits the imaging signal output from the endoscopic camera 5 to the control device 9, and transmits setting data and power output from the control device 9 to the endoscopic camera 5. Here, the setting data refers to control signals, synchronization signals, clock signals, and the like for controlling the endoscopic camera 5.

[0058] The display device 7 displays an observation image based on an imaging signal processed by the control device 9 and various information related to the endoscope system 1 under the control of the control device 9. The display device 7 is implemented using a display monitor such as liquid crystal or organic EL (Electro Luminescence).

[0059] One end of the second transmission cable 8 is detachably connected to the display device 7 , and the other end is detachably connected to the control device 9 . The second transmission cable 8 transmits the image signal processed by the control device 9 to the display device 7 .

[0060] The control device 9 is implemented using the following components: a processor (including hardware such as a GPU (Graphics Processing Unit), FPGA, or CPU) as a processing device; and memory as a temporary storage area used by the processor. The control device 9 comprehensively controls the operation of the light source device 3, the endoscopic camera head 5, and the display device 7 via the first transmission cable 6, the second transmission cable 8, and the third transmission cable 10, in accordance with a program stored in the memory. Furthermore, the control device 9 performs various image processing on the imaging signal input via the first transmission cable 6 and outputs it to the second transmission cable 8.

[0061] One end of the third transmission cable 10 is detachably connected to the light source device 3 , and the other end is detachably connected to the control device 9 . The third transmission cable 10 transmits control data from the control device 9 to the light source device 3 .

[0062] [Functional structure of disposal system]

[0063] Next, the configuration of a treatment system 100 connected to the above-described endoscope system 1 will be described. Figure 2 1 is a diagram showing a schematic configuration of a treatment system to which the endoscope system according to the first embodiment is connected. Figure 2 In FIG. 1 , one side along the central axis Ax of the treatment instrument is referred to as the distal end side Ar1 , and the other side is referred to as the proximal end side Ar2 .

[0064] The treatment system 100 treats a target area of ​​biological tissue (hereinafter referred to as the target area) by applying ultrasonic energy and high-frequency energy to the target area. Furthermore, the treatment system of this embodiment can perform treatments such as coagulation to seal the target area, incision, or simultaneous coagulation and incision. Furthermore, the treatment system 100 includes a treatment instrument 110 and a treatment instrument control device 120.

[0065] The treatment instrument 110 is an ultrasonic treatment instrument that treats a target area by applying ultrasonic energy and high-frequency energy to the target area, and corresponds to the surgical device of the present invention. The treatment instrument 110 includes a handpiece 111 and an ultrasonic transducer 112 .

[0066] The handpiece 111 includes a holding housing 113 , a movable handle 114 , a switch 115 , a rotation knob 116 , a tube 117 , a jaw 118 , and a vibration transmitting member 119 .

[0067] The ultrasonic transducer 112 includes a TD (transducer) case 112 a and an ultrasonic vibrator 112 b .

[0068] The TD housing 112 a supports the ultrasonic transducer 112 b and is detachably connected to the holding housing main body 113 a .

[0069] The ultrasonic vibrator 112b generates ultrasonic vibrations under the control of the treatment instrument control device 120. In the present embodiment, the ultrasonic vibrator 112b is composed of a BLT (Bolt-Tightened Langevin Type Vibrator).

[0070] The holding shell 113 constitutes the appearance of the treatment instrument 110 and supports the entire treatment instrument 110. The holding shell 113 includes: a substantially cylindrical holding shell body 113a coaxial with the central axis Ax; and a fixed handle 113b extending from the main body of the holding shell 113 to the Figure 2 It extends below the cervix and is held by the operator.

[0071] The movable handle 114 is opened and closed by an operator such as a surgeon. This opening and closing operation is an operation of opening and closing the jaw 11 relative to the end portion 119 a on the distal end side Ar1 of the vibration transmission member 119 .

[0072] The switch 115 is provided so as to be exposed to the outside from the side surface of the front end Ar1 of the fixed handle 113b. Furthermore, the switch 115 accepts treatment operations by an operator, such as a surgeon. This treatment operation involves applying ultrasonic energy or high-frequency energy to the target area. In this embodiment, the switch 115 is provided with multiple buttons, each of which is assigned an operation instruction. For example, there is a button for setting the device to the incision mode for treatment, and a button for setting the device to the closed mode for treatment.

[0073] The rotation knob 116 has a generally cylindrical shape coaxial with the central axis Ax and is located on the distal end side Ar1 of the holding housing body 113a. Furthermore, the rotation knob 116 is rotated by an operator, such as a surgeon. This rotational operation causes the rotation knob 116 to rotate relative to the holding housing body 113a about the central axis Ax. Furthermore, rotation of the rotation knob 116 causes the conduit 117, jaws 118, and vibration transmission member 119 to rotate about the central axis Ax.

[0074] The conduit 117 is a cylindrical conduit. A pin (not shown) that rotatably supports the jaw 118 is fixed to the end portion of the conduit 117 on the distal end side Ar1.

[0075] At least a portion of the jaws 118 is made of a conductive material. The jaws 118 open and close relative to the end 119a of the distal end Ar1 of the vibration transmitting member 119 as the operator grips the movable handle 114, thereby gripping the target area.

[0076] The vibration transmission member 119 is made of a conductive material and has a long strip shape extending linearly along the central axis Ax. In addition, the vibration transmission member 119 is inserted into the pipeline 117 in a state where the end 119a of the front end side Ar1 protrudes to the outside. At this time, although the end of the base end side Ar2 in the vibration transmission member 119 is omitted in the specific figure, it is mechanically connected to the ultrasonic transducer 112. That is, the vibration transmission member 12 transmits the ultrasonic vibration generated by the ultrasonic transducer 112 from the end of the base end side Ar2 to the end 119a of the front end side Ar1. In this embodiment, the ultrasonic vibration is a longitudinal vibration that vibrates in the direction along the central axis Ax.

[0077] The treatment instrument control device 120 comprehensively controls the operation of the treatment instrument 110 via the electric cable 130. The treatment instrument control device 120 corresponds to a device control device.

[0078] Specifically, the treatment instrument control device 120 detects a treatment operation of the switch 115 by an operator, such as a surgeon, via the electrical cable 130. Upon detecting this treatment operation, the treatment instrument control device 120 applies ultrasonic energy or high-frequency energy to the target area grasped between the jaw 11 and the end 119a of the distal end Ar1 of the vibration transmission member 119 via the electrical cable 130. In other words, the treatment instrument control device 120 performs treatment on the target area.

[0079] For example, when imparting ultrasonic energy to the target area, the treatment instrument control device 120 supplies driving power to the ultrasonic vibrator 112b via the electrical cable 130. As a result, the ultrasonic vibrator 112b generates longitudinal vibrations (ultrasonic vibrations) that vibrate in the direction along the central axis Ax. Furthermore, the end 119a of the front end side Ar1 of the vibration transmission member 119 vibrates at a desired amplitude due to this longitudinal vibration. Furthermore, ultrasonic vibrations are imparted from the end 119a to the target area held between the jaws 118 and the end 119a. In other words, ultrasonic energy is imparted from the end 119a to the target area.

[0080] Furthermore, for example, when applying high-frequency energy to a target site, the treatment instrument control device 120 supplies high-frequency power between the jaws 118 and the vibration transmission member 119 via the electrical cable 130. This causes a high-frequency current to flow through the target site, which is grasped between the jaws 118 and the end 119a of the distal end Ar1 of the vibration transmission member 119. In other words, high-frequency energy is applied to the target site.

[0081] The treatment instrument control device 120 is communicably connected to the control device 9 and, when the switch 115 is pressed, outputs a signal indicating the switch is pressed to the control device 9. The treatment instrument control device 120 outputs information on the output mode of the treatment instrument 110 to the control device 9.

[0082] [Functional structure of the main parts of the endoscope system]

[0083] Next, the functional configuration of the main parts of the endoscope system 1 will be described. Figure 3 1 is a block diagram showing the functional configuration of the main parts of the endoscope system 1 .

[0084] [Structure of the insertion part]

[0085] First, the structure of the insertion portion 2 will be described. The insertion portion 2 includes an optical system 22 and an illumination optical system 23 .

[0086] The optical system 22 forms an image of the subject by converging reflected light from the subject, return light from the subject, excitation light from the subject, and luminescence emitted by the subject. The optical system 22 is implemented using one or more lenses.

[0087] The illumination optical system 23 irradiates the subject with illumination light supplied from the light guide 4. The illumination optical system 23 is implemented using one or more lenses and the like.

[0088] [Structure of light source device]

[0089] Next, a description will be given of the configuration of the light source device 3 . The light source device 3 includes a condenser lens 30 , a first light source unit 31 , a second light source unit 32 , and a light source control unit 33 .

[0090] The condenser lens 30 condenses the light emitted from the first light source unit 31 and the second light source unit 32 and emits the light toward the light guide 4 .

[0091] Under the control of the light source control unit 33, the first light source unit 31 emits white light (normal light) as visible light, thereby supplying white light to the light guide 4 as illumination light. The first light source unit 31 is constructed using a collimating lens, a white LED lamp, and a driver. Alternatively, the first light source unit 31 can simultaneously emit red, green, and blue LED lamps to supply visible white light. Of course, the first light source unit 31 can also be constructed using a halogen lamp, a xenon lamp, or the like.

[0092] Under the control of the light source control unit 33, the second light source unit 32 emits narrowband light in a wavelength range different from and narrower than that of white light, thereby supplying narrowband light as illumination light to the light guide 4. Here, narrowband light refers to, for example, light in the wavelength range of 400 nm to 430 nm with a center wavelength of 415 nm. The second light source unit 32 is implemented using a collimating lens, a semiconductor laser such as a violet laser diode (LD), and a driver. Furthermore, in the present embodiment, the narrowband light functions as excitation light for exciting advanced glycation end products produced by heat treatment of biological tissue.

[0093] The light source control unit 33 is implemented using a processor (e.g., an FPGA or CPU) as a processing device and a memory (e.g., a temporary storage area) used by the processor. Based on control data input from the control device 9, the light source control unit 33 controls the light emission timing and duration of each of the first light source unit 31 and the second light source unit 32.

[0094] Here, the wavelength characteristics of the light emitted by the first light source unit 31 and the second light source unit 32 will be described. Figure 4 Schematically shows the wavelength characteristics of the light emitted by the first light source unit 31 and the second light source unit 32. Figure 4 In the figure, the horizontal axis represents wavelength (nm) and the vertical axis represents relative intensity. Figure 4 In the middle, curve L WL The curve L represents the wavelength characteristics of the white light emitted by the first light source unit 31. V The wavelength characteristics of the narrowband light (excitation light) emitted by the second light source unit 32 are shown. The second light source unit 32 emits light with a central wavelength (peak wavelength) of 415 nm and a wavelength range of 400 nm to 430 nm. Figure 4 The curve L in WL The wavelength characteristics shown represent the characteristics when a white LED is used as the first light source unit 31 .

[0095] [Structure of endoscope camera]

[0096] return Figure 3 , the structure of the endoscope system 1 is further described.

[0097] Next, the configuration of the endoscopic camera head 5 will be described. The endoscopic camera head 5 includes an optical system 51, a drive unit 52, an imaging element 53, a cut filter 54, an A / D converter 55, a P / S converter 56, an imaging and recording unit 57, and an imaging control unit 58.

[0098] The optical system 51 forms an image of the subject, focused by the optical system 22 of the insertion portion 2, onto the light-receiving surface of the imaging element 53. The optical system 51 is capable of changing the focal length and focus position. The optical system 51 is constructed using a plurality of lenses 511. The optical system 51 changes the focal length and focus position by moving each of the plurality of lenses 511 along the optical axis L1 via the drive unit 52.

[0099] The drive unit 52 moves the multiple lenses 511 of the optical system 51 along the optical axis L1 under the control of the imaging control unit 58. The drive unit 52 is configured using a motor such as a stepping motor, a DC motor, or a voice coil motor, and a transmission mechanism such as gears that transmits the motor's rotation to the optical system 51.

[0100] The imaging element 53 is implemented using a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) image sensor having a plurality of pixels arranged in a two-dimensional matrix. Under the control of the imaging control unit 58, the imaging element 53 receives the subject image (light) formed by the optical system 51 and passed through the cut filter 54, performs photoelectric conversion on the subject image, generates an imaging signal (RAW data), and outputs it to the A / D converter 55. The imaging element 53 includes a pixel unit 531 and a color filter 532.

[0101] Figure 5 1 is a diagram schematically showing the structure of the pixel portion 531. The pixel portion 531 is composed of a plurality of pixels P such as photodiodes that store charges according to the amount of light. nm (n, m are integers greater than 1) are arranged in a two-dimensional matrix. The pixel unit 531 is controlled by the imaging control unit 58 to obtain a plurality of pixels P nm The pixel P in the reading area arbitrarily set as the reading object nm The image signal is read as image data and output to the A / D conversion unit 55 .

[0102] Figure 6 : This is a diagram schematically showing the structure of the color filter 532. The color filter 532 is composed of a Bayer array with 2×2 as one unit. The color filter 532 is composed of a filter R that transmits light in the red wavelength range, two filters G that transmit light in the green wavelength range, and a filter B that transmits light in the blue wavelength range. Figure 5 In the figure, the reference numerals for each filter (e.g. G 11 ) and pixel P nm Corresponding means configured at corresponding pixel positions.

[0103] Figure 7 is a diagram schematically showing the sensitivity characteristics of each filter. Figure 7 In the figure, the horizontal axis represents wavelength (nm) and the vertical axis represents transmission characteristics (sensitivity characteristics). Figure 7 In the middle, curve L B Indicates the transmission characteristics of filter B, curve L G Indicates the transmission characteristics of filter G, curve L R Indicates the transmission characteristics of filter R.

[0104] Filter B transmits light in the blue wavelength range (see Figure 7 The curve L B). In addition, the filter G transmits light in the green wavelength range (see Figure 7 The curve L G ). In addition, the filter R transmits light in the red wavelength range (see Figure 7 The curve L R ). In addition, in the following, the filter R is arranged on the light receiving surface to form a pixel P nm The pixel P is formed by placing the filter G on the light receiving surface. nm The pixel P is formed by placing the filter B on the light receiving surface. nm This will be described as a B pixel.

[0105] According to the imaging element 53 configured in this manner, when receiving the subject image formed by the optical system 51, the imaging element 53 generates color signals (R signal, G signal, and B signal) for each of the R pixel, the G pixel, and the B pixel (see Figures 8A to 8C ).

[0106] return Figure 3 , the structure of the endoscope system 1 is further described.

[0107] The cutoff filter 54 is disposed on the optical axis L1 between the optical system 51 and the imaging element 53. The cutoff filter 54 is provided on the light-receiving surface side (incident surface side) of at least the G pixel of the color filter 532, which is provided with the filter G that transmits the green wavelength range. The cutoff filter 54 blocks light in the wavelength range of the excitation light and transmits light in the wavelength range on the long-wavelength side that is longer than the wavelength range of the excitation light.

[0108] Figure 9 Schematically shows the structure of the cut filter 54. Figure 9 As shown, the filter F constituting the cutoff filter 54 11 Configured on filter G 11 (Refer to Figure 6 ) is configured at a position where the filter G 11 The light-receiving side directly above the .

[0109] Figure 10 is a diagram schematically showing the transmission characteristics of the cut filter 54. Figure 10 In the figure, the horizontal axis represents wavelength (nm) and the vertical axis represents transmittance. Figure 10 In the middle, curve L F The curve L represents the transmission characteristics of the cutoff filter 54. V Indicates the wavelength characteristics of the excitation light.

[0110] The cutoff filter 54 blocks light in the wavelength range of the excitation light and transmits light in a wavelength range longer than the wavelength range of the excitation light. Specifically, the cutoff filter 54 blocks light in a wavelength range shorter than the wavelength range of the excitation light and transmits light in a wavelength range longer than the wavelength range of the excitation light.

[0111] return Figure 3 , continue to explain the structure of the endoscope camera 5.

[0112] Under the control of the imaging control unit 58, the A / D converter 55 performs A / D conversion processing on the analog imaging signal input from the imaging element 53 and outputs the resulting signal to the P / S converter 56. The A / D converter 55 is implemented using an A / D conversion circuit or the like.

[0113] Under the control of the camera control unit 58, the P / S converter 56 performs parallel / serial conversion on the digital camera signal input from the A / D converter 55, and outputs the parallel / serial converted camera signal to the control device 9 via the first transmission cable 6. The P / S converter 56 is implemented using a P / S conversion circuit, etc. In the first embodiment, an E / O converter that converts the camera signal into an optical signal may be provided in place of the P / S converter 56, and the camera signal may be output to the control device 9 via an optical signal. Alternatively, the camera signal may be transmitted to the control device 9 via wireless communication such as Wi-Fi (Wireless Fidelity) (registered trademark).

[0114] The image recording unit 57 records various information related to the endoscopic camera head 5 (e.g., pixel information of the image sensor 53 and characteristics of the cutoff filter 54). Furthermore, the image recording unit 57 records various setting data and control parameters transmitted from the control device 9 via the first transmission cable 6. The image recording unit 57 is configured using a nonvolatile memory or a volatile memory.

[0115] The imaging control unit 58 controls the operation of the drive unit 52, the imaging element 53, the A / D converter 55, and the P / S converter 56 based on the setting data received from the control device 9 via the first transmission cable 6. The imaging control unit 58 is implemented using the following components: a TG (Timing Generator); a processor as a processing device including hardware such as a CPU; and a memory as a temporary storage area used by the processor.

[0116] [Structure of control device]

[0117] Next, the configuration of the control device 9 will be described.

[0118] The control device 9 includes an S / P conversion unit 91 , an image processing unit 92 , an input unit 93 , a recording unit 94 , and a control unit 95 .

[0119] Under the control of the control unit 95, the S / P converter 91 performs serial / parallel conversion on the image data received from the endoscopic camera 5 via the first transmission cable 6 and outputs the data to the image processing unit 92. Furthermore, if the endoscopic camera 5 outputs the imaging signal via an optical signal, an O / E converter that converts the optical signal into an electrical signal may be provided in place of the S / P converter 91. Furthermore, if the endoscopic camera 5 transmits the imaging signal via wireless communication, a communication module capable of receiving wireless signals may be provided in place of the S / P converter 91.

[0120] Under the control of the control unit 95, the image processing unit 92 performs predetermined image processing on the parallel data imaging signal input from the S / P conversion unit 91 and outputs the result to the display device 7. The predetermined image processing herein includes, for example, demosaicing, white balance, gain adjustment, gamma correction, and format conversion. The image processing unit 92 is implemented using a processor comprising hardware such as a GPU or FPGA as a processing device, and a memory serving as a temporary storage area for the processor. The image processing unit 92 includes a generation unit 921, an extraction unit 922, a fluorescence area determination unit 923, an output mode information assignment unit 924, and an output unit 925.

[0121] The generator 921 generates a first image and a second image, the first image including one or more characteristic regions to be removed by the operator, and the second image including one or more cauterized regions after being cauterized by the energy device (treatment instrument 110). Specifically, the generator 921 generates a white light image as the first image based on an imaging signal generated by capturing reflected light and return light from the living tissue when white light is irradiated. Furthermore, the generator 921 generates a fluorescence image as the second image based on an imaging signal generated by capturing fluorescence generated by excitation light, which is irradiated to excite advanced glycation end products (AGEs) produced by thermal treatment of the living tissue, in a fluorescence observation mode described later. Alternatively, the generator 921 can generate a pseudo-color image including one or more characteristic regions (lesion regions) to be removed by the operator based on an imaging signal generated by capturing reflected light and return light from the living tissue when excitation light is irradiated in the fluorescence observation mode of the endoscope system 1 described later.

[0122] The extraction unit 922 extracts a fluorescence area, which is an area of ​​the fluorescence image, from the fluorescence image generated by the generation unit 921 .

[0123] The fluorescence area determination unit 923 determines whether there is a change in the fluorescence area between fluorescence images captured at different times.

[0124] Based on the output mode determination result, the output mode information assigning unit 924 assigns output mode information to the fluorescent area (fluorescent image). Specifically, information about the output mode used when the fluorescent area was generated is assigned to the fluorescent area. Examples of output modes include an incision mode for incising biological tissue and a sealing (coagulation) mode for sealing the incised area.

[0125] The output unit 925 outputs a white light image, a fluorescence image, and a fluorescence image to which output mode information has been assigned by the output mode information assigning unit 924. The output unit 925 also outputs a white light image to which output mode information has been assigned by the output mode information assigning unit 924.

[0126] The input unit 93 receives input of various operations related to the endoscope system 1 and outputs the received operations to the control unit 95. The input unit 93 is configured using a mouse, a foot switch, a keyboard, buttons, switches, a touch panel, and the like.

[0127] The recording unit 94 is implemented using a recording medium such as a volatile memory, a nonvolatile memory, an SSD (Solid State Drive), an HDD (Hard Disk Drive), or a memory card. The recording unit 94 records data including various parameters required for the operation of the endoscope system 1. The recording unit 94 also includes a program recording unit 941 for recording various programs used to operate the endoscope system 1.

[0128] The control unit 95 is implemented using a processor (e.g., a hardware processor such as an FPGA or CPU) as a processing device and a memory (e.g., a temporary storage area) used by the processor. The control unit 95 comprehensively controls the various components that comprise the endoscope system 1. Furthermore, the control unit 95 receives signals from the treatment instrument control device 120 regarding the pressing of the switch 115 (output from the treatment instrument 110).

[0129] [Overview of each observation mode]

[0130] Next, an overview of each observation mode that can be executed by the endoscope system 1 will be described. The normal light observation mode and the fluorescence observation mode will be described in order.

[0131] [Overview of Normal Light Observation Mode]

[0132] First, the normal light observation mode will be described. Figure 11 A diagram schematically illustrating the observation principle in the normal light observation mode.

[0133] Under the control of the control device 9, the light source device 3 emits white light W1 having an intensity distribution shown in the graph G11 toward the biological tissue T1 of the subject by causing the first light source unit 31 to emit light. In this case, a portion of the reflected light and return light (hereinafter referred to as "reflected light WR10, reflected light WG10, and reflected light WB10") reflected by the biological tissue is shielded by the cutoff filter 54, and the remaining portion is incident on the imaging element 53. For example, specifically, the cutoff filter 54 shields the reflected light (reflected light WG10) incident on the G pixel and the reflected light in the wavelength range of the excitation light (excitation light W2 described later). That is, the reflected light and return light based on the irradiation of white light are incident on the filter R and the filter B, and the light in the wavelength range longer than the wavelength range of the excitation light is incident on the filter G. Therefore, the component of the light in the blue wavelength range incident on the pixel becomes smaller than when the cutoff filter 54 is not configured. The light incident on each filter is selectively transmitted according to the filter characteristics shown in graph G12.

[0134] Next, the image processing unit 92 acquires image data (RAW data) from the imaging element 53 of the endoscopic camera head 5 and performs image processing on the signal values ​​of the R pixels, G pixels, and B pixels contained in the acquired image data to generate a white light image. In this case, since the blue component contained in the image data is smaller than in conventional white light observation, the image processing unit 92 performs white balance adjustment processing to adjust the white balance so that the ratio of the red component, green component, and blue component is constant.

[0135] In the normal light observation mode, even when the cut filter 54 is arranged on the light receiving surface side of the G pixel, a natural white light image (observation image) can be observed.

[0136] [Overview of Fluorescence Observation Mode]

[0137] Next, the fluorescence observation mode will be described. Figure 12 This is a diagram schematically showing the observation principle in the fluorescence observation mode.

[0138] In recent years, minimally invasive treatments using endoscopes and laparoscopy have become increasingly common in the medical field. For example, widely performed minimally invasive treatments using endoscopes and laparoscopy include endoscopic submucosal dissection (ESD), laparoscopic endoscopic co-operative gastrectomy (LECS), non-exposed endoscopic wall-inversion surgery (NEWS), and transurethral resection of the bladder tumor (TUR-bt).

[0139] In these minimally invasive treatments, when performing treatment, for example, doctors or other operators use treatment instruments that emit energy such as high-frequency waves, ultrasound, microwaves, etc. to perform heat treatment as a pre-treatment to mark the surgical area, or as a treatment to remove the diseased area, close the incision, or coagulate the incision.

[0140] Furthermore, when amino compounds and reducing sugars are heated, a glycation reaction (Maillard reaction) occurs, in which the amino acids react with the reducing sugars. The end products produced by this Maillard reaction are collectively referred to as advanced glycation end products (AGEs). AGEs are known to contain substances with fluorescent properties. AGEs are known to emit fluorescence with a higher intensity than the autofluorescent substances naturally present in biological tissues. Therefore, the formation of AGEs significantly increases their fluorescence intensity compared to before the formation of AGEs.

[0141] AGEs generated by cauterization during treatment can be visualized by fluorescence observation, and the fluorescence intensity serves as an indicator of the state of thermal treatment. The fluorescence at this time corresponds to the trajectory of the tip (jaws 118 and vibration transmission member 119) of the energy device (treatment instrument 110).

[0142] Specifically, the fluorescence observation mode utilizes the fluorescence properties of AGEs generated in living tissues by thermal treatment with an energy device, etc., to visualize the thermally treated area. Therefore, in fluorescence observation mode, excitation light for exciting AGEs, for example, blue narrowband light with a center wavelength of 415 nm, is irradiated from the light source device 3 onto the living tissue. Consequently, in fluorescence observation mode, a thermally treated image (fluorescence image) that captures fluorescence (for example, green light with a wavelength of 490 to 625 nm) generated by AGEs can be observed.

[0143] Specifically, first, the light source device 3, under the control of the control device 9, emits the excitation light W2 (center wavelength 415nm: refer to Figure G13) by making the second light source unit 32 emit light to irradiate the biological tissue T2 (heat treatment area) to which the subject has been subjected to heat treatment by an energy device or the like. In this case, the reflected light (hereinafter referred to as "reflected light WR20, reflected light WG20, reflected light WB20") including at least the components of the excitation light W2 reflected at the biological tissue T2 (heat treatment area) and the return light is shielded by the cutoff filter 54, and a portion of the components on the long wavelength side is incident on the imaging element 53 (refer to Figure G14). In addition, Figure 11 In FIG, the thickness of the arrow represents the intensity of the component (light amount or signal value) of each line.

[0144] More specifically, if Figure 11 As shown in graph G2, cutoff filter 54 blocks reflected light WG20 incident on G pixels within a wavelength range that includes the wavelength range of excitation light W2. Furthermore, cutoff filter 54 transmits fluorescence WF1 emitted by AGEs in tissue T2 (heat-treated area) (see graph G14). Consequently, reflected light WG20 is prevented from entering G pixels, while fluorescence WF1 does enter. Since cutoff filter 54 is positioned on the light-receiving side (incident side) of the G pixels, it prevents the reflected light WG20 from excitation light W2 from mixing with fluorescence WF1 and burying the fluorescence component.

[0145] Furthermore, reflected light (reflected light WR20 , WB20 ) and fluorescent light WF1 are incident on the R pixel and the B pixel, respectively.

[0146] The image processing unit 92 then acquires image data (RAW data) from the imaging element 53 of the endoscopic camera head 5 and performs image processing on the signal values ​​of the G and B pixels contained in the acquired image data to generate a fluorescence image. In this case, the signal values ​​of the G pixels include fluorescence information representing the fluorescence image emitted from the heat-treated area. Furthermore, the B pixels include background information representing the surrounding biological tissues of the heat-treated area, which serve as the background of the heat-treated area. The image processing unit 92 performs image processing such as gain control, pixel interpolation, and mucosal enhancement on the signal values ​​of the G and B pixels contained in the image data to generate a fluorescence image. In this gain control process, the image processing unit 92 performs processing to make the gain of the signal value for the G pixels greater than the gain of the signal value for the G pixels during normal light observation, while making the gain of the signal value for the B pixels less than the gain of the signal value for the B pixels during normal light observation. Furthermore, the image processing unit 92 performs processing to make the signal values ​​of the G pixels and the signal values ​​of the B pixels equal (1:1). Alternatively, the image processing unit 92 may generate a pseudo-color image by superimposing color information whose hue is changed according to the fluorescence intensity on the fluorescent image.

[0147] [Treatment using an endoscopic system]

[0148] Next, a treatment using the endoscope system 1 of the present disclosure will be described. In this case, the operator inserts the insertion portion 2 into the subject and causes the light source device 3 to illuminate the area containing the treatment target with white light. The operator confirms the treatment target while observing the observation image displayed on the display device 7.

[0149] The operator then performs treatment on the treatment target of the subject while checking the white light image displayed on the display device 7. For example, the operator inserts an energy device (treatment instrument 110) into the subject via the insertion portion 2 to cauterize and remove the treatment target.

[0150] Afterwards, the operator irradiates the treatment object with excitation light and observes the fluorescent image displayed on the display device 7. The operator determines whether the treatment (e.g., resection) at the treatment position has been completed by observing the fluorescent image displayed on the display device 7. When the operator determines that the treatment is completed, the operator ends the operation. Specifically, the operator determines whether the resection of the treatment object has been completed by observing the fluorescent image displayed on the display device 7 and the burnt area that has been burnt and removed by the treatment instrument 110. At this time, when the operator determines that the resection of the treatment object is not completed, the operator switches the observation mode of the endoscope system 1 while repeating the observation of the white light image based on the irradiation of white light and the observation of the fluorescent image based on the irradiation of excitation light to continue the treatment.

[0151] [Handling of endoscope system]

[0152] Next, the processing executed by the endoscope system 1 will be described. Figure 13 This is a flowchart for explaining a thermally denatured region determination process using an endoscope system according to one embodiment. The thermally denatured region determination process is a process executed in the fluorescence observation mode.

[0153] The control unit 95 generates a first fluorescent image (step S101). At this point, the control unit 95 controls the light source control unit 33 to cause the second light source unit 32 to emit light, irradiating the subject with excitation light. The generator 921 generates the first fluorescent image by acquiring an imaging signal from the imaging element 53 of the endoscopic camera head 5. Thus, the first fluorescent image is acquired. In this case, the output unit 925 may also cause the display device 7 to display the first fluorescent image generated by the generator 921.

[0154] Next, the control unit 95 generates a second fluorescent image (step S102). At this point, the control unit 95 controls the light source control unit 33 to cause the second light source unit 32 to emit light, irradiating the subject with excitation light. The generator 921 generates the second fluorescent image by acquiring an imaging signal from the imaging element 53 of the endoscopic camera head 5. Thus, the second fluorescent image is acquired. In this case, the output unit 925 may also cause the display device 7 to display the second fluorescent image generated by the generator 921.

[0155] The second fluorescent image is a fluorescent image based on image data acquired at a later time than the first fluorescent image. The image data acquisition time (imaging timing) is performed, for example, after a predetermined time has passed since the first fluorescent image was acquired.

[0156] Next, the control unit 95 determines whether there has been a change in the fluorescence area between the first and second fluorescence images (step S103). In this step, the extraction unit 922 extracts the area depicting the fluorescence image (fluorescent area) from each fluorescence image. The extraction unit 922 performs contour extraction based on, for example, brightness values ​​to extract one or more fluorescence areas within the image. The fluorescence area determination unit 923 then determines whether there has been a change in the fluorescence area of ​​the second fluorescence image relative to the extracted fluorescence area of ​​the first fluorescence image. In this process, the fluorescence area determination unit 923 detects a change in the fluorescence area by determining the presence of new fluorescence areas in the second fluorescence image that were not present in the first fluorescence image. If the fluorescence area determination unit 923 determines that there has been no change in the fluorescence area (step S103: No), the control unit 95 terminates the process. On the other hand, if the fluorescence area determination unit 923 determines that there has been a change in the fluorescence area (step S103: Yes), the control unit 95 proceeds to step S104.

[0157] In step S104 , the control unit 95 acquires the output mode of the treatment instrument 110 when each fluorescent image is captured. Specifically, the control unit 95 acquires information on the output mode of the treatment instrument 110 from the treatment instrument control device 120 .

[0158] The output mode information assigning unit 924 then assigns information related to the output mode of the treatment instrument 110 to the fluorescence region of each fluorescence image based on the time at which the first and second fluorescence images were captured. In this case, the output mode information assigning unit 924 may also assign information related to the output mode to regions exceeding a specific fluorescence intensity.

[0159] Here, refer to Figure 14 To illustrate the temporal changes of fluorescence images. Figure 14 This is a diagram for explaining the fluorescence image in the fluorescence observation mode. Figure 14 , an example of a fluorescence image having a plurality of fluorescence images is shown, some of which are generated by different output modes. Specifically, the following fluorescence images are shown: Fluorescence image F 11 、F 12 The fluorescence image F is generated by the same output mode. 21 By different from the fluorescence image F 11 、F 12 The output pattern of F is generated. 11 、F 12 The same hue is superimposed on the 21 A different fluorescence image F is superimposed on 11 、F 12 At this time, the fluorescent image F 21 The change in the fluorescence area detected by the fluorescence area determination unit 923 is given a different image from the fluorescence image F. 11 、F 12 The generating unit 921 selects a preset hue according to the output mode and superimposes it on each fluorescent image, thereby generating a fluorescent image for display.

[0160] In addition to the superimposed hue, text information indicating the output mode may also be superimposed. In addition, a fluorescent image of only the fluorescent image of the specified output mode may be displayed. Furthermore, the fluorescent image F in the white light image and the fluorescent image F in the fluorescent image may also be displayed. 11 、F 12 The corresponding areas are displayed by superimposing preset hues.

[0161] For example, the thermally denatured region determination process may be executed at pre-set time intervals or when a surgeon inputs an instruction to execute the detection process. In this case, the second fluorescent image acquired during the previous process may be used as the first fluorescent image. In this case, the process may be restarted from step S102.

[0162] In the first embodiment described above, when the fluorescent area in a fluorescence image changes, output mode information of the treatment instrument is acquired, and information corresponding to the output mode is assigned to the fluorescent image in the fluorescence image and displayed, thereby allowing the operator to understand the output mode of each fluorescent image. According to this first embodiment, the operator can understand the treatment type of the fluorescent image in the fluorescence image.

[0163] (Implementation Method 2)

[0164] Next, Embodiment 2 will be described. While Embodiment 1 described an endoscope system equipped with a rigid endoscope, Embodiment 2 will describe an endoscope system equipped with a flexible endoscope. The endoscope system according to Embodiment 2 will be described below. In Embodiment 2, components identical to those of the endoscope system 1 according to Embodiment 1 are denoted by the same reference numerals, and detailed descriptions thereof will be omitted.

[0165] [Structure of the endoscope system]

[0166] Fig. 18 is a diagram showing a schematic configuration of an endoscope system according to Embodiment 2. Fig. 19 is a block diagram showing a functional configuration of a main portion of an endoscope system according to Embodiment 2.

[0167] The endoscope system 101 is inserted into a patient or other subject to be examined and images the subject's interior. The display device 7 displays images based on the captured image data. A physician or other operator observes the images displayed on the display device 7 to inspect the presence and condition of abnormal areas such as bleeding sites, tumor sites, and abnormal sites captured as examination targets. Furthermore, the physician or other operator inserts treatment instruments, such as energy devices, into the subject's interior through the treatment instrument channel of the endoscope to perform treatment on the subject. The endoscope system 101 includes an endoscope 102 in addition to the aforementioned light source device 3, display device 7, and control device 9.

[0168] [Structure of an endoscope]

[0169] The configuration of the endoscope 102 will be described. The endoscope 102 generates image data by imaging the inside of a subject, and outputs the generated image data to the control device 9. The endoscope 102 includes an insertion portion 121, an operation portion 122, and a universal cable 123.

[0170] The insertion portion 121 is flexible and elongated. It comprises a distal end portion 124 housing a camera device (described later); a freely bendable bending portion 125 composed of a plurality of bendable pieces; and a long, flexible flexible tubular portion 126 connected to the proximal end of the bending portion 125.

[0171] The front end portion 124 is made of glass fiber, etc. The front end portion 124 includes a light guide 241 that forms a light guide path for light supplied from the light source device 3 , an illumination lens 242 provided at the front end of the light guide 241 , and an imaging device 243 .

[0172] The imaging device 243 includes a light-converging optical system 244 , the imaging element 53 of the first embodiment, a cut filter 54 , an A / D converter 55 , a P / S converter 56 , an imaging and recording unit 57 , and an imaging control unit 58 .

[0173] The universal cable 123 has at least a built-in optical fiber 241 and a bundled cable that brings together one or more cables. The bundled cable is a signal line that sends and receives signals between the endoscope 102 and the light source device 3 and the control device 9, including a signal line for sending and receiving setting data, a signal line for sending and receiving camera images (image data), a signal line for sending and receiving timing signals for driving the camera element 53, etc. The universal cable 123 has a connector portion 127 that can be freely attached and detached from the light source device 3. The coiled coil cable 127a of the connector portion 127 extends, and a connector portion 128 that can be freely attached and detached from the control device 9 is provided at the extended end of the coil cable 127a.

[0174] The endoscope system 101 configured in this manner performs the same processing as that of the endoscope system 1 according to the first embodiment described above.

[0175] In the second embodiment described above, similar to the first embodiment, when the fluorescent area in the fluorescence image changes, output mode information of the treatment instrument is acquired, and information corresponding to the output mode is assigned to the fluorescent image in the fluorescence image and displayed, thereby allowing the operator to understand the output mode of each fluorescent image. According to this second embodiment, the operator can understand the treatment type of the fluorescent image in the fluorescence image.

[0176] (Implementation 3)

[0177] Next, Embodiment 3 will be described. While Embodiments 1 and 2 above describe endoscope systems, Embodiment 3 will describe a case where the system is applied to a surgical microscope system. In Embodiment 3, components identical to those of the endoscope system 1 according to Embodiment 1 are denoted by the same reference numerals, and detailed descriptions thereof will be omitted.

[0178] [Structure of surgical microscope system]

[0179] FIG20 is a diagram schematically illustrating the configuration of a surgical microscope system according to Embodiment 3. The surgical microscope system 300 includes a microscope device 310, which serves as a medical imaging device and captures images for observing a subject, and a display device 7. Alternatively, the display device 7 and the microscope device 310 may be integrated.

[0180] The microscope system 310 includes a microscope unit 312 for magnifying and photographing minute portions of a subject; a support unit 313 connected to the base of the microscope unit 312 and including an arm that rotatably supports the microscope unit 312; and a base unit 314 that rotatably holds the base of the support unit 313 and allows for movement on the floor. The base unit 314 includes a light source device 3 for generating white light, first narrowband light, and second narrowband light, etc., which are emitted from the microscope unit 310 toward the subject; and a control device 9 for controlling the operation of the surgical microscope system 300. The light source device 3 and the control device 9 each have at least the same configuration as that of the first embodiment described above. Specifically, the light source device 3 includes a condenser lens 30, a first light source unit 31, a second light source unit 32, and a light source control unit 33. Furthermore, the control device 9 includes an image / picture converter 91, an image processor 92, an input unit 93, a recording unit 94, and a control unit 95. The base portion 314 may be configured to be fixed to a ceiling, a wall, or the like to support the support portion 313 instead of being movably provided on the floor.

[0181] The microscope section 312 is, for example, cylindrical in shape and has the above-mentioned medical imaging device inside. Specifically, the medical imaging device has the same structure as the endoscopic camera head 5 involved in the above-mentioned embodiment 1. For example, the microscope section 312 includes an optical system 51, a drive section 52, an imaging element 53, a cutoff filter 54, an A / D conversion section 55, a P / S conversion section 56, an imaging recording section 57, and an imaging control section 58. In addition, a switch for receiving input of an operation instruction of the microscope device 310 is provided on the side surface of the microscope section 312. A cover glass (not shown) is provided on the opening surface of the lower end portion of the microscope section 312 to protect the interior.

[0182] In the surgical microscope system 300 thus configured, the operator or other user can operate various switches while holding the microscope unit 312 to move the microscope unit 312, perform zoom operations, or switch the illumination light. Furthermore, the microscope unit 312 is preferably shaped elongated in the observation direction to facilitate changing the field of view while holding the microscope unit. Therefore, the microscope unit 312 may also have a shape other than a cylinder, such as a polygonal prism.

[0183] In the third embodiment described above, similar to the first embodiment, in the surgical microscope system 300, when the fluorescent area in the fluorescence image changes, output mode information of the treatment instrument is acquired, and information corresponding to the output mode is assigned to the fluorescent image in the fluorescence image and displayed, thereby allowing the surgeon to understand the output mode of each fluorescent image. According to this third embodiment, the surgeon can understand the treatment type of the fluorescent image in the fluorescence image.

[0184] (Other embodiments)

[0185] By appropriately combining the multiple components disclosed in the endoscope systems according to the above-mentioned embodiments 1 and 2 of the present disclosure or the surgical microscope system according to the above-mentioned embodiment 3, various inventions can be formed. For example, some components can be deleted from all the components described in the endoscope systems or surgical microscope systems according to the above-mentioned embodiments of the present disclosure. In addition, the components described in the endoscope systems or surgical microscope systems according to the above-mentioned embodiments of the present disclosure can also be appropriately combined. Furthermore, this embodiment can be applied as long as the processing is based on the fluorescence emitted by a substance generated by burning, etc.

[0186] In addition, in the embodiment and the modified example, the processing example is described on the premise that the first fluorescent image and the second fluorescent image are images of the same viewing angle. However, when using images of different viewing angles in which the same subject is captured in part, a known method such as pattern matching is used to match the fluorescent areas (thermal denaturation areas), detect changes in the fluorescent areas, and perform setting processing of the thermal denaturation areas generated after the output is turned off.

[0187] In the endoscope system or surgical microscope system according to the embodiment of the present disclosure, the above-mentioned "unit" can be replaced by "unit", "circuit", etc. For example, the control unit can be replaced by a control unit or a control circuit.

[0188] In the descriptions of the flowcharts in this specification, although expressions such as "first," "afterwards," and "next" are used to clarify the sequential relationship between the processes in the steps, the order of the processes required to implement the present invention is not solely defined by these expressions. In other words, the order of the processes in the flowcharts described in this specification can be changed within a range consistent with the order in which they are performed.

[0189] In addition, the programs executed by the devices involved in embodiments 1 to 3 are provided in the form of file data in an installable or executable form recorded on a computer-readable recording medium such as a CD-ROM, floppy disk (FD), CD-R, DVD (Digital Versatile Disk), USB medium, flash memory, etc.

[0190] Furthermore, the program for causing the apparatuses according to Embodiments 1 to 3 to execute may be stored on a computer connected to a network such as the Internet and provided by downloading via the network. Furthermore, the program for causing the information processing apparatuses according to Embodiments 1 to 3 to execute may be provided or distributed via a network such as the Internet.

[0191] In addition, in embodiments 1 and 2, an example in which the light source device 3 and the control device 9 are different entities is described, but a structure in which the light source device 3 and the control device 9 are integrated can also be provided. In addition, in embodiment 3, an example in which the light source device 3 and the control device 9 are integrated is described, but a structure in which the light source device 3 and the control device 9 are different entities can also be provided.

[0192] While several embodiments of the present application have been described in detail above based on the drawings, these are merely examples and the present invention can be implemented in other ways that are variously modified and improved based on the knowledge of those skilled in the art, represented by the methods described in the columns of this disclosure.

[0193] Industrial applicability

[0194] As described above, the medical device, medical system, medical device operating method, and medical device operating program according to the present invention are useful for enabling the operator to understand the thermal denaturation that occurs when the output of the treatment instrument is turned off.

[0195] Description of Reference Numerals

[0196] 1.101: Endoscope system; 2. Insertion unit; 3. Light source device; 4. Light guide; 5. Endoscope camera head; 6. First transmission cable; 7. Display device; 8. Second transmission cable; 9. Control device; 10. Third transmission cable; 21. Eyepiece unit; 22. Optical system; 23. Illumination optical system; 30. Converging lens; 31. First light source unit; 32. Second light source unit; 33. Light source control unit; 51. Optical system; 52. Drive unit; 53. Image sensor; 54. Cutoff filter; 55. A / D converter; 56. P / S converter; 57. Video recording unit; 58. Video control unit; 61. Video connector; 62. Camera head connector; 91. S / P converter; 92. Image processing unit; 93. Input unit; 94. Recording unit; 95. Control unit; 100: Treatment system; 102 : Endoscope; 110: Treatment instrument; 115: Switch; 120: Treatment instrument control device; 121: Insertion part; 122: Operation part; 123: Universal cable; 124: Front end part; 125: Bending part; 126: Flexible tube part; 127, 128: Connector part; 127a: Coil cable; 130: Electric cable; 241: Light guide; 242: Illumination lens; 243: Camera device; 244: Optical system; 300: Surgical microscope system; 310: Microscope device; 312: Microscope part; 313: Support part; 314: Base part; 511: Lens; 531: Pixel part; 532: Color filter; 921: Generation part; 922: Extraction part; 923: Fluorescence area determination part; 924: Output mode information assignment part; 925: Output part; 941: Program recording part.

Claims

1. A medical device comprising: An output mode information imparting unit imparts information on the output mode of the energy device to a fluorescent area in a fluorescent image based on fluorescence generated by the excitation light, based on the output information of the energy device, wherein: The excitation light is light that excites a substance produced by burning with the energy device; as well as An output unit outputs information obtained by assigning the output mode information to the fluorescent area by the output mode information assigning unit.

2. The medical device according to claim 1, wherein The device further includes a generating unit for generating the fluorescent image.

3. The medical device according to claim 1, wherein The output mode is one of an incision mode for incising the treatment object and a closing mode for coagulating the treatment object to close it.

4. The medical device according to claim 1, wherein Also features: an extraction unit that extracts a fluorescent area in the first fluorescent image and the second fluorescent image having different output modes from the energy device; and a fluorescence region determination unit that determines the presence or absence of a new fluorescence region that exists only in the second fluorescence image with respect to the fluorescence region extracted by the extraction unit; The output mode information imparting section imparts information of different output modes to the new fluorescence region and fluorescence regions other than the new fluorescence region.

5. The medical device according to claim 4, wherein The second fluorescent image is an image captured later in time than the first fluorescent image.

6. The medical device according to claim 1, wherein The fluorescence is light generated when the substance is excited.

7. The medical device according to claim 6, wherein: The substances are advanced glycation end products produced by heat denaturation.

8. The medical device according to claim 2, wherein: The generating unit generates a display image that is displayed in a different manner for each type of the output mode assigned by the output mode information assigning unit.

9. The medical device according to claim 2, wherein: The fluorescence shows the trajectory of the front end of the energy device.

10. The medical device according to claim 2, wherein The output mode includes a first mode and a second mode different from the first mode, The generating unit can generate a display image that emphasizes only the fluorescent region assigned the first mode information as the output mode and a display image that emphasizes only the fluorescent region assigned the second mode information as the output mode.

11. The medical device according to claim 10, wherein: The generating unit generates a display image that is displayed in a fluorescent image in a different manner for each type of the output mode assigned by the output mode information assigning unit.

12. The medical device according to claim 10, wherein: The generating unit generates a display image displayed in a white-light image in a manner different for each type of the output mode assigned by the output mode information assigning unit.

13. The medical device according to claim 1, wherein The output mode information imparting unit imparts information on the output mode of the energy device to a region exceeding a specific fluorescence intensity among the fluorescence regions in the fluorescence image.

14. A medical system comprising: an imaging device for capturing an image of the subject; a light source device capable of irradiating excitation light for exciting a substance generated by thermally treating living tissue; as well as a control device electrically connected to the imaging device and capable of communicating with a device control device for controlling an energy device for the cautery treatment target; in, The control device has: an output mode information imparting unit that imparts information on an output mode of the energy device to a fluorescent area in a fluorescence image based on fluorescence generated by excitation light based on output information of the energy device, wherein the excitation light is light that excites a substance generated by cauterization using the energy device; as well as An output unit outputs information obtained by assigning the output mode information to the fluorescent area by the output mode information assigning unit.

15. A method for operating a medical device, performed by the medical device, the method comprising the following steps: an output mode information imparting step, wherein the output mode information imparting unit imparts information on the output mode of the energy device to a fluorescent area in a fluorescence image based on fluorescence generated by excitation light, based on output information of the energy device, wherein the excitation light is light that excites a substance generated by cauterization using the energy device; and In the output step, the output unit outputs information obtained by assigning the output mode information to the fluorescence region by the output mode information assigning unit.

16. A medical device operating program, executed by the medical device, wherein the operating program causes the following steps to be performed: A generating step of generating a fluorescence image based on the excitation light and the fluorescence generated by the excitation light, wherein The excitation light is light that excites a substance produced by burning with an energy device; an output mode information assigning step of assigning information on the output mode of the energy device to a fluorescent area in a fluorescence image based on fluorescence generated by excitation light based on the output information of the energy device, wherein the excitation light is light that excites a substance generated by cauterization using the energy device; as well as The output step outputs information obtained by assigning the output mode information to the fluorescent area in the output mode information assigning step.

Images