Medical device, medical system, method for operating medical device, and program for operating medical device

Abstract

A medical device according to the present disclosure is provided with: a generation unit that generates a first white light image, which is an image including irradiation with white light of a subject to be subjected to cauterization treatment, and which is an image including irradiation with white light of the subject to be subjected to cauterization treatment, a second white light image, which is an image including irradiation with white light of the subject to be subjected to cauterization treatment, and a fluorescent light image based on fluorescent light; the second white light image is an image of which the imaging time is later than that of the first white light image, and the fluorescent light is generated by exciting light for exciting a glycosylation end product generated by burning; a detection unit that detects mist on the basis of the first white light image and the second white light image; and a control unit that executes notification processing for notifying generation of the mist when the mist is detected by the detection unit.

Description

Technical Field

[0001] The present disclosure relates to a medical device, a medical system, an operating method of the medical device, and an operating program of the medical device that performs image processing on an imaging signal of an image of a subject and outputs the resultant image. Background Art

[0002] Conventionally, there is known a technique in which a surgical endoscope is inserted into a subject, and the operator performs treatment by cauterizing the living tissue using a treatment instrument such as an energy device while observing the treatment portion (see, for example, Patent Document 1).

[0003] Furthermore, when biological tissue is burned, it undergoes thermal denaturation, producing advanced glycation end-products (AGEs), also known as "scorches." These AGEs fluoresce when exposed to light of a specific wavelength. By observing the fluorescence emitted by the AGEs, the surgeon can identify the thermally denatured area at the treatment site.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: International Publication No. 2020 / 054723 Summary of the Invention

[0007] Problems to be solved by the invention

[0008] However, water vapor, smoke, etc. generated by burning biological tissues spreads, causing a phenomenon called mist. When mist is generated, the mist may obscure the fluorescence image and prevent the thermally denatured area from being properly detected.

[0009] The present disclosure has been made in view of the above circumstances, and an object thereof is to provide a medical device, a medical system, an operating method of a medical device, and an operating procedure of a medical device that can appropriately detect a thermally denatured area even when mist is generated.

[0010] Solutions for solving problems

[0011] In order to solve the above-mentioned problems and achieve the purpose, the medical device involved in the present disclosure includes: a generation unit that generates a first white light image, a second white light image, and a fluorescence image based on fluorescence, wherein the first white light image is an image based on white light irradiation that includes a treatment object subjected to a cauterization treatment, the second white light image is an image captured later than the first white light image, and the fluorescence is fluorescence generated by excitation light that excites advanced glycation end products generated by the cauterization; a detection unit that detects mist based on the first white light image and the second white light image; and a control unit that, when the detection unit detects the mist, executes a notification process for notifying the generation of the mist.

[0012] Furthermore, in the medical device according to the present disclosure described above, the detection unit detects the mist based on a mist evaluation value calculated from a signal value of an image.

[0013] In addition, in the above-mentioned disclosure of the medical device involved in the present invention, the mist evaluation value includes a first evaluation value calculated based on the brightness value of the white light image, a second evaluation value calculated based on the chroma value of the white light image, and a third evaluation value calculated based on the contrast value of the white light image, and the detection unit compares the mist evaluation value of the first white light image with the mist evaluation value of the second white light image to detect the mist.

[0014] In addition, in the above-mentioned disclosure of the medical device involved in the present disclosure, when the first evaluation value of the second white light image is greater than the first evaluation value of the first white light image, and the second evaluation value of the second white light image is less than the second evaluation value of the first white light image, and the third evaluation value of the second white light image is less than the third evaluation value of the first white light image, the detection unit detects the presence of the mist in the second white light image.

[0015] In addition, the medical device according to the present disclosure further includes a dividing unit that divides the first white-light image and the second white-light image into a plurality of divided areas, and the detecting unit detects the mist for each of the divided areas.

[0016] Furthermore, the medical device according to the present disclosure further includes an extraction unit configured to extract an overlapping region between the fluorescent image and the mist in the fluorescent image when the mist is detected.

[0017] In the medical apparatus according to the present disclosure, in the above disclosure, the control unit performs notification when the overlapping region where the fluorescent image in the fluorescent image overlaps with the mist is extracted.

[0018] Furthermore, the medical device according to the present disclosure further includes a correction unit configured to correct the fluorescence amount in the overlapping region in the fluorescence image in the above disclosure.

[0019] In addition, in the medical device involved in the present disclosure, the detection unit detects the mist based on the mist evaluation value calculated from the signal value of the image, and the correction unit corrects the fluorescence amount by referring to a table that establishes a correspondence between the mist evaluation value and the correction information of the fluorescence amount.

[0020] In the medical device according to the present disclosure, in the above disclosure, the overlapping region is composed of one or more divided regions.

[0021] Furthermore, the medical device according to the present disclosure further includes an extraction unit configured to extract a thermally denatured region in the fluorescence image in the above disclosure.

[0022] In addition, the medical device involved in the present disclosure is further provided with a segmentation unit in the above disclosure, which segments the first white light image and the second white light image into multiple segmented areas, and the detection unit detects the mist according to each of the segmented areas. The medical device is also provided with an output unit, which outputs a display image that shows the thermal denaturation area overlapping with the mist and the thermal denaturation area not overlapping with the mist in different forms.

[0023] In addition, the medical device involved in the present disclosure includes: a detection unit, which detects mist based on a first white light image and a second white light image, wherein the first white light image is an irradiation image based on white light, and the second white light image is an image captured later than the first white light image; an extraction unit, which extracts thermal denaturation information based on a fluorescence image based on fluorescence, wherein the fluorescence is fluorescence generated by excitation light that excites glycation end products generated by burning; and a notification unit, which notifies the thermal denaturation information based on the detection result of the mist by the detection unit.

[0024] In addition, the medical system involved in the present disclosure includes: a medical imaging device that images the inside of a lumen of a subject; a light source device that can irradiate white light and excitation light for exciting advanced glycation end products generated by heat treatment of biological tissue; and a control device, wherein the medical imaging device is detachably mounted on the control device, and the control device includes: a generation unit that generates a first white light image, a second white light image, and a fluorescence image based on fluorescence, the first white light image being an image irradiated with white light and including a treatment object subjected to cauterization treatment; the second white light image being an image captured later than the first white light image, the fluorescence being fluorescence generated by the excitation light that excites the advanced glycation end products generated by the cauterization; a detection unit that detects mist based on the first white light image and the second white light image; and a control unit that, when the detection unit detects the mist, executes a notification process for notifying the generation of the mist.

[0025] In addition, the working method of the medical device involved in the present disclosure is a working method of the medical device performed by the medical device, and the working method includes the following steps: a first white light image generating step of generating a first white light image, the first white light image is an image based on white light irradiation including the treatment object subjected to the burn treatment; a second white light image generating step of generating a second white light image, the second white light image is an image captured later than the first white light image; a fluorescence image generating step of generating a fluorescence image based on fluorescence, the fluorescence being fluorescence generated by excitation light that excites glycation end products generated by the burn; a detection step of detecting mist based on the first white light image and the second white light image; and a notification step of notifying the generation of the mist when the mist is detected in the detection step.

[0026] In addition, the working procedure of the medical device involved in the present disclosure is a working procedure of the medical device executed by the medical device, and the working procedure causes the following steps to be executed: a first white light image generating step of generating a first white light image, the first white light image being an image based on white light irradiation including the treatment object subjected to the burn treatment; a second white light image generating step of generating a second white light image, the second white light image being an image captured later than the first white light image; a fluorescence image generating step of generating a fluorescence image based on fluorescence, the fluorescence being fluorescence generated by excitation light that excites glycation end products generated by the burn; a detection step of detecting mist based on the first white light image and the second white light image; and a notification step of notifying the generation of the mist when the mist is detected in the detection step.

[0027] Effects of the Invention

[0028] According to the present disclosure, there is an effect that a thermally denatured region can be appropriately detected even when mist is generated. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a diagram showing a schematic configuration of an endoscope system according to the first embodiment.

[0030] Figure 2 This is a block diagram showing the functional configuration of the main parts of the endoscope system according to the first embodiment.

[0031] Figure 3 This is a diagram schematically showing wavelength characteristics of light emitted by the first light source unit and the second light source unit according to the first embodiment.

[0032] Figure 4 This is a diagram schematically showing the structure of a pixel portion according to Embodiment 1.

[0033] Figure 5 This is a diagram schematically showing the structure of the color filter according to Embodiment 1.

[0034] Figure 6 It is a diagram schematically showing the sensitivity characteristics of each filter.

[0035] Figure 7A Schematically shows the signal value of the R pixel of the image sensor according to the first embodiment.

[0036] Figure 7B Schematically showing the signal values ​​of the G pixels of the image sensor according to the first embodiment.

[0037] Figure 7C This is a diagram schematically showing the signal value of the B pixel of the image sensor according to the first embodiment.

[0038] Figure 8 This is a diagram schematically showing the structure of the cut filter according to the first embodiment.

[0039] Figure 9 This is a diagram schematically showing the transmission characteristics of the cut filter according to the first embodiment.

[0040] Figure 10 This is a diagram schematically showing the observation principle in the normal light observation mode according to the first embodiment.

[0041] Figure 11 This is a diagram schematically showing the observation principle in the heat treatment observation mode according to the first embodiment.

[0042] Figure 12This is a flowchart for explaining the mist determination process using the endoscope system according to the first embodiment.

[0043] Figure 13 A diagram for explaining the first white-light image and the second white-light image.

[0044] Figure 14 This is a diagram for explaining the detection of the mist generation area.

[0045] Figure 15 A diagram showing an example of a fluorescence image.

[0046] Figure 16 This is a block diagram showing the functional configuration of a main portion of an endoscope system according to a modification of the first embodiment.

[0047] Figure 17 This is a flowchart for explaining the mist determination process using the endoscope system according to the modification of the first embodiment.

[0048] Figure 18 This is a diagram showing a schematic configuration of an endoscope system according to the second embodiment.

[0049] Figure 19 This is a block diagram showing the functional configuration of the main parts of the endoscope system according to the second embodiment.

[0050] Figure 20 This is a diagram showing a schematic configuration of a surgical microscope system according to a third embodiment. DETAILED DESCRIPTION

[0051] The following is a method for implementing the present disclosure and the attached Figure 1 Detailed description will be given below. In addition, the present disclosure is not limited to the following embodiments. In addition, the figures referred to in the following description are merely schematic illustrations of shapes, sizes and positional relationships to the extent that the contents of the present disclosure can be understood. That is, the present disclosure is not limited to the shapes, sizes and positional relationships illustrated in the figures. Moreover, in the description of the drawings, the same parts are marked with the same figure numbers for description. In addition, as an example of the endoscope system involved in the present disclosure, an endoscope system including a rigid endoscope and a medical imaging device is described.

[0052] (Implementation Method 1)

[0053] [Structure of the endoscope system]

[0054] Figure 1 This is a diagram showing a schematic configuration of an endoscope system according to the first embodiment. Figure 1The illustrated endoscope system 1 is a system used in the medical field for observing biological tissue within a subject, such as a living organism. The endoscope system 1 is used when performing surgery or treatment on a subject using a treatment instrument (not shown), such as an energy device capable of thermal treatment. The operator performs surgery or treatment while observing a display device displaying an observation image based on image data captured by a medical imaging device.

[0055] The endoscope system 1 includes an insertion portion 2 , a light source device 3 , a light guide 4 , an endoscopic camera head 5 (endoscope imaging device), a first transmission cable 6 , a display device 7 , a second transmission cable 8 , a control device 9 , and a third transmission cable 10 .

[0056] The insertion portion 2 is a rigid scope having a hard and elongated shape. The insertion portion 2 is inserted into a subject such as a patient via a cannula. The insertion portion 2 is internally provided with an optical system such as a lens for forming an observation image. In addition, the insertion portion 2 may also be partially flexible.

[0057] One end of the light guide 4 is connected to the light source device 3. Under the control of the control device 9, the light source device 3 supplies illumination light to the end of the light guide 4 for irradiation into the subject. The light source device 3 is implemented using the following components: one or more light sources such as LEDs (light-emitting diodes), xenon lamps, and semiconductor laser elements such as LDs (laser diodes); a processor serving as a processing device comprising hardware such as an FPGA (field programmable gate array) and a CPU (central processing unit); and a memory serving as a temporary storage area used by the processor.

[0058] One end of the light guide 4 is detachably connected to the light source device 3 , and the other end is detachably connected to the insertion portion 2 . The light guide 4 guides illumination light supplied from the light source device 3 from one end to the other end, and supplies the illumination light to the insertion portion 2 .

[0059] The endoscopic camera head 5 is detachably connected to the eyepiece portion 21 of the insertion portion 2. The endoscopic camera head 5 is a medical imaging device that, under the control of the control device 9, receives an observation image formed by the insertion portion 2 and performs photoelectric conversion to generate an imaging signal (RAW data). The imaging signal is then output to the control device 9 via the first transmission cable 6.

[0060] One end of the first transmission cable 6 is detachably connected to the control device 9 via a video connector 61, and the other end is detachably connected to the endoscopic camera 5 via a camera connector 62. The first transmission cable 6 transmits the imaging signal output from the endoscopic camera 5 to the control device 9, and transmits setting data and power output from the control device 9 to the endoscopic camera 5. Here, the setting data refers to control signals, synchronization signals, clock signals, and the like for controlling the endoscopic camera 5.

[0061] The display device 7 displays an observation image based on an imaging signal processed by the control device 9 and various information related to the endoscope system 1 under the control of the control device 9. The display device 7 is implemented using a display monitor such as liquid crystal or organic EL (Electro Luminescence).

[0062] One end of the second transmission cable 8 is detachably connected to the display device 7 , and the other end is detachably connected to the control device 9 . The second transmission cable 8 transmits the image signal processed by the control device 9 to the display device 7 .

[0063] The control device 9 is implemented using the following components: a processor (including hardware such as a GPU (Graphics Processing Unit), FPGA, or CPU) as a processing device; and memory as a temporary storage area used by the processor. The control device 9 comprehensively controls the operation of the light source device 3, the endoscopic camera head 5, and the display device 7 via the first transmission cable 6, the second transmission cable 8, and the third transmission cable 10, in accordance with a program stored in the memory. Furthermore, the control device 9 performs various image processing on the imaging signal input via the first transmission cable 6 and outputs it to the second transmission cable 8.

[0064] One end of the third transmission cable 10 is detachably connected to the light source device 3 , and the other end is detachably connected to the control device 9 . The third transmission cable 10 transmits control data from the control device 9 to the light source device 3 .

[0065] [Functional structure of the main parts of the endoscope system]

[0066] Next, the functional configuration of the main parts of the endoscope system 1 will be described. Figure 2 1 is a block diagram showing the functional configuration of the main parts of the endoscope system 1 .

[0067] [Structure of the insertion part]

[0068] First, the structure of the insertion portion 2 will be described. The insertion portion 2 includes an optical system 22 and an illumination optical system 23 .

[0069] The optical system 22 forms an image of the subject by converging reflected light from the subject, return light from the subject, excitation light from the subject, and luminescence emitted by the subject. The optical system 22 is implemented using one or more lenses.

[0070] The illumination optical system 23 irradiates the subject with illumination light supplied from the light guide 4. The illumination optical system 23 is implemented using one or more lenses and the like.

[0071] [Structure of light source device]

[0072] Next, a description will be given of the configuration of the light source device 3 . The light source device 3 includes a condenser lens 30 , a first light source unit 31 , a second light source unit 32 , and a light source control unit 33 .

[0073] The condenser lens 30 condenses the light emitted from the first light source unit 31 and the second light source unit 32 and emits the light toward the light guide 4 .

[0074] Under the control of the light source control unit 33, the first light source unit 31 emits white light (normal light) as visible light, thereby supplying white light to the light guide 4 as illumination light. The first light source unit 31 is constructed using a collimating lens, a white LED lamp, and a driver. Alternatively, the first light source unit 31 can simultaneously emit red, green, and blue LED lamps to supply visible white light. Of course, the first light source unit 31 can also be constructed using a halogen lamp, a xenon lamp, or the like.

[0075] Under the control of the light source control unit 33, the second light source unit 32 emits narrowband light in a wavelength range different from and narrower than that of white light, thereby supplying narrowband light as illumination light to the light guide 4. Here, narrowband light refers to, for example, light in the wavelength range of 400 nm to 430 nm with a center wavelength of 415 nm. The second light source unit 32 is implemented using a collimating lens, a semiconductor laser such as a violet laser diode (LD), and a driver. Furthermore, in the present embodiment, the narrowband light functions as excitation light for exciting advanced glycation end products produced by heat treatment of biological tissue.

[0076] The light source control unit 33 is implemented using a processor (e.g., an FPGA or CPU) as a processing device and a memory (e.g., a temporary storage area) used by the processor. Based on control data input from the control device 9, the light source control unit 33 controls the light emission timing and duration of each of the first light source unit 31 and the second light source unit 32.

[0077] Here, the wavelength characteristics of the light emitted by the first light source unit 31 and the second light source unit 32 will be described. Figure 3 Schematically shows the wavelength characteristics of the light emitted by the first light source unit 31 and the second light source unit 32. Figure 3 In the figure, the horizontal axis represents wavelength (nm) and the vertical axis represents relative intensity. Figure 3 In the middle, curve L WL The curve L represents the wavelength characteristics of the white light emitted by the first light source unit 31. V The wavelength characteristics of the narrowband light (excitation light) emitted by the second light source unit 32 are shown. The second light source unit 32 emits light with a central wavelength (peak wavelength) of 415 nm and a wavelength range of 400 nm to 430 nm. Figure 3 The curve L in WL The wavelength characteristics shown represent the characteristics when a white LED is used as the first light source unit 31 .

[0078] [Structure of endoscope camera]

[0079] return Figure 2 , the structure of the endoscope system 1 is further described.

[0080] Next, the configuration of the endoscopic camera head 5 will be described. The endoscopic camera head 5 includes an optical system 51, a drive unit 52, an imaging element 53, a cut filter 54, an A / D converter 55, a P / S converter 56, an imaging and recording unit 57, and an imaging control unit 58.

[0081] The optical system 51 forms an image of the subject, focused by the optical system 22 of the insertion portion 2, onto the light-receiving surface of the imaging element 53. The optical system 51 is capable of changing the focal length and focus position. The optical system 51 is constructed using a plurality of lenses 511. The optical system 51 changes the focal length and focus position by moving each of the plurality of lenses 511 along the optical axis L1 via the drive unit 52.

[0082] The drive unit 52 moves the multiple lenses 511 of the optical system 51 along the optical axis L1 under the control of the imaging control unit 58. The drive unit 52 is configured using a motor such as a stepping motor, a DC motor, or a voice coil motor, and a transmission mechanism such as gears that transmits the motor's rotation to the optical system 51.

[0083] The imaging element 53 is implemented using a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) image sensor having a plurality of pixels arranged in a two-dimensional matrix. Under the control of the imaging control unit 58, the imaging element 53 receives the subject image (light) formed by the optical system 51 and passed through the cut filter 54, performs photoelectric conversion on the subject image, generates an imaging signal (RAW data), and outputs it to the A / D converter 55. The imaging element 53 includes a pixel unit 531 and a color filter 532.

[0084] Figure 4 Schematic diagram showing the structure of the pixel portion 531. Figure 4 As shown, the pixel portion 531 is composed of a plurality of pixels P such as photodiodes that store charges according to the amount of light. nm (n, m are integers greater than 1) are arranged in a two-dimensional matrix. The pixel unit 531 is controlled by the imaging control unit 58 to obtain a plurality of pixels P nm The pixel P in the reading area arbitrarily set as the reading object nm The image signal is read as image data and output to the A / D conversion unit 55 .

[0085] Figure 5 : This is a diagram schematically showing the structure of the color filter 532. The color filter 532 is composed of a Bayer array with 2×2 as one unit. The color filter 532 is composed of a filter R that transmits light in the red wavelength range, two filters G that transmit light in the green wavelength range, and a filter B that transmits light in the blue wavelength range. Figure 5 In the figure, the reference numerals for each filter (e.g. G 11 ) and pixel P nm Corresponding means configured at corresponding pixel positions.

[0086] Figure 6 is a diagram schematically showing the sensitivity characteristics of each filter. Figure 6 In the figure, the horizontal axis represents wavelength (nm) and the vertical axis represents transmission characteristics (sensitivity characteristics). Figure 6 In the middle, curve L B Indicates the transmission characteristics of filter B, curve L G Indicates the transmission characteristics of filter G, curve L R Indicates the transmission characteristics of filter R.

[0087] Filter B transmits light in the blue wavelength range (see Figure 6 The curve LB ). In addition, the filter G transmits light in the green wavelength range (see Figure 6 The curve L G ). In addition, the filter R transmits light in the red wavelength range (see Figure 6 The curve L R ). In addition, in the following, the filter R is arranged on the light receiving surface to form a pixel P nm The pixel P is formed by placing the filter G on the light receiving surface. nm The pixel P is formed by placing the filter B on the light receiving surface. nm This will be described as a B pixel.

[0088] According to the imaging element 53 configured in this manner, when receiving the subject image formed by the optical system 51, the imaging element 53 generates color signals (R signal, G signal, and B signal) for each of the R pixel, the G pixel, and the B pixel (see Figures 7A to 7C ).

[0089] return Figure 2 , the structure of the endoscope system 1 is further described.

[0090] The cutoff filter 54 is disposed on the optical axis L1 between the optical system 51 and the imaging element 53. The cutoff filter 54 is provided on the light-receiving surface side (incident surface side) of at least the G pixel of the color filter 532, which is provided with the filter G that transmits the green wavelength range. The cutoff filter 54 blocks light in the wavelength range of the excitation light and transmits light in the wavelength range on the long wavelength side that is longer than the wavelength range of the excitation light.

[0091] Figure 8 Schematically shows the structure of the cut filter 54. Figure 8 As shown, the filter F constituting the cutoff filter 54 11 Configured on filter G 11 (Refer to Figure 5 ) is configured at a position where the filter G 11 The light-receiving side directly above the .

[0092] Figure 9 is a diagram schematically showing the transmission characteristics of the cut filter 54. Figure 9 In the figure, the horizontal axis represents wavelength (nm) and the vertical axis represents transmittance. Figure 9 In the middle, curve L F The curve L represents the transmission characteristics of the cutoff filter 54. V Indicates the wavelength characteristics of the excitation light.

[0093] The cutoff filter 54 blocks the wavelength range of the excitation light and transmits wavelengths longer than the wavelength range of the excitation light. Specifically, the cutoff filter 54 blocks light in a wavelength range shorter than the wavelength range of the excitation light and transmits light in a wavelength range longer than the wavelength range of the excitation light.

[0094] return Figure 2 , continue to explain the structure of the endoscope camera 5.

[0095] Under the control of the imaging control unit 58, the A / D converter 55 performs A / D conversion processing on the analog imaging signal input from the imaging element 53 and outputs the resulting signal to the P / S converter 56. The A / D converter 55 is implemented using an A / D conversion circuit or the like.

[0096] Under the control of the camera control unit 58, the P / S converter 56 performs parallel / serial conversion on the digital camera signal input from the A / D converter 55, and outputs the parallel / serial converted camera signal to the control device 9 via the first transmission cable 6. The P / S converter 56 is implemented using a P / S conversion circuit, etc. In the first embodiment, an E / O converter that converts the camera signal into an optical signal may be provided in place of the P / S converter 56, and the camera signal may be output to the control device 9 via an optical signal. Alternatively, the camera signal may be transmitted to the control device 9 via wireless communication such as Wi-Fi (Wireless Fidelity) (registered trademark).

[0097] The image recording unit 57 records various information related to the endoscopic camera head 5 (e.g., pixel information of the image sensor 53 and characteristics of the cutoff filter 54). Furthermore, the image recording unit 57 records various setting data and control parameters transmitted from the control device 9 via the first transmission cable 6. The image recording unit 57 is configured using a nonvolatile memory or a volatile memory.

[0098] The imaging control unit 58 controls the operation of the drive unit 52, the imaging element 53, the A / D converter 55, and the P / S converter 56 based on the setting data received from the control device 9 via the first transmission cable 6. The imaging control unit 58 is implemented using the following components: a TG (Timing Generator); a processor as a processing device including hardware such as a CPU; and a memory as a temporary storage area used by the processor.

[0099] [Structure of control device]

[0100] Next, the configuration of the control device 9 will be described.

[0101] The control device 9 includes an S / P conversion unit 91 , an image processing unit 92 , an input unit 93 , a recording unit 94 , and a control unit 95 .

[0102] Under the control of the control unit 95, the S / P converter 91 performs serial / parallel conversion on the image data received from the endoscopic camera 5 via the first transmission cable 6 and outputs the data to the image processing unit 92. Furthermore, if the endoscopic camera 5 outputs the imaging signal via an optical signal, an O / E converter that converts the optical signal into an electrical signal may be provided in place of the S / P converter 91. Furthermore, if the endoscopic camera 5 transmits the imaging signal via wireless communication, a communication module capable of receiving wireless signals may be provided in place of the S / P converter 91.

[0103] Under the control of the control unit 95, the image processing unit 92 performs predetermined image processing on the parallel data imaging signal input from the S / P conversion unit 91 and outputs the image to the display device 7. The predetermined image processing herein includes processing such as demosaicing, white balance, gain adjustment, gamma correction, and format conversion. The image processing unit 92 is implemented using a processor comprising hardware such as a GPU or FPGA as a processing device, and a memory serving as a temporary storage area used by the processor. The image processing unit 92 includes a generation unit 921, a segmentation unit 922, a calculation unit 923, a detection unit 924, an output unit 925, and an extraction unit 926.

[0104] The generator 921 generates a first image and a second image, the first image including one or more characteristic regions to be removed by the operator, and the second image including one or more ablated regions after being ablated by the energy device. Specifically, the generator 921 generates a white light image as the first image based on an imaging signal generated by capturing reflected light and return light from the biological tissue when white light is irradiated on the biological tissue. Furthermore, the generator 921 generates a fluorescence image as the second image based on an imaging signal generated by capturing fluorescence generated by excitation light in a heat treatment observation mode of the endoscope system 1, described later. The excitation light is light irradiated to excite advanced glycation end products (AGEs) generated by heat treatment of the biological tissue. Alternatively, the generator 921 may generate a pseudo-color image including one or more characteristic regions (lesion regions) to be removed by the operator based on an imaging signal generated by capturing reflected light and return light from the biological tissue when the excitation light is irradiated on the biological tissue, described later, in the fluorescence observation mode of the endoscope system 1, described later.

[0105] The division unit 922 divides the image generated by the generation unit 921 and sets a plurality of division regions. For example, the division unit 922 sets nine division regions of 3×3 for the white light image generated by the generation unit 921. The number of divisions and the size of each division region can be set as appropriate.

[0106] The calculation unit 923 calculates a mist evaluation value for each segmented area. Specifically, the calculation unit 923 calculates an evaluation value for the brightness, chroma, and contrast values ​​within each segmented area. Hereinafter, the evaluation value calculated based on the brightness value is referred to as the first evaluation value, the evaluation value calculated based on the chroma value is referred to as the second evaluation value, and the evaluation value calculated based on the contrast value is referred to as the third evaluation value.

[0107] Here, the brightness value increases as the amount of mist in the abdominal cavity increases, and decreases as the mist generation ceases and gradually disappears. In contrast, the saturation and contrast values ​​(dynamic range) decrease as the amount of mist in the abdominal cavity increases, and increase as the mist generation ceases and gradually disappears. At this point, the brightness value increases as the amount of mist in the abdominal cavity increases because the mist diffusely reflects light, while the saturation and contrast values ​​decrease as the amount of mist in the abdominal cavity increases because the mist is opaque.

[0108] The detection unit 924 detects the generation of mist in each segmented area. The detection unit 924 detects the generation of mist by comparing the first through third evaluation values ​​of white light images captured at different times. For example, the detection unit 924 detects the generation of mist when the temporal variation of the first through third evaluation values ​​satisfies a condition. Specifically, the detection unit 924 detects the generation of mist when the first evaluation value increases and the second and third evaluation values ​​decrease over time.

[0109] When the generation of mist is detected by the detection unit 924 , the output unit 925 outputs information for notifying the divided area in which the mist is detected.

[0110] The extraction unit 926 extracts a region where thermal denaturation has occurred based on the fluorescence image. The extraction unit 926 extracts the thermal denaturation region by, for example, extracting a region with high fluorescence intensity.

[0111] The input unit 93 receives input of various operations related to the endoscope system 1 and outputs the received operations to the control unit 95. The input unit 93 is configured using a mouse, a foot switch, a keyboard, buttons, switches, a touch panel, and the like.

[0112] The recording unit 94 is implemented using a recording medium such as a volatile memory, a nonvolatile memory, an SSD (Solid State Drive), an HDD (Hard Disk Drive), or a memory card. The recording unit 94 records data including various parameters required for the operation of the endoscope system 1. The recording unit 94 also includes a program recording unit 941 for recording various programs used to operate the endoscope system 1.

[0113] The control unit 95 is realized by using a processor as a processing device including hardware such as an FPGA or a CPU, and a memory as a temporary storage area used by the processor. The control unit 95 comprehensively controls the various components constituting the endoscope system 1 .

[0114] [Overview of each observation mode]

[0115] Next, an overview of each observation mode that can be executed by the endoscope system 1 will be described. The normal light observation mode and the fluorescence observation mode will be described in order.

[0116] [Overview of Normal Light Observation Mode]

[0117] First, the normal light observation mode will be described. Figure 10 This is a diagram schematically showing the observation principle in the normal light observation mode.

[0118] Under the control of the control device 9, the light source device 3 emits white light W1 having an intensity distribution shown in the graph G11 toward the biological tissue T1 of the subject by causing the first light source unit 31 to emit light. In this case, a portion of the reflected light and return light (hereinafter referred to as "reflected light WR10, reflected light WG10, and reflected light WB10") reflected by the biological tissue is shielded by the cutoff filter 54, and the remaining portion is incident on the imaging element 53. For example, specifically, the cutoff filter 54 shields the reflected light (reflected light WG10) incident on the G pixel and the reflected light in the wavelength range of the excitation light (excitation light W2 described later). That is, the reflected light and return light based on the irradiation of white light are incident on the filter R and the filter B, and the light in the wavelength range longer than the wavelength range of the excitation light is incident on the filter G. Therefore, the component of the light in the blue wavelength range incident on the pixel becomes smaller than when the cutoff filter 54 is not configured. The light incident on each filter is selectively transmitted according to the filter characteristics shown in graph G12.

[0119] Next, the image processing unit 92 acquires image data (RAW data) from the imaging element 53 of the endoscopic camera head 5 and performs image processing on the signal values ​​of the R pixels, G pixels, and B pixels contained in the acquired image data to generate a white light image. In this case, since the blue component contained in the image data is smaller than in conventional white light observation, the image processing unit 92 performs white balance adjustment processing to adjust the white balance so that the ratio of the red component, green component, and blue component is constant.

[0120] In the normal light observation mode, even when the cut filter 54 is arranged on the light receiving surface side of the G pixel, a natural white light image (observation image) can be observed.

[0121] [Overview of Fluorescence Observation Mode]

[0122] Next, the fluorescence observation mode will be described. Figure 11 This is a diagram schematically showing the observation principle in the fluorescence observation mode.

[0123] In recent years, minimally invasive treatments using endoscopes and laparoscopy have become increasingly common in the medical field. For example, widely performed minimally invasive treatments using endoscopes and laparoscopy include endoscopic submucosal dissection (ESD), laparoscopic endoscopic co-operative gastrectomy (LECS), non-exposed endoscopic wall-inversion surgery (NEWS), and transurethral resection of the bladder tumor (TUR-bt).

[0124] In these minimally invasive treatments, when performing treatment, for example, doctors or other operators use treatment instruments that emit energy such as high-frequency waves, ultrasound, microwaves, etc. to perform heat treatment as a pre-treatment to mark the surgical area, or as a treatment to remove the diseased area, close the incision, or coagulate the incision.

[0125] Furthermore, when amino compounds and reducing sugars are heated, a glycation reaction (Maillard reaction) occurs, in which the amino acids react with the reducing sugars. The end products produced by this Maillard reaction are collectively referred to as advanced glycation end products (AGEs). AGEs are known to contain substances with fluorescent properties. AGEs are known to emit fluorescence with a higher intensity than the autofluorescent substances naturally present in biological tissues. Therefore, the formation of AGEs significantly increases their fluorescence intensity compared to before the formation of AGEs.

[0126] AGEs generated by burning during treatment can be visualized by fluorescence observation, and the fluorescence intensity serves as an indicator of the state of thermal treatment.

[0127] Specifically, the fluorescence observation mode utilizes the fluorescence properties of AGEs generated in living tissues by thermal treatment with an energy device, etc., to visualize the thermally treated area. Therefore, in fluorescence observation mode, excitation light for exciting AGEs, for example, blue narrowband light with a center wavelength of 415 nm, is irradiated from the light source device 3 onto the living tissue. Consequently, in fluorescence observation mode, a thermally treated image (fluorescence image) that captures fluorescence (for example, green light with a wavelength of 490 to 625 nm) generated by AGEs can be observed.

[0128] Specifically, first, the light source device 3, under the control of the control device 9, emits the excitation light W2 (center wavelength 415nm: refer to Figure G13) by making the second light source unit 32 emit light to irradiate the biological tissue T2 (heat treatment area) to which the subject has been subjected to heat treatment by an energy device or the like. In this case, the reflected light (hereinafter referred to as "reflected light WR20, reflected light WG20, reflected light WB20") including at least the components of the excitation light W2 reflected at the biological tissue T2 (heat treatment area) and the return light is shielded by the cutoff filter 54, and a portion of the components on the long wavelength side is incident on the imaging element 53 (refer to Figure G14). In addition, Figure 11 In FIG, the thickness of the arrow represents the intensity of the component (light amount or signal value) of each line.

[0129] More specifically, if Figure 11As shown in graph G2, cutoff filter 54 blocks reflected light WG20 incident on G pixels within a wavelength range that includes the wavelength range of excitation light W2. Furthermore, cutoff filter 54 transmits fluorescence WF1 emitted by AGEs in tissue T2 (heat-treated area) (see graph G14). Consequently, reflected light WG20 is prevented from entering G pixels, while fluorescence WF1 does enter. Since cutoff filter 54 is positioned on the light-receiving side (incident side) of the G pixels, it prevents the reflected light WG20 from excitation light W2 from mixing with fluorescence WF1 and burying the fluorescence component.

[0130] Furthermore, the reflected light (reflected light WR20 , WB20 ) and the fluorescent light WF1 enter the R pixel and the B pixel, respectively.

[0131] The image processing unit 92 then acquires image data (RAW data) from the imaging element 53 of the endoscopic camera head 5 and performs image processing on the signal values ​​of the G and B pixels contained in the acquired image data to generate a fluorescence image. In this case, the signal values ​​of the G pixels include fluorescence information representing the fluorescence image emitted from the heat-treated area. Furthermore, the B pixels include background information representing the surrounding biological tissues of the heat-treated area, which serve as the background of the heat-treated area. The image processing unit 92 performs image processing such as gain control, pixel interpolation, and mucosal enhancement on the signal values ​​of the G and B pixels contained in the image data to generate a fluorescence image. In this gain control process, the image processing unit 92 performs processing to make the gain of the signal value for the G pixels greater than the gain of the signal value for the G pixels during normal light observation, while making the gain of the signal value for the B pixels less than the gain of the signal value for the B pixels during normal light observation. Furthermore, the image processing unit 92 performs processing to make the signal values ​​of the G pixels and the signal values ​​of the B pixels equal (1:1). Alternatively, the image processing unit 92 may generate a pseudo-color image by superimposing color information whose hue is changed according to the fluorescence intensity on the fluorescent image.

[0132] The output unit 925 outputs the generated fluorescence image to the display device 7 .

[0133] [Treatment using an endoscopic system]

[0134] Next, a treatment using the endoscope system 1 of the present disclosure will be described. In this case, the operator inserts the insertion portion 2 into the subject and causes the light source device 3 to illuminate the area containing the treatment target with white light. The operator confirms the treatment target while observing the observation image displayed on the display device 7.

[0135] Thereafter, the operator performs treatment on the treatment target of the subject while checking the white light image displayed on the display device 7. For example, the operator cauterizes and removes the treatment target using an energy device or the like inserted into the subject via the insertion portion 2.

[0136] Afterwards, the operator irradiates the treatment object with excitation light and observes the fluorescent image displayed on the display device 7. The operator determines whether the treatment (e.g., resection) at the treatment position has been completed by observing the fluorescent image displayed on the display device 7. When the operator determines that the treatment is completed, the operator ends the present operation. Specifically, the operator determines whether the resection of the treatment object has been completed by observing the fluorescent image displayed on the display device 7 and observing the burnt area that has been burnt and removed by an energy device or the like. At this time, when the operator determines that the resection of the treatment object is not completed, the operator switches the observation mode of the endoscope system 1 while repeating the observation of the white light image based on the irradiation of white light and the observation of the fluorescent image based on the irradiation of excitation light to continue the treatment.

[0137] [Handling of endoscope system]

[0138] Next, the processing executed by the endoscope system 1 will be described. Figure 12 This is a flowchart for explaining a mist determination process using an endoscope system according to one embodiment.

[0139] The control unit 95 generates a first white-light image (step S101). At this point, the control unit 95 controls the light source control unit 33 to cause the first light source unit 31 to emit light, irradiating the subject with white light. The generator 921 generates the first white-light image by acquiring an imaging signal from the imaging element 53 of the endoscopic camera head 5. In this case, the output unit 925 may also cause the display device 7 to display the first white-light image generated by the generator 921.

[0140] Next, the image processing unit 92 calculates a first mist evaluation value based on the first white-light image (step S102). To calculate the first mist evaluation value, the segmentation unit 922 first divides the first white-light image into a plurality of segmented regions (e.g., 3×3). The calculation unit 923 then calculates the first mist evaluation value based on the signal values ​​of each segmented region. In this embodiment, a first evaluation value based on brightness, a second evaluation value based on chroma, and a third evaluation value based on contrast are calculated for each segmented region as the first mist evaluation value.

[0141] Next, the control unit 95 generates a second white-light image (step S103). At this point, the control unit 95 controls the light source control unit 33 to cause the first light source unit 31 to emit light, irradiating the subject with white light. The generator 921 generates the second white-light image by acquiring an imaging signal from the imaging element 53 of the endoscopic camera head 5. In this case, the output unit 925 may also cause the display device 7 to display the second white-light image generated by the generator 921.

[0142] The second white-light image is based on image data acquired later than the first white-light image. The image data acquisition timing (imaging timing) is, for example, performed after a predetermined time interval has passed since the first white-light image was acquired.

[0143] Next, the image processing unit 92 calculates a second mist evaluation value based on the second white-light image (step S104). During the first mist evaluation value calculation process, the segmentation unit 922 first divides the second white-light image into a plurality of segmented regions (e.g., 3×3). The calculation unit 923 then calculates the second mist evaluation value based on the signal values ​​of each segmented region. In this embodiment, similar to the first mist evaluation value, a first evaluation value based on brightness, a second evaluation value based on chroma, and a third evaluation value based on contrast are calculated for each segmented region as the second mist evaluation value.

[0144] Next, the control unit 95 generates a fluorescence image (step S105). At this point, the control unit 95 controls the light source control unit 33 to cause the second light source unit 32 to emit light, irradiating the subject with excitation light. The generator 921 generates the fluorescence image by acquiring an imaging signal from the imaging element 53 of the endoscopic camera head 5. In this case, the output unit 925 may also cause the display device 7 to display the fluorescence image generated by the generator 921.

[0145] At this time, the control unit 95 sets the divided regions of the fluorescence image to correspond to the divided regions of the white light image. Hereinafter, the divided regions of the white light image are referred to as first divided regions, and the divided regions of the fluorescence image are referred to as second divided regions.

[0146] Furthermore, the fluorescence image is preferably captured at a timing close to or simultaneously with the second white-light image.

[0147] The detection unit 924 then detects the mist generation area based on the first and second mist evaluation values ​​(step S106). Specifically, the detection unit 924 detects mist generation when, for example, the first evaluation value increases while the second and third evaluation values ​​decrease over time. The detection unit 924 detects mist generation for each segmented area.

[0148] Next, the control unit 95 determines whether a mist generation area has been detected (step S107). If the detection unit 924 detects mist generation (step S107: No), the control unit 95 proceeds to step S110. On the other hand, if the detection unit 924 detects mist generation (step S107: Yes), the control unit 95 proceeds to step S108.

[0149] Here, refer to Figure 13 and Figure 14 Describe the detection and processing of mist generation. Figure 13 A diagram for explaining the first white-light image and the second white-light image. Figure 14 This is a diagram for explaining the detection of the mist generation area. Figure 13 (a) shows a first white light image WL1, Figure 13 (b) shows the second white light image WL2. Figure 13 In FIG, an example is shown in which mist is not generated in the first white light image WL1 but is generated in the second white light image WL2. The mist evaluation value is calculated for each divided area WP. In this case, the second mist evaluation value changes significantly in the area where mist M1 is generated compared to the case where mist M1 is not generated. Figure 13 In the case shown, the generation of mist is detected in the four divided areas located at the upper left, upper middle, middle left and center. The detection unit 924 identifies the divided area ( Figure 14 The segmented area WP shown M ) is set as the mist generation area.

[0150] In step S108, the control unit 95 determines whether the area where mist generation was detected overlaps with the fluorescent area. Specifically, the control unit 95 compares the corresponding segmented areas with respect to the first segmented area where mist generation was detected and the second segmented area of ​​the fluorescent image, and determines whether the first segmented area overlaps with the second segmented area containing the fluorescent image. If the control unit 95 determines that the first segmented area does not overlap with the second segmented area containing the fluorescent image (step S108: "No"), the control unit 95 transfers to step S110. Conversely, if the control unit 95 determines that the first segmented area overlaps with the second segmented area containing the fluorescent image (step S108: "Yes"), the control unit 95 transfers to step S109.

[0151] Figure 15 is a diagram illustrating an example of a fluorescence image. The control unit 95 extracts a segmented region WP that overlaps with a segmented region WP in the fluorescence image WF. For example, if a fluorescence image F1 is present in the fluorescence image WF, the control unit 95 extracts the segmented region WP that includes the fluorescence image F1 and uses the extracted segmented region as the overlapping segmented region.

[0152] In step S109, the control unit 95 executes a mist generation notification process. At this point, the control unit 95 causes the display device 7 to display information indicating the generation of mist. For example, text information indicating the generation of mist may be displayed alongside the white light image or fluorescent image, or text information indicating the generation of mist or pseudo-color information may be superimposed on the white light image or fluorescent image, for example, in the mist generation area. Alternatively, the generation of mist may be notified using sound or light.

[0153] In step S110, the control unit 95 executes thermal denaturation information extraction processing. In this process, the extraction unit 926 extracts, for example, regions with high fluorescence intensity from the fluorescence image and identifies these regions as regions undergoing thermal denaturation. The output unit 925 outputs the thermal denaturation information, including the thermally denatured regions extracted by the extraction unit 926, to the recording unit 94 and the display device 7.

[0154] Alternatively, the extraction unit 926 may extract the region based on the intensity of fluorescence having a specific wavelength.

[0155] For example, the mist determination process may be executed at pre-set time intervals or when a technician inputs an instruction to execute the detection process. In this case, the second white-light image acquired during the previous process may be used as the first white-light image. In this case, the process may be restarted from step S103.

[0156] In the first embodiment described above, the presence of mist within the viewing angle is detected based on white light images captured at different times, and notification is provided when mist is detected. According to this first embodiment, the operator is notified of mist generation, and thus, even when mist is generated, the thermally denatured area can be appropriately detected.

[0157] (Variation)

[0158] Next, refer to Figure 16 and Figure 17 A modification of the first embodiment will be described. Figure 16 This is a block diagram illustrating the functional configuration of the main components of an endoscope system according to a modification. The endoscope system according to Modification 1 includes an image processing unit 92A in place of the image processing unit 92 of the endoscope system 1 according to the embodiment. Configurations other than the image processing unit 92A are identical to those of Embodiment 1, and therefore description thereof will be omitted.

[0159] [Structure of the Image Processing Unit]

[0160] Under the control of the control unit 95, the image processing unit 92A performs predetermined image processing on the imaging signal of the parallel data input from the S / P conversion unit 91, and outputs the image to the display device 7. The image processing unit 92 includes a generation unit 921, a division unit 922, a calculation unit 923, a detection unit 924, an output unit 925, an extraction unit 926, and a correction unit 927. The following describes the correction unit, which has a different configuration from that of the embodiment, and mist detection processing according to a modified example.

[0161] The correction unit 927 corrects the fluorescence intensity based on the mist concentration. For example, the correction unit 927 refers to a correction table pre-recorded in the recording unit 94 to perform the fluorescence intensity correction. This correction table associates the mist concentration with a correction coefficient for the fluorescence intensity. The correction unit 927 outputs a correction coefficient based on the mist concentration calculated based on the white light image, for example, the difference between the first and second mist evaluation values, to correct the fluorescence intensity of the fluorescence image superimposed on the mist. The corrected fluorescence intensity is output to the generation unit 921, which generates a fluorescence image for display or calculation.

[0162] [Handling of endoscope system]

[0163] Next, the processing executed by the endoscope system according to the modification will be described. Figure 17 This is a flowchart for explaining the mist determination process using the endoscope system according to the modification.

[0164] The control unit 95 generates the first white-light image and the second white-light image, and calculates the first mist evaluation value and the second mist evaluation value (steps S201 to S204 ), similarly to the embodiment.

[0165] Next, the control unit 95 executes the generation of a fluorescence image, the detection of a mist generation region, and the determination process of overlap with the fluorescence region (steps S205 to S208 ).

[0166] When the control unit 95 determines that the first divided region overlaps with the second divided region including the fluorescent image (step S208 : Yes), it executes a notification process of mist generation (step S209 ).

[0167] The control unit 95 then corrects the fluorescence image (step S210). The correction unit 927 corrects the fluorescence intensity of the fluorescence image that overlaps the mist generation area. Specifically, the correction unit 927 refers to the correction table and multiplies the fluorescence intensity by the correction coefficient. The corrected fluorescence intensity is output to the generation unit 921, which generates a corrected fluorescence image.

[0168] Furthermore, in step S211, the control unit 95 executes thermal denaturation information extraction processing. In this process, the extraction unit 926 extracts regions of high fluorescence intensity from, for example, the fluorescence image or the corrected fluorescence image, and identifies these extracted regions as regions undergoing thermal denaturation. The output unit 925 outputs the thermal denaturation information, including the thermally denatured regions extracted by the extraction unit 926, to the recording unit 94 and the display device 7.

[0169] In the above-described variation, similar to the embodiment, a configuration is employed to detect whether mist has formed within the viewing angle based on white light images captured at different times, and to provide notification when mist is detected. According to this variation, the presence of mist is notified to the operator, etc., enabling appropriate detection of the thermally denatured area even when mist has formed.

[0170] Here, the fluorescence intensity indicated by AGEs becomes a parameter related to the position or depth of thermal denaturation. In this modification, the fluorescence intensity of the fluorescent image overlapping the mist generation area is corrected according to the mist concentration, thereby making the information on the thermal denaturation position or depth more accurate.

[0171] (Implementation Method 2)

[0172] Next, Embodiment 2 will be described. While Embodiment 1 described an endoscope system equipped with a rigid endoscope, Embodiment 2 will describe an endoscope system equipped with a flexible endoscope. The endoscope system according to Embodiment 2 will be described below. In Embodiment 2, components identical to those of the endoscope system 1 according to Embodiment 1 are denoted by the same reference numerals, and detailed descriptions thereof will be omitted.

[0173] [Structure of the endoscope system]

[0174] Figure 18 This is a diagram showing a schematic configuration of an endoscope system according to the second embodiment. Figure 19 This is a block diagram showing the functional configuration of the main parts of the endoscope system according to the second embodiment.

[0175] The endoscope system 100 is inserted into a patient or other subject to be examined and images the subject's interior. The display device 7 displays images based on the captured image data. A physician or other operator observes the images displayed on the display device 7 to inspect the presence and condition of abnormal areas such as bleeding sites, tumor sites, and abnormal sites captured as examination targets. Furthermore, the physician or other operator inserts treatment instruments, such as energy devices, into the subject's interior through the treatment instrument channel of the endoscope to perform treatment on the subject. The endoscope system 100 includes an endoscope 102 in addition to the aforementioned light source device 3, display device 7, and control device 9.

[0176] [Structure of an endoscope]

[0177] The configuration of the endoscope 102 will be described. The endoscope 102 generates image data by imaging the inside of a subject, and outputs the generated image data to the control device 9. The endoscope 102 includes an operation unit 122 and a universal cable 123.

[0178] The insertion portion 121 is flexible and elongated. It comprises a distal end portion 124 housing a camera device (described later); a freely bendable bending portion 125 composed of a plurality of bendable pieces; and a long, flexible flexible tubular portion 126 connected to the proximal end of the bending portion 125.

[0179] The front end portion 124 is made of glass fiber, etc. The front end portion 124 includes a light guide 241 that forms a light guide path for light supplied from the light source device 3 , an illumination lens 242 provided at the front end of the light guide 241 , and an imaging device 243 .

[0180] The imaging device 243 includes a light-converging optical system 244 , the imaging element 53 of the first embodiment, a cut filter 54 , an A / D converter 55 , a P / S converter 56 , an imaging and recording unit 57 , and an imaging control unit 58 .

[0181] The universal cable 123 has at least a built-in optical fiber 241 and a bundled cable that brings together one or more cables. The bundled cable is a signal line that sends and receives signals between the endoscope 102 and the light source device 3 and the control device 9, including a signal line for sending and receiving setting data, a signal line for sending and receiving camera images (image data), a signal line for sending and receiving timing signals for driving the camera element 53, etc. The universal cable 123 has a connector portion 127 that can be freely attached and detached from the light source device 3. The coiled coil cable 127a of the connector portion 127 extends, and a connector portion 128 that can be freely attached and detached from the control device 9 is provided at the extended end of the coil cable 127a.

[0182] The endoscope system 100 configured in this manner performs the same processing as that of the endoscope system 1 according to the first embodiment described above.

[0183] In the second embodiment described above, similar to the first embodiment, the following configuration is employed: based on white light images captured at different times, the presence of mist within the viewing angle is detected, and notification is provided when mist is detected. According to this second embodiment, the operator is notified of mist generation, and thus, even when mist is generated, the thermally denatured area can be appropriately detected.

[0184] (Implementation 3)

[0185] Next, Embodiment 3 will be described. While Embodiments 1 and 2 above describe endoscope systems, Embodiment 3 will describe a case where the system is applied to a surgical microscope system. In Embodiment 3, components identical to those of the endoscope system 1 according to Embodiment 1 are denoted by the same reference numerals, and detailed descriptions thereof will be omitted.

[0186] [Structure of surgical microscope system]

[0187] Figure 20 This figure shows the schematic configuration of a surgical microscope system according to Embodiment 3. The surgical microscope system 300 includes a microscope device 310, which serves as a medical imaging device and captures images for observing a subject, and a display device 7. Alternatively, the display device 7 and the microscope device 310 may be integrated.

[0188] The microscope system 310 includes a microscope unit 312 for magnifying and photographing minute portions of a subject; a support unit 313 connected to the base of the microscope unit 312 and including an arm that rotatably supports the microscope unit 312; and a base unit 314 that rotatably holds the base of the support unit 313 and allows for movement on the floor. The base unit 314 includes a light source device 3 for generating white light, first narrowband light, and second narrowband light, etc., which are emitted from the microscope unit 310 toward the subject; and a control device 9 for controlling the operation of the surgical microscope system 300. The light source device 3 and the control device 9 each have at least the same configuration as that of the first embodiment described above. Specifically, the light source device 3 includes a condenser lens 30, a first light source unit 31, a second light source unit 32, and a light source control unit 33. Furthermore, the control device 9 includes an image / picture converter 91, an image processor 92, an input unit 93, a recording unit 94, and a control unit 95. The base portion 314 may be configured to be fixed to a ceiling, a wall, or the like to support the support portion 313 instead of being movably provided on the floor.

[0189] The microscope section 312 is, for example, cylindrical in shape and has the above-mentioned medical imaging device inside. Specifically, the medical imaging device has the same structure as the endoscopic camera head 5 involved in the above-mentioned embodiment 1. For example, the microscope section 312 includes an optical system 51, a drive section 52, an imaging element 53, a cutoff filter 54, an A / D conversion section 55, a P / S conversion section 56, an imaging recording section 57, and an imaging control section 58. In addition, a switch for receiving input of an operation instruction of the microscope device 310 is provided on the side surface of the microscope section 312. A cover glass (not shown) is provided on the opening surface of the lower end portion of the microscope section 312 to protect the interior.

[0190] In the surgical microscope system 300 thus configured, the operator or other user can operate various switches while holding the microscope unit 312 to move the microscope unit 312, perform zoom operations, or switch the illumination light. Furthermore, the microscope unit 312 is preferably shaped elongated in the observation direction to facilitate changing the field of view while holding the microscope unit. Therefore, the microscope unit 312 may also have a shape other than a cylinder, such as a polygonal prism.

[0191] In the third embodiment described above, the surgical microscope system 300 also employs a configuration similar to the first embodiment to detect the presence of mist within the viewing angle based on white light images captured at different times, and to provide notification when mist is detected. According to this third embodiment, the operator is notified of the presence of mist, enabling appropriate detection of thermally denatured areas even when mist is present.

[0192] (Other embodiments)

[0193] By appropriately combining the multiple components disclosed in the endoscope systems according to Embodiments 1 and 2 of the present disclosure or the surgical microscope system according to Embodiment 3, various inventions can be formed. For example, some components may be deleted from all the components described in the endoscope systems or surgical microscope systems according to the embodiments of the present disclosure. Furthermore, the components described in the endoscope systems or surgical microscope systems according to the embodiments of the present disclosure may be appropriately combined.

[0194] In addition, in the embodiments and variations, the processing examples are described on the premise that the first white light image, the second white light image, and the fluorescent image are images from the same perspective. However, when using images with different perspectives and the same subject captured in a portion, known methods such as pattern matching are used to correspond the segmented areas, perform detection of the mist generation area, and determine the overlap of the fluorescent images.

[0195] In the endoscope system or surgical microscope system according to the embodiment of the present disclosure, the above-mentioned "unit" can be replaced by "unit", "circuit", etc. For example, the control unit can be replaced by a control unit or a control circuit.

[0196] In the descriptions of the flowcharts in this specification, although expressions such as "first," "afterwards," and "next" are used to clarify the sequential relationship between the processes in the steps, the order of the processes required to implement the present invention is not solely defined by these expressions. In other words, the order of the processes in the flowcharts described in this specification can be changed within a range consistent with the order in which they are performed.

[0197] In addition, the programs executed by the devices involved in embodiments 1 to 3 are provided in the form of file data in an installable or executable form recorded on a computer-readable recording medium such as a CD-ROM, floppy disk (FD), CD-R, DVD (Digital Versatile Disk), USB medium, flash memory, etc.

[0198] Furthermore, the program for causing the apparatuses according to Embodiments 1 to 3 to execute may be stored on a computer connected to a network such as the Internet and provided by downloading via the network. Furthermore, the program for causing the information processing apparatuses according to Embodiments 1 to 3 to execute may be provided or distributed via a network such as the Internet.

[0199] In addition, in embodiments 1 and 2, an example in which the light source device 3 and the control device 9 are different entities is described, but a structure in which the light source device 3 and the control device 9 are integrated can also be provided. In addition, in embodiment 3, an example in which the light source device 3 and the control device 9 are integrated is described, but a structure in which the light source device 3 and the control device 9 are different entities can also be provided.

[0200] While several embodiments of the present application have been described in detail above based on the drawings, these are merely examples and the present invention can be implemented in other ways that are variously modified and improved based on the knowledge of those skilled in the art, represented by the methods described in the columns of this disclosure.

[0201] Industrial applicability

[0202] As described above, the medical device, medical system, medical device operating method, and medical device operating program according to the present invention are useful for appropriately detecting a thermally denatured region even when mist is generated.

[0203] Description of Reference Numerals

[0204] 1. 1A: Endoscope system; 2. Insertion unit; 3. Light source device; 4. Light guide; 5. Endoscope camera head; 6. First transmission cable; 7. Display device; 8. Second transmission cable; 9. 9A: Control device; 10. Third transmission cable; 21. Eyepiece unit; 22. Optical system; 23. Illumination optical system; 30. Converging lens; 31. First light source unit; 32. Second light source unit; 33. Light source control unit; 51. Optical system; 52. Drive unit; 53. Image sensor; 54. Cutoff filter; 55. A / D converter; 56. P / S converter; 57. Video recording unit; 58. Video control unit; 61. Video connector; 62. Camera head connector; 91. S / P converter; 92. Image processing unit; 93. Input unit; 94. Recording unit. 95: Control unit; 102: Endoscope; 121: Insertion unit; 122: Operation unit; 123: Universal cable; 124: Front end portion; 125: Bending portion; 126: Flexible tube portion; 127: Connector portion; 127a: Coil cable; 128: Connector portion; 241: Light guide; 242: Illumination lens; 243: Camera device; 244: Optical system; 300: Surgical microscope system; 310: Microscope device; 312: Microscope portion; 313: Support portion; 314: Base portion; 511: Lens; 531: Pixel portion; 532: Color filter; 921: Generation unit; 922: Segmentation unit; 923: Calculation unit; 924: Detection unit; 925: Output unit; 926: Extraction unit; 927: Correction unit; 941: Program recording unit.

Claims

1. A medical device comprising: a generating unit configured to generate a first white-light image, an image including a treatment object subjected to cauterization, irradiated with white light, a second white-light image captured later than the first white-light image, and a fluorescence image generated by excitation light that excites advanced glycation end products generated by the cauterization; a detection unit configured to detect mist based on the first white-light image and the second white-light image; and The control unit executes a notification process for notifying that the mist has been generated when the mist is detected by the detection unit.

2. The medical device according to claim 1, wherein The detection unit detects the mist based on a mist evaluation value calculated from a signal value of an image.

3. The medical device according to claim 2, wherein: The mist evaluation value includes a first evaluation value calculated based on the brightness value of the white light image, a second evaluation value calculated based on the chroma value of the white light image, and a third evaluation value calculated based on the contrast value of the white light image. The detection unit detects the mist by comparing the mist evaluation value of the first white-light image with the mist evaluation value of the second white-light image.

4. The medical device according to claim 3, wherein When the first evaluation value of the second white light image is greater than the first evaluation value of the first white light image, the second evaluation value of the second white light image is less than the second evaluation value of the first white light image, and the third evaluation value of the second white light image is less than the third evaluation value of the first white light image, the detection unit detects the presence of the mist in the second white light image.

5. The medical device according to claim 1, wherein further comprising a dividing unit that divides the first white-light image and the second white-light image into a plurality of divided areas, The detection unit detects the mist for each of the divided areas.

6. The medical device according to claim 5, wherein The apparatus further includes an extraction unit configured to extract an overlapping region between the fluorescent image and the mist in the fluorescent image when the mist is detected.

7. The medical device according to claim 6, wherein: The control unit notifies the user when the overlapping region where the fluorescent image in the fluorescent image overlaps with the mist is extracted.

8. The medical device according to claim 6, wherein The method further includes a correction unit configured to correct the fluorescence amount in the overlapping region in the fluorescence image.

9. The medical device according to claim 8, wherein The detection unit detects the mist based on a mist evaluation value calculated from a signal value of an image. The correction unit corrects the fluorescence amount by referring to a table in which the mist evaluation value and the correction information of the fluorescence amount are associated with each other.

10. The medical device according to claim 8, wherein The overlapping area is composed of one or more segmented areas.

11. The medical device according to claim 1, wherein The method further includes an extraction unit configured to extract a thermally denatured region in the fluorescent image.

12. The medical device according to claim 11, wherein further comprising a dividing unit that divides the first white-light image and the second white-light image into a plurality of divided areas, The detection unit detects the mist in each of the divided areas. The medical device further includes an output unit configured to output a display image showing the thermally denatured region overlapping the mist and the thermally denatured region not overlapping the mist in different forms.

13. A medical device comprising: a detection unit configured to detect mist based on a first white-light image and a second white-light image, wherein the first white-light image is an image obtained by irradiation with white light and the second white-light image is an image captured later than the first white-light image; an extraction unit that extracts thermal denaturation information from a fluorescence image based on fluorescence generated by excitation light that excites advanced glycation end products generated by burning; as well as A notification unit notifies the thermal denaturation information based on the detection result of the mist by the detection unit.

14. A medical system comprising: A medical imaging device for imaging the interior of a lumen of a subject; a light source device capable of emitting white light and excitation light for exciting advanced glycation end products produced by heat treatment of biological tissue; as well as a control device, wherein the medical imaging device is detachably mounted on and off the control device; The control device has: a generating unit configured to generate a first white-light image, an image including a treatment object subjected to cauterization, irradiated with white light, a second white-light image captured later than the first white-light image, and a fluorescence image generated by excitation light that excites advanced glycation end products generated by the cauterization; a detection unit configured to detect mist based on the first white light image and the second white light image; as well as The control unit executes a notification process for notifying that the mist has been generated when the mist is detected by the detection unit.

15. A method for operating a medical device, performed by the medical device, the method comprising the following steps: a first white-light image generating step of generating a first white-light image, the first white-light image being an image including the treatment object to be cauterized by irradiation with white light; a second white-light image generating step of generating a second white-light image, the second white-light image being an image captured later than the first white-light image; a fluorescent image generating step of generating a fluorescent image based on fluorescence generated by excitation light that excites advanced glycation end products generated by burning; a detecting step of detecting mist based on the first white light image and the second white light image; as well as A notifying step of notifying that the mist is generated when the mist is detected in the detecting step.

16. A medical device operating program, executed by the medical device, wherein the operating program causes the following steps to be performed: a first white-light image generating step of generating a first white-light image, the first white-light image being an image including the treatment object to be cauterized by irradiation with white light; a second white-light image generating step of generating a second white-light image, the second white-light image being an image captured later than the first white-light image; a fluorescent image generating step of generating a fluorescent image based on fluorescence generated by excitation light that excites advanced glycation end products generated by burning; a detecting step of detecting mist based on the first white light image and the second white light image; as well as A notifying step of notifying that the mist is generated when the mist is detected in the detecting step.

Images