Medical observation system and medical camera device

Abstract

Provided is a medical observation system and a medical camera device capable of narrow-band light observation and fluorescence observation using one imaging element. The medical observation system includes a light source device (3) capable of irradiating at least one of first narrow-band light and second narrow-band light that excites an advanced glycation end product generated by performing thermal treatment on living tissue; an imaging element (53) having a pixel portion (531) and a color filter (532) and capable of generating image data by capturing at least one of return light from the living tissue and fluorescence from the advanced glycation end product; and a cut filter (54) disposed on the light-receiving surface side of a pixel provided with a green filter, which blocks light on the short-wavelength side of a wavelength band including the second narrow-band light and transmits the first narrow-band light.

Description

Technical Field

[0001] This disclosure relates to a medical observation system and a medical imaging device that generate image data of a subject by photographing it, such as an examination subject. Background Technology

[0002] Conventionally, techniques known in endoscopes involve using two imaging elements, switching between irradiating a first narrowband light and a second narrowband light of different wavelengths depending on the observation mode, and having either imaging element capture an image according to the observation mode (see, for example, Patent Document 1). In this technique, during narrowband light observation, the first imaging element captures an image by irradiating the subject with the first narrowband light. The first imaging element has a light-shielding filter on its light-receiving surface that blocks reflected light from the subject at a predetermined shading rate. Furthermore, in Patent Document 1, during first autofluorescence observation, the first imaging element captures the first autofluorescence emitted from the subject by irradiating it with the first narrowband light as excitation light via the light-shielding filter. In the second autofluorescence observation mode, the second imaging element captures the second autofluorescence emitted from the subject by irradiating it with the second narrowband light as excitation light.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent No. 5371946 Summary of the Invention

[0006] The problem that the invention aims to solve

[0007] However, in the aforementioned Patent Document 1, narrowband light observation and fluorescence observation are performed by using two imaging elements: a second imaging element and a first imaging element with a light-shielding filter on the light-receiving surface. Therefore, a technology that can perform narrowband light observation and fluorescence observation using only one imaging element is desired.

[0008] This disclosure was made in view of the above circumstances, and its object is to provide a medical observation system and a medical imaging device capable of performing narrowband light observation and fluorescence observation using a single imaging element.

[0009] Methods for solving problems

[0010] To address the aforementioned issues and achieve the objectives, the medical observation system disclosed herein comprises: a light source device capable of irradiating living tissue with at least one of a first narrowband light with a wavelength narrower than white light and a second narrowband light with a wavelength shorter than the first narrowband light, wherein the second narrowband light excites advanced glycation end products (AGEs) generated by heat treatment of the living tissue; an imaging element having: a pixel portion having a plurality of pixels arranged in a 2D matrix; and a color filter configured such that any one of a red filter, a green filter, and a blue filter is provided on the light-receiving surface of each of the plurality of pixels, wherein the imaging element is capable of generating image data by capturing at least one of reflected light from the living tissue and fluorescence from the AGEs; and a cutoff filter, at least provided on the light-receiving surface side of the pixel provided with the green filter, which blocks light containing the short wavelength side of the second narrowband light while allowing the first narrowband light to pass through.

[0011] Furthermore, in the medical observation system disclosed herein, the medical observation system further includes an image processing unit that performs image processing on the image data and outputs it to a display device. When the living tissue is irradiated with the first narrowband light and the second narrowband light using the light source device, the image processing unit generates a narrowband light image based on the blue component signal from the pixel configured with the blue filter and the green component signal from the pixel configured with the green filter contained in the image data. On the other hand, when the advanced glycosylation end products are irradiated with only the second narrowband light using the light source device, the image processing unit generates a heat treatment image based on the blue component signal from the pixel configured with the blue filter and the green component signal from the pixel configured with the green filter contained in the image data.

[0012] Furthermore, in the medical observation system disclosed above, when the advanced glycation end products are irradiated only with the second narrowband light using the light source device, the image processing unit makes the gain of the blue component signal less than the gain of the green component signal.

[0013] Furthermore, in the medical observation system disclosed above, when the advanced glycation end products are irradiated only with the second narrowband light using the light source device, the image processing unit adjusts the gain of the blue component signal and the green component signal to keep the ratio of the blue component signal to the green component signal fixed.

[0014] Furthermore, in the medical observation system disclosed above, the light source device is also capable of irradiating white light. When the white light is irradiated onto the living tissue using the light source device, the image processing unit adjusts the white balance to fix the ratio of the values ​​of the red component signal, green component signal, and blue component signal contained in the image data, thereby generating a white image.

[0015] In addition, in the above disclosure, the fluorescence wavelength of the medical observation system is 500nm to 640nm.

[0016] In addition, in the medical observation system disclosed above, the first narrowband light has a wavelength range of 530nm to 550nm, the second narrowband light has a wavelength range of 390nm to 430nm, and the cutoff filter blocks light with a wavelength shorter than 430nm.

[0017] Furthermore, in the aforementioned disclosure, the medical observation system generates the advanced glycation end products by thermal treatment using an energy device.

[0018] In addition, the medical observation system disclosed herein further comprises: an insertion part capable of being inserted into a subject, having an optical system for focusing the reflected light and the fluorescence; and a medical imaging device, wherein the insertion part is detachable from the medical imaging device, and the medical imaging device includes the imaging element and the cutoff filter.

[0019] In addition, the medical observation system disclosed herein further comprises: an endoscope having an insertion portion capable of being inserted into the subject body and having a front end; and a medical imaging device disposed at the front end, the medical imaging device having an imaging element and a cutoff filter.

[0020] In addition, the medical observation system disclosed herein further comprises: a medical imaging device; a support portion that supports the medical imaging device to be rotatable; and a base portion that is movable on the ground and holds the base end of the support portion to be rotatable, wherein the medical imaging device includes the imaging element and the cutoff filter.

[0021] Furthermore, the medical observation system disclosed herein includes a narrowband light observation mode and a heat treatment observation mode. The medical observation system comprises: a light source device capable of illuminating living tissue with two types of blue light: a first type of blue light used in the narrowband light observation mode, which is blue light with high absorbance from hemoglobin in blood and easily reflected by mucosal surfaces; and a second type of blue light used in the heat treatment observation mode, which is blue light that excites advanced glycation end products (AGEs) generated through heat treatment of living tissue; and an imaging element that, in the narrowband light observation mode and the... The imaging element is applicable to any observation mode during the heat treatment observation mode. It has: a pixel section having a plurality of pixels arranged in a 2D matrix; a color filter configured such that any one of a red filter, a green filter, and a blue filter is provided on the light-receiving surface of each of the plurality of pixels, and the imaging element is capable of generating image data by capturing at least one of the reflected light from the living tissue and the fluorescence from the advanced glycation end products; and a cutoff filter, which is provided at least on the light-receiving surface side of the pixel provided with the green filter, to block light in the wavelength band including the fluorescence, while allowing the blue light to pass through.

[0022] Furthermore, in the medical observation system disclosed herein, the blue light is generated from a single light source unit as described above.

[0023] Furthermore, in the medical observation system disclosed herein, the medical observation system further includes an image processing unit that processes the image data and outputs it to a display device. When the living tissue is irradiated with blue light using the light source device in the narrowband light observation mode, the image processing unit generates a narrowband light image based on the blue component signal from the pixel configured with the blue filter contained in the image data. On the other hand, when the advanced glycosylation end products are irradiated with only blue light using the light source device in the heat treatment observation mode, the image processing unit generates a heat treatment image based on the blue component signal from the pixel configured with the blue filter and the green component signal from the pixel configured with the green filter contained in the image data.

[0024] Furthermore, the medical imaging device disclosed herein includes: an imaging element having: a pixel portion having a plurality of pixels arranged in a two-dimensional matrix; a color filter configured such that any one of a red filter, a green filter, and a blue filter is disposed on the light-receiving surface of each of the plurality of pixels; and a cutoff filter disposed at least on the light-receiving surface side of the pixel disposed of the green filter. The imaging element generates image data by capturing at least one of reflected light and fluorescence, wherein the reflected light is irradiated onto living tissue. The fluorescence is the return light from the living tissue under a first narrowband light with a wavelength narrower than that of white light, and the fluorescence is the fluorescence from the advanced glycation end products (AGEs) produced by heat treatment of the living tissue when irradiated with a second narrowband light, wherein the second narrowband light is a narrowband light on the short wavelength side of the wavelength band shorter than that of the first narrowband light, which excites the AGEs. The cutoff filter blocks the light on the short wavelength side of the wavelength band containing the second narrowband light, while allowing the first narrowband light to pass through.

[0025] Furthermore, the medical imaging device disclosed herein includes an imaging element having: a pixel portion having a plurality of pixels arranged in a 2D matrix; and a color filter configured such that any one of a red filter, a green filter, and a blue filter is provided on the light-receiving surface of each of the plurality of pixels. The imaging element generates image data by capturing at least one of reflected light and fluorescence, wherein the reflected light is reflected light from the living tissue when the living tissue is irradiated with a first narrowband light with a wavelength narrower than white light, and the fluorescence is fluorescence from the advanced glycation end products (AGEs) produced by heat treatment of the living tissue when the living tissue is irradiated with a second narrowband light, wherein the second narrowband light is a narrowband light on the short wavelength side of the wavelength shorter than the first narrowband light, which excites the AGEs, and the green filter blocks light on the short wavelength side of the wavelength band containing the second narrowband light, thereby allowing the first narrowband light to pass through.

[0026] Invention Effects

[0027] According to this disclosure, it enables narrowband light observation and fluorescence observation using a single imaging element. Attached Figure Description

[0028] Figure 1 This is a diagram showing the schematic structure of the endoscope system according to Embodiment 1.

[0029] Figure 2 This is a block diagram showing the functional structure of the main parts of the endoscope system according to Embodiment 1.

[0030] Figure 3 This is a diagram schematically showing the wavelength characteristics of the light emitted by the second and third light source units of Embodiment 1.

[0031] Figure 4 This is a diagram schematically showing the structure of the pixel portion in Embodiment 1.

[0032] Figure 5 This is a diagram schematically showing the structure of the color filter according to Embodiment 1.

[0033] Figure 6 This is a schematic diagram showing the sensitivity and wavelength of each filter in Embodiment 1.

[0034] Figure 7A This is a diagram schematically showing the signal values ​​of the G pixels of the camera element in Embodiment 1.

[0035] Figure 7B This is a diagram schematically showing the signal values ​​of the R pixels of the imaging element in Embodiment 1.

[0036] Figure 7C This is a diagram schematically showing the signal value of the B pixel of the imaging element in Embodiment 1.

[0037] Figure 8 This is a diagram schematically showing the structure of the cutoff filter of Embodiment 1.

[0038] Figure 9 This is a diagram schematically illustrating the transmission characteristics of the cutoff filter of Embodiment 1.

[0039] Figure 10 This is a diagram schematically illustrating the observation principle in the narrowband light observation mode of Embodiment 1.

[0040] Figure 11 This is a diagram schematically illustrating the observation principle during the heat treatment observation mode of Embodiment 1.

[0041] Figure 12 This is a diagram schematically illustrating the observation principle in the autofluorescence observation mode of Embodiment 1.

[0042] Figure 13 This is a diagram schematically illustrating the observation principle in the normal light observation mode of Embodiment 1.

[0043] Figure 14 This is a flowchart illustrating a summary of the processes performed by the endoscope system of Embodiment 1.

[0044] Figure 15 It is shown Figure 14 A flowchart outlining the narrowband light observation mode processing.

[0045] Figure 16 It is shown Figure 14 A flowchart outlining the process of heat treatment observation mode.

[0046] Figure 17 It is shown Figure 14 A flowchart outlining the autofluorescence observation mode.

[0047] Figure 18 It is shown Figure 14 A flowchart outlining the typical light observation mode.

[0048] Figure 19 This is a flowchart illustrating a summary of the processes performed by the endoscope system of Embodiment 2.

[0049] Figure 20 It is shown Figure 19 A flowchart outlining the process of video recording and processing.

[0050] Figure 21 It is shown Figure 19 A flowchart outlining the display process.

[0051] Figure 22 This is a diagram showing an example of an image displayed by the display device according to Embodiment 2.

[0052] Figure 23 This is a diagram showing an example of an image displayed by the display device according to Embodiment 2.

[0053] Figure 24 This is a diagram showing another example of an image displayed by the display device 7 according to Embodiment 2.

[0054] Figure 25A This is a diagram showing another example of an image displayed by the display device according to Embodiment 2.

[0055] Figure 25B This is a diagram showing another example of an image displayed by the display device according to Embodiment 2.

[0056] Figure 25C This is a diagram showing another example of an image displayed by the display device according to Embodiment 2.

[0057] Figure 26 This is a diagram showing an example of an image displayed by the display device according to Embodiment 2.

[0058] Figure 27 This is a graph showing the relationship between fluorescence intensity and the depth of heat treatment.

[0059] Figure 28 This is a diagram showing another example of an image displayed by the display device according to Embodiment 2.

[0060] Figure 29 This is a diagram showing the schematic structure of the endoscope system according to Embodiment 3.

[0061] Figure 30 This is a block diagram showing the functional structure of the main parts of the endoscope system according to Embodiment 3.

[0062] Figure 31 This is a diagram showing the schematic structure of the surgical microscope system according to Embodiment 4.

[0063] Figure 32 This is a diagram schematically showing the structure of the cutoff filter of Modified Example 1 of Embodiments 1 to 4.

[0064] Figure 33A The diagram schematically illustrates the manufacturing method of the cutoff filter of Modified Example 1 of Embodiments 1 to 4.

[0065] Figure 33B The diagram schematically illustrates the manufacturing method of the cutoff filter of Modified Example 1 of Embodiments 1 to 4.

[0066] Figure 34 This is a diagram schematically showing the transmission characteristics of filter G of the color filter in variant example 2 of embodiments 1 to 4.

[0067] Figure 35 This is a diagram schematically showing the structure of the cutoff filter of variant 3 of embodiments 1 to 4. Detailed Implementation

[0068] The following is related to the appendix. Figure 1 The following details the methods for implementing this disclosure. However, this disclosure is not limited to the embodiments described below. Furthermore, the figures referred to in the following description are merely schematic representations of shapes, sizes, and positional relationships to the extent that the content of this disclosure can be understood. That is, this disclosure is not limited to the shapes, sizes, and positional relationships illustrated in the figures. Moreover, in the description of the figures, the same reference numerals are used to describe the same parts. Furthermore, as an example of a medical observation system of this disclosure, an endoscope system equipped with a rigid endoscope and a medical imaging device will be described.

[0069] (Implementation Method 1)

[0070] [Structure of an endoscopic system]

[0071] Figure 1 This is a diagram showing the schematic structure of the endoscope system according to Embodiment 1. Figure 1 The endoscope system 1 shown is used in the medical field and is a system for observing living tissues inside a living body or other specimen. Furthermore, in Embodiment 1, as endoscope system 1, a device using… Figure 1 The rigid endoscope system (insertion part 2) shown is described herein, but the system is not limited to this; for example, an endoscope system with a flexible endoscope could also be used. Furthermore, as endoscope system 1, an endoscope system equipped with a medical imaging device for photographing the subject can be used, and surgery or procedures can be performed while displaying an image based on the image data captured by the medical imaging device on a display device. Figure 1 The endoscope system 1 shown is used when performing surgery or treatment on a subject using a treatment instrument (not shown) such as an electrosurgical scalpel or energy device capable of heat treatment.

[0072] Figure 1 The endoscope system 1 shown includes an insertion part 2, a light source device 3, a light guide 4, an endoscope camera 5 (an endoscope camera device), a first transmission cable 6, a display device 7, a second transmission cable 8, a control device 9, and a third transmission cable 10.

[0073] The insertion part 2 has a rigid or at least partially flexible elongated shape. The insertion part 2 is inserted into the body of a patient or other subject through a cannula. The insertion part 2 is internally provided with an optical system such as a lens for forming an observation image.

[0074] The light source device 3 is connected to one end of the light guide 4, and under the control of the control device 9, provides illumination light to one end of the light guide 4 to illuminate the subject. The light source device 3 is implemented using the following components: any one or more of the following light sources: LED (Light Emitting Diode), xenon lamp, and LD (laser diode) semiconductor laser elements; a processor serving as a processing device with hardware such as FPGA (Field Programmable Gate Array) and CPU (Central Processing Unit); and a memory serving as temporary storage for the processor. Furthermore, the light source device 3 and the control device 9 can be configured as follows: Figure 1 It can be configured to communicate independently as shown, or it can be configured as an integrated unit.

[0075] One end of the light guide 4 is detachably connected to the light source device 3, and the other end is detachably connected to the insertion part 2. The light guide 4 guides the illumination light provided by the light source device 3 from one end to the other end, thereby providing it to the insertion part 2.

[0076] The endoscope camera 5 is detachably connected to the eyepiece 21 of the insertion part 2. Under the control of the control device 9, the endoscope camera 5 receives the observation image formed by the insertion part 2 and performs photoelectric conversion to generate image data (RAW data), and outputs the image data to the control device 9 via the first transmission cable 6.

[0077] One end of the first transmission cable 6 is detachably connected to the control device 9 via a video connector 61, and the other end of the first transmission cable 6 is detachably connected to the endoscope camera 5 via a camera connector 62. The first transmission cable 6 transmits image data output from the endoscope camera 5 to the control device 9, and also transmits setting data and power, etc., output from the control device 9 to the endoscope camera 5. Here, the setting data includes control signals, synchronization signals, clock signals, etc., that control the endoscope camera 5.

[0078] Under the control of the control device 9, the display device 7 displays images based on image data processed in the control device 9, as well as various information related to the endoscope system 1. The display device 7 is implemented using a display monitor such as a liquid crystal or an organic EL (electroluminescence) display.

[0079] One end of the second transmission cable 8 is detachably connected to the display device 7, and the other end of the second transmission cable 8 is detachably connected to the control device 9. The second transmission cable 8 transmits the image data, which has undergone image processing in the control device 9, to the display device 7.

[0080] The control device 9 is implemented using a processor, which is a processing device equipped with hardware such as a GPU (Graphics Processing Unit), FPGA, or CPU, and a memory serving as a temporary storage area for the processor. The control device 9 controls the operation of the light source device 3, the endoscope camera 5, and the display device 7 in a unified manner, according to a program recorded in the memory, via the first transmission cable 6, the second transmission cable 8, and the third transmission cable 10. Furthermore, the control device 9 performs various image processing operations on the image data input via the first transmission cable 6 and outputs the data to the second transmission cable 8.

[0081] One end of the third transmission cable 10 is detachably connected to the light source device 3, and the other end of the third transmission cable 10 is detachably connected to the control device 9. The third transmission cable 10 transmits control data from the control device 9 to the light source device 3.

[0082] [Functional structure of the main components of an endoscope system]

[0083] Next, the functional structure of the main parts of the above-mentioned endoscope system 1 will be described. Figure 2 This is a block diagram showing the functional structure of the main parts of the endoscope system 1.

[0084] [Structure of the insertion part]

[0085] First, the structure of the insertion part 2 will be described. The insertion part 2 has an optical system 22 and an illumination optical system 23.

[0086] The optical system 22 focuses light from the subject, including reflected light, return light, excitation light, and light emitted by the subject, thereby forming an image of the subject. The optical system 22 is implemented using one or more lenses.

[0087] The illumination optical system 23 illuminates the subject with illumination light provided from the light guide 4. The illumination optical system 23 is implemented using one or more lenses, etc.

[0088] [Structure of the light source device]

[0089] Next, the structure of the light source device 3 will be described. The light source device 3 includes a condenser lens 30, a first light source unit 31, a second light source unit 32, a third light source unit 33, and a light source control unit 34.

[0090] The focusing lens 30 focuses the light emitted by the first light source 31, the second light source 32 and the third light source 33 and directs it out to the light guide 4.

[0091] Under the control of the light source control unit 34, the first light source unit 31 provides white light (normal light) as illumination light to the light guide 4 by emitting white light, which is visible light. The first light source unit 31 is constructed using a collimating lens, a white LED lamp, and a driver, etc. Alternatively, the first light source unit 31 can also provide visible light, i.e., white light, by simultaneously emitting red LED lamps, green LED lamps, and blue LED lamps. Of course, the first light source unit 31 can also be constructed using halogen lamps, xenon lamps, etc.

[0092] Under the control of the light source control unit 34, the second light source unit 32 emits a first narrowband light with a predetermined wavelength, thereby providing the first narrowband light as illumination light to the light guide 4. Here, the wavelength of the first narrowband light is 530nm to 550nm (center wavelength is 540nm). The second light source unit 32 is composed of a green LED lamp, a collimating lens, a transmission filter that transmits light from 530nm to 550nm, and a driver.

[0093] Under the control of the light source control unit 34, the third light source unit 33 provides the light guide 4 with second narrowband light as illumination by emitting second narrowband light with a wavelength different from that of the first narrowband light. Here, the wavelength of the second narrowband light is 400nm to 430nm (center wavelength is 415nm). The third light source unit 33 is implemented using a collimating lens, a semiconductor laser such as a violet LD (laser diode), and a driver.

[0094] The light source control unit 34 is implemented using a processor, which is a processing device with hardware such as an FPGA or CPU, and a memory, which is a temporary storage area used by the processor. Based on the control data input from the control device 9, the light source control unit 34 controls the light emission timing and light emission time of the first light source unit 31, the second light source unit 32, and the third light source unit 33.

[0095] Here, the wavelength characteristics of the light emitted by the second light source unit 32 and the third light source unit 33 will be explained. Figure 3 This is a diagram schematically showing the wavelength characteristics of the light emitted by the second light source unit 32 and the third light source unit 33, respectively. Figure 3 In the graph, the horizontal axis represents wavelength (nm), and the vertical axis represents wavelength characteristics. Additionally, in... Figure 3 In the middle, the broken line L NG The broken line L represents the wavelength characteristics of the first narrowband light emitted by the second light source unit 32. V This indicates the wavelength characteristics of the second narrowband light emitted by the third light source unit 33. Additionally, in Figure 3 In the middle, curve L B The blue band is represented by curve L. G The green band is represented by curve L. R The red band is indicated.

[0096] like Figure 3 The broken line L NG As shown, the second light source unit 32 emits a first narrowband light with a center wavelength (peak wavelength) of 540 nm and a wavelength range of 530 nm to 550 nm. In addition, the third light source unit 33 emits a second narrowband light with a center wavelength (peak wavelength) of 415 nm and a wavelength range of 400 nm to 430 nm.

[0097] In this way, the second light source unit 32 and the third light source unit 33 respectively emit first narrowband light and second narrowband light with different wavelengths.

[0098] [Structure of an endoscopic camera]

[0099] return Figure 2 The structure of endoscope system 1 will be further described.

[0100] Next, the structure of the endoscope camera 5 will be described. The endoscope camera 5 includes an optical system 51, a drive unit 52, an image sensor 53, a cutoff filter 54, an A / D converter 55, a P / S converter 56, an image recording unit 57, and an image control unit 58.

[0101] The optical system 51 images the subject image formed by the convergence of the optical system 22 of the insertion section 2 onto the light-receiving surface of the imaging element 53. The optical system 51 can change the focal length and focal point position. The optical system 51 is constructed using multiple lenses 511. The optical system 51 moves the multiple lenses 511 on the optical axis L1 respectively by the drive unit 52, thereby changing the focal length and focal point position.

[0102] Under the control of the camera control unit 58, the drive unit 52 moves the plurality of lenses 511 of the optical system 51 along the optical axis L1. The drive unit 52 is constructed using motors such as stepper motors, DC motors, and voice coil motors, and transmission mechanisms such as gears that transmit the rotation of the motors to the optical system 51.

[0103] The imaging element 53 is implemented using an image sensor, such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor), which has multiple pixels arranged in a 2D matrix. Under the control of the imaging control unit 58, the imaging element 53 receives the subject image (light) formed by the optical system 51 and passed through the cutoff filter 54, performs photoelectric conversion to generate image data (RAW data), and outputs it to the A / D conversion unit 55. The imaging element 53 has a pixel unit 531 and a color filter 532.

[0104] Figure 4 This is a schematic diagram illustrating the structure of the pixel section 531. (As shown...) Figure 4 As shown, the pixel unit 531 consists of multiple pixels P, such as photodiodes that store charges corresponding to the amount of light. nm (Integers n=1 or greater, integers m=1 or greater) are arranged in a 2D matrix. The pixel unit 531, under the control of the camera control unit 58, extracts data from multiple pixels P. nm The pixel P in the readout area is arbitrarily set as the readout object. nm The image signal is read out as image data and output to the A / D conversion unit 55.

[0105] Figure 5 This is a schematic diagram illustrating the structure of the color filter 532. (As shown...) Figure 5As shown, the color filter 532 is composed of Bayer arrays arranged in 2×2 units. The color filter 532 is composed of a filter R that transmits red light, two filters G that transmit green light, and a filter B that transmits blue light.

[0106] Figure 6 This is a schematic diagram showing the sensitivity and wavelength of each filter. Figure 6 In the graph, the horizontal axis represents wavelength (nm), and the vertical axis represents transmission characteristics (sensitivity characteristics). Furthermore, in... Figure 6 In the middle, curve L B Curve L represents the transmission characteristics of filter B. G Curve L represents the transmission characteristics of filter G. R This indicates the transmission characteristics of filter R.

[0107] like Figure 6 curve L B As shown, filter B allows light in the blue wavelength range to pass through. Additionally, as... Figure 6 curve L G As shown, filter G allows light in the green wavelength range to pass through. Furthermore, as... Figure 6 curve L R As shown, filter R allows light in the red band to pass through. Furthermore, the following will describe pixel P formed by placing filter R on the light-receiving surface. nm Pixel P, marked as R, is formed by placing a filter G on the light-receiving surface. nm Pixel P, labeled G, is formed by placing a filter B on the light-receiving surface. nm Let's use the B pixel as an example.

[0108] Based on the imaging element 53 configured in this way, when it receives the image of the subject formed by the optical system 51, such as Figures 7A to 7C As shown, color signals (R signal, G signal, and B signal) for R pixel, G pixel, and B pixel are generated respectively.

[0109] return Figure 2 The structure of endoscope system 1 will be further described.

[0110] A cutoff filter 54 is disposed on the optical axis L1 of the optical system 51 and the imaging element 53. The cutoff filter 54 is at least disposed on the light-receiving surface side (incident surface side) of the G pixel of the color filter 532, which transmits light in the green band. The cutoff filter 54 blocks light in the short wavelength band containing the second narrowband light and allows light in the band containing the first narrowband light that is longer than the second narrowband light to pass through.

[0111] Figure 8This is a schematic diagram illustrating the structure of the cutoff filter 54. (As shown...) Figure 8 As shown, the filter F that constitutes the cutoff filter 54 11 Configured with filter G 11 (Refer to Figure 5 ) at the position of filter G 11 It is formed on the side of the light-receiving surface directly above.

[0112] Figure 9 This is a diagram schematically illustrating the transmission characteristics of the cutoff filter 54. Figure 8 In the diagram, the horizontal axis represents wavelength (nm), and the vertical axis represents transmission characteristics. Additionally, in... Figure 8 In the middle, the broken line L F The broken line L represents the transmission characteristics of the cutoff filter 54. NG The broken line L represents the wavelength characteristics of the first narrowband light. V This indicates the wavelength characteristics of the second narrowband light.

[0113] like Figure 9 As shown, the cutoff filter 54 blocks the second narrowband light and allows light in the longer wavelength band to pass through, starting from the wavelength of the second narrowband light. Specifically, the cutoff filter 54 blocks light in the short wavelength band (400 nm and above, but less than 430 nm) that includes the wavelength of the second narrowband light, and allows light in the longer wavelength band (compared to 400 nm to 430 nm) that includes the second narrowband light to pass through.

[0114] return Figure 2 The structure of the endoscope camera 5 will be further explained.

[0115] Under the control of the camera control unit 58, the A / D conversion unit 55 performs A / D conversion processing on the analog image data input from the camera element 53 and outputs it to the P / S conversion unit 56. The A / D conversion unit 55 is implemented using A / D conversion circuits and the like.

[0116] Under the control of the camera control unit 58, the P / S conversion unit 56 performs parallel / serial conversion on the digital image data input from the A / D conversion unit 55, and outputs the image data that has undergone parallel / serial conversion to the control device 9 via the first transmission cable 6. The P / S conversion unit 56 is implemented using a P / S conversion circuit or the like. Alternatively, in Embodiment 1, an E / O conversion unit that converts image data into optical signals may be provided instead of the P / S conversion unit 56, outputting the image data to the control device 9 via optical signals. For example, the image data may also be transmitted to the control device 9 via wireless communication such as Wi-Fi (Wireless Fidelity).

[0117] The video recording unit 57 records various information related to the endoscope camera 5 (such as pixel information of the imaging element 53 and characteristics of the cutoff filter 54). Additionally, the video recording unit 57 records various setting data and control parameters transmitted from the control device 9 via the first transmission cable 6. The video recording unit 57 is constructed using non-volatile memory and volatile memory.

[0118] The camera control unit 58 controls the operation of the drive unit 52, the camera element 53, the A / D converter 55, and the P / S converter 56 based on the setting data received from the control device 9 via the first transmission cable 6. The camera control unit 58 is implemented using a TG (Timing Generator), a processor as a processing device with hardware such as a CPU, and a memory as a temporary storage area used by the processor.

[0119] [Structure of the control device]

[0120] Next, the structure of the control device 9 will be explained.

[0121] The control device 9 includes an S / P conversion unit 91, an image processing unit 92, an input unit 93, a recording unit 94, and a control unit 95.

[0122] Under the control of the control unit 95, the S / P converter 91 performs serial-to-parallel conversion on the image data received from the endoscope camera 5 via the first transmission cable 6 and outputs it to the image processing unit 92. Alternatively, when the endoscope camera 5 outputs image data via optical signals, an O / E converter that converts optical signals to electrical signals can be provided instead of the S / P converter 91. Furthermore, when the endoscope camera 5 transmits image data wirelessly, a communication module capable of receiving wireless signals can be provided instead of the S / P converter 91.

[0123] Under the control of the control unit 95, the image processing unit 92 performs prescribed image processing on the image data of the parallel data input from the S / P conversion unit 91 and outputs it to the display device 7. Here, the prescribed image processing includes depigmentation, white balance processing, gain adjustment processing, gamma correction processing, and format conversion processing. The image processing unit 92 is implemented using a processor as a processing device equipped with hardware such as a GPU or FPGA and a memory as a temporary storage area used by the processor.

[0124] The input unit 93 accepts inputs for various operations related to the endoscope system 1 and outputs the accepted operations to the control unit 95. The input unit 93 is composed of a mouse, foot switch, keyboard, buttons, switches, and touch panel.

[0125] The recording unit 94 is implemented using recording media such as volatile memory, non-volatile memory, SSD (Solid State Drive), HDD (Hard Disk Drive), or memory cards. The recording unit 94 records data including various parameters required for the operation of the endoscope system 1. In addition, the recording unit 94 has a program recording unit 941 that records various programs used to operate the endoscope system 1.

[0126] The control unit 95 is implemented using a processor, which is a processing device with hardware such as an FPGA or CPU, and a memory, which is a temporary storage area used by the processor. The control unit 95 provides unified control over all components constituting the endoscope system 1.

[0127] [Summary of each observation mode]

[0128] Next, a summary of the observation modes performed by the endoscope system 1 will be given. Furthermore, the following descriptions will proceed in the order of narrowband light observation mode, heat treatment observation mode, autofluorescence observation mode, and normal light observation mode.

[0129] [Overview of Narrowband Light Observation Mode]

[0130] First, the narrowband light observation mode will be explained. Figure 10 This is a diagram that schematically illustrates the observation principle in narrowband light observation mode.

[0131] Narrow Band Imaging (NBI) utilizes the fact that hemoglobin in blood strongly absorbs light around 415 nm to emphasize the capillaries and surface structures of mucosa in living tissues. Specifically, in NBI, the living tissue is irradiated with two narrow bands of light (530 nm–550 nm) and a second narrow band of light (390 nm–445 nm), both of which are easily absorbed by hemoglobin in the blood. This allows for the emphasizing and display of deep mucosal vessels and blood flow information that is difficult to discern with ordinary light (white light).

[0132] Specifically, such as Figure 10As shown in the curve G1, firstly, under the control of the control device 9, the light source device 3 causes the second light source section 32 and the third light source section 33 to emit light, thereby illuminating the first narrowband light W1 and the second narrowband light W2 onto the living tissue O1 (mucosa) of the subject. In this case, at least a portion of the reflected light containing multiple components and the returned light (hereinafter referred to as "reflected light WR1, WR2, WG1, WG2, WB1, WB2") reflected by the living tissue O1 such as the subject is blocked by the cutoff filter 54, and the remaining portion is incident on the imaging element 53. Furthermore, in the following description, the reflected light from the first narrowband light W1 will be referred to as reflected light WR1, reflected light WG1, and reflected light WB1, and the reflected light from the second narrowband light W2 will be referred to as reflected light WR2, reflected light WG2, and reflected light WB2. Figure 10 In this context, the thickness of the lines represents the intensity of the components (light quantity or signal value) of each line.

[0133] More specifically, such as Figure 10 The broken line L of the curve G2 F As shown, the cutoff filter 54 blocks the reflected light WG2 incident on the G pixel, that is, the reflected light WG2 in the short wavelength band that includes the second narrowband light W2.

[0134] Furthermore, the cutoff filter 54 allows the reflected light WG1, which includes the band of the first narrowband light W1 with a wavelength longer than that of the second narrowband light W2, to be transmitted. In addition, the reflected light (reflected light WR1, WR2, WB1, WB2) of the first narrowband light W1 and the second narrowband light W2 after being reflected by the subject are incident on the R pixel and the B pixel, respectively.

[0135] Next, as Figure 10 As shown in graph G3, the transmission characteristics (sensitivity characteristics) of the R, G, and B pixels are different. Specifically, the B pixel is not sensitive to the reflected light WB1 of the first narrowband light W1, so the output value corresponding to the amount of light received by the reflected light WB1 is a small value. On the other hand, the B pixel is sensitive to the reflected light WB2 of the second narrowband light W2, so the output value corresponding to the amount of light received by the reflected light WB1 is a large value.

[0136] Subsequently, the image processing unit 92 acquires image data (RAW data) from the imaging element 53 of the endoscope camera 5, and performs image processing on the signal values ​​of the G pixels and B pixels contained in the acquired image data to generate a pseudo-color image (narrowband image). In this case, the signal values ​​of the G pixels contain information about the deep mucosa of the subject. In addition, the signal values ​​of the B pixels contain information about the surface mucosa of the subject. Therefore, the image processing unit 92 performs image processing such as gain control processing, pixel interpolation processing, and mucosal emphasis processing on the signal values ​​of the G pixels and B pixels contained in the image data to generate a pseudo-color image, and outputs the pseudo-color image to the display device 7. Here, the pseudo-color image is an image generated using only the signal values ​​of the G pixels and B pixels. In addition, although the image processing unit 92 acquires the signal values ​​of the R pixels, it deletes them instead of using them to generate the pseudo-color image.

[0137] In this way, the narrowband light observation mode can highlight and display blood vessels and blood flow information deep in the mucosa that are difficult to see with white light (normal light).

[0138] [Overview of heat treatment observation mode]

[0139] Next, the observation mode for heat treatment will be explained. Figure 11 This is a diagram that schematically illustrates the observation principle during heat treatment observation mode.

[0140] In recent years, minimally invasive treatments using endoscopy and laparoscopy have been widely adopted in the medical field. For example, minimally invasive procedures using endoscopy and laparoscopy include endoscopic submucosal dissection (ESD), laparoscopic-endoscopic combined gastrectomy (LECS), and non-exposed endoscopic wall-inversion surgery.

[0141] In these low-invasive treatments, during procedures such as marking the surgical area, surgeons use energy devices like high-frequency scalpels and electrosurgical scalpels to perform thermal treatment or thermal marking of living tissue. Additionally, during actual procedures, surgeons also use energy devices for the removal and coagulation of living tissue from the patient.

[0142] In reality, surgeons rely on sight, touch, and intuition to determine the degree of heat treatment applied to living tissue using energy devices. Therefore, in treatments using conventional energy devices, surgeons often struggle to accurately determine the appropriate level of heat treatment during procedures, making it a highly skill-intensive process. Consequently, surgeons desire a technology that allows visualization of the burning state of the heat-treated area when heat treatment is applied to living tissue using energy devices.

[0143] In addition, when amino acids and reducing sugars are heated, a glycation reaction (Maillard reaction) occurs. The final products resulting from this Maillard reaction are collectively referred to as advanced glycation end products (AGEs). As a characteristic of AGEs, they are known to contain substances with fluorescent properties.

[0144] In other words, AGEs are generated when amino acids and reducing sugars in living tissue are heated and undergo the Maillard reaction during heat treatment using an energy device. AGEs generated through this heating process can be visualized through fluorescence observation. Furthermore, AGEs are known to emit stronger fluorescence than autofluorescent substances originally present in living tissue.

[0145] In other words, the heat treatment observation mode is an observation method that visualizes the heat-treated area by utilizing the fluorescence properties of AGEs generated in living tissue through heat treatment using energy devices or the like. Therefore, in the heat treatment observation mode, blue light with a wavelength around 415 nm, used to excite AGEs, is irradiated onto the living tissue from the light source device 3. Thus, the heat treatment observation mode can observe heat treatment images (fluorescence images) obtained by capturing images of the fluorescence generated from AGEs (e.g., green light with wavelengths of 490–625 nm).

[0146] Specifically, such as Figure 11 As shown in curve G11, firstly, under the control of the control device 9, the light source device 3 causes the third light source section 33 to emit light, thereby irradiating the living tissue O2 (heat-treated area) that has undergone heat treatment of the subject by energy equipment, etc., with the second narrowband light W2 as the excitation light (center wavelength 415nm). In this case, as Figure 11 As shown in curve G12, at least the components of the second narrowband light W2 reflected by the living tissue O2 (heat treatment area) and the reflected light containing the return light (hereinafter referred to as "reflected light WR10, reflected light WG10, reflected light WB10") are blocked by the cutoff filter 54, and a portion of the long-wavelength components are incident on the imaging element 53. Additionally, in Figure 11In this context, the thickness of the lines represents the intensity of the components (light quantity or signal value) of each line.

[0147] More specifically, such as Figure 11 As shown in curve G12, the cutoff filter 54 blocks the reflected light WG10 incident on pixel G, specifically the short-wavelength reflected light WG10 that includes the second narrowband light W2. Furthermore, as... Figure 11 As shown in graph G12, the cutoff filter 54 allows the fluorescence (WF1) emitted by the AGEs in the living tissue O2 (heat-treated area) to pass through. Therefore, the reflected light (reflected light WR10, reflected light WB10) and the fluorescence (WF1) are incident on the R pixel and B pixel, respectively. Additionally, the fluorescence (WF1) is incident on the G pixel. Thus, since the G pixel has the cutoff filter 54 positioned on the light-receiving side (incident side), it is possible to prevent the fluorescence component from being buried in the reflected light WG10 of the second narrowband light W2, which serves as the excitation light.

[0148] In addition, such as Figure 11 The broken line L of the fluorescence properties in the curve G12 NG As shown, although the G pixel is sensitive to fluorescence, the output value is small because fluorescence is a tiny reaction.

[0149] Subsequently, the image processing unit 92 acquires image data (RAW data) from the imaging element 53 of the endoscope camera 5, and performs image processing on the signal values ​​of the G pixels and B pixels contained in the acquired image data to generate a pseudo-color image (heat-treated fluorescence image). In this case, the signal value of the G pixels contains fluorescence information emitted from the heat-treated area. In addition, the signal value of the B pixels contains background information of the living tissue surrounding the heat-treated area. Therefore, the image processing unit 92 performs image processing such as gain control processing, pixel interpolation processing, and mucosal emphasis processing on the signal values ​​of the G pixels and B pixels contained in the image data to generate a pseudo-color image, and outputs the pseudo-color image (heat-treated image) to the display device 7. In this case, the image processing unit 92 performs gain control processing such that the gain for the signal value of the G pixels is greater than the gain for the signal value of the G pixels under normal light observation, while the gain for the signal value of the B pixels is less than the gain for the signal value of the B pixels under normal light observation. Furthermore, the image processing unit 92 performs gain control processing to make the signal values ​​of the G pixels and B pixels the same (1:1).

[0150] In this way, the heat treatment observation mode can easily observe the O2 (heat treatment area) of living tissues subjected to heat treatment, such as energy equipment.

[0151] [Overview of Autofluorescence Observation Mode]

[0152] Next, the autofluorescence observation mode will be explained. Figure 12 This is a diagram schematically illustrating the observation principle in autofluorescence observation mode.

[0153] Autofluorescence Imaging (AFI) is an observation method that easily distinguishes between normal tissue and diseased tissues such as tumors by exciting fluorescent substances such as collagen in the submucosa of living tissue. In AFI, blue light at approximately 415 nm is alternately irradiated as the excitation light for the autofluorescent substances, and green light at approximately 540 nm is used as the reference light reflected from the mucosal surface of the living tissue. Then, in AFI, the imaging element 53 captures images of the fluorescent components emitted by the fluorescent substances within the living tissue and the reflected light from the reference light returning from normal living tissue, displaying a pseudo-color image that distinguishes between normal and diseased tissues.

[0154] Specifically, such as Figure 12 As shown in curve G21, firstly, under the control of the control device 9, the light source device 3 causes the second light source section 32 and the third light source section 33 to emit light alternately, so that the first narrowband light W1 (center wavelength 540nm) serving as reference light and the second narrowband light W2 (center wavelength 415nm) serving as excitation light sequentially (alternatingly) irradiate (interchangeably irradiate) the living tissue O3 of the subject. In this case, at least a portion of the reflected light and return light (hereinafter referred to as "reflected light WR20, reflected light WG20, reflected light WB20"), which includes multiple components reflected by the subject, is blocked by the cutoff filter 54, and the remaining portion is incident on the imaging element 53.

[0155] More specifically, such as Figure 12 As shown in curve G22, the cutoff filter 54 blocks the reflected light WG20 incident on pixel G, specifically the short-wavelength reflected light WG20 that includes the band of the second narrowband light W2. Specifically, in autofluorescence observation mode, when the second narrowband light W2 is irradiated, fluorescence WF10 (center wavelength 540nm) is incident on pixel G. Furthermore, in autofluorescence observation mode, when the second narrowband light W2 is irradiated, fluorescence WF10 generated from fluorescent substances within living tissue and reflected light WB20 of the second narrowband light W2 reflected by O3 in living tissue are incident on pixel B, and fluorescence WF10 generated from fluorescent substances within living tissue and reflected light WR20 of the second narrowband light W2 reflected by O3 in living tissue are incident on pixel R.

[0156] Furthermore, in autofluorescence observation mode, when the first narrowband light W1 is irradiated, the reflected light WG30 of the first narrowband light W1 (reference light) reflected by the living tissue O3 is incident on pixel G. Also, in autofluorescence observation mode, when the first narrowband light W1 is irradiated, the reflected light WB30 ​​of the first narrowband light W1 (reference light) reflected by the living tissue O3 is incident on pixel B, and the reflected light WR30 of the first narrowband light W1 (reference light) reflected by the living tissue O3 is incident on pixel R. Additionally, in Figure 12 In this context, the thickness of the lines represents the intensity of the components (light quantity or signal value) of each line.

[0157] Subsequently, the image processing unit 92 acquires image data (RAW data) from the imaging element 53 of the endoscope camera 5, and generates a pseudo-color image (autofluorescence image) by performing image processing on the signal values ​​of the G pixels contained in the acquired image data. In this case, the signal values ​​of the G pixels include: fluorescence information emitted from the fluorescent substance in the living tissue when irradiated by the second narrowband light W2 (essential component 1); and background information of the reference reflected light (essential component 2) including the reflected light and return light of the first narrowband light W1 after being reflected from the living tissue when irradiated by the first narrowband light W1 (reference light). At this time, regarding the reference reflected light from the living tissue irradiated by the first narrowband light W1 (reference light), compared with normal tissue and areas with thickened superficial mucosa, the amount of light in areas with blood vessels or inflammation is smaller. Therefore, the image processing unit 92 performs emphasis processing, so that areas with weak fluorescence information (essential component 1) from the fluorescent substance in the living tissue and strong background information (essential component 2) of the reference reflected light from the living tissue are further emphasized. Specifically, the image processing unit 92 generates a pseudo-color image to display the region estimated to be a tumor in magenta. For example, the image processing unit 92 assigns the hue of the fluorescence information (essential component 1) of the region emitted by fluorescent substances within the living tissue to the blue and red areas of the pseudo-color image, and assigns the hue of the background information (essential component 2) of the reference reflected light from the living tissue to the green area of ​​the pseudo-color image. Thus, the region estimated to be a tumor is represented in magenta, while normal mucosa, blood vessels, or areas with inflammation are represented in green. Furthermore, the image processing unit 92 removes the signal values ​​of the B and R pixels contained in the image data without using them.

[0158] In contrast, as mentioned above Figure 12 As shown, in autofluorescence observation mode, the image processing unit 92 performs gain control processing to make the gain of the signal value for the G pixel greater than the gain of the signal value for the G pixel during normal light observation.

[0159] In this way, the autofluorescence observation mode can observe lesion areas (abnormal areas) such as tumors and normal areas by observing the autofluorescence from living tissue, thereby highlighting them with different hues.

[0160] [Overview of typical light observation modes]

[0161] Next, the typical light observation mode will be explained. Figure 13 It is a diagram that schematically illustrates the observation principle in the normal light observation mode.

[0162] like Figure 13 As shown, firstly, under the control of the control device 9, the light source device 3 causes the first light source section 31 to emit light, thereby irradiating white light W3 onto the living tissue O4 of the subject. In this case, a portion of the reflected light and return light reflected by the living tissue (hereinafter referred to as "reflected light WR40, reflected light WG40, reflected light WB40") is blocked by the cutoff filter 54, and the remaining portion is incident on the imaging element 53. Specifically, as Figure 13 As shown, the cutoff filter 54 blocks the reflected light (WG40) incident on the G pixel, specifically the reflected light in the short wavelength band containing the second narrowband light W2. Therefore, as... Figure 13 As shown, compared to the state where the cutoff filter 54 is not configured, the blue band light incident on the G pixel has a smaller component.

[0163] Next, the image processing unit 92 acquires image data (RAW data) from the imaging element 53 of the endoscope camera 5, performs image processing on the signal values ​​of the R pixels, G pixels, and B pixels contained in the acquired image data, and generates a white light image. In this case, since the blue component contained in the image data is smaller than that observed in previous white light, the image processing unit 92 performs white balance adjustment processing to fix the ratio of the red, green, and blue components.

[0164] Thus, even with a cutoff filter 54 configured on the light-receiving side of the G pixel, a natural white image can be observed in normal light viewing mode.

[0165] [Management of the endoscopic system]

[0166] Next, the processing performed by the endoscope system 1 will be explained. Figure 14 This is a flowchart showing an overview of the processes performed by the endoscope system 1. Furthermore, the image processing unit 92 performs various image processing steps for developing image data; however, for the sake of simplicity, only characteristic image processing steps for each observation mode will be described below.

[0167] like Figure 14As shown, firstly, the control unit 95 determines whether the endoscope system 1 is set to narrowband light observation mode (step S1). If the control unit 95 determines that the endoscope system 1 is set to narrowband light observation mode (step S1: Yes), the endoscope system 1 proceeds to step S2, which will be described later. Conversely, if the control unit 95 determines that the endoscope system 1 is not set to narrowband light observation mode (step S1: No), the endoscope system 1 proceeds to step S4, which will be described later.

[0168] In step S2, the endoscope system 1 performs narrowband light observation mode processing. After step S2, the endoscope system 1 proceeds to step S3, which will be described later.

[0169] [Narrowband light observation mode processing]

[0170] Figure 15 This is to show the above. Figure 14 A flowchart outlining the narrowband light observation mode processing in step S2.

[0171] like Figure 15 As shown, the control unit 95 controls the light source control unit 34, and by causing the second light source unit 32 and the third light source unit 33 to emit light, the first narrowband light and the second narrowband light are irradiated toward the subject (step S11).

[0172] Next, the control unit 95 controls the camera control unit 58 so that the camera element 53 can capture the subject image obtained by the convergence of the optical system 22 and the optical system 51 after passing through the cutoff filter 54 (step S12).

[0173] Then, the control unit 95 causes the image processing unit 92 to perform gain control processing on the image data input via the A / D conversion unit 55, the P / S conversion unit 56, and the S / P conversion unit 91 (step S13).

[0174] Next, the control unit 95 causes the image processing unit 92 to perform de-mosaic processing on the image data after gain control processing (step S14), and causes the image processing unit 92 to perform high-quality processing on the image data after de-mosaic processing and generate a pseudo-color image (step S15).

[0175] Then, the control unit 95 causes the image processing unit 92 to output a pseudo-color image to the display device 7 (step S16). As a result, surgeons and other personnel can observe the subject while viewing the narrowband light image.

[0176] Next, the control unit 95 determines whether a switching signal for switching the observation mode of the endoscope system 1 has been input from the input unit 93 (step S17). If the control unit 95 determines that a switching signal for switching the observation mode of the endoscope system 1 has been input from the input unit 93 (step S17: Yes), the endoscope system 1 returns to normal operation. Figure 14 The main program. In contrast, if the control unit 95 determines that no switching signal for switching the observation mode of the endoscope system 1 has been input from the input unit 93 (step S17: No), the endoscope system 1 returns to the above-mentioned step S11.

[0177] return Figure 14 Continuing with the explanation after step S3.

[0178] In step S3, the control unit 95 determines whether an instruction signal to end the observation of the subject has been input from the input unit 93. If the control unit 95 determines that an instruction signal to end the observation of the subject has been input from the input unit 93 (step S3: Yes), the endoscope system 1 terminates this process. Conversely, if the control unit 95 determines that no instruction signal to end the observation of the subject has been input from the input unit 93 (step S3: No), the endoscope system 1 returns to step S1 as described above.

[0179] In step S4, the control unit 95 determines whether the endoscope system 1 is set to the heat treatment observation mode. If the control unit 95 determines that the endoscope system 1 is set to the heat treatment observation mode (step S4: Yes), the endoscope system 1 proceeds to step S5, which will be described later. Conversely, if the control unit 95 determines that the endoscope system 1 is not set to the heat treatment observation mode (step S4: No), the endoscope system 1 proceeds to step S6, which will be described later.

[0180] In step S5, the endoscope system 1 performs heat treatment observation mode processing. After step S5, the endoscope system 1 proceeds to step S3.

[0181] [Heat treatment observation mode processing]

[0182] Figure 16 This is to show the above. Figure 14 A flowchart outlining the heat treatment observation mode processing in step S5.

[0183] like Figure 16 As shown, the control unit 95 controls the light source control unit 34 to make the third light source unit 33 emit light, thereby irradiating the subject with the second narrow band light (step S51).

[0184] Next, the control unit 95 controls the camera control unit 58 so that the camera element 53 can capture the subject image obtained by the convergence of the optical system 22 and the optical system 51 after passing through the cutoff filter 54 (step S52).

[0185] Subsequently, the control unit 95 instructs the image processing unit 92 to perform gain control processing on the image data input via the A / D conversion unit 55, the P / S conversion unit 56, and the S / P conversion unit 91 (step S53). In this case, the image processing unit 92 performs gain control processing as follows: it makes the gain for the signal value of the G pixel included in the image data greater than the gain for the signal value of the G pixel under normal light observation, while making the gain corresponding to the signal value of the B pixel less than the gain for the signal value of the B pixel under normal light observation. Furthermore, the image processing unit 92 performs gain control processing to make the signal values ​​of the G pixel and the B pixel the same (1:1).

[0186] Next, the control unit 95 causes the image processing unit 92 to perform de-mosaic processing on the image data after gain control processing (step S54), and causes the image processing unit 92 to perform high-quality processing on the image data after de-mosaic processing and generate a pseudo-color image (thermal processed image) (step S55).

[0187] Then, the control unit 95 causes the image processing unit 92 to output a pseudo-color image to the display device 7 (step S56). As a result, surgical personnel such as doctors can observe the patient while viewing the heat treatment image.

[0188] Next, the control unit 95 determines whether a switching signal for switching the observation mode of the endoscope system 1 has been input from the input unit 93 (step S57). If the control unit 95 determines that a switching signal for switching the observation mode of the endoscope system 1 has been input from the input unit 93 (step S57: Yes), the endoscope system 1 returns to normal operation. Figure 14 The main program. In contrast, if the control unit 95 determines that no switching signal for switching the observation mode of the endoscope system 1 has been input from the input unit 93 (step S57: No), the endoscope system 1 returns to the above-mentioned step S51.

[0189] return Figure 14 Continue with the explanation following step S6.

[0190] In step S6, the control unit 95 determines whether the endoscope system 1 is set to autofluorescence observation mode. If the control unit 95 determines that the endoscope system 1 is set to autofluorescence observation mode (step S6: Yes), the endoscope system 1 proceeds to step S7, which will be described later. Conversely, if the control unit 95 determines that the endoscope system 1 is not set to autofluorescence observation mode (step S6: No), the endoscope system 1 proceeds to step S8, which will be described later.

[0191] In step S7, the endoscope system 1 performs autofluorescence observation mode processing. After step S7, the endoscope system 1 proceeds to step S3.

[0192] [Autofluorescence observation mode processing]

[0193] Figure 17 This is to show the above. Figure 14 A flowchart outlining the autofluorescence observation mode in step S7.

[0194] like Figure 17 As shown, the control unit 95 controls the light source control unit 34 to make the second light source unit 32 and the third light source unit 33 emit light respectively, thereby sequentially (alternatingly) irradiating the first narrowband light and the second narrowband light toward the subject (step S71).

[0195] Next, the control unit 95 controls the camera control unit 58 so that the camera element 53 can capture the subject image obtained by the convergence of the optical system 22 and the optical system 51 after passing through the cutoff filter 54 (step S72).

[0196] Then, the control unit 95 causes the image processing unit 92 to perform de-mosaic processing on the image data input via the A / D conversion unit 55, the P / S conversion unit 56, and the S / P conversion unit 91 (step S73).

[0197] Next, the control unit 95 instructs the image processing unit 92 to perform tone conversion processing on the de-mosaiced image data and generate a pseudo-color image (step S74). In this case, the signal value of the G pixel includes: fluorescence information emitted from fluorescent material within the living tissue when illuminated by the second narrowband light W2 (essential component 1); and background information of the reference reflected light (essential component 2) including the reflected light and return light of the first narrowband light W1 (reference light) reflected from the living tissue. At this time, regarding the reference reflected light from the living tissue irradiated by the first narrowband light W1 (reference light), the amount of light in areas with blood vessels or inflammation is smaller compared to normal tissue and areas with thickened superficial mucosa. Therefore, the image processing unit 92 performs emphasis processing, further emphasizing areas where the fluorescence information (essential component 1) emitted from the fluorescent material within the living tissue is weak and the background information (essential component 2) of the reference reflected light from the living tissue is strong. Specifically, the image processing unit 92 generates a pseudo-color image to display the region estimated to be a tumor in magenta. For example, the image processing unit 92 assigns the hue of the fluorescence information (essential component 1) of the region emitted by fluorescent substances within the living tissue to the blue and red areas of the pseudo-color image, and assigns the hue of the background information (essential component 2) of the reference reflected light from the living tissue to the green area of ​​the pseudo-color image. Thus, the region estimated to be a tumor is represented in magenta, while normal mucosa or areas with blood vessels or inflammation are represented in green. Furthermore, the image processing unit 92 removes the signal values ​​of the B and R pixels contained in the image data without using them.

[0198] Then, the control unit 95 causes the image processing unit 92 to output a pseudo-color image to the display device 7 (step S75). As a result, surgeons and other personnel can observe abnormal areas of the subject, including tumors, while viewing the autofluorescence image.

[0199] Next, the control unit 95 determines whether a switching signal for switching the observation mode of the endoscope system 1 has been input from the input unit 93 (step S76). If the control unit 95 determines that a switching signal for switching the observation mode of the endoscope system 1 has been input from the input unit 93 (step S76: Yes), the endoscope system 1 returns to normal operation. Figure 14 The main program. In contrast, if the control unit 95 determines that no switching signal for switching the observation mode of the endoscope system 1 has been input from the input unit 93 (step S76: No), the endoscope system 1 returns to the above-mentioned step S71.

[0200] return Figure 14 Continue with the explanation following step S8.

[0201] In step S8, the control unit 95 determines whether the endoscope system 1 is set to normal light observation mode. If the control unit 95 determines that the endoscope system 1 is set to normal light observation mode (step S8: Yes), the endoscope system 1 proceeds to step S9, which will be described later. Conversely, if the control unit 95 determines that the endoscope system 1 is not set to normal light observation mode (step S8: No), the endoscope system 1 proceeds to step S3.

[0202] [Processed in standard light observation mode]

[0203] Figure 18 This is to show the above. Figure 14 A flowchart outlining the typical light observation mode in step S9.

[0204] like Figure 18 As shown, the control unit 95 controls the light source control unit 34 to make the first light source unit 31 emit light, thereby irradiating white light toward the subject (step S91).

[0205] Next, the control unit 95 controls the camera control unit 58 to cause the camera element 53 to capture the subject image obtained by the convergence of the optical system 22 and the optical system 51 after passing through the cutoff filter 54 (step S92).

[0206] Then, the control unit 95 causes the image processing unit 92 to perform de-mosaic processing on the image data input via the A / D conversion unit 55, the P / S conversion unit 56, and the S / P conversion unit 91 (step S93).

[0207] Next, the control unit 95 instructs the image processing unit 92 to perform white balance adjustment processing on the de-mosaiced image data and generate a white image (step S94). Specifically, since the blue component in the image data is smaller than that observed under white light, the image processing unit 92 performs white balance adjustment processing to adjust the white balance in a way that keeps the ratio of the red, green, and blue components fixed and generates a white image.

[0208] Then, the control unit 95 causes the image processing unit 92 to output a white image to the display device 7 (step S95). As a result, surgical personnel such as doctors can observe the subject while viewing the white image.

[0209] Next, the control unit 95 determines whether a switching signal for switching the observation mode of the endoscope system 1 has been input from the input unit 93 (step S96). If the control unit 95 determines that a switching signal for switching the observation mode of the endoscope system 1 has been input from the input unit 93 (step S96: Yes), the endoscope system 1 returns to normal operation. Figure 14The main program. In contrast, if the control unit 95 determines that no switching signal for switching the observation mode of the endoscope system 1 has been input from the input unit 93 (step S96: No), the endoscope system 1 returns to the above-mentioned step S71.

[0210] According to Embodiment 1 described above, a cutoff filter 54 is provided on the light-receiving surface side of the pixel on which the filter G is provided. The cutoff filter 54 blocks the light on the short wavelength side of the band containing the second narrowband light, while allowing the first narrowband light to pass through. Therefore, it is possible to observe the narrowband light using a single imaging element 53 and to observe the fluorescence generated by heat treatment using an energy device or the like.

[0211] Furthermore, according to Embodiment 1, in the narrowband light observation mode, the image processing unit 92 generates a narrowband light image (pseudo-color image) based on the blue component signal and the green component signal. On the other hand, in the heat treatment observation mode, it generates a heat treatment image (pseudo-color image) based on the blue component signal and the green component signal. Therefore, it is possible to perform narrowband light observation and fluorescence observation generated by heat treatment using an energy device or the like using a single imaging element 53.

[0212] Furthermore, according to Embodiment 1, when the light source device 3 only illuminates the advanced glycosylation end products with the second narrow band light, the image processing unit 92 makes the gain of the blue component signal less than the gain of the green component signal, so that the fluorescence contained in the heat-treated image can be emphasized from the background.

[0213] Furthermore, according to Embodiment 1, in the normal light observation mode, the image processing unit 92 adjusts the white balance to fix the ratio of the values ​​of the red component signal, green component signal and blue component signal contained in the image data and generates a white image. Therefore, it is possible to perform narrowband light observation, fluorescence observation generated by heat treatment using an energy device or the like, and normal light observation using a single imaging element 53.

[0214] (Implementation Method 2)

[0215] Next, Embodiment 2 will be described. The endoscope system of Embodiment 2 has the same structure as the endoscope system 1 of Embodiment 1 described above, but the processing performed is different. Specifically, in Embodiment 1, multiple observation modes are switched between, while in Embodiment 2, multiple observation modes are alternately performed to generate two image data sets with different characteristics. Surgeons, such as doctors, record these data sets manually, or, under predetermined conditions, switch and record the content of the image displayed on the display device. The processing performed by the endoscope system of Embodiment 2 will be described below. Furthermore, in Embodiment 2, the same reference numerals are used for structures identical to those in the endoscope system 1 of Embodiment 1, and detailed descriptions are omitted.

[0216] [Management of the endoscopic system]

[0217] Figure 19 This is a flowchart illustrating a summary of the processes performed by the endoscope system 1 according to Embodiment 2. Furthermore, in Figure 19 In order to simplify the explanation, the above-described normal optical observation and thermal treatment observation of the endoscope system 1 will be explained.

[0218] like Figure 19 As shown, firstly, the endoscope system 1 illuminates the living tissue of the subject with white light or a second narrow-band light, and performs image recording processing (step S101) to capture and record the reflected light, fluorescent light, etc. from the living tissue, and performs image display processing (step S102) to display an image based on the captured image data. The details of the image recording processing and display processing will be described later.

[0219] Next, the control unit 95 determines whether an instruction signal to end the observation of the subject has been input from the input unit 93 (step S103). If the control unit 95 determines that an instruction signal to end the observation of the subject has been input from the input unit 93 (step S103: Yes), the endoscope system 1 ends this process. Conversely, if the control unit 95 determines that no instruction signal to end the observation of the subject has been input from the input unit 93 (step S103: No), the endoscope system 1 returns to step S101 described above.

[0220] [Video Recording Processing]

[0221] Next, regarding the above... Figure 19 The details of the video recording process in step S101 will be explained. Figure 20 This is a flowchart illustrating an overview of the video recording process.

[0222] like Figure 20As shown, the control unit 95 controls the light source control unit 34 and illuminates the subject with white light by causing the first light source unit 31 to emit light (step S201).

[0223] Next, the control unit 95 controls the camera control unit 58 so that the camera element 53 can capture the subject image after the transmission cut-off filter 54, which is the subject image obtained by the convergence of the optical system 22 and the optical system 51 (step S202).

[0224] Then, the control unit 95 causes the image processing unit 92 to perform prescribed image processing on the image data input via the A / D conversion unit 55, the P / S conversion unit 56 and the S / P conversion unit 91 and generate a white image (step S203).

[0225] Next, the control unit 95 records the white image generated by the image processing unit 92 into the recording unit 94 (step S204).

[0226] Then, the control unit 95 controls the light source control unit 34 to make the third light source unit 33 emit light, thereby irradiating the second narrow band light toward the subject (step S205).

[0227] Next, the control unit 95 controls the camera control unit 58 so that the camera element 53 can capture the subject image after the transmission cut-off filter 54, which is the subject image obtained by the convergence of the optical system 22 and the optical system 51 (step S206).

[0228] Then, the control unit 95 causes the image processing unit 92 to perform prescribed image processing on the image data input via the A / D conversion unit 55, the P / S conversion unit 56, and the S / P conversion unit 91 and generate a heat treatment image (step S207).

[0229] Next, the control unit 95 determines whether a recording signal for recording a heat treatment image has been input from the input unit 93 (step S208). Specifically, when a surgeon or other personnel performs heat treatment on living tissue using an energy device or the like by operating the input unit 93, the control unit 95 determines whether a recording signal for recording the captured heat treatment image to the recording unit 94 has been input from the input unit 93. If the control unit 95 determines that a recording signal for recording a heat treatment image has been input from the input unit 93 (step S208: Yes), the endoscope system 1 proceeds to step S209, which will be described later. Conversely, if the control unit 95 determines that a recording signal for recording a heat treatment image has not been input from the input unit 93 (step S209: No), the endoscope system 1 proceeds to step S210, which will be described later.

[0230] In step S209, the control unit 95 records the thermal processing image generated by the image processing unit 92 onto the recording unit 94. After step S209, the endoscope system 1 returns to the above-described... Figure 19 The main program.

[0231] In step S210, the control unit 95 determines whether the specified conditions are met. Specifically, the control unit 95 determines whether the energy device or the like has started heat treatment based on the drive signal input from the energy device or the like. Furthermore, the control unit 95 determines whether the specified conditions are met based on the state of the heat-treated living tissue and the fluorescence intensity in the heat treatment image generated by the image processing unit 92. For example, if the fluorescence intensity is above a specified threshold, the control unit 95 determines that the specified conditions are met. Furthermore, the control unit 95 determines whether the specified conditions are met based on the area of ​​the fluorescent region of the heat-treated living tissue in the heat treatment image generated by the image processing unit 92. For example, the control unit 95 determines whether the area of ​​the fluorescent region in the heat treatment image is above a specified threshold. If the control unit 95 determines that the specified conditions are met (step S210: Yes), the endoscope system 1 proceeds to step S211, which will be described later. Conversely, if the control unit 95 determines that the specified conditions are not met (step S210: No), the endoscope system 1 returns to the above-described... Figure 19 The main program.

[0232] In step S211, the control unit 95 records the thermal processing image generated by the image processing unit 92 onto the recording unit 94. After step S211, the endoscope system 1 returns to the above-described... Figure 19 The main program.

[0233] [Display Processing]

[0234] Next, regarding the above... Figure 19 The summary of the display processing in step S102 will be explained. Figure 21 This is a flowchart showing an overview of the display process.

[0235] like Figure 21 As shown, the control unit 95 determines whether an instruction signal indicating the display of a white image and a heat treatment image has been input from the input unit 93 (step S301). If the control unit 95 determines that an instruction signal indicating the display of a white image and a heat treatment image has been input from the input unit 93 (step S301: Yes), the endoscope system 1 proceeds to step S302, which will be described later. Conversely, if the control unit 95 determines that no instruction signal indicating the display of a white image and a heat treatment image has been input from the input unit 93 (step S301: No), the endoscope system 1 proceeds to step S305, which will be described later.

[0236] In step S302, the control unit 95 causes the image processing unit 92 to output a white image and a heat treatment image, thereby causing the display device 7 to display the white image and the heat treatment image. Figure 22 This is a diagram showing an example of an image displayed by display device 7. For example... Figure 22 As shown, the control unit 95 causes the image processing unit 92 to display the composite image P1, obtained by combining a white image and a heat treatment image, on the display device 7. In this case, the image processing unit 92 performs the composite image with a 1:1 ratio between the white image and the heat treatment image. Of course, the image processing unit 92 can also appropriately change the composite ratio based on the instruction signal input from the input unit 93. Furthermore, the image processing unit 92 can also composite only the fluorescent regions in the heat treatment image, such as pixels with a signal value of the heat treatment image above a threshold, from the white image. Thus, by observing the composite image P1, which combines the white image and the heat treatment image, surgeons and other personnel can intuitively grasp the heat treatment area Z1, including the location where heat treatment is performed, such as energy devices. Of course, in Embodiment 2, an overlay image, in which the heat treatment image is superimposed on the white image, is also possible.

[0237] Next, the control unit 95 determines whether a switching signal for changing the display mode of the image displayed on the display device 7 has been input from the input unit 93 (step S303). If the control unit 95 determines that a switching signal for changing the display mode of the image displayed on the display device 7 has been input from the input unit 93 (step S303: Yes), the endoscope system 1 proceeds to step S304, which will be described later. Conversely, if the control unit 95 determines that no switching signal for changing the display mode of the image displayed on the display device 7 has been input from the input unit 93 (step S303: No), the endoscope system 1 returns to... Figure 19 The main program.

[0238] In step S304, the control unit 95 controls the display mode of the image displayed on the display device 7 by generating a white image and a heat treatment image corresponding to the switching signal input from the input unit 93 and outputting them to the display device 7. After step S304, the endoscope system 1 returns to normal operation. Figure 19 The main program.

[0239] Figure 23 This is a diagram showing an example of an image displayed by display device 7. For example... Figure 23 As shown, the control unit 95 causes the image processing unit 92 to generate a display image P2 in which the white image P10 and the heat treatment image P11 are side by side, and outputs it to the display device 7. Thus, surgical personnel such as doctors can intuitively grasp the heat treatment area Z1, including the location of the heat treatment equipment such as energy devices, by comparing and observing the white image P10 and the heat treatment image P11.

[0240] Figure 24 This is another example of an image displayed by display device 7. (See diagram below.) Figure 24 As shown, the control unit 95 can also cause the image processing unit 92 to generate a display image P3 and output it to the display device 7. This display image P3 places the white image P10 and the heat treatment image P11 side by side, and makes the display area of ​​the heat treatment image P11 smaller than the display area of ​​the white image P10. Therefore, surgical personnel such as doctors can intuitively grasp the heat treatment area Z1, including the location of the energy device undergoing heat treatment, by comparing and observing the white image P10 and the heat treatment image P11. Furthermore, the control unit 95 can also cause the image processing unit 92 to generate the display image P3 by changing the display ratio of the heat treatment image P11 and the white image P10 within the display image P3 based on the instruction signal from the input unit 93.

[0241] Figures 25A-25C This is another example of an image displayed by display device 7. (See diagram below.) Figures 25A-25C As shown, the control unit 95 can also control the image processing unit 92 to process the white image P10 according to the number of switching signals input from the input unit 93. Figure 25A ), display image P3 ( Figure 25B ), heat treatment image P11 ( Figure 25C The sequence of images is switched and output, allowing the display device 7 to display them. This enables surgeons and other personnel to observe the desired images through simple operation.

[0242] return Figure 21 Continue with the explanation following step S305.

[0243] In step S305, the control unit 95 determines whether a predetermined condition is met. Specifically, the control unit 95 determines whether the energy device or the like has started or stopped heat treatment based on the drive signal input from the energy device or the like. Furthermore, the control unit 95 determines whether a predetermined condition is met based on the state of the heat-treated living tissue and the amount of fluorescence emitted in the heat treatment image generated by the image processing unit 92. For example, if the amount of fluorescence emitted is above a predetermined threshold, the control unit 95 determines that a predetermined condition is met. The control unit 95 also determines whether a predetermined condition is met based on the area of ​​the fluorescent region of the heat-treated living tissue in the heat treatment image generated by the image processing unit 92. For example, the control unit 95 determines whether the area of ​​the fluorescent region in the heat treatment image is above a predetermined threshold. If the control unit 95 determines that a predetermined condition is met (step S305: Yes), the endoscope system 1 proceeds to step S306, which will be described later. Conversely, if the control unit 95 determines that a predetermined condition is not met (step S305: No), the endoscope system 1 proceeds to step S309, which will be described later.

[0244] In step S306, the control unit 95 generates a composite image by combining the white image and the heat treatment image and outputs it to the display device 7, whereby the display device 7 displays the composite image. After step S306, the endoscope system 1 returns to... Figure 19 The main program.

[0245] Figure 26 This is a diagram showing an example of an image displayed by display device 7. Figure 27 This is a graph showing the correspondence between fluorescence intensity and the depth of heat treatment. Figure 27 In the diagram, the vertical axis represents fluorescence intensity, and the horizontal axis represents the depth of heat treatment into living tissue. Additionally, in... Figure 27 In the figure, the straight line Ly represents the correlation between fluorescence intensity and the depth of heat treatment in living tissue.

[0246] like Figure 26 As shown, the control unit 95 causes the image processing unit 92 to combine a white image and a heat-treated image to obtain a composite image P20, which is then displayed on the display device 7. In this case, the image processing unit 92 generates the composite image P20 by combining the white image and the heat-treated image at a ratio of 1:1. Furthermore, as... Figure 26 and Figure 27 As shown, the image processing unit 92 generates a composite image P20 that emphasizes the color of the fluorescent region based on the amount of fluorescence emitted from the heat-treated region contained in the heat-treated image. For example, as... Figure 26 As shown, the image processing unit 92 uses blue to generate a heat-treated area Z2 with weak fluorescence, and uses green to generate a heat-treated area Z1 with stronger fluorescence than heat-treated area Z2. The heat-treated area Z2 with weak fluorescence represents an area marked using a treatment device such as an energy device before surgical personnel, such as doctors, use an electrosurgical scalpel to remove abnormal areas like tumors. Furthermore, as... Figure 28 As shown in the heat treatment image P21, the image processing unit 92 can also generate the heat treatment area Z3 using yellow based on the amount of fluorescence emission. Therefore, surgeons and other personnel can intuitively understand the state of the heat treatment based on the color.

[0247] In step S307, the control unit 95 determines whether an indication signal indicating that the image displayed by the display device 7 is a heat treatment image has been input from the input unit 93. If the control unit 95 determines that an indication signal indicating that the image displayed by the display device 7 is a heat treatment image has been input from the input unit 93 (step S307: Yes), the endoscope system 1 proceeds to step S308, which will be described later. Conversely, if the control unit 95 determines that no indication signal indicating that the image displayed by the display device 7 is a heat treatment image has been input from the input unit 93 (step S307: No), the endoscope system 1 proceeds to step S309, which will be described later.

[0248] In step S308, the control unit 95 instructs the image processing unit 92 to generate a heat treatment image and output it to the display device 7, thereby causing the display device 7 to display the heat treatment image. For example, the control unit 95 instructs the image processing unit 92 to generate the aforementioned image. Figure 25C The heat treatment image P11 is output to the display device 7. After step S308, the endoscope system 1 returns to... Figure 19 The main program.

[0249] In step S309, the control unit 95 instructs the image processing unit 92 to generate a white image and output it to the display device 7, thereby causing the display device 7 to display the white image. For example, the control unit 95 instructs the image processing unit 92 to generate the image described above. Figure 25A The white image P10 is output to the display device 7. After step S309, the endoscope system 1 returns to... Figure 19 The main program.

[0250] According to Embodiment 2 described above, a cutoff filter 54 is provided on the light-receiving surface side of the pixel on which the filter G is provided. The cutoff filter 54 blocks the light on the short wavelength side of the band containing the second narrowband light, while allowing the first narrowband light to pass through. Therefore, it is possible to observe narrowband light using a single imaging element 53 and to observe fluorescence generated by heat treatment using energy devices, etc.

[0251] (Implementation Method 3)

[0252] Next, Embodiment 3 will be described. Although Embodiments 1 and 2 described above are endoscope systems equipped with rigid endoscopes, Embodiment 3 will describe an endoscope system equipped with a flexible endoscope. Hereinafter, the endoscope system of Embodiment 3 will be described. Furthermore, in Embodiment 3, structures identical to those in the endoscope system 1 of Embodiment 1 will be labeled with the same reference numerals, and detailed descriptions will be omitted.

[0253] [Structure of an endoscopic system]

[0254] Figure 29 This is a diagram showing the schematic structure of the endoscope system according to Embodiment 3. Figure 30 This is a block diagram showing the functional structure of the main parts of the endoscope system according to Embodiment 3.

[0255] Figure 29 and Figure 30 The endoscopic system 100 shown is inserted into the body of a patient or other subject to take images of the body. The display device 7 displays images based on the acquired image data. By observing the images displayed on the display device 7, surgeons can check the presence and condition of bleeding sites, tumor sites, and abnormal areas in the examined area. Furthermore, surgeons can insert instruments such as energy devices into the body of the subject through the endoscope's instrument channel to perform procedures on the subject. In addition to the aforementioned light source device 3, display device 7, and control device 9, the endoscopic system 100 also includes an endoscope 102.

[0256] [Structure of an endoscope]

[0257] The structure of endoscope 102 will be described. Endoscope 102 generates image data by taking pictures inside the body of the subject and outputs the generated image data to control device 9. Endoscope 102 includes an operation unit 122 and a universal cable 123.

[0258] The insertion portion 121 has a flexible, elongated shape. The insertion portion 121 has: a front end portion 124, which houses the camera device described later; a flexible bending portion 125, which is composed of multiple bending blocks; and a long, flexible tube portion 126, which is connected to the base end of the bending portion 125 and is flexible.

[0259] The front end portion 124 is made of glass fiber or the like. The front end portion 124 has: a light guide 241 that forms a light guiding path for light provided from the light source device 3; an illumination lens 242 disposed at the front end of the light guide 241; and an imaging device 243.

[0260] The imaging device 243 includes an optical system 244 for focusing light, an imaging element 53 as described in Embodiment 1, a cutoff filter 54, an A / D converter 55, a P / S converter 56, an image recording unit 57, and an image control unit 58. Furthermore, in Embodiment 3, the imaging device 243 functions as a medical imaging device.

[0261] The universal cable 123 has at least a built-in light guide 241 and a focusing cable that combines one or more cables. The combining cable is a signal line that transmits and receives signals between the endoscope 102, the light source device 3, and the control device 9. It includes signal lines for transmitting and receiving setting data, signal lines for transmitting and receiving image data, and signal lines for transmitting and receiving timing signals for driving the imaging element 53. The universal cable 123 has a connector portion 127 that is detachable from the light source device 3. A coiled cable 127a extends from the connector portion 127, and a connector portion 128 that is detachable from the control device 9 is located at the extended end of the coiled cable 127a.

[0262] The endoscope system 100 configured in this way performs the same processing as the endoscope system 1 of Embodiment 1 described above.

[0263] According to Embodiment 3 described above, it has the same effect as Embodiment 1, and it is possible to perform narrow-band light observation and fluorescence observation by heat treatment using only one imaging element 53 and energy device, etc., so the diameter of the insertion part 121 can be reduced.

[0264] (Implementation Method 4)

[0265] Next, Embodiment 4 will be described. Although Embodiments 1 to 3 described above were endoscope systems, Embodiment 4 will describe a system suitable for a surgical microscope. Furthermore, in Embodiment 4, structures identical to the endoscope system 1 in Embodiment 1 will be labeled with the same reference numerals, and detailed descriptions will be omitted.

[0266] [Structure of a surgical microscope system]

[0267] Figure 31 This is a diagram showing the schematic structure of the surgical microscope system according to Embodiment 4. Figure 31 The surgical microscope system 300 shown includes a display device 17 and a microscope device 310, which is a medical imaging device for observing a subject by capturing images. Alternatively, the display device 17 and the microscope device 310 may be integrated into one unit.

[0268] The microscope apparatus 310 includes: a microscope section 312 for magnifying and photographing minute parts of a subject; a support section 313 connected to the base end of the microscope section 312 and including an arm that supports the microscope section 312 for rotation; and a base section 314 that holds the base end of the support section 313 for rotation and is movable on the ground. The base section 314 includes: a light source device 3 that generates white light, a first narrowband light, and a second narrowband light, etc., illuminating the subject from the microscope apparatus 310; and a control device 9 that controls the operation of the surgical microscope system 300. Furthermore, the light source device 3 and the control device 9 each have at least the same structure as in Embodiment 1 described above. Specifically, the light source device 3 includes a condenser lens 30, a first light source section 31, a second light source section 32, a third light source section 33, and a light source control section 34. Additionally, the control device 9 includes an S / P conversion section 91, an image processing section 92, an input section 93, a recording section 94, and a control section 95. The base portion 314 may not be installed on the ground in a movable manner, but may be configured to be fixed to the ceiling, wall, etc. to support the support portion 313.

[0269] The microscope section 312 is, for example, cylindrical, and houses the aforementioned medical imaging device. Specifically, the medical imaging device has the same structure as the endoscope camera 5 of Embodiment 1 described above. For example, the microscope section 312 includes an optical system 51, a drive unit 52, an imaging element 53, a cutoff filter 54, an A / D converter 55, a P / S converter 56, an image recording unit 57, and an image control unit 58. Furthermore, a switch for receiving operation instructions from the microscope device 310 is provided on the side of the microscope section 312. A glass cover (not shown) protecting the interior is provided at the opening at the lower end of the microscope section 312.

[0270] In this configuration, the surgical microscope system 300 allows the user, such as a surgeon, to operate various switches while manipulating the microscope unit 312, enabling movement, zooming, and switching of the illumination light. Furthermore, the shape of the microscope unit 312 is preferably elongated in the observation direction to facilitate easy handling and changing of the field of view. Therefore, the shape of the microscope unit 312 can also be other than cylindrical, for example, it could be a polygonal prism.

[0271] According to Embodiment 4 described above, the same effect as Embodiment 1 can be obtained in the surgical microscope system 300, and the microscope section 312 can also be miniaturized.

[0272] (Modification 1 of Embodiments 1-4)

[0273] Next, a variation of Embodiments 1 to 4 will be described. In Variation 1 of Embodiments 1 to 4, only the structure of the cutoff filter differs. Hereinafter, the structure of the cutoff filter in Variation 1 of Embodiments 1 to 4 will be described.

[0274] Figure 32 This is a diagram schematically showing the structure of the cutoff filter of Modified Example 1 of Embodiments 1 to 4. Figure 32 The cutoff filter 54A shown is at least disposed on the light-receiving surface side (incident surface side) of the G pixel of the color filter 532, where the filter G is disposed, and on the light-receiving surface side (incident surface side) of the R pixel, where the filter R is disposed. It blocks light in the short-wavelength band containing the second narrowband light, while allowing light in the longer-wavelength band containing the first narrowband light compared to the second narrowband light to pass through. Specifically, as... Figure 32 As shown, filter F constitutes the cutoff filter 54A. 11 Configured with filter G 11 (Refer to Figure 5 The position of the filter G is configured in the filter G. 11 The light-receiving side is directly above it. Furthermore, the filter F... 21 Configured with filter R 21 (Refer to Figure 5 The position of the filter R is configured in the filter R. 21 The side of the surface directly above the light-receiving surface.

[0275] [Manufacturing method of cut-off filter]

[0276] Next, the manufacturing method of the cut-off filter 54A will be described. Figure 33A and Figure 33B This is a schematic diagram illustrating the manufacturing method of the cutoff filter 54A.

[0277] like Figure 33A As shown, firstly, a coating apparatus (not shown) that performs the coating of the light-shielding film coats the light-shielding film in a column corresponding to the column formed by arranging G pixels and R pixels in the vertical direction of the cut-off filter 54A. Figure 33A Next, the coating apparatus applies the light-shielding film to a column in the horizontal direction of the cut-off filter 54A that corresponds to the column formed by the arrangement of G pixels and R pixels. Figure 33B ).

[0278] According to the modified example 1 of embodiments 1 to 4 described above, the cutoff filter 54A can be easily connected.

[0279] Furthermore, although in the modified example 1 of embodiments 1 to 4 described above, two coating processes were performed in the vertical and horizontal directions, for example, it is also possible to manufacture the product by applying a mask to the part corresponding to the B pixel, coating the entire surface with a light-shielding film, and then removing the mask.

[0280] (Modification 2 of Embodiments 1-4)

[0281] Next, variations 2 of embodiments 1 to 4 will be described. In variations 2 of embodiments 1 to 4, the cutoff filter 54 of embodiment 1 is omitted, and the transmission characteristics of the filter G of the color filter are different. Hereinafter, the structure of the color filter in variations 2 of embodiments 1 to 4 will be described. Furthermore, in variations 2 of embodiments 1 to 4, the same reference numerals are used for structures identical to those in the endoscope system 1 of embodiment 1, and detailed descriptions are omitted.

[0282] Figure 34 This is a diagram schematically illustrating the transmission characteristics of filter G in the color filter of variant example 2 of embodiments 1-4. Figure 34 In the diagram, the horizontal axis represents wavelength, and the vertical axis represents transmission characteristics. Figure 34 In the middle, curve L G10 This indicates the transmission characteristics of filter G.

[0283] like Figure 34 curve L G10 As shown, filter G blocks light in the short-wavelength band compared to 415nm. That is, filter G blocks light in the short-wavelength band that includes the second narrowband light, allowing light in the longer-wavelength band compared to the second narrowband light that includes the first narrowband light to pass through.

[0284] According to the modified example 2 of embodiments 1 to 4 described above, the cutoff filter 54 can be omitted, thus allowing for a simpler structure.

[0285] (Modification 3 of Embodiments 1-4)

[0286] Next, variations 3 of embodiments 1 to 4 will be described. In variations 3 of embodiments 1 to 4, the structure is different from the cutoff filter 54 of embodiment 1 described above. Hereinafter, the structure of the cutoff filter in variations 3 of embodiments 1 to 4 will be described. In addition, in variations 3 of embodiments 1 to 4, structures that are the same as those in the endoscope system 1 of embodiment 1 described above will be labeled with the same reference numerals and detailed descriptions will be omitted.

[0287] Figure 35 This is a diagram schematically showing the structure of the cutoff filter of variant 3 of embodiments 1 to 4. Figure 35The cutoff filter 54C shown is disk-shaped and includes: a transmission section 541 that allows light of all wavelengths to pass through; and a transmission section 542 that blocks light of shorter wavelengths that include the second narrowband light, while allowing light of longer wavelengths that include the first narrowband light compared to the second narrowband light to pass through. The cutoff filter 54C is rotated about the optical axis L1 by a drive unit such as a motor (not shown).

[0288] According to the variation 3 of the above-described embodiments 1 to 4, it achieves the same effect as the above-described embodiments 1 to 4.

[0289] Furthermore, although in Variation 3 of Embodiments 1 to 4, the wavelength of light incident on the imaging element 53 is limited by rotating the cutoff filter 54C, an electronic filter or the like that can block light of a specified wavelength based on the current value may be used instead of the cutoff filter 54C.

[0290] (Other implementation methods)

[0291] Various inventions can be formed by appropriately combining the multiple structural elements disclosed in the medical observation systems of embodiments 1 to 4 of this disclosure. For example, some structural elements may be deleted from all the structural elements described in the medical observation systems of the embodiments of this disclosure. Furthermore, the structural elements described in the medical observation systems of the embodiments of this disclosure may be appropriately combined.

[0292] Furthermore, in the medical observation systems of embodiments 1 to 4 of this disclosure, the term "section" can be replaced with "unit," "circuit," or the like. For example, the control section can be replaced with a control unit or a control circuit.

[0293] Furthermore, although expressions such as "firstly," "afterwards," and "next" are used in the flowchart descriptions of this specification to explicitly indicate the sequential relationship between steps, the order of processes required to implement this invention is not uniquely limited by these expressions. That is, the order of processes in the flowcharts described in this specification can be changed without contradiction.

[0294] The above description of several embodiments of this application is based on the accompanying drawings. However, these are illustrative, and the invention can be implemented in various modifications and variations based on the knowledge of those skilled in the art, starting from the manner described in this disclosure.

[0295] Label Explanation

[0296] 1. 100: Endoscopic system;

[0297] 2: Insertion section;

[0298] 3: Light source device;

[0299] 4: Optical guide;

[0300] 5: Endoscopic camera;

[0301] 6: First transmission cable;

[0302] 7: Display device;

[0303] 8: Second transmission cable;

[0304] 9: Control device;

[0305] 10: Third transmission cable;

[0306] 21: Eyepiece section;

[0307] 22: Optical system;

[0308] 23: Illumination optical system;

[0309] 30: Condensing lens;

[0310] 31: First light source section;

[0311] 32: Second light source section;

[0312] 33: Third Light Source Section;

[0313] 34: Light source control unit;

[0314] 51: Optical system;

[0315] 52: Drive unit;

[0316] 53: Camera components;

[0317] 54, 54A, 54C: Cut-off filters;

[0318] 55: A / D conversion unit;

[0319] 56: P / S converter;

[0320] 57: Video Recording Department;

[0321] 58: Camera Control Unit;

[0322] 61: Video connector;

[0323] 62: Camera connector;

[0324] 91: S / P conversion unit;

[0325] 92: Image Processing Unit;

[0326] 93: Input section;

[0327] 94: Records Department;

[0328] 95: Control Department;

[0329] 102: Endoscope;

[0330] 121: Insertion section;

[0331] 122: Operations Department;

[0332] 123: General purpose cable;

[0333] 124: Front end;

[0334] 125: Curved section;

[0335] 126: Flexible tube section;

[0336] 127: Connector section;

[0337] 127a: Coil cable;

[0338] 128: Connector section;

[0339] 241: Optical guide;

[0340] 242: Illumination lens;

[0341] 243: Camera device;

[0342] 244: Optical system;

[0343] 300: Surgical microscope system;

[0344] 310: Microscope apparatus;

[0345] 312: Microscope section;

[0346] 313: Support part;

[0347] 314: Base section;

[0348] 511: Lens;

[0349] 531: Pixel section;

[0350] 532: Color filter;

[0351] 541, 542: Transmitting section;

[0352] 941: Program Recording Department.

Claims

1. A medical observation system comprising: a light source device capable of irradiating at least one of first narrow-band light having a wavelength band narrower than that of white light and second narrow-band light having a shorter wavelength than the first narrow-band light, the second narrow-band light exciting an advanced glycation end product generated by performing a thermal treatment on a living tissue; an imaging element having a pixel section including a plurality of pixels arranged in a two-dimensional matrix and a color filter configured to have any one of a red filter, a green filter, and a blue filter provided on a light-receiving surface of each of the plurality of pixels, the imaging element being capable of generating image data by capturing at least one of returning light from the living tissue and fluorescent light from the advanced glycation end product; and a cut filter provided at least on the light-receiving surface side of the pixel provided with the green filter, the cut filter blocking light having a shorter wavelength than the wavelength band of the second narrow-band light and transmitting the first narrow-band light, wherein the advanced glycation end product is generated by performing the thermal treatment using an energy device.

2. The medical observation system according to claim 1, wherein the medical observation system further comprises an image processing section that performs image processing on the image data and outputs the image data to a display device, wherein in a case where the first narrow-band light and the second narrow-band light are irradiated on the living tissue using the light source device, the image processing section generates a narrow-band light image from a blue component signal from the pixel provided with the blue filter and a green component signal from the pixel provided with the green filter included in the image data, and wherein in a case where only the second narrow-band light is irradiated on the advanced glycation end product using the light source device, the image processing section generates a thermal treatment image from the blue component signal from the pixel provided with the blue filter and the green component signal from the pixel provided with the green filter included in the image data.

3. The medical observation system according to claim 2, wherein in a case where only the second narrow-band light is irradiated on the advanced glycation end product using the light source device, the image processing section sets a gain of the blue component signal to be smaller than a gain of the green component signal.

4. The medical observation system according to claim 3, wherein in a case where only the second narrow-band light is irradiated on the advanced glycation end product using the light source device, the image processing section adjusts the gains of the blue component signal and the green component signal so as to fix a ratio of the blue component signal to the green component signal.

5. The medical observation system according to claim 4, wherein the light source device is further capable of irradiating white light. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ In a case where the living tissue is irradiated with the white light by the light source device, the image processing section adjusts a white balance so that a ratio of values of a red component signal, a green component signal, and a blue component signal included in the image data is fixed, thereby generating a white image.

6. The medical observation system according to claim 1, wherein The band of the fluorescence is 500 nm to 640 nm.

7. The medical observation system according to claim 1, wherein The band of the first narrow-band light is 530 nm to 550 nm, the band of the second narrow-band light is 390 nm to 430 nm, and the cut filter blocks light on a short wavelength side shorter than 430 nm.

8. The medical observation system according to claim 1, wherein The medical observation system further includes: an insertion section that is insertable into a subject, and has an optical system that condenses the return light and the fluorescence; and a medical camera device, the insertion section is detachable with respect to the medical camera device, the medical camera device includes the imaging element and the cut filter.

9. The medical observation system according to claim 1, wherein The medical observation system further includes: a medical camera device, the insertion section is detachable with respect to the medical camera device, the medical camera device includes the imaging element and the cut filter.

10. The medical observation system according to claim 1, wherein The medical observation system further includes: a medical camera device; a support section that supports the medical camera device so as to be rotatable; and a base section that is movable on a floor, and holds a base end section of the support section so as to be rotatable, the medical camera device includes the imaging element and the cut filter.

11. A medical observation system that includes a narrow-band light observation mode and a thermal treatment observation mode, wherein The medical observation system includes: a light source device that is capable of illuminating a living tissue with two kinds of blue light, a first kind of the blue light being blue light that illuminates a living tissue at the time of the narrow-band light observation mode, and being blue light in which hemoglobin in blood has a high absorbance and is easily reflected on a mucous membrane surface layer, and a second kind of the blue light being blue light that illuminates a living tissue at the time of the thermal treatment observation mode, and being blue light that excites an advanced glycation end product generated by a living tissue being thermally treated; an imaging element that is common to either of the narrow-band light observation mode and the thermal treatment observation mode, the imaging element including a pixel section having a plurality of pixels arranged in a 2-dimensional matrix, and a color filter configured to have any one of a red filter, a green filter, and a blue filter provided on a light-receiving surface of each of the plurality of pixels, the imaging element being capable of generating image data by capturing at least one of return light from the living tissue and fluorescence from the advanced glycation end product; and and ​ A cut filter is provided at least on the light-receiving surface side of the pixel provided with the green filter, and blocks light in a wavelength band including the fluorescence while transmitting the two kinds of blue light.

12. The medical observation system according to claim 11, wherein The two kinds of blue light are generated from one light source section.

13. The medical observation system according to claim 12, wherein The medical observation system further includes an image processing section that performs image processing on the image data and outputs the image data to a display device, When the blue light is irradiated to the living tissue using the light source device in the narrow-band light observation mode, the image processing section generates a narrow-band light image based on a blue component signal from the pixel provided with the blue filter included in the image data, and, on the other hand, When only the blue light is irradiated to the advanced glycosylation end product using the light source device in the thermal treatment observation mode, the image processing section generates a thermal treatment image based on a blue component signal from the pixel provided with the blue filter and a green component signal from the pixel provided with the green filter included in the image data.

14. A medical imaging device including: an imaging element having a pixel section including a plurality of pixels arranged in a two-dimensional matrix, and a color filter configured to have any one of a red filter, a green filter, and a blue filter provided on a light-receiving surface of each of the plurality of pixels; and a cut filter provided at least on the light-receiving surface side of the pixel provided with the green filter, The imaging element generates image data by capturing at least one of return light from a living tissue when a first narrow-band light narrower than a wavelength band of white light is irradiated to the living tissue, and fluorescence from an advanced glycosylation end product when a second narrow-band light shorter in wavelength than the first narrow-band light is irradiated to the advanced glycosylation end product generated by performing thermal treatment on the living tissue, and the advanced glycosylation end product is excited, The cut filter blocks light on a short wavelength side of a wavelength band including the second narrow-band light while transmitting the first narrow-band light.

15. A medical imaging device including an imaging element having a pixel section including a plurality of pixels arranged in a two-dimensional matrix, and a color filter configured to have any one of a red filter, a green filter, and a blue filter provided on a light-receiving surface of each of the plurality of pixels, The imaging element generates image data by imaging at least one of return light from a living body tissue when first narrow-band light narrower than a wavelength band of white light is irradiated to the living body tissue, and fluorescent light from an advanced glycation end product generated by performing a thermal treatment on the living body tissue when second narrow-band light shorter in wavelength than the first narrow-band light is irradiated to the advanced glycation end product to excite the advanced glycation end product, The green filter blocks light on a short wavelength side of a wavelength band of the second narrow-band light, and transmits the first narrow-band light.

16. An imaging method characterized by comprising: a light source irradiating narrow-band light that excites an advanced glycation end product to a living body tissue, a blue pixel of an imaging element imaging light from the living body tissue and the fluorescent light from the advanced glycation end product after passing through a blue filter, a green pixel of the imaging element imaging light from the living body tissue and the fluorescent light from the advanced glycation end product after passing through a cut filter that blocks light on a short wavelength side of a wavelength of the fluorescent light, and further passing through a green filter that mainly transmits light of a green wavelength band.

Images