Endoscope system and operating method thereof

Abstract

The present invention provides an endoscope system and an operating method thereof that can reliably perform analysis processing even when using a rolling shutter imaging sensor and can display the analysis results in a clear image with high visual recognition. The imaging processor (45) reads out a signal and outputs a first image signal while exposing the imaging sensor (44) to a first illumination light during a first illumination period. The imaging processor (45) reads out a signal and outputs a second image signal after exposing the imaging sensor to a second illumination light during a second illumination period. The imaging processor (45) reads out a signal and outputs a second image signal while extinguishing the first illumination light and the second illumination light during an extinguishing period.

Description

Technical Field

[0001] The present invention relates to an endoscope system for switching and emitting a plurality of illumination lights with different luminous spectra and a working method thereof. Background Art

[0002] In the medical field, medical images are widely used for diagnosis. For example, an endoscope system, which includes a light source device, an endoscope, and a processor device, is a device that uses medical images. An endoscope system captures an endoscopic image, a medical image, by illuminating an object with illumination light and capturing the illuminated object. The endoscopic image is then displayed on a monitor and used for diagnosis.

[0003] Furthermore, in recent endoscope systems, the visibility of lesions on the observation object is enhanced by illuminating the observation object with light having a specific wavelength band, thereby facilitating lesion detection. However, special observation images obtained using light having a specific wavelength band are displayed in colors different from the color of the observation object, such as pseudo-color. Therefore, they are often difficult for users to see compared to the normal observation images used for normal observation. Therefore, Patent Document 1 detects lesion areas from special observation images and highlights them in the normal observation image, making it possible to observe lesions within the normal observation image that users are familiar with.

[0004] Previous technical literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Laid-Open No. 2011-135983 Summary of the Invention

[0007] Technical issues to be solved by the invention

[0008] In recent years, CMOS (Complementary Metal Oxide Semiconductor) type imaging sensors have been used for photographing objects under observation. In CMOS type imaging sensors, a rolling shutter method is adopted in which signals are read out in row units in the row direction and in order in the column direction in the case of a plurality of pixels arranged in the row direction and the column direction. However, the rolling shutter method sometimes produces a difference in exposure time between different rows, in which case, distortion of the image of the object under observation sometimes occurs. Therefore, as described in the above-mentioned patent document 1, when detecting a lesion area, the lesion area may not be accurately detected due to image distortion, etc., with respect to the image obtained by photographing based on the rolling shutter method. Therefore, it is required that even when using an imaging sensor using the rolling shutter method, analysis processing such as lesion detection can be reliably performed, and these analysis results can be displayed on a clear image with high visual recognition.

[0009] An object of the present invention is to provide an endoscope system and an operating method thereof that can reliably execute analysis processing even when a rolling shutter imaging sensor is used, and can display the analysis results in a clear image with high visibility.

[0010] Means for solving technical problems

[0011] The present invention comprises: a light source portion, which emits a first illumination light and a second illumination light having different luminous spectra; a light source processor, which automatically switches and emits the first illumination light with a first luminous pattern and emits the second illumination light with a second luminous pattern; an imaging sensor, wherein a plurality of pixels are arranged in row and column directions; and an imaging processor, which reads out signals from the pixels in a column-wise order in row units in the row direction, the imaging processor performing the following processing: in a first illumination period of emitting the first illumination light, the imaging sensor is exposed to the first illumination light, the signal is read out to output a first image signal, and after the imaging sensor is exposed to the second illumination light in a second illumination period of emitting the second illumination light, the imaging sensor is set to an unexposed state in an extinguishing period of extinguishing the first illumination light and the second illumination light, the signal is read out to output a second image signal.

[0012] Preferably, if the imaging sensor has a global reset function, the light source processor switches to the second illumination period after the first illumination period and after the imaging processor performs a global reset. Preferably, if the imaging sensor does not have a global reset function, the light source processor switches to the off period after the first illumination period, and after the imaging processor performs signal readout in the off period following the first illumination period, switches to the second illumination period. Preferably, if the imaging sensor does not have a global reset function, after performing signal readout related to the second illumination light in the off period, the light source processor continues the off period in order to perform signal readout in the off state of the first and second illumination lights by the imaging processor, and after performing signal readout in the off state, switches to the first illumination period.

[0013] Preferably, white light is used as the first illumination light, and violet, blue, green, or red light is used as the second illumination light. Preferably, any of violet, blue, green, or red light is emitted during the second illumination period, or at least two of violet, blue, green, or red light are switched and emitted in a specific order during the second illumination period. Preferably, the second illumination light is either a first special light in which the amount of violet light is greater than the amount of light of other colors, or a second special light in which the amount of green light is greater than the amount of light of other colors.

[0014] The first light-emitting pattern is preferably any of a 1A light-emitting pattern in which the number of frames of the first lighting period is the same in each first lighting period, and a 1B light-emitting pattern in which the number of frames of the first lighting period is different in each first lighting period. The second light-emitting pattern is preferably any of the following patterns: a 2A pattern in which the number of frames of the second lighting period is the same in each second lighting period, and the light-emitting spectrum of the second illumination light is the same in each second lighting period; a 2B pattern in which the number of frames of the second lighting period is the same in each second lighting period, and the light-emitting spectrum of the second illumination light is different in each second lighting period; a 2C pattern in which the number of frames of the second lighting period is different in each second lighting period, and the light-emitting spectrum of the second illumination light is the same in each second lighting period; and a 2D pattern in which the number of frames of the second lighting period is different in each second lighting period, and the light-emitting spectrum of the second illumination light is different in each second lighting period.

[0015] Preferably, in the specific light emission pattern, the first illumination period is longer than the second illumination period. Preferably, the first illumination period of the light source processor is two frames or more. Preferably, a display control unit is provided for displaying an analysis result display image on the display unit, which displays an analysis result obtained through analysis processing based on the second image signal, in addition to the display image based on the first image signal.

[0016] The present invention is a working method of an endoscope system, which includes: a light source unit that emits a first illumination light and a second illumination light; a light source processor that automatically switches between emitting the first illumination light and the second illumination light, emitting the first illumination light in a first light-emitting pattern and emitting the second illumination light in a second light-emitting pattern; an imaging sensor having a plurality of pixels arranged in a row direction and a column direction; and an imaging processor that reads out signals from the pixels in a column direction in units of rows in the row direction. In the working method of the endoscope system, the imaging processor includes the following steps: reading out signals and outputting a first image signal in a state where the first illumination light is exposed to the imaging sensor during a first illumination period of emitting the first illumination light, and reading out signals and outputting a second image signal in a state where the imaging sensor is set to an unexposed state during an extinguishing period of extinguishing the first illumination light and the second illumination light after exposing the imaging sensor to the second illumination light during a second illumination period of emitting the second illumination light.

[0017] Effects of the Invention

[0018] According to the present invention, even when a rolling shutter imaging sensor is used, detection and analysis processing can be reliably performed, and the analysis results can be displayed in a clear image with high visibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is an external view of the endoscope system.

[0020] Figure 2 This is a block diagram showing the functions of the endoscope system according to the first embodiment.

[0021] Figure 3 It is a graph showing the spectrum of violet light V, blue light B, green light G, and red light R.

[0022] Figure 4 This is an explanatory diagram showing a specific light emission pattern, which is a light emission pattern in the analysis processing mode.

[0023] Figure 5 This is a graph showing the spectral transmittance of each color filter of the imaging sensor.

[0024] Figure 6 This is an explanatory diagram showing the arrangement of pixels in an imaging sensor.

[0025] Figure 7 This is an explanatory diagram showing a rolling shutter.

[0026] Figure 8 This diagram illustrates the lighting control, analysis processing, and image display in the analysis processing mode in chronological order.

[0027] Figure 9 This is an explanatory diagram showing imaging control when the imaging sensor has an analysis processing mode during a global reset.

[0028] Figure 10 This is an explanatory diagram showing imaging control in the case where the imaging sensor does not have an analysis processing mode for global reset.

[0029] Figure 11 This means that the camera sensor does not have a global reset and is Figure 10 Explanatory diagram of imaging control in analysis processing modes in different cases.

[0030] Figure 12 This is a flowchart showing a series of flows in the analysis processing mode.

[0031] Figure 13 It is a graph showing the emission spectrum of the first special light.

[0032] Figure 14 It is a graph showing the emission spectrum of the second special light. DETAILED DESCRIPTION

[0033] exist Figure 1In the embodiment, the endoscope system 10 includes an endoscope 12, a light source device 14, a processor device 16, a display 18, and a user interface 19. The endoscope 12 is optically connected to the light source device 14 and electrically connected to the processor device 16. The endoscope 12 includes an insertion portion 12a for insertion into the body of an observation subject, an operating portion 12b provided at the base end of the insertion portion 12a, a bending portion 12c provided at the distal end of the insertion portion 12a, and a distal end portion 12d. The bending portion 12c is bent by operating the angle knob 12e of the operating portion 12b. The distal end portion 12d is oriented in a desired direction by the bending portion 12c.

[0034] Furthermore, the operation portion 12b includes, in addition to the angle knob 12e, a mode switching SW (mode switching switch) 12f used for switching the modes, a still image acquisition instruction portion 12g used for instructing acquisition of a still image of the observation object, and a zoom lens 43 (see FIG. Figure 2 ) operation.

[0035] The endoscope system 10 has three modes: normal observation mode, special observation mode, and analysis processing mode. In normal observation mode, the observation object is illuminated with white light to capture the image, and a normal observation image in natural color is displayed on the display 18. In special observation mode, the observation object is illuminated with special light having a wavelength band different from that of normal light to capture the image, and a special observation image with specific structures emphasized is displayed on the display 18. In analysis processing mode, the first and second illumination lights, each having a different emission spectrum, are switched, and the image based on the first illumination light is processed to be displayed on the display 18. Meanwhile, the image based on the second illumination light is analyzed to obtain an index value related to the observation object. The results of the analysis processing are displayed superimposed on the display image.

[0036] The processor device 16 is electrically connected to a display 18 and a user interface 19. The display 18 outputs an image of the observation object or information accompanying the image of the observation object. The user interface 19 includes a keyboard, a mouse, a touchpad, and the like, and has the function of receiving input operations such as function settings. Furthermore, an external recording unit (not shown) for recording images or image information may also be connected to the processor device 16.

[0037] exist Figure 2In the embodiment, the light source device 14 includes a light source unit 20 and a light source processor 21 for controlling the light source unit 20. The light source unit 20 includes, for example, a plurality of semiconductor light sources, which are individually turned on and off. When turned on, the amount of light emitted by each semiconductor light source is controlled to emit illumination light that illuminates the object under observation. In this embodiment, the light source unit 20 includes four color LEDs: a V-LED (Violet Light Emitting Diode) 20a, a B-LED (Blue Light Emitting Diode) 20b, a G-LED (Green Light Emitting Diode) 20c, and an R-LED (Red Light Emitting Diode) 20d.

[0038] like Figure 3 As shown, the V-LED 20a generates violet light V with a center wavelength of 405±10 nm and a wavelength range of 380-420 nm. The B-LED 20b generates blue light B with a center wavelength of 450±10 nm and a wavelength range of 420-500 nm. The G-LED 20c generates green light G with a wavelength range of 480-600 nm. The R-LED 20d generates red light R with a center wavelength of 620-630 nm and a wavelength range of 600-650 nm.

[0039] The light source processor 21 controls the V-LED 20a, B-LED 20b, G-LED 20c, and R-LED 20d. By independently controlling each LED 20a-20d, the light source processor 21 can independently vary the light intensity to emit violet light V, blue light B, green light G, or red light R. Furthermore, in normal observation mode, the light source processor 21 controls each LED 20a-20d so that white light is emitted with a light intensity ratio of Vc:Bc:Gc:Rc among the violet light V, blue light B, green light G, and red light R. Furthermore, Vc, Bc, Gc, and Rc are greater than 0.

[0040] Furthermore, in special observation mode, the light source processor 21 controls each LED 20a-20d to emit special light, where the light intensity ratio of violet light V, blue light B, green light G, and red light R, which are short-wavelength, narrow-band light, is Vs:Bs:Gs:Rs. This light intensity ratio Vs:Bs:Gs:Rs differs from the light intensity ratio Vc:Bc:Gc:Rc used in normal observation mode and is appropriately set according to the observation objective. For example, when emphasizing superficial blood vessels, Vs is preferably set greater than the other Bs, Gs, and Rs. When emphasizing mid- to deep-layer blood vessels, Gs is preferably set greater than the other Vs, Gs, and Rs.

[0041] And, as Figure 4As shown, when in the analysis processing mode, when automatically switching between emitting the first illumination light and the second illumination light, the light source processor 21 emits the first illumination light with a first illumination pattern and emits the second illumination light with a second illumination pattern. Specifically, the first illumination pattern is preferably either a 1A illumination pattern in which the number of frames in the first illumination period is the same in each first illumination period, or a 1B illumination pattern in which the number of frames in the first illumination period is different in each first illumination period. The second light-emitting pattern is preferably any one of the following patterns: a pattern 2A in which the number of frames in the second lighting period is the same in each second lighting period and the light-emitting spectrum of the second lighting light is the same in each second lighting period; a pattern 2B in which the number of frames in the second lighting period is the same in each second lighting period and the light-emitting spectrum of the second lighting light is different in each second lighting period; a pattern 2C in which the number of frames in the second lighting period is different in each second lighting period and the light-emitting spectrum of the second lighting light is the same in each second lighting period; and a pattern 2D in which the number of frames in the second lighting period is different in each second lighting period and the light-emitting spectrum of the second lighting light is different in each second lighting period. In addition, the light-emitting spectrum of the first lighting light may be the same or different in each first lighting period.

[0042] Furthermore, regarding the specific light emitting pattern, the first lighting period is preferably longer than the second lighting period, and the first lighting period is preferably set to more than 2 frames. Figure 4 In the embodiment, when the first light emission pattern is set to pattern 1A and the second light emission pattern is set to pattern 2A (the number of frames in the second illumination period is the same, and the emission spectrum of the second illumination light is the same), the first illumination period is set to two frames and the second illumination period is set to one frame. The first illumination light is used to generate a display image displayed on the display 18. Therefore, it is preferable to obtain a clear image by illuminating the observation object with the first illumination light.

[0043] For example, the first illumination light is preferably white light. Meanwhile, the second illumination light is used for analysis and processing, and therefore, it is preferable to obtain an image suitable for analysis and processing by illuminating the observation object with the second illumination light. For example, when analyzing and processing based on the shape information of multiple blood vessels of varying depths, it is preferable to use violet light V, blue light B, green light G, and red light R as the second illumination light. In this case, when the second light emission pattern is set to pattern 2A (same number of frames in the second illumination period, same emission spectrum of the second illumination light) or pattern 2C (different number of frames in the second illumination period, same emission spectrum of the second illumination light), it is preferable to use any one of violet light V, blue light B, green light G, and red light R. On the other hand, when the second light emitting pattern is set to the 2B pattern (number of frames in the second lighting period: the same, the light emitting spectrum of the second lighting light: different) or the 2D pattern (number of frames in the second lighting period: different, the light emitting spectrum of the second lighting light: different), in the second lighting period, it is preferred to switch and emit at least two of the purple light V, blue light B, green light G, and red light R in a specific order. Figure 8 In the embodiment, the three lights of purple light V, green light G and red light R are emitted in this order.

[0044] The switching pattern between the first illumination period and the second illumination period, i.e., the first and second light emission patterns, is set by the imaging control of the imaging sensor 44 by the imaging processor 45, and will be described in detail later. Furthermore, a frame is a unit of time period that includes at least the period from a specific timing to the end of signal readout in the imaging sensor 44. In this embodiment, a CMOS imaging sensor 44 is used, and therefore, the frame refers to the period from the start of signal readout to the end of signal readout.

[0045] For example, when obtaining the shape information of blood vessels related to the surface blood vessels existing within a depth of 50 μm from the mucosal surface, the middle blood vessels existing within a depth of 200 μm from the mucosal surface, and the deep blood vessels existing within a depth of 600 μm from the mucosal surface, and performing analysis and processing based on the shape information of these surface, middle and deep blood vessels, it is preferred to use purple light V to emphasize the surface blood vessels, green light G to emphasize the middle blood vessels, and red light R to emphasize the deep blood vessels.

[0046] In this specification, the term "light intensity ratio" includes the case where the ratio of at least one semiconductor light source is 0 (zero). Therefore, it also includes the case where one or more of the semiconductor light sources are not illuminated. For example, if the light intensity ratio of violet light V, blue light B, green light G, and red light R is 1:0:0:0, even if only one of the semiconductor light sources is illuminated and the other three are not illuminated, the light intensity ratio still exists.

[0047] like Figure 2 As shown, the light emitted by each LED 20a to 20d passes through an optical path coupling portion 23 formed of a reflector, lens, or the like and enters a light guide 25. The light guide 25 is built into the endoscope 12 and the universal cord (the cord that connects the endoscope 12, the light source device 14, and the processor device 16). The light guide 25 transmits the light from the optical path coupling portion 23 to the distal end portion 12d of the endoscope 12.

[0048] The distal end portion 12d of the endoscope 12 is provided with an illumination optical system 30a and an imaging optical system 30b. The illumination optical system 30a includes an illumination lens 32, through which illumination light propagated by the light guide 25 is irradiated onto the observation object. The imaging optical system 30b includes an objective lens 42 and an imaging sensor 44. Light from the observation object, which is generated by the illumination light, is incident on the imaging sensor 44 via the objective lens 42 and the zoom lens 43. As a result, an image of the observation object is formed on the imaging sensor 44. The zoom lens 43 is a lens for magnifying the observation object and is moved between a telephoto end and a wide-angle end by operating the zoom operation unit 12h.

[0049] The imaging sensor 44 is a primary color sensor and includes three types of pixels: B pixels (blue pixels) having a blue filter, G pixels (green pixels) having a green filter, and R pixels (red pixels) having a red filter. Figure 5 As shown, the blue filter BF primarily transmits light in the blue band, specifically, light in the wavelength band of 380 to 560 nm. The transmittance of the blue filter BF reaches a peak around wavelengths of 460 to 470 nm. The green filter GF primarily transmits light in the green band, specifically, light in the wavelength band of 460 to 620 nm. The red filter RF primarily transmits light in the red band, specifically, light in the wavelength band of 580 to 760 nm.

[0050] Furthermore, the image sensor 44 is a CMOS (Complementary Metal Oxide Semiconductor) type, such as Figure 6 As shown, there are a plurality of pixels Px arranged in the row direction and the column direction. The plurality of pixels Px include B pixels, G pixels, and R pixels. The imaging processor 45 controls the CMOS imaging sensor 44 to capture images. Specifically, the imaging processor 45 reads out signals from the pixels Px in the order of the column direction in the row unit LU in the row direction. This signal reading method is called a rolling shutter. When the imaging sensor 44 has pixels Px in the 1st to Nth rows in the column direction, as shown in FIG. Figure 7 As shown, signal reading is performed in the time sequence of the pixel group of the 1st row, the pixel group of the 2nd row, ..., the pixel group of the Nth row (N is a natural number greater than or equal to 2). Figure 7The readout end line FL indicates the timing at which the readout of each row of pixel groups ends. T1, T2, and TN on the readout end line FL indicate the timing at which the readout of the pixel groups in the 1st, 2nd, and Nth rows ends, with T1 < T2 < TN. Details of imaging control in analysis processing mode will be described later.

[0051] As described above, the imaging processor 45 reads out signals from the imaging sensor 44, thereby outputting image signals from the imaging sensor 44. In normal observation mode, while the imaging sensor 44 is exposed to white light, the imaging processor 45 reads out signals, thereby outputting Bc image signals from the B pixels of the imaging sensor 44, Gc image signals from the G pixels, and Rc image signals from the R pixels. In special observation mode, while the imaging sensor 44 is exposed to special light, the imaging processor 45 reads out signals, thereby outputting Bs image signals from the B pixels of the imaging sensor 44, Gs image signals from the G pixels, and Rs image signals from the R pixels.

[0052] In the analysis processing mode, the imaging processor 45 reads out a signal while exposing the imaging sensor 44 to the first illumination light during the first illumination period, thereby outputting a first image signal from the imaging sensor 44 (refer to FIG. Figures 9 to 11 ). The first image signal includes a B1 image signal output from a B pixel, a G1 image signal output from a G pixel, and an R1 image signal output from an R pixel. Furthermore, the image processing processor 45 exposes the image sensor 44 to the second illumination light during the second illumination period, and then reads out the signal by setting the image sensor 44 to an unexposed state during the extinguishing period in which the first and second illumination lights are extinguished, thereby outputting a second image signal from the image sensor 44 (refer to Figures 9 to 11 ). The second image signal includes a B2 image signal output from a B pixel, a G2 image signal output from a G pixel, and an R2 image signal output from an R pixel.

[0053] As described above, the first image signal obtained by performing signal readout (rolling shutter) during exposure to illumination light has a sufficient exposure time (a maximum exposure time of 1 / 60 is achieved for 60 frames), and is therefore preferably used as a display image displayed on the monitor 18. As long as this display image is observed with the naked eye in a dynamic image format, visual recognition is not reduced by the rolling effect (image distortion) caused by the use of a rolling shutter. On the other hand, the second image signal obtained by performing signal readout after exposure to illumination light is set to unexposed, and the exposure time can be made the same in the 1st to Nth lines, so it is not affected by the rolling effect (image distortion) caused by the use of a rolling shutter. Therefore, the second image signal can accurately output the shape of the observed object, making it an image signal suitable for analysis processing such as shape analysis. Details of the camera control in the analysis processing mode will be described later.

[0054] like Figure 2 As shown, CDS / AGC (Correlated Double Sampling / Automatic Gain Control) circuit 46 performs correlated double sampling (CDS) or automatic gain control (AGC) on the analog image signal obtained from image sensor 44. The image signal passed through CDS / AGC circuit 46 is converted into a digital image signal by A / D (Analog / Digital) converter 48. The A / D-converted digital image signal is input to processor device 16.

[0055] The processor device 16 includes an image acquisition unit 50, a DSP (Digital Signal Processor) 52, a noise reduction unit 54, an image processing switching unit 56, an image processing unit 58, and a display control unit 60. The image processing unit 58 includes a normal observation image generation unit 62, a special observation image generation unit 64, and an analysis processing unit 66.

[0056] The image acquisition unit 50 acquires the color image input from the endoscope 12. The color image includes a blue image, a green image, and a red image output from the B pixel, the G pixel, and the R pixel of the image sensor 44. The acquired color image is sent to the DSP 52. The DSP 52 performs various signal processing such as defect correction processing, offset processing, gain correction processing, matrix processing, gamma conversion processing, de-mosaic processing, and YC conversion processing on the received color image. In the defect correction processing, the signal of the defective pixel of the image sensor 44 is corrected. In the offset processing, the dark current component is removed from the image signal subjected to the defect correction processing, and an accurate zero level is set. The gain correction processing adjusts the signal level of the color image by multiplying the image signal of each color after the offset processing by a specific gain. The image signal of each color after the gain correction processing is subjected to matrix processing to improve color reproducibility.

[0057] The brightness and saturation of the color image are then adjusted through gamma conversion. Demosaicing (also known as isotropy or synchronization) is performed on the matrix-processed color image, and interpolation is used to generate signals for the missing colors in each pixel. Demosaicing ensures that all pixels have signals for each RGB color. The DSP 52 performs YC conversion on the demosaiced color image and outputs the luminance signal Y and the color difference signals Cb and Cr to the noise reduction unit 54.

[0058] The noise reduction unit 54 performs noise reduction processing based on, for example, a moving average method or a median filter method on the color image subjected to demosaicing processing by the DSP 56. The color image with reduced noise is input to the image processing unit 56.

[0059] The image processing switching unit 56 switches the destination of the image signal from the noise reduction unit 54 to one of the normal observation image generating unit 62, the special observation image generating unit 64, and the analysis processing unit 66, depending on the set mode. Specifically, when the normal observation mode is set, the image signal from the noise reduction unit 54 is input to the normal observation image generating unit 62. When the special observation mode is set, the image signal from the noise reduction unit 54 is input to the special observation image generating unit 64. When the analysis processing mode is set, the image signal from the noise reduction unit 54 is input to the analysis processing unit 66.

[0060] The normal observation image generation unit 62 performs normal observation image processing on the input Rc, Gc, and Bc image signals for one frame. This normal observation image processing includes color conversion processing such as 3×3 matrix processing, grayscale conversion processing, and three-dimensional LUT (Look Up Table) processing, as well as color enhancement processing and structure enhancement processing such as spatial frequency enhancement. The Rc, Gc, and Bc image signals that have undergone normal observation image processing are input to the display control unit 60 as normal observation images.

[0061] The special observation image generation unit 64 performs special observation image processing on the input Rs, Gs, and Bs image signals for one frame. This special observation image processing includes color conversion processing such as 3×3 matrix processing, grayscale conversion processing, and three-dimensional LUT (Look Up Table) processing, as well as color enhancement processing and structure enhancement processing such as spatial frequency enhancement. The Rs, Gs, and Bs image signals that have undergone special observation image processing are input to the display control unit 60 as special observation images.

[0062] The analysis processing unit 66 performs the same normal observation image processing as described above on the input R1, G1, and B1 image signals for one frame. The R1, G1, and B1 image signals that have undergone the normal observation image processing are used as display images. Furthermore, the analysis processing unit 66 performs analysis processing on the input R2, G2, and B2 image signals for a specific frame. Furthermore, the analysis processing unit 66 performs display control processing to display the analysis results, i.e., the analysis results, on the display image.

[0063] For example, when the first light-emitting pattern is set to the 1A light-emitting pattern and the second light-emitting pattern is set to the 2B pattern (the number of frames in the second illumination period: the same, the light-emitting spectrum of the second illumination light: different), when the white light W is emitted as the first illumination light for 2 frames and the purple light V, green light G and red light R as the second illumination light are respectively used to illuminate the observation object for 1 frame during the emission of the white light W, as shown in FIG. Figure 8As shown, the R2 image signal, G2 image signal, and B2 image signal obtained by illumination with purple light V are analyzed and processed to obtain analysis result V. Similarly, the R2 image signal, G2 image signal, and B2 image signal obtained by illumination with green light G are analyzed and processed to obtain analysis result G. Furthermore, the R2 image signal, G2 image signal, and B2 image signal obtained by illumination with red light R are analyzed and processed to obtain analysis result R. After the analysis processing related to red light R is completed, these analysis results V, G, and R are displayed on the display image as a combined analysis result T. Furthermore, the analysis results V, G, and R can be displayed on the display image individually, or an analysis result obtained by combining at least two of the analysis results V, G, and R can be displayed on the display image.

[0064] The analysis processing includes, for example, blood vessel extraction processing for extracting blood vessel shape information, index value calculation processing for calculating an index value related to the blood vessel based on the extracted blood vessel shape information, and superimposed display control processing for superimposing the calculated index value on a display image. An image with an analysis result display showing the analysis processing results is input to the display control unit 60.

[0065] The display control unit 60 controls the display 18 so that the image output from the image processing unit 58 is displayed. Specifically, the display control unit 60 converts the normal observation image, the special observation image, or the image for display with analysis results into a video signal that can be displayed in full color on the display 18. The converted video signal is input to the display 18. As a result, the normal observation image, the special observation image, or the image for display with analysis results is displayed on the display 18.

[0066] Next, the details of the light source control and imaging control in the analysis processing mode will be described. When the CMOS imaging sensor 44 has a global reset that can discharge all the charges accumulated in the pixels Px without reading out the signals, as shown in FIG. Figure 9 As shown, the imaging processor 45 activates the global reset GR after completing the signal readout for the first lighting period. After the global reset GR is activated, the light source processor 21 switches to the second lighting period. Then, after completing the signal readout for the second lighting period, the light source processor 21 switches to the off period. Furthermore, after completing the signal readout for the off period, the imaging processor 45 activates the global reset. Then, after completing the global reset GR, the light source processor 21 switches to the first lighting period.

[0067] On the other hand, when the CMOS type imaging sensor 44 does not have a global reset, as shown in FIG. Figure 10As shown, after the signal readout for the first illumination period is completed, the light source processor 21 switches to the blackout period. The imaging processor 45 reads out the signal during the blackout period. The image signal (blank signal) obtained in the signal readout during the blackout period is a signal containing charge obtained by exposing the first illumination light to the imaging sensor 44 during the first illumination period. Therefore, it is discarded in order to prevent the components of the first illumination light from being mixed into the components of the second illumination light emitted thereafter. After the light source processor 21 reads out the signal during the blackout period following the first illumination light, it switches to the second illumination period. Furthermore, after the signal readout for the second illumination period is completed, the light source processor 21 switches to the first illumination period.

[0068] In addition, Figure 10 In the second illumination period, after the signal reading related to the second illumination light is completed, it is immediately switched to the first illumination period. However, sometimes even in the extinguishing period, charge accumulation occurs due to slight exposure. Therefore, if Figure 11 As shown, after the light source processor 21 completes signal readout related to the second illumination light, it may continue the off-period in order to perform signal readout by the imaging processor 45 in the off-state state of the first and second illumination lights. Then, after the light source processor 21 completes signal readout in the off-state, it switches to the first illumination period. Furthermore, the image signal obtained by the off-period signal readout is discarded as a blank signal.

[0069] Then, follow Figure 12 The flowchart shown illustrates the analysis and processing mode. Switching to analysis and processing mode by operating mode switch SW12f automatically switches between the first illumination period, during which the first illumination light is emitted, and the second illumination period, during which the second illumination light is emitted, using a specific light pattern. Thus, the first and second illumination lights, automatically switched with the specific light pattern, illuminate the object under observation.

[0070] The imaging processor 45 reads signals from pixels in the imaging sensor 44 in the column direction, row by row, while exposing the imaging sensor 44 to the first illumination light during the first illumination period. This outputs a first image signal from the imaging sensor 44. Furthermore, after exposing the imaging sensor 44 to the second illumination light during the second illumination period, the imaging processor 45 reads signals while the imaging sensor 44 is unexposed during the first illumination light and second illumination light extinguishing periods. This outputs a second image signal from the imaging sensor 44. The display control unit 60 displays an analysis result display image on the display 18, along with the display image based on the first image signal, that shows the analysis result obtained through analysis processing based on the second image signal.

[0071] In the above embodiment, purple light V, blue light B, green light G, and red light R are used as the second illumination light, but light of other emission spectra may also be used. Figure 13 As shown in FIG. 1 , as the second illumination light, the first special light may be used, which makes the light amount of the purple light V greater than the light amounts of the other blue light B, green light G, and red light R. Figure 14 As shown, a second special light can be used as the second illumination light, where the amount of green light G is greater than that of the other violet light V, blue light B, and red light R. Furthermore, both the first and second special lights can be used as the second illumination light, or the second emission pattern can be set to a 2B pattern or a 2D pattern, with the first and second special lights emitted alternately. Furthermore, when the first special light is used, chromatic aberration expansion processing can be performed on the second image signal to expand the color difference between normal areas and abnormal areas (e.g., lesions) included in the observation object. The second image signal that has undergone chromatic aberration expansion processing can be analyzed.

[0072] Furthermore, in the above-described embodiment, the first image signal based on the first illumination light is used for a display image, and the second image signal based on the second illumination light is used only for analysis and processing and not for display on the display 18. However, the second image signal may also be used for display on the display 18. In this case, the display image based on the first image signal and the display image based on the second image signal are switched and displayed on the display 18. Whether the image based on the second image signal is displayed on the display 18 or not is preferably configurable as appropriate via the user interface 19.

[0073] In the above-described embodiment, the hardware configuration of the processing units (processing units) that perform various processes, such as the light source processor 21, the imaging processor 45, and the normal observation image generating unit 62, the special observation image generating unit 64, and the analysis processing unit 66 included in the image processing unit 58, is composed of the following various processors. These various processors include general-purpose processors such as CPUs (Central Processing Units) that execute software (programs) and function as various processing units; processors such as FPGAs (Field Programmable Gate Arrays) whose circuit configuration can be modified after manufacture, such as programmable logic devices (PLDs); and processors with circuit configurations specifically designed to perform various processes, such as dedicated electrical circuits.

[0074] A processing unit may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs or a CPU and an FPGA). In addition, multiple processing units may be composed of one processor. As an example of a plurality of processing units composed of one processor, first, there is a method in which a processor is composed of a combination of one or more CPUs and software, such as represented by a computer such as a client or a server, and the processor functions as multiple processing units. Second, there is a method in which a processor is used to implement the functions of the entire system including multiple processing units by one IC (Integrated Circuit) chip, such as represented by a System On Chip (SoC). In this way, various processing units are composed of one or more of the above-mentioned various processors as a hardware structure.

[0075] More specifically, the hardware structure of these various processors is an electric circuit (circuitry) in which circuit elements such as semiconductor elements are combined. Furthermore, the hardware structure of the storage unit is a storage device such as an HDD (hard disc drive) or an SSD (solid state drive).

[0076] Explanation of symbols

[0077] 10-Endoscope system, 12-Endoscope, 12a-Insertion portion, 12b-Operation portion, 12c-Bending portion, 12d-Front end portion, 12e-Angle knob, 12f-Mode switching switch, 12g-Still image acquisition instruction portion, 12h-Zoom operation portion, 14-Light source device, 16-Processor device, 18-Display, 19-User interface, 20-Light source portion, 20a-V-LED, 20b-B-LED, 20c-G-LED, 20d-R-LED, 21-Light source processor, 23-Optical path combination , 25-light guide, 30a-illumination optical system, 30b-imaging optical system, 32-illumination lens, 42-objective lens, 43-zoom lens, 44-imaging sensor, 45-imaging processor, 46-CDS / AGC circuit, 48-A / D converter, 50-image acquisition unit, 52-DSP, 54-noise reduction unit, 56-image processing switching unit, 58-image processing unit, 60-display control unit, 62-normal observation image generation unit, 64-special observation image generation unit, 66-analysis processing unit, Px-pixel.

Claims

1. An endoscope system comprising: a light source unit that emits a first illumination light and a second illumination light having different emission spectra; a light source processor configured to automatically switch between the first illumination light and the second illumination light and emit the first illumination light in a first light emission pattern and the second illumination light in a second light emission pattern; A camera sensor, wherein a plurality of pixels are arranged in row and column directions; and The imaging processor reads signals from the pixels in the order of the columns in the row direction in units of rows in the row direction. The camera processor performs the following processing: In a state where the imaging sensor is exposed to the first illumination light during a first illumination period in which the first illumination light is emitted, the signal is read out to output a first image signal. After the imaging sensor is exposed to the second illumination light during a second illumination period in which the second illumination light is emitted, the imaging sensor is set to an unexposed state during an extinguishing period in which the first illumination light and the second illumination light are extinguished, and the signal is read out to output a second image signal. The first lighting period is longer than the second lighting period, The first lighting period is more than 2 frames, In the case where the camera sensor has a global reset, The light source processor switches to the second lighting period after the first lighting period and after the global reset operation performed by the imaging processor.

2. An endoscope system comprising: a light source unit that emits a first illumination light and a second illumination light having different emission spectra; a light source processor configured to automatically switch between the first illumination light and the second illumination light and emit the first illumination light in a first light emission pattern and the second illumination light in a second light emission pattern; A camera sensor, wherein a plurality of pixels are arranged in row and column directions; and The imaging processor reads signals from the pixels in the order of the columns in the row direction in units of rows in the row direction. The camera processor performs the following processing: In a state where the imaging sensor is exposed to the first illumination light during a first illumination period in which the first illumination light is emitted, the signal is read out to output a first image signal. After the imaging sensor is exposed to the second illumination light during a second illumination period in which the second illumination light is emitted, the imaging sensor is set to an unexposed state during an extinguishing period in which the first illumination light and the second illumination light are extinguished, and the signal is read out to output a second image signal. The first lighting period is longer than the second lighting period, The first lighting period is more than 2 frames, In the case where the camera sensor does not have a global reset, The light source processor switches to the light-off period after the first lighting period, and switches to the second lighting period after the light-off period after the first lighting period has read out a signal by the imaging processor.

3. An endoscope system comprising: a light source unit that emits a first illumination light and a second illumination light having different emission spectra; a light source processor configured to automatically switch between the first illumination light and the second illumination light and emit the first illumination light in a first light emission pattern and the second illumination light in a second light emission pattern; A camera sensor, wherein a plurality of pixels are arranged in row and column directions; and The imaging processor reads signals from the pixels in the order of the columns in the row direction in units of rows in the row direction. The camera processor performs the following processing: In a state where the imaging sensor is exposed to the first illumination light during a first illumination period in which the first illumination light is emitted, the signal is read out to output a first image signal. After the imaging sensor is exposed to the second illumination light during a second illumination period in which the second illumination light is emitted, the imaging sensor is set to an unexposed state during an extinguishing period in which the first illumination light and the second illumination light are extinguished, and the signal is read out to output a second image signal. The first lighting period is longer than the second lighting period, The first lighting period is more than 2 frames, In the case where the camera sensor does not have a global reset, After the light source processor performs signal reading related to the second illumination light during the extinction period, the extinction period is continued in order to perform signal reading based on the camera processor in the extinction state of the first illumination light and the second illumination light, and after performing signal reading in the extinction state, it switches to the first illumination period.

4. The endoscope system according to any one of claims 1 to 3, wherein: White light is used as the first illumination light, and violet light, blue light, green light, or red light is used as the second illumination light.

5. The endoscope system according to claim 4, wherein: During the second lighting period, any one of the purple light, the blue light, the green light or the red light is emitted, or during the second lighting period, at least two of the purple light, the blue light, the green light or the red light are switched in a specific order and emitted.

6. The endoscope system according to any one of claims 1 to 3, wherein: The second illumination light is either first special light in which the light intensity of purple light is greater than that of light of other colors, or second special light in which the light intensity of green light is greater than that of other colors.

7. The endoscope system according to any one of claims 1 to 3, wherein: The first light emitting pattern is any one of a 1A light emitting pattern in which the number of frames of the first lighting period is the same in each of the first lighting period and a 1B light emitting pattern in which the number of frames of the first lighting period is different in each of the first lighting period.

8. The endoscope system according to any one of claims 1 to 3, wherein: The second light emitting pattern is any of the following patterns: The number of frames of the second lighting period is the same in each of the second lighting periods, and the luminous spectrum of the second lighting light is the same 2A pattern in each of the second lighting periods; The number of frames of the second lighting period is the same in each of the second lighting periods, and the luminous spectrum of the second lighting light is a different secondB pattern in each of the second lighting periods; The number of frames of the second lighting period is different in each of the second lighting periods, and the luminous spectrum of the second illumination light is the same 2C pattern in each of the second lighting periods; and The number of frames of the second lighting period is different in each of the second lighting periods, and the emission spectrum of the second illumination light is a different second D pattern in each of the second lighting periods.

9. The endoscope system according to any one of claims 1 to 3, comprising: The display control unit displays, on the display unit, an image with analysis result display that displays an analysis result obtained by analysis processing based on the second image signal in relation to the display image based on the first image signal.

10. An operating method of an endoscope system, the endoscope system comprising: a light source unit that emits a first illumination light and a second illumination light; a light source processor that automatically switches between the first illumination light and the second illumination light and emits the first illumination light in a first light emission pattern and the second illumination light in a second light emission pattern; an imaging sensor having a plurality of pixels arranged in row and column directions; and an imaging processor that reads signals from the pixels in order in the column direction in units of rows in the row direction, the operating method of the endoscope system comprising the following steps: The imaging processor performs the signal readout to output a first image signal while the imaging sensor is exposed to the first illumination light during a first illumination period of emitting the first illumination light, and then performs the signal readout to output a second image signal while the imaging sensor is unexposed during an extinguishing period of extinguishing the first illumination light and the second illumination light, after exposing the imaging sensor to the second illumination light during a second illumination period of emitting the second illumination light. The first lighting period is longer than the second lighting period, The first lighting period is more than 2 frames, In the case where the camera sensor has a global reset, The light source processor switches to the second lighting period after the first lighting period and after the global reset operation performed by the imaging processor.

11. An operating method of an endoscope system, the endoscope system comprising: a light source unit that emits a first illumination light and a second illumination light; a light source processor that automatically switches between the first illumination light and the second illumination light and emits the first illumination light in a first light emission pattern and the second illumination light in a second light emission pattern; an imaging sensor having a plurality of pixels arranged in row and column directions; and an imaging processor that reads signals from the pixels in order in the column direction in units of rows in the row direction, the operating method of the endoscope system comprising the following steps: The imaging processor performs the signal readout to output a first image signal while the imaging sensor is exposed to the first illumination light during a first illumination period of emitting the first illumination light, and then performs the signal readout to output a second image signal while the imaging sensor is unexposed during an extinguishing period of extinguishing the first illumination light and the second illumination light, after exposing the imaging sensor to the second illumination light during a second illumination period of emitting the second illumination light. The first lighting period is longer than the second lighting period, The first lighting period is more than 2 frames, In the case where the camera sensor does not have a global reset, The light source processor switches to the light-off period after the first lighting period, and switches to the second lighting period after the light-off period after the first lighting period has read out a signal by the imaging processor.

12. An operating method of an endoscope system, the endoscope system comprising: a light source unit that emits a first illumination light and a second illumination light; a light source processor that automatically switches between the first illumination light and the second illumination light and emits the first illumination light in a first light emission pattern and the second illumination light in a second light emission pattern; an imaging sensor having a plurality of pixels arranged in row and column directions; and an imaging processor that reads signals from the pixels in order in the column direction in units of rows in the row direction, the operating method of the endoscope system comprising the following steps: The imaging processor performs the signal readout to output a first image signal while the imaging sensor is exposed to the first illumination light during a first illumination period of emitting the first illumination light, and then performs the signal readout to output a second image signal while the imaging sensor is unexposed during an extinguishing period of extinguishing the first illumination light and the second illumination light, after exposing the imaging sensor to the second illumination light during a second illumination period of emitting the second illumination light. The first lighting period is longer than the second lighting period, The first lighting period is more than 2 frames, In the case where the camera sensor does not have a global reset, After the light source processor performs signal reading related to the second illumination light during the extinction period, the extinction period is continued in order to perform signal reading based on the camera processor in the extinction state of the first illumination light and the second illumination light, and after performing signal reading in the extinction state, it switches to the first illumination period.

Images