Endoscope system and method of operating the same

Abstract

To provide an endoscope system capable of displaying an endoscope image with reduced degradation of image quality in a case of acquiring a plurality of types of endoscope images, and an operation method of the system.SOLUTION: The endoscope system irradiates a subject with each of a plurality of illumination beams in a preset order, photographs the subject in accordance with a preset first imaging frame rate, in a first period of irradiation with a first illumination beam included in the illumination beams, acquires a first image photographed in the first period, generates a first image for display based on the acquired first image in accordance with a first display frame rate higher than the first imaging frame rate, and displays the first image for display on a display.SELECTED DRAWING: Figure 14

Description

Technical Field

[0001] The present invention relates to an endoscope system capable of displaying a plurality of types of endoscope images and an operating method thereof.

Background Art

[0002] In the medical field, diagnosis using an endoscope system including a light source device, an endoscope, and a processor device is widely performed. In diagnosis using an endoscope system, by a method called image-enhanced endoscopy or image-enhanced endoscopy (IEE), an image obtained by photographing an observation target as a subject with an endoscope (hereinafter referred to as an endoscope image) is used to emphasize and display the surface structure, lesion, biological information, etc. of the observation target, and diagnostic support information for a doctor to diagnose the observation target is obtained.

[0003] In IEE, various types of endoscope images are obtained by a method of digitally processing an endoscope image obtained by imaging an observation target or a method of using an endoscope image obtained by illuminating an observation target with specific illumination light and imaging it. For example, by a doctor selecting a specific type of endoscope image, biological information such as a region where blood vessels are concentrated or a region with low oxygen saturation in an observation target is determined, and these regions are emphasized and displayed on a display or the like. Such a display is useful as diagnostic support information for a doctor to diagnose an observation target.

[0004] When acquiring two types of image signals, a first imaging signal and a second imaging signal, necessary for generating an oxygen saturation image, while providing a lighting-off period for the illumination light, by shortening the readout time of the second imaging signal compared to the readout time of the first imaging signal that becomes a normal observation image, a decrease in the luminance and resolution of the normal observation image and a decrease in the frame rate are suppressed, and a decrease in the luminance and resolution of the oxygen saturation image using the normal observation image and a decrease in the frame rate are also suppressed. An endoscope system and the like are known (Patent Document 1).

Prior Art Documents

Patent Documents

[0005] [Patent Document 1] International Publication No. 2015 / 136963 [Summary of the Invention] [Problems to be Solved by the Invention]

[0006] When acquiring a plurality of types of endoscopic images by irradiating each of a plurality of types of illumination lights, if the frame rate of shooting is the same as when irradiating a single illumination light, the frame rate of shooting of each type of endoscopic image will decrease. In the case of Patent Document 1, when performing display based on a specific type of endoscopic image among the captured endoscopic images, the reading time can be shortened by selecting and reading pixels for other endoscopic images, etc., and the frame rate of the endoscopic image to be displayed can be increased. However, depending on the content of the required IEE, there may be cases where it is desired to perform display or image analysis, etc. while suppressing a decrease in the frame rate or a decrease in image quality caused by reading of selected pixels for two or more types of endoscopic images.

[0007] An object of the present invention is to provide an endoscopic system and an operating method thereof that can display an endoscopic image with suppressed deterioration in image quality when acquiring a plurality of types of endoscopic images. [Means for Solving the Problems]

[0008] The endoscope system of the present invention includes an endoscope having an imaging unit, a light source unit that irradiates a subject with each of a plurality of illumination lights having different spectra, and a processor device having a processor. The processor controls the light source unit to irradiate the subject with each of the plurality of illumination lights in a preset order in the first observation mode, and controls the imaging unit to photograph the subject according to a preset first imaging frame rate during a first period when the first illumination light included in the plurality of illumination lights is irradiated. The processor acquires a first image captured by the imaging unit during the first period, generates a first display image according to a first display frame rate higher than the first imaging frame rate based on the first image, and controls to display the first display image on a display.

[0009] The first display image preferably includes an image that does not change the first image.

[0010] In the first observation mode, the processor preferably controls the imaging unit to photograph the subject according to a preset second imaging frame rate during a second period when the second illumination light included in the plurality of illumination lights is irradiated, and acquires a second image captured by the imaging unit during the second period.

[0011] The processor preferably controls to display the second image on the display.

[0012] The processor preferably controls the light source unit to repeatedly irradiate the subject with a pattern composed of each of the plurality of illumination lights.

[0013] The processor preferably controls the light source unit to repeatedly irradiate the subject with a pattern composed of the first illumination light and the second illumination light to irradiate the subject with the first illumination light or the second illumination light.

[0014] The processor preferably controls the light source unit to intermittently irradiate each of the plurality of illumination lights.

[0015] The processor preferably controls the imaging unit such that, in a period of one frame consisting of an imaging period during which the imaging unit performs imaging and a readout period during which an image signal obtained by imaging is read out, the imaging period is longer than the readout period.

[0016] The processor preferably controls the light source unit such that, within the imaging period, the turning off and irradiation of any one of a plurality of illumination lights are each performed at least once.

[0017] The processor preferably controls the light source unit such that the irradiation period during which any one of a plurality of illumination lights is irradiated is changed to perform irradiation of the illumination light.

[0018] The processor preferably controls the imaging unit such that the exposure period is changed based on the changed irradiation period.

[0019] The processor preferably controls the light emission amount of the illumination light irradiated by the light source unit in one irradiation.

[0020] The light emission amount is preferably calculated by the irradiation period during which the light source unit irradiates the illumination light and the instantaneous light emission amount which is the light emission amount per unit time of the illumination light.

[0021] In a second observation mode that can be switched with the first observation mode, the processor controls the light source unit to irradiate a third illumination light included in a plurality of illumination lights to a subject, controls the imaging unit to photograph the subject according to a preset third imaging frame rate during a third period in which the third illumination light is irradiated, obtains a third image photographed by the imaging unit during the third period, and when displaying the third image on a display, preferably generates a display third image according to a third display frame rate that is lower than the third imaging frame rate.

[0022] A method of operating an endoscope system of the present invention is a method of operating an endoscope system including an endoscope having an imaging unit, a light source unit that irradiates a subject with each of a plurality of illumination lights having different spectra, and a processor device having a processor. The processor controls the light source unit to irradiate the subject with each of the plurality of illumination lights in a preset order in the first observation mode, controls the imaging unit to photograph the subject according to a preset first imaging frame rate during a first period when the first illumination light included in the plurality of illumination lights is irradiated, acquires a first image photographed by the imaging unit during the first period, generates a first display image according to a first display frame rate higher than the first imaging frame rate based on the first image, and performs control to display the first display image on a display.

Advantages of the Invention

[0023] According to the present invention, when acquiring a plurality of types of endoscope images, it is possible to display an endoscope image with suppressed deterioration in image quality.

Brief Description of the Drawings

[0024]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Mode for Carrying Out the Invention

[0025] As shown in FIG. 1, the endoscope system 10 includes an endoscope 12, a light source device 13, a processor device 14, a display 15, and a keyboard 16. The endoscope 12 is optically connected to the light source device 13 and electrically connected to the processor device 14. Various connections may not be limited to wired ones but may be wireless or via a network.

[0026] The endoscope 12 has an insertion portion 12a that is inserted into the body of a subject having an object to be observed, an operation portion 12b provided at the proximal end portion of the insertion portion 12a, and a bending portion 12c and a distal end portion 12d provided on the distal end side of the insertion portion 12a. The bending portion 12c bends by operating the angle knob 12e of the operation portion 12b. The distal end portion 12d includes an imaging sensor 45 that is an imaging unit, and is directed in a desired direction by the bending operation of the bending portion 12c. A forceps channel (not shown) for inserting a treatment instrument or the like is provided from the insertion portion 12a to the distal end portion 12d. The treatment instrument is inserted into the forceps channel from the forceps opening 12h. Also, air supply, water supply, or suction is also performed from the forceps opening 12h.

[0027] In addition to the angle knob 12e, the operation portion 12b has a zoom operation portion 12f for changing the imaging magnification, a mode change switch 12g used for switching the observation mode, and a freeze switch 12i for acquiring a still image. Note that the switching operation of the observation mode, the zoom operation, or the still image acquisition operation may be an operation using the mode change switch 12g, the zoom operation portion 12f, or the freeze switch, or a keyboard 16, a foot switch (not shown), or the like.

[0028] The endoscope system 10 includes a first observation mode and a second observation mode. In the first observation mode, a plurality of types of endoscope images are acquired, and one type or two or more types of endoscope images are displayed on the display 15. The second observation mode is an observation mode for acquiring one type of endoscope image, and the acquired one type of endoscope image is displayed on the display 15.

[0029] The types of endoscope images are distinguished by the type of illumination light and / or the content of image processing, etc. The type of illumination light is distinguished by the spectrum (spectral characteristics) of the illumination light. Therefore, when the spectra of the illumination light during imaging are different from each other, the captured endoscope images are of different types.

[0030] In this embodiment, in the first observation mode, a first image obtained by photographing an observation target using first illumination light and a second image obtained by photographing the observation target using second illumination light are acquired. In the second observation mode, a third image obtained by photographing the observation target using third illumination light is acquired.

[0031] In the first observation mode, as the first illumination light, a normal image (first image), which is an endoscopic image with a natural color obtained by imaging the observation target using white light as the illumination light, and a special image (second image) obtained by emitting special light (second illumination light), which is illumination light having a specific spectrum different from white light, to photograph the observation target are acquired. For example, the two types of images are arranged and displayed on the display 15. In the second observation mode, as the third illumination light, a normal image (third image) using white light is acquired, and the normal image (third image) is displayed on the display 15. The first illumination light and the third illumination light, or the second illumination light and the third illumination light, may be the same or different. In this embodiment, the first illumination light and the third illumination light are the same white light.

[0032] In this embodiment, the second image is a color difference expansion processed image obtained by performing color difference expansion processing on a special image obtained by emitting second illumination light to photograph the observation target. As the second image, alternatively, a special image obtained without performing color difference expansion processing on an endoscopic image obtained by emitting special light, which is illumination light having a specific spectrum different from white light, to photograph the observation target, or a pseudo-color processed image obtained by performing pseudo-color processing may be used. The color difference expansion processing or the pseudo-color processing will be described later.

[0033] The processor device 14 has a processor and is electrically connected to the display 15 and the keyboard 16. The display 15 displays an endoscopic image and / or various information obtained by the endoscope. The keyboard 16 functions as a user interface that receives input operations such as function settings. Note that an external storage (not shown) for storing an image or image information may be connected to the processor device 14.

[0034] As shown in FIG. 2, the light source device 13 emits illumination light for irradiating an observation target, and includes a light source unit 20 and a light source processor 21 that controls the light source unit 20. The light source processor 21 is controlled by the central control unit 51 of the processor device 14.

[0035] The light source unit 20 is composed of, for example, semiconductor light sources such as LEDs (Light Emitting Diodes) of a plurality of colors, a combination of a laser diode and a phosphor, or a xenon lamp or a halogen light source. Then, each of a plurality of illumination lights having different spectra is irradiated onto the observation target. The light source unit 20 includes an optical filter or the like for adjusting the wavelength band of the light emitted from the LED or the like. The light source processor 21 controls the light quantity of the illumination light by turning on / off each LED or the like and adjusting the drive current and drive voltage of each LED or the like. The light source processor 21 also controls the wavelength band of the illumination light by changing the optical filter or the like.

[0036] As shown in FIG. 3, in the present embodiment, the light source unit 20 has four-color LEDs: a V-LED (Violet Light Emitting Diode) 20a, a B-LED (Blue Light Emitting Diode) 20b, a G-LED (Green Light Emitting Diode) 20c, and an R-LED (Red Light Emitting Diode) 20d.

[0037] As shown in FIG. 4, the V-LED 20a generates violet light V with a central wavelength of 410 ± 10 nm and a wavelength range of 380 to 420 nm. The B-LED 20b generates blue light B with a central wavelength of 450 ± 10 nm and a wavelength range of 420 to 500 nm. The G-LED 20c generates green light G with a wavelength range of 480 to 600 nm. The R-LED 20d generates red light R with a central wavelength of 620 to 630 nm and a wavelength range of 600 to 650 nm.

[0038] The light source processor 21 controls the V-LED 20a, B-LED 20b, G-LED 20c, and R-LED 20d. When the light source processor 21 is set to the first observation mode and acquiring a normal image, and when it is set to the second observation mode, the light source processor 21 controls each of the LEDs 20a to 20d to emit first illumination light with a combination of light intensity ratios among the purple light V, blue light B, green light G, and red light R being Vc:Bc:Gc:Rc. The first illumination light is white light.

[0039] When the light source processor 21 is set to the first observation mode and acquiring a special image, the light source processor 21 controls each of the LEDs 20a to 20d to emit special light with a combination of light intensity ratios among the purple light V, blue light B, green light G, and red light R being Vs:Bs:Gs:Rs. The special light is, for example, the second illumination light. It is preferable that the second illumination light can obtain an endoscope image that emphasizes superficial blood vessels. Therefore, it is preferable that the light intensity of the purple light V is greater than the light intensity of the blue light B. For example, as shown in FIG. 5, the ratio of the light intensity Vs1 of the purple light V to the light intensity Bs1 of the blue light B is set to "4:1".

[0040] In this specification, the combination of light intensity ratios includes the case where the ratio of at least one semiconductor light source is 0 (zero). Therefore, it includes the case where any one or two or more of the semiconductor light sources are not lit. For example, even when only one of the semiconductor light sources is lit and the other three are not lit, such as when the combination of light intensity ratios among the purple light V, blue light B, green light G, and red light R is 1:0:0:0, it has a light intensity ratio and is one of the combinations of light intensity ratios.

[0041] As described above, the combinations of the light intensity ratios of the purple light V, blue light B, green light G, and red light R emitted in the first illumination light and the second illumination light, that is, the types of illumination light, are different from each other. Therefore, the first illumination light and the second illumination light have different spectra. The light source processor 21 controls each of the LEDs 20a to 20d so as to repeat a pattern composed of each of the plurality of illumination lights and irradiate the observation target with each of the plurality of illumination lights in a preset order.

[0042] In the present embodiment, in the first observation mode, illumination lights of different types, such as the first illumination light and the second illumination light, are automatically switched and emitted. In the second observation mode, the first illumination light is continuously emitted. When the first illumination light and the second illumination light are automatically switched and emitted, for example, a first period in which the first illumination light is continuously emitted and a second period in which the second illumination light is continuously emitted are alternately repeated. More specifically, after performing the first period of emitting the first illumination light for a preset number of frames, the second period of emitting the second illumination light is performed for a preset number of frames. Then, it becomes the first period again, and the pattern consisting of the first period and the second period is repeated. Note that three or more types of illumination lights may be switched and emitted. Also in this case, similar to the case of two types of illumination lights, the illumination lights are emitted for a period of a preset number of frames in each illumination light in a preset order, and this pattern is repeated. Note that the pattern of the illumination light may be a pattern in which the same type of illumination light is used two or more times, and there is no limitation on the illumination light to be used.

[0043] Note that a "frame" refers to a unit for controlling the imaging sensor 45 (see FIG. 2) that images the observation target, and "one frame" refers to a period including at least an imaging period in which imaging is performed with light from the observation target to expose the imaging sensor 45 and a readout period in which an image signal is read out. One endoscopic image taken corresponds to one frame. In the present embodiment, various periods such as the first period, the second period, or the third period are defined corresponding to the "frame" which is the unit of imaging. Note that the various periods such as the first, second, or third may be the same number of frames as any one of them or different numbers of frames from each other, and can be set without limitation.

[0044] As shown in FIG. 6, in this embodiment, the first period of the first illumination light L1 is irradiated for 2 frames (2FL), and then the second period of the second illumination light L2 is irradiated for 1 frame (1FL). The combination of the first illumination light L1 and the second illumination light L2 is defined as a pattern P1, and the pattern P1 is repeated.

[0045] The light emitted from each of the LEDs 20a to 20e is incident on the light guide 41 through an optical path coupling portion (not shown) composed of a mirror, a lens, or the like. The light guide 41 is built into the endoscope 12 and a universal cord (a cord connecting the endoscope 12, the light source device 13, and the processor device 14). The light guide 41 propagates the light from the optical path coupling portion to the tip portion 12d of the endoscope 12.

[0046] An illumination optical system 30a and an imaging optical system 30b are provided at the tip portion 12d of the endoscope 12. The illumination optical system 30a has an illumination lens 42, and the illumination light propagated by the light guide 41 is irradiated onto the observation target through the illumination lens 42. The imaging optical system 30b includes an imaging drive unit 36, an objective lens 43, a zoom lens 44, and an imaging sensor 45. Various lights such as reflected light, scattered light, and fluorescence from the observation target are incident on the imaging sensor 45 through the objective lens 43 and the zoom lens 44. Thereby, an image of the observation target is formed on the imaging sensor 45. The zoom lens 44 moves freely between the tele end and the wide end by operating the zoom operation unit 12f, and enlarges or reduces the observation target imaged on the imaging sensor 45.

[0047] The imaging sensor 45 is a color imaging device, and images the optical image of the observation target and outputs an image signal. In this embodiment, a CMOS (Complementary Metal - Oxide Semiconductor) image sensor is adopted. As shown in FIG. 7, a plurality of pixels 38 that generate pixel signals by photoelectric conversion are formed on the imaging surface 45a of the imaging sensor 45. The pixels 38 are two - dimensionally arranged in a matrix in the row direction (X - direction) and the column direction (Y - direction).

[0048] On the light incident side of the imaging sensor 45, a color filter array 39 is provided. The color filter array 39 has a blue (B) filter 39a, a green (G) filter 39b, and a red (R) filter 39c. Any one of these filters is arranged on each pixel 38. The color arrangement of the color filter array 39 is a Bayer array, where the G filter 39b is arranged every other pixel in a checkered pattern, and the B filter 39a and the R filter 39c are arranged in a square lattice pattern on the remaining pixels.

[0049] Hereinafter, the pixel 38 on which the B filter 39a is arranged is referred to as a B pixel, the pixel 38 on which the G filter 39b is arranged is referred to as a G pixel, and the pixel 38 on which the R filter 39c is arranged is referred to as an R pixel. In each pixel row of even numbers (0, 2, 4, ···, N - 1), B pixels and G pixels are alternately arranged. In each pixel row of odd numbers (1, 3, 5, ···, N), G pixels and R pixels are alternately arranged. Here, N is a positive integer, and a pixel row refers to one row of pixels 38 arranged in the row direction. A pixel column refers to one column of pixels 38 arranged in the column direction.

[0050] The color filter array 39 has the spectral characteristics shown in FIG. 8. The B filter 39a has a high light transmittance for a wavelength band of, for example, 380 nm to 560 nm. The G filter 39b has a high light transmittance for a wavelength band of, for example, 450 nm to 630 nm. The R filter 39c has a high light transmittance for a wavelength band of, for example, 580 nm to 760 nm.

[0051] The imaging sensor 45 is driven by an imaging drive unit 36, receives return light from an observation object illuminated by illumination light through the color filter array 39 by a plurality of pixels 38, and outputs an image signal. The imaging sensor 45 outputs a BGR image signal composed of a B pixel signal, a G pixel signal, and an R pixel signal as the image signal.

[0052] In this embodiment, a CMOS image sensor is used as the imaging sensor 45. Generally, a CMOS image sensor performs an imaging operation in a rolling shutter method. In the rolling shutter method, the imaging sensor 45 executes signal readout by a "sequential readout method". In the sequential readout method, for all pixels 38, signal readout is performed in order, one pixel row at a time, from the first pixel row "0" to the last pixel row "N".

[0053] As a reset method, the imaging sensor 45 can execute a "sequential reset method" and a "batch reset method". In the sequential reset method, reset is performed in order, one pixel row at a time, from the first pixel row "0" to the last pixel row "N". In the batch reset method, all pixel rows are reset simultaneously in a batch. In this embodiment, reset is performed by the sequential reset method.

[0054] In this embodiment, a rolling shutter type CMOS image sensor is used as the imaging sensor 45, but it is not limited to this, and a global shutter type CMOS image sensor may be used. Furthermore, as the imaging sensor 45, instead of a CMOS image sensor, a CCD (Charge Coupled Device) image sensor may be used.

[0055] Note that instead of the imaging sensor 45 provided with a primary color color filter, a complementary color imaging sensor provided with complementary color filters of C (cyan), M (magenta), Y (yellow), and G (green) may be used. When a complementary color imaging sensor is used, four-color image signals of CMYG are output. Therefore, by converting the four-color image signals of CMYG into three-color image signals of RGB by complementary color-primary color conversion, RGB image signals similar to those of the imaging sensor 45 can be obtained. Also, instead of the imaging sensor 45, a monochrome sensor without a color filter may be used.

[0056] The imaging sensor 45 is driven and controlled by the central control unit 51 (see Fig. 2) via the imaging drive unit 36. The central control unit 51 controls the light emission of the light source unit 20 through the light source processor 21 in synchronization with the driving of the imaging sensor 45. By controlling the imaging sensor 45 to capture the observation target illuminated by the first illumination light L1 which is white light, a Bc image signal is output from the B pixels of the imaging sensor 45, a Gc image signal is output from the G pixels, and an Rc image signal is output from the R pixels. Similarly, by controlling the imaging sensor 45 to capture the observation target illuminated by the second illumination light L2 which is special light, a Bs image signal is output from the B pixels of the imaging sensor 45, a Gs image signal is output from the G pixels, and an Rs image signal is output from the R pixels.

[0057] The central control unit 51 includes a frame rate control unit 63 (see Fig. 2). The frame rate indicates the number of frames per unit time, and the unit fps (frames per second) is used. The frame rate control unit 63 controls the frame rate such as the shooting frame rate when shooting an endoscope image or the display frame rate when displaying an endoscope image. The frame rate control unit 63 will be described later.

[0058] The CDS / AGC (Correlated Double Sampling / Automatic Gain Control) circuit 46 performs correlated double sampling (CDS) and automatic gain control (AGC) on the analog image signal obtained from the imaging sensor 45. The image signal that has passed through the CDS / AGC circuit 46 is converted into a digital image signal by the A / D (Analog / Digital) converter 47. The digital image signal after A / D conversion is input to the processor device 14.

[0059] In the processor device 14, a program related to processing such as imaging control or image processing is stored in a program memory (not shown). In the processor device 14, the central control unit 51 constituted by a processor or the like causes the program in the program memory to operate, thereby realizing the functions of the central control unit 51, the image acquisition unit 52, the DSP (Digital Signal Processor) 53, the noise reduction unit 54, the memory 55, the image processing unit 56, the display control unit 57, and the video signal generation unit 58. Further, the central control unit 51 receives information from the endoscope 12 and the light source device 13, and based on the received information, controls each part of the processor device 14 and also controls the endoscope 12 or the light source device 13. In addition, information such as an instruction from the keyboard 16 is also received.

[0060] As shown in FIG. 9, the frame rate control unit 63 provided in the central control unit 51 includes a frame rate adjustment unit 71, a shooting frame rate control unit 72, and a display frame rate control unit 73.

[0061] The frame rate adjustment unit 71 adjusts the shooting frame rate and the display frame rate so as to satisfy preset conditions. The conditions regarding the shooting frame rate and the display frame rate are set according to the type of the endoscope image to be acquired. Therefore, the conditions regarding the shooting frame rate and the display frame rate are set according to the type of the illumination light when acquiring the endoscope image.

[0062] In the first observation mode, the frame rate adjustment unit 71 sends an instruction to the shooting frame rate control unit 72 to shoot the observation target according to the first shooting frame rate during the first period when the first illumination light L1 is irradiated, and when displaying the obtained first image on the display 15, sends an instruction to the display frame rate control unit 73 to display the first image according to the first display frame rate.

[0063] Also, in the first observation mode, during the second period when the second illumination light L2 is irradiated, the frame rate adjustment unit 71 sends an instruction to the imaging frame rate control unit 72 to capture the observation target according to the second imaging frame rate, and when displaying the obtained second image on the display 15, it sends an instruction to the display frame rate control unit 73 to display the second image according to the second display frame rate.

[0064] A condition that the first display frame rate is higher than the first imaging frame rate is preset as a condition for the first image to be acquired by the frame rate adjustment unit 71. Therefore, the frame rate adjustment unit 71 sends instructions to the imaging frame rate control unit 72 and the display frame rate control unit 73 respectively, based on the preset first imaging frame rate and the first display frame rate that is higher than the first imaging frame rate.

[0065] Also, a condition that the second imaging frame rate is the same as the second display frame rate is preset as a condition for the second image to be acquired. Therefore, the frame rate adjustment unit 71 sends instructions to the imaging frame rate control unit 72 and the display frame rate control unit 73 respectively, based on the second display frame rate that is the same as the preset second imaging frame rate.

[0066] The imaging frame rate control unit 72 adjusts the frame rate of imaging based on the instruction regarding the first imaging frame rate sent from the frame rate adjustment unit 71. In the first observation mode, during the first period when the first illumination light L1 is irradiated, the imaging frame rate control unit 72 controls the imaging sensor 45 and the like to capture the observation target according to the first imaging frame rate. The operation of the display frame rate control unit 73 will be described later.

[0067] The image acquisition unit 52 acquires the digital image signal of the endoscope image input from the endoscope 12. The image acquisition unit 52 acquires the image signal of the observation target illuminated by each illumination light for each frame.

[0068] The acquired image signal is transmitted to the DSP53. The DSP53 performs digital signal processing such as color correction processing on the received image signal. The noise reduction unit 54 performs noise reduction processing on the image signal that has undergone color correction processing and the like by the DSP53, for example, by a moving average method or a median filter method. The image signal with reduced noise is stored in the memory 55.

[0069] The image processing unit 56 acquires the noise-reduced image signal from the memory 55. Then, signal processing such as color conversion processing, color enhancement processing, and structure enhancement processing is performed on the acquired image signal as necessary to generate a color endoscope image in which the observation target is shown. The image processing unit 56 includes a normal image processing unit 61 and a special image processing unit 62.

[0070] In the first observation mode and the second observation mode, the normal image processing unit 61 performs color conversion processing and image processing for normal images such as color enhancement processing and structure enhancement processing on the image signal for the normal image after noise reduction for one frame that is input. The image signal on which the image processing for the normal image has been performed is input to the display control unit 57 as a normal image.

[0071] In the first observation mode, the special image processing unit 62 performs color conversion processing and image processing for special images such as color enhancement processing and structure enhancement processing on the image signal of the special image after noise reduction for one frame that is input. The image signal on which the image processing for the special image has been performed is input to the display control unit 57 as a special image.

[0072] The endoscope image generated by the image processing unit 56 is a normal image or a special image, and the contents of the color conversion processing, color enhancement processing, and structure enhancement processing differ depending on the type of the endoscope image. In the case of a normal image, the image processing unit 56 performs the above various signal processes so that the observation target has a natural color tone to generate a normal image. In the case of a special image, the image processing unit 56 performs various signal processes such as emphasizing the blood vessels of the observation target, for example, to generate a special image.

[0073] Here, the color difference expansion process will be described in the case where the second image is a color difference expansion process image obtained by performing a color difference expansion process on an endoscopic image obtained by capturing an observation target while emitting the second illumination light L2. In the color difference expansion process, first, a first signal ratio (Bs / Gs) representing the ratio of the Bs image signal to the Gs image signal and a second signal ratio (Gs / Rs) representing the ratio of the Rs image signal to the Gs image signal are calculated. Then, based on the first signal ratio and the second signal ratio, a color difference expansion process for expanding the color difference between a plurality of observation target ranges is performed, and a color difference expansion image is generated based on the first signal ratio and the second signal ratio after the color difference expansion process. The color difference expansion process is a type of color enhancement process. The color difference expansion image becomes the second image.

[0074] Regarding the color difference expansion process, as shown in FIG. 10, it is preferable to expand the distance between a plurality of observation target ranges in a two-dimensional space composed of the first signal ratio (Bs / Gs) and the second signal ratio (Gs / Rs). Specifically, in the two-dimensional space, with the position of the first range (indicated by 1 surrounded by a circle) among the plurality of observation target ranges maintained before and after the color difference expansion process, the distance between the first range and the second range (indicated by 2 surrounded by a circle), the distance between the first range and the third range (indicated by 3 surrounded by a circle), and the distance between the first range and the fourth range (indicated by 4 surrounded by a circle) are preferably expanded. The color difference expansion process is preferably performed by a method of adjusting the radial distance and the angle after performing a polar coordinate transformation on the first signal ratio and the second signal ratio. Note that the first range is a normal part where no lesion or the like exists, and the second to fourth ranges are preferably abnormal parts where a lesion or the like may exist. Due to the color difference expansion process, the range E1 in the two-dimensional space before the color difference expansion process is expanded to the range E2 after the color difference expansion process, so the color difference is emphasized. For example, an image is obtained in which the color difference between the abnormal part and the normal part is emphasized. It is preferable to use this image as the second image.

[0075] Also, the pseudo-color process will be described in the case where the second image is a pseudo-color process image obtained by performing a pseudo-color process on an endoscopic image obtained by capturing an observation target while emitting the second illumination light L2. In the pseudo-color process, the Bs image signal is assigned to the B channel and the G channel for display, and the Gs image signal is displayed usingThis is a process of assigning to the R channel. By this pseudo-color processing, an image in which blood vessels or structures at a specific depth such as superficial blood vessels are emphasized can be obtained. This image may be used as the second image.

[0076] The display control unit 57 receives the endoscopic image generated by the image processing unit 56 and performs control for display on the display 15 according to the control of the central control unit 51. As shown in FIG. 11, the display control unit 57 includes a display image generation unit 74. The display image is an image for display on the display 15, and the display image generation unit 74 generates the display image from the endoscopic image generated by the image processing unit 56.

[0077] Regarding the frame rate when displaying the display image, the display control unit 57 follows the instructions of the frame rate adjustment unit 71 provided in the central control unit 51. The display control unit 57 generates the display image according to the instructions of the frame rate adjustment unit 71, thereby performing control for displaying the endoscopic image on the display 15.

[0078] The instructions of the frame rate adjustment unit 71 are made for each type of endoscopic image. Therefore, in a specific type of endoscopic image, the instruction from the frame rate adjustment unit 71 is, for example, a rate at which the shooting frame rate and the display frame rate are different. If necessary, the display control unit 57 adjusts the number of display images generated by the display image generation unit 74, thereby controlling the image displayed on the display 15 according to the instructions of the frame rate adjustment unit 71.

[0079] Instructions for a specific type of endoscopic image by the frame rate adjustment unit 71 specifically include the following three cases. First, in the instruction from the frame rate adjustment unit 71, the display frame rate is the same as the shooting frame rate. In this case, the display image generation unit 74 generates the captured endoscopic image as the display image as it is. Second, in the instruction from the frame rate adjustment unit 71, the display frame rate is lower than the shooting frame rate. In this case, the display image generation unit 74 generates, as the display image, an endoscopic image selected from the captured endoscopic images so as to match the display frame rate. Third, in the instruction from the frame rate adjustment unit 71, the display frame rate is higher than the shooting frame rate. In this case, the display image generation unit 74 generates the captured endoscopic image as the display image as it is, and newly generates a display image based on the captured endoscopic image, and uses these together as the display image.

[0080] In the present embodiment, for the first image, the frame rate adjustment unit 71 sends a condition to the display control unit 57 that the display frame rate is higher than the shooting frame rate. For the second image, shooting is performed but not displayed, that is, a specific imaging frame rate where the imaging frame rate is higher than 0 fps and the display frame rate is 0 fps is sent to the display control unit 57. In this case, in order for the display image generation unit 74 to newly generate a first display image based on the captured first image, for example, frame interpolation is performed based on the captured past first images.

[0081] In frame interpolation, an endoscopic image generated based on a captured past endoscopic image is used as an interpolation frame. The method for generating the interpolation frame can be selected according to the case. For example, methods such as an addition average method, a motion vector method, or a replication method performed using the captured endoscopic image can be mentioned. By using the interpolation frame together with the frame of the original captured endoscopic image, the number of frames of the endoscopic image to be displayed can be increased to satisfy the condition that the display frame rate is higher than the shooting frame rate.

[0082] In the method performed by the addition averaging method, for example, when using a plurality of past endoscopic images that have been taken to generate a new image as a display image, the average value of the image signal values is calculated for each of the endoscopic images to be used, and the display image is generated based on the value obtained by additively averaging using the number of endoscopic images for which these average values are used, and this can be used as an interpolation frame. Additive averaging means simple averaging, but in some cases, weighted averaging in which weighting is performed on any of the endoscopic images may be used. The weighting ratio can be set in advance. Also, it is not limited to using two endoscopic images, and additive averaging may be performed using three or more endoscopic images to generate a display image.

[0083] In the present embodiment, according to the instruction of the frame rate adjustment unit 71, for the first image, the first imaging frame rate, which is the imaging frame rate of the first image, is set to 30 fps, and the first display frame rate, which is the display frame rate of the first image, is set to 60 fps. Also, according to the instruction of the frame rate adjustment unit 71, for the second image, the second imaging frame rate, which is the imaging frame rate of the second image, is set to 15 fps, and the second display frame rate, which is the display frame rate of the second image, is set to 0. Note that the first illumination light L1 and the second illumination light L2 repeat the irradiation pattern P1 in a pattern of two frames and one frame (FIG. 6), and the imaging frame rate and the display frame rate of the first image and the second image also repeat the frame rates of the values of the above instructions.

[0084] As shown in FIG. 12, in the imaging frame, the first image 81 is taken in two frames by the first illumination light L1 in the first period, and then the second image 82 is taken in one frame by the second illumination light L2 in the subsequent second period. Then, this combined pattern P1 is repeated. The numbers attached to the first image 81, the interpolation frame 83, and the second image 82 described in the columns of the imaging frame, the interpolation frame, and the display frame are numbered from 1 to 9 in the order of the imaging time. The second image 82 is shaded. The interpolation frame is shaded differently from the second image 82.

[0085] As shown in the column of the shooting frame in FIG. 12, for the first image 81, the frame rate adjustment unit 71 sets the shooting frame rate to 30 fps and the display frame rate to 60 fps, and for the second image 82, the shooting frame rate is set to 15 fps and the display frame rate is set to 0 fps. Therefore, as shown in the column of the interpolated frame, the display image generation unit 74 generates a new display image as the interpolated frame 83 at a rate of 30 fps based on the captured first image 81. Then, as shown in the column of the display frame, the captured first image 81 and the interpolated frame 83 are combined to 60 fps and displayed on the display 15. As described above, all the display frames are images based on the first image 81. Therefore, the first display image includes an image that does not change the first image 81 and the interpolated frame 83 obtained by changing the first image 81.

[0086] In this embodiment, the interpolated frame 83 is generated by the addition average method. The interpolated frame 83 marked as "1 + 2" indicates that it is an interpolated frame 83 generated by the addition average of two first images 81, namely the first first image 81 and the second first image 81. The first first image 81 and the second first image 81 take the addition average of the image signals at a ratio of 50:50 te However, the ratio is not limited to this. When displaying the interpolated frame 83, as the display frame, the interpolated frame 83 is arranged between the first images 81 used to generate the interpolated frame 83 in time series. In the motion vector method, the replication method, or the like, the interpolated frame 83 can be arranged and displayed in the same manner. As described above, according to the first display frame rate higher than the first imaging frame rate, the first display image, which is the display image of the first image 81, can be generated and the conditions of the first imaging frame rate and the first display frame rate can be satisfied.

[0087] In the method using the motion vector method, for example, when using a plurality of past endoscopic images that have been captured to generate a new image as a display image, the average value of the image signal values is calculated for each of the endoscopic images to be used, and based on the motion vectors calculated based on each of these endoscopic images, the endoscopic image is weighted-averaged, and a display image is generated based on the obtained value, and this can be used as an interpolation frame.

[0088] As shown in FIG. 13, when performing the method using the motion vector method, the processor device 14 includes a motion detection unit 84. The motion detection unit 84 may detect the relative motion between the imaging sensor 45 and the observation target by a physical method such as a gyro sensor in addition to calculating the motion vector based on the endoscopic image. Further, when calculating the motion vector based on the endoscopic image, it may be calculated based on a structure such as a shape, blood vessel, lesion, or incision shown in the endoscopic image, or a landmark such as a treatment tool.

[0089] In the method using the duplication method, a past endoscopic image that has been captured is duplicated to generate a display image, and this can be used as an interpolation frame. The endoscopic image to be duplicated may be the endoscopic image captured immediately before, or an endoscopic image captured in the past other than immediately before. In the addition average method, when using the two immediately preceding endoscopic images, if the ratio is set to 100:0, it is the same as generating an interpolation frame by the duplication method using the endoscopic image captured two frames before, and if it is set to 0:100, it is the same as generating an interpolation frame by the duplication method using the single endoscopic image captured immediately before. Further, the endoscopic image to be duplicated may be duplicated to generate one interpolation frame or two or more interpolation frames according to the relationship between the set imaging frame rate and the display frame rate.

[0090] By the method as described above, the display image generation unit 74 generates a display image according to the instruction of the frame rate adjustment unit 71, whereby the display frame rate can be adjusted. Note that the past endoscopic images used when the display image generation unit 74 generates the interpolation frame 83 as a display image can be one or two or more frames immediately before the timing of generating the interpolation frame 83, but in some cases, three or more frames may be used. Also, instead of two consecutive images taken immediately before the generation of the interpolation frame 83, endoscopic images taken in the past at a temporally distant time may be used.

[0091] The display image generated in the display control unit 57 is generated in the video signal generation unit 58 into a video signal for display on the display 15 and sent to the display 15. The display 15 displays the display image sent from the video signal generation unit 58.

[0092] As a method of displaying the display image in the first observation mode, when displaying two types of endoscopic images, i.e., the display image based on the first image 81 and the display image based on the second image 82, they may be displayed on the same display 15, or for example, they may be respectively displayed on different displays. Also, since the display image based on the first image 81 is adjusted to a state where the display frame rate is higher than the shooting frame rate and is thus smoothly displayed, the display areas of the first image 81 and the second image 82 may be made different, such as displaying with a larger display area compared to the display area of the display image based on the second image 82.

[0093] As shown in FIG. 14, in the first observation mode, when displaying the second image 82a obtained by performing color difference expansion processing as the first image 81 and the second image 82, the first image 81 is displayed in a large-area region of the display 15, and the second image 82a is displayed in a small-area region. Since the first image 81 is an image with a natural color tone by white light, for example, it is an image familiar to a doctor who performs an examination. Therefore, since the first display frame rate of the first image 81 is adjusted to be higher than the first shooting frame rate, it is smooth and easy to view, and it can be viewed without discomfort even when displayed in a large-area region. In FIG. 14, the observation target shown in the first image 81 and the second image 82a has erythema 92.

[0094] Since the second image 82a is an image in which the structure such as superficial blood vessels is emphasized and displayed by special light and color enhancement display processing, lesions where blood vessels are dense are emphasized and displayed. Therefore, even when the second image 82a is displayed in a relatively small-area region, since the lesions and the like are emphasized and displayed, it is possible to prevent overlooking the lesions and the like by displaying them side by side with the first image 81.

[0095] In the first observation mode, the first image 81 may be displayed, and the second image 82b obtained by performing pseudo-color processing may be displayed. When the second image 82b is displayed, an image analysis result screen 91 using the second image 82b may be further displayed.

[0096] As shown in FIG. 15, for example, in the first observation mode, the first image 81 is displayed in a large-area region of the display 15, and a second image 82b subjected to pseudo-color processing in a small-area region, and an image analysis result screen 91 using the second image 82b are displayed. The redness 92 included in the second image 82b is subjected to image analysis using AI (artificial intelligence) and / or image analysis for measuring physical quantities such as oxygen saturation or distance measurement, and the result is shown, for example, as a lesion region 93, by the image analysis result screen 91. Therefore, while the doctor proceeds with the observation using the easy-to-see first image 81 with the display frame rate adjusted using white light, the doctor can make a diagnosis while immediately obtaining, from the image analysis result screen 91, information regarding whether the redness 92 is, for example, tumorous or non-tumorous. Also, the image for which the image analysis is performed may be any type of endoscopic image to be acquired, and can be, for example, any one or more of the first image, the second image, and / or the third image. At this time, the image analysis may be performed simultaneously, or may be performed in parallel by different image analysis units respectively. Also, the image analysis can be performed using a device other than the processor device 14. Thus, since appropriate types of endoscopic images can be acquired according to various image analyses and used for each image analysis, the accuracy of the image analysis can be improved.

[0097] Note that, as shown in FIG. 16, in the first observation mode, only the second image 82b obtained by performing pseudo-color processing on a small-area region while displaying the first image 81 in a large-area region of the display 15 may be displayed. Also, in the first observation mode, when the first image 81 is displayed and the second image 82 is not displayed, it is not necessary to display anything in the small-area region while displaying the first image 81 in the large-area region of the display 15, or an image other than the second image 82 may be displayed. Also, the first image 81 may be displayed in the small-area region and the second image 82 may be displayed in the large-area region. Note that the second image 82 is a general term for the second image and includes the second image 82a and the second image 82b.

[0098] Next, a series of operations in the first observation mode of the endoscope system according to the present embodiment will be described with reference to the flowchart shown in FIG. 17. Starting an endoscope examination in the first observation mode, various frame rates are set (step ST110). The frame rates set in the first observation mode are the first shooting frame rate and the first display frame rate of the first image 81, and the second shooting frame rate and the second display frame rate of the second image 82. After setting various frame rates, observation in the first observation mode is started (step ST120). Since the illumination light is irradiated onto the observation target by the pattern P1, the first image 81 is acquired first. The first image 81 is acquired according to the first shooting frame rate (step ST130).

[0099] Next, the second image 82 is acquired according to the second shooting frame rate (step ST140). After the first image 81 is acquired, a first display image is generated according to the first display frame rate (step ST150). When the second image 82 is to be displayed (Y in step ST160), a second display image is generated according to the second display frame rate (step ST170). Next, the generated first display image and second display image are displayed on the display 15. When the second image 82 is not to be displayed (N in step ST160), the first display image is displayed.

[0100] Since the endoscopic system 10 is configured as described above, even when two types of endoscopic images are acquired, the first image 81 can suppress a decrease in image quality during display by adjusting the display frame, and at the same time, a high-quality image can be obtained without subsampling pixels or the like in the second image 82. This can also be applied when three or more types of endoscopic images are acquired. A plurality of types of endoscopic images can be acquired according to the purpose, and each endoscopic image can be acquired with an image quality suitable for the individual purpose. As a result, it becomes possible to display two or more types of endoscopic images with suppressed image quality degradation, and without requiring special effort, the doctor can use a plurality of endoscopic images with different illumination lights and the like acquired during the examination to proceed with the examination while acquiring a plurality of image analysis results. Therefore, the doctor can proceed with the diagnosis using high-quality endoscopic images while simultaneously obtaining a plurality of image analysis results useful for diagnosis as diagnostic support information.

[0101] The imaging sensor 45 is an imaging sensor 45 that performs signal reading by a sequential reading method and performs reset by a sequential reset method, and performs an imaging operation by a rolling shutter method. However, in the above imaging operation by the rolling shutter method, the exposure period may be adjusted by adjusting the irradiation period, which is the period during which the illumination light is irradiated, and the imaging operation may be performed by a pseudo global shutter method.

[0102] In this case, each of the plurality of illumination lights may be intermittently irradiated under the control of the light source unit 20. Intermittent irradiation means irradiating so as to provide a light-off period before and after the irradiation period. Preferably, the irradiation period is included at least once in the period of one frame. Therefore, preferably, in the period of one frame, the irradiation period and the light-off period are each included at least once.

[0103] As shown in FIG. 18, in the pseudo global shutter method, the illumination light is irradiated during the irradiation period C2, and then turned off during the light-off period B2, and this is repeated. In the imaging sensor 45, exposure is performed during the exposure period C1, which is the same period as the irradiation period C2, and then reading is performed during the reading period B1, which is the same period as the light-off period B2 during which the illumination light is turned off. Note that in FIG. 18, the sequential reading method in which pixel rows are sequentially read out as time elapses is indicated by diagonal lines.

[0104] The illumination light is intermittently irradiated during the irradiation period C2 in accordance with the exposure period C1 of the imaging sensor. In FIG. 18, in the line of the imaging sensor 45, the region exposed by the illumination light is schematically shown by hatching. By adopting such a pseudo global shutter method, it is preferable because problems such as color mixing do not occur when the illumination light is switched. The imaging period A1 is a period including the exposure period C1 and the reading period B1, and is the period of one frame.

[0105] Note that the imaging period A1 during which the imaging sensor 45 performs imaging and the reading period B1 during which the image signal obtained by imaging is read out are such that the imaging period A1 is longer than the reading period B1. Thereby, an exposure period for exposing all lines of the imaging sensor 45 can be provided within one frame.

[0106] For example, the imaging period A1 can be set to 1 / 45 sec (seconds). Also, the exposure period C1 and the reading period B1 can each be set to 1 / 90 sec. Therefore, it can be said that the shooting frame rate in this case is 45 fps.

[0107] Note that the light source unit 20 may change the irradiation period C2 for irradiating any of the plurality of illumination lights to perform illumination light irradiation. For example, the irradiation period C2 when irradiating the illumination light intermittently is selected by the user from 1 / 90 sec, 1 / 200 sec, or 1 / 400 sec.

[0108] In this case, the exposure period C1 may be changed based on the changed irradiation period C2. That is, the shutter speed can be controlled by the irradiation period C2 of the illumination light. The shutter speed controlled by the irradiation period C2 of the illumination light can be, for example, 1 / 90 sec, 1 / 200 sec, or 1 / 400 sec.

[0109] In this embodiment, the first illumination light and the second illumination light are switched and emitted (see FIG. 6), and the pattern P1 of continuously shooting two frames with the first illumination light L1 and then switching to the second illumination light L2 to shoot one frame is repeated. Therefore, out of 45 fps, the first image 81 by the first illumination light L1 is acquired at 30 fps, and the second image 82 by the second illumination light L2 is acquired at 15 fps.

[0110] Also, the emission amount can be calculated by the emission integral amount composed of the irradiation period C2 during which the light source unit 20 irradiates the illumination light and the instantaneous emission amount D1 which is the emission amount per unit time of the illumination light. Therefore, the emission amount of the illumination light irradiated by the light source unit 20 in one irradiation of the illumination light can be controlled.

[0111] Also, the emission amount can be controlled by APC (Automatic Power Control). Similarly in this case, the emission amount can be calculated by the emission integral amount composed of the irradiation period C2 during which the light source unit 20 irradiates the illumination light and the instantaneous emission amount D1 which is the emission amount per unit time of the illumination light, and can be controlled by the irradiation period C2 and / or the instantaneous emission amount D1 of the illumination light. Note that a light meter or the like (not shown) may be used to acquire the instantaneous emission amount D1.

[0112] As described above, the endoscope system 10 can be provided with a second observation mode that can be switched with the first observation mode. In the second observation mode, the illumination light is not switched, and the observation target is observed with the same illumination light. In the second observation mode, the third illumination light included in the plurality of illumination lights is irradiated onto the observation target, and the imaging sensor 45 is controlled to capture the observation target according to a preset third imaging frame rate during a third period when the third illumination light is irradiated. The imaging sensor 45 acquires a third image captured during the third period. When the third image is displayed on the display, it is preferable to display the third image according to a third display frame rate that is equal to or lower than the third imaging frame rate. The third imaging frame rate is the frame rate at which imaging is performed during the irradiation of the third illumination light, and the third display frame rate is the frame rate when the third image is displayed on the display 15.

[0113] In the second observation mode, in order to display the third image according to the third display frame rate that is equal to or lower than the third imaging frame rate, interpolation frames are not generated. Therefore, in the second observation mode, display images are not generated.

[0114] Also, in the second observation mode, the imaging sensor 45 performs an imaging operation by a rolling shutter method. As shown in FIG. 19, as the rolling shutter method, the illumination light is constantly irradiated. In the imaging sensor 45, constant exposure is performed, and pixel rows are sequentially read out over time. A period A2 from the first pixel row to the last pixel row until the completion of readout after exposure is one frame. For example, the imaging period A2 can be set to 1 / 60 sec. Therefore, in this case, the imaging frame rate is 60 fps.

[0115] In the second observation mode, since the imaging sensor 45 performs an imaging operation by the rolling shutter method as described above, the imaging period A2 during which the imaging sensor 45 performs imaging is a period obtained by combining the exposure period and the readout period for reading out the image signal obtained by imaging, and is a period of one frame. In the present embodiment, the period of one frame, which is the imaging period A2, is 1 / 60 sec.

[0116] Also, in the second observation mode, since the illumination light is always on, the light amount can be controlled by the emission amount D2 of the illumination light. Similar to the first observation mode, the emission amount D2 can be controlled by the APC, and in this case as well, the light amount can be controlled by the emission amount D2 of the illumination light. Also, the shutter speed can be controlled by the imaging period A2 of the imaging sensor 45.

[0117] In the present embodiment, in the second observation mode, the third imaging frame rate by the third illumination light can be, for example, 60 fps. The third display frame rate is equal to or lower than the third imaging frame rate, and can be, for example, 60 fps, 50 fps, or 59.94 fps.

[0118] Note that in the above embodiment, the present invention is applied to the case of processing endoscopic images, but the present invention can also be applied to a processor device, a medical image processing device, or a medical image processing system that processes medical images other than endoscopic images.

[0119] Note that in the present embodiment, in the first observation mode, two types of images, the first image and the second image, are acquired, but three or more types of images may be acquired. Also, in the first observation mode, tumors and non-tumors are discriminated by image analysis using AI for the second image, and the result is displayed on the display 15. However, image analysis may be performed using any of the acquired multiple types of images, or image analysis may be performed on generated images such as interpolation frames. That is, image analysis may be performed on one or two or more of the three types of endoscopic images, the first image, the interpolation frame based on the first image, and the second image.

[0120] As described above, among the plurality of types of acquired images, in one or two or more types of endoscopic images, analysis based on image information, for example, acquisition of diagnostic support information by AI, or acquisition of physical quantity measurement information such as oxygen saturation or measurement of the distance in the subject, etc. may be performed. As the endoscopic image used for image analysis, according to the type of image analysis, the type of endoscopic image that can obtain good analysis results can be selected and used. In this case, the plurality of analyses may be performed simultaneously or in parallel.

[0121] For example, in the first observation mode, a first image which is a normal image, a second image which is a special image, and a third image which is a special image different from the second image are acquired, and the oxygen saturation of the subject is calculated by performing an analysis using the first image and the second image, the detection of the lesion possessed by the subject is performed by performing another analysis using the second image, and diagnostic information regarding the malignancy of the detected lesion can be acquired by performing yet another analysis using the third image. In this way, by performing different image analysis processes using a plurality of types of endoscopic images, it is possible to automatically obtain a plurality of diagnostic information during the observation by the endoscope.

[0122] Also, the analysis may be performed in the processor device 14 or may be performed using another device. For example, when performing a plurality of analyses, the analysis may be performed in an image processing device (not shown) connected to the processor device 14. For the analysis result, a display 15 may be connected to the image processing device, and the image processing device may perform control to display the analysis result on the display 15. In this case, endoscopic images such as the first image 81, the second image 82, and / or the interpolation frame 83 to be displayed on the display 15 may be sent from the processor device 14 to the image processing device, and the image processing device may perform control to display these images and the analysis result on the display 15.

[0123] In addition, when displaying the endoscopic image and / or the analysis result, etc., it may be displayed on a plurality of displays 15, or it may be displayed on a small portable terminal device such as a tablet (not shown). At the time of display, the screen layout, etc. can be set in advance according to the device to be displayed.

[0124] In the above embodiment, the hardware structure of the processing unit that executes various processes such as the central control unit 51, the image acquisition unit 52, the DSP 53, the noise reduction unit 54, the image processing unit 56, the display control unit 57, and the video signal generation unit 58 included in the processor device 14 is various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (program) and functions as various processing units, a programmable logic device (PLD), which is a processor such as an FPGA (Field Programmable Gate Array) whose circuit configuration can be changed after manufacture, and an application-specific electric circuit, which is a processor having a circuit configuration designed specifically for executing various processes.

[0125] One processing unit may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same or different types (for example, a plurality of FPGAs, or a combination of a CPU and an FPGA). Also, a plurality of processing units may be composed of one processor. As an example of configuring a plurality of processing units with one processor, first, as represented by a computer such as a client or a server, one processor is configured by a combination of one or more CPUs and software, and this processor functions as a plurality of processing units. Second, as represented by a System On Chip (SoC), there is a form in which a processor that realizes the functions of the entire system including a plurality of processing units with one IC (Integrated Circuit) chip is used. Thus, various processing units are configured using one or more of the above various processors as a hardware structure.

[0126] Furthermore, the hardware structure of these various processors is, more specifically, an electrical circuit (circuitry) in the form of a combination of circuit elements such as semiconductor elements.

Explanation of Signs

[0127] 10 Endoscope system 12 Endoscope 12a Insertion part 12b Operation part 12c Bending part 12d Tip part 12e Angle knob 12f Zoom operation part 12g Mode switch 12h Forceps opening 12i Freeze switch 13 Light source device 14 Processor device 15 Display 16 Keyboard 20 Light source part 20a V-LED 20b B-LED 20c G-LED 20d R-LED Processor for 21 light sources 30a Illumination optical system 30b Imaging optical system 36 Imaging drive unit 38 Pixel 39 Color filter array 39a Blue filter 39b Green filter 39c Red filter 41 Light guide 42 Illumination lens 43 Objective lens 44 Zoom lens 45 Imaging sensor 45a Imaging surface 46 CDS / AGC circuit 47 A / D converter 51 Central control unit 52 Image acquisition unit 53 DSP 54 Noise reduction unit 55 Memory 56 Image processing unit 57 Display control unit 58 Video signal generation unit 61 Normal image processing unit 62 Special image processing unit 63 Frame rate control unit 71 Frame rate adjustment unit 72 Shooting frame rate control unit 73 Display frame rate control unit 74 Display image generation unit 81 First image 82, 82a, 82b Second image 83 Interpolation frame 84 Motion detection unit 91 Image analysis result screen 92 Flushing 93 Lesion area A1, A2 Imaging period B1 Readout period B2 Extinguishing period C1 Exposure period C2 Irradiation period D1 Instant light emission amount D2 Emission light Amount E1, E2 Range L1 First illumination light L2 Second illumination light P1 Pattern X Row direction Y Column direction ST110~ST180 Steps

Claims

1. An endoscope having an imaging unit, a light source unit that irradiates a subject with each of a first illumination light and a second illumination light having a spectrum different from that of the first illumination light, and a processor device having a processor, wherein the processor controls the light source unit so as to intermittently irradiate the subject with each of the first illumination light and the second illumination light in a first observation mode, and controls the imaging unit so as to photograph the subject according to a preset first imaging frame rate during a first period in which the first illumination light is irradiated, and photograph the subject according to a preset second imaging frame rate during a second period in which the second illumination light is irradiated, sequentially acquires a plurality of first images photographed by the imaging unit during the first period, sequentially acquires a plurality of second images photographed by the imaging unit during the second period, sequentially displays the acquired plurality of first images in a first area of a display according to a first display frame rate higher than the first imaging frame rate, sequentially displays the acquired plurality of second images in a second area different from the first area of the display according to a second display frame rate lower than the first display frame rate, wherein the first imaging frame rate is higher than the second imaging frame rate, and the first area is larger than the second area. An endoscope system.

2. The endoscope system according to claim 1, wherein the processor controls the light source unit so as to repeatedly irradiate the subject with a pattern composed of each of a plurality of illumination lights.

3. The endoscope system according to claim 1, wherein the processor controls the light source unit so as to repeatedly irradiate the subject with the first illumination light or the second illumination light with a pattern composed of the first illumination light and the second illumination light.

4. The endoscope system according to any one of claims 1 to 3, wherein the processor controls the imaging unit so that the imaging period is longer than the reading period in a period of one frame composed of an imaging period in which the imaging unit performs imaging and a reading period in which an image signal obtained by imaging is read out.

5. The endoscope system according to claim 4, wherein the processor controls the light source unit so that at least once each of the extinguishing and irradiation of any one of a plurality of illumination lights is performed during the imaging period.

6. The endoscope system according to claim 5, wherein the processor controls the light source unit to perform irradiation of the illumination light by changing an irradiation period for performing irradiation of any one of the plurality of illumination lights.

7. The endoscope system according to claim 6, wherein the processor controls the imaging unit to change an exposure period based on the changed irradiation period.

8. The endoscope system according to any one of claims 1 to 7, wherein the processor controls a light emission amount of illumination light irradiated by the light source unit in one irradiation.

9. The endoscope system according to claim 8, wherein the light emission amount is calculated by an irradiation period during which the light source unit performs irradiation of the illumination light and an instantaneous light emission amount which is a light emission amount per unit time of the illumination light.

10. In a second observation mode that is switchable with the first observation mode, the processor controls the light source unit to irradiate a third illumination light included in the plurality of illumination lights to the subject, controls the imaging unit to photograph the subject according to a preset third imaging frame rate during a third period when the third illumination light is irradiated, acquires a third image photographed by the imaging unit during the third period, When the third image is displayed on a display, generates a display third image according to a third display frame rate that is equal to or lower than the third imaging frame rate. The endoscope system according to any one of claims 1 to 9.

11. The endoscope system according to any one of claims 1 to 10, wherein the first illumination light is white light, and the second illumination light has a higher light intensity ratio of purple light to green light than the first illumination light.

12. An operating method of an endoscope system including an endoscope having an imaging unit, a light source unit that irradiates a subject with each of a first illumination light and a second illumination light having a spectrum different from that of the first illumination light, and a processor device having a processor, the method comprising: a step in which the processor controls the light source unit to intermittently irradiate the subject with each of the first illumination light and the second illumination light in a first observation mode; a step of controlling the imaging unit to photograph the subject according to a preset first imaging frame rate during a first period when the first illumination light is irradiated, and photograph the subject according to a preset second imaging frame rate during a second period when the second illumination light is irradiated; a step of sequentially acquiring a plurality of first images photographed by the imaging unit during the first period; The step of sequentially acquiring a plurality of second images captured by the imaging unit during the second period; The step of sequentially displaying the acquired plurality of first images in a first area of the display according to a first display frame rate higher than the first imaging frame rate; The step of sequentially displaying the acquired plurality of second images in a second area different from the first area of the display according to a second display frame rate lower than the first display frame rate, and The first imaging frame rate is higher than the second imaging frame rate, and the first area is larger than the second area. A method of operating an endoscope system.

Images