Endoscope device, processor, color emphasis method

Abstract

An endoscope device (1) has a light source device (3) that emits light of a plurality of colors at a first light amount ratio / second light amount ratio to generate first illumination light / second illumination light, an endoscope (2) that has an imaging element (21) that images return light to generate an imaging signal, and a processor (4). The processor (4) causes the light source device (3) to emit light while switching the first illumination light and the second illumination light, generates a first image signal from the imaging signal related to the first illumination light, generates a second image signal from the imaging signal related to the second illumination light, and performs color emphasis on the first image signal based on the second image signal. The second illumination light is light obtained by adjusting the second light amount ratio in a manner such that the second image signal related to a reference portion becomes substantially achromatic.

Description

Technical Field

[0001] This invention relates to an endoscopic apparatus, processor, and method for color enhancement of an image obtained by emitting multiple lights with different center wavelengths. Background Technology

[0002] Previously, endoscopic devices that clearly display images of superficial blood vessels and fine surface structures have been proposed, making a significant contribution to the diagnosis of lesions.

[0003] For example, Japanese Patent Application Publication No. 2013-176 describes the following technique: obtaining a normal light image by taking images of a subject with equal light intensity values ​​of B light, G light, and R light; obtaining a special light image by taking images of a subject with light intensity values ​​of B light > G light > R light; and combining the special light image and the normal light image to obtain a composite image in which surface microvessels are clearly defined.

[0004] In addition, for example, Japanese Patent No. 6050286 discloses the following technique: In a feature space formed by multiple color information, the coordinates of a specific range within the first to third ranges where the observed object is distributed in the test body are moved into the reference range, and the two ranges outside the specific range are moved to separate them from each other, thereby generating an image that emphasizes the color difference between the normal and abnormal parts.

[0005] Furthermore, for example, Japanese Patent No. 3228627 discloses a technique for emphasizing IHb color as follows: calculating the hemoglobin concentration (IHb) and the average value of IHb <IHb>, and emphasizing the deviation of IHb from the average value <IHb>.

[0006] However, in the above-mentioned prior art, the hue of the processed image changes significantly, the color reproducibility is reduced, and it becomes an image with a different color reproduction than usual, which requires getting used to in diagnosis.

[0007] The present invention was made in view of the above circumstances, and its object is to provide an endoscope device, processor, and color emphasis method that can emphasize the display of minute color differences without reducing color reproducibility. Summary of the Invention

[0008] Methods for solving problems

[0009] An endoscope device according to one aspect of the present invention comprises: a light source device having a plurality of semiconductor light-emitting elements having different center wavelengths of emitted light, wherein the plurality of semiconductor light-emitting elements emit light at a certain light intensity ratio to generate illumination light; an endoscope having an imaging element that captures an image of return light from a subject irradiated by the illumination light, generating an imaging signal having multiple color components; and a processor connected to the light source device and the endoscope, the processor performing the following processing: controlling the light source device to switch between a first illumination light and a second illumination light to emit light, wherein the light intensity ratio of the first illumination light is set for observation. The subject has a first light intensity ratio, and the second illumination light has a second light intensity ratio that is different from the first light intensity ratio. A first image signal is generated by receiving a first imaging signal related to the first illumination light from the endoscope, and a second image signal is generated by receiving a second imaging signal related to the second illumination light from the endoscope. A corrected image signal is generated based on the second image signal and the first image signal is color-enhanced. The second illumination light is light obtained by adjusting the second light intensity ratio in a way that makes the second image signal related to the reference portion of the subject colorless within a specified error range.

[0010] In one aspect of the present invention, a processor controls a light source device to switch between a first illumination light and a second illumination light to emit light. The first illumination light has a light intensity ratio of multiple light rays with different center wavelengths set as a first light intensity ratio for observing a subject. The second illumination light has a light intensity ratio set as a second light intensity ratio different from the first light intensity ratio. A first image signal is generated based on a first imaging signal obtained by imaging the return light from the subject illuminated by the first illumination light. A second image signal is generated based on a second imaging signal obtained by imaging the return light from the subject illuminated by the second illumination light. A corrected image signal is generated based on the second image signal, after color enhancement of the first image signal. The second light intensity ratio is adjusted to make the second image signal, related to a reference portion of the subject, colorless within a specified error range, thereby controlling the light source device.

[0011] In one aspect of the color enhancement method of the present invention, a first illumination light and a second illumination light are switched to emit light. The light intensity ratio of multiple lights with different center wavelengths in the first illumination light is set as a first light intensity ratio for observing the subject. The light intensity ratio in the second illumination light is set as a second light intensity ratio different from the first light intensity ratio. A first image signal is generated based on a first imaging signal obtained by imaging the return light from the subject illuminated by the first illumination light. A second image signal is generated based on a second imaging signal obtained by imaging the return light from the subject illuminated by the second illumination light. A corrected image signal is generated by color-enhancing the first image signal based on the second image signal, so that the second image signal related to a reference portion of the subject becomes colorless within a specified error range, and the second light intensity ratio is adjusted accordingly. Attached Figure Description

[0012] Figure 1 This is a diagram showing the structure of the endoscope device according to the first embodiment of the present invention.

[0013] Figure 2 This is a block diagram showing the structure of the color-emphasis section in the first embodiment described above.

[0014] Figure 3 This is a flowchart illustrating the color emphasis display processing performed by the endoscope device of the first embodiment described above.

[0015] Figure 4 The first embodiment described above is a timing diagram showing the alternating emission of white light WL and color-emphasis light CE when the color-emphasis mode is enabled.

[0016] Figure 5 The timing diagram relating to the first embodiment described above is an example of a case in which white light WL is emitted when the color emphasis mode is turned off.

[0017] Figure 6 The first embodiment described above is a diagram showing an example of setting the light intensity of each LED in the first illumination light and the second illumination light when the color emphasis mode is turned on in normal viewing mode.

[0018] Figure 7 The diagram relating to the first embodiment described above is an example of a typical observation mode in which the light intensity ratio of each LED in the second illumination light differs depending on the location of the subject.

[0019] Figure 8The first embodiment described above is illustrated by a diagram showing an example of a typical observation mode in which the light intensity ratio of each LED in the second illumination light varies depending on the location of the subject and the distance from the tip of the endoscope to the location of the subject.

[0020] Figure 9 The first embodiment described above is illustrated by a graph showing an example of an NBI observation mode in which the light intensity ratio of the purple LED and the green LED in the second illumination light is different depending on the location of the subject.

[0021] Figure 10 The first embodiment described above is illustrated by a graph showing an example of changes in pixel signals of normal tissue pixels and diseased tissue pixels in a corrected image signal after the first image signal and the second image signal are synthesized by the color emphasis section.

[0022] Figure 11 The diagrams relating to the first embodiment described above are for illustrating other examples of synthesis methods for the synthesis section.

[0023] Figure 12 The first embodiment described above is a graph showing an example of how the color emphasis changes based on the distance of a component of a first image signal in the a*b* plane from the L* axis.

[0024] Figure 13 The diagram relating to the first embodiment described above illustrates an example of a region where color emphasis is applied in the a*b* plane. Detailed Implementation

[0025] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, the present invention is not limited to the embodiments described below.

[0026] Furthermore, in the accompanying drawings, identical or corresponding elements are appropriately labeled with the same reference numerals. It should also be noted that the drawings are schematic, and the length relationships and ratios of elements within a single drawing may sometimes differ from reality. Moreover, among multiple drawings, there may sometimes be sections where the length relationships and ratios differ from each other.

[0027] [First Implementation Method]

[0028] Figures 1 to 13 This represents the first embodiment of the present invention. Figure 1 This is a diagram showing the structure of the endoscope device 1.

[0029] The endoscope device 1 includes an endoscope 2, a light source device 3, a processor 4, a display 5, and an input device 6.

[0030] Endoscope 2 is configured as an electronic endoscope that can be inserted into the body cavity of the subject to be examined, and to photograph the biological tissues of the subject and output the photographic signal.

[0031] The light source device 3 supplies illumination light to the endoscope 2 so that the subject in the dark can be observed.

[0032] The processor 4 is connected to the light source device 3 and the endoscope 2, and generates and outputs image signals for observation and / or recording based on the camera signals output from the endoscope 2.

[0033] The display 5 is equipped with display devices such as LCD (Liquid-Crystal Display) and organic EL (Electro-Luminescence) display, and displays the observed image corresponding to the image signal output from the processor 4.

[0034] Input device 6 has operating components such as switches and buttons, and allows users such as surgeons to perform input operations, outputting indication signals corresponding to the operation content to processor 4.

[0035] Endoscope 2 is connected to processor 4 in a detachable manner, for example, via a universal cable (not shown), and to light source device 3 in a detachable manner via an optical cable (not shown).

[0036] The endoscope 2 has a slender insertion part 2a that can be inserted into the body of the patient and an operation part 2b provided at the base of the insertion part 2a.

[0037] A camera unit 21 and an illumination optical system 22 are provided at the front end 2c of the insertion part 2a.

[0038] Inside the aforementioned optical cable and endoscope 2, a light guide 7 for transmitting illumination light is inserted. The exit end of the light guide 7 is positioned opposite the illumination optical system 22. Thus, the illumination light transmitted via the light guide 7 illuminates the subject through the illumination optical system 22.

[0039] The camera unit 21 has an objective lens optical system 21a and an image sensor 21b.

[0040] The objective lens optical system 21a images the reflected light from the subject, which is illuminated by the illumination light emitted from the illumination optical system 22, onto the imaging element 21b.

[0041] The imaging element 21b captures an optical image of the subject imaged by the objective lens optical system 21a, generates an imaging signal with multiple color components, and outputs the generated imaging signal. Specifically, the imaging element 21b is configured as an image sensor such as a CCD or CMOS, in which multiple pixels are arranged in a matrix, for example, with a primary color Bayer color filter (which, for example, could also be a complementary color filter).

[0042] The operation unit 2b is shaped so that the user can hold and operate it, and is provided with a mirror switch 23 and a mirror memory 24.

[0043] The mirror switch 23 has operating components such as switches and buttons. By performing user input operations, it outputs an indication signal corresponding to the operation content to the processor 4.

[0044] The endoscope memory 24 stores endoscope information, including the ID number of the endoscope 2, the spectral sensitivity characteristics of the camera unit 21, and other information inherent to the endoscope 2.

[0045] The signal lines connected to the camera element 21b, the signal lines connected to the endoscope switch 23, and the signal lines connected to the endoscope memory 24 are disposed inside the endoscope 2 and in the aforementioned general-purpose cable, and are electrically connected to the processor 4 via the general-purpose cable.

[0046] Therefore, when the endoscope 2 is electrically connected to the processor 4 and the processor 4 is powered on, the control unit 47 of the processor 4 (described later) reads the endoscope information stored in the endoscope memory 24. Additionally, an indication signal output from the endoscope switch 23 is sent to the control unit 47. Furthermore, the imaging signal output from the imaging element 21b is sent to the preprocessing circuit 40 within the processor 4 (described later).

[0047] The light source device 3 includes a light source controller 31, a light source unit 32, a beam combiner 33, and a condenser lens 34.

[0048] The light source controller 31 includes, for example, a control circuit, which controls the light emission of the light source unit 32 according to the lighting control signal output from the processor 4.

[0049] The light source unit 32 functions as a light source and has multiple semiconductor light-emitting elements (specifically, LEDs 32a to 32d described below) with different center wavelengths of emitted light, so that the multiple semiconductor light-emitting elements emit light in a certain light intensity ratio to generate illumination light.

[0050] Specifically, the light source unit 32 includes, for example, a purple LED (Light-Emitting Diode) 32a, a blue LED 32b, a green LED 32c, and a red LED 32d.

[0051] The purple LED 32a emits purple light (hereinafter also referred to as V light) with a center wavelength in the purple region. In particular, in this embodiment, since an endoscope system capable of performing NBI observation mode is envisioned, the purple LED 32a emits narrowband purple light with a wavelength of 390 to 445 (nm).

[0052] The blue LED32b emits blue light (hereinafter also referred to as B light) with a center wavelength in the blue region. As will be described later, this blue light is preferably narrowband light.

[0053] The green LED 32c emits green light (hereinafter also referred to as G light) with a center wavelength belonging to the green region. As described above, in this embodiment, since an NBI viewing mode is envisioned, the green LED 32c emits narrowband green light with a wavelength of 530 to 550 (nm).

[0054] The red LED32d emits red light (hereinafter also referred to as R light) with a center wavelength belonging to the red region. As will be described later, this red light is also preferably narrowband light.

[0055] Each LED 32a to 32d of the light source unit 32 emits light or fades individually based on the control of the light source controller 31, according to its own light intensity. In addition, the light intensity mentioned here refers to the amount of illumination light emitted during the capture (exposure) of an image by the imaging element 21b.

[0056] Furthermore, the spectral sensitivity characteristics of the imaging unit 21, represented by the spectral sensitivity characteristic information stored in the endoscope memory 24, vary depending on the model of the endoscope 2 (and thus on the individual). For example, the wavelength of light received by the imaging element 21b via the primary color Bayer array R (red), G (green), and B (blue) filters is not limited to the red, green, and blue bands respectively, and in fact, it sometimes has sensitivity over a wider frequency band.

[0057] Therefore, as the light source used in the light source unit 32 that accurately controls the illumination light, it is preferable to use a light source with narrow bands and discrete spectra of each color light.

[0058] For example, when using an LED as the light source of the light source unit 32, it is preferable to use an LED that generates a light color by using light emitted from the LED itself, rather than using an LED that generates a light color by using a phosphor.

[0059] In addition, not limited to LEDs, laser light sources such as semiconductor lasers (LDs) can also be used as the light source of the light source unit 32.

[0060] This reduces color mixing when shooting with a primary color filter and improves the accuracy of color emphasis.

[0061] The combiner 33 combines the light emitted from each of the LEDs 32a to 32d in the light source unit 32 and emits it.

[0062] The focusing lens 34 converges the light emitted from the combiner 33 to the incident end of the light guide 7.

[0063] The processor 4 includes a pre-processing circuit 40, an A / D converter 41, a WB (white balance) processing unit 42, a simultaneous processing unit 43, a color enhancement unit 44, a sharpness enhancement unit 45, a display control unit 46, and a control unit 47. The pre-processing circuit 40, which precedes the A / D converter 41, is an analog circuit. Furthermore, the WB processing unit 42, the simultaneous processing unit 43, the color enhancement unit 44, the sharpness enhancement unit 45, and the display control unit 46, which are subsequent stages to the A / D converter 41, are digital circuits, and the control unit 47 is also a digital circuit.

[0064] Here, the digital circuit section of the processor 4 is configured, for example, to read and execute the processing program stored in a storage device (or recording medium) such as a memory by a processor including a CPU (Central Processing Unit) such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array) to realize the functions of each section.

[0065] However, this is not the only limitation; for example, each part of the processor 4 can also be configured as a dedicated electronic circuit to perform its respective function.

[0066] The preprocessing circuit 40 amplifies the image signal output from the camera unit 21 of the endoscope 2, and then performs noise removal processing such as correlation double sampling.

[0067] The A / D converter 41 performs A / D conversion on the analog camera signal output from the pre-processing circuit 40 to generate a digital image signal. The digital image signal generated by the A / D converter 41 is output to the WB processing unit 42 and the control unit 47.

[0068] The WB processing unit 42 performs white balance processing on the image signal with multiple color components output from the A / D converter 41.

[0069] Simultaneous processing unit 43 performs simultaneous processing (also known as de-mosaicing) on ​​the image signal with multiple color components output from A / D converter 41. That is, when the imaging element 21b is, for example, a single-board imaging element equipped with primary color Bayer color filters as described above, the imaging signal becomes a signal with one color component per pixel. Therefore, in the case of a G pixel equipped with a G filter, the simultaneous processing unit 43 performs the following simultaneous processing: interpolating the R component and B component missing from the pixel signal at the G pixel position based on the R component of the surrounding R pixels and the B component of the B pixels, thereby generating the RGB component of the pixel position of the G pixel. By performing the same simultaneous processing on pixels equipped with other color filters, an image signal with RGB components uniformly present at each pixel position is generated.

[0070] The color enhancement unit 44 performs color enhancement processing on the image signal output from the simultaneous processing unit 43 under the control of the control unit 47. The color enhancement processing performed by the color enhancement unit 44 will be described in detail later.

[0071] The sharpness enhancement unit 45 performs sharpness enhancement processing on the image signal output from the color enhancement unit 44 based on the control of the control unit 47.

[0072] The display control unit 46 generates an image signal that is distributed from the image signal output from the sharpness enhancement unit 45 to the R channel, G channel and B channel of the display 5, and outputs the generated image signal to the display 5.

[0073] The control unit 47 is a controller that receives indication signals from the input device 6 and the endoscope switch 23 and performs unified control over the entire endoscope device 1, which includes the processor 4.

[0074] The control unit 47 includes a memory 47a that stores the processing program executed by the control unit 47.

[0075] Furthermore, the memory 47a stores in advance information such as color adjustment coefficients for adjusting the light amount of each LED 32a to 32d according to the type of illumination light, and emphasis coefficients used in the color emphasis processing performed by the color emphasis unit 44.

[0076] In addition, both the color adjustment coefficient and the emphasis coefficient are stored in memory 47a, but at least one of them can also be stored in memory (not shown) within the light source controller 31.

[0077] As described above, when the endoscope 2 is electrically connected to the processor 4 and the processor 4 is powered on, the control unit 47 reads the endoscope information stored in the endoscope memory 24.

[0078] Furthermore, the control unit 47 sets the observation mode of the endoscope device 1 based on the instruction signal output from the observation mode switching switch (not shown) provided on the input device 6 and / or the endoscope switch 23. Here, as observation modes that can be set in the endoscope device 1, there are, for example, a normal observation mode and a special light observation mode. In addition, in this embodiment, as an example of a special light observation mode, the NBI (Narrow-Band Imaging) observation mode is given, but it is not limited to this.

[0079] Furthermore, the control unit 47 sets the color emphasis mode of the endoscope device 1 to be turned on / off according to the indication signal output from the color emphasis mode setting switch (not shown) provided on the input device 6 and / or the endoscope switch 23.

[0080] That is, the endoscope device 1 of this embodiment can set the color emphasis mode to be turned on / off in each observation mode. Therefore, it is possible to select and set any one of the following: color emphasis mode on in normal observation mode, color emphasis mode off in normal observation mode, color emphasis mode on in NBI observation mode, and color emphasis mode off in NBI observation mode.

[0081] The control unit 47 generates an illumination control signal for emitting illumination light from the light source device 3 to be turned on / off according to the spectral sensitivity characteristic information of the camera unit 21 shown in the endoscope information read from the endoscope memory 24, and outputs the illumination control signal to the light source controller 31.

[0082] When the color emphasis mode is turned on, the control unit 47 controls the light source device 3 to switch between the first illumination light and the second illumination light to emit light. The light intensity ratio of each LED 32a to 32d in the first illumination light becomes the first light intensity ratio for observing the subject. The light intensity ratio of each LED 32a to 32d in the second illumination light becomes the second light intensity ratio, which is different from the first light intensity ratio.

[0083] On the other hand, when the color emphasis mode is off, the control unit 47 controls the light source device 3 to emit the first illumination light for observing the subject.

[0084] If the camera unit 21 captures the reflected light from the subject that has been illuminated by the illumination light emitted from the light source device 3, the control unit 47 extracts the brightness information of the subject from the image signal output by the A / D converter 41, generates an illumination control signal that makes the subject have an appropriate brightness based on the current brightness information, and outputs the illumination control signal to the light source controller 31.

[0085] Furthermore, when the color emphasis mode is enabled, the control unit 47 reads the emphasis coefficients from the memory 47a and controls the color emphasis unit 44 to perform color emphasis processing using the read emphasis coefficients.

[0086] Figure 2 This is a block diagram representing the structure of the color emphasis section 44.

[0087] The color enhancement unit 44 includes a first memory 44a, a second memory 44b, an L*a*b* conversion unit 44c, a synthesis unit 44d, and an RGB conversion unit 44e.

[0088] The first memory 44a is a memory that stores a first image signal obtained by processing a first imaging signal, which is generated by capturing the reflected light from a subject illuminated by the first illumination light. Here, the first illumination light is illumination light in which the light intensity ratio of each of the LEDs 32a to 32d is set to a first light intensity ratio for observing the subject. Specific examples of the first illumination light are white light (WL) in normal observation mode, NBI illumination light in NBI observation mode, etc.

[0089] The second memory 44b is a memory that stores a second image signal obtained by processing the second imaging signal, which is generated by imaging the return light from the subject illuminated by the second illumination light. Here, the second illumination light is illumination light emitted by LEDs 32a to 32d, which are the same as the LEDs emitting the first illumination light, with a second light intensity ratio different from the first light intensity ratio.

[0090] The second illumination light, also known as color enhancement light (CE), is light obtained by adjusting the second light intensity ratio within a specified error range to make the second image signal related to the reference part of the subject colorless (for example, in the RGB color space, R=G=B, and in the CIE (International Commission on Illumination) L*a*b* color space, a*=b*=0, etc.).

[0091] Here, as an example of a reference portion of the subject, a normal part of the subject (e.g., normal mucosa or other normal tissue) where blood vessels do not extend can be cited. Alternatively, a biological model can also be used as a reference portion of the subject. Therefore, in the second image signal, the reference portion of the subject becomes achromatic (white or other grayscale), while abnormal areas outside the reference portion are colored.

[0092] The L*a*b* conversion unit 44c reads the first image signal (RGB components are (R1, G1, B1)) stored in the first memory 44a in the RGB color space and converts it into the first image signal (L1*, a1*, b1*) in the L*a*b* color space.

[0093] Furthermore, the L*a*b* conversion unit 44c reads the second image signal (RGB components are (R2, G2, B2)) stored in the second memory 44b in the RGB color space and converts it into the second image signal (L2*, a2*, b2*) in the L*a*b* color space.

[0094] The compositing unit 44d generates a corrected image signal after color enhancement of the first image signal (L1*, a1*, b1*) based on the second image signal (L2*, a2*, b2*). When compositing the image signal, the compositing unit 44d uses the enhancement coefficient sent from the control unit 47 as described later.

[0095] In addition, the first and second camera signals are signals obtained by capturing images at different times (see reference). Figure 4 Therefore, the compositing unit 44d preferably performs compositing after aligning the first image signal (L1*, a1*, b1*) and the second image signal (L2*, a2*, b2*). This reduces image blurring and color deviation caused by positional shifts.

[0096] The RGB conversion unit 44e converts the L*a*b* color space signal synthesized by the synthesis unit 44d into an RGB color space signal and outputs it.

[0097] Figure 3 This is a flowchart illustrating the color enhancement display processing performed by the endoscope device 1. Figure 4 This is a timing diagram showing the alternating emission of white light (WL) and color-accurate light (CE) when color-accurate mode is enabled. Figure 5 This is a timing diagram illustrating an example of emitting white light (WL) when the color emphasis mode is off.

[0098] Additionally, in either the normal viewing mode or the special light viewing mode (in this case, the NBI viewing mode), when the color emphasis mode is enabled, Figure 3 The color enhancement display processing shown is performed by the endoscope device 1.

[0099] like Figure 4 As shown, when the color emphasis mode is enabled, white light (WL) and color emphasis light (CE) are emitted alternately for each frame of video capture. However, Figure 4This is a diagram illustrating one example, and is not limited to this. For example, variations such as emitting one frame of color-enhanced light CE after emitting two consecutive frames of white light WL can also be performed.

[0100] Furthermore, here, it is envisioned that irradiation... Figure 4 Color emphasis is applied to the combinations of (WL1, CE1), (WL2, CE2), ... in the images captured by the various illuminations shown. However, to improve the video frame rate, further color emphasis can be applied based on the combinations of (CE1, WL2), (CE2, WL3), ...

[0101] Additionally, when the color emphasis mode is off, it is preferable to simply prevent the color emphasis light (CE) from emitting light, rather than... Figure 5 As shown, white light WL is emitted when the color-enhancing light CE is emitted, thereby increasing the camera frame rate. By increasing the camera frame rate, it is possible to generate easily observable moving images, even in situations with motion.

[0102] In the main process (not shown), when entering... Figure 3 During the processing shown, the control unit 47 and the light source controller 31 set the light intensity of each LED 32a to 32d for emitting the first illumination light (step S1). Here, the case where the first illumination light is white light WL (the case of normal observation mode) will be described as an example, and the light intensity of each LED 32a to 32d is set as follows.

[0103] Figure 6 This is a chart showing the light levels of each LED 32a to 32d in the first and second illumination lights when the color emphasis mode is enabled in normal viewing mode.

[0104] Based on the brightness information extracted from the image signal output from the A / D converter 41, the control unit 47 sets the light intensity Gw of the green LED 32c in the white light WL. Furthermore, the control unit 47 reads the color adjustment coefficients αwv, αwb, and αwr for the white light WL from the memory 47a, and multiplies them by the light intensity Gw of the green LED 32c. This allows the control unit 47 to calculate the light intensity Gv of the purple LED 32a (Gv = αwv × Gw), the light intensity Gb of the blue LED 32b (Gb = αwb × Gw), and the light intensity Gr of the red LED 32d (Gr = αwr × Gw) respectively (see reference). Figure 6 (WL column).

[0105] Furthermore, in this embodiment, the reflected light from the subject is obtained by simultaneously emitting light from both the purple LED 32a and the blue LED 32b, as captured by the B-pixel of the imaging element 21b. This is to compensate for the amount of light emitted by the blue LED 32b with the amount of light emitted by the purple LED 32a; therefore, in normal viewing mode, the purple LED 32a is essentially treated as an LED emitting blue light as well. However, if there is a surplus of light emitted by the blue LED 32b, the blue LED 32b can be emitted without emitting light from the purple LED 32a, thus obtaining the reflected light from the subject captured by the B-pixel.

[0106] The control unit 47 generates an illumination control signal that can obtain the light amount of each LED 32a to 32d as set, and outputs the illumination control signal to the light source controller 31.

[0107] The light source controller 31 supplies driving current to each semiconductor light-emitting element, and the light source device 3 emits a first illumination light based on the illumination control signal, for example, emitting white light WL (step S2).

[0108] Then, the imaging element 21b captures the reflected light from the subject to generate a first imaging signal, which is received by the processor 4 from the endoscope 2. The processor 4 processes the first imaging signal through the preprocessing circuit 40 to the simultaneous processing unit 43 to generate, for example, a first image signal (R1, G1, B1) having RGB components (step S3). The first image signal (R1, G1, B1) generated here is stored in the first memory 44a.

[0109] The L*a*b* conversion unit 44c reads the first image signal (R1, G1, B1) stored in the first memory 44a and converts it into the first image signal (L1*, a1*, b1*) in the L*a*b* color space (step S4).

[0110] Next, the control unit 47 and the light source controller 31 set the light intensity of each LED 32a to 32d for emitting the second illumination light (step S5). Here, when the first illumination light is white light WL, the second illumination light becomes the color-enhancing light CE of white light WL.

[0111] In order to suppress the brightness (or luminance) variation of the first illumination light and the second illumination light, the control unit 47 basically sets the light intensity of the green LED 32c in the color-emphasis light CE to be the same as the light intensity Gw of the green LED 32c in the white light WL.

[0112] However, when the light intensity of the green LED32c in the color-emphasis light CE is set to Gw, if at least one of the light intensity of the purple LED32a, the blue LED32b, and the red LED32d calculated based on the second light intensity ratio exceeds the maximum light intensity of each color, the control unit 47 is set to reduce the light intensity of each color while maintaining the second light intensity ratio, so that the light intensity of the purple LED32a, the blue LED32b, and the red LED32d are all below the maximum light intensity.

[0113] therefore, Figure 6 The color adjustment factor Kg shown is basically set to 1, and when Kg = 1 and the second light intensity ratio cannot be achieved, it is set to an appropriate value such as Kg < 1. Then, the set Kg is multiplied by the light intensity Gw of the green LED32c in the white light WL to calculate the light intensity Gw × Kg of the green LED32c in the color-enhancing light CE.

[0114] Furthermore, the control unit 47 reads the color adjustment coefficient Kv for the second light intensity ratio from the memory 47a and multiplies it by the light intensity αwv×Gw of the purple LED 32a in the white light WL, thereby calculating the light intensity Gv of the purple LED 32a = αwv×Gw×Kv. It is self-evident that the color adjustment coefficient Kv (and the color adjustment coefficients Kb and Kr described below) becomes a different value depending on whether the color adjustment coefficient Kg is 1 or a value other than 1, in order to maintain the second light intensity ratio.

[0115] Similarly, the control unit 47 reads the color adjustment coefficient Kb for the second light intensity ratio from the memory 47a and multiplies it by the light intensity αwb×Gw of the blue LED 32b in the white light WL, thereby calculating the light intensity Gb of the blue LED 32b = αwb×Gw×Kb.

[0116] The control unit 47 reads the color adjustment coefficient Kr for the second light intensity ratio from the memory 47a and multiplies it by the light intensity αwr×Gw of the red LED 32d in the white light WL, thereby calculating the light intensity Gr of the red LED 32d = αwr×Gw×Kr (refer to...). Figure 6 (CE column).

[0117] Here, the aforementioned color adjustment coefficients Kg, Kv, Kb, and Kr are calculated in advance based on the spectral reflectance of the reference portion of the subject, the spectral sensitivity characteristics of the imaging unit 21, and the spectral luminous intensity characteristics of the light source unit 32, as coefficients that ensure the second image signal becomes colorless within a specified error range (e.g., coefficients where the average values ​​of the R, G, and B components are the same within a specified error range), and are stored in the memory 47a.

[0118] Here, regarding the spectral reflectance of the reference portion of the test subject, for example, the technique described in Japanese Patent Application Publication No. 2000-14629 (however, of course, other suitable techniques may also be applied) is used, and the spectral reflectance measured at a specified distance is used.

[0119] Therefore, the values ​​of color adjustment coefficients Kg, Kv, Kb, and Kr generally vary depending on the combination of endoscope 2, light source device 3, and processor 4, and can therefore be stored in memory 47a as a table corresponding to the combination of models.

[0120] Furthermore, if we give a specific example of the second light intensity ratio when Kg = 1, it is as follows:

[0121] V-light: B-light: G-light: R-light = (αwv × Kv): (αwb × Kb): 1: (αwr × Kr).

[0122] This second light intensity ratio can also be changed in real time. For example, in the second image signal (R2, G2, B2) in the RGB color space, the control unit 47 can also adjust the color adjustment coefficients Kv, Kb, and Kr with respect to the second light intensity ratio in real time so that the average of each signal value of the reference part (the average is represented by the symbol <>) becomes <R2> = <G2> = <B2> within a specified error range.

[0123] Alternatively, in the second image signal (L2*, a2*, b2*) of the L*a*b* color space, the average <a2*> of the a* component and the average <b2*> of the b* component of the reference part can be calculated, and the control unit 47 can adjust the color adjustment coefficients Kv, Kb, and Kr in real time so that <a2*> and <b2*> are 0 within the specified error range.

[0124] For example, in<a*> To reduce the value of the color adjustment factor Kr during timing,<a*> When it is negative, increase the value of the color adjustment coefficient Kr.<b*> To increase the values ​​of the color adjustment coefficients Kv and Kb during the timing,<b*> When the values ​​are negative, the color adjustment coefficients Kv and Kb are reduced. This process can also be performed recursively, for example, to ensure that the reference portion of the second image signal remains achromatic within a specified error range.

[0125] Alternatively, it can be described as follows: First, the surgeon or other user operates the endoscope 2 to calculate the average at a predetermined location within the image, placing the normal area where blood vessels do not extend. Then, the user operates the input device 6 to instruct the calculation of the average at the predetermined location. Thus, the user can select the area they wish to make colorless.

[0126] The control unit 47 generates an illumination control signal that can obtain the light amount of each LED 32a to 32d as set, and outputs the illumination control signal to the light source controller 31.

[0127] The light source device 3 emits a second illumination light based on the illumination control signal, which is the color-enhancing light CE of white light WL (step S6).

[0128] Then, the imaging element 21b captures the reflected light from the subject to generate a second imaging signal, which is received by the processor 4 from the endoscope 2. The processor 4 processes the second imaging signal through the preprocessing circuit 40 to the simultaneous processing unit 43 to generate, for example, a second image signal (R2, G2, B2) having RGB components (step S7). The second image signal (R2, G2, B2) generated here is stored in the second memory 44b.

[0129] The L*a*b* conversion unit 44c reads the second image signal (R2, G2, B2) stored in the second memory 44b and converts it into a second image signal (L2*, a2*, b2*) in the L*a*b* color space (step S8).

[0130] Then, the compositing unit 44d synthesizes the first image signal (L1*, a1*, b1*) and the second image signal (L2*, a2*, b2*) for each pixel in the L*a*b* color space to generate a color-emphasized corrected image signal (step S9).

[0131] Figure 10 This is a graph illustrating examples of pixel signal changes in normal tissue pixels and diseased tissue pixels in a corrected image signal synthesized by the color emphasis section after combining the first and second image signals.

[0132] As described above, normal tissue without extended blood vessels is selected as the baseline portion of the subject. In this case, such as Figure 10 As shown in the column for normal tissue, the second illumination light is set in such a way that the normal tissue in the second image is achromatic (white or grayscale), so that the a* and b* components in the second image signal both take the value of 0 within the specified error range.

[0133] Therefore, the (as*, bs*) components of the synthesized corrected image signal do not change from the a* and b* components of the first image signal, but become as shown in Equation 1.

[0134] [Number 1]

[0135] (as*,bs*)=(a1*,b1*).

[0136] In contrast, compared to normal tissue, known diseased tissue, such as inflamed biological tissue, has a decreased reflectivity of light in the blue-green frequency band (and an increased absorptivity of light in the blue-green frequency band). In the pixel signals obtained by photographing such diseased tissue, the equation R = G = B does not hold; for example, the signal values ​​become G < R and B < R.

[0137] Thus, in the case of diseased tissue, even when illuminated by the second illumination light, it will not become achromatic (white or other grayscale), and at least one of the a* and b* components of the second image signal has a value other than 0 (a2*, b2*).

[0138] The synthesis unit 44d multiplies the emphasis coefficient C received from the control unit 47 with the components (a2*, b2*) of the second image signal, and then adds it to the components (a1*, b1*) of the first image signal, thereby calculating the components (as*, bs*) of the color-emphasized corrected image signal as shown in the following mathematical formula 2.

[0139] [Number 2]

[0140] (as*,bs*)=(a1*+C×a2*,b1*+C×b2*)

[0141] Equation 2 also applies to normal tissue, but in normal tissue (a2*, b2*) = (0, 0), thus only yielding the result shown in Equation 1. Furthermore, if we assume the emphasis coefficient C = 1, then the simple addition of the components (a1*, b1*) of the first image signal and the components (a2*, b2*) of the second image signal becomes the components (as*, bs*) of the corrected image signal.

[0142] On the other hand, the synthesis unit 44d performs an operation, for example, as shown in the following mathematical formula 3, on the L* component (brightness component) of the corrected image signal using the second emphasis coefficient C2 received from the control unit 47, thereby calculating Ls* as the L* component of the synthesized corrected image signal.

[0143] [Number 3]

[0144] Ls*=L1*+C2×(L2*-L1*)

[0145] The operation of mathematical formula 3 is performed by multiplying the difference value obtained by subtracting the brightness component L1* of the first image signal from the brightness component L2* of the second image signal by the second emphasis coefficient C2, and then adding it to the brightness component L1* of the first image to perform the correction.

[0146] As described above, if the light intensity of the green LED32c in the color-enhanced light CE is set to be the same as the light intensity Gw of the green LED32c in the white light WL, then L1* and L2* will become equal or close values, and Ls* will not change much relative to L1*.

[0147] Therefore, in mathematical formula 3, the brightness does not change much, while mathematical formula 2 mainly emphasizes the hue.

[0148] Furthermore, the synthesis method of the synthesis section 44d is not limited to the method described above, and various other methods can be applied.

[0149] Figure 11 This is a diagram illustrating other examples of the synthesis method of the synthesis section 44d.

[0150] In the above calculation method based on mathematical formula 2, such as Figure 10 As shown in the lesion / corrected image section, the hue of the components (as*, bs*) of the corrected image signal is close to the hue of the components (a1*, b1*) of the first image signal, but sometimes some hue changes occur.

[0151] therefore, Figure 11 The synthesis method shown is a method that does not change the hue.

[0152] Let r1 be the distance of the component (a1*, b1*) of the first image signal from the origin (0, 0) (i.e., the L* axis) on the a*b* plane (L* = L1* plane) (refer to mathematical formula 5 described later), and let θ be the angle with the a* axis.

[0153] Similarly, the distance of the components (a2*, b2*) of the second image signal from the origin (0, 0) on the a*b* plane (L*=L2* plane) is set as r2.

[0154] At this time, the synthesis unit 44d uses the emphasis coefficient C received from the control unit 47 to extend the distance r1 based on the distance r2, thereby calculating the components (as*, bs*) of the color-emphasized corrected image signal.

[0155] [Number 4]

[0156] (as*,bs*)

[0157] =([r1+C×r2]cosθ,[r1+C×r2]sinθ)

[0158] According to the calculation method shown in the mathematical formula 4, not only are the a*b* plane components of the second image signal emphasized by the emphasis coefficient C and added to the a*b* plane components of the first image signal, but the angle between the components (as*, bs*) of the corrected image signal and the a* axis is the same as the angle θ between the components (a1*, b1*) of the first image signal and the a* axis. Therefore, color emphasis can be performed without changing the hue.

[0159] Furthermore, the L* component (luminance component) of the corrected image signal can be obtained, for example, through the operation shown in the above mathematical formula 3.

[0160] Furthermore, as a synthesis method for the synthesis unit 44d, a weighted summation of the components (a1*, b1*) of the first image signal and the components (a2*, b2*) of the second image signal can be used, and the weighting can also be normalized. On the other hand, regarding the L* component, the value of the L1* component of the first image signal can be directly used as the L* component of the corrected image signal without changing it.

[0161] Thus, after the composite unit 44d generates the corrected image signal, the RGB conversion unit 44e converts the corrected image signal in the L*a*b* color space into a signal in the RGB color space and outputs it (step S10).

[0162] Then, the sharpness enhancement unit 45 performs sharpness enhancement processing, the display control unit 46 generates an image signal, and displays the color-enhanced corrected image on the display 5 (step S11).

[0163] Then, the control unit 47 determines whether the setting to end the color emphasis display process has been performed (step S12) based on the indication signal from the input device 6 or the mirror switch 23. If not, it returns to step S1 and repeats the above-described process.

[0164] On the other hand, in step S12, if it is determined that an end setting has been performed, the process returns to the main process (not shown).

[0165] In addition, Figure 6 The diagram shows an example of the light intensity setting for each of the LEDs 32a to 32d in the second illumination light, but the light intensity ratio of LEDs 32a to 32d can also be changed depending on the part of the subject being examined. Figure 7 This is a chart showing examples of typical observation modes in which the light intensity ratio of each LED 32a to 32d in the second illumination light differs depending on the location of the subject.

[0166] Taking the digestive tract as an example, the color of the baseline portion of the specimen varies depending on which part of the specimen is being examined, such as the esophagus (first part), stomach (second part), or large intestine (third part).

[0167] Therefore, in Figure 7 In the process, color adjustment coefficients (Kv1, Kb1, Kr1) for the first part, color adjustment coefficients (Kv2, Kb2, Kr2) for the second part, and color adjustment coefficients (Kv3, Kb3, Kr3) for the third part are prepared in advance as color adjustment coefficients (Kv, Kb, Kr) and stored in the memory 47a of the control unit 47.

[0168] Here, as described above, the color adjustment coefficients used for each part are determined in advance based on the spectral reflectance of the reference portion of each part, the spectral sensitivity characteristics of the imaging unit 21, and the spectral luminous intensity characteristics of the light source unit 32, which are measured using the technology described in Japanese Patent Application Publication No. 2000-14629, and are coefficients that ensure the second image signal becomes colorless within a specified error range.

[0169] Furthermore, during the examination, any color adjustment factor corresponding to the area of ​​the subject can be selected. Alternatively, the color adjustment factor can be manually selected by the user through input device 6 or mirror switch 23. Alternatively, machine learning, which has proven effectiveness in image recognition, such as CNN (Convolutional Neural Network), can be used to determine the area of ​​the subject captured in the image, and the color adjustment factor can be automatically selected based on the determination result.

[0170] In addition, Figure 7 The scenario assumes Kg = 1, but it can also be set to Kg ≠ 1 as needed, similar to the above. In this case, different color adjustment coefficients such as Kg1, Kg2, and Kg3 can be set depending on the location of the subject. Here, Kg1 is the color adjustment coefficient multiplied by the amount of white light WL Gw in the first color emphasis light (first CE) irradiating the first location; Kg2 is the color adjustment coefficient multiplied by the amount of white light WL Gw in the second color emphasis light (second CE) irradiating the second location; and Kg3 is the color adjustment coefficient multiplied by the amount of white light WL Gw in the third color emphasis light (third CE) irradiating the third location.

[0171] and, Figure 8 This is a diagram showing an example of a typical observation mode in which the light intensity ratio of each LED 32a to 32d in the second illumination light varies depending on the location of the subject and the distance from the anterior end 2c of the endoscope 2 to the location of the subject.

[0172] When the distance from the tip 2c of the endoscope 2 to the part of the object being examined is short, the camera unit 21 can directly acquire the return light from the object being illuminated by the illumination light emitted from the illumination optical system 22.

[0173] In contrast, when the distance from the front end 2c of the endoscope 2 to the part of the object being examined increases, the light returning from one part is used as secondary light to illuminate other parts, and the light incident on the camera unit 21 from other parts sometimes becomes the return light of the illumination light and the return light of the secondary light.

[0174] Thus, the light incident from the subject onto the camera unit 21 includes not only second-order light, but more generally higher-order reflected light and scattered light (hereinafter, multiple scattered light), the effects of which sometimes occur depending on the distance to the subject.

[0175] Because the red band of light is more dispersed in the multiple scattered light in organisms, the reflectance of the subject will vary depending on the distance from the front end 2c to the subject, even when irradiated with the same illumination light.

[0176] Therefore, the distance from the tip 2c of the endoscope 2 to the subject is prepared in advance according to near, medium, and far distances, with Kc1n for near distance, Kc1m for medium distance, and Kc1f for far distance, serving as the color adjustment coefficient Kc1 for the first part (here "c" represents color, c = v (purple), b (blue), r (red)). Similarly, for the color adjustment coefficients Kc2 and Kc3 used for the second and third parts, Kc2n for near distance, Kc2m for medium distance, Kc2f for far distance, Kc3n for near distance, Kc3m for medium distance, and Kc3f for far distance are also prepared in advance.

[0177] In addition, regarding Figure 8 Alternatively, it can be set to Kg≠1 in the same way as above, and include c=g (green) in the color "c". Prepare Kg1n, Kg1m, and Kg1f according to the distance to the subject (the same applies to the second and third parts). This point is the same.

[0178] The color adjustment coefficients corresponding to these distances can be determined, for example, using the technology described in Japanese Patent Application Publication No. 2000-14629, based on the spectral reflectance of each part measured at close, medium, and long distances.

[0179] As a specific example of the determined color adjustment coefficient in a living organism, one can cite a color adjustment coefficient that causes the light intensity of the red LED 32d to decrease with increasing distance, and Kr1n > Kr1m > Kr1f (the same applies to the second and third parts). Furthermore, in order to suppress the decrease in total light intensity that accompanies the decrease in the light intensity of the red LED 32d, the light intensity of each of the green LED 32c, purple LED 32a, and blue LED 32b can be slightly increased with increasing distance.

[0180] Furthermore, this example shows the distance divided into three stages: near, medium, and far. However, it can also be divided into two stages, or even four or more stages. Alternatively, interpolation can be used to determine the color adjustment coefficient corresponding to any distance.

[0181] The color adjustment coefficients prepared in this way are pre-stored in the memory 47a of the control unit 47. During the inspection, any color adjustment coefficient corresponding to the part of the subject and the distance to the subject can be selected for use. At this time, the selection can be manually set by the user operating input device 6 or the mirror switch 23. For example, regarding the distance, it can also be automatically set by the control unit 47 based on the distance measured by various technologies such as laser-based distance measurement and image recognition-based distance measurement.

[0182] Furthermore, in the above description, the color adjustment coefficient varies not only according to the location of the subject but also according to the distance. However, the color adjustment coefficient may also vary according to the angle between the direction of the front end 2c of the endoscope 2 and the surface of the subject, or in addition to the distance, the color adjustment coefficient may also vary according to the angle between the direction of the front end 2c of the endoscope 2 and the surface of the subject.

[0183] Furthermore, while the above description illustrates an example of enabling the color emphasis mode in normal viewing mode, the endoscope device 1 of this embodiment, as described above, can also enable the color emphasis mode in NBI viewing mode. Here, the color emphasis display processing flow in NBI viewing mode is also as described above. Figure 3 The process shown is the same.

[0184] Figure 9 This is a chart showing examples of different NBI observation modes where the light intensity ratio of the purple LED32a and green LED32c in the second illumination light is adjusted according to the location of the subject.

[0185] The NBI observation mode involves irradiating two narrow bands of light that are easily absorbed by hemoglobin in the blood. Specifically, as mentioned above, it involves irradiating violet light with a wavelength of 390–445 nm and green light with a wavelength of 530–550 nm to emphasize the observation mode that displays the capillaries and fine patterns of the mucosa.

[0186] Therefore, in NBI observation mode, the purple LED32a and green LED32c emit light at a first light intensity ratio to generate NBI illumination light as the first illumination light.

[0187] Specifically, the control unit 47 calculates the light quantity Gv of the purple LED 32a by multiplying the light quantity GN of the green LED 32c in the NBI illumination light by the color adjustment coefficient αNv for the NBI illumination light read from the memory 47a (refer to...). Figure 9 (NBI column) (Step S1).

[0188] Then, the light source device 3 emits NBI illumination light of a set amount (step S2), and the imaging element 21b captures the reflected light from the subject to generate a first imaging signal. During this imaging, the B pixel mainly captures the reflected purple light, and the G pixel mainly captures the reflected green light.

[0189] Simultaneous processing unit 43 allocates the image signal corresponding to the camera signal obtained from the B pixel to the R channel and the B channel, and allocates the image signal corresponding to the camera signal obtained from the G pixel to the G channel, and then performs simultaneous processing (de-mosaic processing) to generate, for example, a first image signal (R1, G1, B1) with RGB components (step S3).

[0190] The first image signal (R1, G1, B1) is stored in the first memory 44a and converted into a first image signal (L1*, a1*, b1*) in the L*a*b* color space by the L*a*b* conversion unit 44c (step S4).

[0191] Next, the control unit 47 and the light source controller 31 set the light intensity of the purple LED 32a and green LED 32c for emitting the second illumination light (color emphasis light CE of NBI illumination light) in the NBI observation mode (step S5).

[0192] First, to suppress variations in brightness (or luminance), the light intensity of the green LED 32c in the color-emphasis light CE is essentially set to be the same as the light intensity GN of the green LED 32c in the NBI illumination light, which is the same as the case in the normal observation mode described above. Additionally, to achieve the second light intensity ratio, the light intensity of the green LED 32c in the color-emphasis light CE is varied as needed, which is also the same as the case in the normal observation mode described above.

[0193] Furthermore, the control unit 47 reads the color adjustment coefficient Kvx of the second light intensity ratio from the memory 47a ("x" represents the part number, x=1 for the first part, x=2 for the second part, and x=3 for the third part), and multiplies it by the light intensity αNv×GN of the purple LED 32a in the NBI illumination light, thereby calculating the light intensity Gv of the purple LED 32a = αNv×GN×Kvx (refer to...). Figure 9 (First to third CE columns).

[0194] Here, the color adjustment coefficient Kvx is a coefficient that makes the color of the reference portion of the subject in the second image (R2, G2, B2) stored in the second memory 44b of the color emphasis unit 44 become achromatic (white or other grayscale), that is, a coefficient that is (a2*, b2*)≈(0,0).

[0195] Then, the light source device 3 generates a color-enhancing light CE as a second illumination light by emitting green light and purple light with a set second light intensity ratio (step S6).

[0196] The imaging element 21b captures the reflected light from the subject illuminated by the color-intensity light CE, generating a second imaging signal. During this imaging, the B pixel primarily captures the reflected violet light, and the G pixel primarily captures the reflected green light.

[0197] Simultaneous processing unit 43 allocates the image signal corresponding to the camera signal obtained from the B pixel to the R channel and the B channel, and allocates the image signal corresponding to the camera signal obtained from the G pixel to the G channel, and then performs simultaneous processing (de-mosaic processing) to generate a second image signal (R2, G2, B2) (step S7).

[0198] The second image signal (R2, G2, B2) is stored in the second memory 44b and converted into a second image signal (L2*, a2*, b2*) in the L*a*b* color space by the L*a*b* conversion unit 44c (step S8).

[0199] Then, the synthesis unit 44d synthesizes the first image signal (L1*, a1*, b1*) and the second image signal (L2*, a2*, b2*) using any of the methods described above for the normal viewing mode to generate a color-emphasized corrected image signal (step S9).

[0200] Thus, after the composite unit 44d generates the corrected image signal, the RGB conversion unit 44e converts the corrected image signal in the L*a*b* color space into a signal in the RGB color space and outputs it (step S10).

[0201] Then, the sharpness enhancement unit 45 performs sharpness enhancement processing, and the display control unit 46 generates an image signal. While generating the image signal, the display control unit 46 performs pseudo-color processing, allocating the image signal from the G channel to the R channel, allocating the image signal from the B channel to the G channel, and allocating the image signal from the B channel to the B channel. Based on the image signal generated in this way, a corrected image of the NBI viewing mode with color enhancement is displayed on the display 5 (step S11).

[0202] Then, as described above, it is determined whether to end the process (step S12). If it has not ended, return to step S1; if it has ended, return to the main process (not shown).

[0203] In addition, Figure 9 The example shown illustrates how the color adjustment factor Kvx varies depending on the location of the subject, but it can also be found in reference [reference missing]. Figure 8 As explained, the color adjustment coefficient Kv varies depending on the distance and angle.

[0204] In addition, in the above description, NBI illumination was performed by a combination of purple LED 32a and green LED 32c, but for example, a second NBI illumination can be performed by further adding an amber LED that emits amber light (so-called 5LED structure) to the light source device 3.

[0205] In this case, the control unit 47 controls the light source device 3 to generate a second NBI illumination light (first illumination light) by emitting red light, green light and amber light in a first light intensity ratio, and to generate a color-enhancing light as the second illumination light by emitting red light, green light and amber light in a second light intensity ratio.

[0206] Furthermore, color emphasis can be applied in the same way as described above, not only in NBI observation mode, but also in special light observation modes such as infrared light observation mode or fluorescence observation mode.

[0207] Figure 12 It is a graph showing an example of how the amount of color emphasis changes based on the distance of the components (a1*, b1*) of the first image signal in the a*b* plane from the L* axis.

[0208] For example, when the components (a1*, b1*) of the first image signal are color-emphasized with a certain amount of color emphasis, the color emphasis becomes excessive in areas with high chroma, and the calculated values ​​sometimes exceed the achievable color range.

[0209] Therefore, as Figure 12 As shown, the color emphasis is preferably varied according to the distance of the components (a1*, b1*) of the first image signal from the L* axis.

[0210] First, the distance of the component (a1*, b1*) of the first image signal from the L* axis is calculated as the distance r1 from the origin (0, 0) in the a*b* plane, as shown in the following mathematical formula 5.

[0211] [Number 5]

[0212]

[0213] Similarly, the distance of the corrected image signal component (as*, bs*) from the L* axis is calculated as the distance rs from the origin (0, 0) in the a*b* plane, as shown in the following mathematical formula 6.

[0214] [Number 6]

[0215]

[0216] Figure 12 The dashed lines in the diagram represent the case where distances rs and r1 are equal, without any color emphasis.

[0217] on the other hand, Figure 12 The functions f1 to f3 shown represent several examples of functions f that emphasize color based on distance r1. Function f uses a constant k (0 < k) and γ (0 < γ < 1) to represent exponentiation, as shown in, for example, in the following mathematical expression 7.

[0218] [Number 7]

[0219] rs=f(r1)=k·r1γ

[0220] Here, the power γ primarily determines the shape of the curve of function f. When the value of γ is close to 1, as shown by the double-dotted line function f3, the color emphasis decreases. When the value of γ is close to 0, as shown by the solid line function f1, the color emphasis increases. When the value of γ is moderate, as shown by the single-dotted line function f2, the color emphasis is moderate. That is, if we set γ1 for function f1, γ2 for function f2, and γ3 for function f3, then the relationship 0 < γ1 < γ2 < γ3 < 1 exists.

[0221] Furthermore, these functions f1 to f3 are examples of processor 4 setting the emphasis coefficient C in such a way that as the distance r1 from the L* axis in the a*b* plane increases, the emphasis coefficient C decreases monotonically and gradually approaches 0.

[0222] Alternatively, the setting method for the emphasis coefficient C represented by these functions f1 to f3 can be prepared in advance, for example, a setting method that allows the user to select a color emphasis amount that corresponds to their preference.

[0223] In addition, such as Figure 12As shown in the upper right corner of the graph, the maximum value of distance r1 is equal to the maximum value of distance rs, therefore even with... Figure 12 The method shown for color emphasis will not exceed the achievable color range.

[0224] Therefore, by utilizing Figure 12 The setting method shown changes the emphasis coefficient C, which can suppress excessive color emphasis in high-chroma areas of the color space and effectively emphasize colors in low-chroma areas.

[0225] then, Figure 13 This is an example diagram showing an area where color is emphasized in the a*b* plane.

[0226] In the a*b* plane, the area where color is emphasized is as follows: Figure 13 As shown, it can also be limited to a certain range.

[0227] That is, for the components (a1*, b1*) of the first image signal, pixels whose distance r1 from the origin (0, 0) on the a*b* plane is less than a specified distance rth (component (a1*, b1*) enter the specified distance rth. Figure 13 The pixels in the shaded area shown are color-emphasized, but pixels at a distance r1 greater than or equal to a predetermined distance rth are not color-emphasized. The components (a1*, b1*) of the first image signal are directly used as the components (as*, bs*) of the corrected image signal. Furthermore, smoothing can be performed in a way that prevents abrupt changes in color emphasis at distance r1 = rth.

[0228] In addition, in the above description, the emphasis coefficient for component a* and the emphasis coefficient for component b* are both set to the same C, but this is not a limitation. The emphasis coefficients for components a* and b* can also be set separately.

[0229] According to this first embodiment, a first image signal related to a first illumination light used for observing the subject is generated, a second image signal related to a second illumination light that makes a reference portion of the subject colorless is generated, and the first image signal is color-emphasized based on the second image signal, so that minute color differences can be emphasized without reducing color reproducibility.

[0230] At this point, in the second image signal, the portion with a spectral reflectance different from the reference portion does not become achromatic but has color. Therefore, by simply adding the second image signal to the first image signal, color emphasis based on color difference can be performed.

[0231] Furthermore, since color enhancement is performed using a second image signal obtained by illuminating a second light, unlike the case where color enhancement is performed by only processing the first image signal, the generation of artifacts dependent on image processing conditions can be suppressed.

[0232] In addition, the first and second image signals are converted from RGB color space signals to L*a*b* color space signals. The a* and b* components of the second image signal are added to the a* and b* components of the first image signal respectively to emphasize the color. Therefore, the luminance component L* is separated, and hue and chroma can be processed efficiently to emphasize the color.

[0233] At this point, after multiplying the a* and b* components of the second image signal by the emphasis coefficient C, they are added to the a* and b* components of the first image signal, respectively, thereby controlling the amount of color emphasis.

[0234] Furthermore, by setting the emphasis coefficient C in a manner that monotonically decreases and gradually approaches 0 as the distance r1 of the first image signal from the L* axis increases, it is possible to suppress excessive color emphasis in high chroma regions and effectively emphasize colors in low chroma regions.

[0235] Furthermore, by not emphasizing color when the distance r1 from the first image signal to the L* axis is greater than or equal to a predetermined distance rth, it is possible to suppress color emphasis in areas with high chroma.

[0236] Furthermore, by correcting the brightness component based on mathematical formula 3, it is possible to obtain a brightness component of the corrected image signal that appropriately balances the brightness of the first image signal and the brightness of the second image signal.

[0237] On the other hand, when using mathematical formula 4 to extend the distance r1 of the first image signal from the L* axis based on the distance r2 of the second image signal from the L* axis, color emphasis can be achieved without changing the hue.

[0238] In addition, when the first illumination light is set to white light WL and the color emphasis mode is turned on, it is possible to observe the image in the normal viewing mode by emphasizing its color.

[0239] At this time, since the green light in the second illumination light and the green light in the first illumination light are made to be as equal in quantity as possible, the variation in brightness (or luminance) can be suppressed.

[0240] However, as needed, within the range of maximum light intensity for each color, the light intensity of each color is reduced while maintaining the second light intensity ratio. Therefore, even if the green light of the second illumination light cannot be made to have the same light intensity as the green light of the first illumination light, a corrected image with appropriate color emphasis can still be obtained.

[0241] Furthermore, when the first illumination light is used as the NBI illumination light and the color emphasis mode is enabled, the image in the NBI observation mode can be observed with color emphasis. At this time, color-emphasis-based observation can be performed on either the NBI observation mode that combines green and purple light or the second NBI observation mode that combines red, green, and amber light.

[0242] Furthermore, by adjusting the second light intensity ratio in real time, it is possible to provide appropriate color emphasis for each frame of a dynamic image at all times.

[0243] It should be noted that the above explanation uses the CIE L*a*b* color space as an example, but it is not limited to this; other color systems can also be used. For example, the YCrCb luminance color space can also be used. In this case, in the above explanation, simply replace L* with Y, a* with Cr, and b* with Cb.

[0244] Furthermore, the above description mainly describes the case of an endoscope device equipped with a processor, but it is not limited thereto. The present invention may be a single processor, a color enhancement method that performs color enhancement in the same way as the endoscope device, a computer program for enabling a computer to perform the same processing as the endoscope device, a computer-readable non-transitory recording medium for recording the computer program, etc.

[0245] Furthermore, the present invention is not limited to the embodiments described above, and can be further modified and embodied by adapting the constituent elements without departing from its spirit during the implementation phase. Additionally, various inventive methods can be formed by appropriately combining the multiple constituent elements disclosed in the above embodiments. For example, several constituent elements may be deleted from all the constituent elements shown in the embodiments. Furthermore, constituent elements from different embodiments may be appropriately combined. Thus, various modifications and applications can be made without departing from the spirit of the invention.

Claims

1. An endoscope device characterized by comprising: having: a light source device including a plurality of semiconductor light emitting elements that emit light having different center wavelengths, and configured to generate illumination light by causing the plurality of semiconductor light emitting elements to emit light in a certain light quantity ratio; an endoscope including an imaging element configured to image return light from an object that has been irradiated with the illumination light, and generate an imaging signal having a plurality of color components; and a processor configured to perform the following processing: controlling the light source device to switch between a first illumination light obtained by causing the plurality of semiconductor light emitting elements to emit light in a first light quantity ratio and a second illumination light obtained by causing the plurality of semiconductor light emitting elements to emit light in a second light quantity ratio different from the first light quantity ratio, generating a first image signal based on an imaging signal obtained by imaging an object that has been illuminated with the first illumination light, generating a second image signal based on an imaging signal obtained by imaging the object that has been illuminated with the second illumination light, generating a corrected image signal based on color emphasis of the second image signal and the first image signal, wherein the second illumination light is obtained by adjusting the second light quantity ratio in such a manner that the second image signal relating to a reference portion of the object becomes achromatic within a prescribed error range.

2. The endoscope device according to claim 1, wherein the processor performs the following processing: converting the first image signal and the second image signal from signals in an RGB color space to signals in a CIE L*a*b* color space, in the CIE L*a*b* color space, adding the a* component and the b* component of the second image signal to the a* component and the b* component of the first image signal, respectively, thereby performing color emphasis on the first image signal.

3. The endoscope device according to claim 2, wherein the processor performs color emphasis on the first image signal by adding the a* component and the b* component of the second image signal to the a* component and the b* component of the first image signal, respectively, after multiplying the a* component and the b* component of the second image signal by an emphasis coefficient.

4. The endoscope device according to claim 3, wherein the processor sets the emphasis coefficient in such a manner that the emphasis coefficient monotonically decreases and gradually approaches 0 as a distance of the first image signal from an L* axis in an a*b* plane of the CIE L*a*b* color space increases.

5. The endoscope device according to claim 4, wherein the processor does not perform color emphasis in a case where the distance of the first image signal from the L* axis in the a*b* plane of the CIE L*a*b* color space is equal to or greater than a prescribed distance.

6. The endoscope device according to claim 3, wherein the processor further multiplies a difference value obtained by subtracting the L* component of the first image signal from the L* component of the second image signal by a second emphasis coefficient, and adds the L* component of the first image signal to the product.

7. The endoscope device according to claim 1, wherein the processor performs the following processing: ​ ​ ​ ​ ​ ​ ​ converting the first and second image signals from signals in an RGB color space to signals in a CIE L*a*b* color space, lengthening a distance of the first image signal from an L* axis in an a*b* plane of the CIE L*a*b* color space according to a distance of the second image signal from the L* axis in the a*b* plane, thereby color emphasizing the first image signal.

8. The endoscope apparatus according to claim 1, wherein the light source device includes a semiconductor light emitting element that emits red light, a semiconductor light emitting element that emits green light, and a semiconductor light emitting element that emits blue light, the processor controls the light source device, causes the light source device to generate white light as the first illumination light by emitting red light, green light, and blue light in the first light amount ratio, causes the light source device to generate color emphasized light as the second illumination light by emitting red light, green light, and blue light in the second light amount ratio.

9. The endoscope apparatus according to claim 8, wherein the processor performs the following processing: when the light amount of green light in the second illumination light is the same as the light amount of green light in the first illumination light, calculates the light amounts of red light and blue light based on the second light amount ratio, when both the calculated light amounts of red light and blue light are below the maximum light amount, sets the light amount of green light in the second illumination light to be the same as the light amount of green light in the first illumination light, when at least one of the calculated light amounts of red light and blue light exceeds the maximum light amount, sets to reduce the light amounts of green light, red light, and blue light while maintaining the second light amount ratio so that both the light amounts of red light and blue light are below the maximum light amount.

10. The endoscope apparatus according to claim 1, wherein the light source device includes a semiconductor light emitting element that emits green light and a semiconductor light emitting element that emits purple light, the processor controls the light source device, causes the light source device to generate NBI illumination light as the first illumination light by emitting green light and purple light in the first light amount ratio, causes the light source device to generate color emphasized light as the second illumination light by emitting green light and purple light in the second light amount ratio.

11. The endoscope apparatus according to claim 1, wherein the light source device includes a semiconductor light emitting element that emits red light, a semiconductor light emitting element that emits green light, and a semiconductor light emitting element that emits amber light, the processor controls the light source device, causes the light source device to generate NBI illumination light as the first illumination light by emitting red light, green light, and amber light in the first light amount ratio, causes the light source device to generate color emphasized light as the second illumination light by emitting red light, green light, and amber light in the second light amount ratio.

12. The endoscope apparatus according to claim 1, wherein The processor adjusts the second light amount ratio in real time so that the second image signal becomes achromatic within a prescribed error range.

13. The endoscope apparatus according to claim 1, wherein The second illumination light is set so that normal tissue in the second image signal becomes achromatic.

14. The endoscope apparatus according to claim 2, wherein The a* component and the b* component in the second image signal each take a value of 0 within a prescribed error range.

15. A processor, comprising: The processor is configured to perform the following processing: control a light source apparatus to switch a first illumination light and a second illumination light, the first illumination light being obtained by emitting a plurality of lights having different center wavelengths at a first light amount ratio, the second illumination light being obtained by emitting the plurality of lights at a second light amount ratio different from the first light amount ratio, generate a first image signal based on an imaging signal obtained by imaging an object illuminated by the first illumination light, generate a second image signal based on an imaging signal obtained by imaging the object illuminated by the second illumination light, generate a corrected image signal based on color emphasis of the second image signal and the first image signal, wherein the second illumination light is obtained by adjusting the second light amount ratio in a manner so that the second image signal related to a reference portion of the object becomes achromatic within a prescribed error range.

16. A color emphasis method, comprising: switching a first illumination light and a second illumination light to emit light, the first illumination light being obtained by setting a light amount ratio of a plurality of lights having different center wavelengths to a first light amount ratio, the second illumination light being obtained by setting the light amount ratio of the plurality of lights having different center wavelengths to a second light amount ratio different from the first light amount ratio, generate a first image signal based on an imaging signal obtained by imaging an object illuminated by the first illumination light, generate a second image signal based on an imaging signal obtained by imaging the object illuminated by the second illumination light, generate a corrected image signal based on color emphasis of the second image signal and the first image signal, adjust the second light amount ratio in a manner so that the second image signal related to a reference portion of the object becomes achromatic within a prescribed error range.

Images