Imaging device and imaging system

Abstract

Provided is an imaging device that makes it possible to improve the image quality of an oxygen saturation image while maintaining the image quality of a color image.　This imaging device comprises: an imaging unit that captures an image of a subject and generates a red signal, a blue signal, a green signal, and an infrared signal; and an image processing unit that processes the signals generated by the imaging unit to generate images. The image processing unit generates an oxygen saturation image indicating the distribution of the oxygen saturation of the subject by synthesizing the red signal and the infrared signal, and generates a color image of the subject by correcting either or both of the red signal and the blue and green signals, and then synthesizing these signals.

Description

Imaging device and imaging system

[0001] The present invention relates to an imaging device and an imaging system.

[0002] Imaging devices that simultaneously capture a color image of a subject and an oxygen saturation image showing the distribution of oxygen saturation using light irradiated onto the human body and reflected light have been known. For example, Patent Document 1 discloses a technology for generating an oxygen saturation image by irradiating the subject with normal light and specific light (800 nm). Patent Document 2 discloses a spectroscopic prism that splits incident light into red, blue, green, and infrared light.

[0003] Patent No. 7346357 Patent No. 6917183

[0004] However, as in Patent Document 1, there is a problem in that it is difficult to obtain an oxygen saturation image with the image quality required for diagnosis by only combining normal light and specific light.

[0005] The present invention has been made in consideration of the above-mentioned situation, and its purpose is to provide an imaging device that can improve the image quality of oxygen saturation images while maintaining the image quality of color images.

[0006] (Invention 1) In order to solve the above problems, the imaging device of the present invention comprises an imaging unit that images a subject and generates red, blue, green, and infrared signals, and an image processing unit that processes the signals generated by the imaging unit to generate an image, wherein the image processing unit generates an oxygen saturation image showing the distribution of oxygen saturation in the subject by combining the red and infrared signals, and generates a color image of the subject by correcting at least one of the red, blue, and green signals and then combining them.

[0007] Furthermore, the following configurations can be exemplified as preferred embodiments of the present invention.

[0008] (Invention 2) In the imaging device according to (Invention 1) above, the image processing unit corrects the representative value of the red signal, the representative value of the blue signal, and the representative value of the green signal so that they approach each other.

[0009] (Invention 3) In the imaging device described above in (Invention 2), the image processing unit corrects the red signal to generate a corrected red signal so that a representative value of the red signal approaches a representative value of the blue signal and a representative value of the green signal, and generates the color image by combining the corrected red signal, the blue signal, and the green signal.

[0010] (Invention 4) In the imaging device described above in (Invention 2), the image processing unit: corrects the blue signal to generate a corrected blue signal so that a representative value of the blue signal approaches a representative value of the red signal; corrects the green signal to generate a corrected green signal so that a representative value of the green signal approaches a representative value of the red signal; generates the color image by combining the red signal, the corrected blue signal, and the corrected green signal; and further performs gamma correction on the color image with a correction coefficient γ<1.

[0011] (Invention 5) In the imaging device described in (Invention 1) above, the imaging unit comprises: a spectral prism that spectrally separates incident light into infrared light, blue light, red light, and green light; an infrared imaging element that photoelectrically converts the infrared light spectrally separated by the spectral prism to generate an infrared image as the infrared signal; a blue imaging element that photoelectrically converts the blue light spectrally separated by the spectral prism to generate a blue image as the blue signal; a red imaging element that photoelectrically converts the red light spectrally separated by the spectral prism to generate a red image as the red signal; and a green imaging element that photoelectrically converts the green light spectrally separated by the spectral prism to generate a green image as the green signal; and the image processing unit generates the oxygen saturation image by combining the red image and the infrared image, and generates the color image by combining the red image, the blue image, and the green image after correcting at least one of them.

[0012] (Invention 6) An imaging system comprising: a visible light source that irradiates a subject with visible light in which the amount of red light is greater than that of blue light and green light; an infrared light source that irradiates the subject with infrared light; and the imaging device according to claim 1 that images the subject using reflected light of the light irradiated from the visible light source and the infrared light source.

[0013] (Invention 7) In the imaging system according to (Invention 6) above, the visible light source includes a white light source that emits white light, and a red light source that emits red light.

[0014] (Invention 8) In the imaging system described above (Invention 7), the image processing unit reduces the amount of red light emitted from the red light source when the histogram of the red signal is saturated.

[0015] According to the present invention, even if the amount of red light is increased to improve the quality of the oxygen saturation image, the imbalance in the color image can be eliminated. That is, the quality of the oxygen saturation image can be improved while maintaining the quality of the color image. Note that the problems, configurations, and advantages other than those described above will become clear from the description of the following embodiments.

[0016] FIG. 1 is a schematic diagram of an imaging system; FIG. 2 is a diagram showing an example of the structure of a spectral prism; FIG. 3 is a flowchart of imaging control processing; FIG. 4 is a diagram showing an example of a histogram of luminance values; FIG. 5 is a diagram explaining color image generation method 1; FIG. 6 is a diagram explaining color image generation method 2; FIG. 7 is a diagram showing another application example of color image generation method 1; and FIG. 8 is a diagram showing another application example of color image generation method 2.

[0017] Hereinafter, an embodiment of the invention will be described with reference to the drawings. The embodiment realizes a highly versatile imaging device and imaging system, thereby contributing to "9. Build resilient infrastructure, promote inclusive and sustainable industrialization, promote innovation and build resilient infrastructure" of the Sustainable Development Goals (SDGs) advocated by the United Nations.

[0018] [Configuration of Imaging System 1] FIG. 1 is a schematic diagram of the imaging system 1. The imaging system 1 is a system that irradiates a subject with light to capture an image, processes the captured image to generate an oxygen saturation image and a color image, and outputs the generated oxygen saturation image and color image. As shown in FIG. 1, the imaging system 1 mainly includes, for example, an illumination unit 10, an imaging unit 20, an image processing unit 30, and a display unit 40. Note that the imaging system 1 according to this embodiment may omit the display unit 40 and output the oxygen saturation image and color image to an external display or external memory. The present invention may also be conceived as an imaging device including the imaging unit 20 and the image processing unit 30.

[0019] The subject may be, for example, the outer surface of a human body, the inside of a human body after thoracotomy or abdominal surgery, or the inside of a human body observed with an endoscope. The oxygen saturation image is an image showing the distribution of oxygen saturation in the subject (an image that visualizes areas with high and low oxygen saturation). The color image is an image of the subject captured in color. In other words, the imaging system 1 can be used primarily in the medical field.

[0020] As an example, when the imaging system 1 is installed in an operating room, the illumination unit 10 is composed of a ceiling light or a shadowless lamp (white light) and a red light source and an infrared light source installed separately. The imaging unit 20 is installed in a location where it can capture images of the surgical field. The display unit 40 is installed in a location where it can be seen by the surgeon (operator) performing the surgery. As another example, when the imaging system 1 is applied to a rigid endoscope system, the imaging unit 20 is built into the rigid endoscope, and the illumination unit 10 and the image processing unit 30 are connected to the rigid endoscope. Light emitted from the illumination unit 10 is irradiated onto the surgical field from the tip of the rigid endoscope, and the reflected light is incident on the tip of the rigid endoscope and received by the imaging unit 20. However, the specific use of the imaging system 1 is not limited to the above example.

[0021] The illumination unit 10 irradiates a subject with light. More specifically, the illumination unit 10 irradiates the subject with visible light including red light, blue light, green light, and infrared light, and infrared light. Furthermore, the visible light irradiated from the illumination unit 10 has a greater amount of red light than blue light and green light. The illumination unit 10 according to this embodiment includes a visible light source 11 that irradiates the subject with visible light, and an infrared light source 12 that irradiates the subject with infrared light.

[0022] The visible light emitted by the visible light source 11 is, for example, white light with a wavelength in the range of 380 nm to 760 nm. Furthermore, the visible light emitted by the visible light source 11 has, for example, a greater amount of red light than other wavelengths. As an example, the visible light source 11 may include a white light source that emits white light and a red light source that emits red light of a single wavelength (e.g., 660 nm). As another example, the visible light source 11 may include a red light source that emits red light of a single wavelength, a blue light source that emits blue light of a single wavelength, and a green light source that emits green light of a single wavelength. The infrared light source 12 emits infrared light of a single wavelength (e.g., 850 nm). The light emitted from the illumination unit 10 may be diffused light or highly directional laser light.

[0023] [Configuration of Image Capturing Unit 20] The image capturing unit 20 captures an image of a subject and generates a red image, a blue image, a green image, and an infrared image. The image capturing unit 20 is a camera unit that integrates an optical lens 21, a spectral prism 22, and image sensors 23, 24, 25, and 26 (image capturing elements). The optical lens 21 collects light outside the image capturing unit 20 (i.e., reflected light irradiated onto the subject from the illumination unit 10) and makes it incident on the spectral prism 22. The spectral prism 22 disperses the incident light that has entered through the optical lens 21 and outputs the light to the image sensors 23, 24, 25, and 26, respectively.

[0024] 2 is a diagram showing an example of the structure of the spectral prism 22. The configuration of the spectral prism 22 is already well known, as disclosed in Patent Document 2, and therefore a detailed description will be omitted. However, for example, the spectral prism 22 is configured as follows: The spectral prism 22 is composed of a first prism 221, a second prism 222, a third prism 223, and a fourth prism 224. The first prism 221, the second prism 222, and the third prism 223 each have an incident surface 221a, 222a, and 223a, a reflecting surface 221b, 222b, and 223b, and an exit surface 221c, 222c, and 223c. The fourth prism 224 has an incident surface 224a and an exit surface 224c.

[0025] The incident surface 221a of the first prism 221 is disposed perpendicular to the optical axis of the incident light L (split light L1+L2+L3+L4). The exit surface 221c of the first prism 221 faces the image sensor 23. The exit surface 222c of the second prism 222 faces the image sensor 24. The exit surface 223c of the third prism 223 faces the image sensor 25. The exit surface 224c of the fourth prism 224 faces the image sensor 26.

[0026] The reflecting surface 221b of the first prism 221 and the incident surface 222a of the second prism 222 form a joint surface A that is inclined with respect to the optical axis of the incident light L, downstream of the incident surface 221a in the optical path of the incident light L. Therefore, of the incident light L that enters the first prism 221 through the incident surface 221a, a portion (split light L1) is reflected at the joint surface A toward the incident surface 221a, and the rest (split light L2+L3+L4) is transmitted and enters the second prism 222. The optical axis of the split light L2+L3+L4 coincides with the optical axis of the incident light L.

[0027] Furthermore, the reflecting surface 222b of the second prism 222 and the incident surface 223a of the third prism 223 form a joint surface B that is tilted with respect to the optical axis of the light beams L2+L3+L4, downstream of the joint surface A in the optical path of the light beams L2+L3+L4. Therefore, of the light beams L2+L3+L4 that enter the second prism 222 through the incident surface 222a, a portion (light beam L2) is reflected at the joint surface B toward the incident surface 222a, and the other portion (light beam L3+L4) is transmitted and enters the third prism 223. The optical axis of the light beams L3+L4 coincides with the optical axis of the incident light L.

[0028] Furthermore, the reflecting surface 223b of the third prism 223 and the incident surface 224a of the fourth prism 224 form a joint surface C that is inclined with respect to the optical axis of the split light L3+L4, downstream of the joint surface B in the optical path of the split light L3+L4. Therefore, of the split light L3+L4 that enters the third prism 223 through the incident surface 223a, a portion (split light L3) is reflected at the joint surface C toward the incident surface 223a, and the other portion (split light L4) is transmitted and enters the fourth prism 224. The optical axis of the split light L4 coincides with the optical axis of the incident light L.

[0029] Then, the split light beam L1 reflected by the bonding surface A is totally reflected by the incident surface 221a and is guided to the image sensor 23 through the exit surface 221c. The split light beam L2 reflected by the bonding surface B is totally reflected by the bonding surface A and is guided to the image sensor 24 through the exit surface 222c. The split light beam L3 reflected by the bonding surface C is totally reflected by the bonding surface B and is guided to the image sensor 25 through the exit surface 223c. Furthermore, the split light beam L4 incident on the fourth prism 224 through the incident surface 224a is guided to the image sensor 26 through the exit surface 224c.

[0030] That is, the light splitting prism 22 splits (spectralizes) the incident light L into light beams L1, L2, and L3, and guides them to the image sensors 23, 24, 25, and 26. The light beam L1 is an example of infrared light, the light beam L2 is an example of blue light, the light beam L3 is an example of red light, and the light beam L4 is an example of green light.

[0031] The image sensors 23, 24, 25, and 26 are an assembly of a plurality of pixels arranged in a matrix. The image sensors 23 to 26 photoelectrically convert the light beams L1, L2, L3, and L4 output from the spectral prism 22 to generate images of each color. The image sensors 23 to 26 then transmit image data representing the images of each color to the image processing unit 30 via a cable. The image sensors 23 to 26 are, for example, complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD).

[0032] Image sensor 23 is an example of an "infrared imaging element," image sensor 24 is an example of a "blue imaging element," image sensor 25 is an example of a "red imaging element," and image sensor 26 is an example of a "green imaging element." Furthermore, the image captured by image sensor 23 is an example of an "infrared image (infrared signal)," the image captured by image sensor 24 is an example of a "blue image (blue signal)," the image captured by image sensor 25 is an example of a "red image (red signal)," and the image captured by image sensor 26 is an example of a "green image (green signal)."

[0033] 1 , the image processing unit 30 is, for example, an integrated circuit product in which various electronic components are integrated on a single chip configured by an integrated circuit. The image processing unit 30 includes, for example, a memory 31, a CPU (Central Processing Unit) 32, and an ISP (Image Signal Processing) 33.

[0034] The memory 31 includes at least a RAM (Random Access Memory) and a ROM (Read Only Memory). The memory 31 temporarily stores programs and control data required for executing the operations of the image processing unit 30, as well as data or information generated during operation of each unit of the image processing unit 30. The RAM is, for example, a work memory used when each unit of the image processing unit 30 operates. The ROM stores and holds, in advance, programs and control data for controlling each unit of the image processing unit 30, for example.

[0035] The CPU 32 is a processor that controls the overall operation of the image processing unit 30. The CPU 32 performs control processing for overseeing the operation of each unit of the image processing unit 30, data input / output processing between each unit of the image processing unit 30, data calculation processing, and data storage processing. The CPU 32 operates in accordance with the program and control data stored in the memory 31. The CPU 32 uses the memory 31 during operation, and transfers data generated or acquired by the CPU 32 to the memory 31 for temporary storage.

[0036] The ISP 33 is a processor that executes various types of image processing within the image processing unit 30. The ISP 33 reads image data from the memory 31 and performs various types of image processing using the read image data. The ISP 33 uses the memory 31 during operation, and transfers data or information generated or acquired by the ISP 33 to the memory 31 for temporary storage.

[0037] [Configuration of Display Unit 40] The display unit 40 displays the oxygen saturation image and color image output from the image processing unit 30 in the imaging control process described below. As the display unit 40, for example, a known display can be used.

[0038] [Image capture control process] Fig. 3 is a flowchart of the image capture control process. The image capture control process involves capturing an image of a subject using light irradiated from the illumination unit 10, processing the captured image to generate an oxygen saturation image and a color image, and displaying the generated oxygen saturation image and color image on the display unit 40. The image processing unit 30 starts the image capture control process shown in Fig. 3 in response to, for example, an instruction to start the image capture control process being received from an operator via an operation unit (not shown).

[0039] First, the image processing unit 30 turns on the illumination unit 10 to irradiate the subject with predetermined amounts of light (white light, red light, and infrared light) (S11). The visible light source 11 according to this embodiment irradiates not only white light containing red, blue, and green light but also red light of a single wavelength (660 nm), so that the amount of red light irradiated onto the subject is greater than the amounts of blue light and green light. The infrared light source 12 also irradiates infrared light of a single wavelength (850 nm). Note that, in order to suppress the contribution of blue light and green light to the red and infrared images, it is preferable that the amount of light from the white light source be less than the amount of light from the red light source, and less than the amount of light from the white light source.

[0040] Next, the image processing unit 30 causes the imaging unit 20 to capture an image of the subject and acquires the image data generated by the imaging unit 20. The image processing unit 30 then generates a histogram of pixel values ​​for each color image (S12). FIG. 4 is a diagram showing an example of a histogram of luminance values. As shown in FIG. 4, each color image has a plurality of pixels (V11, V12, ...) arranged in a matrix. Each pixel is assigned a pixel value ranging from 0 to 255. The pixel value is a value that indicates the degree of light intensity (gradation) of the pixel. In other words, the larger the pixel value of a pixel, the higher the gradation, and the smaller the pixel value of a pixel, the lower the gradation.

[0041] The image processing unit 30 then counts the number of pixels having the same pixel value for each color image, and generates a histogram with the pixel value on the horizontal axis and the pixel count on the vertical axis, as shown in Fig. 4, for example. This histogram often exhibits a normal distribution centered around the mode M, as shown in the upper right diagram of Fig. 4. However, because the maximum pixel value is 255, if the amount of light emitted from the illumination unit 10 is too large, there is a possibility that a portion to the right of the mode M will fall outside the histogram, as shown in the lower right diagram of Fig. 4. In this embodiment, the state shown in the lower right diagram of Fig. 4 is referred to as a "saturated histogram."

[0042] Next, the image processing unit 30 determines whether the histogram of each color image is saturated (S13). For example, as shown in Fig. 4, the image processing unit 30 may determine that the histogram is saturated when the sum of the mode M and the standard deviation σ is greater than the maximum pixel value (M + σ > 255). However, the specific method for determining whether the histogram is saturated is not particularly limited.

[0043] Next, if the image processing unit 30 determines that at least one of the histograms of each color image is saturated (S13: No), it adjusts the light intensity of the illumination unit 10 (S14) and repeats the process from step S11 onward. In this embodiment, the light intensity of red light is greater than that of the other colors, so the histogram of the red image is most likely to be saturated. In this case, the image processing unit 30 reduces the light intensity of red light emitted from the red light source.

[0044] Note that the image processing unit 30 may notify the operator of the need to adjust the light intensity, instead of directly instructing the illumination unit 10 to adjust the light intensity via a cable (not shown). The image processing unit 30 repeatedly executes the processes of steps S11 to S14 until it determines that the histograms of all colors are not saturated (S13: No). When it determines that the histograms of all colors are not saturated (S13: Yes), the image processing unit 30 proceeds to the processes of step S15 and subsequent steps.

[0045] Next, the image processing unit 30 generates an oxygen saturation image by combining (overlapping) the infrared image generated by the image sensor 23 and the red image generated by the image sensor 25 (S15). The method of combining the infrared image and the red image is well known, so a detailed description is omitted. The amount of reflected red light varies depending on the amount of reduced hemoglobin (Hb) in the subject. Therefore, by irradiating the subject with red light in addition to white light (i.e., increasing the amount of red light), an oxygen saturation image can be obtained that clearly shows the distribution of Hb levels compared to when only white light is irradiated (i.e., the image quality of the oxygen saturation image is improved).

[0046] The infrared image shows areas strongly influenced by HbO2, which is abundant in arterial blood. Meanwhile, the red image shows areas strongly influenced by Hb, which is abundant in venous blood. Therefore, for example, by changing the color of the infrared image and the color of the red image to create a composite image, it is possible to generate a distribution image of arteries and veins with higher resolution than conventional methods. However, because HbO2 absorbs blue and green light more easily than Hb, if the influence of other color components in the red image becomes too great, it becomes difficult to depict the distribution of venous blood.

[0047] Note that the wavelength bands of blue and green light in white light are broad, so some of that light will be incident on the red sensor. Therefore, if the light intensity of the white light source is large, the proportion of blue and green light in the light received by the red sensor will be large. Therefore, by separately irradiating red light in addition to white light, it is possible to reduce noise in the red image due to blue and green light in the white light. Therefore, it is desirable to make the light intensity of red light sufficiently greater than the light intensity of blue and green light.

[0048] Next, the image processing unit 30 generates a color image by combining the blue image generated by the image sensor 24, the red image generated by the image sensor 25, and the green image generated by the image sensor 26 (S16). Here, by irradiating the subject with red light in addition to white light (i.e., increasing the amount of red light), the balance of the red, blue, and green images that make up the color image becomes poor (i.e., the red in the color image becomes too strong).

[0049] Therefore, in order to eliminate the imbalance of each color in the color image, the image processing unit 30 generates a color image by correcting at least one of the red image, blue image, and green image and then combining them. More specifically, the image processing unit 30 generates the color image by correcting the representative value of the red image and the representative value of the blue image and the representative value of the green image so that they approach each other. A specific method for generating a color image will be described later with reference to FIGS. 5 to 8.

[0050] Next, the image processing unit 30 causes the display unit 40 to display the oxygen saturation image generated in step S15 and the color image generated in step S16 (S17). The display unit 40 may simultaneously display the oxygen saturation image and the color image side by side, or may switch between them according to the operator's selection. The image processing unit 30 then repeatedly executes the processes of steps S15 to S17 on the images captured by the imaging unit 20 at a predetermined frame rate, thereby displaying an image of the surgical field on the display unit 40.

[0051] Next, the image processing unit 30 repeatedly executes the processes of steps S15 to S17 until a predetermined frame period is reached (S18: No). Meanwhile, the image processing unit 30 executes the process of step S12 again when the predetermined frame period is reached (S18: Yes). Alternatively, the image processing unit 30 may execute the process of step S12 again when an external interrupt signal (e.g., an instruction to adjust the light intensity) is input (S18: Yes) instead of at the predetermined frame period. In other words, the adjustment of the light intensity of the illumination unit 10 does not need to be performed for each frame, but may be performed at any timing.

[0052] [Color Image Generation Method 1] Fig. 5 is a diagram illustrating color image generation method 1. In the example of Fig. 5, the histograms of the red image, blue image, and green image each have a normal distribution. In each histogram, the mode of the red image has a higher pixel value than the mode of the blue image and the mode of the green image. In the example of Fig. 5, the image processing unit 30 corrects the red image so that the representative value of the red image approaches the representative value of the blue image and the representative value of the green image.

[0053] The representative value is a value that represents an image. The representative value is also a value that is identified from the pixel values ​​of the image. Specific processing will be explained below using the average value as an example of the representative value. However, the representative value may also be the mode or median. The same applies to generation method 2, which will be described later.

[0054] First, the image processing unit 30 calculates an average value AR of the pixel values ​​of the red image, an average value AB of the pixel values ​​of the blue image, and an average value AG of the pixel values ​​of the green image. Next, the image processing unit 30 calculates a correction coefficient KR for bringing the average value AR closer to the average values ​​AB and AG. There are no particular limitations on the method for calculating the correction coefficient KR, but it may be calculated using the following equation 1, for example: KR = AR - (AB + AG) / 2 (Equation 1)

[0055] Next, the image processing unit 30 subtracts the correction coefficient KR from the pixel values ​​of all pixels in the red image to generate a corrected red image (corrected red signal). That is, the corrected red image is an image obtained by translating the histogram of the red image leftward (in the direction of decreasing pixel values). The image processing unit 30 then generates a color image by combining (overlapping) the three monochromatic images (the corrected red image, blue image, and green image). Methods for generating a color image from three monochromatic images are already well known, so a description thereof will be omitted.

[0056] [Color Image Generation Method 2] Fig. 6 is a diagram illustrating color image generation method 2. In the example of Fig. 6, the histograms of the red, blue, and green images each have a normal distribution. In each histogram, the mode of the red image has a higher pixel value than the mode of the blue and green images. In the example of Fig. 6, the image processing unit 30 corrects the blue image so that the average value of the blue image approaches the average value of the red image, and corrects the green image so that the average value of the green image approaches the average value of the red image.

[0057] First, the image processing unit 30 calculates the average value AR of the pixel values ​​of the red image, the average value AB of the pixel values ​​of the blue image, and the average value AG of the pixel values ​​of the green image. Next, the image processing unit 30 calculates a correction coefficient KB for bringing the average value AB closer to the average value AR, and a correction coefficient KG for bringing the average value AG closer to the average value AR. The correction coefficients KB and KG may be the same value or different values. There are no particular limitations on the method for calculating the correction coefficients KB and KG, but for example, if they are to be the same value, they may be calculated using Equation 1 (i.e., the same as KR), and if they are to be different values, they may be calculated using Equation 2 and Equation 3. KB = AR - AB ... (Equation 2) KG = AR - AG ... (Equation 3)

[0058] Next, image processing unit 30 adds correction coefficient KB to the pixel values ​​of all pixels in the blue image to generate a corrected blue image (corrected blue signal). That is, the corrected blue image is an image obtained by translating the histogram of the blue image to the right (in the direction to increase pixel values). Image processing unit 30 also adds correction coefficient KG to the pixel values ​​of all pixels in the green image to generate a corrected green image (corrected green signal). That is, the corrected green image is an image obtained by translating the histogram of the green image to the right (in the direction to increase pixel values).

[0059] Image processing unit 30 may adjust (decrease) correction coefficient KB calculated by equation 2 so that the histogram of the corrected blue image does not saturate. Similarly, image processing unit 30 may adjust (decrease) correction coefficient KG calculated by equation 3 so that the histogram of the corrected green image does not saturate. The same applies when correction coefficients KB and KG are set to the same value.

[0060] The image processing unit 30 then generates a color image by combining the three monochrome images (red image, corrected blue image, and corrected green image). Methods for generating a color image from three monochrome images are already well known, so a detailed description will be omitted. Here, the red image, corrected blue image, and corrected green image have high gradations, so simply combining them as is may result in overexposure and other issues. Therefore, the image processing unit 30 further performs gamma correction on the generated color image. Since the gamma correction method is already well known, a detailed description will be omitted, but the gamma coefficient γ is set to γ<1 to reduce gradation.

[0061] [Application Example to Other Images] Fig. 7 is a diagram showing another application example of color image generation method 1. In the example of Fig. 7, the histogram of the red image does not have a normal distribution, while the histograms of the blue image and green image have normal distributions. Furthermore, in each histogram, the mode of the red image has a higher pixel value than the mode of the blue image and the mode of the green image. In this way, generation method 1 can be applied to images whose histograms do not have a normal distribution. The same applies to generation method 2.

[0062] FIG. 8 is a diagram showing another application example of color image generation method 2. In the example of FIG. 8, the histogram of the red image is saturated, while the histograms of the blue image and green image are not saturated. Furthermore, in each histogram, the mode of the red image has a higher pixel value than the mode of the blue image and the mode of the green image. In this way, generation method 2 can also be applied to images with saturated histograms. The same applies to generation method 1. If an image with a saturated histogram is corrected as in FIG. 8, the processes of steps S13 and S14 can be omitted.

[0063] [Effects of the embodiment] According to the above embodiment, by increasing the amount of red light incident on the imaging unit 20, an oxygen saturation image in which the distribution of Hb levels can be clearly observed can be obtained. Furthermore, by correcting the red image and / or the blue and green images, the imbalance among the three monochromatic images can be eliminated. As a result, the image quality of the oxygen saturation image can be improved while maintaining the image quality of the color image.

[0064] Furthermore, according to generation methods 1 and 2, the imbalance between the three monochrome images can be appropriately eliminated by correcting the average value of the red image, the average value of the blue image, and the average value of the green image so that they approach each other. Furthermore, according to generation method 1, only one image needs to be corrected, so the amount of calculation required for correction can be reduced. On the other hand, according to generation method 2, images are synthesized in a high gradation state, so black crush can be prevented. Furthermore, according to generation method 2, color images are gamma corrected (γ<1), so white blowout can also be prevented.

[0065] However, the image correction method is not limited to the above example, and all of the red image, blue image, and green image may be corrected. Furthermore, the method is not limited to adding (subtracting) a correction coefficient to (from) a pixel value, and may involve multiplying (dividing) the pixel value by the correction coefficient. As yet another example, the image processing unit 30 may correct at least one of the red image, blue image, and green image based on the difference in the amount of red light, blue light, and green light irradiated from the illumination unit 10.

[0066] Furthermore, in the above embodiment, an example of a four-plate configuration including four image sensors 23 to 26 has been described, but the configuration of the image sensors is not limited to this. As another example, the imaging unit may have a two-plate configuration including an RGB sensor that photoelectrically converts red light, blue light, and green light to generate an image including red, blue, and green signals, and an IR sensor that photoelectrically converts infrared light to generate an image including an infrared signal. As yet another example, the imaging unit may have a three-plate configuration including an GB sensor that photoelectrically converts blue and green light to generate an image including blue and green signals, an R sensor that photoelectrically converts red light to generate an image including a red signal, and an IR sensor that photoelectrically converts infrared light to generate an image including an infrared signal.

[0067] Furthermore, according to the above embodiment, when the histogram of the image is saturated, a natural color image can be obtained by adjusting the amount of light from the illumination unit 10.

[0068] Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, alterations, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure. Furthermore, the components of the various embodiments described above may be combined in any manner without departing from the spirit of the invention.

[0069] The present disclosure is useful as an imaging device and an imaging system that can improve the image quality of an oxygen saturation image while maintaining the image quality of a color image.

[0070] 1: Imaging system 2: Generation method 10: Illumination unit 11: Visible light source 12: Infrared light source 20: Imaging unit 21: Optical lens 22: Spectroscopic prism 23-26: Image sensor 30: Image processing unit 31: Memory 32: CPU 40: Display unit 221: First prism 222: Second prism 223: Third prism 224: Fourth prism

Claims

1. An imaging device comprising: an imaging unit that captures an image of a subject and generates red, blue, green, and infrared signals; and an image processing unit that processes the signals generated by the imaging unit to generate an image, wherein the image processing unit generates an oxygen saturation image that shows the distribution of oxygen saturation in the subject by combining the red and infrared signals; and generates a color image of the subject by correcting at least one of the red, blue, and green signals and then combining them.

2. An imaging device according to claim 1, wherein the image processing unit corrects the representative value of the red signal, the representative value of the blue signal, and the representative value of the green signal so that they become closer to each other.

3. An imaging device according to claim 2, wherein the image processing unit corrects the red signal to generate a corrected red signal so that the representative value of the red signal approaches the representative value of the blue signal and the representative value of the green signal, and generates the color image by combining the corrected red signal, the blue signal, and the green signal.

4. An imaging device according to claim 2, wherein the image processing unit corrects the blue signal to generate a corrected blue signal so that a representative value of the blue signal approaches a representative value of the red signal, corrects the green signal to generate a corrected green signal so that a representative value of the green signal approaches a representative value of the red signal, generates the color image by combining the red signal, the corrected blue signal, and the corrected green signal, and further performs gamma correction on the color image with a correction coefficient γ<1.

5. An imaging device according to claim 1, wherein the imaging section comprises: a spectral prism that spectrally separates incident light into infrared light, blue light, red light, and green light; an infrared imaging element that photoelectrically converts the infrared light spectrally separated by the spectral prism to generate an infrared image as the infrared signal; a blue imaging element that photoelectrically converts the blue light spectrally separated by the spectral prism to generate a blue image as the blue signal; a red imaging element that photoelectrically converts the red light spectrally separated by the spectral prism to generate a red image as the red signal; and a green imaging element that photoelectrically converts the green light spectrally separated by the spectral prism to generate a green image as the green signal; and the image processing section generates the oxygen saturation image by combining the red image and the infrared image, and generates the color image by combining the red image, the blue image, and the green image after correcting at least one of them.

6. An imaging system comprising: a visible light source that irradiates a subject with visible light in which the amount of red light is greater than that of blue light and green light; an infrared light source that irradiates the subject with infrared light; and the imaging device according to claim 1 that images the subject using reflected light of the light irradiated from the visible light source and the infrared light source.

7. An imaging system according to claim 6, wherein the visible light source includes a white light source that emits white light and a red light source that emits red light.

8. An imaging system according to claim 7, wherein the image processing unit reduces the amount of red light emitted from the red light source when the histogram of the red signal is saturated.

Images