Image signal processing method

Abstract

To provide an image signal processing method which reduces noise by removing a high frequency component that may cause color noise.SOLUTION: An image signal processing method comprises the steps of: determining an image signal for determining an image signal of one color being a correction object; calculating a linear sum of an image signal of two colors other than the correction object; subtracting the linear sum from the image signal of the correction object; extracting a low frequency component from a subtraction calculation result; performing the second subtraction of subtracting the image signal passing through a low frequency extraction part from the subtraction calculation result; calculating a color difference from the image signal of three colors; determining a first coefficient on the basis of a function from the color difference; multiplying the calculation result of the second subtraction with the first coefficient determined by a color difference function part; comparing the multiplication result with any parameter; selecting any of the result and the parameter; subtracting the selection result from the image signal of the correction object; and setting the subtraction result as the image signal after correction to the image signal of the correction object.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an image signal processing method for reducing color noise. [Background technology]

[0002] In recent years, image signal processing methods have been proposed for reducing color noise, which is noise related to color in image data.

[0003] Color noise is noise that occurs when color information that should not actually exist in image data is included for some reason.

[0004] Therefore, when image data containing this color noise is output onto the rear LCD screen of a digital camera, a monitor screen, or paper media, color information that does not actually exist is displayed as an image signal, causing a sense of discomfort to the photographer.

[0005] Patent Document 1 discloses that in image data that has undergone white balance correction, color difference data is calculated for each pixel, and for image data of pixels whose color difference data is within a first color difference range and whose brightness is within a specified brightness range, color noise reduction processing is performed so that the color difference data of the pixel is reduced. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent No. 5460202 Summary of the Invention [Problem to be solved by the invention]

[0007] However, the image processing device disclosed in Patent Document 1 has a problem in that color differences are eliminated by performing color noise reduction processing to reduce color difference data as described above.

[0008] In other words, there is a problem in that the disappearance of color differences in an image that has undergone color noise reduction processing results in an achromatic image.

[0009] Furthermore, each color of the image signal contains high-frequency noise components, and the high-frequency noise generates monochromatic, isolated color noise.

[0010] Therefore, when a low-pass filter or the like is applied to suppress isolated color noise, there is a problem that the color noise may become color noise that spreads to the surrounding area.

[0011] The present invention has been made in view of the above circumstances, and has as its object to provide an image signal processing method that reduces noise by removing high frequency components that cause color noise. [Means for solving the problem]

[0012] A first invention as a means for solving the above problem is an image signal processing method for processing three-color image signals, the method comprising: an image signal determination unit for determining an image signal of one color to be corrected; a linear sum calculation unit for calculating a linear sum of image signals of two colors other than the image signal to be corrected; a first subtraction unit for subtracting the linear sum calculated by the linear sum calculation unit from the image signal to be corrected; a low-frequency extraction unit for extracting low-frequency components from the calculation result of the first subtraction unit; a second subtraction unit for subtracting an image signal that has passed through the low-frequency extraction unit from the calculation result of the first subtraction unit; a color difference function unit for calculating color differences from the three-color image signals and determining a first coefficient from the color differences based on a function; a multiplication unit for multiplying the calculation result of the second subtraction unit by the first coefficient determined by the color difference function unit; a threshold comparison and selection unit for comparing the calculation result of the multiplication unit with an arbitrary parameter and selecting either the calculation result or the parameter; choice and a noise elimination unit that subtracts the result of the selection by the noise elimination unit from the image signal to be corrected, and the result of the calculation by the noise elimination unit is used as the corrected image signal for the image signal to be corrected.

[0013] In addition, a second invention, which is a means for solving the above-mentioned problem, is an image signal processing method according to the first invention, characterized in that the color difference function unit calculates a plurality of color differences using two color image signals from the three color image signals, determines each coefficient from the plurality of color differences based on a function, and determines a first coefficient by one of a linear sum, maximum value, minimum value, or median value of the each coefficient. [Effects of the Invention]

[0014] According to the present invention, it is possible to provide an image signal processing method that reduces noise by removing high frequency components that cause color noise. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a block diagram showing a configuration of a solid-state imaging device according to an embodiment of the present invention. [Figure 2] Flowchart for noise reduction according to an embodiment of the present invention DETAILED DESCRIPTION OF THE INVENTION

[0016] The best mode for carrying out the present invention will be described below with reference to the accompanying drawings, but the present invention is not limited to this embodiment.

[0017] In this embodiment, 100 denotes a lens barrel, 101 denotes a solid-state imaging element, 1011 denotes a microlens, 1012 denotes a photoelectric conversion unit, 1013 denotes a wiring unit, 102 denotes a CPU, 1021 denotes an image signal determination unit, 1022 denotes a linear sum calculation unit, 1023 denotes a first subtraction unit, 1024 denotes a low-frequency extraction unit, 1025 denotes a second subtraction unit, 1026 denotes a color difference function unit, 1027 denotes a multiplication unit, and 1028 denotes a threshold comparison and selection unit.

[0018] FIG. 1 is a diagram showing the configuration of an imaging device according to this embodiment.

[0019] The imaging device 10 is made up of a lens barrel 100 and a camera body (not shown).

[0020] Lens barrel 100 is detachable from the camera body, and the mount of lens barrel 100 and the mount of the camera body are connected by a bayonet mechanism, etc. Lens barrel 100 may also be fixed to the camera body.

[0021] The solid-state imaging device 101 includes a microlens 1011, a photoelectric conversion unit 1012, a wiring unit 1013, and the like.

[0022] The microlens 1011 is a lens that condenses the light beam incident on the lens barrel 100 so that each photoelectric conversion unit 1012 is suitable for receiving the light.

[0023] The photoelectric conversion unit 1012 converts the light beam that has passed through the microlens 1011 into an image signal.

[0024] The image signal converted by the photoelectric conversion unit 1012 will be described. The photoelectric conversion unit 1012 that converts light beams into image signals is arranged in each pixel. When the solid-state imaging device 101 is a Bayer type, the photoelectric conversion unit 1012 arranged in each pixel converts the image signal into an image signal of the same color as the color filter arranged in the microlens 1011 and the photoelectric conversion unit 1012. Generally, color filters of three colors, red (R), green (G), and blue (B), are used, and in this case, three types of image signals are converted.

[0025] On the other hand, when the solid-state imaging device 101 is a stacked type, image signals corresponding to the number of stacked photoelectric conversion units are converted from one pixel. In this embodiment, the photoelectric conversion units 1012 are stacked in three layers to obtain image signals of three colors, red (R), green (G), and blue (B), from one pixel.

[0026] The wiring unit 1013 transfers the image signals converted by the photoelectric conversion units 1012 from each photoelectric conversion unit 1012 to the CPU 102 .

[0027] The CPU 102 performs various processes on the image signals converted by the photoelectric conversion unit 1012 and transferred via the wiring unit 1013 .

[0028] The image signal determination unit 1021 determines an image signal to be corrected from the image signals converted by the photoelectric conversion unit 1012 and transferred via the wiring unit 1031 .

[0029] The linear sum calculation unit 1022 calculates a linear sum from image signals other than the image signal to be corrected determined by the image signal determination unit 1021. A specific calculation method will be described later.

[0030] The first subtraction unit 1023 subtracts the linear sum of the image signals other than the correction target calculated by the linear sum calculation unit 1022 from the correction target image signal determined by the image signal determination unit 1021.

[0031] The low-frequency extraction unit 1024 extracts low-frequency components from the result of calculation by the first subtraction unit 1023 .

[0032] The second subtraction unit 1025 subtracts the image signal extracted by the low-frequency extraction means 1024 from the calculation result of the first subtraction unit 1023 .

[0033] The color difference function 1026 calculates the color difference between the image signals converted by the photoelectric conversion unit 1012 of the solid-state image sensor 101, and returns a coefficient based on the calculated color difference function.

[0034] The multiplication unit 1027 multiplies the result calculated by the second subtraction means 1025 by the coefficient returned by the color difference function.

[0035] The threshold comparison and selection unit 1028 compares the result calculated by the multiplication unit 1027 with an arbitrary parameter and selects an appropriate value.

[0036] The noise removal unit 1029 subtracts the appropriate value selected by the threshold comparison and selection unit 1028 from the image signal to be corrected determined by the image signal determination unit 1021. The calculation result is set as a corrected image signal.

[0037] Next, an embodiment of the image signal processing method according to the present invention will be described with reference to the flowchart of FIG.

[0038] 1, a light beam passing through lens barrel 100 reaches solid-state imaging element 101, passes through microlenses 1011, and is guided to photoelectric conversion section 1012, where it is converted into an image signal. In the solid-state imaging element 101 used in the present embodiment, three layers of photoelectric conversion section 1012 are stacked per pixel. Therefore, three types of image signals are converted from one pixel.

[0039] In step #1, the image signal determination unit 1021 determines an image signal to be corrected from among the three types of image signals converted by the photoelectric conversion unit 1012 of the solid-state image sensor 101.

[0040] In step #2, the linear sum calculation unit 1022 calculates a linear sum using two types of image signals other than the image signal to be corrected that was determined by the image signal determination unit 1021 in step #1.

[0041] In step #3, the first subtraction unit 1023 subtracts the linear sum calculated by the linear sum calculation unit 1022 in step #2 from the image signal to be corrected determined in step #1 by the image signal determination unit 1021. The result of this subtraction is set as the first subtraction result.

[0042] In step #4, the low-frequency extraction unit 1024 extracts low-frequency components by resizing the result calculated by the first subtraction unit 1023 in step #3. Note that, as long as the low-frequency components can be extracted, a low-pass filter may also be used instead of resizing.

[0043] In step #5, the second subtraction unit 1025 subtracts the low frequency components extracted in step #4 from the first subtraction result, which is the calculation result of step #3, thereby making it possible to extract high frequency components.

[0044] In step #6, the color difference function unit 1026 calculates the color differences between the three types of image signals converted by the photoelectric conversion unit 1012. Furthermore, a coefficient is returned based on the function from the calculated color differences.

[0045] In step #7, the multiplication unit 1027 multiplies the result calculated in step #5 by the coefficient determined in step #6.

[0046] In step #8, the threshold comparison and selection unit 1028 compares the result calculated in step #7 with the parameter.

[0047] In step #9, the noise removal unit 1029 subtracts the comparison result selected in step #8 from the image signal to be corrected determined in step #1.

[0048] The result of the calculation in step #9 is the corrected image signal obtained by removing noise from the image signal to be corrected in step #1.

[0049] The calculations performed in each of the above steps will be specifically described below.

[0050] In this embodiment, the correction method will be specifically described by taking green (G) from among the image signals converted by the photoelectric conversion unit 1012 as the image signal to be corrected.

[0051] Since the image signal to be corrected is green (G), in step #1 the image signal determination unit 1021 determines green (G) as the image signal to be corrected.

[0052] Next, in step #2, the linear sum calculation unit 1022 calculates a linear sum using red (R) and blue (B) image signals other than green (G) image signal to be corrected.

[0053] A specific calculation method performed by the linear sum calculation unit 1022 in this embodiment will be described.

[0054] If the red (R) and blue (B) image signals are R and B respectively and the coefficient is α, the linear sum can be expressed as follows: Linear sum: α*R+(1-α)B (0<α<1)

[0055] Next, in step #3, the calculation of the first subtractor 1023 will be described. If the green (G) image signal converted by the photoelectric converter 1012 is G and the green (G) after calculation is G1, the following equation is obtained. G1=G-(α*R+(1-α)B) (0<α<1)

[0056] In step #4, low-frequency components are extracted from G1 calculated in step #3. Specifically, a reduction process is applied to G1, which is the calculated green (G), and then the image signal G2 is generated by enlarging it to its original size (i.e., resizing). Alternatively, the image signal G2 may be generated by applying a low-pass filter.

[0057] In step #5, the low-frequency image signal G2 extracted in step #4 is subtracted from G1 calculated in step #4. By subtracting the low-frequency image signal G2 from G1, it is possible to extract only the high-frequency components. The image signal from which only the high-frequency components have been extracted is designated as image signal G3. As a result, magenta noise is reduced.

[0058] In step #6, the color differences of (RG), (GB), and (BR) are found. Next, for the color difference (RG), the coefficient f1 is found using the Gaussian function exp(-a*(RG)^2). Similarly, coefficients f2 and f3 are found from the Gaussian function for the other color differences (GB) and (BR). The coefficient f is determined using the coefficients f1 to f3 found for each color difference.

[0059] There are various methods for determining the coefficient f from the coefficients f1 to f3. For example, it may be a linear sum of f1 to f3, or it may be the maximum, minimum, or median value. Furthermore, these methods may be used depending on the state of noise contained in the image.

[0060] Furthermore, in step #6, the image signal can be corrected with high accuracy by using all three types of color differences. Alternatively, coefficients f1 and f2 can be calculated using a Gaussian function from the color differences (RG) and (GB), which correspond to CbCr in YCbCr. These can also be used to determine the coefficient f, as described above. In this case, the amount of calculation can be reduced.

[0061] In step #7, the image signal G3 extracted in step #5 is multiplied by the coefficient f calculated in step #6, and the multiplication result is set as G4.

[0062] In step #8, the multiplication result G4 is compared with the parameters. Specifically, the multiplication result G4 is clipped so that it falls within the range of -ThMin to +ThMax. If the clipped value is G5, then G5 = Min(+ThMax, Max(-ThMin, G4)).

[0063] In step #9, the clipped G5 obtained in step #8 is used to set G = G - G5, thereby making it possible to remove color noise components from the image signal that was corrected in step #1.

[0064] Furthermore, in step #6 of this embodiment, the coefficients are calculated by applying a Gaussian function to each color difference. However, it is possible to apply any function other than a Gaussian function, as long as it has the characteristic of attenuating as the color difference, which is the image signal difference, increases. For example, exp(-a*|RG|^b) or a / (|RG|^b+1) can be used. (However, |X| is the absolute value of X, and b>0.)

[0065] Among the high-frequency color noise, green / magenta noise is particularly noticeable at high ISO settings. A major cause of this green / magenta noise is the independent behavior of the high frequencies of green (G) compared to the other colors red (R) and blue (B). Therefore, assuming the independent G component as G1 = G-(α*R+(1-α)B) (0<α<1), and extracting only the high-frequency components, it is possible to reduce the green / magenta noise. Furthermore, because this alone cannot avoid the destruction of the color patterns in the original image (color bleeding, loss of color detail), we performed intensity adjustment based on color differences in step #6 and threshold setting in step #7 before noise reduction.

[0066] Furthermore, in the above embodiments, the solid-state imaging device 101 is assumed to be a stacked type, but a Bayer type solid-state imaging device having one type of image signal per pixel may also be used.

[0067] In the case of a stacked solid-state imaging device, since signals can be acquired from the same position, it is easy to prevent the loss of high-frequency information and to obtain a color noise suppression effect. Also, in the case of a sensor with mixed colors, the high-frequency information of the target signal is also contained in other color signals, so it is easy to prevent the loss of high-frequency information and to obtain a color noise suppression effect.

[0068] Furthermore, a Bayer-type solid-state imaging device can only obtain one type of image signal from one pixel. Therefore, the other two types of image signals can be obtained by interpolating from surrounding pixels. When a Bayer-type solid-state imaging device is used in the present invention, the image signal for one pixel is converted into three types by interpolation processing, and then the image signal to be corrected in step #1 is determined, allowing the subsequent processing to be the same as when a stacked-type solid-state imaging device is used. [Explanation of symbols]

[0069] 10. Imaging device 100 Lens barrel 101 Solid-state imaging element 1011 Micro Lens 1012 Photoelectric conversion unit 1013 Wiring layer 102 CPU 1021 Image signal determination unit 1022 Linear sum calculation section 1023 First subtraction section 1024 Low Frequency Extraction Unit 1025 Second subtraction section 1026 Color Difference Function 1027 Multiplication Unit 1028 Threshold comparison and selection unit 1029 Noise Reduction Section

Claims

1. 1. An image signal processing method for processing three-color image signals, an image signal determination unit that determines an image signal of one color to be corrected; a linear sum calculation unit that calculates a linear sum of image signals of two colors other than the correction target; a first subtraction unit that subtracts the linear sum calculated by the linear sum calculation unit from the image signal to be corrected; a low-frequency extraction unit that extracts low-frequency components from the calculation result of the first subtraction unit; a second subtraction unit that subtracts the image signal that has passed through the low-frequency extraction unit from the calculation result of the first subtraction unit; a color difference function unit that calculates color differences from the three-color image signals and determines a first coefficient based on a function from the color differences; a multiplication unit that multiplies the calculation result of the second subtraction unit by the first coefficient determined by the color difference function unit; a threshold comparison and selection unit that compares the calculation result of the multiplication unit with an arbitrary parameter and selects either the calculation result or the parameter; a noise removal unit that subtracts a selection result from the threshold comparison and selection unit from the image signal to be corrected; The image signal processing method is characterized in that the calculation result of the noise removal unit is used as a corrected image signal for the image signal to be corrected.

2. 2. The image signal processing method according to claim 1, wherein the color difference function unit calculates a plurality of color differences using image signals of two colors from the three-color image signals, determines each coefficient based on a function from the plurality of color differences, and determines a first coefficient by one of a linear sum, a maximum value, a minimum value, and a median value of the each coefficient.

Images