Image signal processor

Abstract

To provide an image signal processor which reduces the occurrence of tone jump and discoloration.SOLUTION: The image signal processor according to one embodiment of an invention includes a bit extension processing unit 100 which generates extension data by extending, by a first bit, the number of bits of input data including a Bayer image and a white image, and an output unit 200 which receives the extension data, adjusts the first number of bits to a second number of bits, and generates output data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image signal processor for processing an image signal. [Background technology]

[0002] In recent years, as the functions of electronic devices have become more diverse, there has been an increasing demand for improved image capture functions using electronic devices, which has led to a demand for techniques for improving the quality of images captured through electronic devices.

[0003] Image sensors used in imaging devices have a structure in which color filters that transmit specific wavelength components of light (R, G, B) for each pixel are attached to the sensor surface. There are various types of color filter color arrangements, and the Bayer arrangement, which consists of three types of filters that transmit only specific wavelengths of red (R), green (G), and blue (B), is often used.

[0004] In recent years, as image sensor pixels become finer, the amount of light incident on each pixel decreases and the signal to noise ratio (SNR) increases, which can cause the performance of the image sensor to deteriorate rapidly. In order to increase the transmittance of the filter and achieve high sensitivity in such low-light environments, image sensors that include white filters in addition to filters that transmit only specific wavelengths of light such as RGB (Red, Green, Blue) are becoming more widely used. Summary of the Invention [Problem to be solved by the invention]

[0005] An embodiment of the present invention provides an image signal processor that reduces the occurrence of tone jumps and discoloration by making rounding errors smaller than random noise in a process for generating a noise-reduced Bayer image from an image containing white pixels. [Means for solving the problem]

[0006] An image signal processor according to an embodiment of the present invention may include a bit expansion processing unit that expands the number of bits of input data including a Bayer image and a white image by a first bit to generate expanded data; and an output unit that receives the expanded data, adjusts the first number of bits to a second number of bits, and generates output data.

[0007] An image signal processor according to another embodiment of the present invention may include a bit extension processing unit that extends a number of bits of input data by a first number of bits to generate extended data; a storage unit that sets a register value for calculating a noise amount based on a gain of the input data; an output bit number determination unit that adjusts the first number of bits to a second number of bits based on the register value; and an output unit that generates output data based on the extended data and the second number of bits.

[0008] According to yet another embodiment of the present invention, an image signal processor may include a bit extension processing unit that extends the number of bits of input data to generate extended data; a storage unit that sets a register value for calculating a noise amount based on a gain of the input data; a shadow noise measuring unit that measures shadow noise of the input data; an output bit number determination unit that determines a number of output bits based on the register value and the shadow noise; and an output unit that receives the extended data and the output bit number and generates output data having a different number of bits from the extended data. Effect of the Invention

[0009] The embodiment of the present invention can provide an effect of improving the quality of an image signal by reducing the occurrence of tone jumps and discoloration during re-mosaic processing of an RGBW pattern.

[0010] Furthermore, the embodiments of the present invention are for illustrative purposes only, and those skilled in the art may make various modifications, changes, substitutions, and additions within the technical spirit and scope of the appended claims, and such modifications and variations should be deemed to fall within the scope of the following claims. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram illustrating an imaging device according to one embodiment of the present invention. [Diagram 2] 2 is a diagram for explaining the operation of the bit extension processing unit in FIG. 1; [Diagram 3] 2 is a diagram for explaining a bit extension operation of the bit extension processing unit in FIG. 1; [Figure 4] FIG. 2 is a block diagram illustrating an imaging device according to another embodiment of the present invention. [Diagram 5] FIG. 13 is a block diagram illustrating an imaging device according to yet another embodiment of the present invention. [Figure 6] 10 is a flowchart illustrating a method for processing an image signal according to yet another embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Various embodiments will be described below with reference to the accompanying drawings. However, it should be understood that the present disclosure is not limited to specific embodiments, but includes various modifications, equivalents, and / or alternatives of the embodiments. The embodiments of the present disclosure can provide various effects that can be recognized directly or indirectly through the present disclosure.

[0013] FIG. 1 is a block diagram illustrating an imaging device 1 according to one embodiment of the present invention.

[0014] Referring to FIG. 1, an imaging device 1 may include an image sensing device 10 and an image signal processor (ISP).

[0015] Here, the image sensing device 10 may be a complementary metal oxide semiconductor image sensor (CIS) that converts an optical signal into an electrical signal. The image sensing device 10 may have its overall operations, such as on / off, operation mode, operation timing, and sensitivity, controlled by a timing controller (not shown).

[0016] The image sensing device 10 may include a pixel array 20 and a readout circuit 300 .

[0017] The pixel array 20 may include a plurality of pixels arranged in a two-dimensional matrix structure (e.g., arranged in a row direction and a column direction). Each of the pixels in the pixel array 20 can generate a pixel signal by converting an optical signal into an electrical signal and output the pixel signal to the readout circuit 30.

[0018] The readout circuit 30 may process the analog pixel signals output from each of the pixels to generate digital data corresponding to the pixel signals. For example, the readout circuit 30 may include an analog-to-digital converter for analog-to-digital conversion of the pixel signals.

[0019] The image signal processor (ISP) can perform video signal processing on the image data input from the readout circuit 30. The image signal processor (ISP) can perform video signal processing for reducing noise on the corrected image data and improving image quality, such as gamma correction, color filter array interpolation, color matrix, color correction, and color enhancement. The image signal processor (ISP) can compress the image data generated by performing the video signal processing for improving image quality to generate an image file, or can restore the image data from the image file. The video compression format can be a lossless format or a lossy format. As an example of the compression format, in the case of a still image, the JPEG (Joint Photographic Experts Group) format or the JPEG 2000 format can be used. In the case of a moving image, a moving image file can be generated by compressing a plurality of frames according to the MPEG (Moving Picture Experts Group) standard. The video file can be generated according to, for example, the Exif (Exchangeable image file format) standard.

[0020] The image data applied from the image sensing device 10 to the image signal processor (ISP) may be an RGBW pattern in which a Bayer image and a white image are mixed. The image signal processor (ISP) may perform signal processing to generate a noise-reduced Bayer image using a white image in an RGBW (Red, Green, Blue, White) pattern including the Bayer image and the white image.

[0021] Such an image signal processor (ISP) may include a bit expansion processing unit 100, an output unit 200, and a storage unit 300.

[0022] Here, the bit extension processing unit 100 can extend the number of bits of the input data to the decimal part side and output the extended data. That is, the bit extension processing unit 100 can obtain the value of the extended data up to the decimal point range and increase the number of bits of the extended data to be greater than the number of bits of the input data. For example, assuming that the input data is N bits (10 bits), the bit extension processing unit 100 can extend the input data up to the decimal point and output it as extended data having N+A (10+A) bits. (Here, N may be an integer greater than 0.) Here, A may be a data range corresponding to a decimal point smaller than the least significant bit (LSB) of the input data.

[0023] On the other hand, when shooting in a relatively dark low-light environment with high gain, random noise may increase and the standard deviation of the noise may become larger after re-mosaic processing.In addition, when the number of output bits is expanded, the amount of information transmitted by the image signal processor (ISP) may increase.

[0024] Therefore, the output unit 200 may output the number of output bits as is or may output the number of output bits after restricting it. That is, the output unit 200 may receive the extension data, output the output data, and generate the output data based on the number of output bits read from the storage unit 300. Here, the number of output bits may indicate the number of decimal points that must be included in the output data that is finally output. The number of output bits may be set to an arbitrary number that does not cause pseudocolor according to the amount of noise after the re-mosaic process and does not increase the amount of information to be transmitted too much.

[0025] For example, the output unit 200 can output data having N+B (10+B) bits, with the A value of the extended data having N+A (10+A) bits adjusted to the B value based on the number of output bits set in the storage unit 300. (Here, B may be the same as A or a smaller value than A.) As a result, in a low light environment, the number of output bits is limited rather than extended, thereby preventing an increase in the amount of data due to the transmission of meaningless bits caused by increased random noise.

[0026] The storage unit 300 may preset and store the number of output bits corresponding to a gain value of pixel data in order to adjust the number of output bits of the output data. The image signal processor (ISP) may perform a digital calculation operation to calculate a gain on pixel data during a re-mosaic operation. For example, the image signal processor (ISP) may perform calculations by adding a gain to pixel data during image signal processing such as noise reduction, white balance adjustment, demosaic processing, hue correction, gamma correction, and hue conversion.

[0027] The amount of noise after the re-mosaic process varies depending on the amount of noise in the input data, so the number of output bits may vary depending on the amount of noise in the input data, which may depend mainly on the gain (sensitivity) setting of the image sensing device 10.

[0028] Therefore, the image signal processor (ISP) can determine the relationship between the gain value and the number of output bits in advance, and specify the number of output bits based on the relationship and store it in the storage unit 300. Here, the relationship between the gain value and the number of bits can be obtained by previously processing pixel data and then converting the processing result into data.

[0029] As an example, the storage unit 300 may include a register. The image signal processor (ISP) may check in advance the relationship between the gain of pixel data and the number of output bits, and implement the checked relationship information in software. Then, the image signal processor (ISP) (or a central processing unit (CPU) inside or outside the ISP) may store the number of output bits set when the software is executed in the register.

[0030] FIG. 2 is a diagram for explaining the operation of the bit extension processing unit 100 in FIG.

[0031] 2, the input data applied to the bit expansion processor 100 may be an RGBW pattern in which a Bayer image and a white image are mixed. The RGBW pattern may include color pixels of red, blue, and green pixels, and a white pixel.

[0032] The color pixels (R, G, B) may be pixels to which light of a specific wavelength is incident through a corresponding color filter. And the white pixel (W) is a pixel without a corresponding color filter and may be referred to as a texture pixel, a black and white pixel, or a mono pixel. For example, the color pixels (R, G, B) can obtain a color pixel value by incident light of a specific hue (e.g., red, green, blue, etc.) by a color filter that transmits only incident light of a specific wavelength. In contrast, the white pixel (W) can obtain a white pixel value by incident light (e.g., visible light) that is not limited to a specific hue.

[0033] In the embodiment of FIG. 2, an RGBW pattern having a size of 64 pixels arranged in an 8×8 matrix has been described as an example, but in embodiments of the present invention, the size of the RGBW pattern and the arrangement of each pixel are not limited to this.

[0034] As the pixels of the image sensing device 10 become finer, the amount of incident light to each pixel decreases, and the signal to noise ratio (SNR) decreases, which may rapidly deteriorate the performance of the image sensing device 10. In order to increase the transmittance of the filter and achieve high sensitivity in such a low-illumination environment, the image sensing device 10 including a white filter as well as a filter that transmits only specific wavelengths of light such as RGB (Red, Green, Blue) is becoming more widely used.

[0035] A white image can have a relatively strong signal to noise ratio (SNR) characteristic compared to an RGB image. Since a white image is designed to receive light of all bands compared to an RGB image that receives light of a specific band, it has a characteristic of having relatively less signal noise even when capturing an image in a low light environment.

[0036] The bit expansion processor 100 can separate a Bayer image and a white image by binning the RGBW pattern. Here, "binning" may mean summing pixel values ​​of unit pixels having the same hue in sub-pixel units. For example, the bit expansion processor 100 can generate a Bayer image and a white image by summing pixel values ​​of unit pixels having the same hue in the RGBW pattern in units of four (2×2) (sub-pixel units) (4-summation).

[0037] The Image Signal Processor (ISP) can output an image with a good signal-to-noise ratio by performing re-mosaic processing using a white image, especially when a high gain is applied in a low-light environment.

[0038] The RGBW pattern is a single-chip image sensor with R, G, B, and W filters arranged in a mosaic shape. Therefore, when generating a color image, a re-mosaic process is required as color coding to generate RGB pixel values ​​corresponding to each pixel.

[0039] Here, the "re-mosaic processing" may refer to an operation of outputting a high quality image signal using pixel signals. In another embodiment, the "re-mosaic processing" may refer to an image process of converting an original image into an image having a Bayer pattern.

[0040] The bit extension processor 100 may generate a converted image by performing a re-mosaic process on an original image using a re-mosaic method. According to an embodiment, the re-mosaic method may be a method of performing a re-mosaic process based on a preset reference formula. The reference formula may be a formula for calculating pixel values ​​of each pixel included in the re-mosaic processed image. For example, the reference formula may be a formula for calculating pixel values ​​of the re-mosaic processed image by applying weighted values ​​to pixel values ​​of adjacent pixels in the original image.

[0041] According to an embodiment, the bit extension processor 100 may include a re-mosaic processing module (not shown) for performing the re-mosaic process. Here, the re-mosaic processing module may be implemented as a software block executed by a predetermined processor, or as a combination of a dedicated hardware block and a processing unit.

[0042] Typically, an image signal processor can convert an RGBW pattern into a Bayer pattern by performing various signal processes on the RGBW pattern during re-mosaic processing. For example, a digital calculation operation for applying gain to pixel data can be performed during re-mosaic processing. In this process, quantization error can accumulate, and the accumulated quantization error can cause color difference between images. Therefore, in order to reduce noise and perform fast calculations during re-mosaic processing of an RGBW pattern, a rounding process can be performed without expressing the decimal part with bits of pixel data values, or the decimal part can be limited to a preset limited number of digits.

[0043] However, when shooting in a relatively bright, high-illumination environment with low gain, the standard deviation of random noise may become smaller than a certain value, and the effect of rounding error may become greater than the effect of random noise. For example, if the amount of random noise becomes smaller than the least significant bit "1" after re-mosaic processing, the effect of rounding error may become greater and false color may occur. As a result, when rounding processing is performed during re-mosaic processing, tone jumps and discoloration may occur in the output image due to the effect of rounding error.

[0044] Therefore, the bit extension processor 100 according to an embodiment of the present invention can obtain pixel values ​​of a Bayer image up to the decimal point range during re-mosaic processing, and can extend the number of bits of the input data to output extended data with the extended number of bits. Thus, the number of bits of input data input to the bit extension processor 100 may be N bits, but the number of bits of extended data finally output by the bit extension processor 100 after re-mosaic processing may be (N+A) bits. Thus, the embodiment of the present invention can prevent tone jumps and discoloration from occurring in the output image by making the rounding error smaller than random noise.

[0045] FIG. 3 is a diagram for explaining the bit extension operation of the bit extension processing unit in FIG.

[0046] A specific method for extending bits by the bit extension processor 100 will be described with reference to FIG.

[0047] 3 shows an example of pixels used during bit extension processing in the bit extension processing unit 100. For example, a Bayer pattern having a size of 81 pixels arranged in a 9×9 matrix and a White pattern having a size of 81 pixels arranged in a 9×9 matrix will be described as examples.

[0048] In the embodiment of Figure 3, the Bayer pattern and the White pattern each have a size of 81 pixels arranged in a 9 x 9 matrix, but in embodiments of the present invention, the sizes of the Bayer pattern and the White pattern and the arrangement of each pixel are not limited to this.

[0049] In the Bayer pattern, bit expansion processing may be performed using pixels of the same hue as the target pixel (TP) among 9×9 pixels centered on the target pixel (TP) to be processed. In the Bayer pattern, the target pixel (TP) may be assumed to be a green pixel. There may be 41 green pixels in the Bayer pattern, including the target pixel (TP). During re-mosaic processing, 41 white pixels at the same position as the green pixel may also be used to reduce noise.

[0050] First, the bit extension processing unit 100 can calculate a statistic r for a pixel for performing the bit extension process as shown in the following [Formula 1]. In an embodiment of the present invention, the statistic r can be calculated as 1 bit of an integer part and 13 bits of a decimal part.

number

[0051] Here, the statistic r can be calculated by taking the pixel value of the selected green pixel as Gi and the pixel value of the selected white pixel as Wi.

[0052] Then, the similarity index Si between the target pixel (TP) to be processed and the selected pixel can be calculated as shown in the following [Mathematical Formula 2].

number

[0053] Here, the similarity index Si can be calculated by assuming that the pixel value of the green pixel to be processed is Gc and the pixel value of the white pixel to be processed is Wc. The more similar the pixel value Gc of the green pixel and the pixel value Wc of the white pixel are, the larger the similarity index Si can be.

[0054] In the above [Equation 2], the estimated value V(Gc, Wc) of the noise variance of the pixel to be processed can be obtained as shown in the following [Equation 3].

number

[0055] Here, k is a parameter related to the amount of shot noise, and the higher the gain (sensitivity) during shooting, the larger the estimated noise variance value V(Gc,Wc) may be. And, Vo indicates the noise variance value of the image sensing device 10 when light is blocked.

[0056] Next, the bit extension processing unit 100 can calculate the weighted average value Z of the white pixels according to the following [Equation 4].

number

[0057] Here, the weighted average value Z of the white pixels may be calculated based on the pixel values ​​Wi of a total of five selected white pixels including the target pixel (TP) to be processed and four pixels located above, below, left and right of the target pixel (TP). Depending on the embodiment, the weighted average value Z may be set to an integer part of 10 bits and a decimal part of 2 bits.

[0058] In the above [Equation 4], the similarity index Si is used to calculate the weighted average value Z, but in the embodiment of the present invention, the method of calculating the weighted average value Z is not limited to this. In other embodiments, an edge may be determined, and the weighted average value may be calculated by averaging pixel values ​​along the edge direction.

[0059] In addition, the pixel value Gout after the re-mosaic process can be calculated as shown in the following [Mathematical Equation 5].

number

[0060] The pixel value Gout can be calculated based on the statistic r calculated by the above [Mathematical Formula 1] and the weighted average value Z calculated by the above [Mathematical Formula 4]. Here, the statistic r has an integer part of 1 bit and a decimal part of 13 bits, and the weighted average value Z may be set to an integer part of 10 bits and a decimal part of 2 bits. In this case, the rZ value has an integer part of 11 bits and a decimal part of 15 bits. For example, when the illuminance is high and the random noise after re-mosaic is about 1 / 8 under the lowest sensitivity condition, about 4 bits are required in the decimal part to sufficiently reduce the rounding error and prevent pseudo-color, and no more is required. Therefore, the decimal part can be made to leave the upper 4 bits of the total 15 bits and discard the remaining 11 bits. When the most significant bit (MSB) of the integer part is "1", that is, when the integer part exceeds 1023, the integer part may be clipped to 1023. As a result, the integer part has 10 bits and the decimal part has 4 bits, totaling 14 bits, and the pixel value Gout can be a 14-bit integer.

[0061] FIG. 4 is a block diagram illustrating an imaging device according to another embodiment of the present invention.

[0062] 4, the imaging device 1_1 may include an image sensing device 10 and an image signal processor (ISP). Here, the image signal processor (ISP) may include a bit extension processing unit 100, an output unit 200, an output bit number determination unit 250, and a storage unit 300_1.

[0063] The embodiment of Fig. 4 is different from the embodiment of Fig. 1 in that an output bit number determining unit 250 is further included between the storage unit 300_1 and the output unit 200, and all other components are the same. Therefore, in Fig. 4, the same components as those in Fig. 1 are denoted by the same reference numerals, and duplicated explanations will be omitted.

[0064] The storage unit 300_1 may include a register. The image signal processor (ISP) may store a register value for calculating a noise amount by a gain value of pixel data in the storage unit 300_1. Here, the register value may include a parameter k related to the shot noise amount described in [Equation 3] above, and a noise variance value V0 when light is blocked described in [Equation 6] below.

[0065] The output bit number determining unit 250 may adjust the bit value of the output data output from the output unit 200 based on the register value stored in the storage unit 300_1. In the embodiment of FIG. 4, the output bit number determining unit 250 may determine whether to expand the bit value of the output data output from the output unit 200. For example, the output bit number determining unit 250 may calculate the number of output bits by calculating a noise standard deviation based on the register value stored in the storage unit 300_1. According to an embodiment, the output bit number determining unit 250 may calculate the number of output bits to be 1LSB to 2LSB.

[0066] The output unit 200 adjusts the A value of the extended data having N+A (10+A) bits to the B value based on the number of output bits set by the output bit number determination unit 250, and outputs data having N+B (10+B) bits (here, B may be the same as A or a value smaller than A).

[0067] FIG. 5 is a block diagram illustrating an imaging device according to yet another embodiment of the present invention.

[0068] 5, the imaging device 1_2 may include an image sensing device 10 and an image signal processor (ISP). Here, the image signal processor (ISP) may include a bit extension processing unit 100, an output unit 200, an output bit number determination unit 250_1, a storage unit 300_1, and a shadow noise measurement unit 400.

[0069] The embodiment of Fig. 5 is different from the embodiment of Fig. 4 in that it further includes a dark noise measurement unit 400, and the output bit number determination unit 250_1 receives the output of the dark noise measurement unit 400, but all other components are the same. Therefore, in Fig. 5, the same components as those in Fig. 4 are denoted by the same reference numerals, and duplicated explanations will be omitted.

[0070] The storage unit 300_1 may include a register. The image signal processor (ISP) may store a register value for calculating the amount of noise using a gain value of pixel data in the storage unit 300_1. Here, the register value may include a parameter k related to the amount of shot noise described in [Equation 3] above.

[0071] Noise may occur in dark areas depending on the exposure time and temperature of the image sensing device 10. That is, the pixel array 20 of the image sensing device 10 may include optical black pixels to compensate for dark signals of active pixels.

[0072] The dark noise measurement unit 400 can measure the dark noise of the optical black pixel in the input data and output it to the output bit number determination unit 250_1. As an example, the noise variance value V0 of the dark noise can be calculated as shown in the following [Equation 6]. For example, the dark noise calculated by the dark noise measurement unit 400 may indicate the noise variance value V0 when light is blocked, which is explained in [Equation 7] described later.

number

[0073] In the above [Equation 6], Ii may represent a pixel value, and N may represent the number of pixels used in the variance calculation, where I may be an index indicating a pixel.

[0074] The output bit number determining unit 250_1 may adjust the bit value of the output data output from the output unit 200 based on the register value stored in the storage unit 300_1 and the shadow noise measured by the shadow noise measuring unit 400. That is, the output bit number determining unit 250_1 may adjust the bit value of the output data in consideration of not only the register value but also the shadow noise.

[0075] In the embodiment of Fig. 5, the number of output bits determining unit 250_1 may determine whether to expand the bit value of the output data output from the output unit 200. For example, the number of output bits determining unit 250_1 may calculate the number of output bits by calculating a noise standard deviation based on a register value and a shadow noise. Depending on the embodiment, the number of output bits determining unit 250_1 may calculate the number of output bits to be 1LSB to 2LSB.

[0076] The output unit 200 adjusts the A value of the extended data having N+A (10+A) bits to the B value based on the number of output bits set by the output bit number determination unit 250_1, and outputs data having N+B (10+B) bits (here, B may be equal to A or may be smaller than A).

[0077] FIG. 6 is a flowchart illustrating a method for processing an image signal according to yet another embodiment of the present invention.

[0078] A method for determining the number of output bits of output data in an embodiment of the present invention will be described in detail below with reference to Fig. 6. In the following description, an example will be described in which the number of output bits is adjusted by the output bit number determination unit 250.

[0079] First, the noise variance V(I, g) of the image sensing device 10 can be modeled as follows [Equation 7].

number

[0080] In the above [Equation 7], g is the gain during shooting, the parameter k is a parameter proportional to the gain g, V0(g) may represent the noise variance value when light is blocked, and I0 may represent the pixel value when light is blocked, and I may represent the pixel value.

[0081] In this manner, the image signal processor (ISP) can preset register values ​​(i.e., values ​​such as the gain G, the parameter k, and V0) for calculating the amount of noise and store them in the storage unit 300_1 (step S1).

[0082] Assuming that the standard deviation of the noise is multiplied by 1 / R by the re-mosaic process, the noise variance value VA(I, g) after the re-mosaic process can be calculated as follows (Equation 8): (Step S2)

number

[0083] As shown in [Formula 8], if the noise variance value VA(I, g) after the re-mosaic process is about 1 LSB, tone jumps and discoloration may occur in dark areas. This allows the output bit number determination unit 250 to calculate the output bit number to be between 1 LSB and 2 LSB.

[0084] The output bit number determining unit 250 according to an embodiment of the present invention may calculate the number of output bits based on the gain G value, the parameter k value, and the dark noise value when light is blocked (step S3).

number

[0085] The parameter k is proportional to the gain g, and V0(g) is, for example, V00+g 2 Then, as shown in [Equation 9] above, the noise variance value VA(I, g) after the re-mosaic process can be estimated from the gain g and pixel value I.

[0086] Here, the pixel value I can be set to "0" which provides the minimum variance. Depending on the embodiment, the pixel value I may be a pixel value that has a level of brightness at which a problem is visible when a gamma curve is applied in the development process. Therefore, the need for bit extension may change depending on the shape of the gamma curve or tone curve.

[0087] As described above, the output data output from the output unit 200 may have (N+B) bits, where B may be equal to or smaller than A (step S4). Therefore, when the number of bits needs to be expanded based on the noise variance value VA(I, g), the output bit number determination unit 250 may set B to A and output the data. On the other hand, when the number of bits needs to be restricted based on the noise variance value VA(I, g), the output bit number determination unit 250 may set B to a value smaller than A and output the data.

[0088] For example, it may be determined that the number of bits needs to be expanded when the noise variance value VA(I, g) is 0.25, which is smaller than 1. In other words, the output bit number determination section 250 may determine that the bit expansion is necessary when the gain g value is small.

number

[0089] The number of bits B to be extended may be determined as shown in [Formula 10]. In the embodiment of the present invention, the noise variance value VA(I, g) based on the gain g is determined to determine the number of output bits B to be extended. However, the method of determining the number of output bits B to be extended is not limited to this and can be changed.

[0090] As described above, in the embodiment of the present invention, the optimal number of output bits may be set to a number of bits that makes the standard deviation of noise after re-mosaic processing in dark areas 1LSB to 2LSB. In the embodiment of the present invention, an example is described in which the number of bits that makes the standard deviation of noise 1LSB to 2LSB is set as the number of output bits, but the present invention is not limited to this, and the number of output bits may be set to another value.

[0091] For example, let us assume that the input data is 10 bits and that the standard deviation of the noise is 4.0. If we then assume that the standard deviation of the noise becomes 1 / 8.0 during the process of re-mosaicing (rather than bit expansion), then the standard deviation of the noise after re-mosaicing may be 4.0 x (1.0 / 8.0) = 0.5 in 10-bit terms. In such a case, since color tones etc. will occur after re-mosaicing, the bits can be expanded.

[0092] For example, if 4 bits are expanded to output 14 bits, the noise standard deviation of the output image can be 0.5 x 2^4 = 8.0. That is, in a 14-bit image, the noise standard deviation is greater than 2 LSB, so the number of bits may be excessive. The noise standard deviation is about 1 LSB when outputting 11-bit data, and about 2 LSB when outputting 12-bit data, so it can be seen that the optimal number of bits for the output data is 11 to 12 bits.

[0093] In this way, an embodiment of the present invention may investigate (or test, detect) in advance the amount of noise after re-mosaic processing (in dark areas), and determine the number of output bits so that the amount of noise can be 1LSB to 2LSB.

[0094] Here, the amount of noise after the re-mosaic process can be predicted from the number of pixels referenced when calculating the statistic r during the re-mosaic process as shown in Equation 1. In another embodiment, software for calculating the re-mosaic process algorithm in floating point is created, and various images are processed using the software, and then the relationship between the noise standard deviation of the input image and the noise standard deviation of the output image for each gain setting is investigated and calculated.

Claims

1. a bit extension processing unit that extends a number of bits of input data including a Bayer image and a white image by a first number of bits to generate extended data; an output unit configured to receive the extended data, adjust the first number of bits to a second number of bits, and generate output data.

2. 2. The image signal processor of claim 1, wherein the first number of bits is a data range corresponding to a fraction less than the least significant bit of the input data.

3. 2. The image signal processor of claim 1, wherein the second number of bits is set to be equal to or smaller than the first number of bits.

4. 4. The image signal processor of claim 3, further comprising a storage unit for storing the second number of bits as a register value.

5. The second number of bits is 4. The image signal processor of claim 3, wherein the gain is set based on the gain of the input data.

6. The bit extension processing unit 2. The image signal processor of claim 1, further comprising: binning the input data to generate the Bayer image and the white image; and performing a re-mosaic process based on the white image to generate the denoised Bayer image.

7. The bit extension processing unit 2. The image signal processor of claim 1, further comprising: setting said first number of bits based on like-hue pixels at specific locations in said Bayer image and said white image, centered on a target pixel.

8. The bit extension processing unit 8. The image signal processor of claim 7, further comprising: obtaining statistics of the same-hue pixels according to a similarity index between the target pixel and the same-hue pixels; and calculating the first number of bits based on the statistics.

9. The bit extension processing unit 9. The image signal processor according to claim 8, further comprising: an estimate of a noise variance value obtained from a parameter relating to an amount of shot noise and a noise variance value when light is blocked; and said similarity index is calculated based on said estimate.

10. The bit extension processing unit 9. The image signal processor of claim 8, further comprising: a pixel value of the extended data after re-mosaic processing based on the statistic and a weighted average value of white pixels.

11. a bit extension processing unit that extends the number of bits of input data by a first bit to generate extended data; a storage unit that sets a register value for calculating a noise amount based on a gain of the input data; an output bit number determination unit that adjusts the first bit number to a second bit number based on the register value; an output section for generating output data based on the extended data and the second number of bits.

12. The register value is 12. The image signal processor according to claim 11, wherein the parameter is set based on a gain of the input data, a parameter related to an amount of shot noise, and a noise variance value when light is blocked.

13. The input data is 12. The image signal processor of claim 11, comprising a Bayer image and a White image.

14. 12. The image signal processor of claim 11, wherein the second number of bits is set to be equal to or smaller than the first number of bits.

15. The output bit number determination unit 12. The image signal processor of claim 11, further comprising: calculating a noise standard deviation based on the register value to determine the second number of bits.

16. The output bit number determination unit 16. The image signal processor of claim 15, further comprising: setting the second number of bits to a data range corresponding to a 1st least significant bit to a 2nd least significant bit of the input data based on the noise standard deviation.

17. a bit extension processing unit that extends the number of bits of input data to generate extended data; a storage unit that sets a register value for calculating a noise amount based on a gain of the input data; a dark noise measurement unit for measuring the dark noise of the input data; an output bit number determination unit that determines an output bit number based on the register value and the shadow noise; an output section receiving the extended data and the number of output bits and generating output data having a number of bits different from that of the extended data.

18. The output bit number determination unit 18. The image signal processor of claim 17, further comprising: a noise standard deviation calculated from the register value and the shadow noise to determine the number of output bits.

19. The input data is 20. The image signal processor of claim 17, comprising a Bayer image and a White image.

20. 18. The image signal processor of claim 17, wherein the number of bits of the extended data is a data range corresponding to a smaller fractional part than the least significant bit of the input data.

Images