Image signal processor and image signal processing method

Abstract

The invention relates to an image signal processor and an image signal processing method. An image signal processor capable of processing an image signal and an image signal processing method thereof are disclosed. The image signal processor includes: a remosaic processor configured to perform remosaic processing on an input image to generate a converted image; a noise amount estimator configured to estimate a noise amount of the converted image based on a preset noise amount parameter and the input image; a noise reduction degree determiner configured to determine a noise reduction degree and generate noise reduction degree information; and a noise suppression processor configured to generate an output image in which a degree of noise reduction is controlled based on the degree of noise reduction information.

Description

Technical Field

[0001] Technologies and embodiments disclosed in the present disclosure generally relate to an image signal processor capable of processing an image signal and an image signal processing method thereof. Background Art

[0002] Recently, as the number of functions of electronic devices has rapidly increased, the demand for improving image capture functions using these electronic devices has also increased. Therefore, technology is needed to improve the quality of images obtained by electronic devices.

[0003] The image sensor used in the imaging device has a configuration in which color filters that transmit red (R) light, green (G) light, and blue (B) light of specific wavelength bands for each pixel are attached to the surface of the sensor. Although there are various types of color arrays for color filters, a Bayer array that includes three types of filters each transmitting only light of specific wavelengths of red, green, and blue has been widely used.

[0004] Recently, as the size of pixels of image sensors is miniaturized, the amount of light incident on each pixel is reduced, thereby increasing the signal-to-noise ratio (SNR), and thus the performance of the image sensor may deteriorate rapidly. In order to increase the transmittance of the filter located inside the pixel and realize high-sensitivity pixels in low-light environments, the utilization of image sensors including white (W) filters and other filters that transmit only specific wavelengths of light such as red, green, and blue (RGB) is rapidly increasing. Summary of the invention

[0005] Various embodiments of the present disclosure are directed to an image signal processor capable of reducing a degree of tone jump and / or a degree of color change in a process of generating a Bayer image with less noise from an image including white pixels.

[0006] According to an embodiment of the present disclosure, an image signal processor may include: a re-mosaic processor, which is configured to perform re-mosaic processing on an input image to generate a converted image; a noise amount estimator, which is configured to estimate the noise amount of the converted image based on a preset noise amount parameter and the input image; a noise reduction degree determiner, which is configured to determine the noise reduction degree based on the noise amount estimated by the noise amount estimator and generate noise reduction degree information; and a noise suppression processor, which is configured to generate an output image with the noise reduction degree controlled based on the noise reduction degree information.

[0007] According to another embodiment of the present disclosure, an image signal processing method may include the following steps: performing re-mosaic processing on an input image including a Bayer image and a white image to generate a converted image; estimating a noise variance value of the converted image based on a preset noise amount parameter and a pixel value of the input image; determining a degree of noise reduction based on the noise variance value; and suppressing the degree of noise reduction of the converted image based on the degree of noise reduction.

[0008] It is to be understood that both the foregoing general description and the following detailed description of the present disclosure are exemplary and descriptive and are intended to provide further description of the disclosure as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features and advantageous aspects of the present disclosure will become readily apparent with reference to the following detailed description when considered in conjunction with the accompanying drawings.

[0010] Figure 1 is a block diagram illustrating an imaging device according to an embodiment of the present disclosure.

[0011] Figure 2 The present invention is shown in the embodiment of the present invention. Figure 1 A schematic diagram of the operation of the re-mosaic processor is shown.

[0012] Figure 3 is a flowchart illustrating an image signal processing method according to an embodiment of the present disclosure.

[0013] Figure 4 The present invention is a diagram showing an embodiment of the present invention. Figure 1 A block diagram of a computing device corresponding to the imaging device shown. DETAILED DESCRIPTION

[0014] The present disclosure provides embodiments and examples of image signal processors and image signal processing methods for processing image signals, which can be used in a configuration that substantially solves one or more technical or engineering problems and alleviates limitations or shortcomings encountered in some image signal processors in the art. Some embodiments of the present disclosure relate to an image signal processor capable of reducing the degree of hue jumps and / or discoloration in the process of generating a Bayer image with less noise from an image including white pixels. Recognizing the above problems, an image signal processor according to an embodiment of the present disclosure can reduce the degree of hue jumps and / or discoloration during the re-mosaic processing of an RGBW pattern, thereby improving the quality of the image signal.

[0015] Reference will now be made in detail to some embodiments of the present disclosure, examples of which are shown in the accompanying drawings. Whenever possible, the same reference numerals will be used throughout the accompanying drawings to represent the same or similar parts. Although the present disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof are shown as examples in the accompanying drawings. However, the present disclosure should not be construed as being limited to the embodiments set forth herein.

[0016] Hereinafter, various embodiments will be described with reference to the accompanying drawings. However, it should be understood that the present disclosure is not limited to a specific embodiment, but includes various modifications, equivalents and / or substitutions of the embodiments. The embodiments of the present disclosure can provide various effects that can be directly or indirectly recognized by the present disclosure.

[0017] Figure 1 is a block diagram showing an imaging device 1 including a readout circuit 30 according to an embodiment of the present disclosure.

[0018] Reference Figure 1 , the imaging device 1 may include an image sensing device 10 and an image signal processor ISP.

[0019] Here, the image sensing device 10 may be or include a complementary metal oxide semiconductor image sensor (CIS) for converting an optical signal into an electrical signal. The image sensing device 10 may control overall operations such as on / off control, operation mode control, operation timing control, sensitivity control, etc. by using a timing controller (not shown).

[0020] Image sensing device 10 may include a pixel array 20 and a readout circuit 30 .

[0021] The pixel array 20 may include a plurality of pixels arranged continuously (e.g., arranged continuously in row and column directions) in a two-dimensional (2D) matrix structure. Each of the plurality of pixels of the pixel array 20 may generate a pixel signal obtained by converting an optical signal into an electrical signal and output the converted pixel signal to the readout circuit 30.

[0022] The readout circuit 30 may process analog pixel signals output from respective pixels and generate digital data corresponding to the pixel signals. For example, the readout circuit 30 may include an analog-to-digital converter (ADC) for performing ADC conversion of such pixel signals.

[0023] The image signal processor ISP may perform image processing on the image data received from the readout circuit 30. The image signal processor ISP may reduce the noise of the image data and may perform various types of image signal processing (e.g., gamma correction, color filter array interpolation, color matrix, color correction, color enhancement, lens distortion correction, etc.) for image quality improvement of the image data. In addition, the image signal processor ISP may compress the image data created by performing the image signal processing for image quality improvement so that the image signal processor ISP may create an image file using the compressed image data. Alternatively, the image signal processor ISP may restore the image data from the image file. In this case, the scheme for compressing such image data may be a reversible format or an irreversible format. As a representative example of such a compression format, in the case of using a still image, a Joint Photographic Experts Group (JPEG) format, a JPEG 2000 format, etc. may be used. In addition, in the case of using a moving image, a plurality of frames may be compressed according to the Moving Picture Experts Group (MPEG) standard so that a moving image file may be created. For example, an image file may be created according to the Exchangeable Image File Format (Exif) standard.

[0024] The image transmitted from the image sensing device 10 to the image signal processor ISP may be an RGBW pattern in which a Bayer image and a white image are mixed. The image signal processor ISP may perform signal processing to generate a Bayer image with less noise using a white image in an RGBW (red, green, blue, white) pattern including the Bayer image and the white image.

[0025] The image signal processor ISP may include a re-mosaic processor 100 , a noise suppression processor 200 , a storage circuit 300 , a noise amount estimator 400 , and a noise reduction degree determiner 500 .

[0026] Here, the re-mosaic processor 100 may perform a re-mosaic process on the input image to generate a converted image having a Bayer pattern. In this case, the Bayer pattern may refer to a pattern in which green, red, and blue are alternately arranged at a specific ratio according to human visual characteristics. In addition, the re-mosaic process may refer to image processing that converts an original image into an image having a Bayer pattern.

[0027] The noise suppression processor 200 may perform noise suppression processing on the converted image and generate an output image based on the noise reduction degree information received from the noise reduction degree determiner 500. When the noise reduction degree is suppressed, the noise suppression processor 200 may perform noise suppression processing by referring to the input image as a reference value.

[0028] For example, when image capture (i.e., imaging) is performed at a high gain in a relatively dark low-light environment, random noise may increase, and after the re-mosaic processing of the captured image, the standard deviation of the noise may further increase. Therefore, in the case where the random noise must be reduced, the noise suppression processor 200 may increase the degree of noise reduction. On the other hand, if the amount of noise is excessively reduced and reaches a preset condition (e.g., when the amount of noise is less than or equal to the rounding error), a tone jump or discoloration may occur. Therefore, in the case where the noise must be maintained at a predetermined amount or more, the noise suppression processor 200 may suppress the degree of noise reduction.

[0029] The storage circuit 300 may store parameters required for estimating the amount of noise. For example, the storage circuit 300 may include a register, and determine the parameters required for estimating the amount of noise as register values.

[0030] For example, the image signal processor ISP may study (or test, detect) the amount of noise after the re-mosaic process in advance, and store the average amount of noise in the storage circuit 300. Here, the amount of noise to be generated after the re-mosaic process may be estimated from the number of pixels to be referenced when processing the statistical value (r) required for the re-mosaic process. In some other embodiments, software for executing the algorithm (e.g., floating point number) required for the re-mosaic process is pre-written so that various images can be processed using the software. Finally, the relationship between the noise standard deviation of the input image for each gain setting and the noise standard deviation of the converted image for each gain setting may be studied and calculated as needed.

[0031] The noise amount estimator 400 may estimate the noise amount of the converted image based on the input image, the converted image, and the noise amount parameter received from the storage circuit 300. The noise amount estimator 400 may estimate the noise amount in various ways, and the method of estimating the noise amount will be referred to in detail later. Figure 3 Describe in more detail.

[0032] Based on the noise amount estimated by the noise amount estimator 400, the noise reduction degree determiner 500 may determine the noise reduction degree, and may generate information about the determined noise reduction degree (hereinafter referred to as "noise reduction degree information"). The method of calculating the noise reduction degree by the noise reduction degree determiner 500 will be referred to in detail later. Figure 3 Describe in more detail.

[0033] Figure 2 The present invention is shown in the embodiment of the present invention. Figure 1 A schematic diagram of the operation of the re-mosaic processor 100 is shown.

[0034] Reference Figure 2, the input image applied to the re-mosaic processor 100 may be an RGBW pattern in which a Bayer image and a white image are mixed. The RGBW pattern may include color pixels of red (R), blue (B), and green (G) pixels and white pixels.

[0035] Each color pixel (R, G, and B pixel) may be a pixel on which light of a specific wavelength is incident through a corresponding color filter. In addition, a white pixel (W) may be a pixel without a color filter and may be referred to as a texture pixel, a black and white pixel, or a single pixel. For example, a color pixel (R, G, B) may obtain a color pixel value according to incident light of a specific color (e.g., red, green, blue, etc.) through a color filter that transmits only incident light of a specific wavelength. Conversely, a white pixel (W) may obtain a white pixel value according to incident light not limited to a specific color (e.g., visible light).

[0036] exist Figure 2 In the embodiment, although an RGBW pattern having a size of 64 pixels arranged in a (8×8) matrix has been described as an example for convenience of description, the size of the RGBW pattern and the arrangement shape of each pixel are not limited thereto.

[0037] As the pixels of the image sensing device 10 are miniaturized, the amount of incident light to each pixel is reduced, resulting in a decrease in the signal-to-noise ratio (SNR), which will cause rapid degradation in the performance of the image sensing device 10. In order to increase the transmittance of each filter and realize high-sensitivity pixels in response to a low-light environment, the utilization of the image sensing device 10 including a white pixel and a color filter (e.g., an RGB filter) designed to transmit only light having a specific wavelength (e.g., red, green, and blue (RGB)) is rapidly increasing.

[0038] Compared with RGB images, white images may have a relatively strong signal-to-noise ratio (SNR) characteristic. Since white images are designed to receive full-band light compared to RGB images that receive light in a specific band, white images may have a relatively low SNR even when acquired in a low-light environment.

[0039] The re-mosaic processor 100 may separate the Bayer image and the white image from each other by binning the RGBW pattern. Here, "binning" may mean the sum of pixel values ​​of unit pixels having the same color based on a sub-pixel array (SP). For example, the re-mosaic processor 100 may sum (e.g., 4 sums) the pixel values ​​of unit pixels having the same color in the RGBW pattern in units of four (2×2) sub-pixels (i.e., based on sub-pixels), resulting in the formation of a Bayer image and a white image.

[0040] The image signal processor ISP can output an image with good SNR (ie, Bayer image) by performing re-mosaic processing using a white image. In particular, when a high gain is applied to the image in a low illumination environment, a resulting image with excellent SNR can be output.

[0041] The RGBW pattern may be a single-plate image sensor in which R, G, B, and W filters are arranged in a mosaic shape. Therefore, when generating a color image, it is necessary to perform a re-mosaic process as color encoding to generate RGB pixel values ​​corresponding to each pixel. Here, "re-mosaic processing" may be a task of outputting a high-quality image signal using a pixel signal.

[0042] The re-mosaic processor 100 may perform re-mosaic processing on the original image using a re-mosaic method, resulting in a converted image. In some embodiments, the re-mosaic method may be implemented as a method of performing re-mosaic processing based on a preset reference equation. The reference equation may be an equation for calculating the pixel values ​​of each pixel included in the image on which the re-mosaic processing has been performed. For example, the reference equation may be an equation for calculating the pixel values ​​of the re-mosaic processed image obtained when weights are applied to the pixel values ​​of adjacent pixels in the original image.

[0043] In some embodiments, the re-mosaic processor 100 may be implemented as a software block to be executed by a predetermined processor, or may be implemented as a combination of a dedicated hardware block and a processing unit.

[0044] Figure 3 is a flowchart illustrating an image signal processing method according to an embodiment of the present disclosure.

[0045] Reference Figure 3 , the image signal processor ISP may set parameters for calculating the noise amount to adjust the noise reduction degree of the output image, and store the set parameters in the storage circuit 300. That is, the storage circuit 300 may set various types of information related to the noise to be reflected in the noise reduction processing of the re-mosaic processor 100 as parameters, and may store the set information.

[0046] For example, the storage circuit 300 may store not only the pixel value of each of the red, green, and blue pixels of the input image, but also the pixel value generated when light is blocked. In another example, the storage circuit 300 may store the average noise reduction amount of the re-mosaic processor 100. As another example, the storage circuit 300 may store the average noise variance value of the input image and the converted image. In another example, the storage circuit 300 may store parameters related to the shot noise amount and the noise variance value generated when light is blocked.

[0047] In another example, the storage circuit 300 may store a gain value of pixel data of an input image. The image signal processor ISP may perform digital operations to calculate the gain of the pixel data during a re-mosaic operation. For example, the image signal processor ISP may perform various types of operations by adding gain to the pixel data during image signal processing (e.g., noise reduction, white balance adjustment, demosaicing, color correction, gamma correction, color change, etc.).

[0048] The amount of noise generated after the re-mosaic process may vary according to the amount of noise in the input image. The amount of noise in the input image may mainly depend on the gain (sensitivity) setting of the image sensing device 10. Therefore, the image signal processor ISP may specify a parameter related to the amount of noise based on the gain value, and may store the specified parameter in the storage circuit 300. Here, the relationship between the gain value and the amount of noise may be obtained by performing pixel data processing in advance and converting the processing result into data.

[0049] For example, the image signal processor ISP may preset the above-mentioned parameters related to the noise amount, may implement these parameters in software, and may store the software in the storage circuit 300. When the software is executed in the image signal processor ISP (or a CPU (central processing unit) located inside or outside the ISP), the image signal processor ISP may store the parameters as register values ​​in the storage circuit 300.

[0050] Thereafter, the noise amount estimator 400 may estimate the noise amount of the converted image based on the noise amount parameter received from the storage circuit 300 (operation S1). Hereinafter, specific embodiments capable of estimating the noise amount by the noise amount estimator 400 will be described in detail.

[0051] For example, after the re-mosaic processor 100 performs noise processing based on the pixel values ​​of the input image, the noise amount estimator 400 may estimate a noise variance value of the converted image. The noise amount estimator 400 may estimate a noise variance value (V) of the converted image represented by the following Equation 1.

[0052] [Formula 1]

[0053] V=R{k(I-I0)+V0}

[0054] In Equation 1, R may represent the noise variance value (V out ) and the noise variance value of the input image (V in ) ratio (V out / V in) (hereinafter referred to as variance ratio). Here, the variance ratio R may represent an average noise reduction amount to be reflected in the noise reduction processing of the re-mosaic processor 100. The average value of the variance ratio R may be pre-stored in the storage circuit 300. In addition, the average value of the variance ratio R may be set to different values ​​according to the sensitivity at the time of image capture (shooting), and may be a function depending on the pixel value (I) of the input image.

[0055] In the above Equation 1, “I” may represent a value of each red (R) pixel, a value of each green (G) pixel, and a value of each blue (B) pixel of the input image, and “I0” may represent a pixel value obtained when light is blocked.

[0056] In addition, in the above equation 1, "k" may be a parameter related to the amount of shot noise, and "k" may be proportional to the gain. In addition, V0 may represent a noise variance value of the image sensing device 10 when light is blocked. "k" may be a parameter based on the gain value. As the gain at the time of image capture (shooting) increases, the estimated value of the noise variance value (V) may also increase.

[0057] That is, as can be seen from the above equation 1, the noise amount estimator 400 can obtain a first value by subtracting the pixel value (I0) of the input image obtained by light blocking from the pixel value (I) of the input image, obtain a second value by multiplying the first value by a parameter (k) related to the shot noise amount, obtain a third value by adding the noise variance value (V0) obtained by light blocking to the second value, and multiply the third value by the average value of the variance ratio R, thereby estimating the noise variance value (V) of the converted image.

[0058] As another example, the noise amount estimator 400 can estimate the noise amount by distinguishing various regions of the input image. In the target kernel, the input image can be divided into a flat region (i.e., a first region) and a texture region (i.e., a second region). Here, the flat region may mean an area where the difference in pixel values ​​between pixels included in the target kernel is not large (e.g., the difference in pixel values ​​between pixels is less than a first value). The texture region may refer to an area containing texture. The image data corresponding to one frame may include textures of various sizes and shapes. Texture may refer to a collection (aggregation) of pixels having similarities. For example, a subject with a uniform color included in a scene may be identified as a texture. The boundary of the texture may be defined as an edge, and the pixel data may vary significantly inside the edge (inside the texture) and outside (outside the texture). That is, the texture region may refer to an area where the difference in pixel values ​​between pixels included in the target kernel is large (e.g., the difference in pixel values ​​between pixels is greater than or equal to a first value).

[0059] Since the flat region does not include an edge region, it may be relatively easier to reduce noise than the texture region. On the other hand, since the texture region includes an edge region, it may be relatively difficult to reduce noise than the flat region. As a result, although the input image of the re-mosaic processor 100 has the same pixel value (I), the amount of noise of the converted image may vary depending on whether the corresponding image belongs to the flat region or the texture region within the input image.

[0060] As has been described in the above Equation 1, the value of the variance ratio R may be stored in the storage circuit 300. The storage circuit 300 may distinguish between the average value of the variance ratio R in the flat region (hereinafter referred to as R1) and the average value of the variance ratio R in the texture region (hereinafter referred to as R2), and may store the obtained average values.

[0061] The noise amount estimator 400 may determine each region of the input image. When the corresponding region includes a flat region, the noise amount estimator 400 may select a variance ratio R1 as an R value and estimate a noise variance value (V) represented by Equation 1. Then, the noise amount estimator 400 may determine each region of the input image. When the corresponding region includes a texture region, the noise amount estimator 400 may select a variance ratio R2 as an R value and estimate a noise variance value (V) represented by Equation 1.

[0062] As another example, the noise amount estimator 400 may estimate the noise amount by measuring the degree of noise reduction. That is, the noise amount estimator 400 may estimate the noise amount based on the noise amount calculated during the noise reduction process of the re-mosaic processor 100. For example, the noise amount estimator 400 may estimate the noise variance value (V) of the converted image represented by the following equation 2.

[0063] [Equation 2]

[0064] V={k(I-I0)+V0} / N

[0065] In Equation 2, "I" may represent the value of each red (R) pixel, the value of each green (G) pixel, and the value of each blue (B) pixel of the input image. "I0" may represent the pixel value obtained when light is blocked, and "N" may represent the number of adjacent pixels used in the variance calculation. In addition, in the above Equation 2, "k" may be a parameter related to the amount of shot noise, and "V0" may represent the noise variance value of the image sensing device 10 when light is blocked.

[0066] That is, as can be seen from the above Equation 2, the noise amount estimator 400 can obtain a fourth value by subtracting the pixel value (I0) of the input image obtained by light blocking from the pixel value (I) of the input image, obtain a fifth value by multiplying the fourth value by a parameter (k) related to the shot noise amount, obtain a sixth value by adding the noise variance value (V0) obtained by light blocking to the fifth value, and divide the sixth value by the number of adjacent pixels (N), thereby estimating the noise variance value (V) of the converted image.

[0067] During the noise reduction process, the re-mosaic processor 100 may use the pixel values ​​of neighboring pixels adjacent to the target pixel to determine the pixel value of the target pixel. The re-mosaic processor 100 may select N neighboring pixels located around the target pixel, where N is an integer of 2 or greater, and may determine the value of the target pixel by calculating the statistical values ​​of the neighboring pixels (e.g., average pixel value, etc.). Here, as the value of N increases, the value of the noise variance value generated after the re-mosaic process may decrease. For example, when the average value of the pixel values ​​of N neighboring pixels is calculated, the noise variance value (V) may be represented by 1 / N. Therefore, the noise amount estimator 400 may use the number of neighboring pixels (N) to estimate the noise variance value (V), as represented by Equation 2.

[0068] The image signal processor ISP may convert the RGBW pattern into a Bayer pattern (i.e., a converted image) by performing various types of signal processing on the RGBW pattern during the re-mosaic process. For example, during the re-mosaic process, the re-mosaic processor 100 may perform a digital calculation operation of multiplying the pixel data by a gain. Here, quantization errors may accumulate, which may cause a difference in color perception between images. Therefore, in order to reduce noise and quickly perform arithmetic calculations when re-mosaicing the RGBW pattern, the image signal processor ISP may discard the decimal part from the bits of the pixel data value without representing the decimal part, or may limit the decimal part to a preset finite number of bits.

[0069] However, when image capture (shooting) is performed at low gain in a relatively bright high-illuminance environment, the standard deviation of random noise becomes smaller than a certain value, so that the influence of rounding error may be greater than the influence of random noise. For example, if the amount of random noise becomes less than the value "1" after re-mosaic processing, the influence of rounding error may increase, and false color may appear. Therefore, when the amount of noise is excessively reduced during re-mosaic processing, tone jumps or discoloration, etc. may appear in the output image.

[0070] Therefore, the noise reduction degree determiner 500 may calculate the noise reduction degree information (operation S2) based on the noise amount estimated by the noise amount estimator 400. For example, when the noise variance value (V) estimated by the noise amount estimator 400 becomes smaller than the value "1", the noise reduction degree determiner 500 may recognize that a tone jump or a color change occurs in the output image.

[0071] Although for the sake of convenience of description, the embodiments of the present disclosure disclose that false colors appear in the output image when the noise variance value is less than "1", the scope or spirit of the present disclosure is not limited to this. It should be noted that the reference value (standard) of the noise variance value can also be set to another value as needed.

[0072] [Equation 3]

[0073] L=α(IJ)+J

[0074] In Equation 3, "I" may represent the pixel value of the input image before noise reduction, "J" may represent the pixel value of the image converted after noise reduction, and "α" may represent the noise suppression parameter. For example, when α is set to zero "0" (i.e., the first parameter value), there is no suppression effect of noise reduction (hereinafter referred to as the "first reduction degree"), and when α is set to "1" (i.e., the second parameter value), the input pixel value (I) may be output so that the suppression effect of noise reduction can be maximized (hereinafter referred to as the "second reduction degree"). When the noise suppression parameter is set to α, the noise reduction degree determiner 500 may calculate the pixel value (L) obtained when the noise reduction degree is determined by the above Equation 3. That is, the L value may represent the noise reduction degree information obtained by suppressing the noise reduction effect.

[0075] That is, as can be seen from the above Equation 3, the noise reduction degree determiner 500 can obtain the seventh value by subtracting the pixel value (J) of the converted image from the pixel value (I) of the input image, can obtain the eighth value by multiplying the seventh value by the noise suppression parameter (α), and can add the pixel value (J) of the converted image to the eighth value, so that the noise reduction degree determiner 500 can determine the pixel value (L) obtained when determining the noise reduction degree.

[0076] Subsequently, the noise suppression processor 200 may control the reduction degree of the noise amount of the converted image based on the noise reduction degree information (operation S3). That is, the noise suppression processor 200 may suppress the noise reduction degree to prevent the noise variance value of the output image from being equal to or less than "1".

[0077] [Equation 4]

[0078] Ve~α 2 V in +V out (1-α 2 )~V in {α2 +R(1-α 2 )}

[0079] The noise variance value (Ve) of the output image finally output from the noise suppression processor 200 can be obtained as expressed in the above equation 4. Therefore, the noise reduction degree determiner 500 can determine the noise suppression parameter (α) so that the noise variance value (Ve) of the output image output from the noise suppression processor 200 becomes "1" (or another integer value) or greater. The noise suppression parameter (α) determined by the noise reduction degree determiner 500 can be set to "0≤α≤1".

[0080] Therefore, the output image output from the noise suppression processor 200 after the re-mosaic process can be an output image obtained when the noise reduction degree is suppressed. Therefore, the embodiment of the present disclosure can estimate the noise amount of the converted image, and can adjust the noise reduction degree based on the estimated noise amount, so that the tone jump, color change, etc. can be prevented from occurring in the output image.

[0081] Figure 4 The present invention is a diagram showing an embodiment of the present invention. Figure 1 The imaging device 1 shown is a block diagram of a computing device 1000 corresponding to the imaging device 1.

[0082] Reference Figure 4 , the computing device 1000 may represent a computer program product for executing Figure 1 An embodiment of a hardware configuration of an operation of the imaging device 1.

[0083] The computing device 1000 may be mounted on a chip independent of a chip on which an image sensing device is mounted. According to one embodiment, a chip on which the image sensing device 10 is mounted and a chip on which the computing device 1000 is mounted may be implemented in one package (e.g., a multi-chip package (MCP)), but the scope of the present disclosure is not limited thereto.

[0084] in addition, Figure 1 The internal configuration or arrangement of the computing device 1000, the image sensing device 10, and the image signal processor ISP described in the foregoing may vary according to the implementation. For example, at least a portion of the computing device 1000 may be included in the image sensing device 10. Alternatively, at least a portion of the computing device 1000 may be included in the image sensing device 10. In this case, at least a portion of the computing device 1000 may be mounted together on a chip on which the image sensing device 10 is mounted.

[0085] The computing device 1000 may include a processor 1010 , a memory 1020 , an input / output interface 1030 , and a communication interface 1040 .

[0086] Processor 1010 can process execution Figure 1The processor 1010 may be referred to as an image signal processor ISP, but the scope of the present disclosure is not limited thereto.

[0087] The memory 1020 may store data and / or instructions required to perform operations of the components (100, 200, 300, 400, 500) of the image signal processor ISP, and may be accessed by the processor 1010. For example, the memory 1020 may be a volatile memory such as a dynamic random access memory (DRAM), a static random access memory (SRAM), etc. In addition, the memory 1020 may be a non-volatile memory such as a programmable read-only memory (PROM), an erasable PROM (EPROM), an EEPROM (electrically erasable PROM), a flash memory, etc.

[0088] That is, a computer program for executing the operation of the image signal processor ISP disclosed in this document is recorded in the memory 1020 and is executed and processed by the processor 1010, thereby realizing the operation of the image signal processor ISP.

[0089] The input / output interface 1030 is an interface that connects an external input device (eg, keyboard, mouse, touch panel, etc.) and / or an external output device (eg, display) to the processor 1010 to allow transmission and reception of data.

[0090] The communication interface 1040 is a component that can transmit and receive various data with an external device (eg, an application processor, an external memory, etc.), and may be a device that supports wired or wireless communication.

[0091] As apparent from the above description, the image signal processor and the image signal processing method according to the embodiments of the present disclosure can reduce the degree of tone jumping and / or color change during re-mosaic processing of an RGBW pattern, thereby improving the quality of an image signal.

[0092] The embodiments of the present disclosure may provide various effects that can be directly or indirectly recognized through the above-mentioned embodiments of the present disclosure.

[0093] Although a number of exemplary embodiments have been described, it should be understood that modifications and enhancements to the disclosed embodiments and other embodiments are contemplated based on what is described and / or shown in the present disclosure and the following claims. Furthermore, these embodiments may be combined to form additional embodiments.

[0094] CROSS-REFERENCE TO RELATED APPLICATIONS

[0095] This application claims priority to and the benefit of Korean Patent Application No. 10-2023-0117866, filed on September 5, 2023, which is incorporated herein by reference in its entirety and made a part hereof.

Claims

1. An image signal processor, comprising: a remosaic processor that performs a remosaic process on an input image to generate a converted image; a noise amount estimator for estimating a noise amount of the converted image based on a preset noise amount parameter and the input image; a noise reduction degree determiner that determines a noise reduction degree based on the noise amount estimated by the noise amount estimator and generates noise reduction degree information; as well as A noise suppression processor generates an output image in which the noise reduction degree is controlled based on the noise reduction degree information.

2. The image signal processor according to claim 1, wherein: The input image includes a Bayer image and a white image.

3. The image signal processor according to claim 1, wherein: The converted image includes a Bayer image.

4. The image signal processor according to claim 1, wherein: The re-mosaicing process is performed by the following operations: Binning the input image to generate a Bayer image and a white image; and A noise reduction process is performed on the Bayer image based on the white image. 5 . The image signal processor according to claim 1 , further comprising a storage circuit for storing the noise amount parameter.

6. The image signal processor according to claim 1, wherein: The noise suppression processor performs processing to suppress the noise reduction degree using the input image as a reference value based on the noise reduction degree information.

7. The image signal processor according to claim 6, wherein: When the noise variance value of the output image is less than or equal to a preset rounding error, the noise suppression processor suppresses the noise reduction degree.

8. The image signal processor according to claim 1, wherein: The noise amount estimator estimates a noise variance value of the converted image based on pixel values ​​of the input image.

9. The image signal processor according to claim 8, wherein: The noise amount parameter is determined based on an average value of a variance ratio of a noise variance value of the converted image to a noise variance value of the input image.

10. The image signal processor according to claim 9, wherein: The noise amount estimator estimates a noise variance value of the converted image based on the variance ratio, a pixel value of the input image obtained when light is blocked, a noise variance value obtained when light is blocked, and a parameter related to a shot noise amount.

11. The image signal processor according to claim 1, wherein: The noise amount estimator estimates the noise amount as different values ​​in a first area and a second area of ​​the input image.

12. The image signal processor according to claim 11, wherein: The first region includes a flat region where a difference in pixel values ​​between pixels included in the target kernel is less than a preset first value; and The second region includes a texture region in which a difference in pixel values ​​between pixels included in the target kernel is equal to or greater than the first value.

13. The image signal processor according to claim 11, wherein: In the first region, a ratio of a noise variance value of the converted image to a noise variance value of the input image is set to a first variance ratio; and In the second region, a ratio of a noise variance value of the converted image to a noise variance value of the input image is set to a second variance ratio different from the first variance ratio.

14. The image signal processor according to claim 13, wherein: The noise amount estimator: determining the first area and the second area of ​​the input image; estimating a noise variance value of the converted image by applying the first variance ratio to the first region; and A noise variance value of the converted image is estimated by applying the second variance ratio to the second region.

15. The image signal processor according to claim 1, wherein: The noise amount estimator estimates the noise amount based on the noise amount calculated during the noise reduction process performed by the re-mosaic processor.

16. The image signal processor according to claim 15, wherein: The noise amount estimator estimates the noise variance value of the converted image based on the pixel values ​​of the input image, the pixel values ​​of the input image obtained when light is blocked, the noise variance value obtained when light is blocked, parameters related to the shot noise amount, and the average pixel value of neighboring pixels located around the target pixel.

17. The image signal processor according to claim 15, wherein: The noise reduction degree determiner: When the noise suppression parameter is a first parameter value, outputting the noise reduction degree information indicating a first reduction degree; and When the noise suppression parameter is a second parameter value greater than the first parameter value, the noise reduction degree information indicating a second reduction degree is output.

18. A method for processing an image signal, the method comprising the following steps: performing a re-mosaic process on an input image including a Bayer image and a white image to generate a converted image; estimating a noise variance value of the converted image based on a preset noise amount parameter and a pixel value of the input image; determining a degree of noise reduction based on the noise variance value; as well as The noise reduction degree of the converted image is suppressed based on the noise reduction degree.

19. The image signal processing method according to claim 18, wherein: The step of estimating the noise variance value comprises the following steps: Calculating an average value of the variance ratio of the noise variance value of the converted image to the noise variance value of the input image; calculating a first value by subtracting a pixel value of the input image obtained when light is blocked from a pixel value of the input image; calculating a second value by multiplying the first value by a parameter related to the amount of shot noise; calculating a third value by adding a noise variance value obtained when light is blocked to the second value; and The third value is multiplied by the average value of the variance ratios.

20. The image signal processing method according to claim 18, wherein: The step of determining the degree of noise reduction comprises the following steps: calculating a noise suppression parameter corresponding to the noise variance value of the converted image; and suppressing the noise reduction degree based on the noise suppression parameter so that the noise variance value of the converted image is equal to or greater than a preset first value; and Generates an output image with suppressed noise reduction.

Images