Image processing apparatus and image processing method

Abstract

The invention discloses an image processing apparatus and an image processing method. The image processing apparatus includes a gradation map generator configured to selectively identify luminance map data based on a saturation map and a luminance map corresponding to an input image, and configured to generate a gradation map based on at least one of the identified luminance map data. The image processing apparatus also includes a noise corrector configured to generate processed image data in which a noise value for the input image is corrected based on the grayscale map.

Description

TECHNICAL FIELD

[0001] The technology and implementations disclosed in this patent document relate generally to an image processing apparatus and an image processing method capable of performing image conversion. BACKGROUND

[0002] An image sensing apparatus is an apparatus that captures an optical image by converting light into an electrical signal using a photosensitive semiconductor material that reacts to light. As the automobile, medical, computer, and communication industries develop, the demand for high-performance image sensing apparatuses is increasing in various fields such as smartphones, digital cameras, game consoles, Internet of Things (IoT), robots, security cameras, and medical microcameras.

[0003] An image sensing apparatus converts a light signal detected by a pixel array into an electrical signal. When an image sensing apparatus converts a light signal into an electrical signal, a direct current (DC) offset can occur. When a DC offset occurs, the brightness (luminance) of a final image becomes brighter by the amount of the DC offset, and thus the contrast between bright and dark regions of the image can be reduced. SUMMARY

[0004] According to one embodiment of the disclosed technology, an image processing apparatus can include a grayscale map generator configured to selectively identify luminance map data based on a saturation map and a luminance map corresponding to an input image, and configured to generate a grayscale map based on the identified luminance map data; and a noise corrector configured to generate processed image data in which a noise value for the input image is corrected based on the grayscale map.

[0005] According to another embodiment of the disclosed technology, an image processing apparatus can include a grayscale map generator configured to selectively identify luminance map data based on a saturation map and a luminance map corresponding to an input image, and configured to generate a grayscale map based on the identified luminance map data; a dark region detector configured to detect a dark region from an area of the grayscale map; and a noise calculator configured to calculate a noise value for the input image based on pixel data for one or more pixels included in the dark region.

[0006] According to another embodiment of the disclosed technology, an image processing apparatus can include a saturation map generator configured to generate a saturation map based on a standard deviation between pixel data of a plurality of pixels, a luminance map generator configured to generate a luminance map by extracting luminance information from the plurality of pixels, a grayscale map generator configured to selectively identify luminance map data based on the saturation map and the luminance map, and configured to generate a grayscale map based on the identified luminance map data, and a dark region detector configured to detect a dark region from an area of the grayscale map.

[0007] According to another embodiment of the disclosed technology, an image processing method can include identifying data less than or equal to a second threshold value from data of a saturation map for an input image, detecting a dark region based on the data less than or equal to the first threshold value and the data less than or equal to the second threshold value, and calculating a direct current (DC) offset noise value for the input image based on the detected dark region. BRIEF DESCRIPTION OF DRAWINGS

[0008] The above and other features and aspects of the disclosed technology will become more apparent when considered in connection with the following detailed description, taken in conjunction with the accompanying drawings.

[0009] Figure 1 is a block diagram illustrating an example of an image signal processor included in an image processing apparatus based on some implementations of the disclosed technology.

[0010] Figure 2 is a block diagram illustrating an example of a mapping data generator based on some implementations of the disclosed technology.

[0011] Figure 3 is a block diagram illustrating an example of a noise corrector based on some implementations of the disclosed technology.

[0012] Figure 4 is a flowchart illustrating an example operation of an image signal processor based on some implementations of the disclosed technology.

[0013] Figure 5 is a graph illustrating an example of pixel data in a high luminance environment based on some implementations of the disclosed technology.

[0014] Figure 6 is a graph illustrating an example of pixel data in a medium luminance environment based on some implementations of the disclosed technology.

[0015] Figure 7is a graph illustrating an example of pixel data in a low brightness environment based on some implementations of the disclosed technology.

[0016] Figure 8 is a diagram illustrating an example of mapping data based on some implementations of the disclosed technology.

[0017] Figure 9 is a diagram illustrating an example of mapping data based on some implementations of the disclosed technology. Figure 1 is a block diagram of an example of a computing device of an image processing apparatus. DETAILED DESCRIPTION

[0018] The present patent document provides implementations and examples of an image processing apparatus and an image processing method that can perform image conversion that can be used in a configuration for substantially addressing one or more technical problems or engineering problems and mitigating limitations or drawbacks encountered in some image processing apparatuses in the art. Some implementations of the disclosed technology relate to an image processing apparatus that is able to correct direct current (DC) offset noise. Some implementations of the disclosed technology relate to an image processing method that detects DC offset noise and corrects an image signal based on the detected noise. It is recognized that the above-described problems can be addressed by an image processing apparatus that can generate a high-contrast image even when DC offset noise occurs. The image processing apparatus can correct an amplified noise value even when the noise value is amplified by applying an analog gain or a digital gain to the image processing apparatus in a low brightness environment.

[0019] Reference will now be made in detail to some implementations of the disclosed technology, examples of which are illustrated in the accompanying drawings. Wherever possible, like reference numbers will be used throughout the drawings to refer to like or similar elements. While every effort has been made to make this disclosure as accurate and comprehensive as possible, it is understood that the specific embodiments shown in the drawings and described in this document are presented by way of example only and should not be construed as limiting the disclosed technology.

[0020] In the following, various embodiments are described with reference to the accompanying drawings. However, it should be understood that the disclosed technology is not limited to the specific embodiments, but includes various modifications, equivalents, and / or alternatives of the embodiments. Embodiments of the disclosed technology can provide various effects that can be directly or indirectly appreciated by the disclosed technology.

[0021] Some implementations of the disclosed technology relate to an image processing apparatus that is able to correct direct current (DC) offset noise. Other implementations of the disclosed technology can relate to an image processing method that detects DC offset noise and corrects an image signal based on the detected noise. It should be understood that the foregoing general description and the following detailed description of the disclosed technology are both illustrative and explanatory and are intended to provide further explanation of the claimed disclosure.

[0022] Figure 1 is a block diagram illustrating an example of an image signal processor (ISP) 100 included in an image processing apparatus 10 based on some implementations of the disclosed technology.

[0023] Referring to Figure 1 , the image processing apparatus 10 can be embedded in an electronic apparatus, or can be implemented as an electronic apparatus. Here, the electronic apparatus can capture (or photograph) an image, can display a captured image, or can perform an operation based on a captured image. In this case, the electronic apparatus can be, for example, a digital camera, a smartphone, a wearable apparatus, an Internet of Things (IoT) apparatus, a Personal Computer (PC), a tablet, a Personal Digital Assistant (PDA), a Portable Multimedia Player (PMP), a navigation apparatus, a drone, etc., or can be mounted to other apparatuses that function as constituent elements of various electronic apparatuses such as vehicles, medical apparatuses, furniture, manufacturing equipment, security apparatuses, doors, various measuring instruments, etc.

[0024] The image processing apparatus 10 can include an image sensing apparatus and an image signal processor (ISP) 100. The image signal processor 100 can perform at least one image signal processing on image data (IDATA) to generate processed image data (IDATA_P). The image signal processor 100 can reduce noise of the image data (IDATA), and can perform various types of image signal processing (e.g., demosaicing, defective pixel correction, gamma correction, color filter array interpolation, color matrix manipulation, color correction, color enhancement, lens distortion correction, DC offset correction, etc.) for image quality improvement of the image data. Further, the image signal processor 100 can compress the image data created by performing the image signal processing for the image quality improvement, so that the image signal processor 100 can create an image file using the compressed image data. Alternatively, the image signal processor 100 can restore the image data from the image file. In this case, a scheme for compressing such image data can be a reversible format or an irreversible format. As a representative example of such a compression format, in the case of using a still image, a Joint Photographic Experts Group (JPEG) format, a JPEG 2000 format, etc. can be used. Further, in the case of using a moving image, a plurality of frames can be compressed according to a Moving Picture Experts Group (MPEG) standard, so that a moving image file can be created.

[0025] Image data (IDATA) can be generated by an image sensing device that captures an optical image of a scene, although the scope of the disclosed technology is not limited thereto. The image sensing device can include a pixel array including a plurality of pixels configured to sense incident light received from a scene, a control circuit configured to control the pixel array, and a readout circuit configured to output digital image data (IDATA) by converting an analog pixel signal received from the pixel array into the digital image data (IDATA). In some implementations of the disclosed technology, it is assumed that the image data (IDATA) is generated by the image sensing device.

[0026] Image data (IDATA) can include noise due to defects in the pixels or errors that occur in the process of converting the light signal into an electrical signal. In particular, when an analog gain or a digital gain is applied to the image data in a low brightness environment, the noise of the image data can also be amplified. In some implementations, for ease of description and better understanding of the disclosed technology, defects or errors that cause a difference between the image data (IDATA) and the actual image will be collectively referred to as “noise” hereinafter.

[0027] To improve the quality of a color image, the correction accuracy of a defective pixel can be improved. To this end, an image signal processor (ISP) 100 based on some implementations of the disclosed technology can include a mapping data generator 200 and a noise corrector 300.

[0028] The mapping data generator 200 can generate mapping data for detecting noise in the image data (IDATA). In the following description, for ease of description, digital data corresponding to a pixel signal of each pixel will be defined as pixel data, and information indicating a position of a pixel corresponding to a mapping standard and data of the pixel will be defined as mapping data hereinafter. For example, luminance mapping data for the image data (IDATA) can correspond to data representing a position of each pixel for the image data (IDATA) and a luminance value corresponding to the position of each pixel. Later, with reference to Figure 2 A more detailed operation of the mapping data generator 200 is described.

[0029] The noise corrector 300 can calculate a noise value based on the mapping data received from the mapping data generator 200. The noise corrector 300 can generate processed image data (IDATA_P) by correcting the pixel data based on the calculated noise value. For example, the noise corrector 300 can generate the processed image data (IDATA_P) by eliminating DC offset noise of the pixel data based on the mapping data. Later, with reference to Figure 3 A more detailed operation of the noise corrector 300 is described.

[0030] Figure 2 is a block diagram illustrating an example of a mapping data generator 200 based on some implementations of the disclosed technology.

[0031] Referring to Figure 2 , the mapping data generator 200 can include a luminance (Y) image converter 210, a luminance mapping map generator 220, an RGB image separator 230, a median filter 240, a saturation mapping map generator 250, and a grayscale mapping map generator 260. For example, the mapping data generator 200 can receive image data (IDATA), and can transmit grayscale mapping map G(x, y) data to the noise corrector 300.

[0032] According to one embodiment, the luminance image converter 210 can receive image data (IDATA) from an image sensing device, and can convert a color space domain of the received image data (IDATA). For example, the luminance image converter 210 can convert an RGB domain image of the image data (IDATA) into a YUV domain image or a YCbCr domain image. In one example, a color space domain conversion formula can use a Keith Jack conversion method or a Julen conversion method. For example, the Keith Jack conversion method can use formulas such as “Y = (0.257 x R) + (0.504 x G) + (0.098 x B) + 16”, “Cb = (0.439 x R) + (0.368 x G) – (0.071 x B) + 128”, and “Cr = – (0.148 x R) – (0.291 x G) + (0.439 x B) + 128” when attempting to convert an RGB domain image into a YCbCr domain image. For example, when attempting to convert an RGB domain image into a YCbCr domain image, pixel data corresponding to a red channel, pixel data corresponding to a green channel, and pixel data corresponding to a blue channel can be substituted into the formulas of the Keith Jack conversion method, and data corresponding to Y, data corresponding to Cb, and data corresponding to Cr can be calculated.

[0033] According to one embodiment, the image data (IDATA) can correspond to coordinates (x, y) of a pixel array, and the pixel coordinates can include pixels of a unit pattern. For example, (1, 1) of the pixel array can include a red channel pixel, two green channel pixels, or a blue channel pixel included in a unit pattern of a Bayer pattern. In one example, pixel data of the two green channel pixels of the unit pattern can correspond to an average value of pixel data of the two green channel pixels, and can include data X i (x, y). The pixel data can include pixel data corresponding to an “i” channel and pixel coordinate data. For example, pixel data located at coordinates (1, 1) and corresponding to a red channel can be represented by X R(1, 1) indicates. For example, pixel data located at coordinates (1, 1) and corresponding to a blue channel can be represented by X B (1, 1) indicates.

[0034] According to one embodiment, the luminance image converter 210 can transmit luminance data (Y) to the luminance map generator 220. In one example, the luminance image converter 210 can convert an RGB domain image of the image data (IDATA) into a YUV domain image or a YCbCr domain image, and can extract luminance data (Y) from the converted image. For example, the luminance image converter 210 can extract luminance data (Y) from an RGB image using a Keith Jack conversion method or a Julen conversion method, and can transmit the extracted luminance data (Y) to the luminance map generator 220.

[0035] According to one embodiment, the luminance map generator 220 can receive luminance data (Y) from the luminance image converter 210, and can generate a luminance map Y(x, y) based on the received luminance data (Y). For example, the luminance map generator 220 can generate a luminance map Y(x, y) using luminance data (Y), in which the luminance data (Y) and coordinate information (x, y) of the pixel array are combined with each other. In one example, the luminance map can correspond to mapping data indicating luminance data (Y) for each coordinate of the pixel array. In one example, the luminance map generator 220 can transmit the luminance map Y(x, y) to the grayscale map generator 260. The luminance image converter 210 and the luminance map generator 220 according to some implementations of the disclosed technology are merely examples, and can operate as a single module. For example, the luminance map generator 220 can receive image data (IDATA) to identify luminance data (Y), and can generate a luminance map Y(x, y) by combining the identified luminance data (Y) with coordinate data corresponding to the image data (IDATA) of the identified luminance data (Y).

[0036] According to an embodiment, the RGB image separator 230 can receive image data (IDATA) from an image sensing device, and can classify the received image data (IDATA) into an R image (R_i) composed of pixel data corresponding to a red color channel, a G image (G_i) composed of pixel data corresponding to a green color channel, and a B image (B_i) composed of pixel data corresponding to a blue color channel. In one example, a color gamut of the image data (IDATA) can include another color gamut different from an RGB gamut. For example, the image data (IDATA) can correspond to an RGBW gamut image including pixel data (W) corresponding to a white color channel. For example, the image data (IDATA) can be classified by an image separator (not shown) into an R image composed of pixel data corresponding to a red color channel, a G image composed of pixel data corresponding to a green color channel, a B image composed of pixel data corresponding to a blue color channel, and a W image composed of pixel data (W) corresponding to a white color channel.

[0037] According to an embodiment, the R image (R_i), the G image (G_i), and the B image (B_i) generated by the RGB image separator 230 can correspond to data in which coordinate information (x, y) of a pixel array is incorporated. For example, upon receiving the image data (IDATA), the RGB image separator 230 can generate the R image (R_i) by incorporating the pixel data corresponding to the red color channel and the coordinate information (x, y) of the pixel array. In one example, the R image (R_i) can correspond to mapping data indicating that the pixel data corresponding to the red color channel is mapped for each coordinate of the pixel array. For example, upon receiving the image data (IDATA), the RGB image separator 230 can generate the G image (G_i) by incorporating the pixel data corresponding to the green color channel and the coordinate information (x, y) of the pixel array. In one example, the G image (G_i) can correspond to mapping data indicating that the pixel data corresponding to the green color channel is mapped for each coordinate of the pixel array. For example, upon receiving the image data (IDATA), the RGB image separator 230 can generate the B image (B_i) by incorporating the pixel data corresponding to the blue color channel and the coordinate information (x, y) of the pixel array. In one example, the B image (B_i) can correspond to mapping data indicating that the pixel data corresponding to the blue color channel is mapped for each coordinate of the pixel array.

[0038] According to an embodiment, the RGB image separator 230 can transmit the R image (R_i), the G image (G_i), and the B image (B_i) to the median filter 240. In one example, the RGB image separator 230 can separate the image data (IDATA) according to a plurality of color channels (e.g., an R channel, a G channel, and a B channel), and can transmit the separated images to the median filter 240.

[0039] According to an embodiment, the median filter 240 can receive the R image (R_i), the G image (G_i), and the B image (B_i) from the RGB image separator 230. In one example, the median filter 240 can remove noise included in the received R image (R_i), noise included in the G image (G_i), and noise included in the B image (B_i). In one example, the noise to be removed by the median filter 240 can be distinguished from the noise to be processed by the noise corrector 300, which will be described later. For example, the median filter 240 can perform a pre-processing operation to assist the processing operation of the saturation map generator 250, the grayscale map generator 260, or the noise corrector 300. For example, the median filter 240 can remove pixel noise generated due to pixel defects of a pixel array of an image sensing device. In one example, the median filter 240 can perform at least some of the image signal processing operations of the image signal processor (ISP) 100 described above for improving image quality.

[0040] According to an embodiment, the median filter 240 can perform a pre-processing operation for removing noise included in the R image (R_i), noise included in the G image (G_i), and noise included in the B image (B_i) received from the RGB image separator 230. The pre-processing operation can correspond to a pre-processing operation for a subsequent operation (e.g., a saturation map generation operation, a grayscale map generation operation, a dark region detection operation, or a noise calculation operation). In one example, the noise can include noise related to a defective pixel included in the R image (R_i), noise related to a defective pixel included in the G image (G_i), noise related to a defective pixel included in the B image (B_i), noise related to temporary light inflow, or noise related to signal overflow. In one example, the median filter 240 can generate a pre-processed R image (R'_I), a pre-processed G image (G'_I), and a pre-processed B image (B'_I). In one example, the median filter 240 can transmit the pre-processed R image (R'_I), the pre-processed G image (G'_I), and the pre-processed B image (B'_I) to the saturation map generator 250.

[0041] According to an embodiment, the saturation map generator 250 can receive the pre-processed R image (R'_I), the pre-processed G image (G'_I), and the pre-processed B image (B'_I) from the median filter 240. In one example, the saturation map generator 250 can generate a saturation map (S_map) based on the received pre-processed R image (R'_I), the pre-processed G image (G'_I), and the pre-processed B image (B'_I). In one example, the saturation map generator 250 can identify pixel data corresponding to a red channel, pixel data corresponding to a green channel, and pixel data corresponding to a blue channel for each coordinate by using the pre-processed R image (R'_I), the pre-processed G image (G'_I), and the pre-processed B image (B'_I).

[0042] According to one embodiment, the saturation map generator 250 can generate a saturation map wherein the coordinate information (x, y) of the pixel array is combined with the standard deviation according to the coordinates of the pixel data corresponding to the identified red channel, the pixel data corresponding to the identified green channel, and the pixel data corresponding to the identified blue channel. In one example, the standard deviation of the pixel data corresponding to the red channel, the pixel data corresponding to the green channel, and the pixel data corresponding to the blue channel for each coordinate can correspond to the saturation of the image data (IDATA) for each coordinate.

[0043] For example, when the image data (IDATA) is closer to an achromatic color, the pixel data of the R channel, the pixel data of the G channel, and the pixel data of the B channel are identical or similar to each other, and thus the standard deviation of the pixel data of each of the R channel, the G channel, and the B channel is at a low value. For example, when the image data (IDATA) is closer to a primary color, the difference between the pixel data of the R channel, the pixel data of the G channel, and the pixel data of the B channel is at a high value, such that the standard deviation of the pixel data of the R channel, the pixel data of the G channel, and the pixel data of the B channel is higher.

[0044] According to one embodiment, the saturation map generator 250 can generate a saturation map based on pixels corresponding to a first color filter, pixels corresponding to a second color filter, and pixels corresponding to a third color filter. For example, the first color filter can correspond to the R channel, the second color filter can correspond to the G channel, or the third color filter can correspond to the B channel. In one example, the saturation map generator 250 can generate a saturation map based on a standard deviation between pixel data of the pixels corresponding to the first color filter to the third color filter.

[0045] According to one embodiment, the saturation map can be calculated using Equation 1 below

[0046] In Equation 1, σ(x, y) can correspond to a standard deviation for obtaining a saturation map X i may correspond to pixel data for each channel, The average of the pixel data corresponding to the (R, G, B) channels can correspond to (x, y), and (x, y) can correspond to coordinates corresponding to the pixel array. Sigma (σ) can correspond to the standard deviation indicated in Equation 1, which can represent a saturation map of an embodiment

[0047] [Equation 1]

[0048]

[0049] According to one embodiment, when the color of the image data (IDATA) corresponding to (1, 1) is black, the pixel data value corresponding to the red channel, the pixel data value corresponding to the green channel, and the pixel data value corresponding to the blue channel can all correspond to zero "0". At this time, at the (1, 1) position, since the pixel data of the (R, G, B) channels is represented by (0, 0, 0), the standard deviation of the (1, 1) coordinates can correspond to zero "0".

[0050] According to one embodiment, when the color of the image data (IDATA) corresponding to (1, 1) is red, the pixel data value corresponding to the red channel can correspond to 255, and each of the pixel data value corresponding to the green channel and the pixel data value corresponding to the blue channel can correspond to zero "0". At this time, at the (1, 1) position, since the pixel data of the (R, G, B) channels is represented by (255, 0, 0), the standard deviation of the (1, 1) coordinates can correspond to about "120.2".

[0051] According to one embodiment, when the color of the image data (IDATA) corresponding to (1, 1) is gray, each of the pixel data value corresponding to the red channel, the pixel data value corresponding to the green channel, and the pixel data value corresponding to the blue channel can correspond to 100. At this time, since the pixel data of the (R, G, B) channels at the (1, 1) position is represented by (100, 100, 100), the standard deviation of the (1, 1) coordinates can correspond to zero "0".

[0052] According to one embodiment, when the color of the image data (IDATA) corresponding to (1, 1) is white, each of the pixel data value corresponding to the red channel, the pixel data value corresponding to the green channel, and the pixel data value corresponding to the blue channel can correspond to 255. At this time, at the (1, 1) position, the pixel data of the (R, G, B) channels is represented by (255, 255, 255), so the standard deviation of the (1, 1) coordinates can correspond to zero "0".

[0053] The RGB image separator 230, the median filter 240, and the saturation map generator 250 based on some implementations of the disclosed technology are merely examples and can operate as one module. For example, the saturation map generator 250 can receive image data (IDATA), can classify the received image data (IDATA) into an R image (R_i) composed of pixel data corresponding to a red channel, a G image (G_i) composed of pixel data corresponding to a green channel, and a B image (B_i) composed of pixel data corresponding to a blue channel, and can generate a saturation map based on a standard deviation between the pixel data of the R image (R_i), the G image (G_i), and the B image (B_i) The respective components of the RGB image separator 230, the median filter 240, and the saturation map generator 250 based on some implementations of the disclosed technology can be combined with each other, or any one of all the components can be omitted as needed. For example, the median filter 240 can be combined with the RGB image separator 230 or the saturation map generator 250 to operate as a single module. For example, the median filter 240 can be omitted as needed.

[0054] According to one embodiment, the grayscale map generator 260 can receive the luminance map Y(x,y) from the luminance map generator 220 and can receive the saturation map S(x,y) from the saturation map generator 250 In one example, the grayscale map generator 260 can generate a grayscale map G(x,y) based on the received luminance map Y(x,y) and the received saturation map S(x,y) In one example, the grayscale map generator 260 can generate a grayscale map G(x,y) based on the received luminance map Y(x,y) and the received saturation map S(x,y) to identify a luminance and a saturation for each coordinate.

[0055] According to one embodiment, the grayscale map generator 260 can generate a grayscale map G(x,y) in which coordinate information (x,y) is incorporated based on a result of comparing the identified luminance for each coordinate with the identified saturation for each coordinate. In one example, the data for each coordinate of the grayscale map G(x,y) can correspond to a luminance value for a low saturation region of the image data (IDATA). For example, the grayscale map generator 260 can identify a region in which a saturation is less than a threshold value in the image data (IDATA), and can apply the luminance map (Y(x,y)) data to the identified region, thereby indicating a luminance value for a low saturation region of the image data (IDATA).

[0056] According to one embodiment, the gray scale map generator 260 can identify a certain region in which the saturation is greater than or equal to a threshold value in the image data (IDATA), and can convert the identified region into data corresponding to white. In one example, the gray scale map generator 260 can set the pixel data of the region in which the saturation is greater than or equal to the threshold value in the image data (IDATA) to a maximum value 255.

[0057] According to one embodiment, the gray scale map G(x, y) can be calculated using Equation 2. In Equation 2, G(x, y) can correspond to the gray scale map, Y(x, y) can correspond to the luminance map, "Th" can correspond to a threshold value, and (x, y) can correspond to coordinates corresponding to a pixel array. In one example, the maximum value of the pixel data can correspond to 255.

[0058] [Equation 2]

[0059]

[0060] According to one embodiment, when the color corresponding to (1, 1) of the image data (IDATA) is black, the pixel data of the (R, G, B) channel is represented by (0, 0, 0), and the threshold value is 100, the saturation corresponding to (1, 1) is may correspond to zero "0". At this time, because the luminance Y(1, 1) is 16 (i.e., (0.257 x 0) + (0.504 x 0) + (0.098 x 0) + 16 = 16) based on a conversion method such as the Keith Jack conversion method, the luminance Y(1, 1) can correspond to 16. In one example, at the (1, 1) position, the saturation is less than the threshold value, so the gray scale map data G(1, 1) can correspond to 16 indicating the value of the luminance Y(1, 1).

[0061] According to one embodiment, when the color corresponding to (1, 1) of the image data (IDATA) is red, the pixel data of the (R, G, B) channel is represented by (255, 0, 0), and the threshold value is 100, the saturation corresponding to (1, 1) is may correspond to about "120.2". At this time, because the luminance Y(l,l) is 81.5 (i.e., (0.257 x 255) + (0.504 x 0) + (0.098 x 0) + 16 = 81.5) based on a conversion method such as the Keith Jack conversion method, the luminance Y(l,l) can correspond to 81.6. In one example, at the (1,1) position, the saturation is greater than the threshold, so the grayscale map data G(l,l) can correspond to 255.

[0062] According to one embodiment, when the color of the image data (IDATA) corresponding to (1,1) is green, the pixel data of the (R,G,B) channel is represented by (0, 100, 0), and the threshold is 100, the saturation corresponding to (1,1) is may correspond to about 57.7. At this time, because the luminance Y(l,l) is 66.4 (i.e., (0.257 x 0) + (0.504 x 100) + (0.098 x 0) + 16 = 66.4) based on a conversion method such as the Keith Jack conversion method, the luminance Y(l,l) can correspond to 66.4. In one example, at the (1,1) position, the saturation is less than the threshold, so the grayscale map data G(l,l) can correspond to 66.4, which indicates the value of the luminance Y(l,l).

[0063] According to one embodiment, when the color of the image data (IDATA) corresponding to (1,1) is gray, the pixel data of the (R,G,B) channel is represented by (100, 100, 100), and the threshold is 100, the saturation corresponding to (1,1) is may correspond to zero "0". At this time, because the luminance Y(l,l) is 101.9 (i.e., (0.257 x 100) + (0.504 x 100) + (0.098 x 100) + 16 = 101.9) based on a conversion method such as the Keith Jack conversion method, the luminance Y(l,l) can correspond to 101.9. In one example, at the (1,1) position, the saturation is less than the threshold, so the grayscale map data G(l,l) can correspond to 101.9, which indicates the value of the luminance Y(l,l).

[0064] According to one embodiment, when the color of the image data (IDATA) corresponding to (1,1) is white, the pixel data of the (R,G,B) channel is represented by (255, 255, 255), and the threshold is 100, the saturation corresponding to (1,1) is may correspond to zero "0". At this time, because the luminance Y(l, 1) is 235.0 (i.e., (0.257 x 255) + (0.504 x 255) + (0.098 x 255) + 16 = 235.0) based on a conversion method such as the Keith Jack conversion method, the luminance Y(l, 1) can correspond to 235.0. In one example, at the (1, 1) position, the saturation is less than the threshold, so the grayscale map data G(l, 1) can correspond to 235.0 indicating the value of the luminance Y(l, 1).

[0065] Figure 3 is a block diagram illustrating an example of a noise corrector 300 based on some implementations of the disclosed technology.

[0066] Referring to Figure 3 , the noise corrector 300 can include a dark region detector 310 and a noise calculator 320. In one example, the noise corrector 300 can receive grayscale map (i.e., G(x, y)) data and can output processed image data (IDATA P). In one example, the processed image data (IDATA P) can correspond to image data from which noise has been removed from the image data (IDATA). For example, the processed image data (IDATA P) can correspond to image data obtained when DC offset correction has been performed on the image data (IDATA). For example, the processed image data (IDATA P) can correspond to image data obtained when a noise value has been subtracted from pixel data corresponding to pixels of an input image.

[0067] According to one embodiment, the dark region detector 310 can receive grayscale map (i.e., G(x, y)) data from the map data generator 200 and can detect a dark region of the received grayscale map (i.e., G(x, y)) data. In one example, the dark region detector 310 can identify data including the lowest pixel data from the grayscale map (G(x, y)) data. In one example, the dark region detector 310 can detect a region including the lowest pixel data in a region of the grayscale map (G(x, y)) as a dark region.

[0068] According to one embodiment, the dark region detector 310 can identify a coordinate corresponding to the lowest pixel data among all coordinates of the gray scale map G(x, y), and can designate an (N x N) region corresponding to the identified coordinate as a dark region. For example, the dark region detector 310 can identify a coordinate corresponding to the lowest pixel data among all coordinates of the gray scale map G(x, y) as (100, 100), and can detect a (3 x 3) region centered on the identified coordinate (100, 100) as a dark region. For example, the dark region detector 310 can detect a region corresponding to (99, 99), (99, 100), (99, 101), (100, 99), (100, 100), (100, 101), (101, 99), (101, 100), and (101, 101) among the coordinates of the gray scale map (G(x, y)) as a dark region.

[0069] According to one embodiment, the dark region detector 310 can calculate a noise value (BLC i ) based on pixel data of pixels included in the dark region. In one example, the dark region detector 310 can identify an average value of pixel data of each channel of pixels included in a dark region having an (N x N) size as a noise value (BLC i ).

[0070] According to one embodiment, the noise value (BLC i ) can be calculated using Equation 3. In Equation 3, BLC i may be a noise value corresponding to a channel "i", "i" can be a channel identification parameter, N can be a size of a region, and (r, c) can be a coordinate corresponding to a dark region.

[0071] [Equation 3]

[0072] for i = {R, G, B}

[0073] According to one embodiment, the dark zone detector 310 can set the value of N to "3", and can identify coordinates corresponding to the lowest pixel data in the coordinates of the gray scale map G(x, y) centered at (50, 50). At this time, the dark zone detector 310 can detect a (3x3) region centered at the identified coordinates (50, 50) as a dark zone. In one example, the dark zone detector 310 can detect regions corresponding to (49, 49), (49, 50), (49, 51), (50, 49), (50, 50), (50, 51), (51, 49), (51, 50), and (51, 51) in the coordinates of the gray scale map (G(x, y)) as dark zones. At this time, in the coordinates of the gray scale map G(x, y), the (R, G, B) channel pixel data of the image data (IDATA) for (49, 49), (49, 50), (49, 51), (50, 49), (50, 50), (50, 51), (51, 49), (51, 50), and (51, 51) can correspond to (100, 0, 0), (100, 0, 0), (100, 0, 0), (100, 0, 0), (100, 0, 0), (100, 0, 0), (100, 0, 0), (100, 0, 0), and (100, 0, 0), respectively. In this case, the formula "BLC R = 100, BLC G = 0, BLC B = 0" can be satisfied.

[0074] According to one embodiment, the dark zone detector 310 can set the value of N to "5", and can identify coordinates corresponding to the lowest pixel data in the coordinates of the gray scale map G(x, y) centered at (100, 100). At this time, the dark zone detector 310 can detect a (5x5) region centered at the identified coordinates (100, 100) as a dark zone. In one example, the dark zone detector 310 can detect regions corresponding to (98, 98), (98, 99), (98, 100), (98, 101), (98, 102), (99, 98), (99, 99), (99, 100), (99, 101), (99, 102), (100, 98), (100, 99), (100, 100), (100, 101), (100, 102), (101, 98), (101, 99), (101, 100), (101, 101), (101, 102), (102, 98), (102, 99), (102, 100), (102, 101), and (102, 102) in the coordinates of the gray scale map G(x, y) as a dark zone. At this time, in the coordinates of the gray scale map G(x, y), the (R, G, B) channel pixel data of the image data (IDATA) for (98, 98), (98, 99), (98, 100), (98, 101), (98, 102), (99, 98), (99, 99), (99, 100), (99, 101), (99, 102), (100, 98), (100, 99), (100, 100), (100, 101), (100, 102), (101, 98), (101, 99), (101, 100), (101, 101), (101, 102), (102, 98), (102, 99), (102, 100), (102, 101), and (102, 102) can correspond to (50, 50, 30), (51, 50, 30), (52, 50, 30), (53, 50, 30), (54, 50, 30), (55, 50, 30), (56, 50, 30), (57, 50, 30), (58, 50, 30), (59, 50, 30), (60, 50, 30), (61, 50, 30), (62, 50, 30), (63, 50, 30), (64, 50, 30), (65, 50, 30), (66, 50, 30), (67, 50, 30), (68, 50, 30), (69, 50, 30), (70, 50, 30), (71, 50, 30), (72, 50, 30), (73, 50, 30), and (74, 0, 0), respectively. In this case, the formula "BLC R = 62, BLCG = 50, BLC B = 30”.

[0075] The above-described BLC i The calculation method is one example, but the scope of the disclosed technology is not limited thereto. According to one embodiment, the dark area detector 310 can identify coordinates corresponding to the lowest pixel data among the coordinates of the gray scale map G(x, y), and can designate an (M x N) region corresponding to the identified coordinates as a dark area. In one example, the dark area detector 310 can identify the lowest value among the pixel data of each channel for the pixels included in the dark area as a noise value (BLC i ).

[0076] According to one embodiment, the dark area detector 310 can transmit the noise value (BLC i ) to the noise calculator 320. In one example, the dark area detector 310 can detect regions corresponding to (49, 49), (49, 50), (49, 51), (50, 49), (50, 50), (50, 51), (51, 49), (51, 50), and (51, 51) among the coordinates of the gray scale map G(x, y) as dark areas. At this time, among the (R, G, B) channel pixel data of the image data (IDATA) for (49, 49), (49, 50), (49, 51), (50, 49), (50, 50), (50, 50), (50, 51), (51, 49), (51, 50), and (51, 51) in the coordinates of the gray scale map G(x, y), it can correspond to (100, 0, 0), (100, 0, 0), (100, 0, 0), (100, 0, 0), (100, 0, 0), (100, 0, 0), (100, 0, 0), (100, 0, 0), and (100, 0, 0), respectively. In this case, the dark area detector 310 can transmit data of "BLC R = 100, BLC G = 0, and BLC B = 0" to the noise calculator 320.

[0077] According to one embodiment, the noise calculator 320 can receive the noise value (BLC i ) from the dark area detector 310, and can output processed image data (IDATA_P) based on the received noise value (BLC i ). In one example, the noise calculator 320 can subtract (BLC R , BLC G , BLC B) to output processed image data (IDATA P). In more detail, the noise calculator 320 can subtract "BLC R " from all R channel pixel data of the image data (IDATA), can subtract "BLC G " from all G channel pixel data of the image data (IDATA), and can subtract "BLC B " from all B channel pixel data of the image data (IDATA), and thereby can output the processed image data (IDATA P).

[0078] According to one embodiment, the image data (IDATA) can include pixel data X i (x, y) of a pixel located at coordinates (x, y) corresponding to a channel "i", and the processed image data (IDATA P) can include corrected pixel data The corrected pixel data In Equation 4, X i (x, y) can be pixel data of an i channel pixel located at (x, y) before correction, may correspond to corrected pixel data of an i channel pixel located at (x, y), and BLC i may correspond to a noise value corresponding to the i channel.

[0079] [Equation 4]

[0080]

[0081] According to one embodiment, in a case where pixel data of (R, G, B) channels for a position (a, b) of the image data (IDATA) corresponds to (c, d, e) and "BLC R = 100, BLC G = 0, BLC B = 0" is given, it is determined that X R (a, b) = c, X G (a, b) = d, and X B (a, b) = e. As a result, the noise calculator 320 can calculate (as represented by ), can calculate (as represented by ), and can calculate (as represented by In one example, when the value of is less than or equal to zero "0", the noise calculator 320 can set The value of BLC i is determined to be zero "0". The dark region detector 310 and the noise calculator 320 based on some implementations of the disclosed technology are merely examples and can operate as a single module. For example, the noise calculator 320 can receive the grayscale map (G(x, y)) data and can calculate the noise value (BLC i ). Based on the calculated noise value (BLC i ), the noise calculator 320 can subtract (BLC R , BLC G , BLC B ) from the (R, G, B) channel pixel data of the image data (IDATA) and can output the processed image data (IDATA_P). More specifically, the noise calculator 320 can subtract "BLC R " from all R channel pixel data of the image data (IDATA), can subtract "BLC G " from all G channel pixel data of the image data (IDATA), and can subtract "BLC B " from all B channel pixel data of the image data (IDATA), thereby can output the processed image data (IDATA_P).

[0082] Figure 4 is a flowchart illustrating an example operation of an image signal processor based on some implementations of the disclosed technology.

[0083] Referring to Figure 4 , the image processing apparatus 10 can receive (S100) an input image from an image sensing apparatus. In one example, the image processing apparatus 10 can receive image data (IDATA) using the image sensing apparatus. In one example, the image data (IDATA) can be mapped as coordinates corresponding to a pixel array. For example, the image data (IDATA) can include pixel data X(x, y).

[0084] According to one embodiment, the image processing apparatus 10 can identify (S110) data from data of a luminance map for the input image based on a first threshold value. In one example, the image processing apparatus 10 can extract a luminance map for the received image data (IDATA) and can identify data from data of the luminance map based on the first threshold value. In one example, the first threshold value can correspond to data of a saturation map . For example, in connection with Figure 4 Referring to Figure 2, the image processing apparatus 10 can generate a luminance map Y(x, y) using the luminance map generator 220, in which luminance data (Y) is combined with coordinate information (x, y) of the pixel array. In one example, the image processing apparatus 10 can identify data in the data of the luminance map Y(x, y) that satisfies a first threshold condition. In one example, the first threshold condition can correspond to the second threshold described later. In one example, the operation of S110 can correspond to the operation of the luminance map generator 220 illustrated in Figure 2 .

[0085] According to one embodiment, the image processing apparatus 10 can identify (S120) data that is less than or equal to a second threshold from a saturation map for the input image. In one example, the image processing apparatus 10 can extract a saturation map for the received image data (IDATA) and can identify data that is less than or equal to the second threshold in the saturation map data. For example, in conjunction with Figure 4 Referring to Figure 2 , the image processing apparatus 10 can generate a saturation map S(x, y) using the saturation map generator 250. In one example, the image processing apparatus 10 can identify data in the data of the saturation map S(x, y) in which a saturation value is less than or equal to the second threshold.

[0086] According to one embodiment, the image processing apparatus 10 can generate a saturation map based on a standard deviation between pixel data of pixels corresponding to each of a plurality of channels (e.g., a red channel, a green channel, and a blue channel) included in a unit pixel group. In one example, the operation of S120 can correspond to the operation of the RGB separator 230, the median filter 240, and the saturation map generator 250 illustrated in Figure 2 .

[0087] According to one embodiment, the image processing apparatus 10 can detect (S130) a dark region from the data identified based on the first threshold and the data that is less than or equal to the second threshold. In one example, the image processing apparatus 10 can detect a common region between a region corresponding to the data identified based on the first threshold and a region corresponding to the data that is less than or equal to the second threshold from among regions of the image data (IDATA). In one example, the image processing apparatus 10 can detect a region including a pixel having the lowest pixel data among one or more common regions between the region corresponding to the data identified based on the first threshold and the region corresponding to the data that is less than or equal to the second threshold as a dark region. In one example, the operation of S130 can correspond to the operation of the grayscale map generator 260 and Figure 2 . Figure 3 ​the operation of the dark region detector 310.

[0088] According to one embodiment, the image processing apparatus 10 can calculate (S140) a DC offset noise for the input image based on the detected dark region. In one example, the image processing apparatus 10 can calculate an average value between pixel data of pixels included in the detected dark region corresponding to a color channel as a DC offset noise value. For example, the image processing apparatus 10 can determine an average value between pixel data of pixels included in the dark region corresponding to a red color channel as a DC offset noise value for pixels corresponding to the red color channel, can determine an average value between pixel data of pixels included in the dark region corresponding to a green color channel as a DC offset noise value for pixels corresponding to the green color channel, and can determine an average value between pixel data of pixels included in the dark region corresponding to a blue color channel as a DC offset noise value for pixels corresponding to the blue color channel. In one example, the operation of S140 can correspond to Figure 3 the operation of the dark region detector 310.

[0089] According to one embodiment, the image processing apparatus 10 can correct (S150) the input image by subtracting the calculated DC offset noise from the pixel data of the pixels. In one example, the image processing apparatus 10 can subtract a first average value between pixel data of pixels included in the dark region corresponding to a red color channel, a second average value between pixel data of pixels included in the dark region corresponding to a green color channel, and a third average value between pixel data of pixels included in the dark region corresponding to a blue color channel from (R, G, B) channel pixel data of the image data (IDATA). Here, the first average value, the second average value, and the third average value are calculated in operation S140.

[0090] According to one embodiment, the image processing device 10 can eliminate the DC offset noise value from all red channel pixel data of the image data (IDATA) by subtracting the average value between pixel data of pixels corresponding to the red channel among pixels included in the dark region from all red channel pixel data of the image data (IDATA). In one example, the image processing device 10 can eliminate the DC offset noise value from all green channel pixel data of the image data (IDATA) by subtracting the average value between pixel data of pixels corresponding to the green channel among pixels included in the dark region from all green channel pixel data of the image data (IDATA). In one example, the image processing device 10 can eliminate the DC offset noise value from all blue channel pixel data of the image data (IDATA) by subtracting the average value between pixel data of pixels corresponding to the blue channel among pixels included in the dark region from all blue channel pixel data of the image data (IDATA). In one example, the operation of S150 can correspond to Figure 3 the operation of the noise calculator 320 illustrated.

[0091] Figure 5 is a graph illustrating an example of pixel data in a high brightness environment based on some implementations of the disclosed technology.

[0092] Referring to Figure 5 , the pixel data in the high brightness environment can include pixel data corresponding to the image data (IDATA) and pixel data corresponding to the processed image data (IDATA_P). In one example, the image processing device 10 can receive the image data (IDATA) and generate the processed image data (IDATA_P) in an environment in which "20 lux" and 8 times analog gain (i.e., 8x analog gain) are set.

[0093] According to one embodiment, the pixel number according to the pixel data (i.e., represented by "RGB input") for the image data (IDATA) corresponds to a black bar graph. In one example, the pixel number according to the pixel data (i.e., represented by "BLC result") for the processed image data (IDATA_P) corresponds to a dot pattern bar graph. According to one embodiment, because a relatively low gain (e.g., analog gain and / or digital gain) is used in a high brightness environment (e.g., 20 lux), less DC offset distortion due to the gain can occur in the operation of generating the pixel data. Accordingly, the image processing device 10 can identify the noise value (BLC i ) as a relatively low value.

[0094] According to one embodiment, it can be seen that the number of pixels according to pixel data (BLC result) for the processed image data (IDATA_P) is shifted in the pixel data direction in which the number of pixels according to pixel data (BLC result) for the processed image data (IDATA_P) is generally smaller than the number of pixels according to pixel data (RGB input) for the image data (IDATA). In one example, the image processing apparatus 10 can perform the above operation by subtracting the noise value (BLC result) from all pixel data of the image data (IDATA). i ) to generate processed image data (IDATA_P), so that the shift value in the direction of lower pixel data may correspond to the noise value (BLC i ).

[0095] According to one embodiment, in an environment where 20 lux and 8 times (8×) analog gain are set, it can be confirmed that based on the number of pixels according to pixel data (RGB input) for the image data (IDATA), the shift value according to the number of pixels (BLC result) for the processed image data (IDATA_P) is relatively low.

[0096] Figure 6 is a chart illustrating examples of pixel data in a moderately bright environment based on some implementations of the disclosed technology.

[0097] Reference Figure 6 , the pixel data in the medium brightness environment may include pixel data corresponding to the image data (IDATA) and pixel data corresponding to the processed image data (IDATA_P). In one example, the image processing device 10 may receive the image data (IDATA) and generate the processed image data (IDATA_P) in an environment where 1 lux, 16 times analog gain (16×Analog Gain), and 7.99 times digital gain (7.99×Digital Gain) are set.

[0098] According to one embodiment, the number of pixels according to the pixel data (RGB input) for the image data (IDATA) corresponds to a black bar graph. In one example, the number of pixels according to the pixel data (BLC result) for the processed image data (IDATA_P) corresponds to a dot pattern bar graph. According to one embodiment, because a medium gain (e.g., analog gain and / or digital gain) is used in a medium brightness environment (e.g., 1 lux), a medium DC offset distortion due to gain may occur in the operation of generating pixel data. Therefore, the image processing device 10 may convert the noise value (BLC i ) is identified as an intermediate value.

[0099] According to one embodiment, it can be seen that the shift in the direction of pixel data for the number of pixels based on pixel data for the processed image data (BLC Result) from the number of pixels based on pixel data for the image data (RGB Input) is shifted to a medium level in the direction of pixel data for the number of pixels based on pixel data for the processed image data (BLC Result) from the number of pixels based on pixel data for the image data (RGB Input). In one example, the image processing device 10 can generate the processed image data (IDATA P) by subtracting the noise value (BLC i ) from all of the pixel data of the image data (IDATA), such that the shift value in the direction of lower pixel data can correspond to the noise value (BLC i ).

[0100] According to one embodiment, in an environment in which 1 lux, 16 times (16x) analog gain, and 7.99 times (7.99x) digital gain are set, it can be confirmed that the shift value for the number of pixels based on pixel data for the processed image data (BLC Result) from the number of pixels based on pixel data for the image data (RGB Input) is a medium level.

[0101] Figure 7 is a graph showing an example of pixel data in a low brightness environment based on some implementations of the disclosed technology.

[0102] Referring to Figure 7 , the pixel data in the low brightness environment can include pixel data corresponding to the image data (IDATA) and pixel data corresponding to the processed image data (IDATA P). In one example, the image processing device 10 can receive the image data (IDATA) and generate the processed image data (IDATA P) in an environment in which 0.1 lux, 16 times analog gain (16x analog gain), and 16 times digital gain (16x digital gain) are set.

[0103] According to one embodiment, the number of pixels based on pixel data for the image data (RGB Input) corresponds to a black bar graph. In one example, the number of pixels based on pixel data for the processed image data (BLC Result) corresponds to a dot pattern bar graph. According to one embodiment, because a relatively high gain (e.g., analog gain and / or digital gain) is used in a low brightness environment (e.g., 0.1 lux), a large gain-induced DC offset distortion can occur in the operation of generating pixel data. Accordingly, the image processing device 10 can identify the noise value (BLC i ) as a relatively high value.

[0104] According to one embodiment, it can be seen that the pixel quantity according to the pixel data for the processed image data (BLC result) is shifted in a direction of the pixel quantity according to the pixel data for the image data (RGB input) in which the pixel quantity according to the pixel data for the processed image data (BLC result) is generally smaller than the pixel quantity according to the pixel data for the image data (RGB input). In one example, the image processing apparatus 10 can generate the processed image data (IDATA P) by subtracting the noise value (BLC i ) from all of the pixel data of the image data (IDATA) such that the shift value in the direction of the lower pixel data can correspond to the noise value (BLC i ).

[0105] According to one embodiment, in an environment in which 0.1 lux, 16 times (16x) analog gain, and 16 times (16x) digital gain are set, it can be confirmed that the shift value of the pixel quantity according to the pixel data for the processed image data (BLC result) based on the pixel quantity according to the pixel data for the image data (RGB input) is relatively large.

[0106] Figure 8 FIG. 1 is a diagram illustrating an example of mapping data based on some implementations of the disclosed technology.

[0107] Referring to Figure 2 and Figure 8 , the mapping data generator 200 can identify an RGB image 810 from the image data (IDATA), can generate a saturation map 820 using the saturation map generator 250, and can generate a grayscale map 830 using the grayscale map generator 260. In one example, the RGB image 810 can correspond to an image including a black square 800 in which the pixel data of the (R, G, B) channels is (0, 0, 0), a gray background 801 in which the pixel data of the (R, G, B) channels is (100, 100, 100), a white triangle 802 in which the pixel data of the (R, G, B) channels is (255, 255, 255), a red circle in which the pixel data of the (R, G, B) channels is (255, 0, 0), a green star 804 in which the pixel data of the (R, G, B) channels is (0, 255, 0), and a blue pentagon 805 in which the pixel data of the (R, G, B) channels is (0, 0, 255).

[0108] According to one embodiment, because the (R, G, B) channel pixel data for the region corresponding to the square is (0, 0, 0), the saturation map generator 250 can identify the standard deviation as zero "0". Accordingly, the saturation map generator 250 can set the value of the portion of the saturation map 820 corresponding to the square to zero "0".

[0109] According to one embodiment, because the (R, G, B) channel pixel data for the region corresponding to the background is (100, 100, 100), the saturation map generator 250 can identify the standard deviation as zero "0". Accordingly, the saturation map generator 250 can set the value of the portion of the saturation map 820 corresponding to the background to zero "0".

[0110] According to one embodiment, because the (R, G, B) channel pixel data for the region corresponding to the triangle is (0, 0, 0), the saturation map generator 250 can identify the standard deviation as zero "0". Accordingly, the saturation map generator 250 can set the value of the portion of the saturation map 820 corresponding to the triangle to zero "0".

[0111] According to one embodiment, because the (R, G, B) channel pixel data for the region corresponding to the circle is (255, 0, 0), the saturation map generator 250 can identify the standard deviation as about "120.2". Accordingly, the saturation map generator 250 can set the value of the portion of the saturation map 820 corresponding to the circle to about "120.2".

[0112] According to one embodiment, because the (R, G, B) channel pixel data for the region corresponding to the star is (0, 255, 0), the saturation map generator 250 can identify the standard deviation as about "120.2". Accordingly, the saturation map generator 250 can set the value of the portion of the saturation map 820 corresponding to the star to about "120.2".

[0113] According to one embodiment, because the (R, G, B) channel pixel data for the region corresponding to the pentagon is (0, 0, 255), the saturation map generator 250 can identify the standard deviation as about "120.2". Accordingly, the saturation map generator 250 can set the value of the portion of the saturation map 820 corresponding to the pentagon to about "120.2".

[0114] According to one embodiment, because the (R, G, B) channel pixel data for the region corresponding to the square is (0, 0, 0), the grayscale map generator 260 can identify the standard deviation as zero "0". At this time, based on the Keith Jack conversion method, the luminance for the region corresponding to the square is set to, for example, 16 (= (0.257 x 0) + (0.504 x 0) + (0.098 x 0) + 16), and thus the standard deviation value (i.e., the saturation) is less than the threshold value when the threshold value is 100, so that the grayscale map generator 260 can set the value of the portion of the grayscale map 260 corresponding to the square to the luminance value 16.

[0115] According to one embodiment, because the (R, G, B) channel pixel data for the region corresponding to the background is (100, 100, 100), the grayscale map generator 260 can identify the standard deviation as zero "0". At this time, based on the Keith Jack conversion method, the luminance for the region corresponding to the background is set to, for example, 101.9 (= (0.257 x 100) + (0.504 x 100) + (0.098 x 100) + 16), and thus the standard deviation value (i.e., the saturation) is less than the threshold value when the threshold value is 100, so that the grayscale map generator 260 can set the value of the portion of the grayscale map 260 corresponding to the background to the luminance value 101.9.

[0116] According to one embodiment, because the (R, G, B) channel pixel data for the region corresponding to the triangle is (255, 255, 255), the grayscale map generator 260 can identify the standard deviation as zero "0". At this time, based on the Keith Jack conversion method, the luminance for the region corresponding to the triangle is set to, for example, 235.0 (= (0.257 x 255) + (0.504 x 255) + (0.098 x 255) + 16), and thus the standard deviation value (i.e., the saturation) is less than the threshold value when the threshold value is 100, so that the grayscale map generator 260 can set the value of the portion of the grayscale map 260 corresponding to the triangle to the luminance value 230.

[0117] According to one embodiment, because the (R, G, B) channel pixel data for the region corresponding to the circle is (255, 0, 0), the grayscale map generator 260 can identify the standard deviation as "120.2". At this time, because the standard deviation value (the saturation) for the region corresponding to the circle is higher than the threshold value when the threshold value is 100, the grayscale map generator 260 can set the value of the portion of the grayscale map 260 corresponding to the circle to the luminance value 255.

[0118] According to one embodiment, because the (R, G, B) channel pixel data for the region corresponding to the star shape is (0, 255, 0), the gray map generator 260 can identify the standard deviation as "120.2". At this time, because the standard deviation value (saturation) for the region corresponding to the star shape is higher than the threshold value when the threshold value is 100, the gray map generator 260 can set the value of the portion of the gray map 260 corresponding to the star shape to the luminance value 255.

[0119] According to one embodiment, because the (R, G, B) channel pixel data for the region corresponding to the star shape is (0, 255, 0), the gray map generator 260 can identify the standard deviation as "120.2". At this time, because the standard deviation value (saturation) for the region corresponding to the star shape is higher than the threshold value when the threshold value is 100, the gray map generator 260 can set the value of the portion of the gray map 260 corresponding to the star shape to the luminance value 255.

[0120] According to one embodiment, in conjunction with reference to Figure 3 Because the lowest pixel data of the gray map 830 is the pixel data of the pixel included in the region corresponding to the square shape, the dark area detector 310 can calculate the noise value (BLC i ) based on the pixel data of the pixel included in the region corresponding to the square shape. In one example, the lowest pixel data detected by the dark area detector 310 or the average pixel data of the region including the lowest pixel data can correspond to the DC offset. In one example, the dark area detector 310 can eliminate the DC offset noise by subtracting the lowest pixel data or the average pixel data of the region including the lowest pixel data from the pixel data of all pixels for each channel.

[0121] Based on some implementations of the disclosed technology Figure 8The RGB image 810, the saturation map 820, and the grayscale map 830 are merely examples showing a contrast between pixel data corresponding to various shapes included in the image, and the same pixel data can not correspond to the same color or pattern. For example, although the background color of the RGB image 810 and the background color of the grayscale map 830 are depicted as the same color, the (R, G, B) channel pixel data of the area corresponding to the background of the RGB image 810 can correspond to (100, 100, 100), and the value of the area corresponding to the background of the grayscale map 260 can correspond to a luminance value 101.9. For example, although the circle and the triangle of the grayscale map 260 are depicted in the same color, the grayscale map data value of the portion of the grayscale map 260 corresponding to the circle can correspond to 255, and the grayscale map data value of the portion of the grayscale map 260 corresponding to the triangle can correspond to a luminance value 235.0.

[0122] Figure 9 is a block diagram of an example of a computing device 900 of an image signal processor. Figure 1

[0123] Referring to Figure 9 , the computing device 900 can represent an embodiment of a hardware configuration for performing operations of the image signal processor 100. Figure 1

[0124] The computing device 900 can be mounted on a chip that is independent of a chip on which the image sensing device is mounted. According to one embodiment, the chip on which the image sensing device is mounted and the chip on which the computing device 900 is mounted can be implemented in one package (e.g., a multi-chip package (MCP)), but the scope of the disclosed technology is not limited thereto.

[0125] Further, Figure 1 The internal configuration or arrangement of the image sensing device and the image signal processor 100 described in the above can vary according to an embodiment. For example, at least a portion of the image sensing device can be included in the image signal processor 100. Alternatively, at least a portion of the computing device 900 can be included in the image sensing device. In this case, at least a portion of the computing device 900 can be mounted on a chip on which the image sensing device is also mounted.

[0126] The computing device 900 can include a processor 910, a memory 920, an input / output interface 930, and a communication interface 940.

[0127] The processor 910 can process to perform Figure 1 ​​The processor 910 may be data and / or instructions required for the operation of the components (200, 300) of the image signal processor 100 described in

[15] . That is, the processor 910 may refer to the image signal processor 100, but the scope of the disclosed technology is not limited thereto.

[0128] The memory 920 may store data and / or instructions required to perform operations of the components (200, 300) of the image signal processor 100 and may be accessed by the processor 910. For example, the memory 920 may be a volatile memory (e.g., dynamic random access memory (DRAM), static random access memory (SRAM), etc.) or a non-volatile memory (e.g., programmable read-only memory (PROM), erasable programm ...

[0129] That is, a computer program for executing the operations of the image signal processor 100 disclosed in this document may be recorded or stored in the memory 920 and executed and processed by the processor 910 , thereby implementing the operations of the image signal processor 100 .

[0130] The input / output interface 930 is an interface that connects an external input device (eg, keyboard, mouse, touch panel, etc.) and / or an external output device (eg, display) to the processor 910 to allow data transmission and reception.

[0131] The communication interface 940 is a component that can transmit and receive various data with an external device (eg, an application processor, an external memory, etc.), and may be a device that supports wired communication or wireless communication.

[0132] As apparent from the above description, an image processing apparatus based on some implementations of the disclosed technology can generate a high-contrast image even when DC offset noise is present.

[0133] For some embodiments, even when a noise value is amplified by applying an analog gain or a digital gain to the image processing device in a low-brightness environment, the image processing device may correct the amplified noise value.

[0134] Some embodiments of the disclosed technology can provide various effects that can be directly or indirectly recognized as described in this patent document.

[0135] Although a number of exemplary embodiments have been described, it should be understood that modifications and enhancements to the disclosed embodiments and other embodiments may be designed based on the content described and / or illustrated in this patent document. Therefore, the scope of the disclosed technology should not be limited to the above-described embodiments, but should include their equivalents.

[0136] Cross Reference to Related Applications

[0137] This patent document claims priority to and the benefit of Korean Patent Application No. 10-2024-0051143, filed April 17, 2024, the disclosure of which is incorporated herein in its entirety by reference.

Claims

1. An image processing apparatus comprising: a grayscale map generator that selectively identifies luminance map data based on a saturation map and a luminance map corresponding to an input image, and generates a grayscale map based on the identified luminance map data; and a noise corrector that generates processed image data in which a noise value for the input image is corrected based on the grayscale map. 2.The image processing apparatus according to claim 1, further comprising: a saturation map generator that generates the saturation map based on pixels corresponding to a first color filter, pixels corresponding to a second color filter, and pixels corresponding to a third color filter. The saturation map generator:

3. The image processing apparatus according to claim 2, wherein generates the saturation map based on a standard deviation between pixel data of pixels corresponding to the first color filter, pixel data of pixels corresponding to the second color filter, and pixel data of pixels corresponding to the third color filter. The grayscale map generator generates the grayscale map by:

4. The image processing apparatus according to claim 1, wherein converting data having a threshold value or more among data of the saturation map into data corresponding to white; and converting data less than the threshold value among data of the saturation map into data of the luminance map. 5.The image processing apparatus according to claim 1, wherein the processed image data is image data in which the noise value is subtracted from pixel data of pixels corresponding to the input image. 6.The image processing apparatus according to claim 1, wherein the noise value is a DC offset noise value associated with pixel data of pixels corresponding to the input image. The noise corrector includes:

7. The image processing apparatus according to claim 1, wherein a dark area detector that detects a dark area from an area of the grayscale map; and a noise calculator that calculates the noise value for the input image based on pixel data for at least one pixel included in the dark area. The dark area detector:

8. The image processing apparatus according to claim 7, wherein identifies data including lowest pixel data among data of the grayscale map; and detects an area corresponding to the data including the identified lowest pixel data as the dark area. The noise calculator:

9. The image processing apparatus according to claim 7, wherein determines pixel data for the at least one pixel included in the dark area as the noise value. The noise calculator calculates:

10. The image processing apparatus according to claim 7, wherein a first color average obtained by calculating an average between pixel data of pixels corresponding to a first color filter among pixels included in the dark area; a second color average obtained by calculating an average between pixel data of pixels corresponding to a second color filter among pixels included in the dark area; and a third color average obtained by calculating an average between pixel data of pixels corresponding to a third color filter among pixels included in the dark area. The noise calculator corrects the input image by:

11. The image processing apparatus according to claim 10, wherein subtracting the first color average from pixel data for pixels corresponding to the first color filter among pixels corresponding to the input image; ​ subtracting the second color average from pixel data of pixels corresponding to the second color filter among pixels corresponding to the input image; and subtracting the third color average from pixel data of pixels corresponding to the third color filter among pixels corresponding to the input image. 12.An image processing apparatus comprising: a saturation map generator that generates a saturation map based on a standard deviation between pixel data for a plurality of pixels; a luminance map generator that generates a luminance map by extracting luminance information from the plurality of pixels; a grayscale map generator that selectively identifies luminance map data based on the saturation map and the luminance map, and generates a grayscale map based on the identified luminance map data; and a dark region detector that detects a dark region from an area of the grayscale map. 13.An image processing method comprising the steps of: identifying data less than or equal to a second threshold from data of a saturation map for an input image; identifying data less than or equal to a second threshold from data of a saturation map for an input image; detecting a dark region based on data less than or equal to the first threshold and data less than or equal to the second threshold; and calculating a direct current (DC) offset noise value for the input image based on the detected dark region. 14.The image processing method of claim 13, further comprising the steps of: classifying and separating pixels corresponding to the input image into pixels corresponding to each of a plurality of channels. 15.The image processing method of claim 14, wherein, the plurality of channels include a red channel, a green channel, and a blue channel.

16. The image processing method of claim 14, wherein, the step of classifying and separating the pixels corresponding to the input image into pixels corresponding to each of the plurality of channels includes the step of: performing pre-processing to remove pixel noise from the input image. 17.The image processing method of claim 14, further comprising the steps of: generating the saturation map based on a standard deviation between pixel data of pixels corresponding to each of the plurality of channels among pixels included in a unit pixel group.

18. The image processing method of claim 13, wherein, the step of detecting the dark region includes the step of: detecting a common area between an area corresponding to data identified based on the first threshold and an area corresponding to data less than or equal to the second threshold from an area of the input image.

19. The image processing method of claim 13, wherein, the step of calculating the DC offset noise includes the step of: calculating an average between pixel data of pixels corresponding to a channel of a color among pixels included in the detected dark region.

20. The image processing method of claim 19, wherein, the step of calculating the DC offset noise further includes the step of: correcting the input image by subtracting the calculated average from pixel data of pixels corresponding to the channel of the color among pixels corresponding to the input image.

Images