Image processing apparatus and method

Abstract

To provide an image processing apparatus and method for correcting a target pixel in a kernel including a texture having directionality of a check pattern.SOLUTION: The image processing apparatus 100 includes a check pattern determiner configured to determine whether a texture in a kernel including a target pixel has a directionality of a predetermined check pattern, a diagonal pattern determiner configured to determine whether the texture has a directionality of a diagonal pattern when it is determined that the texture does not have the directionality of the predetermined check pattern, and a target pixel corrector configured to correct the target pixel based on a determination result of the check pattern determiner or the diagonal pattern determiner.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technical concepts of the present disclosure relate to an apparatus and method for processing images. [Background technology]

[0002] Image sensing devices are devices that capture optical images using the properties of photosensitive semiconductor materials that react to light. With the development of industries such as automobiles, medicine, computers, and communications, there is an increasing demand for high-performance image sensing devices in various fields such as smartphones, digital cameras, game consoles, the Internet of Things, robots, security cameras, and medical micro cameras.

[0003] An original image captured by an image sensing device may include an image that does not correspond to a normal image due to defective pixels. Since the image due to such defective pixels causes a decrease in image quality, a process for correcting the image due to the defective pixels is necessary. Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present disclosure is to provide an image processing device that corrects a target pixel in a kernel that includes a texture having a directional checker pattern.

[0005] An object of the present disclosure is to provide an image processing device that determines whether a texture in a kernel has a checkered pattern directionality.

[0006] An object of the present disclosure is to provide an image processing device that corrects a target pixel using a pixel that has the same color as the target pixel.

[0007] An object of the present disclosure is to provide an image processing device that determines whether or not a texture in a kernel has a diagonal pattern directionality.

[0008] The technical problems to be solved by the present disclosure are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by a person having ordinary skill in the technical field to which the present disclosure pertains from the following description. [Means for solving the problem]

[0009] An image processing device according to an exemplary embodiment of the present disclosure may include a check pattern determination unit that determines whether a texture in a kernel including a target pixel has a directionality of a predetermined check pattern, a diagonal pattern determination unit that determines whether the texture has a directionality of a diagonal pattern if it is determined that the texture does not have the directionality of the predetermined check pattern, and a target pixel correction unit that corrects the target pixel based on the determination result by the check pattern determination unit or the diagonal pattern determination unit.

[0010] According to one embodiment, the checkerboard pattern determination unit can determine whether the texture has a predetermined checkerboard directionality based on the vertical gradient sum of at least some pixels in the kernel, the horizontal gradient sum of at least some pixels, the slash gradient sum of at least some pixels, and the backslash gradient sum of at least some pixels.

[0011] According to one embodiment, the check pattern determination unit can determine whether the texture has a predetermined check pattern directionality based on whether a first direction corresponding to the maximum gradient sum and a second direction corresponding to the minimum gradient sum among the vertical gradient sum, the horizontal gradient sum, the slash gradient sum, and the backslash gradient sum are perpendicular to each other.

[0012] According to one embodiment, the check pattern determination unit can determine whether a texture has a predetermined check pattern directionality based on whether the value obtained by multiplying the minimum gradient sum among the vertical gradient sum, the horizontal gradient sum, the slash gradient sum, and the backslash gradient sum by a first ratio is greater than the remaining gradient sums excluding the minimum gradient sum.

[0013] According to one embodiment, the check pattern determination unit can determine whether the texture has a predetermined check pattern directionality based on a first sum obtained by summing a first gradient sum for the first row of the kernel, a second gradient sum for the first column of the kernel, a portion of the third gradient sum for the last column of the kernel, and a portion of the fourth gradient sum for the last row of the kernel; a second sum obtained by summing the first gradient sum, the third gradient sum, a portion of the second gradient sum, and a portion of the fourth gradient sum; a third sum obtained by summing the third gradient sum, the fourth gradient sum, a portion of the first gradient sum, and a portion of the second gradient sum; and a fourth sum obtained by summing the second gradient sum, the fourth gradient sum, a portion of the first gradient sum, and a portion of the third gradient sum.

[0014] According to one embodiment, the check pattern determination unit can determine whether the texture has a predetermined check pattern directionality based on whether the smallest sum among the first to fourth sums is smaller than the value obtained by multiplying the average brightness value of the kernel by the second ratio.

[0015] According to an embodiment, the checker pattern determining unit may calculate the average brightness value of the kernel by subtracting the offset value from the pixel values ​​of the green pixels in the kernel.

[0016] According to an embodiment, the check pattern determining unit may calculate the first to fourth gradient sums using pixels having the same color as the target pixel.

[0017] According to one embodiment, the check pattern determination unit can determine whether the texture has a predetermined check pattern directionality based on whether the value obtained by subtracting the minimum sum value from the maximum sum value among the first to fourth sum values ​​is greater than the value obtained by multiplying the average brightness value of the kernel by a third ratio.

[0018] According to one embodiment, the diagonal pattern determination unit may determine that the texture does not have a diagonal pattern direction if, among the vertical gradient sum of at least some pixels in the kernel, the horizontal gradient sum of at least some pixels, the slash direction gradient sum of at least some pixels, and the backslash direction gradient sum of at least some pixels, the slash direction gradient sum of at least some pixels is the smallest, a fifth slash direction gradient sum between an adjacent pixel of the target pixel and a pixel located in the slash direction of the adjacent pixel is greater than the average brightness value of the kernel multiplied by the fourth ratio, and the slash direction gradient sum is greater than the average brightness value of the kernel.

[0019] According to one embodiment, the diagonal pattern determination unit may determine that the texture does not have a diagonal pattern direction if, among the vertical gradient sum of at least some pixels in the kernel, the horizontal gradient sum of at least some pixels, the forward gradient sum of at least some pixels, and the backslash gradient sum of at least some pixels, the backslash gradient sum is the smallest, a sixth backslash gradient sum between an adjacent pixel of the target pixel and a pixel located in the backslash direction of the adjacent pixel is greater than the average brightness value of the kernel multiplied by the fourth ratio, and the backslash gradient sum is greater than the average brightness value of the kernel.

[0020] According to one embodiment, when the target pixel correction unit determines that the texture has a predetermined checkered pattern directionality, the target pixel correction unit can correct the target pixel based on a third direction corresponding to the second smallest gradient sum among the vertical gradient sum of at least some of the pixels in the kernel, the horizontal gradient sum of at least some of the pixels, the forward gradient sum of at least some of the pixels, and the backslash gradient sum of at least some of the pixels.

[0021] According to an embodiment, the target pixel correcting unit may correct the target pixel with an average value of pixel values ​​of pixels located in a third direction from the target pixel and having the same color as the target pixel.

[0022] According to one embodiment, if the target pixel correction unit determines that the texture does not have the directionality of a predetermined checkered pattern and does not have the directionality of a diagonal pattern, it can correct the target pixel based on a fourth direction corresponding to the smaller gradient sum among the vertical gradient sum of at least some of the pixels in the kernel and the horizontal gradient sum of at least some of the pixels.

[0023] According to an embodiment, the target pixel correcting unit may correct the target pixel with an average value of pixel values ​​of pixels located in a fourth direction from the target pixel and having the same color as the target pixel.

[0024] According to one embodiment, if the target pixel correction unit determines that the texture does not have a predetermined checkered pattern direction but has a diagonal pattern direction, the target pixel can be corrected with the average value of the pixel values ​​of pixels in the kernel that have the same color as the target pixel.

[0025] According to one embodiment, if the target pixel correction unit determines that the texture does not have a predetermined checkered pattern direction but has a diagonal pattern direction, it can correct the target pixel with the median value of the pixel values ​​of pixels in the kernel that have the same color as the target pixel.

[0026] According to one embodiment, if the target pixel correction unit determines that the texture does not have the directionality of a predetermined checkerboard pattern but has the directionality of a diagonal pattern, it can correct the target pixel based on the direction corresponding to the smallest gradient sum among the vertical gradient sum of at least some of the pixels in the kernel, the horizontal gradient sum of at least some of the pixels, the slash-direction gradient sum of at least some of the pixels, and the backslash-direction gradient sum of at least some of the pixels.

[0027] An image processing device according to an exemplary embodiment of the present disclosure may include a check pattern determination unit that determines whether a texture in a kernel has a direction of a predetermined check pattern based on a gradient sum of at least a portion of pixels in the kernel including the target pixel, and a target pixel correction unit that, when it is determined that the texture has a direction of the predetermined check pattern, corrects the target pixel based on a direction corresponding to the second smallest gradient sum among the vertical gradient sum, horizontal gradient sum, slash gradient sum, and backslash gradient sum of at least a portion of the pixels.

[0028] An image processing method according to an exemplary embodiment of the present disclosure may include a step of determining whether the texture in the kernel has a checkered pattern directionality based on the gradient sums for the first row, last row, first column, and last column of the kernel including the target pixel, and a step of correcting the target pixel based on the determination result. The above briefly summarized features of the present disclosure are illustrative aspects of the detailed description of the present disclosure that follows and are not intended to limit the scope of the present disclosure. [Effects of the Invention]

[0029] The image processing device according to the exemplary embodiment of the present disclosure can correct a target pixel in a kernel that includes a texture having a checkered pattern directionality. The image processing device according to the exemplary embodiment of the present disclosure can determine whether the texture in the kernel has the directionality of a checkered pattern.

[0030] The image processing device according to the exemplary embodiment of the present disclosure can correct the target pixel using a pixel having the same color as the target pixel. The image processing device according to the exemplary embodiment of the present disclosure can determine whether the texture in the kernel has a diagonal pattern directionality.

[0031] The effects obtained by the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by a person having ordinary skill in the technical field to which the present disclosure pertains from the following description. [Brief explanation of the drawings]

[0032] [Figure 1] FIG. 1 is a block diagram illustrating an image processing device according to an exemplary embodiment of the present disclosure. [Figure 2] FIG. 1 is a diagram illustrating a check pattern according to an exemplary embodiment of the present disclosure. [Figure 3] FIG. 1 is a diagram illustrating a check pattern according to an exemplary embodiment of the present disclosure. [Figure 4] 1 is a flowchart illustrating an image processing method according to an exemplary embodiment of the present disclosure. [Figure 5] FIG. 1 is a diagram illustrating an image processing method according to an exemplary embodiment of the present disclosure. [Figure 6] 1 is a flowchart illustrating an image processing method according to an exemplary embodiment of the present disclosure. [Figure 7] FIG. 1 is a diagram illustrating an image processing method according to an exemplary embodiment of the present disclosure. [Figure 8] FIG. 1 is a diagram illustrating an image processing method according to an exemplary embodiment of the present disclosure. [Figure 9] 1 is a flowchart illustrating an image processing method according to an exemplary embodiment of the present disclosure. [Figure 10] 2 is a block diagram illustrating an example of a computing device corresponding to the image processing device of FIG. 1. DETAILED DESCRIPTION OF THE INVENTION

[0033] Hereinafter, with reference to the accompanying drawings, detailed descriptions will be given of embodiments of the present disclosure so that those skilled in the art can easily implement the present disclosure, however, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein.

[0034] In describing embodiments of the present disclosure, if a detailed description of known configurations or functions is deemed to unnecessarily obscure the gist of the present disclosure, the detailed description will be omitted. In addition, in the drawings, parts that are not related to the description of the present disclosure will be omitted, and similar parts will be designated by similar reference numerals.

[0035] In this disclosure, when a component is "coupled," "coupled," or "connected" to another component, it means not only a direct connection, but also an indirect connection where there is another component between them. Furthermore, when a component "includes" or "has" another component, it does not mean that the other component is excluded, but that the component may further include the other component, unless otherwise specified.

[0036] In this disclosure, terms such as first and second are used only to distinguish one component from another, and do not limit the order or importance of the components unless otherwise specified. Therefore, within the scope of this disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly, a second component in one embodiment may be referred to as a first component in another embodiment.

[0037] In this disclosure, components that are distinguished from one another are used to clearly describe the characteristics of each component and do not necessarily mean that the components are separate. That is, multiple components may be integrated and consist of a single hardware or software unit, or a single component may be distributed and consist of multiple hardware or software units. Therefore, even if not otherwise specified, such integrated or distributed embodiments are also included within the scope of this disclosure.

[0038] In this disclosure, the components described in various embodiments do not necessarily mean essential components, and some may be optional components. Therefore, an embodiment consisting of a subset of the components described in one embodiment is also included in the scope of the present disclosure. Also, an embodiment including other components in addition to the components described in various embodiments is also included in the scope of the present disclosure.

[0039] In this disclosure, expressions of positional relationships used in this specification, such as upper, lower, left side, right side, etc., are described for the convenience of explanation, and when the drawings shown in this specification are viewed upside down, the positional relationships described in this specification may be interpreted in reverse.

[0040] In this disclosure, each of the phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B, or C," "at least one of A, B, and C," and "at least one of A, B, or C" can include any one or all possible combinations of the items listed together in that phrase.

[0041] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to FIGS. FIG. 1 is a block diagram illustrating an image processing device according to an exemplary embodiment of the present disclosure. FIG. 2 is a diagram illustrating a check pattern according to an exemplary embodiment of the present disclosure. FIG. 3 is a diagram illustrating a check pattern according to an exemplary embodiment of the present disclosure.

[0042] FIG. 1 will be described below with reference to FIGS. 2 and 3. FIG. Referring to FIG. 1, an image processing device 100 can perform at least one image signal processing on image data IDATA to generate processed image data IDATA_P.

[0043] The image processing device 100 may perform image signal processing on the image data IDATA to reduce noise and improve image quality, such as demosaicing, defective pixel correction, gamma correction, color filter array interpolation, color matrix, color correction, color enhancement, and lens distortion correction. The image processing device 100 may also compress image data that has undergone image signal processing for improving image quality to generate an image file, or restore image data from the image file. The image compression format may be a lossless format or a lossy format. Examples of compression formats that can be used for still images include the Joint Photographic Experts Group (JPEG) format and the JPEG2000 format. For moving images, a moving image file may be generated by compressing multiple frames according to the Moving Picture Experts Group (MPEG) standard.

[0044] The image data IDATA may be generated by an image sensing device that captures an optical image of a scene, although the scope of the present invention is not limited thereto. The image sensing device may include a pixel array including a plurality of pixels for detecting light incident from a scene, a control circuit for controlling the pixel array, and a readout circuit that converts analog pixel signals received from the pixel array into digital image data IDATA and outputs the digital image data. In this disclosure, the image data IDATA will be described assuming that it is generated by the image sensing device.

[0045] The pixel array may include a color filter array (CFA) in which color filters are arranged according to a certain pattern (e.g., a Bayer pattern, a quad Bayer pattern, a nona Bayer pattern, an RGBW pattern, etc.) so that each pixel array can detect light of a predetermined wavelength band. The pattern of the image data IDATA can be determined according to the type of pattern the CFA has.

[0046] The image processing device 100 may be, but is not limited to, a computing device mounted on a chip separate from a chip on which an image sensing device is mounted. The chip on which the image sensing device is mounted and the chip on which the image processing device 100 is mounted may communicate with each other via a predetermined interface. According to one embodiment, the chip on which the image sensing device is mounted and the chip on which the image processing device 100 is mounted may be implemented in a single package, for example, a multi-chip package (MCP), but the scope of the present invention is not limited thereto.

[0047] The image processing device 100 may include a check pattern determining unit 110, a diagonal pattern determining unit 120, and / or a target pixel correcting unit .

[0048] The check pattern determination unit 110 can determine whether a texture in a kernel including a target pixel has a directionality of a predetermined check pattern.

[0049] A pixel array of an image sensing device may include defective pixels that prevent a color image from being captured correctly due to process limitations or the inflow of temporary noise. The pixel array may also include phase difference detection pixels that acquire information related to phase difference to achieve an autofocus function. Like defective pixels, phase difference detection pixels may be unable to capture a color image. Images resulting from pixels that prevent a normal color image from being captured, such as defective pixels and phase detection pixels, need to be corrected. In this disclosure, pixels that prevent a normal color image from being captured, such as defective pixels and phase difference detection pixels, are referred to as "target pixels," and a process for correcting the target pixels will be described.

[0050] The kernel may include a target pixel at the center. The kernel may also include a texture, which may refer to a collection of pixels that have similarity. For example, but not limited to, objects with similar colors in an image may be recognized as a texture in the kernel. Referring to Figures 2 and 3, assuming a 5x5 kernel, the shaded areas in (a), (b), (c), and (d) of Figure 2 and (a), (b), (c), and (d) of Figure 3 may represent textures.

[0051] The texture in the kernel may have directionality. Specifically, the texture in the kernel may have directionality corresponding to a horizontal direction, a vertical direction, a diagonal direction (slash direction and backslash direction), etc. Also, the texture in the kernel may have a checkered pattern directionality in which horizontal and vertical textures are mixed. For example, the texture 210 of the first row of the kernel in FIG. 2(a) may have a horizontal directionality and the texture 220 of the first column may have a vertical directionality, so the texture of the kernel in FIG. 2(a) may have a checkered pattern directionality. Also, the texture of the first row 230 of the kernel in FIG. 2(b) may have a horizontal directionality and the texture of the last column 240 may have a vertical directionality, so the texture of the kernel in FIG. 2(b) may have a checkered pattern directionality. Furthermore, the texture 250 of the last row of the kernel in FIG. 2(c) may have horizontal directionality, and the texture 260 of the last column may have vertical directionality, so the texture of the kernel in FIG. 2(c) may have checkered pattern directionality. Furthermore, the texture 270 of the last row of the kernel in FIG. 2(d) may have horizontal directionality, and the texture 280 of the first column may have vertical directionality, so the texture of the kernel in FIG. 2(d) may have checkered pattern directionality. Furthermore, the kernels in FIGS. 3(a), (b), (c), and (d) include vertical and horizontal textures, so they can be said to include textures 310-380 having checkered pattern directionality. However, examples of kernels including textures having checkered pattern directionality are not limited to those described above.

[0052] The checkerboard pattern determination unit 110 can determine whether the texture has a predetermined checkerboard directionality based on the vertical gradient sum of at least some pixels in the kernel, the horizontal gradient sum of at least some pixels, the forward gradient sum of at least some pixels, and the backslash gradient sum of at least some pixels. The gradient can indicate a difference in pixel values, and the gradient sum can be a sum of the difference values ​​of pixel values. More specific details of calculating the gradient sum will be described later.

[0053] The check pattern determination unit 110 can determine whether the texture has a predetermined check pattern directionality based on whether a first direction corresponding to the maximum gradient sum and a second direction corresponding to the minimum gradient sum are perpendicular to each other among the vertical gradient sum, the horizontal gradient sum, the slash gradient sum, and the backslash gradient sum.

[0054] Furthermore, the checker pattern determination unit 110 can determine whether a texture has a predetermined checker pattern directionality based on whether the value obtained by multiplying the minimum gradient sum among the vertical gradient sum, the horizontal gradient sum, the slash gradient sum, and the backslash gradient sum by a first ratio is greater than the remaining gradient sums excluding the minimum gradient sum. More specific details of determining texture directionality based on the vertical gradient sum, the horizontal gradient sum, the slash gradient sum, and the backslash gradient sum will be described later.

[0055] The check pattern determination unit 110 can determine whether the texture has the directionality of a predetermined check pattern based on the first gradient sum for the first row of the kernel, the second gradient sum for the first column of the kernel, the third gradient sum for the last column of the kernel, and the fourth gradient sum for the last row of the kernel. Specifically, the check pattern determination unit 110 can determine whether the texture has the directionality of a predetermined check pattern based on a first sum obtained by summing the first gradient sum, the second gradient sum, a portion of the third gradient sum, and a portion of the fourth gradient sum, a second sum obtained by summing the first gradient sum, the third gradient sum, a portion of the second gradient sum, and a portion of the fourth gradient sum, a third sum obtained by summing the third gradient sum, the fourth gradient sum, a portion of the first gradient sum, and a portion of the second gradient sum, and a fourth sum obtained by summing the second gradient sum, the fourth gradient sum, a portion of the first gradient sum, and a portion of the third gradient sum. For example, the check pattern determination unit 110 can determine whether a texture has a predetermined check pattern directionality based on a first sum obtained by summing a first gradient sum for the first row of the kernel, a second gradient sum for the first column of the kernel, a portion of the third gradient sum for the last column, and a portion of the fourth gradient sum for the last row of the kernel in Fig. 2(a), a second sum obtained by summing the first gradient sum, the third gradient sum, a portion of the second gradient sum, and a portion of the fourth gradient sum of the kernel in Fig. 2(b), a third sum obtained by summing the third gradient sum, the fourth gradient sum, a portion of the first gradient sum, and a portion of the second gradient sum of the kernel in Fig. 2(c), and a fourth sum obtained by summing the second gradient sum, the fourth gradient sum, a portion of the first gradient sum, and a portion of the third gradient sum of the kernel in Fig. 2(d). More specific details regarding the methods for calculating the gradient sums and sums will be described later.

[0056] In addition, the check pattern determination unit 110 can determine whether the texture has the directionality of a predetermined check pattern based on whether the smallest sum among the first to fourth sums described above is smaller than the value obtained by multiplying the average brightness value of the kernel by the second ratio.

[0057] Furthermore, the check pattern determination unit 110 can determine whether the texture has a predetermined check pattern directionality based on whether the value obtained by subtracting the minimum sum of the first to fourth sums from the maximum sum is greater than the value obtained by multiplying the average brightness value of the kernel by a third ratio. The check pattern determination unit 110 can calculate the average value of values ​​obtained by removing an offset value (e.g., pedestal offset value) from the pixel values ​​of green pixels in the kernel as the average brightness value of the kernel. More specific details of determining the texture directionality will be described later.

[0058] The diagonal pattern determination unit 120 can determine whether the texture in the kernel has a diagonal pattern directionality. For example, the diagonal pattern determination unit 120 can determine which direction gradient sum has the smallest gradient sum among the vertical gradient sum of at least some pixels of the kernel, the horizontal gradient sum of at least some pixels, the forward gradient sum of at least some pixels, and the backslash gradient sum of at least some pixels. The diagonal pattern determination unit 120 can also determine whether the gradient sum of an adjacent pixel of a target pixel and a pixel located diagonally from the adjacent pixel is greater than the average brightness value of the kernel multiplied by a fourth ratio. The diagonal pattern determination unit 120 can also determine whether the diagonal gradient sum is greater than the average brightness value of the kernel. Specifically, the diagonal pattern determination unit 120 can determine that the texture does not have a diagonal pattern direction if, among the vertical gradient sum of at least some pixels in the kernel, the horizontal gradient sum of at least some pixels, the slash direction gradient sum of at least some pixels, and the backslash direction gradient sum of at least some pixels, the slash direction gradient sum is the smallest, the fifth slash direction gradient sum between an adjacent pixel of the target pixel and a pixel located in the slash direction of the adjacent pixel is greater than the value obtained by multiplying the average brightness value of the kernel by the fourth ratio, and the slash direction gradient sum is greater than the average brightness value of the kernel.

[0059] In addition, the diagonal pattern determination unit 120 may determine that the texture does not have a diagonal pattern direction if, among the vertical gradient sum of at least some pixels in the kernel, the horizontal gradient sum of at least some pixels, the forward gradient sum of at least some pixels, and the backslash gradient sum of at least some pixels, the backslash gradient sum is the smallest, the sixth backslash gradient sum between an adjacent pixel of the target pixel and a pixel located in the backslash direction of the adjacent pixel is greater than the average brightness value of the kernel multiplied by the fourth ratio, and the backslash gradient sum is greater than the average brightness value of the kernel. Specific details regarding a method for determining whether a texture has a diagonal pattern direction will be described later.

[0060] The target pixel correction unit 130 may correct the target pixel based on the determination result of the checkerboard pattern determination unit or the diagonal pattern determination unit. For example, if the target pixel correction unit 130 determines that the texture has a predetermined checkerboard pattern directionality, the target pixel correction unit 130 may correct the target pixel based on a third direction corresponding to the second smallest gradient sum among the vertical gradient sum of at least some pixels in the kernel, the horizontal gradient sum of at least some pixels, the forward gradient sum of at least some pixels, and the backslash gradient sum of at least some pixels. Specifically, the target pixel correction unit 130 may correct the target pixel using the average value of pixel values ​​of pixels located in the third direction from the target pixel and having the same color as the target pixel.

[0061] Furthermore, if the target pixel correction unit 130 determines that the texture does not have the directionality of a predetermined checkered pattern or the directionality of a diagonal pattern, the target pixel correction unit 130 may correct the target pixel based on a fourth direction corresponding to a smaller gradient sum among the vertical gradient sum of at least some pixels in the kernel and the horizontal gradient sum of at least some pixels in the kernel. Specifically, the target pixel correction unit 130 may correct the target pixel using an average value of pixel values ​​of pixels located in the fourth direction from the target pixel and having the same color as the target pixel.

[0062] If the target pixel correction unit 130 determines that the texture does not have a predetermined checkered pattern direction but has a diagonal pattern direction, it may correct the target pixel using an average pixel value of pixels in the kernel that have the same color as the target pixel. If the target pixel correction unit 130 determines that the texture does not have a predetermined checkered pattern direction but has a diagonal pattern direction, it may correct the target pixel using a median pixel value of pixels in the kernel that have the same color as the target pixel. If the target pixel correction unit 130 determines that the texture does not have a predetermined checkered pattern direction but has a diagonal pattern direction, it may correct the target pixel based on the direction corresponding to the smallest gradient sum among the vertical gradient sum of at least some pixels in the kernel, the horizontal gradient sum of at least some pixels, the forward gradient sum of at least some pixels, and the backslash gradient sum of at least some pixels. More specific details of correcting the target pixel will be described later.

[0063] FIG. 4 is a flowchart illustrating an image processing method according to an exemplary embodiment of the present disclosure. FIG. 5 is a diagram for explaining an image processing method according to an exemplary embodiment of the present disclosure.

[0064] FIG. 4 will be described below with reference to FIG. Referring to FIG. 4, an image processing method (S400) according to an exemplary embodiment of the present disclosure may determine whether the texture in the kernel has the directionality of a predetermined checkerboard pattern. In step S410, the image processing method may calculate a gradient sum for the predetermined checkerboard pattern. Referring to FIG. 5, the predetermined checkerboard pattern may refer to, but is not limited to, checkerboard patterns such as those shown in (a), (b), (c), and (d) of FIG. 5. (k) of FIG. 5 may be a 5×5 kernel in which the center pixel corresponding to the blue pixel position of the Bayer pattern is the target pixel. (a), (b), (c), and (d) of FIG. 5 may also be kernels such as (k). B0 to B7 shown in (a), (b), (c), and (d) of FIG. 5 may represent pixel values ​​of blue pixels. The gradient sum for a given checkerboard pattern may refer to, for example, the first sum obtained by adding together the first gradient sum for the first row corresponding to the texture, the second gradient sum for the first column, a portion of the third gradient sum for the last column, and a portion of the fourth gradient sum for the last row in the kernel of FIG. 5(a). Specifically, the first gradient sum may be the sum of the difference between B0 and B1 and the difference between B1 and B2. The second gradient sum may be the sum of the difference between B0 and B3 and the difference between B3 and B5. The third gradient sum may be the sum of the difference between B2 and B4 and the difference between B4 and B7. The fourth gradient sum may be the sum of the difference between B5 and B6 and the difference between B6 and B7. The first sum may be a sum of the difference between B0 and B1, the difference between B1 and B2, the difference between B0 and B3, the difference between B3 and B5, the difference between B4 and B7, and the difference between B6 and B7. If the first sum is diff_check1, diff_check1 can be expressed as the following [Equation 1].

[0065] [Formula 1] diff_check1=abs(B0-B1)+abs(B1-B2)+abs(B0-B3)+abs(B3-B5)+abs(B4-B7)+abs(B6-B7)

[0066] Furthermore, the gradient sum for a given check pattern may refer to a second sum obtained by summing the first gradient sum for the first row corresponding to the texture, a portion of the second gradient sum for the first column, a third gradient sum for the last column, and a portion of the fourth gradient sum for the last row in the kernel (b) of Figure 5. Specifically, the second sum may be the sum of the difference between B0 and B1, the difference between B1 and B2, the difference between B2 and B4, the difference between B4 and B7, the difference between B3 and B5, and the difference between B5 and B6. If the second sum is diff_check2, diff_check2 can be expressed as follows:

[0067] [Formula 2] diff_check2=abs(B0-B1)+abs(B1-B2)+abs(B2-B4)+abs(B4-B7)+abs(B3-B5)+abs(B5-B6)

[0068] Furthermore, the gradient sum for a given check pattern may refer to a third sum obtained by summing a portion of the first gradient sum for the first row corresponding to the texture, a portion of the second gradient sum for the first column, a third gradient sum for the last column, and a fourth gradient sum for the last row in the kernel (c) of Figure 5. Specifically, the third sum may be a sum of the difference between B2 and B4, the difference between B4 and B7, the difference between B5 and B6, the difference between B6 and B7, the difference between B0 and B1, and the difference between B0 and B3. Let the third sum be diff_check3, which can be expressed as follows:

[0069] [Formula 3] diff_check3=abs(B2-B4)+abs(B4-B7)+abs(B5-B6)+abs(B6-B7)+abs(B0-B1)+abs(B0-B3)

[0070] Furthermore, the gradient sum for a given check pattern may refer to a fourth sum obtained by summing a portion of the first gradient sum for the first row corresponding to the texture, a second gradient sum for the first column, a portion of the third gradient sum for the last column, and a fourth gradient sum for the last row in the kernel (d) of Figure 5. Specifically, the fourth sum may be a sum of the difference between B0 and B3, the difference between B3 and B5, the difference between B5 and B6, the difference between B6 and B7, the difference between B1 and B2, and the difference between B2 and B4. Assuming the fourth sum is diff_check4, diff_check4 can be expressed as follows:

[0071] [Formula 4] diff_check4=abs(B0-B3)+abs(B3-B5)+abs(B5-B6)+abs(B6-B7)+abs(B1-B2)+abs(B2-B4)

[0072] Although the gradient sum has been described as being calculated based on blue pixels, even if the pixel corresponding to the center pixel of the kernel is green or red, the gradient sum can be calculated using the above-described method using pixels having the same color as the color corresponding to the center pixel. However, the method for calculating the gradient sum according to the exemplary embodiment of the present disclosure is not limited to using pixel values ​​of pixels of the same color.

[0073] In step S420, the image processing method may determine whether the direction corresponding to the minimum gradient sum and the direction corresponding to the maximum gradient sum among the vertical gradient sum, horizontal gradient sum, forward gradient sum, and backslash gradient sum are perpendicular (orthogonal). If the vertical gradient sum is the minimum gradient sum and the horizontal gradient sum is the maximum gradient sum, it may be determined that the vertical direction corresponding to the minimum gradient sum and the horizontal direction corresponding to the maximum gradient sum are orthogonal to each other. If the kernel contains a texture with a single directionality rather than a texture with multiple directions, such as a checkerboard pattern, the direction corresponding to the minimum gradient sum and the direction corresponding to the maximum gradient sum are likely to be orthogonal to each other. Therefore, if the direction corresponding to the minimum gradient sum and the direction corresponding to the maximum gradient sum are not orthogonal, this may be because the texture does not have a specific directionality, such as a checkerboard pattern.

[0074] Let the condition that the direction corresponding to the minimum gradient sum and the direction corresponding to the maximum gradient sum are perpendicular be the first condition (cond_not_90degree), the direction corresponding to the minimum gradient sum be dir_1st, the direction corresponding to the maximum gradient sum be dmax, and the gradient sum for each of the four directions be 4-dir gradient sum. Then, the first condition (cond_not_90degree) can be expressed as follows [Equation 5].

[0075] [Formula 5] cond_not_90degree=!orthogonal(dir_1st, dmax)

[0076] Here, dir_1st=MIN(4-dir gradient sum), dmax=MAX(4-dir gradient sum).

[0077] In this case, the gradient sums in the vertical, horizontal, slash, and backslash directions can be calculated using at least a portion of the pixels in the kernel (e.g., pixels of the same color), and the method for calculating the gradient sums in the four directions is not limited.

[0078] In step S430, the image processing method may compare the minimum value of the gradient sum for a predetermined check pattern with the average brightness value of the kernel. The gradient sum for a predetermined check pattern may refer to the first to fourth sums, as described above. The check pattern corresponding to the minimum sum among the first to fourth sums may be the pattern most similar to the texture pattern of the kernel. For example, if the first sum is the minimum sum, the kernel may include a texture pattern similar to that shown in FIG. 5(a). Furthermore, the smaller the magnitude of the minimum sum, the higher the probability that the kernel has a pattern corresponding to the check pattern corresponding to the minimum sum. Therefore, the image processing method may determine whether the kernel is likely to not have a specific directionality, such as a check pattern, by comparing the minimum sum with the average brightness value of the kernel. Furthermore, the image processing method may determine whether the texture has the directionality of the predetermined check pattern based on whether the minimum sum is smaller than the value obtained by multiplying the average brightness value of the kernel by a second ratio. If the second condition is that the minimum sum is smaller than the average brightness value of the kernel multiplied by the second ratio, the second ratio may be a user parameter for adjusting the second condition. If the second condition is cond_check_min_ratio, the second condition can be expressed as the following [Equation 6].

[0079] [Formula 6] cond_check_min_ratio=(min_check <avg_grn_blc*r_th_checker_min_ratio)

[0080] Here, min_check=MIN(diff_check1 to 4), avg_grn_blc is the average brightness value of the kernel, and r_th_check_min_ratio may mean the second ratio.

[0081] The average brightness value of the kernel may be calculated based on the average value of the pixel values ​​of the green pixels in the kernel. For example, the average brightness value of the kernel may be the average value of the pixel values ​​of the green pixels after removing the offset value, but the method for calculating the average brightness value of the kernel is not limited to the above.

[0082] In step S440, the image processing method may compare the difference between the maximum and minimum values ​​of the gradient sums for a predetermined check pattern with the average brightness value of the kernel. The gradient sums for the predetermined check pattern may be the first to fourth sums, as described above. Furthermore, the image processing method may determine whether the texture has the directionality of the predetermined check pattern based on whether the difference between the maximum and minimum sums among the first to fourth sums is greater than the average brightness value of the kernel multiplied by the third ratio. The greater the difference between the maximum and minimum sums, the more likely the kernel is to include the texture of the specific check pattern. For example, if the first sum is the minimum sum, the greater the difference between the first and maximum sums is above a predetermined value, the more likely the kernel is to include a texture similar to that of FIG. 5(a). The third condition is defined as the condition that the difference between the maximum summed value and the minimum summed value is greater than the value obtained by multiplying the kernel brightness value by the third ratio, and if the third condition is defined as cond_check_minmax_ratio, the third condition can be expressed as follows [Equation 7].

[0083] [Formula 7] cond_check_minmax_ratio=((max_check-min_check)>avg_grn_blc*r_th_checker_minmax_ratio)

[0084] Here, max_check=MAX(diff_check1-4), min_check=MIN(diff_check1-4), avg_grn_blc is the average brightness value of the kernel, and r_th_check_minmax_ratio may be a third ratio, which may be a user parameter for adjusting the third condition.

[0085] In step S450, the image processing method may determine whether the texture in the kernel has a specific direction. Specifically, the image processing method may determine whether the texture has a strong direction in any one of the vertical, horizontal, slash, and backslash directions. For example, the image processing method may determine whether the texture has a specific direction by determining whether the value obtained by multiplying the minimum gradient sum among the vertical gradient sum, the horizontal gradient sum, the slash gradient sum, and the backslash gradient sum by a first ratio is greater than the remaining gradient sums excluding the minimum gradient sum. If the value obtained by multiplying the minimum gradient sum by a first ratio is greater than the remaining gradient sums excluding the minimum gradient sum, this may mean that the magnitude of the minimum gradient sum is not significantly smaller than the remaining gradient sums. In this case, this may mean that the texture in the kernel does not have a strong direction. For example, if the vertical gradient sum is the minimum gradient sum, and the value obtained by multiplying the vertical gradient sum by the first ratio is greater than the horizontal gradient sum, the forward gradient sum, and the backslash gradient sum, it can be determined that the texture in the kernel does not have strong vertical direction direction. The fourth condition can be defined as the condition that the value obtained by multiplying the minimum gradient sum among the vertical gradient sum, the horizontal gradient sum, the forward gradient sum, and the backslash gradient sum is greater than the remaining gradient sums excluding the minimum gradient sum. If the fourth condition is cond_check_str_dir, it can be expressed as the following [Equation 8].

[0086] [Formula 8] cond_check_str_dir=(r_th_checker_str_dir*dir_1st>other_gradient_sums)

[0087] Here, dir_1st=MIN(4-dir gradient sum), other_gradient_sums is the gradient sum of the three directions excluding dir_1st among the vertical direction gradient sum, the horizontal direction gradient sum, the forward direction gradient sum, and the backslash direction gradient sum, and r_th_check_str_dir may be a first ratio. The first ratio may be a user parameter for adjusting the fourth condition.

[0088] In step S460, the image processing method may determine whether the texture in the kernel has the directionality of a checkerboard pattern. Specifically, the image processing method may determine whether the texture in the kernel has the directionality of a predetermined checkerboard pattern depending on whether the first to fourth conditions are satisfied. For example, the image processing method may determine that the texture in the kernel has the directionality of a predetermined checkerboard pattern if all of the first to fourth conditions are satisfied. Specifically, the image processing method determines that the kernel does not have a specific direction among the four directions (e.g., determines that the kernel has a direction similar to a predetermined checkerboard pattern) if the first direction corresponding to the maximum gradient sum and the second direction corresponding to the minimum gradient sum among the four directions are perpendicular (if the first condition is satisfied). If the minimum sum among the first to fourth sums is smaller than the average brightness value of the kernel multiplied by the second ratio (if the second condition is satisfied), the kernel is determined to have a high probability of including the texture of the checkerboard pattern corresponding to the minimum sum. If the difference between the maximum and minimum sums among the first to fourth sums is greater than the average brightness value of the kernel multiplied by the third ratio (if the third condition is satisfied), the kernel is determined to have a high probability of including the texture of the predetermined checkerboard pattern. If the minimum gradient sum among the four directions multiplied by the first ratio is greater than the remaining gradient sums excluding the minimum gradient sum (if the fourth condition is satisfied), the kernel is determined to have no specific direction, and thus can conclude that the texture in the kernel has a specific checkerboard pattern direction.

[0089] If the condition that the texture in the kernel has a directionality of a predetermined check pattern is homo_checker, homo_checker can be expressed as the following [Equation 9].

[0090] [Formula 9] homo_checker=cond_not_90degree & cond_check_min_ratio & cond_check_minmax_ratio & cond_check_str_dir

[0091] Although a method for determining that a texture in a kernel has the directionality of a predetermined check pattern when all of the first to fourth conditions are satisfied has been described, the image processing method according to an exemplary embodiment of the present disclosure is not limited to this, and may determine that a texture has the directionality of a predetermined check pattern when at least some of the first to fourth conditions are satisfied.

[0092] FIG. 6 is a flowchart illustrating an image processing method according to an exemplary embodiment of the present disclosure. FIG. 7 is a diagram illustrating an image processing method according to an exemplary embodiment of the present disclosure. FIG. 8 is a diagram for explaining an image processing method according to an exemplary embodiment of the present disclosure.

[0093] FIG. 6 will be described below with reference to FIGS. 6, an image processing method (S600) according to an exemplary embodiment of the present disclosure may determine whether a texture in a kernel has a diagonal pattern directionality. Specifically, in step S610, the image processing method may determine whether a diagonal gradient sum is the smallest gradient sum among four gradient sums. In other words, the image processing method may determine whether a slash gradient sum or a backslash gradient sum is the smallest gradient sum among a vertical gradient sum, a horizontal gradient sum, a slash gradient sum, and a backslash gradient sum.

[0094] In step S620, if the gradient sum in the diagonal direction is the minimum gradient sum, the image processing method may compare the minimum gradient sum with the average brightness value of the kernel. Specifically, if the gradient sum in the forward direction is the minimum gradient sum, the image processing method may determine whether the gradient sum in the forward direction is greater than the average brightness value of the kernel. Also, if the gradient sum in the backslash direction is the minimum gradient sum, the image processing method may determine whether the gradient sum in the backslash direction is greater than the average brightness value of the kernel.

[0095] In step S630, the image processing method may compare the diagonal gradient sum between a neighboring pixel of the target pixel and a pixel located diagonally from the neighboring pixel with the average brightness value of the kernel. Specifically, the image processing method may determine whether a fifth gradient sum in the slash direction between a neighboring pixel of the target pixel and a pixel located diagonally from the neighboring pixel is greater than the average brightness value of the kernel multiplied by a fourth ratio. Referring to FIG. 7, (k) of FIG. 7 may be a 5×5 kernel in which the center pixel corresponding to a blue pixel is the target pixel. In this case, the neighboring pixels of the target pixel may be green pixels corresponding to G12, G21, G23, and G32. Also, (a) of FIG. 7 shows the neighboring pixels of the target pixel and the pixels located diagonally from the neighboring pixels. Specifically, the neighboring pixels of the target pixel and the pixels located diagonally from the neighboring pixels may be green pixels corresponding to G03, G12, G21, G30, G14, G23, G32, and G41. The fifth gradient sum in the slash direction between an adjacent pixel of the target pixel and a pixel located in the slash direction of the adjacent pixel may be the sum of the difference between G03 and G12, the difference between G12 and G21, the difference between G21 and G30, the difference between G14 and G23, the difference between G23 and G32, and the difference between G32 and G41. If the fifth gradient sum is diff_no_dia1, then the fifth gradient sum can be expressed as follows:

[0096] [Formula 10] diff_no_dia1=abs(G03-G12)+abs(G12-G21)+abs(G21-G30)+abs(G14-G23)+abs(G23-G32)+abs(G32-G41)

[0097] Referring to FIG. 8 , (k) of FIG. 8 may be a 5×5 kernel in which the central pixel corresponding to a green pixel is the target pixel. Neighboring pixels of the target pixel may be red and blue pixels corresponding to R12, B21, B23, and R32. (a) of FIG. 8 also shows neighboring pixels of the target pixel and pixels located in the slash direction of the neighboring pixels. Specifically, the neighboring pixels of the target pixel and pixels located in the slash direction of the neighboring pixels may be red and blue pixels corresponding to B03, R12, B21, R30, R14, B23, R32, and B41. A fifth gradient sum in the slash direction between the neighboring pixel of the target pixel and a pixel located in the slash direction of the neighboring pixel may be a value calculated based on the difference between B03 and B21, the difference between B23 and B41, the difference between R12 and R30, and the difference between R14 and R32. Specifically, the fifth gradient sum may be calculated by multiplying the gradient for a blue pixel by a blue gain, multiplying the gradient for a red pixel by a red gain, adding the gradient sum for the blue pixel multiplied by the blue gain and the gradient sum for the red pixel multiplied by the red gain, and multiplying the result by a predetermined ratio. The blue gain may represent the average value of pixel values ​​of all blue pixels relative to the average value of pixel values ​​of all green pixels in the previous frame. The red gain may represent the average value of pixel values ​​of all red pixels relative to the average value of pixel values ​​of all green pixels in the previous frame. The predetermined ratio may be a value for ensuring that the number of gradients used to calculate the fifth gradient sum when the central pixel corresponds to a blue pixel or a red pixel is the same as the number of gradients used to calculate the fifth gradient sum when the central pixel corresponds to a green pixel. For example, for a 5x5 kernel, as described above, the total number of gradients used to calculate the fifth gradient sum when the central pixel corresponds to a red pixel or a blue pixel is six, and the total number of gradients used to calculate the fifth gradient sum when the central pixel corresponds to a green pixel is four, so the predetermined ratio may be, but is not limited to, 3 / 2.If the fifth gradient sum is diff_no_dia1, the predetermined ratio is 3 / 2, the blue gain is B_gain, and the red gain is R_gain, the fifth gradient sum can be expressed as the following [Equation 11].

[0098] [Formula 11] diff_no_dia1=[{abs(B03-B21)+abs(B23-B41)}*B_gain+{abs(R12-R30)+abs(R14-R32)}*R_gain]*3 / 2

[0099] The image processing method may determine whether a sixth gradient sum in a backslash direction between a neighboring pixel of a target pixel and a pixel located in a backslash direction of the neighboring pixel is greater than a value obtained by multiplying the average brightness value of the kernel by a fourth ratio. Referring to FIG. 7, as described above, the neighboring pixels of the target pixel may be green pixels corresponding to G12, G21, G23, and G32. Also, FIG. 7(b) shows the neighboring pixels of the target pixel and the pixels located in the backslash direction of the neighboring pixels. Specifically, the neighboring pixels of the target pixel and the pixels located in the backslash direction of the neighboring pixels may be green pixels corresponding to G01, G12, G23, G34, G10, G21, G32, and G43. The sixth gradient sum in the backslash direction between an adjacent pixel of the target pixel and a pixel located in the backslash direction of the adjacent pixel may be the sum of the difference between G01 and G12, the difference between G12 and G23, the difference between G23 and G34, the difference between G10 and G21, the difference between G21 and G32, and the difference between G32 and G43. If the sixth gradient sum is diff_no_dia2, then the sixth gradient sum can be expressed as follows:

[0100] [Formula 12] diff_no_dia2=abs(G01-G12)+abs(G12-G23)+abs(G23-G34)+abs(G10-G21)+abs(G21-G32)+abs(G32-G43)

[0101] Referring to FIG. 8 , neighboring pixels of a target pixel may be red and blue pixels corresponding to R12, B21, B23, and R32. Also, FIG. 8 (b) shows neighboring pixels of the target pixel and pixels located in the backslash direction of the neighboring pixels. Specifically, the neighboring pixels of the target pixel and pixels located in the backslash direction of the neighboring pixels may be red and blue pixels corresponding to B01, R12, B23, R34, R10, B21, R32, and B43. A sixth gradient sum in the backslash direction between the neighboring pixel of the target pixel and the pixel located in the backslash direction of the neighboring pixel may be a value calculated based on the difference between B01 and B23, the difference between B21 and B43, the difference between R10 and R32, and the difference between R12 and R34. Specifically, the sixth gradient sum can be calculated by multiplying the gradient for the blue pixel by the blue gain, multiplying the gradient for the red pixel by the red gain, and adding the gradient sum for the blue pixel multiplied by the blue gain and the gradient sum for the red pixel multiplied by the red gain, and then multiplying the result by a predetermined ratio. The predetermined ratio may be a value that ensures that the number of gradients used to calculate the sixth gradient sum when the central pixel corresponds to a blue pixel or a red pixel is equal to the number of gradients used to calculate the sixth gradient sum when the central pixel corresponds to a green pixel. For example, for a 5×5 kernel, as described above, the number of gradients used to calculate the sixth gradient sum when the central pixel corresponds to a red pixel or a blue pixel is six in total, and the number of gradients used to calculate the sixth gradient sum when the central pixel corresponds to a green pixel is four in total. Therefore, the predetermined ratio may be, but is not limited to, 3 / 2. If the sixth gradient sum is diff_no_dia2, the predetermined ratio is 3 / 2, the blue gain is B_gain, and the red gain is R_gain, the sixth gradient sum can be expressed as follows:

[0102] [Formula 13] diff_no_dia2=[{abs(B01-B23)+abs(B21-B43)}*B_gain+{abs(R10-R32)+abs(R12-R34)}*R_gain]*3 / 2

[0103] The image processing method may determine whether the fifth gradient sum or the sixth gradient sum is greater than the average brightness value of the kernel multiplied by the fourth ratio.

[0104] In step S640, the image processing method may determine whether the texture in the kernel has a diagonal pattern directionality. Specifically, if the fifth condition is defined as: among the vertical gradient sum, the horizontal gradient sum, the slash gradient sum, and the backslash gradient sum, the slash gradient sum is the smallest; the fifth gradient sum is greater than the average brightness value of the kernel multiplied by the fourth ratio; and the slash gradient sum is greater than the average brightness value of the kernel, the image processing method may determine that the texture does not have a diagonal pattern directionality if the fifth condition is satisfied. If the fifth condition is defined as cond_no_dia1, the fifth condition can be expressed as follows:

[0105] [Formula 14] cond_no_dia1=(dmin_dir==d1_sum)&(diff_no_dia1>(avg_grn_blc<<1))&(d1_sum>avg_grn_blc)

[0106] Here, dmin_dir=MIN(4-dir gradient sum), d1_sum is the gradient sum in the slash direction, and avg_grn_blc is the average brightness value of the kernel. In this case, the fourth ratio is assumed to be 2, but is not limited to this.

[0107] Furthermore, if the sixth condition is that among the vertical gradient sum, horizontal gradient sum, forward gradient sum, and backslash gradient sum, the backslash gradient sum is the smallest, the sixth gradient sum is greater than the average brightness value of the kernel multiplied by the fourth ratio, and the backslash gradient sum is greater than the average brightness value of the kernel, the image processing method can determine that the texture does not have a diagonal pattern directionality if the sixth condition is satisfied. If the sixth condition is cond_no_dia2, the sixth condition can be expressed as follows:

[0108] [Formula 15] cond_no_dia2=(dmin_dir==d2_sum)&(diff_no_dia2>(avg_grn_blc<<1))&(d2_sum>avg_grn_blc)

[0109] Here, dmin_dir=MIN(4-dir gradient sum), d2_sum is the gradient sum in the backslash direction, and avg_grn_blc is the average brightness value of the kernel. In this case, the fourth ratio is assumed to be 2, but is not limited to this.

[0110] The image processing method can determine that the texture does not have a diagonal pattern directionality if either the fifth or sixth condition is satisfied. If the texture does not have a diagonal pattern directionality as cond_no_dia, then cond_no_dia can be expressed as the following [Equation 16].

[0111] [Formula 16] cond_no_dia=(cond_no_dia1 | cond_no_dia2)

[0112] FIG. 9 is a flowchart illustrating an image processing method according to an exemplary embodiment of the present disclosure. Referring to FIG. 9, the image processing method may determine in step S910 whether a texture in a kernel including a target pixel has a direction of a predetermined check pattern. Step S910 may correspond to the image processing method (S400) of FIG. 4. For example, if the first to fourth conditions described above are satisfied, the image processing method may determine that the texture has a direction of a predetermined check pattern. In this case, the predetermined check pattern may be, but is not limited to, any of the check patterns shown in FIG. 2.

[0113] If the image processing method determines that the texture has a predetermined checkered pattern, it may correct the target pixel based on the direction corresponding to the second smallest gradient sum among the four directional gradient sums in step S920. Specifically, if the direction corresponding to the second smallest gradient sum among the vertical, horizontal, slash, and backslash gradient sums is defined as the third direction, the target pixel may be corrected using the average value of pixel values ​​of pixels located in the third direction relative to the target pixel and having the same color as the target pixel. For example, if the second smallest gradient sum among the vertical, horizontal, slash, and backslash gradient sums is the vertical direction, the third direction is the vertical direction, and the image processing method may correct the target pixel using the average value of pixel values ​​of pixels located in the vertical direction relative to the target pixel and having the same color as the target pixel.

[0114] If the image processing method determines that the texture does not have the directionality of a predetermined checkered pattern, it may determine in step S930 whether the texture has the directionality of a diagonal pattern. Step S930 may correspond to the image processing method (S600) of FIG. 6. Specifically, the image processing method may determine that the texture does not have the directionality of a diagonal pattern if at least one of the fifth and sixth conditions described above is satisfied. In other words, if at least one of the fifth and sixth conditions described above is satisfied, the image processing method may determine that the texture does not have the directionality of a diagonal pattern, but has the directionality of one of the checkered patterns shown in FIG. 3.

[0115] If the image processing method determines that the texture has a diagonal pattern directionality, it may correct the target pixel based on pixels in the kernel that have the same color as the target pixel in step S940. In other words, if the fifth and sixth conditions are not both satisfied, the image processing method may determine that the texture has a diagonal pattern directionality, and may correct the target pixel based on pixels in the kernel that have the same color as the target pixel. For example, the image processing method may correct the target pixel using an average pixel value of pixels in the kernel that have the same color as the target pixel. Alternatively, the image processing method may correct the target pixel using a median pixel value of pixels in the kernel that have the same color as the target pixel. Alternatively, the image processing method may correct the target pixel based on a direction corresponding to the smallest gradient sum among the vertical gradient sum, horizontal gradient sum, forward gradient sum, and backslash gradient sum. For example, the target pixel may be corrected using an average pixel value of pixels in the kernel that have the same color as the target pixel and are located in a direction corresponding to the smallest gradient sum relative to the target pixel. The method for correcting the target pixel when it is determined that the texture has a diagonal pattern directionality is not limited to the above-described method.

[0116] If the image processing method determines that the texture does not have a diagonal pattern directionality, it may correct the target pixel based on the direction corresponding to the smaller gradient sum among the vertical gradient sum and the horizontal gradient sum in step S950. For example, if the direction corresponding to the smaller gradient sum among the vertical gradient sum and the horizontal gradient sum is defined as the fourth direction, the image processing method may correct the target pixel using the average value of pixel values ​​of pixels located in the fourth direction from the target pixel and having the same color as the target pixel.

[0117] FIG. 10 is a block diagram illustrating an example of a computing device that corresponds to the image processing device of FIG. Referring to FIG. 10, a computing device 1000 may represent one embodiment of a hardware configuration for performing the operations of the image processing device 100 of FIG.

[0118] The computing device 1000 may be mounted on a chip separate from the chip on which the image sensing device is mounted. According to one embodiment, the chip on which the image sensing device is mounted and the chip on which the computing device 1000 is mounted may be implemented in a single package, for example, a multi-chip package (MCP), although the scope of the present invention is not limited thereto.

[0119] Computing device 1000 may include a processor 1010 , a memory 1020 , an input / output interface 1030 , and a communication interface 1040 .

[0120] The processor 1010 is capable of processing data and / or instructions necessary to perform the operations of the image processing device 100 configuration described in FIG.

[0121] The memory 1020 can store data and / or instructions necessary to perform the configuration operations of the image processing device 100 and can be accessed by the processor 1010. For example, the memory 1020 can be implemented as volatile memory (e.g., Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), etc.) or non-volatile memory (e.g., Programmable Read Only Memory (PROM), Erasable PROM (EPROM), Electrically Erasable PROM (EEPROM), flash memory, etc.).

[0122] That is, a computer program for performing the operations of the image processing device 100 disclosed in this document is recorded in the memory 1020 and is executed and processed by the processor 1010, thereby realizing the operations of the image processing device 100.

[0123] The input / output interface 1030 may provide an interface that connects an external input device (e.g., a keyboard, a mouse, a touch panel, etc.) and / or an external output device (e.g., a display) to the processor 1010, enabling data to be transmitted and received.

[0124] The communication interface 1040 is configured to be able to send and receive various data to and from external devices (for example, an application processor, an external memory, etc.), and may be a device that can support wired or wireless communication.

[0125] The above description merely exemplifies the technical concept of the present disclosure, and various modifications and variations are possible within the scope of the essential characteristics of the present disclosure, provided that such modifications and variations are made by a person skilled in the art to which the present disclosure pertains. Therefore, the embodiments disclosed in the present disclosure are intended to illustrate, rather than limit, the technical concept of the present disclosure, and such embodiments do not limit the scope of the technical concept of the present disclosure. The scope of protection of the present disclosure should be interpreted by the scope of the claims below, and all technical concepts within the scope equivalent thereto should be interpreted as being within the scope of the present disclosure.

Claims

1. a check pattern determination unit that determines whether a texture in a kernel including a target pixel has a directionality of a predetermined check pattern; a diagonal pattern determination unit that determines whether the texture has a diagonal pattern directionality when it is determined that the texture does not have the directionality of the predetermined check pattern; a target pixel correction unit that corrects the target pixel based on a determination result by the check pattern determination unit or the diagonal pattern determination unit; 12. An image processing device comprising:

2. The check pattern determination unit 2. The image processing device of claim 1, wherein the determination of whether the texture has the directionality of the predetermined checkered pattern is based on a vertical gradient sum of at least some of the pixels in the kernel, a horizontal gradient sum of the at least some of the pixels, a slash gradient sum of the at least some of the pixels, and a backslash gradient sum of the at least some of the pixels.

3. The check pattern determination unit 3. The image processing device of claim 2, wherein the determination of whether the texture has the directionality of the predetermined checkered pattern is based on whether a first direction corresponding to a maximum gradient sum and a second direction corresponding to a minimum gradient sum are perpendicular to each other among the vertical gradient sum, the horizontal gradient sum, the slash gradient sum, and the backslash gradient sum.

4. The check pattern determination unit 3. The image processing device according to claim 2, wherein the determination as to whether the texture has the directionality of the predetermined checkered pattern is based on whether a value obtained by multiplying a minimum gradient sum among the vertical gradient sum, the horizontal gradient sum, the slash gradient sum, and the backslash gradient sum by a first ratio is greater than the remaining gradient sums excluding the minimum gradient sum.

5. The check pattern determination unit 2. The image processing device of claim 1, wherein the image processing device determines whether the texture has the directionality of the predetermined checkered pattern based on a first sum obtained by summing a first gradient sum for a first row of the kernel, a second gradient sum for a first column of the kernel, a portion of a third gradient sum for a last column of the kernel, and a portion of a fourth gradient sum for a last row of the kernel; a second sum obtained by summing the first gradient sum, the third gradient sum, a portion of the second gradient sum, and a portion of the fourth gradient sum; a third sum obtained by summing the third gradient sum, the fourth gradient sum, a portion of the first gradient sum, and a portion of the second gradient sum; and a fourth sum obtained by summing the second gradient sum, the fourth gradient sum, a portion of the first gradient sum, and a portion of the third gradient sum.

6. The check pattern determination unit 6. The image processing device of claim 5, wherein the determination as to whether the texture has the directionality of the specified checker pattern is based on whether the smallest sum value among the first sum value to the fourth sum value is smaller than the value obtained by multiplying the average brightness value of the kernel by a second ratio.

7. The check pattern determination unit The image processing apparatus according to claim 6, wherein the average brightness value of the kernel is calculated by subtracting an offset value from the pixel values ​​of green pixels in the kernel.

8. The check pattern determination unit The image processing apparatus according to claim 5 , wherein the first gradient sum to the fourth gradient sum are calculated using pixels having the same color as the target pixel.

9. The check pattern determination unit 6. The image processing device according to claim 5, wherein the image processing device determines whether the texture has the directionality of the specified check pattern based on whether a value obtained by subtracting the minimum sum value from the maximum sum value of the first to fourth sum values ​​is greater than a value obtained by multiplying the average brightness value of the kernel by a third ratio.

10. The diagonal pattern determination unit among a vertical gradient sum of at least some pixels in the kernel, a horizontal gradient sum of the at least some pixels, a slash gradient sum of the at least some pixels, and a backslash gradient sum of the at least some pixels, the slash gradient sum is the smallest; a fifth gradient sum in the slash direction between an adjacent pixel of the target pixel and a pixel located in the slash direction of the adjacent pixel is greater than a value obtained by multiplying the average brightness value of the kernel by a fourth ratio; The image processing apparatus according to claim 1 , wherein if the gradient sum in the slash direction is greater than the average brightness value of the kernel, it is determined that the texture does not have the directionality of the diagonal pattern.

11. The diagonal pattern determination unit among a vertical gradient sum of at least some pixels in the kernel, a horizontal gradient sum of the at least some pixels, a forward gradient sum of the at least some pixels, and a backslash gradient sum of the at least some pixels, the backslash gradient sum is the smallest; a sixth gradient sum in the backslash direction between an adjacent pixel of the target pixel and a pixel located in the backslash direction of the adjacent pixel is greater than a value obtained by multiplying the average brightness value of the kernel by a fourth ratio; The image processing apparatus according to claim 1 , wherein if the gradient sum in the backslash direction is greater than the average brightness value of the kernel, it is determined that the texture does not have the directionality of the diagonal pattern.

12. The target pixel correction unit 2. The image processing device of claim 1, wherein if it is determined that the texture has the directionality of the predetermined checkered pattern, the target pixel is corrected based on a third direction corresponding to a second smallest gradient sum among a vertical gradient sum of at least some of the pixels in the kernel, a horizontal gradient sum of the at least some of the pixels, a slash gradient sum of the at least some of the pixels, and a backslash gradient sum of the at least some of the pixels.

13. The target pixel correction unit The image processing device according to claim 12 , wherein the target pixel is corrected by an average value of pixel values ​​of pixels located in the third direction with respect to the target pixel and having the same color as the target pixel.

14. The target pixel correction unit 2. The image processing device of claim 1, wherein if it is determined that the texture does not have the directionality of the predetermined checkered pattern and does not have the directionality of the diagonal pattern, the target pixel is corrected based on a fourth direction corresponding to a smaller gradient sum among a vertical gradient sum of at least some of the pixels in the kernel and a horizontal gradient sum of the at least some of the pixels.

15. The target pixel correction unit The image processing device according to claim 14 , wherein the target pixel is corrected by an average value of pixel values ​​of pixels located in the fourth direction with respect to the target pixel and having the same color as the target pixel.

16. The target pixel correction unit 2. The image processing device according to claim 1, wherein when it is determined that the texture does not have the directionality of the predetermined checkered pattern but has the directionality of the diagonal pattern, the target pixel is corrected by an average value of pixel values ​​of pixels in the kernel that have the same color as the target pixel.

17. The target pixel correction unit 2. The image processing device according to claim 1, wherein when it is determined that the texture does not have the directionality of the predetermined checkered pattern but has the directionality of the diagonal pattern, the target pixel is corrected with a median value of pixel values ​​of pixels in the kernel that have the same color as the target pixel.

18. The target pixel correction unit 2. The image processing device of claim 1, wherein when it is determined that the texture does not have the directionality of the predetermined checkered pattern but has the directionality of the diagonal pattern, the target pixel is corrected based on a direction corresponding to a minimum gradient sum among a vertical gradient sum of at least some of the pixels in the kernel, a horizontal gradient sum of the at least some of the pixels, a slash gradient sum of the at least some of the pixels, and a backslash gradient sum of the at least some of the pixels.

19. a check pattern determination unit that determines whether a texture in a kernel has a directionality of a predetermined check pattern based on a gradient sum of at least some pixels in the kernel including the target pixel; a target pixel correction unit that corrects the target pixel based on a direction corresponding to a second smallest gradient sum among a vertical gradient sum, a horizontal gradient sum, a slash gradient sum, and a backslash gradient sum of the at least some pixels when it is determined that the texture has the directionality of the predetermined checkered pattern; 12. An image processing device comprising:

20. determining whether the texture in the kernel has a checkered pattern directionality based on gradient sums for the first row, last row, first column, and last column of the kernel including the target pixel; correcting the target pixel based on the determination result; An image processing method comprising:

Images