Image processing device, method, display chip, computer device and storage medium

Abstract

The application discloses an image processing device, method, display chip, computer equipment and storage medium, and belongs to the field of image processing. The device comprises: a green channel interpolation determination module configured to determine the green channel interpolation of a target pixel in a Bayer image in a horizontal direction and in a vertical direction; a comprehensive weight determination module configured to determine the comprehensive weight of the target pixel in the horizontal direction and in the vertical direction; a green channel value determination module configured to determine the green channel value of the target pixel; and a non-green channel value determination module configured to determine the value of a second channel of the target pixel based on the green channel value of the target pixel. The application determines the green channel value of the target pixel through the comprehensive weight of the target pixel in the horizontal direction and in the vertical direction, jointly judges the interpolation direction by using the color difference and the color difference gradient, compensates for the deficiency of the color difference gradient in the high-frequency region by the color difference, and thus reduces the pseudo-color spots and zipper-shaped jagged edges in the generated image.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to an image processing apparatus, method, display chip, computer device, and storage medium. Background Technology

[0002] In the field of image processing, to acquire color images using a single sensor, a Bayer filter is typically placed in front of the sensor, so that each pixel can only record one of the three colors: red, green, and blue. The resulting image is called a Bayer image. In a Bayer image, to obtain a complete color image, it is necessary to calculate and fill in the missing two color values ​​for each pixel. This process is called demosaicing.

[0003] The current mainstream de-mosaic methods rely on the assumption of constant local color difference and color gradients. Specifically, the assumption determines the color difference of each pixel in the horizontal and vertical directions, and the color gradients in the horizontal and vertical directions are then determined. The interpolation direction is then determined based on these color gradients to restore the green channel. Finally, the red or blue channel is restored based on the assumption of constant local color difference. However, in high-frequency regions, the assumption of constant local color difference fails because the color difference fluctuates drastically between adjacent pixels. Therefore, the color gradient cannot accurately reflect the true structural changes of the image, and the determination of the interpolation direction in high-frequency regions is insufficient. This results in obvious false-color spots and jagged edges in the generated image. Summary of the Invention

[0004] This application provides an image processing apparatus, method, display chip, computer device, and storage medium that can utilize color difference and color difference gradient to jointly determine the interpolation direction, using color difference to compensate for the deficiency of color difference gradient in the high-frequency region, thereby reducing false color spots and zipper-like jagged edges in the generated image. The technical solution is as follows: In a first aspect, an image processing apparatus is provided, the apparatus comprising: The green channel interpolation determination module is configured to determine the green channel interpolation of a target pixel in a Bayer image in the horizontal direction and the green channel interpolation in the vertical direction, wherein the Bayer image includes the value of a first channel of the target pixel, and the first channel is either the blue channel or the red channel. The comprehensive weight determination module is configured to determine the comprehensive weight of the target pixel in the horizontal direction and the comprehensive weight in the vertical direction, wherein the comprehensive weight indicates the comprehensive situation of the color difference and color difference gradient of the target pixel in the corresponding direction; The green channel value determination module is configured to determine the green channel value of the target pixel based on the green channel interpolation in the horizontal direction and the green channel interpolation in the vertical direction, the comprehensive weight of the target pixel in the horizontal direction and the comprehensive weight in the vertical direction. The non-green channel value determination module is configured to determine the value of the second channel of the target pixel based on the green channel value of the target pixel; wherein, when the first channel is the blue channel, the second channel is the red channel, and when the first channel is the red channel, the second channel is the blue channel.

[0005] In one possible implementation, the comprehensive weight determination module is configured as follows: The overall color difference weights and overall color difference gradient weights corresponding to the pixels in the target window in the horizontal and vertical directions are determined respectively. The target window refers to the window centered on the target pixel in the Bayer image. The overall color difference weights indicate the degree of overall color deviation of the pixels in the target window in the corresponding direction, and the overall color difference gradient weights indicate the degree of overall change of color difference of the pixels in the target window in the corresponding direction. Determine the color difference weight ratio, which indicates the overall color difference and the degree of asymmetry in the overall color difference gradient of the pixels in the target window in the horizontal and vertical directions; Based on the overall color difference weight and overall color difference gradient weight in the horizontal direction, as well as the proportion of color difference weight, the comprehensive weight of the target pixel in the horizontal direction is determined. Based on the overall color difference weight and overall color difference gradient weight in the vertical direction, as well as the proportion of color difference weight, the comprehensive weight of the target pixel in the vertical direction is determined.

[0006] In one possible implementation, the comprehensive weight determination module is configured as follows: Based on the color difference of multiple first pixels in the target window in the horizontal direction, the color difference intensity of the target window in the horizontal direction is determined, wherein the multiple first pixels have blue channel values ​​or red channel values ​​in the Bayer image; Based on the color difference of the plurality of first pixels in the vertical direction, the color difference intensity of the target window in the vertical direction is determined; The overall color difference weight in the horizontal direction is determined based on the color difference intensity in the horizontal direction, and the overall color difference weight in the vertical direction is determined based on the color difference intensity in the vertical direction.

[0007] In one possible implementation, the comprehensive weight determination module is configured as follows: Based on the color difference gradient of multiple second pixels in the target window in the horizontal direction, the color difference gradient intensity of the target window in the horizontal direction is determined, and the multiple second pixels have green channel values ​​in the Bayer image; Based on the color difference gradient of the plurality of second pixels in the vertical direction, the color difference gradient intensity of the target window in the vertical direction is determined; The overall color difference gradient weight in the horizontal direction is determined based on the color difference gradient intensity in the horizontal direction, and the overall color difference gradient weight in the vertical direction is determined based on the color difference gradient intensity in the vertical direction.

[0008] In one possible implementation, the comprehensive weight determination module is configured as follows: A first weight is determined based on the difference between the color difference intensity in the horizontal direction and the color difference intensity in the vertical direction; The second weight is determined based on the ratio between the color difference intensity in the horizontal direction and the color difference intensity in the vertical direction; The third weight is determined based on the difference between the color difference gradient intensity in the horizontal direction and the color difference gradient intensity in the vertical direction. The fourth weight is determined based on the ratio between the color difference gradient intensity in the horizontal direction and the color difference gradient intensity in the vertical direction. The color difference weight ratio is determined based on the first weight, the second weight, the third weight, and the fourth weight.

[0009] In one possible implementation, the green channel interpolation determination module is configured as follows: Determine the initial interpolation values ​​of the green channel in the horizontal direction and the initial interpolation values ​​of the green channel in the vertical direction for the target pixel; Determine the color gradient of the target pixel in the first channel in the horizontal direction and the color gradient of the first channel in the vertical direction; Based on the color gradient of the first channel of the target pixel in the horizontal direction, the initial interpolation of the green channel of the target pixel in the horizontal direction is corrected to obtain the green channel interpolation of the target pixel in the horizontal direction. Based on the color gradient of the first channel of the target pixel in the vertical direction, the initial interpolation of the green channel of the target pixel in the vertical direction is corrected to obtain the green channel interpolation of the target pixel in the vertical direction.

[0010] In one possible implementation, the green channel interpolation determination module is configured as follows: Determine the adaptive weight of the target pixel, wherein the adaptive weight indicates the trend of change between the color gradient of the green channel of the neighboring pixels of the target pixel and the color gradient of the first channel of the neighboring pixels; The green channel interpolation of the target pixel in the horizontal direction is determined based on the initial interpolation of the green channel of the target pixel in the horizontal direction, the color gradient of the first channel of the target pixel in the horizontal direction, and the adaptive weight.

[0011] Secondly, an image processing method is provided, the method comprising: Determine the green channel interpolation of the target pixel in the horizontal direction and the green channel interpolation in the vertical direction in the Bayer image, wherein the Bayer image includes the value of the first channel of the target pixel, and the first channel is either the blue channel or the red channel; Determine the overall weight of the target pixel in the horizontal direction and the overall weight in the vertical direction, wherein the overall weight indicates the overall situation of the color difference and color difference gradient of the target pixel in the corresponding direction; The green channel value of the target pixel is determined based on the green channel interpolation in the horizontal direction and the green channel interpolation in the vertical direction, the comprehensive weight of the target pixel in the horizontal direction and the comprehensive weight in the vertical direction. Based on the green channel value of the target pixel, the value of the second channel of the target pixel is determined; wherein, if the first channel is the blue channel, the second channel is the red channel, and if the first channel is the red channel, the second channel is the blue channel.

[0012] In one possible implementation, determining the combined weight of the target pixel in the horizontal direction and the combined weight in the vertical direction includes: The overall color difference weights and overall color difference gradient weights corresponding to the pixels in the target window in the horizontal and vertical directions are determined respectively. The target window refers to the window centered on the target pixel in the Bayer image. The overall color difference weights indicate the degree of overall color deviation of the pixels in the target window in the corresponding direction, and the overall color difference gradient weights indicate the degree of overall change of color difference of the pixels in the target window in the corresponding direction. Determine the color difference weight ratio, which indicates the overall color difference and the degree of asymmetry in the overall color difference gradient of the pixels in the target window in the horizontal and vertical directions; Based on the overall color difference weight and overall color difference gradient weight in the horizontal direction, as well as the proportion of color difference weight, the comprehensive weight of the target pixel in the horizontal direction is determined. Based on the overall color difference weight and overall color difference gradient weight in the vertical direction, as well as the proportion of color difference weight, the comprehensive weight of the target pixel in the vertical direction is determined.

[0013] In one possible implementation, determining the overall color difference weights of pixels within the target window in the horizontal and vertical directions includes: Based on the color difference of multiple first pixels in the target window in the horizontal direction, the color difference intensity of the target window in the horizontal direction is determined, wherein the multiple first pixels have blue channel values ​​or red channel values ​​in the Bayer image; Based on the color difference of the plurality of first pixels in the vertical direction, the color difference intensity of the target window in the vertical direction is determined; The overall color difference weight in the horizontal direction is determined based on the color difference intensity in the horizontal direction, and the overall color difference weight in the vertical direction is determined based on the color difference intensity in the vertical direction.

[0014] In one possible implementation, determining the overall color difference gradient weights of pixels within the target window in the horizontal and vertical directions includes: Based on the color difference gradient of multiple second pixels in the target window in the horizontal direction, the color difference gradient intensity of the target window in the horizontal direction is determined, and the multiple second pixels have green channel values ​​in the Bayer image; Based on the color difference gradient of the plurality of second pixels in the vertical direction, the color difference gradient intensity of the target window in the vertical direction is determined; The overall color difference gradient weight in the horizontal direction is determined based on the color difference gradient intensity in the horizontal direction, and the overall color difference gradient weight in the vertical direction is determined based on the color difference gradient intensity in the vertical direction.

[0015] In one possible implementation, determining the color difference weighting percentage includes: A first weight is determined based on the difference between the color difference intensity in the horizontal direction and the color difference intensity in the vertical direction; The second weight is determined based on the ratio between the color difference intensity in the horizontal direction and the color difference intensity in the vertical direction; The third weight is determined based on the difference between the color difference gradient intensity in the horizontal direction and the color difference gradient intensity in the vertical direction. The fourth weight is determined based on the ratio between the color difference gradient intensity in the horizontal direction and the color difference gradient intensity in the vertical direction. The color difference weight ratio is determined based on the first weight, the second weight, the third weight, and the fourth weight.

[0016] In one possible implementation, determining the green channel interpolation of the target pixel in the Bayer image in the horizontal direction and the green channel interpolation in the vertical direction includes: Determine the initial interpolation values ​​of the green channel in the horizontal direction and the initial interpolation values ​​of the green channel in the vertical direction for the target pixel; Determine the color gradient of the target pixel in the first channel in the horizontal direction and the color gradient of the first channel in the vertical direction; Based on the color gradient of the first channel of the target pixel in the horizontal direction, the initial interpolation of the green channel of the target pixel in the horizontal direction is corrected to obtain the green channel interpolation of the target pixel in the horizontal direction. Based on the color gradient of the first channel of the target pixel in the vertical direction, the initial interpolation of the green channel of the target pixel in the vertical direction is corrected to obtain the green channel interpolation of the target pixel in the vertical direction.

[0017] In one possible implementation, correcting the initial interpolation of the green channel of the target pixel in the horizontal direction based on the color gradient of the first channel of the target pixel in the horizontal direction to obtain the interpolated green channel of the target pixel in the horizontal direction includes: Determine the adaptive weight of the target pixel, wherein the adaptive weight indicates the trend of change between the color gradient of the green channel of the neighboring pixels of the target pixel and the color gradient of the first channel of the neighboring pixels; The green channel interpolation of the target pixel in the horizontal direction is determined based on the initial interpolation of the green channel of the target pixel in the horizontal direction, the color gradient of the first channel of the target pixel in the horizontal direction, and the adaptive weight.

[0018] Thirdly, a display chip is provided, the display chip including the image processing apparatus provided in the first aspect.

[0019] Fourthly, an electronic rearview mirror is provided, the electronic rearview mirror including the display chip described in the third aspect.

[0020] Fifthly, a computer device is provided, the computer device including a memory and a processor, the memory for storing computer programs, and the processor for executing the computer programs stored in the memory to implement the image processing method provided in the second aspect.

[0021] In a sixth aspect, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a processor, the computer program implements the image processing method provided in the second aspect.

[0022] The technical solution provided in this application can bring at least the following beneficial effects: In this embodiment, the comprehensive weights of the target pixel in the horizontal and vertical directions are determined. These comprehensive weights indicate the combined situation of the color difference and color difference gradient of the target pixel in the corresponding direction. The green channel value of the target pixel is determined by the comprehensive weights of the target pixel in the horizontal and vertical directions. In this way, the color difference and color difference gradient are used together to determine the interpolation direction, and the color difference compensates for the deficiency of the color difference gradient in the high-frequency region, thereby reducing false color spots and zipper-like jagged edges in the generated image. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of a de-mosaic process provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application; Figure 3 This is a schematic diagram of a Bayer image provided in an embodiment of this application; Figure 4 This is a schematic diagram of a target window provided in an embodiment of this application; Figure 5 This is a comparative schematic diagram of reducing false color provided in an embodiment of this application; Figure 6 This is a schematic diagram illustrating the determination of overall color difference weighting according to an embodiment of this application; Figure 7 This is a schematic diagram illustrating how to determine the overall color difference gradient weights according to an embodiment of this application; Figure 8 This is a schematic diagram illustrating the determination of a first weight provided in an embodiment of this application; Figure 9 This is a schematic diagram illustrating the determination of a second weight provided in an embodiment of this application; Figure 10 This is a schematic diagram illustrating the determination of a third weight provided in an embodiment of this application; Figure 11 This is a schematic diagram illustrating the determination of a fourth weight provided in an embodiment of this application; Figure 12 This is a comparative schematic diagram of reducing false color provided in an embodiment of this application; Figure 13 This is a schematic diagram of an image processing method provided in an embodiment of this application; Figure 14 This is a flowchart of an image processing method provided in an embodiment of this application; Figure 15 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application; Figure 16 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0026] In image processing, a color image consists of multiple pixels, each typically containing three color channels: red (R), green (G), and blue (B). Each color channel has a corresponding value (also called the value of that color channel), and a sensor needs to simultaneously acquire the values ​​of all three color channels to generate a color image. However, sensors can only record the intensity of light, not its color. Therefore, a color filter needs to be superimposed on the sensor to allow only specific wavelengths of light to enter, thus enabling color discrimination. But this means that each photosensitive location of the sensor can only acquire the value of one color channel. Therefore, three sensors are needed simultaneously to acquire the values ​​of each color channel to obtain a single color image. However, this imaging method is expensive to manufacture and bulky, and is rarely used in practice.

[0027] To reduce cost and size, a single sensor is typically used, with a color filter array (CFA) superimposed on top. This allows the filter above each photosensitive location to pass through one of the three color channels (red, green, and blue), while filtering out the other two. In other words, each photosensitive location of the sensor only acquires the value of one of the three color channels: red, green, and blue. Currently, the most widely used CFA is the Bayer filter. It consists of multiple filter units, each using a fixed color arrangement. Each filter unit includes one red filter, one blue filter, and two green filters, meaning the red and blue filters each account for 25%, and the green filters account for 50%.

[0028] Since each photosensitive location only acquires the value of one color channel, each pixel in the acquired Bayer image has only the value of one color channel. If the Bayer image is directly output, a noticeable mosaic effect will occur. Therefore, the Bayer image needs to undergo upsampling interpolation to recover the values ​​of the other two missing color channels for each pixel, thus merging them into a complete color RGB image. This process of generating a color RGB image from a Bayer image is called demosaicing. Specifically, as... Figure 1 As shown, each pixel in the Bayer image ( Figure 1 The small square in the Bayer image shown contains only the value of one color channel. After de-mosaicing, each pixel in the generated color RGB image contains the values ​​of three color channels: red, green, and blue.

[0029] Currently, the mainstream demosaic methods employ the assumption of constant local color difference and color difference gradient. Specifically, bilinear interpolation is first performed in the horizontal and vertical directions. Then, based on the assumption of constant local color difference, the color difference of each pixel in the horizontal and vertical directions is determined. The color difference gradient in the horizontal and vertical directions is determined based on the color difference in the two directions. The interpolation direction is then determined based on the color difference gradient in the two directions to restore the green channel. Finally, based on the assumption of constant local color difference, the red or blue channel is restored.

[0030] However, in the high-frequency region, the assumption of constant local color difference fails because the color difference fluctuates drastically between adjacent pixels. As a result, the color difference gradient is difficult to accurately reflect the real structural changes of the image. The judgment of the interpolation direction in the high-frequency region is insufficient, which leads to obvious false color spots and zipper-like jagged edges in the generated image.

[0031] Based on this, embodiments of this application can determine the comprehensive weight of the target pixel in the horizontal direction and the comprehensive weight in the vertical direction. This comprehensive weight indicates the comprehensive situation of the color difference and color difference gradient of the target pixel in the corresponding direction. Then, the green channel value of the target pixel is determined by the comprehensive weight of the target pixel in the horizontal direction and the comprehensive weight in the vertical direction. In this way, it is equivalent to using color difference and color difference gradient to jointly determine the interpolation direction, using color difference to compensate for the deficiency of color difference gradient in the high-frequency region, thereby reducing false color spots and zipper-like jagged edges in the generated image.

[0032] Figure 2 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application. Figure 2 As shown, the device includes a green channel interpolation determination module 201, a comprehensive weight determination module 202, a green channel value determination module 203, and a non-green channel value determination module 204.

[0033] The green channel interpolation determination module 201 is configured to determine the green channel interpolation in the horizontal direction and the green channel interpolation in the vertical direction of a target pixel in a Bayer image, the Bayer image including the value of a first channel of the target pixel, the first channel being either a blue channel or a red channel. The comprehensive weight determination module 202 is configured to determine the comprehensive weight of the target pixel in the horizontal direction and the comprehensive weight in the vertical direction, the comprehensive weight indicating the comprehensive situation of the color difference and color difference gradient of the target pixel in the corresponding direction. The green channel value determination module 203 is configured to determine the green channel value of the target pixel based on the green channel interpolation in the horizontal direction and the green channel interpolation in the vertical direction, the comprehensive weight in the horizontal direction and the comprehensive weight in the vertical direction. The non-green channel value determination module 204 is configured to determine the value of a second channel of the target pixel based on the green channel value of the target pixel; wherein, if the first channel is a blue channel, the second channel is a red channel, and if the first channel is a red channel, the second channel is a blue channel.

[0034] During the mosaic removal process, as described above Figure 1 As shown, since the Bayer image contains the most pixels with green channel values ​​(i.e., green channel values), and the green channel has richer detail and brightness information, it is usually prioritized for recovery. Since pixels containing red or blue channel values ​​do not inherently contain green channel values, these pixels are typically used as target pixels. The missing green channel values ​​are then determined for these pixels, providing a reliable reference for subsequently recovering the other channel value (blue or red channel value). For example, assuming a pixel containing red channel values ​​is used as the target pixel, the missing green channel value is determined for the target pixel, and then the missing blue channel value is determined.

[0035] In some embodiments, to ensure spatial correlation between local pixels and reduce computational load, a sliding window approach is typically used to scan the Bayer image. Demosaic processing is performed on the center pixel within a defined window, where the window size is smaller than the Bayer image size, and the center pixel within the window is used as the target pixel. For example, such as... Figure 3 As shown, the window size is x y and x represent the number of pixels contained in the horizontal and vertical directions, respectively. Assuming a window size of 10×9, with two center pixels (5th row, 5th column and 5th row, 6th column), and using a vertical sliding step of 1 and a horizontal sliding step of 2 as an example, the window is slid. In the diagram, R represents that the pixel contains red channel values, B represents that the pixel contains blue channel values, Gr represents that the pixel contains green channel values ​​and its left and right adjacent pixels contain red channel values, Gb represents that the pixel contains green channel values ​​and its left and right adjacent pixels contain blue channel values, and R(4,4) represents x and y coordinates of 4 and 4 respectively, corresponding to... Figure 3 The pixel in the fifth row and fifth column, Gr(4,5), represents x and y coordinates of 5 and 4 respectively, which corresponds to... Figure 3 The pixel in the fifth row and sixth column of the window, based on the description above, is the target pixel that contains either a blue channel value or a red channel value. Therefore, R(4,4) can be used as the target pixel within this window.

[0036] In some embodiments, the green channel interpolation determination module 201 is configured to: determine the initial interpolation of the green channel of the target pixel in the horizontal direction and the initial interpolation of the green channel in the vertical direction; determine the color gradient of the first channel of the target pixel in the horizontal direction and the color gradient of the first channel of the target pixel in the vertical direction; based on the color gradient of the first channel of the target pixel in the horizontal direction, correct the initial interpolation of the green channel of the target pixel in the horizontal direction to obtain the green channel interpolation of the target pixel in the horizontal direction; and based on the color gradient of the first channel of the target pixel in the vertical direction, correct the initial interpolation of the green channel of the target pixel in the vertical direction to obtain the green channel interpolation of the target pixel in the vertical direction.

[0037] The horizontal and vertical directions of the target pixel can be defined based on the arrangement of the color filter array and are not fixed. For example, the horizontal direction can be defined as east-west and the vertical direction as north-south. Alternatively, the horizontal and vertical directions can be defined as up-down, left-right, or other relative directions depending on the actual processing requirements. Interpolation refers to the estimation of the missing green channel value of the target pixel, calculated by weighting and averaging the green channel values ​​of the pixels surrounding the target pixel.

[0038] In some embodiments, the green channel values ​​of pixels located in the same row as the target pixel in the horizontal direction within the target window are weighted, summed, and averaged to obtain the initial interpolation of the green channel of the target pixel in the horizontal direction. Similarly, the green channel values ​​of pixels located in the same column as the target pixel in the vertical direction within the target window are weighted, summed, and averaged to obtain the initial interpolation of the green channel of the target pixel in the vertical direction. Here, pixels located in the same row or column as the target pixel refer to pixels with green channel values ​​in the Bayer image, and the target window refers to a window in the Bayer image centered on the target pixel, with a size smaller than the size of the window in the scanned Bayer image described above.

[0039] For example, such as Figure 4 As shown, Figure 4 For the above Figure 3 The 5×5 area cropped from the window shown is taken as the target window, and the target pixel is the center pixel within the target window. The target pixel contains the blue channel value B12. The initial interpolation of the green channel of the target pixel in the horizontal direction can be calculated according to the following formula (1), and the initial interpolation of the green channel of the target pixel in the vertical direction can be calculated according to the following formula (2): Gh12 = (G11 + G13) / 2 (1) Gv12 = (G7 + G17) / 2(2) Wherein, Gh12 is the initial interpolation of the green channel of the target pixel in the horizontal direction, G11 and G13 are the green channel values ​​of pixels in the same row as the target pixel in the horizontal direction within the target window, Gv12 is the initial interpolation of the green channel of the target pixel in the vertical direction, and G7 and G17 are the green channel values ​​of pixels in the same column as the target pixel in the vertical direction within the target window.

[0040] Taking a target window size of 5×5 as an example, within the target window, pixels that are in the same row as the target pixel and have a green channel value include the pixels to the left and right of the target pixel. The weight of these two pixels is 1. Similarly, within the target window, pixels that are in the same column as the target pixel and have a green channel value include the pixels to the top and bottom of the target pixel. The weight of these two pixels is 1.

[0041] In practical applications, a larger target window results in more pixels with green channel values ​​that are in the same row and column as the target pixel. This allows for the inclusion of more green channel values ​​when calculating Gh12 and Gv12. Furthermore, different weights can be assigned to these pixels based on their distance from the target pixel, with pixels closer to the target pixel receiving greater weights and pixels farther away receiving smaller weights. This ensures that the green channel values ​​of these pixels have different effects on the initial interpolation of the target pixel's green channel in the horizontal and vertical directions, thereby improving the accuracy of the initial interpolation of the target pixel's green channel in both directions.

[0042] In some embodiments, the value of the first channel of the target pixel (hereinafter referred to as the first channel value) is subtracted from the first channel values ​​of pixels located in the same row as the target pixel in the horizontal direction within the target window, resulting in multiple horizontal differences. These multiple horizontal differences are then weighted, summed, and averaged to obtain the color gradient of the target pixel in the first channel in the horizontal direction. Similarly, the first channel value of the target pixel is subtracted from the first channel values ​​of pixels located in the same column as the target pixel in the vertical direction within the target window, resulting in multiple vertical differences. These multiple vertical differences are then weighted, summed, and averaged to obtain the color gradient of the target pixel in the first channel in the vertical direction. Pixels located in the same row or column as the target pixel refer to pixels with a first channel value in the Bayer image.

[0043] For example, as described above Figure 4 As shown, continuing the example above, the target pixel is the center pixel within the target window, containing the blue channel value B12. The color gradient of the first channel of the target pixel in the horizontal direction can be calculated using the following formula (3), and the color gradient of the first channel of the target pixel in the vertical direction can be calculated using the following formula (4): Bgh12 = [ (B12 - B10) + (B12 - B14) ] / 2 (3) Bgv12 = [ (B12 - B2) + (B12 - B22) ] / 2 (4) Wherein, Bgh12 is the color gradient of the first channel of the target pixel in the horizontal direction, B10 and B14 are the first channel values ​​of the pixels in the same row as the target pixel in the horizontal direction within the target window, Bgv12 is the color gradient of the first channel of the target pixel in the vertical direction, and B2 and B22 are the first channel values ​​of the pixels in the same column as the target pixel in the vertical direction within the target window.

[0044] Taking a target window size of 5×5 as an example, within the target window, pixels in the same row as the target pixel and having a first channel value include the pixels to the left and right of the target pixel. The first channel value of the target pixel is subtracted from the first channel values ​​of these two pixels to obtain two horizontal differences, each with a weight of 1. Similarly, within the target window, pixels in the same column as the target pixel and having a first channel value include the pixels above and below the target pixel. The first channel value of the target pixel is subtracted from the first channel values ​​of these two pixels to obtain two vertical differences, each with a weight of 1.

[0045] In practical applications, a larger target window results in more pixels that are in the same row and column as the target pixel and have a first channel value. This allows for the inclusion of more horizontal and vertical differences when calculating Bgh12 and Bgv12. Furthermore, different weights can be assigned to these horizontal and vertical differences based on their distance from the target pixel. Pixels closer to the target pixel receive larger weights for their calculated horizontal or vertical differences, while pixels farther away receive smaller weights. This ensures that the first channel values ​​of these pixels have different effects on the color gradient of the target pixel in the horizontal and vertical directions, improving the accuracy of the target pixel's color gradient in the horizontal and vertical directions.

[0046] In some embodiments, the green channel interpolation determination module 201 is configured to: determine the adaptive weight of the target pixel, wherein the adaptive weight indicates the trend of change between the color gradient of the green channel of the neighboring pixels of the target pixel and the color gradient of the first channel of the neighboring pixels; and determine the green channel interpolation of the target pixel in the horizontal direction based on the initial interpolation of the green channel of the target pixel in the horizontal direction, the color gradient of the first channel of the target pixel in the horizontal direction, and the adaptive weight.

[0047] The neighboring pixels of the target pixel include pixels located around the target pixel that have green channel values ​​and first channel values. The process of determining the adaptive weights of the target pixel includes: determining the color gradient of the green channel of the target pixel's neighboring pixels; determining the color gradient of the first channel of the target pixel's neighboring pixels; and determining the ratio of the green channel color gradient of the target pixel's neighboring pixels to the total color gradients of the target pixel's neighboring pixels. This ratio is the adaptive weight. The total color gradients of the target pixel's neighboring pixels refer to the sum of the green channel color gradient and the first channel color gradient of the neighboring pixels.

[0048] For example, as described above Figure 4 As shown, continuing the example in the above embodiment, the target pixel is the center pixel within the target window, containing the blue channel value B12. The color gradient of the green channel of the neighboring pixels of the target pixel can be calculated using the following formula (5), and the color gradient of the first channel of the neighboring pixels of the target pixel can be calculated using the following formula (6): Gg12 = (|G7 - G13| + |G13 - G17| + |G17 - G11| + |G11 - G7|) / 4(5) Bg12 = (|B2 - B14| + |B14 - B22| + |B22 - B10| + |B10 - B2|) / 4(6) Wherein, Gg12 is the color gradient of the green channel of the neighboring pixels of the target pixel, G7, G13, G17 and G11 are the green channel values ​​of the neighboring pixels of the target pixel, Bg12 is the color gradient of the first channel of the neighboring pixels of the target pixel, and B2, B14, B22 and B10 are the first channel values ​​of the neighboring pixels of the target pixel.

[0049] Taking a target window size of 5×5 as an example, when determining the color gradient of the green channel of the target pixel's neighboring pixels, the target pixel's neighboring pixels include the pixels directly above, below, to the left, and to the right of the target pixel. The weight of the difference obtained by subtracting the green channel values ​​of these pixels is 1. Similarly, when determining the color gradient of the first channel color of the target pixel's neighboring pixels, the target pixel's neighboring pixels include the pixels directly above, below, to the left, and to the right of the target pixel. The weight of the difference obtained by subtracting the first channel values ​​of these pixels is 1.

[0050] In practical applications, a larger target window means a greater number of neighboring pixels within the target pixel. This allows for the inclusion of more subtraction differences between the green channel values ​​of neighboring pixels when calculating Gg12. Furthermore, different weights can be assigned to these differences based on their distance from the target pixel. The closer a neighboring pixel is to the target pixel, the greater its weight; conversely, the farther away a neighboring pixel is, the smaller its weight. This results in different effects of the subtraction differences on the green channel color gradient of the target pixel's neighboring pixels, thus improving the accuracy of the green channel color gradient calculation.

[0051] Similarly, when calculating Bg12, more differences obtained by subtracting the first channel values ​​of neighboring pixels can be included. Furthermore, based on the distance between the target pixel and its neighboring pixels, different weights can be assigned to these differences. The closer the neighboring pixels are to the target pixel, the greater the weight of the differences obtained by subtracting the first channel values ​​of the neighboring pixels, and the farther the neighboring pixels are from the target pixel, the smaller the weight of the differences obtained by subtracting the first channel values ​​of the neighboring pixels. This results in different effects of the differences obtained by subtracting the first channel values ​​of the neighboring pixels on the color gradient of the target pixel's neighboring pixels, thereby improving the accuracy of the color gradient of the first channel of the target pixel's neighboring pixels.

[0052] The adaptive weight of the target pixel can be calculated according to the following formula (7): wgtapt = Gg12 / (Gg12 + Bg12) (7) Where wgtapt is the adaptive weight of the target pixel.

[0053] In some embodiments, based on the initial interpolation of the green channel of the target pixel in the horizontal direction, the color gradient of the first channel of the target pixel in the horizontal direction, and the adaptive weights, the interpolation of the green channel of the target pixel in the horizontal direction can be calculated according to the following formula (8): Gh12' = Gh12 + wgtapt Bgh12(8) Where Gh12' is the green channel interpolation of the target pixel in the horizontal direction.

[0054] In other embodiments, based on the initial interpolation of the green channel of the target pixel in the horizontal direction, the color gradient of the first channel of the target pixel in the horizontal direction, and the adaptive weights, the interpolation of the green channel of the target pixel in the horizontal direction can be calculated according to the following formula (9): Gh12' = Gh12 + wgtapt Bgh12 k(9) Where k is a set weight value, used to further adjust the proportion of the initial interpolation of the green channel of the target pixel in the horizontal direction by correcting the color gradient of the first channel in the horizontal direction.

[0055] In some embodiments, based on the initial interpolation of the green channel of the target pixel in the vertical direction, the color gradient of the first channel of the target pixel in the vertical direction, and the adaptive weights, the green channel interpolation of the target pixel in the vertical direction can be calculated according to the following formula (10): Gv12' = Gv12 + wgtapt Bgv12 (10) Where Gv12' is the green channel value of the target pixel in the vertical direction.

[0056] In other embodiments, based on the initial interpolation of the green channel of the target pixel in the vertical direction, the color gradient of the first channel of the target pixel in the vertical direction, and the adaptive weights, the green channel interpolation of the target pixel in the vertical direction can be calculated according to the following formula (11): Gv12' = Gv12 + wgtapt Bgv12 k(11) Where k is the set weight value, which is used to further adjust the proportion of the initial interpolation of the green channel of the target pixel in the vertical direction by correcting the color gradient of the first channel in the vertical direction.

[0057] In the process of correcting the horizontal green channel interpolation, the color gradients of the green channels and the first channel of the target pixel are determined. Adaptive weights are then obtained using these color gradients. These adaptive weights reflect the changing trend between the color gradients of the green channels and the first channel of the target pixel's neighbors. The proportion of the initial horizontal green channel interpolation of the target pixel corrected by the color gradient of the first channel in the horizontal direction is determined using these adaptive weights, thus obtaining the horizontal green channel interpolation of the target pixel. In other words, the initial horizontal green channel interpolation of the target pixel is corrected by using the color gradient of the first channel in the horizontal direction, replacing the assumption of constant local color difference and compensating for the shortcomings of this assumption. Similarly, in the process of correcting the vertical green channel interpolation, the adaptive weights determine the proportion of the initial vertical green channel interpolation of the target pixel corrected by the color gradient of the first channel in the vertical direction, thus obtaining the vertical green channel interpolation of the target pixel. In other words, the initial interpolation of the green channel in the vertical direction of the target pixel is corrected by using the color gradient of the first channel in the vertical direction. This uses the color gradient to replace the assumption of constant local color difference in correcting the initial interpolation of the green channel in the vertical direction, thus compensating for the shortcomings of the assumption. For example... Figure 5 As shown, Figure 5 The left image shows an image with false color generated in highly saturated regions, while the right image shows an image generated after replacing the assumption of constant local color difference with a color gradient. Figure 5It can be seen that using color gradient to replace the assumption of constant local color difference to correct the initial interpolation of the green channel in both directions can effectively reduce false color spots in the generated image.

[0058] In some embodiments, the initial interpolation of the green channels of the target pixel in the horizontal and vertical directions, or the initial interpolation of the green channels of the target pixel in the horizontal and vertical directions, can also be calculated using the assumption of constant local color difference. If the initial interpolation of the green channels of the target pixel in the horizontal and vertical directions is calculated using the assumption of constant local color difference, then it can also be corrected in the manner described above, and this application embodiment will not elaborate further.

[0059] In some embodiments, the comprehensive weight determination module 202 is configured to: determine the overall color difference weights and overall color difference gradient weights of pixels within a target window in the horizontal and vertical directions, respectively. The target window refers to a window centered on the target pixel in a Bayer image. The overall color difference weight indicates the degree of overall color deviation of pixels within the target window in the corresponding direction, and the overall color difference gradient weight indicates the degree of overall color difference variation of pixels within the target window in the corresponding direction. The module further determines the color difference weight percentage, which indicates the combined degree of asymmetric distribution of overall color difference and overall color difference gradient in the horizontal and vertical directions of pixels within the target window. Based on the overall color difference weight, overall color difference gradient weight, and color difference weight percentage in the horizontal direction, the module determines the comprehensive weight of the target pixel in the horizontal direction. Finally, based on the overall color difference weight, overall color difference gradient weight, and color difference weight percentage in the vertical direction, the module determines the comprehensive weight of the target pixel in the vertical direction.

[0060] In some embodiments, the comprehensive weight determination module 202 is configured to: determine the color difference intensity of the target window in the horizontal direction based on the color difference of multiple first pixels within the target window in the horizontal direction, wherein the multiple first pixels have blue channel values ​​or red channel values ​​in the Bayer image; determine the color difference intensity of the target window in the vertical direction based on the color difference of the multiple first pixels in the vertical direction; determine the overall color difference weight in the horizontal direction based on the color difference intensity in the horizontal direction; and determine the overall color difference weight in the vertical direction based on the color difference intensity in the vertical direction.

[0061] Color difference refers to the difference between the green channel value and the non-green channel value (red channel value or blue channel value) within the same pixel. For example, for a single pixel, the difference between the green channel value and the blue channel value of that pixel is determined as the color difference of that pixel. This color difference is used to characterize the relative color difference within that pixel (i.e., the difference between green and blue). A larger color difference indicates a more significant deviation between the two color channels; a smaller difference indicates that the values ​​of the two color channels are closer and the color is more uniform.

[0062] In some embodiments, the horizontal green channel interpolation of the plurality of first pixels is determined, and the horizontal green channel interpolation of the plurality of first pixels is subtracted from their respective first channel values ​​to obtain the color difference of the plurality of first pixels in the horizontal direction. The absolute values ​​of the color differences of the plurality of first pixels in the horizontal direction are weighted, summed, and averaged to obtain the color difference intensity of the target window in the horizontal direction. Similarly, the vertical green channel interpolation of the plurality of first pixels is determined, and the vertical green channel interpolation of the plurality of first pixels is subtracted from their respective first channel values ​​to obtain the color difference of the plurality of first pixels in the vertical direction. The absolute values ​​of the color differences of the plurality of first pixels in the vertical direction are weighted, summed, and averaged to obtain the color difference intensity of the target window in the vertical direction.

[0063] For example, as described above Figure 4 As shown, continuing with the example in the above embodiment, taking the pixel with the blue channel value B12 among the plurality of first pixels as an example, the implementation process of determining the green channel interpolation in the horizontal and vertical directions of the plurality of first pixels is as described above, and will not be repeated here. The color difference of the first pixel in the horizontal direction can be calculated according to the following formula (12), and the color difference of the first pixel in the vertical direction can be calculated according to the following formula (13): Gch12 = Gh12' - B12(12) Gcv12 = Gv12' - B12(13) Wherein, Gch12 is the color difference of the first pixel with a blue channel value in the horizontal direction, and Gcv12 is the color difference of the first pixel with a blue channel value in the vertical direction. In some other embodiments, if the first pixel is a pixel with a red channel value R12, the color difference of the first pixel in the horizontal direction can be calculated according to the following formula (14), and the color difference of the first pixel in the vertical direction can be calculated according to the following formula (15): Gch12 = Gh12' - R12(14) Gcv12 = Gv12' - R12(15) Wherein, Gch12 is the color difference of the first pixel with the red channel value in the horizontal direction, and Gcv12 is the color difference of the first pixel with the red channel value in the vertical direction.

[0064] The color difference intensity of the target window in the horizontal direction can be calculated using the following formula (16), and the color difference intensity of the target window in the vertical direction can be calculated using the following formula (17): diffh = (|Gch0| + |Gch2| + |Gch4| + |Gch10| + |Gch12| + |Gch14| + |Gch20| + |Gch22| + |Gch24| + |Gch6| + |Gch8| + |Gch16| + |Gch18|) / 13(16) diffv = (|Gcv0| + |Gcv2| + |Gcv4| + |Gcv10| + |Gcv12| + |Gcv14| + |Gcv20| + |Gcv22| + |Gcv24| + |Gcv6| + |Gcv8| + |Gcv16| + |Gcv18|) / 13(17) Wherein, diffh represents the color difference intensity of the target window in the horizontal direction, Gch0, Gch2, Gch4, Gch10, Gch12, Gch14, Gch20, Gch22, and Gch24 represent the color differences of multiple first pixels with blue channel values ​​in the horizontal direction, and Gch6, Gch8, Gch16, and Gch18 represent the color differences of multiple first pixels with red channel values ​​in the horizontal direction; diffv represents the color difference intensity of the target window in the vertical direction, Gcv0, Gcv2, Gcv4, Gcv10, Gcv12, Gcv14, Gcv20, Gcv22, and Gcv24 represent the color differences of multiple first pixels with blue channel values ​​in the vertical direction, and Gcv6, Gcv8, Gcv16, and Gcv18 represent the color differences of multiple first pixels with red channel values ​​in the vertical direction.

[0065] In some embodiments, the process of determining the overall color difference weight in the horizontal direction includes: determining the overall color difference weight in the horizontal direction to be 1 in response to the color difference intensity in the horizontal direction being less than or equal to a first threshold; and determining the overall color difference weight in the horizontal direction to be less than 1 in response to the color difference intensity in the horizontal direction being greater than the first threshold. Similarly, the process of determining the overall color difference weight in the vertical direction includes: determining the overall color difference weight in the vertical direction to be 1 in response to the color difference intensity in the vertical direction being less than or equal to a first threshold; and determining the overall color difference weight in the vertical direction to be less than 1 in response to the color difference intensity in the vertical direction being greater than the first threshold.

[0066] In some embodiments, the overall color difference weight in the horizontal direction can be calculated according to the following formula (18): wgtdh= MIN((k1 (diffh - th1) + 1), 1)(18) Where wgtdh is the overall color difference weight in the horizontal direction, MIN((k1 (diffh - th1) + 1), 1) is the selection (k1) The smaller of (diffh - th1) + 1 and 1, where k1 is the first slope and less than 0, and th1 is the first threshold. In the formula, the first threshold and the first slope are empirical values ​​that are set manually.

[0067] In other words, when the color difference intensity in the horizontal direction is less than or equal to the first threshold, (k1) If (diffh - th1) + 1) is greater than 1, the overall color difference weight in the horizontal direction is 1; if the color difference intensity in the horizontal direction is greater than the first threshold, (k1) (diffh - th1) + 1) is less than 1, so the overall color difference weight in the horizontal direction is based on (k1) (diffh -th1) + 1) is calculated to show that the color difference intensity in the horizontal direction and the overall color difference weight in the horizontal direction satisfy a linear relationship.

[0068] In some embodiments, the overall color difference weight in the vertical direction can be calculated according to the following formula (19): wgtdv == MIN((k1 (diffv - th1) + 1), 1)(19) Where wgtdv is the overall color difference weight in the vertical direction, MIN((k1 (diffv - th1) + 1), 1) is the selection (k1) The smaller of (diffv - th1) + 1 and 1, where k1 is the first slope and less than 0, and th1 is the first threshold. In the formula, the first threshold and the first slope are empirical values ​​that are set manually.

[0069] In other words, when the color difference intensity in the vertical direction is less than or equal to the first threshold, (k1) If (diffv - th1) + 1) is greater than 1, the overall color difference weight in the vertical direction is 1; if the color difference intensity in the vertical direction is greater than the first threshold, (k1) If (diffv - th1) + 1) is less than 1, the overall color difference weight in the vertical direction is determined according to (k1). (diffv -th1) + 1) is calculated to show that the color difference intensity in the vertical direction and the overall color difference weight in the vertical direction satisfy a linear relationship.

[0070] Based on the above, the process of determining the overall color difference weight in the horizontal direction and the overall color difference weight in the vertical direction can be divided into two cases, such as... Figure 6As shown, th1 is the first threshold, the horizontal axes diffh and diffv represent the color difference intensity in the horizontal and vertical directions, respectively, and the vertical axes wgtdh and wgtdv represent the overall color difference weight in the horizontal and vertical directions, respectively. When diffh is less than or equal to th1, (k1 (diffh - th1) + 1) is greater than 1, and wgtdh is 1 based on the above formula (18); when diffh is greater than th1, (k1 (diffh - th1) + 1) is less than 1, so wgtdh is calculated based on the above formula (18); when diffv is less than or equal to th1, (k1 (diffv - th1) + 1) is greater than 1, so wgtdv is 1 based on the above formula (19); when diffv is greater than th1, (k1 (diffv - th1) + 1) is less than 1, and wgtdv is calculated based on the above formula (19).

[0071] In some embodiments, the comprehensive weight determination module 202 is configured to: determine the color difference gradient intensity of the target window in the horizontal direction based on the color difference gradient of multiple second pixels within the target window in the horizontal direction, wherein the multiple second pixels have green channel values ​​in the Bayer image; determine the color difference gradient intensity of the target window in the vertical direction based on the color difference gradient of the multiple second pixels in the vertical direction; determine the overall color difference gradient weight in the horizontal direction based on the color difference gradient intensity in the horizontal direction; and determine the overall color difference gradient weight in the vertical direction based on the color difference gradient intensity in the vertical direction.

[0072] Color gradient refers to the magnitude of the color difference between adjacent pixels in an image in the horizontal or vertical direction. For example, for three adjacent pixels, the difference in color between the two outermost pixels can be used as the color gradient of the middle pixel. Color gradient can be used to quantify the rate and trend of color difference changes within a local area. A larger color gradient indicates a more drastic color change, while a smaller color gradient indicates a more gradual color change.

[0073] In some embodiments, for each of the plurality of second pixels, the color difference in the horizontal direction of the first pixel located in the same row and adjacent to the second pixel in the target window is subtracted to obtain the color difference gradient of the second pixel in the horizontal direction. The absolute values ​​of the color difference gradients of the plurality of second pixels in the horizontal direction are then weighted, and the average value is taken to obtain the color difference gradient intensity of the target window in the horizontal direction. Similarly, for each of the plurality of second pixels, the color difference in the vertical direction of the first pixel located in the same column and adjacent to the second pixel in the target window is subtracted to obtain the color difference gradient of the second pixel in the vertical direction. The absolute values ​​of the color difference gradients of the plurality of second pixels in the vertical direction are then weighted, and the average value is taken to obtain the color difference gradient intensity of the target window in the vertical direction.

[0074] For example, as described above Figure 4 As shown, continuing with the example in the above embodiment, taking the pixel with the green channel value G11 among the plurality of second pixels as an example, the color difference gradient of the second pixel in the horizontal direction can be calculated according to the following formula (20), and the color difference gradient of the second pixel in the vertical direction can be calculated according to the following formula (21): Gcgh11 = Gch10 - Gch12 (20) Gcgv11 = Gcv6 - Gcv16 (21) Wherein, Gcgh11 is the color difference gradient of the second pixel in the horizontal direction, and Gcgv11 is the color difference gradient of the second pixel in the vertical direction. Gch10 and Gch12 are the color differences in the horizontal direction of two first pixels that are in the same row as the second pixel and are adjacent to each other on the left and right, respectively. Gcv6 and Gcv16 are the color differences in the vertical direction of two first pixels that are in the same column as the second pixel and are adjacent to each other on the top and bottom, respectively.

[0075] The color difference gradient intensity of the target window in the horizontal direction can be calculated according to the following formula (22), and the color difference gradient intensity of the target window in the vertical direction can be calculated according to the following formula (23): gradh = (|Gcgh1| + |Gcgh3| + |Gcgh11| + |Gcgh13| + |Gcgh21| + |Gcgh23| + |Gcgh7| + |Gcgh17|) / 8(22) gradv = (|Gcgv5| + |Gcgv7| + |Gcgv9| + |Gcgv15| + |Gcgv17| + |Gcgv19|+ |Gcgv11| + |Gcgv13|) / 8(23) Wherein, gradh is the color difference gradient intensity of the target window in the horizontal direction, and Gcgh1, Gcgh3, Gcgh11, Gcgh13, Gcgh21, Gcgh23, Gcgh7 and Gcgh17 are the color difference gradients of multiple second pixels in the horizontal direction; gradv is the color difference gradient intensity of the target window in the vertical direction, and Gcgv5, Gcgv7, Gcgv9, Gcgv15, Gcgv17, Gcgv19, Gcgv11 and Gcgv13 are the color difference gradients of multiple second pixels in the vertical direction.

[0076] In some embodiments, the process of determining the overall color difference gradient weight in the horizontal direction includes: determining the overall color difference gradient weight in the horizontal direction to be 1 in response to the color difference gradient intensity in the horizontal direction being less than or equal to a second threshold; and determining the overall color difference gradient weight in the horizontal direction to be less than 1 in response to the color difference gradient intensity in the horizontal direction being greater than the second threshold. Similarly, the process of determining the overall color difference gradient weight in the vertical direction includes: determining the overall color difference gradient weight in the vertical direction to be 1 in response to the color difference gradient intensity in the vertical direction being less than or equal to a second threshold; and determining the overall color difference gradient weight in the vertical direction to be less than 1 in response to the color difference gradient intensity in the vertical direction being greater than the second threshold.

[0077] In some embodiments, the overall color difference gradient weight in the horizontal direction can be calculated according to the following formula (24): wgtgh = MIN((k2 (gradh – th2) + 1), 1)(24) Where wgtgh is the overall color difference gradient weight in the horizontal direction, MIN((k2 (gradh – th2) + 1), 1) is the selection of (k2) The smaller of (gradh – th2) + 1 and 1, where k2 is the second slope and less than 0, and th2 is the second threshold. In the formula, the second threshold and the second slope are empirically set values.

[0078] In other words, when the chromatic difference gradient intensity in the horizontal direction is less than or equal to the second threshold, (k2) If (gradh – th2) + 1) is greater than 1, the overall color difference gradient weight in the horizontal direction is 1; if the color difference gradient intensity in the horizontal direction is greater than the second threshold, (k2) (gradh – th2) + 1) is less than 1, so the overall color difference gradient weights in the horizontal direction are calculated according to (k2). (gradh – th2) + 1) is calculated to show that the color difference gradient intensity in the horizontal direction and the overall color difference gradient weight in the horizontal direction satisfy a linear relationship.

[0079] In some embodiments, the overall color difference gradient weight in the vertical direction can be calculated according to the following formula (25): wgtgv = MIN((k2 (gradv – th2) + 1), 1)(25) Where wgtgv is the overall color difference weight in the vertical direction, MIN((k2) (gradv – th2) + 1), 1) is the selection (k2) The smaller of (gradv – th2) + 1 and 1, where k2 is the second slope and less than 0, and th2 is the second threshold. In the formula, the second threshold and the second slope are empirically set values.

[0080] In other words, when the chromatic difference gradient intensity in the vertical direction is less than or equal to the second threshold, (k2) If (gradv – th2) + 1) is greater than 1, the overall color difference gradient weight in the vertical direction is 1; if the color difference gradient intensity in the vertical direction is greater than the second threshold, (k2) (gradv – th2) + 1) is less than 1, so the overall color difference gradient weights in the vertical direction are calculated according to (k2). (gradv – th2) + 1) is calculated to show that the color difference gradient intensity in the vertical direction and the overall color difference gradient weight in the vertical direction satisfy a linear relationship.

[0081] Based on the above, the determination process of the overall color difference gradient weights in the horizontal and vertical directions can be divided into two cases, such as... Figure 7 As shown, th2 is the second threshold, the horizontal axes gradh and gradv represent the color difference gradient strength in the horizontal and vertical directions, respectively, and the vertical axes wgtgh and wgtgv represent the overall color difference gradient weights in the horizontal and vertical directions, respectively. When gradh is less than or equal to th2, (k2 (gradh – th2) + 1) is greater than 1, so wgtgh is 1 based on the above formula (24); when gradh is greater than th2, (k2 (gradh – th2) + 1) is less than 1, so wgtgh is calculated based on the above formula (24); when gradv is less than or equal to th2, (k2 (gradv – th2) + 1) is greater than 1, so wgtgv is 1 based on the above formula (25); when gradv is greater than th2, (k2 (gradv – th2) + 1) is less than 1, and wgtgv is calculated based on the above formula (25).

[0082] In other embodiments, the overall color difference gradient weights in the horizontal direction and the overall color difference gradient weights in the vertical direction can also be determined according to a first threshold and a first slope. That is, the threshold and slope used to determine the overall color difference gradient weights in the horizontal direction and the overall color difference gradient weights in the vertical direction can be the same as those used to determine the overall color difference weights in the horizontal direction and the overall color difference weights in the vertical direction.

[0083] In some embodiments, the comprehensive weight determination module 202 is configured to: determine a first weight based on the difference between the color difference intensity in the horizontal direction and the color difference intensity in the vertical direction of the target window; determine a second weight based on the ratio between the color difference intensity in the horizontal direction and the color difference intensity in the vertical direction of the target window; determine a third weight based on the difference between the color difference gradient intensity in the horizontal direction and the color difference gradient intensity in the vertical direction of the target window; determine a fourth weight based on the ratio between the color difference gradient intensity in the horizontal direction and the color difference gradient intensity in the vertical direction of the target window; and determine the color difference weight percentage based on the first, second, third, and fourth weights.

[0084] In some embodiments, the process of determining the first weight includes: subtracting the color difference intensity in the horizontal direction from the color difference intensity in the vertical direction of the target window and taking the absolute value to obtain the color difference intensity difference value. If the color difference intensity difference value is greater than or equal to a third threshold, the first weight is determined to be 1; if the color difference intensity difference value is less than the third threshold, the first weight is determined to be less than 1.

[0085] In some embodiments, the first weight can be calculated according to the following formula (26): wgt1 = MIN((k3 (|diffh - diffv| - th3) ​​+ 1), 1)(26) Where wgt1 is the first weight, MIN((k3) (|diffh - diffv| - th3) ​​+ 1), 1) is the selection (k3) The smaller of (|diffh - diffv| - th3) ​​+ 1) and 1, where k3 is the third slope and is greater than 0, and th3 is the third threshold. In the formula, the third threshold and the third slope are empirically set values.

[0086] In other words, when the color difference intensity difference is greater than or equal to the third threshold, (k3) If (|diffh - diffv| - th3) ​​+ 1) is greater than 1, then the first weight is 1; if the color difference intensity difference is less than the third threshold, (k3) If (|diffh - diffv| - th3) ​​+ 1) is less than 1, the first weight is determined according to (k3). The result is obtained by calculating (|diffh - diffv| - th3) ​​+ 1), which means that the color difference intensity difference and the first weight satisfy a linear relationship.

[0087] Based on the above, the process of determining the first weight can be divided into two cases, such as... Figure 8 As shown, th3 is the third threshold, the horizontal axis |diffh - diffv| represents the color difference intensity difference, and the vertical axis wgt1 represents the first weight. When |diffh - diffv| is greater than or equal to th3, (k3 (|diffh - diffv| - th3) ​​+ 1) is greater than 1, so wgt1 is 1 based on formula (26); when |diffh - diffv| is less than th3, (k3 (|diffh - diffv| - th3) ​​+ 1) is less than 1, and wgt1 is calculated based on the above formula (26).

[0088] In some embodiments, the process of determining the second weight includes: dividing the larger of the color difference intensity in the horizontal direction and the color difference intensity in the vertical direction of the target window by the smaller color difference intensity to obtain a color difference intensity ratio. In response to a color difference intensity ratio greater than or equal to a fourth threshold, the second weight is determined to be 1; in response to a color difference intensity ratio less than the fourth threshold, the second weight is determined to be less than 1.

[0089] In some embodiments, the second weight can be calculated according to the following formula (27): wgt2 = MIN((k4 (diffmax / diffmin - th4) + 1), 1)(27) Where wgt2 is the first weight, MIN((k4) (diffmax / diffmin - th4) + 1), 1) is the selection of (k4) The smaller of (diffmax / diffmin - th4) + 1 and 1, where k4 is the fourth slope and is greater than 0, and th4 is the fourth threshold. In the formula, the fourth threshold and the fourth slope are empirical values ​​that are set manually.

[0090] In other words, when the color difference intensity ratio is greater than or equal to the fourth threshold, (k4) If (diffmax / diffmin - th4) + 1) is greater than 1, the second weight is 1; when the color difference intensity ratio is less than the fourth threshold, (k4) If (diffmax / diffmin - th4) + 1) is less than 1, the second weight is determined according to (k4). The result is obtained by calculating (diffmax / diffmin - th4) + 1, which means that the color difference intensity ratio and the second weight satisfy a linear relationship.

[0091] Based on the above, the process of determining the second weight can be divided into two cases, such as... Figure 9 As shown, th4 is the fourth threshold, the horizontal axis diffmax / diffmin is the color difference intensity ratio, and the vertical axis wgt2 is the second weight. When diffmax / diffmin is greater than or equal to th4, (k4 (diffmax / diffmin - th4) + 1) is greater than 1, so wgt2 is 1 based on formula (27); when diffmax / diffmin is less than th4, (k4 (diffmax / diffmin - th4) + 1) is less than 1, so wgt2 is calculated based on the above formula (27).

[0092] In some embodiments, the process of determining the third weight includes: subtracting the color difference gradient intensity in the horizontal direction from the color difference gradient intensity in the vertical direction of the target window and taking the absolute value to obtain the color difference gradient intensity difference. If the color difference gradient intensity difference is less than or equal to a fifth threshold, the third weight is determined to be 1; if the color difference gradient intensity difference is greater than the fifth threshold, the third weight is determined to be less than 1.

[0093] In some embodiments, the third weight can be calculated according to the following formula (28): wgt3 = MIN((k5 (|gradh - gradv| - ​​th5) + 1), 1)(28) Where wgt3 is the third weight, MIN((k5) (|gradh - gradv| - ​​th5) + 1), 1) is the selection of (k5) The smaller of (|gradh - gradv| - ​​th5) + 1) and 1, where k5 is the fifth slope and less than 0, and th5 is the fifth threshold. In the formula, the fifth threshold and the fifth slope are manually set, adjustable empirical values.

[0094] In other words, when the color difference gradient intensity difference is less than or equal to the fifth threshold, (k5) (|gradh -gradv| - ​​th5) + 1) is greater than 1, so the third weight is 1; when the difference in color gradient intensity is greater than the fifth threshold, (k5) If (|gradh - gradv| - ​​th5) + 1) is less than 1, the third weight is determined according to (k5). (|gradh - gradv| - ​​th5) + 1) is calculated to show that the color difference gradient intensity difference and the third weight satisfy a linear relationship.

[0095] Based on the above, the process of determining the third weight can be divided into two cases, such as... Figure 10 As shown, th5 is the fifth threshold, the horizontal axis |gradh - gradv| represents the color gradient intensity difference, and the vertical axis wgt3 represents the third weight. When |gradh - gradv| is less than or equal to th5, (k5 (|gradh - gradv| - ​​th5) + 1) is greater than 1, so wgt1 is 1 based on formula (28); when |gradh - gradv| is greater than th5, (k5 (|gradh - gradv| - ​​th5) + 1) is less than 1, and wgt3 is calculated based on the above formula (28).

[0096] In some embodiments, the process of determining the fourth weight includes: dividing the larger of the color difference gradient intensities in the horizontal and vertical directions of the target window by the smaller color difference gradient intensity to obtain a color difference gradient intensity ratio. If the color difference gradient intensity ratio is less than or equal to a sixth threshold, the fourth weight is determined to be 1; if the color difference gradient intensity ratio is greater than the sixth threshold, the fourth weight is determined to be less than 1.

[0097] In some embodiments, the fourth weight can be calculated according to the following formula (29): wgt4 = MIN((k6 (gradmax / gradmin - th6) + 1), 1)(29) Where wgt4 is the fourth weight, MIN((k6) (gradmax / gradmin - th6) + 1), 1) is the selection of (k6) The smaller of (gradmax / gradmin - th6) + 1 and 1, where k6 is the sixth slope and less than 0, and th6 is the sixth threshold. In the formula, the sixth threshold and the sixth slope are manually set, adjustable empirical values.

[0098] In other words, when the ratio of color difference gradient intensity is less than or equal to the sixth threshold, (k6) If (gradmax / gradmin - th6) + 1) is greater than 1, then the fourth weight is 1; when the ratio of color difference gradient intensity is greater than the sixth threshold, (k6) If (gradmax / gradmin - th6) + 1) is less than 1, the fourth weight is determined according to (k6). (gradmax / gradmin -th6) + 1) is calculated to show that the ratio of color difference gradient intensity and the fourth weight satisfy a linear relationship.

[0099] Based on the above, the process of determining the fourth weight can be divided into two cases, such as... Figure 11 As shown, th6 is the sixth threshold, the horizontal axis gradmax / gradmin is the ratio of color difference gradient intensity, and the vertical axis wgt4 is the fourth weight. When gradmax / gradmin is greater than or equal to th6, (k6 (gradmax / gradmin - th6) + 1) is greater than 1, so wgt4 is 1 based on formula (29); when gradmax / gradmin is less than th6, (k6 (gradmax / gradmin - th6) + 1) is less than 1, so wgt4 is calculated based on the above formula (29).

[0100] In some embodiments, based on the first weight, the second weight, the third weight, and the fourth weight, the color difference weight ratio can be calculated according to the following formula (30): wgt = wgt1 wgt2 wgt3 wgt4 (30) Where wgt is the weighting percentage of color difference.

[0101] In other embodiments, the color difference weight ratio can be calculated based on the first weight, the second weight, the third weight, and the fourth weight according to the following formula (31): wgt = (wgt1 wgt2 + wgt3 wgt4) / 2(31) Where wgt is the weighting percentage of color difference.

[0102] In addition, the color difference weight ratio can be determined in other ways. For example, based on the first weight, second weight, third weight, and fourth weight, the color difference weight ratio can also be calculated using the following formula (32): wgt = (wgt1 + wgt2 + wgt3 + wgt4) / 4 (32) Where wgt is the weighting percentage of color difference.

[0103] In some embodiments, based on the overall color difference weight in the horizontal direction, the overall color difference gradient weight in the horizontal direction, and the proportion of color difference weight, the comprehensive weight of the target pixel in the horizontal direction can be calculated according to the following formula (33): wgth = wgtgh (1 - wgt) + wgtdh wgt (33) Where wgth is the overall weight in the horizontal direction.

[0104] In some embodiments, based on the overall color difference weight in the vertical direction, the overall color difference gradient weight in the vertical direction, and the proportion of color difference weight, the comprehensive weight of the target pixel in the vertical direction can be calculated according to the following formula (34): wgtv = wgtgv (1 - wgt) + wgtdv wgt (34) Where wgtv is the overall weight in the vertical direction.

[0105] In determining the overall weight in the horizontal direction, the color difference weight and gradient weight in the horizontal direction are further determined by determining the color difference intensity and gradient intensity within the window. Furthermore, the first and second weights are determined based on the difference and ratio of the color difference intensity in the horizontal and vertical directions; the third and fourth weights are determined based on the difference and ratio of the color difference gradient intensity in the horizontal and vertical directions. These four weights are used to obtain the proportion of color difference weights, and then the overall weight in the horizontal direction is obtained using the color difference weights, the color difference gradient weights, and the proportion of color difference weights. In other words, the proportion of color difference weights is used to adjust the ratio of the horizontal color difference weights and gradient weights in the overall weight in the horizontal direction, so that the horizontal color difference weights and gradient weights jointly determine the overall weight in the horizontal direction, with color difference compensating for the deficiency of color difference gradient in high-frequency regions. Similarly, in determining the overall weight in the vertical direction, the color difference weight and gradient weight in the vertical direction are further determined by determining the color difference intensity and gradient intensity within the window. Then, the overall weight in the vertical direction is obtained using the vertical color difference weight, the vertical color difference gradient weight, and the proportion of the color difference weight. In other words, the proportion of the vertical color difference weight and the color difference gradient weight in the overall weight in the vertical direction is adjusted by using the proportion of the color difference weight, so that the vertical color difference weight and the color difference gradient weight jointly determine the overall weight in the vertical direction, with color difference compensating for the deficiency of color difference gradient in the high-frequency region. For example... Figure 12 As shown, Figure 12 The left image shows an image with pseudocolor generated in the high-frequency region, while the right image shows an image generated after using chromatic difference and chromatic difference gradient to determine the combined weights of the two directions. Figure 12 It can be seen that using color difference and color difference gradient to jointly determine the comprehensive weight of the two directions can effectively reduce false color spots in the generated image.

[0106] In some embodiments, based on the green channel interpolation of the target pixel in the horizontal direction, the green channel interpolation of the target pixel in the vertical direction, the comprehensive weight of the target pixel in the horizontal direction, and the comprehensive weight of the target pixel in the vertical direction, the green channel value of the target pixel can be calculated according to the following formula (35): Gi12 = (Gh12' wgth + Gv12' wgtv) / (wgth + wgtv)(35) Gi12 is the green channel value of the target pixel.

[0107] In some embodiments, the process of determining the value of the second channel of a target pixel (hereinafter referred to as the second channel value) includes: determining the green channel value of each of the neighboring pixels of the target pixel; subtracting the green channel value of each neighboring pixel from its respective second channel value to obtain multiple color difference values; taking the average of the multiple color difference values ​​after weighted summation to obtain the average color difference value; and subtracting the green channel value of the target pixel from the average color difference value to obtain the second channel value of the target pixel. The neighboring pixels of the target pixel refer to the pixels in the Bayer image that are located around the target pixel and have the closest second channel value to the target pixel.

[0108] For example, as described above Figure 4 As shown, continuing the example above, the target pixel is the center pixel within the target window, containing the blue channel value B12. The second channel values ​​of the neighboring pixels of this target pixel are R6, R8, R16, and R18, respectively. The process of determining the green channel values ​​of the neighboring pixels of this target pixel is as described above and will not be repeated here. The second channel value of this target pixel can be calculated according to the following formula (36): Ri12 = Gi12 - (Gc6 + Gc8 + Gc16 + Gc18) / 4 (36) Among them, Ri12 is the second channel value of the target pixel, and Gc6, Gc8, Gc16 and Gc18 are multiple color difference values.

[0109] In some embodiments, for pixels located in the same row as the target pixel, adjacent to it, and having a green channel value, the first channel value and the second channel value of the pixel can be calculated using the assumption of constant local color difference, for example, as described above. Figure 4 As shown, continuing the example above, the target pixel is the center pixel within the target window, containing the blue channel value B12. The adjacent pixels in the same row as the target pixel have green channel values ​​G11 and G13 respectively. Taking the pixel with green channel value G13 as an example, the first channel value of the pixel can be calculated according to the following formula (37), and the second channel value of the pixel can be calculated according to the following formula (38): Bi13 = G13 - (Gc12 + Gc14) / 2 (37) Ri13 = G13 - (Gc8 + Gc18) / 2 (38) In this context, Bi13 and Ri13 represent the first channel value and the second channel value of the pixel, respectively. Gc12 and Gc14 represent the color difference values ​​of adjacent pixels in the same row as the pixel, and Gc8 and Gc18 represent the color difference values ​​of adjacent pixels in the same column as the pixel.

[0110] Based on the above description, the complete implementation process is as follows: Figure 13 As shown, within each sliding window (the sliding window is 9×10, please refer to...) Figure 3 The steps for processing target pixels within a window include: Step 1. Obtain the color gradient of the green channel and the color gradient of the first channel of the target pixel from the target window (RBg represents Rg or Bg in the figure), and further obtain the adaptive weights.

[0111] Step 2. Obtain the initial interpolation of the green channel and the color gradient of the first channel of the target pixel in both horizontal and vertical directions from the target window, and further obtain the green channel interpolation in both horizontal and vertical directions by combining adaptive weights.

[0112] Step 3. Using the target window and the interpolation of the green channels in both the horizontal and vertical directions, the color difference in both directions is obtained, and then the color difference gradient in both directions is obtained.

[0113] Step 4. Calculate the absolute value and weighted average of the color difference and color difference gradient in the horizontal and vertical directions, respectively, and then map them to obtain the color difference weights and color difference gradient weights in the horizontal and vertical directions.

[0114] Step 5. Calculate the absolute value and ratio of the color difference in the horizontal and vertical directions and the color difference gradient in the horizontal and vertical directions, respectively, and map them to obtain four weights. Then multiply the four weights to obtain the color difference weight ratio.

[0115] Step 6. Based on the color difference weight ratio, the color difference weight in the horizontal and vertical directions, and the color difference gradient weight in the horizontal and vertical directions, calculate the comprehensive weight in the horizontal and vertical directions, thereby further correcting the green channel value of the target pixel. Finally, combine it with the target window to obtain the RGB output of the target pixel.

[0116] In this embodiment, the comprehensive weights of the target pixel in the horizontal and vertical directions are determined. These comprehensive weights indicate the combined situation of the color difference and color difference gradient of the target pixel in the corresponding direction. The green channel value of the target pixel is determined by the comprehensive weights of the target pixel in the horizontal and vertical directions. In this way, the color difference and color difference gradient are used together to determine the interpolation direction, and the color difference compensates for the deficiency of the color difference gradient in the high-frequency region, thereby reducing false color spots and zipper-like jagged edges in the generated image.

[0117] This application provides an image processing method that can be executed by a processing device. The flow of the method is as follows: Figure 14As shown, including but not limited to steps 1401 to 1404.

[0118] In step 1401, the green channel interpolation in the horizontal direction and the green channel interpolation in the vertical direction of the target pixel in the Bayer image are determined. The Bayer image includes the value of a first channel of the target pixel, which is either the blue channel or the red channel.

[0119] In some embodiments, determining the green channel interpolation of a target pixel in a Bayer image in the horizontal direction and the green channel interpolation in the vertical direction includes: determining the initial green channel interpolation of the target pixel in the horizontal direction and the initial green channel interpolation in the vertical direction; determining the color gradient of the target pixel in the first channel in the horizontal direction and the color gradient of the first channel in the vertical direction; correcting the initial green channel interpolation of the target pixel in the horizontal direction based on the color gradient of the first channel in the horizontal direction to obtain the green channel interpolation of the target pixel in the horizontal direction; and correcting the initial green channel interpolation of the target pixel in the vertical direction based on the color gradient of the first channel in the vertical direction to obtain the green channel interpolation of the target pixel in the vertical direction.

[0120] In some embodiments, based on the color gradient of the target pixel in the first channel in the horizontal direction, the initial interpolation of the green channel of the target pixel in the horizontal direction is corrected to obtain the green channel interpolation of the target pixel in the horizontal direction, including: determining an adaptive weight for the target pixel, wherein the adaptive weight indicates the trend of change between the color gradient of the green channel of the target pixel and the color gradient of the first channel of the neighboring pixels. The green channel interpolation of the target pixel in the horizontal direction is determined based on the initial interpolation of the green channel of the target pixel in the horizontal direction, the color gradient of the first channel of the target pixel in the horizontal direction, and the adaptive weight.

[0121] In step 1402, the overall weight of the target pixel in the horizontal direction and the overall weight in the vertical direction are determined. The overall weight indicates the overall situation of the color difference and color difference gradient of the target pixel in the corresponding direction.

[0122] In some embodiments, determining the comprehensive weight of the target pixel in the horizontal direction and the comprehensive weight in the vertical direction includes: determining the overall color difference weight and the overall color difference gradient weight of the pixels within the target window in the horizontal and vertical directions, respectively. The target window refers to a window centered on the target pixel in the Bayer image. The overall color difference weight indicates the degree of overall color deviation of the pixels within the target window in the corresponding direction, and the overall color difference gradient weight indicates the degree of overall color difference change of the pixels within the target window in the corresponding direction. The color difference weight percentage is determined, indicating the combined degree of asymmetric distribution of the overall color difference and overall color difference gradient of the pixels within the target window in the horizontal and vertical directions. Based on the overall color difference weight, overall color difference gradient weight, and color difference weight percentage in the horizontal direction, the comprehensive weight of the target pixel in the horizontal direction is determined. Based on the overall color difference weight, overall color difference gradient weight, and color difference weight percentage in the vertical direction, the comprehensive weight of the target pixel in the vertical direction is determined.

[0123] In some embodiments, determining the overall color difference weights of pixels within a target window in the horizontal and vertical directions includes: determining the color difference intensity of the target window in the horizontal direction based on the color differences of a plurality of first pixels within the target window in the horizontal direction, wherein the plurality of first pixels have blue channel values ​​or red channel values ​​in the Bayer image; determining the color difference intensity of the target window in the vertical direction based on the color differences of the plurality of first pixels in the vertical direction; determining the overall color difference weight in the horizontal direction based on the color difference intensity in the horizontal direction; and determining the overall color difference weight in the vertical direction based on the color difference intensity in the vertical direction.

[0124] In some embodiments, determining the overall color difference gradient weights of pixels within a target window in the horizontal and vertical directions includes: determining the color difference gradient intensity of the target window in the horizontal direction based on the color difference gradients of multiple second pixels within the target window in the horizontal direction, wherein the multiple second pixels have green channel values ​​in the Bayer image; determining the color difference gradient intensity of the target window in the vertical direction based on the color difference gradients of the multiple second pixels in the vertical direction; determining the overall color difference gradient weight in the horizontal direction based on the color difference gradient intensity in the horizontal direction; and determining the overall color difference gradient weight in the vertical direction based on the color difference gradient intensity in the vertical direction.

[0125] In some embodiments, determining the color difference weight ratio includes: determining a first weight based on the difference between the color difference intensity in the horizontal direction and the color difference intensity in the vertical direction; determining a second weight based on the ratio between the color difference intensity in the horizontal direction and the color difference intensity in the vertical direction; determining a third weight based on the difference between the color difference gradient intensity in the horizontal direction and the color difference gradient intensity in the vertical direction; determining a fourth weight based on the ratio between the color difference gradient intensity in the horizontal direction and the color difference gradient intensity in the vertical direction; and determining the color difference weight ratio based on the first, second, third, and fourth weights.

[0126] In step 1403, the green channel value of the target pixel is determined based on the green channel interpolation in the horizontal direction and the green channel interpolation in the vertical direction, the comprehensive weight of the target pixel in the horizontal direction and the comprehensive weight in the vertical direction.

[0127] In step 1404, the value of the second channel of the target pixel is determined based on the green channel value of the target pixel; wherein, if the first channel is the blue channel, the second channel is the red channel, and if the first channel is the red channel, the second channel is the blue channel.

[0128] In the image processing method provided in this application embodiment, the comprehensive weight of the target pixel in the horizontal direction and the comprehensive weight in the vertical direction are determined. This comprehensive weight indicates the combined situation of the color difference and color difference gradient of the target pixel in the corresponding direction. The green channel value of the target pixel is determined by the comprehensive weight of the target pixel in the horizontal direction and the comprehensive weight in the vertical direction. This is equivalent to using color difference and color difference gradient to jointly determine the interpolation direction, using color difference to compensate for the deficiency of color difference gradient in the high-frequency region, thereby reducing false color spots and zipper-like jagged edges in the generated image.

[0129] It should be noted that the above embodiments of the device are only illustrated by the division of the above functional modules when implementing their functions. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the methods and device embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the device embodiments, which will not be repeated here.

[0130] Figure 15This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. The device can vary considerably depending on its configuration or performance. It may include one or more processors 1501 and one or more memories 1502. The one or more memories 1502 store at least one computer program, which is loaded and executed by the one or more processors 1501 to enable the device to implement the image processing methods provided in the various method embodiments described above. Of course, the device may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The device may also include other components for implementing its functions, which will not be elaborated upon here. Figure 15 The image processing device shown can be a computer device.

[0131] In an exemplary embodiment, a computer device is also provided, which is capable of implementing the image processing methods provided in the various method embodiments described above. This computer device may be, for example, a smartphone, tablet computer, media player, laptop computer, or desktop computer. The terminal may also be referred to as a user device, portable terminal, laptop terminal, desktop terminal, or other names. The computer device may also be a server or any other device capable of implementing the image processing methods described above.

[0132] See Figure 16 This illustration shows a schematic diagram of the structure of an image processing device according to an embodiment of this application. The image processing device 1600 includes an image processing unit 1601, and the image processing unit 1601 includes various units for executing instructions. Figure 16 In this process, the image processing device 1601 is coupled to the memory 1602, and it should be understood that the image processing device 1600 also supports other memory configurations known in the art.

[0133] The memory 1602 may include one or more computer-readable storage media, which may be non-transitory, and the computer-readable storage media stores at least one computer program, which is loaded and executed by the image processing apparatus 1601 to enable the computer device 1600 to implement any of the above-described image processing methods.

[0134] The memory 1602 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1602 is used to store at least one instruction, which is executed by the processing device 1601 to cause the computer device 1600 to implement the image processing method provided in the above method embodiments.

[0135] In one possible implementation, the aforementioned computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0136] Figure 16 A display 1606 coupled to the image processing apparatus 1601 is also shown. For example, the display 1606 is connected to a display controller 1604, which is also connected to the image processing apparatus 1601, thus coupling the display 1606 to the image processing apparatus 1601. In some cases, the image processing apparatus 1600 can be used for wireless communication. Figure 16 Also shown are a speaker 1609 and a microphone 1610 coupled to the image processing device 1601 via an encoder / decoder 1611; and a wireless antenna 1608 coupled to the wireless controller 1605.

[0137] Display 1606 is used to display a UI (User Interface). This UI may include graphics, text, icons, video, and any combination thereof. When display 1606 is a touch screen, it also has the ability to collect touch signals on or above its surface. In this case, display 1606 can also provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display 1606, located on the front panel of the terminal; in other embodiments, there may be at least two displays 1606, located on different surfaces of the terminal or in a folded design; in still other embodiments, display 1606 may be a flexible display, located on a curved or folded surface of the terminal. Furthermore, display 1606 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. Display 1606 may be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0138] Microphone 1610 is used to collect sound waves from the user and the environment, and input the sound waves to image processing device 1601 for processing. Multiple microphones 1610 can be used for stereo sound acquisition or noise reduction, each positioned at a different location on the terminal. Microphone 1610 can also be an array microphone or an omnidirectional microphone. Speaker 1609 is used to convert electrical signals from image processing device 1601 into sound waves. Speaker 1609 can be a traditional thin-film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement.

[0139] The image processing device 1601 and the memory 1602 may be contained in a system-in-package or a system-on-a-chip device.

[0140] Input device 1607 and power supply 1603 are coupled to system-on-chip device 1612. Optionally, as... Figure 16 As shown, when one or more optional boxes are present, the display 1606, input device 1607, speaker 1609, microphone 1610, wireless antenna 1608, and power supply 1603 are external to the system-on-a-chip device 1612. However, each of the display 1606, input device 1607, speaker 1609, microphone 1610, wireless antenna 1608, and power supply 1603 can be coupled to components of the system-on-a-chip device 1612, such as interfaces or controllers.

[0141] Power supply 1603 is used to power the various components in the terminal. Power supply 1603 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1603 includes a rechargeable battery, the rechargeable battery can support wired or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0142] In one possible implementation, the image processing device 1601 and the memory 1602 may be integrated into a set-top box, server, music player, video player, entertainment unit, navigation device, personal digital assistant (PDA), fixed location data unit, computer, laptop computer, tablet computer, communication device, mobile phone or other similar device.

[0143] The methods described in conjunction with the embodiments of this application can be directly implemented in software modules executed by the image processing apparatus. The software modules may reside in random access memory (RAM), flash memory, read-only memory (ROM), electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, removable disks, compact disc read-only memory (CDROM), or any other form of storage medium known in the art. The storage medium is coupled to the processing apparatus, thereby enabling the image processing apparatus to read information from and write information to the storage medium. Optionally, the storage medium may be integral with the image processing apparatus.

[0144] Those skilled in the art will understand that Figure 16 The structure shown does not constitute a limitation on the image processing device. The image processing device may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0145] In an exemplary embodiment, a chip is also provided, comprising a processor and a memory storing at least one computer program. The at least one computer program is loaded and executed by one or more processors to enable a computer device equipped with the chip to implement any of the aforementioned image processing methods. In an exemplary embodiment, a chip is also provided, comprising any of the aforementioned image processing apparatuses.

[0146] In an exemplary embodiment, an electronic rearview mirror is also provided, which includes any of the chips described above.

[0147] In an exemplary embodiment, a computer device is also provided, comprising a processor and a memory storing at least one computer program. The at least one computer program is loaded and executed by one or more processors to enable the computer device to implement any of the image processing methods described above.

[0148] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one computer program that is loaded and executed by a processor of a computer device to enable the computer to implement any of the above-described image processing methods.

[0149] In one possible implementation, the aforementioned computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0150] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the image processing methods described above.

[0151] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the first grayscale images involved in this application were all obtained with full authorization.

[0152] It should be understood that "at least one" as mentioned herein refers to one or more, and "multiple" refers to two or more. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In addition, in order to clearly describe the technical solutions of the embodiments of this application, the terms "first," "second," etc., are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and the terms "first," "second," etc., are not necessarily different.

[0153] The above descriptions are embodiments provided in this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image processing apparatus characterized by comprising: The device includes: The green channel interpolation determination module is configured to determine the green channel interpolation of a target pixel in a Bayer image in the horizontal direction and the green channel interpolation in the vertical direction, wherein the Bayer image includes the value of a first channel of the target pixel, and the first channel is either the blue channel or the red channel. The comprehensive weight determination module is configured to determine the comprehensive weight of the target pixel in the horizontal direction and the comprehensive weight in the vertical direction, wherein the comprehensive weight indicates the comprehensive situation of the color difference and color difference gradient of the target pixel in the corresponding direction; The green channel value determination module is configured to determine the green channel value of the target pixel based on the green channel interpolation in the horizontal direction and the green channel interpolation in the vertical direction, the comprehensive weight of the target pixel in the horizontal direction and the comprehensive weight in the vertical direction. The non-green channel value determination module is configured to determine the value of the second channel of the target pixel based on the green channel value of the target pixel; wherein, when the first channel is the blue channel, the second channel is the red channel, and when the first channel is the red channel, the second channel is the blue channel.

2. The apparatus as claimed in claim 1, characterized in that, The comprehensive weight determination module is configured as follows: The overall color difference weights and overall color difference gradient weights corresponding to the pixels in the target window in the horizontal and vertical directions are determined respectively. The target window refers to the window centered on the target pixel in the Bayer image. The overall color difference weights indicate the degree of overall color deviation of the pixels in the target window in the corresponding direction, and the overall color difference gradient weights indicate the degree of overall change of color difference of the pixels in the target window in the corresponding direction. Determine the color difference weight ratio, which indicates the overall color difference and the degree of asymmetry in the overall color difference gradient of the pixels in the target window in the horizontal and vertical directions; Based on the overall color difference weight and overall color difference gradient weight in the horizontal direction, as well as the proportion of color difference weight, the comprehensive weight of the target pixel in the horizontal direction is determined. Based on the overall color difference weight and overall color difference gradient weight in the vertical direction, as well as the proportion of color difference weight, the comprehensive weight of the target pixel in the vertical direction is determined.

3. The apparatus as described in claim 2, characterized in that, The comprehensive weight determination module is configured as follows: Based on the color difference of multiple first pixels in the target window in the horizontal direction, the color difference intensity of the target window in the horizontal direction is determined, wherein the multiple first pixels have blue channel values ​​or red channel values ​​in the Bayer image; Based on the color difference of the plurality of first pixels in the vertical direction, the color difference intensity of the target window in the vertical direction is determined; The overall color difference weight in the horizontal direction is determined based on the color difference intensity in the horizontal direction, and the overall color difference weight in the vertical direction is determined based on the color difference intensity in the vertical direction.

4. The apparatus as described in claim 3, characterized in that, The comprehensive weight determination module is configured as follows: Based on the color difference gradient of multiple second pixels in the target window in the horizontal direction, the color difference gradient intensity of the target window in the horizontal direction is determined, and the multiple second pixels have green channel values ​​in the Bayer image; Based on the color difference gradient of the plurality of second pixels in the vertical direction, the color difference gradient intensity of the target window in the vertical direction is determined; The overall color difference gradient weight in the horizontal direction is determined based on the color difference gradient intensity in the horizontal direction, and the overall color difference gradient weight in the vertical direction is determined based on the color difference gradient intensity in the vertical direction.

5. The apparatus as described in claim 4, characterized in that, The comprehensive weight determination module is configured as follows: A first weight is determined based on the difference between the color difference intensity in the horizontal direction and the color difference intensity in the vertical direction; The second weight is determined based on the ratio between the color difference intensity in the horizontal direction and the color difference intensity in the vertical direction; The third weight is determined based on the difference between the color difference gradient intensity in the horizontal direction and the color difference gradient intensity in the vertical direction. The fourth weight is determined based on the ratio between the color difference gradient intensity in the horizontal direction and the color difference gradient intensity in the vertical direction. The color difference weight ratio is determined based on the first weight, the second weight, the third weight, and the fourth weight.

6. The apparatus as claimed in claim 1, characterized in that, The green channel interpolation determination module is configured as follows: Determine the initial interpolation values ​​of the green channel in the horizontal direction and the initial interpolation values ​​of the green channel in the vertical direction for the target pixel; Determine the color gradient of the target pixel in the first channel in the horizontal direction and the color gradient of the first channel in the vertical direction; Based on the color gradient of the first channel of the target pixel in the horizontal direction, the initial interpolation of the green channel of the target pixel in the horizontal direction is corrected to obtain the green channel interpolation of the target pixel in the horizontal direction. Based on the color gradient of the first channel of the target pixel in the vertical direction, the initial interpolation of the green channel of the target pixel in the vertical direction is corrected to obtain the green channel interpolation of the target pixel in the vertical direction.

7. The apparatus as claimed in claim 6, characterized in that, The green channel interpolation determination module is configured as follows: Determine the adaptive weight of the target pixel, wherein the adaptive weight indicates the trend of change between the color gradient of the green channel of the neighboring pixels of the target pixel and the color gradient of the first channel of the neighboring pixels; The green channel interpolation of the target pixel in the horizontal direction is determined based on the initial interpolation of the green channel of the target pixel in the horizontal direction, the color gradient of the first channel of the target pixel in the horizontal direction, and the adaptive weight.

8. An image processing method, characterized in that, The method includes: Determine the green channel interpolation of the target pixel in the horizontal direction and the green channel interpolation in the vertical direction in the Bayer image, wherein the Bayer image includes the value of the first channel of the target pixel, and the first channel is either the blue channel or the red channel; Determine the overall weight of the target pixel in the horizontal direction and the overall weight in the vertical direction, wherein the overall weight indicates the overall situation of the color difference and color difference gradient of the target pixel in the corresponding direction; The green channel value of the target pixel is determined based on the green channel interpolation in the horizontal direction and the green channel interpolation in the vertical direction, the comprehensive weight of the target pixel in the horizontal direction and the comprehensive weight in the vertical direction. Based on the green channel value of the target pixel, the value of the second channel of the target pixel is determined; wherein, if the first channel is the blue channel, the second channel is the red channel, and if the first channel is the red channel, the second channel is the blue channel.

9. The method as described in claim 8, characterized in that, Determining the comprehensive weight of the target pixel in the horizontal direction and the comprehensive weight in the vertical direction includes: The overall color difference weights and overall color difference gradient weights corresponding to the pixels in the target window in the horizontal and vertical directions are determined respectively. The target window refers to the window centered on the target pixel in the Bayer image. The overall color difference weights indicate the degree of overall color deviation of the pixels in the target window in the corresponding direction, and the overall color difference gradient weights indicate the degree of overall change of color difference of the pixels in the target window in the corresponding direction. Determine the color difference weight ratio, which indicates the overall color difference and the degree of asymmetry in the overall color difference gradient of the pixels in the target window in the horizontal and vertical directions; Based on the overall color difference weight and overall color difference gradient weight in the horizontal direction, as well as the proportion of color difference weight, the comprehensive weight of the target pixel in the horizontal direction is determined. Based on the overall color difference weight and overall color difference gradient weight in the vertical direction, as well as the proportion of color difference weight, the comprehensive weight of the target pixel in the vertical direction is determined.

10. The method as described in claim 9, characterized in that, The determination of the overall color difference weights of pixels within the target window in the horizontal and vertical directions includes: Based on the color difference of multiple first pixels in the target window in the horizontal direction, the color difference intensity of the target window in the horizontal direction is determined, wherein the multiple first pixels have blue channel values ​​or red channel values ​​in the Bayer image; Based on the color difference of the plurality of first pixels in the vertical direction, the color difference intensity of the target window in the vertical direction is determined; The overall color difference weight in the horizontal direction is determined based on the color difference intensity in the horizontal direction, and the overall color difference weight in the vertical direction is determined based on the color difference intensity in the vertical direction.

11. The method as described in claim 10, characterized in that, The determination of the overall color difference gradient weights of pixels within the target window in the horizontal and vertical directions includes: Based on the color difference gradient of multiple second pixels in the target window in the horizontal direction, the color difference gradient intensity of the target window in the horizontal direction is determined, and the multiple second pixels have green channel values ​​in the Bayer image; Based on the color difference gradient of the plurality of second pixels in the vertical direction, the color difference gradient intensity of the target window in the vertical direction is determined; The overall color difference gradient weight in the horizontal direction is determined based on the color difference gradient intensity in the horizontal direction, and the overall color difference gradient weight in the vertical direction is determined based on the color difference gradient intensity in the vertical direction.

12. The method as described in claim 11, characterized in that, Determining the color difference weighting percentage includes: A first weight is determined based on the difference between the color difference intensity in the horizontal direction and the color difference intensity in the vertical direction; The second weight is determined based on the ratio between the color difference intensity in the horizontal direction and the color difference intensity in the vertical direction; The third weight is determined based on the difference between the color difference gradient intensity in the horizontal direction and the color difference gradient intensity in the vertical direction. The fourth weight is determined based on the ratio between the color difference gradient intensity in the horizontal direction and the color difference gradient intensity in the vertical direction. The color difference weight ratio is determined based on the first weight, the second weight, the third weight, and the fourth weight.

13. The method as described in claim 8, characterized in that, The determination of the green channel interpolation of the target pixel in the Bayer image in the horizontal direction and the green channel interpolation in the vertical direction includes: Determine the initial interpolation values ​​of the green channel in the horizontal direction and the initial interpolation values ​​of the green channel in the vertical direction for the target pixel; Determine the color gradient of the target pixel in the first channel in the horizontal direction and the color gradient of the first channel in the vertical direction; Based on the color gradient of the first channel of the target pixel in the horizontal direction, the initial interpolation of the green channel of the target pixel in the horizontal direction is corrected to obtain the green channel interpolation of the target pixel in the horizontal direction. Based on the color gradient of the first channel of the target pixel in the vertical direction, the initial interpolation of the green channel of the target pixel in the vertical direction is corrected to obtain the green channel interpolation of the target pixel in the vertical direction.

14. The method as described in claim 13, characterized in that, The step of correcting the initial interpolation of the green channel of the target pixel in the horizontal direction based on the color gradient of the first channel of the target pixel in the horizontal direction to obtain the interpolation of the green channel of the target pixel in the horizontal direction includes: Determine the adaptive weight of the target pixel, wherein the adaptive weight indicates the trend of change between the color gradient of the green channel of the neighboring pixels of the target pixel and the color gradient of the first channel of the neighboring pixels; The green channel interpolation of the target pixel in the horizontal direction is determined based on the initial interpolation of the green channel of the target pixel in the horizontal direction, the color gradient of the first channel of the target pixel in the horizontal direction, and the adaptive weight.

15. A display chip, characterized in that, The display chip includes the image processing apparatus as described in any one of claims 1-7.

16. A computer device, characterized in that, The computer device includes a memory and a processor, the memory being used to store computer programs, and the processor being used to execute the computer programs stored in the memory to implement the method according to any one of claims 8-14.

17. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 8-14.

Images