Image data processing method and image signal processor
By calculating and adjusting the local white balance gain value of the image before demosaicing, the problem of local illumination differences in the image is solved, and higher quality image reconstruction results are achieved.
Patent Information
- Application Number
- CN202511055658.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-28
- Publication Date
- 2025-11-11
AI Technical Summary
Existing technologies cannot effectively adapt to local lighting differences in images before demosaicing, resulting in color deviation and edge artifacts. Furthermore, the white balance algorithm and the demosaicing algorithm are strongly coupled, affecting image quality.
By extracting sub-channel data from the original image, calculating local white balance gain values, adjusting the white balance of each sub-image, and then performing de-mosaic processing, including weighted averaging and reciprocal adjustment of local white balance gain values, to optimize color compensation in local areas.
It improves the color reproduction and detail of images, reduces color distortion and artifacts, and enhances the accuracy and efficiency of de-mosaic processing.
Smart Images
Figure CN120935341A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of image data processing technology, and particularly relates to an image data processing method and an image signal processor. The image data processing method optimizes the local white balance gain value, Background Technology
[0002] In the field of digital image processing, demosaicing is a crucial step in generating color images using Bayer array sensors, and its performance directly affects image detail restoration and color accuracy. Demosaicing algorithms rely on a key premise: the input image has already undergone white balance correction to eliminate illumination color cast. However, in practical applications, the accuracy of white balance correction is often affected by factors such as the complexity of scene lighting and local reflection characteristics. If demosaicing is directly applied to uncorrected or insufficiently corrected images, the algorithm needs to calculate missing color information within a larger solution space, leading to color distortion, increased artifacts, and other problems that severely impact image quality.
[0003] To address the aforementioned issues, before de-mosaicing, a global white balance algorithm (such as the gray-world hypothesis or perfect reflection method) can be used to adjust the color gain of the original sensor data, eliminating global illumination color shifts and making the overall image tone more neutral. This separates the color correction and pixel interpolation steps, confining the de-mosaic problem to a normalized color space, thereby reducing algorithm complexity and improving color consistency during the interpolation process based on the assumption of "optimal global white balance."
[0004] However, global white balance correction struggles to adapt to localized lighting differences in images, failing to compensate for localized color casts. This results in color deviations or edge artifacts in the demosaiced areas. Furthermore, the strong coupling between white balance and demosaic algorithms means that the color distribution characteristics output by the white balance algorithm may violate the color continuity assumptions required for the demosaic process, thus limiting the quality of image reconstruction. Summary of the Invention
[0005] In a first aspect, embodiments of this application provide an image data processing method, including:
[0006] Extract sub-channel data from the original image. The sub-channel data contains the color value of each pixel in the original image. The original image consists of at least two sub-images.
[0007] The local white balance gain value corresponding to the sub-image is calculated based on the sub-channel data; the local white balance gain value is determined according to the color value of each pixel in the sub-image.
[0008] The white balance image is obtained by adjusting the color values of pixels based on the local white balance gain value.
[0009] Perform demosaic processing on the white balance image to obtain the target image.
[0010] In some feasible embodiments, it also includes:
[0011] The original image is divided into sub-images according to a preset size; the sub-images have the same image size; the sub-images have a smaller image size than the original image.
[0012] In some feasible embodiments, obtaining the white balance gain value corresponding to the sub-image includes:
[0013] Based on the sub-channel data corresponding to the sub-image, the color values of the pixels in the sub-image are summed to obtain the first color sum value, the second color sum value, and the third color sum value.
[0014] The local white balance gain value of the sub-image is determined based on the ratio between the total color values corresponding to the sub-images; the local white balance gain value includes a first local white balance gain value associated with a first color sum value, a second local white balance gain value associated with a second color sum value, and a third local white balance gain value associated with a third color sum value.
[0015] In some feasible embodiments, summing the color values of pixels in the sub-image based on the sub-channel data corresponding to the sub-image includes:
[0016] Obtain the color threshold;
[0017] If the color value of a pixel is greater than the color threshold, then the color value of the pixel is discarded.
[0018] If the color value of a pixel is less than or equal to the color threshold, the total color value is calculated based on the color value of the pixel.
[0019] In some feasible embodiments, determining the local white balance gain value corresponding to a sub-image based on the ratio between the total pixel values corresponding to the sub-images further includes:
[0020] Get the number of valid pixels; valid pixels are the pixels that participate in the calculation of the total pixel value;
[0021] The local white balance gain coefficient is determined based on the number of effective pixels and the effective pixel number threshold.
[0022] The local white balance gain value is determined by multiplying the local white balance gain value coefficient and the ratio.
[0023] In some feasible embodiments, determining the local white balance gain value corresponding to a sub-image based on the ratio between the total color values corresponding to the sub-images further includes:
[0024] Obtain the local white balance threshold of the sub-image; the local white balance threshold is determined based on the global white balance gain value; the global white balance gain value is determined based on the color values of the pixels in the original image;
[0025] If the local white balance gain value is greater than the local white balance threshold, then the local white balance threshold is determined as the local white balance gain value.
[0026] In some feasible embodiments, determining the local white balance gain value corresponding to a sub-image based on the ratio between the total color values corresponding to the sub-images further includes:
[0027] Obtain the first local white balance gain value of the first sub-image and the second local white balance gain value of the second sub-image; the first sub-image and the second sub-image are adjacent;
[0028] The weighted average of the first local white balance gain value and the second local white balance gain value is calculated to obtain the local white balance gain value.
[0029] In some feasible embodiments, the original image includes at least the previous frame image and the current frame image; determining the local white balance gain value corresponding to the sub-image based on the ratio between the total color values corresponding to the sub-images further includes:
[0030] Obtain the previous local white balance gain value of the preceding frame image and the current local white balance gain value of the current frame image; the preceding frame image and the current frame image are adjacent frame images;
[0031] The local white balance gain value is obtained by calculating the weighted average of the previous local white balance gain value and the current local white balance gain value.
[0032] In some feasible embodiments, after performing demosaic processing on the white balance image, the process further includes:
[0033] The target image is obtained by adjusting the color values of the demosaic pixels based on the reciprocal of the local white balance gain value.
[0034] Secondly, embodiments of this application provide an image signal processor, including: a preprocessing module, a calculation module, and a demosaic module;
[0035] The preprocessing module is used to extract sub-channel data from the original image. The sub-channel data contains the color value of each pixel in the original image; the original image includes at least two sub-images.
[0036] The calculation module is used to calculate the local white balance gain value corresponding to the sub-image based on the sub-channel data; the local white balance gain value is determined according to the color value of each pixel in the sub-image;
[0037] The demosaic module is used to adjust the color values of pixels based on local white balance gain values to obtain a white balance image;
[0038] The demosaic module is also used to perform demosaic processing on white balance images to obtain the target image.
[0039] As described above, this application provides an image data processing method and an image signal processor. The image signal processor extracts sub-channel data from the original image and determines the local white balance gain value corresponding to each sub-image in the original image based on the sub-channel data. Then, it adjusts the color values of pixels in each sub-image according to the local white balance gain values to obtain a white balance image. Finally, it performs demosaic processing on the white balance image to obtain the target image. The image signal processor optimizes each sub-image one by one based on the calculation of local white balance gain values to solve the problem of poor optimization in local areas, thereby improving its adaptability to the demosaic process. Attached Figure Description
[0040] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0041] Figure 1 This is a flowchart of image data processing provided in an embodiment of this application;
[0042] Figure 2 A flowchart of local white balance gain calculation provided in an embodiment of this application;
[0043] Figure 3 A flowchart of high-saturation pixel filtering provided in an embodiment of this application;
[0044] Figure 4 A flowchart for adjusting local white balance gain based on the number of effective pixels provided in this application embodiment;
[0045] Figure 5 A flowchart illustrating the adjustment of a local white balance threshold based on a global white balance threshold, provided in an embodiment of this application.
[0046] Figure 6 This is a flowchart illustrating the weighted adjustment of local white balance gain values provided in an embodiment of this application.
[0047] Figure 7 This is a flowchart illustrating the optimization of local white balance gain values based on adjacent frame images provided in an embodiment of this application.
[0048] Figure 8 This is a schematic diagram of the image signal processor structure. Detailed Implementation
[0049] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.
[0050] Image sensors can employ Bayer filter arrays, meaning each pixel acquired by the sensor records only one color: red (R), green (G), or blue (B). This results in a mosaic effect in the data acquired by the image sensor, where each pixel contains only a single color. Therefore, an image signal processor can incorporate a de-mosaic module to process the data acquired by the image sensor in a de-mosaic manner, thereby achieving image reconstruction and improving image quality.
[0051] To improve the demosaic effect, a white balance module can be set before the demosaic module. This allows the white balance module to optimize the proportions of various colors in the image before the demosaic module performs image reconstruction. However, the white balance module and the demosaic module are highly coupled; that is, the white balance algorithm provided by the white balance module and the demosaic algorithm provided by the demosaic module need to have a high degree of compatibility, otherwise it will be difficult to improve image quality.
[0052] To solve the above problems, such as Figure 1 As shown, this application provides an image data processing method, the method comprising:
[0053] S100: Extract sub-channel data from the original image. The sub-channel data contains the color value of each pixel in the original image.
[0054] In some embodiments, the sub-channel data of the original image is the color data of the pixels acquired by the image sensor. That is, the R channel, G channel, and B channel color data can be extracted from the original image; these color data represent the color value corresponding to each pixel in the original image. The R channel, G1 channel, G2 channel, and B channel can also be extracted from the original image. In other words, the sub-channel data includes two data channels used to describe green.
[0055] The extracted sub-channel data can be processed to reduce data volume and improve processing efficiency. It can also effectively filter noise and artifacts in the image while preserving image details, thus improving the efficiency and quality of subsequent image processing.
[0056] Understandably, due to the acquisition characteristics of the Bayer filter array, the color value of each pixel in the original image corresponds to only one color. Therefore, based on different color values, the white balance gain value corresponding to each pixel can be determined, thus adjusting the white balance of the original image.
[0057] S200: Obtain the local white balance gain value corresponding to the sub-image.
[0058] In some embodiments, the original image can be divided into multiple sub-images of the same size. That is, the original image is divided according to a preset size to obtain sub-images. The sub-images have the same image size, and the image size of each sub-image is smaller than the image size of the original image.
[0059] The size division of sub-images can be adaptively adjusted according to the complexity of the image content to ensure that each sub-image contains sufficient information. The specific division sizes include, but are not limited to: 32×32, 16×16, 8×8, 4×4, 2×2, 32×24, and 16×12.
[0060] Based on the division of sub-images, white balance adjustment can be performed on each sub-image separately, which requires obtaining the local white balance gain value corresponding to the sub-image.
[0061] Understandably, local white balance adjustments can more accurately compensate for color temperature differences in each sub-image, thereby improving the overall image's color reproduction and detail, resulting in a more refined image processing effect. This is beneficial for improving the accuracy and efficiency of subsequent de-mosaic processing.
[0062] The local white balance gain value is determined based on the color value of each pixel in the sub-image. Furthermore, the types of local white balance gain values are adapted to the types of color values. For example, if a sub-image contains three color values, then the corresponding local white balance gain values for that sub-image are also three, adjusting the gain of red, green, and blue colors separately to ensure accurate compensation for each color and avoid color deviation. Through this meticulous local adjustment, the image details and color transitions become more natural, providing higher-quality input data for subsequent de-mosaic processing and improving the overall image quality.
[0063] S300: Adjusts the color value of the pixel based on the local white balance gain value to obtain a white balance image.
[0064] In some embodiments, after obtaining the local white balance gain value, the color value of the pixel can be adjusted using the white balance gain value. For example, the white balance gain can be multiplied by the color value of the pixel to compensate for the color temperature of each sub-image. As a result, after each sub-image has undergone local white balance correction, the overall image quality will be improved.
[0065] Furthermore, adjusting the image quality of each sub-image using local white balance helps avoid significant color differences between local images and other parts of the image in extreme scenes. For example, when the lighting conditions of a local image are abnormal, or in scenes with uniform color values, local white balance processing can provide better correction results for that local image.
[0066] S400: Performs demosaic processing on the white balance image to obtain the target image.
[0067] In some embodiments, a white balance image can be obtained after white balancing the original image, and the white balance image can be de-mosaiced to perform full color reconstruction of the original image, thereby improving image quality.
[0068] like Figure 2 As shown, the local white balance gain value of a sub-image can be calculated based on the color values of pixels in the sub-image. The steps include:
[0069] The color values of pixels in the sub-image are summed based on the sub-channel data corresponding to the sub-image to obtain the first color sum, the second color sum, and the third color sum.
[0070] The local white balance gain value for a sub-image is determined based on the ratio between the total color values corresponding to the sub-images. The local white balance gain value includes a first local white balance gain value associated with a first color sum value, a second local white balance gain value associated with a second color sum value, and a third local white balance gain value associated with a third color sum value.
[0071] In some embodiments, the sub-channel data corresponding to the sub-image is used to provide the color values of the pixels in the sub-image. Therefore, the color sum and value of the pixels in the sub-image can be calculated based on the color values provided by the sub-channel data, namely, the first color sum, the second color sum, and the third color sum. The first color sum can be the red value and r_sum, the second color sum can be the green value and g_sum, and the third color sum can be the blue value and b_sum.
[0072] After obtaining the sum of each color and value in each sub-image, the local white balance gain value corresponding to the sub-image can be determined by calculating the ratios between these sums, thereby precisely adjusting the color temperature of each sub-image. For example, when calculating the local white balance gain value for the red value, the ratios of the sum of red, green, and blue values to the sum of red values can be calculated separately, and the maximum value among these three ratios can be taken as the local white balance gain value corresponding to the red value, i.e., the first local white balance gain value in the sub-image. Similarly, the second local white balance gain value corresponding to the green value and the third local white balance gain value corresponding to the blue value can be obtained in the same way.
[0073] Thus, taking the maximum sum of red values in a sub-image as an example, it indicates that the red in the sub-image is too strong. Therefore, based on the maximum sum of red values, the first local white balance gain value can be 1, indicating that no enhancement or reduction of red is applied. Furthermore, the second and third local white balance gain values will both be greater than 1, indicating that white balance correction can enhance the green and blue components in the sub-image, thereby making the sub-image more color-balanced and improving the overall visual effect.
[0074] like Figure 3 As shown, when there are pixels in a sub-image whose color values differ significantly from other pixel values, these pixels need to be processed to prevent them from affecting the overall color balance of the image. The steps include:
[0075] Get the color threshold.
[0076] If the color value of a pixel is greater than the color threshold, the color value of the pixel is discarded.
[0077] If the color value of a pixel is less than or equal to the color threshold, the total color value is calculated based on the color value of the pixel.
[0078] In some embodiments, a color threshold can be preset to avoid interference from highly saturated pixels on the overall color balance of the image.
[0079] For example, during the pixel value summation process, it can be first determined whether the color value of a pixel exceeds a color threshold. If the color value is less than the color threshold, the pixel can be considered a valid pixel, and its pixel value can be added to the total color value to ensure the accuracy of the calculation.
[0080] For example, if during the pixel value calculation process, it is determined that the color value of a pixel is greater than the color threshold, then the pixel can be treated as an invalid pixel and its pixel value can be discarded to avoid introducing the high saturation features of the pixel into the white balance gain process and the image.
[0081] In this way, by selecting high-saturation pixels, the efficiency of calculating local white balance gain values can be effectively improved, and the effect of white balance correction can be further enhanced, which in turn helps to improve image quality.
[0082] like Figure 4 As shown, the local white balance gain value can be dynamically adjusted based on the number of effective pixels. The steps include:
[0083] Get the number of valid pixels. Valid pixels are the pixels that participate in the calculation of the total pixel value.
[0084] The local white balance gain coefficient is determined based on the number of effective pixels and the effective pixel number threshold.
[0085] The local white balance gain value is determined by multiplying the local white balance gain value coefficient and the ratio.
[0086] In some embodiments, a local white balance gain value obtained under conditions of sufficient effective pixels signifies an effective adjustment to the sub-image. Conversely, a local white balance gain value obtained under conditions of insufficient effective pixels signifies an ineffective adjustment to the sub-image and may further negatively impact image quality.
[0087] Therefore, when determining the local white balance gain value, the calculated local white balance gain value can also be dynamically adjusted by setting an effective pixel count threshold.
[0088] For example, the ratio between the effective pixel count threshold and the effective pixel count can be used as the adjustment coefficient for the local white balance gain value, thereby making the local white balance gain value closer to the actual image requirements and avoiding color distortion caused by over-adjustment.
[0089] In other embodiments, if there are no valid pixels in a sub-image, the local white balance gain value can be set to 1 by default to avoid affecting the image quality of the sub-image due to unnecessary adjustments.
[0090] like Figure 5 As shown, after obtaining the local white balance gain value based on the ratio of each color and value, the local white balance gain value can be further optimized to improve the white balance effect. The steps include:
[0091] Obtain the local white balance threshold of the sub-image. The local white balance threshold is determined based on the global white balance gain value. The global white balance gain value is determined based on the color values of pixels in the original image.
[0092] If the local white balance gain value is greater than the local white balance threshold, then the local white balance threshold is determined as the local white balance gain value.
[0093] In some embodiments, a global white balance gain value corresponding to the original image can also be calculated based on the sub-channel data. The global white balance gain value is calculated based on the color values of pixels in the original image. Furthermore, when obtaining the local white balance gain value corresponding to each sub-image, the local white balance gain value can be adjusted using the global white balance gain value to avoid over-correction of the sub-images affecting the color balance of the overall image.
[0094] For example, the global white balance gain value can be used as a local white balance threshold. When the local white balance threshold is greater than the global white balance gain value, the local white balance gain is considered too high, and the global white balance gain value can be directly used as the local white balance threshold, avoiding over-correction and ensuring overall color balance. This optimization not only improves the accuracy of local white balance but also enhances the overall visual effect of the image, making the colors more natural and harmonious.
[0095] For example, a floating range can be set based on the global white balance gain value, within which the local white balance gain value can be dynamically adjusted to ensure that the correction effect is both accurate and consistent with the overall image. The floating range can be flexibly adjusted according to the image characteristics to adapt to different scene requirements and further improve image quality.
[0096] It should be noted that the embodiments of this application do not limit the setting method of the local white balance threshold. The embodiments of this application aim to dynamically adjust the local white balance threshold based on the global white balance threshold, so that the local white balance effect of the sub-image does not deviate from the color balance framework of the overall image. Furthermore, through the coordinated optimization of global and local values, it is ensured that the white balance correction of each sub-image is both accurate and consistent with the overall color tone, thereby achieving true color reproduction of the image.
[0097] like Figure 6 As shown, smoothing can also be performed based on the local white balance gain values of adjacent sub-images to reduce color abrupt changes between adjacent regions. The steps include:
[0098] Obtain the first local white balance gain value of the first sub-image and the second local white balance gain value of the second sub-image. The first sub-image and the second sub-image are adjacent.
[0099] The weighted average of the first local white balance gain value and the second local white balance gain value is calculated to obtain the local white balance gain value.
[0100] In some embodiments, after calculating the local white balance gain value of each sub-image, the local white balance gain value for correcting adjacent sub-images can be determined based on the local white balance gain value of each sub-image.
[0101] For example, weights can be set for the first local white balance gain value of the first sub-image and the second local white balance gain value of the second sub-image, and weighted calculations can be performed in combination with the weights to obtain the local white balance gain values used to correct the first and second sub-images.
[0102] When setting weights, the characteristics of pixel values in the sub-images can be considered. For example, when the color balance of the first and second sub-images is similar, the weight can be set to 0.5 to ensure a smooth color transition between the two sub-images. Conversely, when the color balance of the first and second sub-images differs significantly, the weight of the sub-image with higher color balance can be increased to enhance its color correction effect. This weight allocation strategy not only effectively reduces color differences between adjacent sub-images but also maintains overall image color consistency, further improving image quality.
[0103] This method of adjusting the local white balance gain value can make the color transition between adjacent sub-images more natural. Even when the adjacent sub-images themselves have a large color difference, a smooth color transition can be achieved through reasonable weight allocation, avoiding obvious color block segmentation and ensuring the natural harmony of the overall image color.
[0104] like Figure 7 As shown, the local white balance gain value can also be optimized based on multiple frames that make up the original image. The steps include:
[0105] Obtain the previous local white balance gain value of the preceding frame image and the current local white balance gain value of the current frame image. The preceding frame image and the current frame image are adjacent frame images.
[0106] The local white balance gain value is obtained by calculating the weighted average of the previous local white balance gain value and the current local white balance gain value.
[0107] In some embodiments, the original image can be processed frame by frame to achieve better processing results. During frame-by-frame processing, the local white balance gain value of each frame can be recorded as a reference. Furthermore, by comparing the gain value changes between consecutive frames, the weights are dynamically adjusted to ensure smooth color transitions between frames.
[0108] For example, when obtaining the local white balance gain value of the current frame, the local white balance gain value of the previous frame can be obtained. Then, by using weight allocation and weighted calculation, the gain values of the previous frame and the current frame can be combined to improve the quality of the local white balance gain value and make the image color more balanced.
[0109] In other embodiments, when adjusting the local white balance gain of the current frame in conjunction with the previous frame, the local white balance gain values of multiple previous frames can be used for multi-level weighted calculation to improve the effect of the local white balance gain of the current frame.
[0110] In this way, in scenarios where the lighting conditions for image acquisition are dynamically changing, the local white balance gain value can be optimized based on a multi-level weighting method, effectively addressing color deviations caused by changes in lighting and ensuring that the image can maintain true color reproduction under different lighting conditions.
[0111] It is understandable that, given the local white balance gain value of the original image obtained based on the above embodiments, the local white balance gain value can be used to perform multiple local white balance corrections on the original image to obtain a white balance image. This approach, based on the characteristics of local correction, prevents color distortion that may result from global white balance correction, ensuring accurate color reproduction in each region, thereby improving the subsequent de-mosaic processing effect.
[0112] In some embodiments, after performing demosaic processing on a white balance image, the color values of the demosaic pixels can be adjusted based on the reciprocal of the local white balance gain value to obtain the target image.
[0113] In particular, when adjusting the color value of demosaic pixels based on the reciprocal of the local white balance gain value, the adjustment can also be made on a sub-image basis.
[0114] It's important to note that while local white balance correction can effectively correct the effects of lighting color casts on images, the pursuit of improved local white balance results in some color temperature deviations from the real scene that occur during the demosaicing process. Therefore, after demosaicing the white balance image, it's necessary to restore the image to its true colors using the reciprocal of the local white balance gain value to avoid global color casts.
[0115] like Figure 8 As shown in the figure, this application provides an image signal processor, including: a preprocessing module, a calculation module, and a de-mosaic module.
[0116] The preprocessing module is used to extract sub-channel data from the original image. The sub-channel data contains the color value of each pixel in the original image. The original image consists of at least two sub-images.
[0117] The calculation module is used to obtain the local white balance gain value corresponding to the sub-image. The local white balance gain value is determined based on the color value of each pixel in the sub-image.
[0118] The demosaic module is used to adjust the color values of pixels based on local white balance gain values to obtain a white balance image.
[0119] The demosaic module is also used to perform demosaic processing on white balance images to obtain the target image.
[0120] In some embodiments, the preprocessing module can be a hardware circuit integrated into the image signal processor to perform efficient data extraction and processing based on the hardware circuit, ensuring real-time performance.
[0121] The computing module can be a hardware-software co-processor (firmware) module in the image signal processor. By combining software algorithms with hardware acceleration, it can achieve fast and accurate calculation of local white balance gain values, thereby further improving image processing efficiency.
[0122] The demosaic module can also be a dedicated hardware unit integrated into the image signal processor, utilizing hardware acceleration technology to efficiently complete the demosaic process. Furthermore, the demosaic module can integrate white balance processing functionality; that is, the demosaic module can first perform white balance correction on the original image based on the white balance gain value provided by the computation module, and then perform demosaic processing, thereby ensuring that the target image achieves optimal results in color reproduction and detail rendering.
[0123] This application provides an image data processing method and an image signal processor. The image signal processor extracts sub-channel data from the original image and determines the local white balance gain value corresponding to each sub-image in the original image based on the sub-channel data. Then, it adjusts the color values of pixels in each sub-image according to the local white balance gain values to obtain a white balance image. Finally, it performs demosaic processing on the white balance image to obtain the target image. The image signal processor optimizes each sub-image one by one based on the calculation of local white balance gain values to solve the problem of poor optimization in local areas, thereby improving its adaptability to the demosaic process and improving image processing quality.
[0124] Similar parts between the embodiments provided in this application can be referred to mutually. The specific implementation methods provided above are only a few examples under the overall concept of this application and do not constitute a limitation on the scope of protection of this application. For those skilled in the art, any other implementation methods extended from the solution of this application without creative effort shall fall within the scope of protection of this application.
Claims
1. An image data processing method, characterized in that, include: Sub-channel data is extracted from the original image, the sub-channel data containing the color value of each pixel in the original image; the original image comprises at least two sub-images. Calculate the local white balance gain value corresponding to the sub-image based on the sub-channel data; The local white balance gain value is determined based on the color value of each pixel in the sub-image; The color values of the pixels are adjusted based on the local white balance gain value to obtain a white balance image; The white balance image is subjected to demosaic processing to obtain the target image.
2. The method according to claim 1, characterized in that, Also includes: The original image is divided into sub-images according to a preset size; the sub-images have the same image size; the image size of the sub-images is smaller than the image size of the original image.
3. The method according to claim 2, characterized in that, The step of obtaining the white balance gain value corresponding to the sub-image includes: Based on the sub-channel data corresponding to the sub-image, the color values of the pixels in the sub-image are summed to obtain a first color sum value, a second color sum value, and a third color sum value; The local white balance gain value corresponding to the sub-image is determined based on the ratio between the total color values corresponding to the sub-image; the local white balance gain value includes a first local white balance gain value associated with the first color sum value, a second local white balance gain value associated with the second color sum value, and a third local white balance gain value associated with the third color sum value.
4. The method according to claim 3, characterized in that, The summation of the color values of pixels in the sub-image based on the sub-channel data corresponding to the sub-image includes: Obtain the color threshold; If the color value of the pixel is greater than the color threshold, then the color value of the pixel is discarded. If the color value of the pixel is less than or equal to the color threshold, then the total color value is calculated based on the color value of the pixel.
5. The method according to claim 3, characterized in that, The step of determining the local white balance gain value corresponding to the sub-image based on the ratio between the total pixel values corresponding to the sub-image further includes: Obtain the number of valid pixels; the valid pixels are the pixels that participate in the calculation of the total pixel value; The local white balance gain coefficient is determined based on the number of effective pixels and the effective pixel number threshold. The local white balance gain value is determined based on the product of the local white balance gain value coefficient and the ratio.
6. The method according to claim 5, characterized in that, The step of determining the local white balance gain value corresponding to the sub-image based on the ratio between the total color values corresponding to the sub-image further includes: Obtain the local white balance threshold of the sub-image; the local white balance threshold is determined based on the global white balance gain value; the global white balance gain value is determined based on the color values of pixels in the original image; If the local white balance gain value is greater than the local white balance threshold, then the local white balance threshold is determined to be the local white balance gain value.
7. The method according to claim 5, characterized in that, The step of determining the local white balance gain value corresponding to the sub-image based on the ratio between the total color values corresponding to the sub-image further includes: Obtain the first local white balance gain value of the first sub-image and the second local white balance gain value of the second sub-image; the first sub-image and the second sub-image are adjacent; The weighted average of the first local white balance gain value and the second local white balance gain value is calculated to obtain the local white balance gain value.
8. The method according to claim 5, characterized in that, The original image includes at least the previous frame image and the current frame image; the step of determining the local white balance gain value corresponding to the sub-image based on the ratio between the total color values corresponding to the sub-image further includes: Obtain the preceding local white balance gain value of the preceding frame image and the current local white balance gain value of the current frame image; the preceding frame image and the current frame image are adjacent frame images; The local white balance gain value is obtained by calculating the weighted average of the preceding local white balance gain value and the current local white balance gain value.
9. The method according to claim 1, characterized in that, After performing demosaic processing on the white balance image, the process further includes: The target image is obtained by adjusting the color values of the demosaic pixels based on the reciprocal of the local white balance gain value.
10. An image signal processor, characterized in that, include: The module includes a preprocessing module, a computation module, and a de-mosaic module. The preprocessing module is used to extract sub-channel data from the original image, the sub-channel data containing the color value of each pixel in the original image; the original image includes at least two sub-images; The calculation module is used to calculate the local white balance gain value corresponding to the sub-image based on the sub-channel data; the local white balance gain value is determined according to the color value of each pixel in the sub-image; The demosaic module is used to adjust the color value of the pixel based on the local white balance gain value to obtain a white balance image; The demosaic module is also used to perform demosaic processing on the white balance image to obtain the target image.