White balance correction methods, systems, devices, and media for Bayer images
By generating overexposure markers in Bayer images and filtering out valid pixels, and calculating white balance correction coefficients, the problem of overexposure pixels affecting the correction effect is solved, thus improving the image processing efficiency of the FPGA platform.
Patent Information
- Application Number
- CN202611232864.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-14
- Publication Date
- 2026-11-17
AI Technical Summary
Existing white balance correction schemes for Bayer images are difficult to adapt to resource-constrained FPGA embedded platforms, resulting in low image processing efficiency and overexposed pixels affecting the correction effect.
By generating a 1-bit overexposure flag based on a preset overexposure threshold, the system iterates through each row starting from the third row of pixels, filters out valid pixels, calculates the white balance correction coefficient, and caches the flag in the FPGA instead of the original pixel data, thus reducing RAM requirements.
It effectively avoids the impact of overexposed pixels on the correction coefficient, reduces the cache space requirement in the white balance processing, improves image processing efficiency, and is compatible with FPGA motherboards for industrial cameras with limited resources.
Smart Images

Figure CN123123736A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing, and in particular relates to a white balance correction method, system, device and medium for Bayer images. Background Technology
[0002] In industrial visual inspection and machine vision imaging scenarios, the raw images output by industrial cameras are mostly Bayer images in RGGB format. Due to factors such as ambient lighting and camera sensor characteristics, the aforementioned Bayer images are prone to color cast issues. It is necessary to eliminate color cast through white balance correction to ensure image color consistency and provide a reliable foundation for subsequent image processing (such as depigmentation, feature extraction, and defect detection).
[0003] Considering the frequent occurrence of strong light and backlighting in industrial settings, some pixels in the generated Bayer images are overexposed (pixel values reach the sensor's maximum output value), and these overexposed pixels lack effective color information. To avoid overexposed pixels affecting white balance correction, a technical solution for suppressing locally overexposed pixels has been proposed in existing technologies.
[0004] However, the aforementioned existing white balance correction schemes for suppressing overexposed pixels are difficult to adapt to resource-constrained FPGA embedded platforms, thus reducing image processing efficiency.
[0005] Therefore, there is an urgent need for a white balance correction scheme for Bayer images to solve the aforementioned problems. Summary of the Invention
[0006] This application proposes a white balance correction method, system, device, and medium for Bayer images, which is adapted to the FPGA platform to improve image processing efficiency while suppressing overexposed pixels.
[0007] To achieve the above objectives, this application proposes the following technical solutions: In a first aspect of this application, a white balance correction method for Bayer images is provided, comprising: Based on a preset overexposure threshold, starting from the second row of pixels, a flag indicating whether each pixel is overexposed is generated row by row; the flag indicating whether a pixel is overexposed is represented by a 1-bit character. Starting from the third row of pixels, traverse row by row. For each row of pixels, using two pixel columns as the step size, check whether the flags of the four adjacent pixels in the same column of the current pixel row and the previous pixel row are not overexposed. If so, obtain two pixels from the four adjacent pixels as valid pixels. If not, directly check whether the flags of the four adjacent pixels in the next step size are not overexposed. The total number of pixel rows in the Bayer image is not less than 4. The pixel values of the effective pixels in the four different component channels of the Bayer image are superimposed to obtain the total pixel value of each component channel; From the four different component channels, the total pixel value with the largest value is selected as the reference value, and the ratio of the reference value to the total pixel value in each component channel is calculated as the white balance correction coefficient of the corresponding component channel. Calculate the product of each pixel and the white balance correction coefficient in its corresponding component channel, and use this product as the white balance corrected pixel value.
[0008] Optionally, after calculating the product of each pixel and the white balance correction coefficient in its corresponding component channel as the white balance corrected pixel value, the white balance correction method further includes: If the pixel value after white balance correction exceeds the preset overexposure threshold, the pixel value after white balance correction will be truncated to the overexposure threshold.
[0009] Optionally, in calculating the ratio of the reference value to the superposition result in each component channel as the white balance correction coefficient for the corresponding component channel, the white balance correction method further includes: If the total pixel value of a certain component channel is 0, then the white balance correction factor for that component channel is set to 1.
[0010] Optionally, based on a preset overexposure threshold, starting from the second row of pixels, a flag indicating whether each pixel is overexposed is generated row by row, including: Receive the data stream of pixel values from the Bayer image line by line; Based on a preset overexposure threshold, the value of each received pixel is compared to generate an overexposure flag; if the pixel value of a certain pixel is above the overexposure threshold, an overexposure flag for that pixel is generated.
[0011] Optionally, starting from the third row of pixels, traverse row by row. For each row of pixels, using two pixel columns as the step size, sequentially check whether the flags of the four adjacent pixels in the same column of the current and previous rows are not overexposed, including: Starting from the second row of pixels, the flags of each row of pixels are cached row by row based on the preset RAM; wherein, when the flag of the i-th row of pixels is input into the preset RAM, the flags of the pixels in the same column of the (i-1)-th row are read from the preset RAM; 3≤i≤n, where n represents the total number of pixel rows in the Bayer image; The flags of the (i-1)th row are read from the preset RAM and cached in the register. The pixel values and flags of the pixels in the same column of the i-th row are obtained and cached in the register. The column positions of the flags cached in the register are adjacent and the total number is not less than 2. The pixels in the same column refer to the pixels in the same column position as the flags read from the previous row. Using two pixel columns as the step size, retrieve the flags of four adjacent pixels in the same column of the i-th row and the (i-1)-th row from the register; Determine whether the markers of four adjacent pixels are not overexposed.
[0012] Optionally, before superimposing the pixel values of the effective pixels in the four different component channels of the Bayer image to obtain the total pixel value of each component channel, the white balance correction method further includes: Based on the parity of the row number in the current pixel row, the component channels corresponding to the two pixels in the current pixel row are determined; among them, the four different component channels are divided into R channel, Gr channel, B channel and Gb channel; Gr channel represents the G channel adjacent to R channel, and Gb channel represents the G channel adjacent to B channel.
[0013] Optionally, the white balance correction method is executed by an FPGA, based on a pipelined processing mode, and implemented in parallel using hardware circuitry within the FPGA.
[0014] In a second aspect of this application, a white balance correction system for Bayer images is provided, comprising: The flag generation module is used to generate a flag indicating whether each pixel is overexposed, starting from the second row of pixels, based on a preset overexposure threshold; the flag indicating whether a pixel is overexposed is represented by a 1-bit character. The effective pixel acquisition module is used to traverse row by row starting from the third row of pixels. For each row of pixels, it uses two pixel columns as the step size to determine whether the flags of the four adjacent pixels in the same column of the current pixel row and the previous pixel row are not overexposed. If so, it acquires two pixels from the four adjacent pixels as effective pixels. If not, it directly determines whether the flags of the four adjacent pixels in the next step size are not overexposed. The total number of pixel rows in the Bayer image is not less than 4. The pixel total value acquisition module is used to superimpose the pixel values of the effective pixels in the four different component channels of the Bayer image to obtain the pixel total value of each component channel. The coefficient calculation module is used to select the total pixel value with the largest value from four different component channels as the reference value, and calculate the ratio of the reference value to the total pixel value in each component channel as the white balance correction coefficient of the corresponding component channel. The pixel value correction module is used to calculate the product of each pixel and the white balance correction coefficient in its component channel, which is then used as the white balance corrected pixel value.
[0015] In a third aspect of this application, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory are connected via the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the white balance correction method described in any of the first aspects.
[0016] In a fourth aspect of this application, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a processor, the computer program implements the white balance correction method described in any of the first aspects.
[0017] The beneficial effects of this application are as follows: A white balance correction method for Bayer images includes: generating an overexposed flag for each pixel, starting from the second row of pixels, based on a preset overexposure threshold; wherein the overexposed flag is represented by a 1-bit character; traversing row by row starting from the third row of pixels, for each row of pixels, determining whether the flags of the four adjacent pixels in the same column of the current pixel row and the previous pixel row are not overexposed, if so, obtaining two pixels from the four adjacent pixels as valid pixels; if not, directly determining whether the flags of the four adjacent pixels in the next step are not overexposed; superimposing the pixel values of the valid pixels in the four different component channels of the Bayer image to obtain the total pixel value of each component channel; selecting the largest total pixel value from the four different component channels as a reference value, and calculating the ratio of the reference value to the total pixel value in each component channel as the white balance correction coefficient of the corresponding component channel; calculating the product of each pixel and the white balance correction coefficient of its component channel as the white balance corrected pixel value.
[0018] Based on the above processing, this application starts from the third row of pixels in the image, and uses the two pixels in the current row and the two pixels in the same column position of the previous row as a check group. Only when all four pixels are not overexposed are the two pixels in the current row obtained as valid pixels, which are then used to calculate the white balance correction coefficient. Thus, by filtering the valid pixels, the influence of overexposed pixels on the correction coefficient can be avoided, thus avoiding the risk of the white balance correction coefficient being easily distorted.
[0019] Meanwhile, this application uses a flag indicating whether an image is overexposed to determine the validity of pixels. Compared to the existing method of directly caching pixel values, this application effectively reduces the cache space requirement during white balance processing. Taking an 8-bit pixel depth as an example, compared to a 1-bit flag, this application can reduce the cache space requirement for the previous row of pixels to one-eighth of the original, significantly saving FPGA resources. It can be directly adapted to FPGA motherboards of industrial cameras with limited resources, solving the limitation of high hardware resource consumption in existing technologies and improving image processing efficiency. Attached Figure Description
[0020] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating a white balance correction method provided in this application; Figure 2 This is a schematic diagram of the hardware implementation of a white balance correction method provided in this application; Figure 3 This is a schematic diagram of the structure of a white balance correction system provided in this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation
[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this application, but not all embodiments.
[0022] Due to factors such as ambient lighting and camera sensor characteristics, images are prone to color casts, which need to be eliminated through white balance correction. Existing white balance correction algorithms mostly calculate the correction coefficient by statistically analyzing the channel mean of all pixels in the image. However, industrial scenes often involve strong light and backlighting, causing some pixels to be overexposed. Overexposed pixels do not contain effective color information; including them in the statistics will distort the calculated channel mean during white balance correction, leading to deviations in the calculated correction coefficient and ultimately affecting the white balance correction effect.
[0023] To avoid overexposed pixels affecting white balance correction, existing technologies have proposed techniques to suppress locally overexposed pixels. For example, the paper "A Local Overexposure Suppression Downsampling White Spot Algorithm for Video Investigation" discloses a downsampling white spot algorithm based on local overexposure suppression, which can be used for color cast correction of video frames or single images.
[0024] Existing white balance correction schemes for suppressing overexposed pixels are difficult to adapt to resource-constrained FPGA embedded platforms, reducing image processing efficiency. For example, in the field of machine vision imaging, industrial cameras often require real-time image processing. To achieve cross-line pixel verification within an FPGA, it is usually necessary to cache the original pixel data of the previous line. However, the original pixel data is 8 / 10 / 12 bits, resulting in a large RAM resource consumption, making it difficult to adapt to resource-constrained FPGA embedded platforms and limiting the engineering application of white balance correction schemes in the field of machine vision.
[0025] Therefore, there is an urgent need for a Bayer image white balance correction scheme that can avoid interference from overexposed pixels and optimize FPGA resource usage, in order to solve the problems of low correction accuracy and high hardware resource consumption in the existing technology.
[0026] To address the aforementioned problems, this application provides a white balance correction method for Bayer images, such as... Figure 1 As shown, the method includes the following steps: S1. Based on the preset overexposure threshold, starting from the second row of pixels, generate a flag indicating whether each pixel is overexposed, row by row; where the flag indicating whether it is overexposed is represented by a 1-bit character.
[0027] S2. Starting from the third row of pixels, traverse row by row. For each row of pixels, using two pixel columns as the step size, sequentially determine whether the markers of the four adjacent pixels in the same column of the current and previous pixel rows are not overexposed. If so, obtain two pixels from the four adjacent pixels in the current row as valid pixels. If not, directly determine whether the markers of the four adjacent pixels in the next step size are not overexposed. The total number of pixel rows in the Bayer image is no less than four.
[0028] S3. Superimpose the pixel values of the effective pixels in the four different component channels of the Bayer image to obtain the total pixel value of each component channel.
[0029] S4. Select the total pixel value with the largest value from the four different component channels as the reference value, and calculate the ratio of the reference value to the total pixel value in each component channel as the white balance correction coefficient of the corresponding component channel.
[0030] S5. Calculate the product of each pixel and the white balance correction coefficient in its component channel, and use it as the pixel value after white balance correction.
[0031] Based on the above processing, this application starts from the third row of pixels in the image, and uses the two pixels in the current row and the two pixels in the same column position of the previous row as a check group. Only when all four pixels are not overexposed are the two pixels in the current row obtained as valid pixels, which are then used to calculate the white balance correction coefficient. Thus, by filtering the valid pixels, the influence of overexposed pixels on the correction coefficient can be avoided, thus avoiding the risk of the white balance correction coefficient being easily distorted.
[0032] Meanwhile, this application uses a flag indicating whether an image is overexposed to determine the validity of pixels. Compared to the existing method of directly caching pixel values, this application effectively reduces the cache space requirement during white balance processing. Taking an 8-bit pixel depth as an example, compared to a 1-bit flag, this application can reduce the cache space requirement for the previous row of pixels to one-eighth of the original, significantly saving FPGA resources. It can be directly adapted to FPGA motherboards of industrial cameras with limited resources, solving the limitation of high hardware resource consumption in existing technologies and improving image processing efficiency.
[0033] The Bayer image in this application corresponds to a raw format image. Considering that each photosensitive element on an image sensor cannot distinguish the quantity of each color, each photosensitive element is equipped with a color filter to capture only one of the three primary colors in order to capture a color image. Therefore, a Bayer image can include red (R), green (G), and blue (B) pixels. Since image sensors lack color adaptability, the color information in the raw format image is inaccurate. By performing white balance processing on the raw format image, the color information in the raw format image is made closer to human visual color, reducing the color difference between the image color and human visual perception.
[0034] In this application, the color filter array (CFA) of the image sensor adopts a Bayer array (a Bayer array consists of alternating green-red and blue-green filters). Furthermore, given that the human eye is more sensitive to green wavelengths, the weight of green pixels is typically increased in a Bayer array to enhance the sampling of green information. Therefore, a Bayer array typically contains 50% green information and 25% each of red and blue information.
[0035] Therefore, based on the different arrangement of color filters in different image sensors, the arrangement format of R, G, and B pixels in the Bayer image of this application is different, which may include GRBG format, RGGB format, BGGR format and GBRG format, etc. This application does not make a specific limitation, and the following content will describe this application in RGGB format.
[0036] For each pixel in the Bayer image of this application, it is divided into four different component channels: R channel, corresponding to pixels containing R color; B channel, corresponding to pixels containing B color; Gr channel, representing the G channel adjacent to the R channel, corresponding to G color pixels in the same pixel row as the R channel; and Gb channel, representing the G channel adjacent to the B channel, corresponding to G color pixels in the same pixel row as the R channel.
[0037] Based on the arrangement of R, G, and B pixels in the aforementioned Bayer image, it can be seen that the B and Gb channels, and the R and Gr channels, are alternately distributed according to the pixel row arrangement. Within the same row, the B and Gb channels, or the R and Gr channels, are alternately distributed according to the pixel column arrangement. Therefore, the component channel to which any pixel belongs can be determined based on its position.
[0038] In some embodiments, step S1 may include the following: S101. Starting from the second row of pixels, receive the data stream of pixel values in the Bayer image row by row.
[0039] S102. Based on a preset overexposure threshold, compare the received pixel value with the value of each pixel to generate an overexposure flag; wherein, if the pixel value of a certain pixel is above the overexposure threshold, an overexposure flag for that pixel is generated.
[0040] For step S101, the data stream corresponds to the pixel values of all pixels in a Bayer image frame. During the data stream reading process, the pixels in a frame are read sequentially, row by row and column by column. In this application, the row index y of the pixel row starts from 0, with the row index y=1 for the second row and y=2 for the third row.
[0041] For pixels in the first row, they can be ignored or discarded during the calculation of the white balance correction coefficient. For example, an overexposure flag can be generated for the first row of pixels, but it will not be cached or used during the white balance correction process. Step S5 calculates the white balance corrected pixel value based on all pixels.
[0042] For step S102, the 1-bit character flag indicating whether the image is overexposed is represented by binary 0 and 1. In practice, the overexposed flag is usually 1, and the non-overexposed flag is usually 0. The overexposure threshold is usually the maximum pixel value that can be represented by the pixel value in the pixel. Taking an 8-bit pixel depth as an example, the corresponding maximum pixel value is 255, and the overexposure threshold is usually 255. The overexposure threshold in this application can be adjusted, for example, it can be set to 254, 253, or 252, and is not limited to the maximum pixel value that can be represented by the pixel depth.
[0043] In practice, before step S1, the white balance method provided in this application also includes initialization and parameter setting steps, including the following: Initialization of statistical variables: Define four cumulative variables sum_r, sum_g1, sum_g2, and sum_b to store the total pixel value of valid pixels in the R channel, Gr channel, Gb channel, and B channel, respectively, and set their initial values to 0.
[0044] Pre-setting parameters: Determine the width, height, and overexposure threshold of the Bayer image to be processed (corresponding to the maximum pixel value of the camera sensor, 8-bit depth is set to 255, 10-bit depth is set to 1023, and 12-bit depth is set to 4095). The input image strictly follows the RGGB Bayer format, and the corresponding pixel arrangement is: row 1 (RG|RG|...), row 2 (GB|GB|...), row 3 (RG|RG|...), row 4 (GB|GB|...).
[0045] In some embodiments, for step S2, starting from the 3rd row of the Bayer image (row index y=2, index counted from 0), traverse row by row to the last row (row index y=n-1). Each row is divided into two pixel groups by column with a step size of 2, i.e., (x,y) and (x+1,y) are the current pixel group. Simultaneously, the pixel groups (x,y-1) and (x+1,y-1) at the same column position of the previous row (y-1) are obtained as a check group, forming four adjacent pixels in the same column. It is determined whether the two pixels in the current group and the two pixels in the check group are both underexposed (i.e., their pixel values are less than the overexposure threshold). If neither is underexposed, the two pixels in the current pixel row are obtained as valid pixels. Here, x is an integer representing the column index of the pixel, 0≤x≤m-1, and m represents the total number of pixel columns in the Bayer image.
[0046] If the aforementioned step S2 is performed by the FPGA, it may include the following steps: S201. Starting from the second row of pixels, cache the flag of each row of pixels row by row based on a preset RAM. Specifically, when the flag of a pixel in the i-th row is input into the preset RAM, read the flag of the pixel in the same column of the (i-1)-th row from the preset RAM. 3 ≤ i ≤ n, where n represents the total number of pixel rows in the Bayer image.
[0047] S202: Read the flags of the (i-1)th row from the preset RAM and cache them in a register. Obtain the pixel value and flag of the pixel in the same column of the i-th row and cache them in the register. The column positions of the flags cached in the register are adjacent and the total number is not less than 2. The pixel in the same column refers to the pixel in the same column position as the flag read from the previous row.
[0048] S203. Using two pixel columns as the step size, obtain the flags of four adjacent pixels in the same column of the i-th row and the (i-1)-th row from the register.
[0049] S204. Determine whether the flags of four adjacent pixels are not overexposed. If so, obtain two pixels from the current pixel row as valid pixels; otherwise, discard the four pixels.
[0050] For step S201, the preset RAM can cache the flags of one row of pixels. Specifically, the flags of each pixel in the second row of pixels are directly cached in the preset RAM. The RAM can be cleared in advance, or the existing cached data can be discarded directly.
[0051] For step S3, since the first row of pixels does not contain pixels from the previous row, it is impossible to determine whether the first row of pixels is overexposed based on the markers of the four adjacent pixels, and therefore the first row of pixels cannot be selected as the basis for white balance correction. Furthermore, considering that the pixel array in a Bayer image is arranged in 2×2 pixel blocks (four pixels in two adjacent rows and two columns), if the effective pixel count starts from the second row of pixels, the total number of pixels used to calculate the white balance correction coefficients in the four different component channels will be different, affecting the accuracy of the white balance correction coefficient calculation results. Therefore, this application chooses to obtain effective pixel points starting from the third row of pixels and use them to calculate the white balance correction coefficients.
[0052] For the caching of flags in rows 3 to n, when the flag of the pixel in row i is input into the preset RAM, the flag of the pixel in the same column of row i-1 is read from the preset RAM. In this application, n ≥ 3.
[0053] For step S202, the flags of pixels in the same column of the (i-1)th row read from step S201 are cached in a register. Simultaneously, the pixel values and flags of pixels in the same column of the i-th row are retrieved from a preset RAM and cached in the aforementioned register. Thus, multiple flags in the same column of the current and previous rows can be obtained in the register. Considering that the column positions of the flags cached in the register are adjacent and the total number is not less than 2 (i.e., the number of flags cached in the register is not less than 4), it is typically set to read the flags of 2 adjacent pixels and cache the flags of 4 adjacent pixels in the register, such as... Figure 2 As shown.
[0054] For step S203, with a step size of two pixel columns, the flags of four adjacent pixels are obtained from the register to ensure that the four pixel flags used to determine valid pixels conform to the format of Bayer images (such as RGGB, GRBG format, etc.).
[0055] To determine whether the flags of four adjacent pixels are all overexposed, we can use the sum or product of the four flags. For example, if 1 represents an overexposed flag and 0 represents a non-overexposed flag, then if the sum of the flags of four adjacent pixels is not 0, then an overexposed flag exists. Similarly, if 0 represents an overexposed flag and 1 represents a non-overexposed flag, then if the product of the flags of four adjacent pixels is 0, then an overexposed flag exists.
[0056] For step S2, if the flags of all four adjacent pixels (RGGB) formed by the i-th row and the (i-1)-th row are not overexposed, then the two pixels in the i-th row are obtained as valid pixels. Here, the i-th row corresponds to the current row of this application, i.e., the pixel row used to determine whether an image is overexposed, which is cached in RAM. If not, this application directly determines whether the flags of all four adjacent pixels in the next step are not overexposed, and does not consider the two pixels in the current row as valid pixels.
[0057] In some embodiments, prior to step S3, the white balance correction method provided in this application further includes the following steps: S6. Based on the parity of the row number in the current pixel row, determine the component channels corresponding to the two pixels in the current pixel row; among them, the four different component channels are divided into R channel, Gr channel, B channel and Gb channel; Gr channel represents the G channel adjacent to R channel, and Gb channel represents the G channel adjacent to B channel.
[0058] Based on the distribution rules of the four component channels in the aforementioned Bayer image, it can be seen that different component channels are distributed alternately along different pixel rows and different pixel columns. Therefore, considering that the four adjacent pixels are divided into a fixed form in this application, the two pixels in the same row can be directly assigned to the R channel or Gr channel, and to the B channel or Gb channel, based on the size of the column number in the pixel (which can be understood as the left and right sides).
[0059] To distinguish between the R and Gr channels and the B and Gb channels in different pixel rows, the two component channels corresponding to the current two pixels can be determined based on the parity of the row number. Furthermore, the channel corresponding to each pixel can be further determined based on the column number. Taking a Bayer image in RGGB format as an example, pixels in odd-numbered rows correspond to the R and Gr component channels, while pixels in even-numbered rows correspond to the B and Gb component channels. Pixels in odd-numbered columns correspond to the R and Gb component channels, while pixels in even-numbered columns correspond to the Gr and B component channels. Therefore, the component channels corresponding to a pixel can be determined based on its row and column number. Considering that the four adjacent pixels selected in this application correspond to the Bayer image format, the two component channels corresponding to the current two pixels can be directly determined based on the parity of the row number.
[0060] Taking a Bayer image in RGGB format as an example, based on the definition of four accumulation variables sum_r, sum_g1, sum_g2, and sum_b, if two pixels in the current row are valid pixels, for step S3, when the current row is an odd-numbered row (y=2,4,6...), (x,y) corresponds to the R channel and is accumulated to sum_r; (x+1,y) corresponds to the Gr channel and is accumulated to sum_g1. When the current row is an even-numbered row (y=3,5,7...), (x,y) corresponds to the Gb channel and is accumulated to sum_g2; (x+1,y) corresponds to the B channel and is accumulated to sum_b, thus obtaining the total pixel value after superposition in each component channel.
[0061] In step S4, this application selects the total pixel value with the largest value as the reference value, which can ensure that the calculated white balance correction coefficient is not less than 1. Therefore, after white balance correction, the problem of correcting overexposed pixels in the Bayer image to non-overexposed pixels can be avoided.
[0062] For step S4, in the process of calculating the ratio of the reference value to the superposition result in each component channel as the white balance correction coefficient for the corresponding component channel, the white balance correction method further includes: If the total pixel value of a certain component channel is 0, then the white balance correction factor of that component channel is directly set to 1.
[0063] Specifically, the maximum value (max_sum) of the total pixel values (sum_r, sum_g1, sum_g2, sum_b) is calculated. This max_sum value is used as the baseline for calculating the white balance correction coefficient for each component channel. If the total pixel value of a channel is 0, the white balance correction coefficient for that channel is set to 1.0 to avoid division by zero errors. The specific calculation formula is as follows: The R channel coefficient kr is: kr = max_sum / sum_r (sum_r ≠ 0), otherwise kr = 1.0; The Gr channel coefficient kg1 is: kg1 = max_sum / sum_g1 (sum_g1 ≠ 0), otherwise kg1 = 1.0; The Gb channel coefficient kg2 is: kg2 = max_sum / sum_g2 (sum_g2 ≠ 0), otherwise kg2 = 1.0; The coefficient kb for channel B is: kb = max_sum / sum_b (sum_b ≠ 0), otherwise kb = 1.0.
[0064] After obtaining the white balance correction coefficients for each component channel, for step S5, all pixels in the entire image are traversed, and the component channel to which the pixel belongs is determined according to the parity of the row and column where the pixel is located. The pixel value is multiplied by the white balance correction coefficient of the corresponding component channel to obtain the white balance corrected pixel value.
[0065] After calculating the product of each pixel and the white balance correction coefficient in its corresponding component channel as the white balance corrected pixel value, the white balance correction method provided in this application also includes the following: If the white balance corrected pixel value exceeds the preset overexposure threshold, the white balance corrected pixel value will be truncated to the overexposure threshold. That is, the corrected pixel value will be truncated within the range of [0, overexpose_thresh] to avoid overexposure after correction. Here, overexpose_thresh corresponds to the preset overexposure threshold.
[0066] Taking a Bayer image in RGGB format as an example, the product rule for each pixel and the white balance correction coefficient in its corresponding component channel is as follows: Odd rows (y=0,2,4...): Even columns (x=0,2,4...) correspond to the R channel and are multiplied by kr; Odd columns (x+1) correspond to the Gr channel and are multiplied by kg1. Here, x represents the column index of the pixel, 0≤x≤m-1, and m represents the total number of pixel columns in the Bayer image.
[0067] Even rows (y=1,3,5...): Even columns (x=0,2,4...) correspond to the Gb channel, multiplied by kg2; Odd columns (x+1) correspond to the B channel, multiplied by kb.
[0068] Subsequently, a white-balanced RGGB format Bayer image can be output for use in subsequent image processing workflows.
[0069] Based on the above processing, this application uses a dual filtering method of double pixel grouping + cross-line overexposure verification to filter out effective pixels starting from the 3rd row and remove overexposed pixels to avoid distortion of the total pixel value after superposition caused by overexposed pixels, ensuring the accuracy of the white balance correction coefficient calculation result and effectively eliminating image color cast.
[0070] Meanwhile, this application uses a 1-bit overexposure flag cache to replace the original pixel data. Compared with the traditional method of caching the original pixel (8 / 10 / 12 bits / pixel), the RAM usage is reduced to 1 / 8 to 1 / 12 of the original, which greatly saves FPGA resources and is compatible with embedded platforms with limited resources.
[0071] Taking an 8-bit RGGB format Bayer image (width 1920px, height 1080px) as an example, with the overexposure threshold set to 255, the white balance correction method provided in this application is implemented in the following specific steps: Step 1: Initialize sum_r=0, sum_g1=0, sum_g2=0, sum_b=0.
[0072] Step 2: Input a 1920×1080 Bayer single-channel image in RGGB format. The first row of pixels in the image is arranged as RGRG..., the second row is arranged as GBGB..., the third row is arranged as RGRG..., and so on.
[0073] Step 3: Starting from row 3 (y=2), traverse row by row to row 1079 (y=1079). In each row, group pixels into pairs starting from x (0) with a step size of 2 to construct four adjacent pixels. Determine if none of the four adjacent pixels are overexposed. If so, select the valid pixels and overlay their pixel values onto the corresponding component channels.
[0074] Specifically, when y=2 (odd rows) and x=0, the two pixels in the current row belong to the current group, with corresponding coordinates (0,2) and (1,2), and the two pixels in the previous row belong to the check group, with coordinates (0,1) and (1,1). It checks whether the values of the four adjacent pixels at coordinates (0,2), (1,2), (0,1), and (1,1) are all less than 255. If they are all less than 255, the pixel value of (0,2) is accumulated to sum_r, and the pixel value of (1,2) is accumulated to sum_g1.
[0075] When y=3 (even-numbered rows) and x=0, the coordinates of the current group are (0,3) and (1,3), and the coordinates of the check group are (0,2) and (1,2). If none of the four pixels are overexposed, the pixel value of (0,3) is accumulated to sum_g2, and the pixel value of (1,3) is accumulated to sum_b. Repeat the above operation until all valid pixels are counted.
[0076] Step 4: Assuming the statistics are completed, sum_r = 125600, sum_g1 = 156800, sum_g2 = 152300, sum_b = 118900, calculate max_sum = 156800.
[0077] Step 5: Calculate the correction coefficients: kr=156800 / 125600≈1.248, kg1=156800 / 156800=1.0, kg2=156800 / 152300≈1.029, kb=156800 / 118900≈1.319.
[0078] Step 6, Pixel-by-Pixel Correction: For y=0 (odd rows) and x=0 (even columns), the corresponding R channel is corrected by multiplying the pixel value by 1.248, resulting in a truncation between 0 and 255. For y=1 (even rows) and x=0 (even columns), the G2 channel is corrected by multiplying the pixel value by 1.029, resulting in a truncation between 0 and 255. The remaining pixels are corrected according to the rules for their respective channels.
[0079] Step 7: Output the corrected 1920×1080 RGGB Bayer image to complete the white balance correction.
[0080] In some embodiments, the white balance correction method provided in this application is executed by an FPGA, based on a pipelined processing mode, and implemented in parallel using hardware circuits within the FPGA.
[0081] In the white balance correction scheme provided in this application, pixel data of any frame of Bayer image is continuously input into the FPGA row by row and column by column in the form of a data stream. The pixel clock within the FPGA is synchronized with the system clock. Each pixel in the Bayer image corresponds to a 1-bit flag to indicate whether it is overexposed. For example, 1 indicates overexposure, and 0 indicates no overexposure. The flag is generated by comparing the pixel with a preset overexposure threshold.
[0082] In some embodiments, the overexposure threshold in this application can be adjusted. Specifically, this application pre-defines an externally configurable register in the FPGA to store the overexposure threshold. This allows for adjustment of the overexposure threshold, rather than setting it to a hard-coded constant. For example, if it is found during operation that the correction effect is significantly affected by nonlinearity in the highlight area, a lower threshold (e.g., changing 255 to 240) can be input into the aforementioned register in the host computer software and written to the register in real time, without modifying or resynthesizing the FPGA code.
[0083] In the white balance correction scheme provided in this application, a RAM can be set up in the FPGA to cache only one row of pixel flags (used to indicate whether it is overexposed). Considering that the size of the flag is only 1 bit, the RAM capacity can be the product of the image width and 1 bit, i.e., m bits. It is not necessary to cache the pixel data of the original pixels in this RAM.
[0084] The white balance correction method provided in this application is executed by an FPGA. The FPGA can be configured with the following units to implement white balance correction for overexposure, as detailed below: The overexposure flag generation unit receives a data stream of pixel values from the input Bayer image, compares each pixel value with a preset overexposure threshold, and generates a 1-bit flag. Then, it synchronously outputs the original pixel data and the corresponding overexposure flag. Specifically, the flag is set to 1 if the pixel value is greater than or equal to the overexposure threshold, and set to 0 otherwise.
[0085] The overexposure flag generation unit can be implemented using combinational logic circuits in an FPGA. It compares the pixel value with the overexposure threshold and generates a 1-bit flag. Taking 8-bit pixel data as an example, the input consists of 8 bits of pixel data and an overexposure threshold of 255. A comparator generates a 1-bit flag, delayed by one clock cycle.
[0086] The overexposure flag buffer unit utilizes a single-port RAM to buffer all overexposure flags from the previous row. When a pixel in the current row is input, the RAM reads the overexposure flag from the same column of the previous row for subsequent verification. Simultaneously, it writes the overexposure flags of the current row pixel by pixel into the RAM, overwriting the previous row's flags, preparing for the next row's verification. When a line synchronization signal (HSYNC) is detected, the RAM read / write address is reset, and the buffering of the new row's flags is initiated.
[0087] Taking a pixel width of 1920 as an example, the capacity of a single-port RAM is 1920 bits. The read and write clocks are synchronized with the pixel clock (e.g., 100MHz). The line synchronization signal (HSYNC) can trigger an address reset, and the read and write delay is 1 clock cycle.
[0088] To improve the writing and reading efficiency of overexposed flags, the single-port RAM can be replaced with a dual-port RAM, which can simultaneously write and read the flags.
[0089] The effective pixel filtering and channel accumulation unit receives the pixel value of the current row, the flag of the current row, the flag of the previous row, and the row number and column number identifier. Ignoring the data of the first two rows (y=0, y=1), starting from the third row (y=2), the pixels are grouped according to the double-pixel grouping rule. It checks whether the flags of the current group and the group at the same position in the previous row are both 0 (neither overexposed). If the condition is met, the pixel values are accumulated to the total pixel value of the corresponding channel according to the row and column parity. Finally, the total pixel value of each of the four component channels (sum_r, sum_g1, sum_g2, sum_b) is output.
[0090] The aforementioned effective pixel filtering and channel accumulation unit can be implemented using sequential logic circuits in an FPGA. Specifically, the row and column numbers of each pixel are generated by a counter, and the logic circuits that assign different pixels to different component channels can be implemented using a multiplexer. The pixel value accumulation can be achieved using a 32-bit adder to ensure that the total pixel value does not overflow.
[0091] The white balance correction coefficient calculation unit receives the total pixel value of the four channels when the FPGA detects the frame synchronization signal VSYNC, indicating that the traversal of the entire frame image is complete. It then calculates the maximum value (max_sum) of the total pixel value across the four channels. The white balance correction coefficient for each component channel is calculated sequentially and stored in the coefficient register for use by the correction module.
[0092] In this application, the calculation of the white balance correction coefficients can be performed using a hardware divider to implement accumulation and division operations. The white balance correction coefficients for each component channel are initially calculated as floating-point numbers. These floating-point coefficients are then converted to 16-bit fixed-point numbers and stored in a coefficient register. The conversion from floating-point to fixed-point numbers is achieved through a shift operation. This application converts the white balance correction coefficients to 16-bit fixed-point numbers (8 integer bits + 8 decimal bits) and stores them in a 16-bit register.
[0093] Based on the aforementioned processing, the white balance correction coefficients in this application adopt a 16-bit fixed-point number format, including an integer part and a fractional part. The allocation ratio of the integer bit width and the fractional bit width in this fixed-point number format can be changed to balance the range of numerical representations and calculation precision, thereby adapting to industrial cameras with different bit depths and resolutions (8 / 10 / 12 bits). Specifically, in FPGAs, the fixed-point format can be adjusted through parameterized macro definitions and selective truncation / shifting, allowing a format to be selected during the configuration phase. Therefore, by adjusting the quantization granularity of numerical calculations, this application enables the solution to be compatible with various industrial camera hardware platforms, ranging from low-cost 8-bit sensors to high-precision 12-bit sensors.
[0094] The image correction and output unit receives the original Bayer pixel data stream and the correction coefficients in the coefficient register, determines the channel to which the pixel belongs based on the parity of the row and column, performs fixed-point multiplication on the pixel value and the corresponding correction coefficient, truncates the result to the range of 0 to the overexposure threshold, and synchronously outputs the corrected pixel data in the order of the original data stream.
[0095] Specifically, a fixed-point multiplier is used in the FPGA to perform the multiplication operation between the pixel value and the correction coefficient. A comparator is used to truncate the pixel value and output the 8-bit corrected pixel value. The pipeline is delayed by 3 clock cycles.
[0096] Based on the above processing, the entire FPGA implementation provided in this application, from pixel input to correction output, has only a pipeline delay of 3 to 4 clock cycles, meeting the high-speed data stream input requirements of industrial cameras (such as 100fps and above). The overexposure flag buffer unit, effective pixel filtering, and channel accumulation unit are all linked to the line synchronization signal (HSYNC) to ensure that data processing is not disrupted when switching between different pixel lines. When the frame synchronization signal (VSYNC) is detected, the FPGA resets all accumulated variables and coefficient registers, initiating the processing of a new frame of Bayer image.
[0097] The white balance correction method provided in this application has a simple process, does not require pre-decoding of the Bayer image, and adopts a pipelined design in FPGA implementation, with low latency, which can be matched with the high-speed data stream input of industrial cameras to meet the needs of real-time image processing. Furthermore, the white balance correction scheme provided in this application has a division-by-zero process and a pixel value truncation mechanism, which can be adapted to industrial cameras with different lighting environments, different resolutions, and different bit depths. At the same time, it supports adapting to different models of industrial cameras by adjusting the overexposure threshold and fixed point accuracy, thereby improving the applicability of the scheme.
[0098] Taking an 8-bit Bayer image as an example, the FPGA uses a Xilinx Artix-7 series chip. Based on combinational logic circuits, an overexposure flag generation unit, an effective pixel filtering and channel accumulation unit are constructed to realize the functions of each unit. An overexposure flag buffer unit is built based on preset RAM. For the correction coefficient calculation unit, a hardware divider is used to implement accumulation and division operations, and floating-point to fixed-point conversion is achieved through shift operations. For the image correction and output unit, a fixed-point multiplier is used to implement the multiplication operation between pixel values and correction coefficients, and a comparator is used to truncate pixel values.
[0099] In practice, the pixel clock is 100MHz and the system clock is 100MHz. Timing convergence of all modules is constrained to ensure no setup or hold-time violations. The five units mentioned above employ pipelined operation to achieve real-time processing and reduce data latency.
[0100] Tests showed that the FPGA implementation only requires 240 bytes of RAM (1920×1 bit), which is 87.5% less than the traditional cached original pixels (1920×8 bits=1920 bytes). The processing latency is 3 clock cycles, which can meet the real-time processing requirements of more than 100fps. The corrected image has no obvious color cast and no interference in overexposed areas.
[0101] In summary, the white balance correction scheme provided in this application, compared with existing white balance correction schemes, solves the problem of low correction accuracy caused by overexposed pixels through dual-pixel grouping and cross-line overexposure verification. Simultaneously, the design of buffering overexposure flags addresses the issue of high FPGA resource consumption. Furthermore, it achieves pipelined algorithm collaboration on the FPGA, improving the real-time performance of the scheme.
[0102] In some embodiments, this application also provides a white balance correction system for Bayer images, such as... Figure 3 As shown, the system includes: The flag generation module 301 is used to generate a flag indicating whether each pixel is overexposed, starting from the second row of pixels, based on a preset overexposure threshold. The overexposure flag is represented by a 1-bit character.
[0103] The effective pixel acquisition module 302 is used to traverse row by row starting from the third row of pixels. For each row of pixels, it uses two pixel columns as the step size to sequentially determine whether the flags of the four adjacent pixels in the same column of the current pixel row and the previous pixel row are all not overexposed. If so, it acquires two pixels from the four adjacent pixels in the current pixel row as effective pixels. If not, it directly determines whether the flags of the four adjacent pixels in the next step size are all not overexposed. The total number of pixel rows in the Bayer image is not less than 4 rows.
[0104] The pixel total value acquisition module 303 is used to superimpose the pixel values of the effective pixels in the four different component channels of the Bayer image to obtain the pixel total value of each component channel.
[0105] The coefficient calculation module 304 is used to select the total pixel value with the largest value from four different component channels as the reference value, and calculate the ratio of the reference value to the total pixel value in each component channel as the white balance correction coefficient of the corresponding component channel.
[0106] The pixel value correction module 305 is used to calculate the product of each pixel and the white balance correction coefficient in its component channel, which is then used as the pixel value after white balance correction.
[0107] This application also provides an electronic device, such as... Figure 4 As shown, it includes a processor 401, a communication interface 402, a memory 403, and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404. Memory 403 is used to store computer programs; When the processor 401 executes the program stored in the memory 403, it implements any of the above-mentioned white balance correction methods.
[0108] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.
[0109] The communication interface is used for communication between the aforementioned electronic devices and other devices.
[0110] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.
[0111] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0112] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements any of the white balance correction method steps described above.
[0113] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the white balance correction method steps in the above embodiments.
[0114] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A white balance correction method for Bayer images, characterized in that, include: Based on a preset overexposure threshold, starting from the second row of pixels, a flag indicating whether each pixel is overexposed is generated row by row; the flag indicating whether a pixel is overexposed is represented by a 1-bit character. Starting from the third row of pixels, traverse row by row. For each row of pixels, using two pixel columns as the step size, check whether the flags of the four adjacent pixels in the same column of the current pixel row and the previous pixel row are not overexposed. If so, obtain two pixels from the four adjacent pixels as valid pixels. If not, directly check whether the flags of the four adjacent pixels in the next step size are not overexposed. The total number of pixel rows in the Bayer image is not less than 4. The pixel values of the effective pixels in the four different component channels of the Bayer image are superimposed to obtain the total pixel value of each component channel; From the four different component channels, the total pixel value with the largest value is selected as the reference value, and the ratio of the reference value to the total pixel value in each component channel is calculated as the white balance correction coefficient of the corresponding component channel. Calculate the product of each pixel and the white balance correction coefficient in its corresponding component channel, and use this product as the white balance corrected pixel value.
2. The white balance correction method according to claim 1, characterized in that, After calculating the product of each pixel and the white balance correction coefficient in its corresponding component channel as the white balance corrected pixel value, the white balance correction method further includes: If the pixel value after white balance correction exceeds the preset overexposure threshold, the pixel value after white balance correction will be truncated to the overexposure threshold.
3. The white balance correction method according to claim 1, characterized in that, In calculating the ratio of the reference value to the superposition result in each component channel, as the white balance correction coefficient for the corresponding component channel, the white balance correction method further includes: If the total pixel value of a certain component channel is 0, then the white balance correction factor for that component channel is set to 1.
4. The white balance correction method according to claim 1, characterized in that, Based on a preset overexposure threshold, starting from the second row of pixels, a flag indicating whether each pixel is overexposed is generated row by row, including: Receive the data stream of pixel values from the Bayer image line by line; Based on a preset overexposure threshold, the value of each received pixel is compared to generate an overexposure flag; if the pixel value of a certain pixel is above the overexposure threshold, an overexposure flag for that pixel is generated.
5. The white balance correction method according to claim 1, characterized in that, Starting from the third row of pixels, iterate through each row. For each row, using two pixel columns as the step size, sequentially check whether the flags of the four adjacent pixels in the same column of the current and previous rows are not overexposed, including: Starting from the second row of pixels, the flags of each row of pixels are cached row by row based on the preset RAM; wherein, when the flag of the i-th row of pixels is input into the preset RAM, the flags of the pixels in the same column of the (i-1)-th row are read from the preset RAM; 3≤i≤n, where n represents the total number of pixel rows in the Bayer image; The flags of the (i-1)th row are read from the preset RAM and cached in the register. The pixel values and flags of the pixels in the same column of the i-th row are obtained and cached in the register. The column positions of the flags cached in the register are adjacent and the total number is not less than 2. The pixels in the same column refer to the pixels in the same column position as the flags read from the previous row. Using two pixel columns as the step size, retrieve the flags of four adjacent pixels in the same column of the i-th row and the (i-1)-th row from the register; Determine whether the markers of four adjacent pixels are not overexposed.
6. The white balance correction method according to claim 1, characterized in that, Before superimposing the pixel values of the effective pixels in the four different component channels of the Bayer image to obtain the total pixel value of each component channel, the white balance correction method further includes: Based on the parity of the row number in the current pixel row, the component channels corresponding to the two pixels in the current pixel row are determined; among them, the four different component channels are divided into R channel, Gr channel, B channel and Gb channel; Gr channel represents the G channel adjacent to R channel, and Gb channel represents the G channel adjacent to B channel.
7. The white balance correction method according to claim 1, characterized in that, The white balance correction method is executed by an FPGA, based on a pipelined processing mode, and implemented in parallel using hardware circuits within the FPGA.
8. A white balance correction system for Bayer images, characterized in that, include: The flag generation module is used to generate a flag indicating whether each pixel is overexposed, starting from the second row of pixels, based on a preset overexposure threshold; the flag indicating whether a pixel is overexposed is represented by a 1-bit character. The effective pixel acquisition module is used to traverse row by row starting from the third row of pixels. For each row of pixels, it uses two pixel columns as the step size to determine whether the flags of the four adjacent pixels in the same column of the current pixel row and the previous pixel row are not overexposed. If so, it acquires two pixels from the four adjacent pixels as effective pixels. If not, it directly determines whether the flags of the four adjacent pixels in the next step size are not overexposed. The total number of pixel rows in the Bayer image is not less than 4. The pixel total value acquisition module is used to superimpose the pixel values of the effective pixels in the four different component channels of the Bayer image to obtain the pixel total value of each component channel. The coefficient calculation module is used to select the total pixel value with the largest value from four different component channels as the reference value, and calculate the ratio of the reference value to the total pixel value in each component channel as the white balance correction coefficient of the corresponding component channel. The pixel value correction module is used to calculate the product of each pixel and the white balance correction coefficient in its component channel, which is then used as the white balance corrected pixel value.
9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory are connected through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the white balance correction method according to any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, A computer-readable storage medium stores a computer program that, when executed by a processor, implements the white balance correction method according to any one of claims 1-7.
