Bayer format image white balance correction method based on FPGA

Abstract

The invention relates to the technical field of image processing, in particular to a Bayer format image white balance correction method based on an FPGA (Field Programmable Gate Array), which comprises the following steps of: carrying out pixel classification on Bayer format image data according to type labels; counting a global pixel mean value and a local pixel mean value corresponding to the type label in the whole image after pixel classification; performing gain compensation on the total image based on the global pixel mean value and the local pixel mean value; and performing demosaicing on the Bayer format image data after gain compensation to generate RGB image data. The parallel computing capability and pipeline design of the FPGA are utilized, the algorithm is optimized, resource occupation is reduced, and while the white balance precision is guaranteed, the real-time performance and adaptability of the white balance technology are improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a Bayer format image white balance correction method based on FPGA. BACKGROUND

[0002] In a digital imaging system, automatic white balance (AWB) is one of the key steps of image signal processing (ISP). Its core goal is to eliminate the influence of scene illumination color temperature on object color, ensuring that a white object appears true white under different lighting conditions and restores the natural color of the object. If the white balance is not accurate, the image will have obvious color deviation (such as yellow or blue), which seriously affects the visual quality of the viewer. Especially in the fields of industrial detection, security monitoring, medical imaging, etc., color accuracy is directly related to the reliability of subsequent analysis. During image acquisition, due to different color temperatures of light sources, the image may exhibit color deviation. For example, an image taken under an incandescent lamp will be yellow, while an image taken under a fluorescent lamp will be blue. Automatic white balance adjusts the gain of the red, green, and blue (RGB) channels in the image to maintain color consistency under different lighting conditions.

[0003] Although existing automatic white balance techniques can achieve white balance function to some extent, they still have shortcomings in practical application: 1. Poor adaptability to complex lighting conditions. Existing algorithms are usually designed for specific lighting conditions, and the white balance effect is unstable under complex lighting conditions (such as mixed light sources, low light, high dynamic range scenes), and color deviation is prone to occur. In addition, existing algorithms usually use fixed calculation methods and cannot dynamically adjust the white balance strategy according to scene changes, resulting in poor adaptability in different scenes and lack of dynamic adjustment capability. In addition, real-time automatic white balance based on FPGA needs to complete image processing in a very short time, but existing algorithms often increase computational complexity and resource occupation while improving accuracy, making it difficult to balance real-time and efficiency. In addition, there is insufficient support for Bayer format images. Many existing algorithms are designed directly for RGB format images, while actual image detectors usually output RGGB RGB format data. Directly processing Bayer format data increases computational complexity, while converting to RGB format introduces additional delay and resource consumption. Therefore, how to improve the adaptability and real-time performance of white balance technology under the condition of reducing resource occupation has become a problem to be solved. SUMMARY

[0004] In view of this, the present invention aims to provide a Bayer format image white balance correction method based on FPGA. By utilizing the parallel computing power and pipeline design of FPGA, the algorithm is optimized to reduce resource consumption, thereby improving the real-time performance and adaptability of white balance technology while ensuring white balance accuracy.

[0005] To achieve the above objectives, the technical solution created by this invention is implemented as follows: This invention provides a Bayer format image white balance correction method based on FPGA, comprising the following steps: classifying Bayer format image data into pixels according to type labels; calculating the global pixel mean and local pixel mean of the entire image corresponding to the type labels after pixel classification; performing gain compensation on the entire image based on the global pixel mean and the local pixel mean; and performing de-mosaic on the gain-compensated Bayer format image data to generate RGB image data.

[0006] Furthermore, the Bayer format image data is classified into pixels according to the type label, specifically including: determining the type label corresponding to the pixel based on the coordinates of the pixel; and generating the type label corresponding to the coordinates of the pixel.

[0007] Further, after classifying pixels, the global pixel mean and local pixel mean corresponding to the type label in the entire image are calculated, specifically including: calculating the global pixel mean corresponding to the type label in the entire image based on the type label; dividing the entire image into multiple sub-modules; and calculating the local pixel mean corresponding to the type label in each sub-module.

[0008] Furthermore, gain compensation is performed on the entire image based on the global pixel mean and the local pixel mean, specifically including: calculating dynamic gain based on the global pixel mean and the local pixel mean; and compensating the entire image for the dynamic gain based on the dynamic gain.

[0009] Further, the dynamic gain is calculated based on the global pixel mean and the local pixel mean, specifically including: calculating the global gain of the entire image based on the global pixel mean; calculating the local gain of the sub-module based on the local pixel mean; and performing gain fusion based on the global gain and the local gain to obtain the dynamic gain.

[0010] Further, the pixel coordinates with odd-numbered rows and columns represent R-pixels, the pixel coordinates with odd-numbered rows and even-numbered columns represent Gr-pixels, the pixel coordinates with even-numbered rows and odd-numbered columns represent Gb-pixels, and the pixel coordinates with even-numbered rows and even-numbered columns represent B-pixels. The global pixel mean of the entire image is calculated based on these type labels, specifically including: calculating the single-class pixel mean for each type label in the entire image; merging and calculating the merged pixel mean of Gr-pixels and Gb-pixels in the entire image; and calculating the average of the single-class pixel mean of R-pixels, the single-class pixel mean of B-pixels, and the merged pixel mean to obtain the grayscale pixel mean of the entire image.

[0011] Furthermore, calculating the global gain of the entire image based on the global pixel mean specifically includes: calculating the single-class pixel gain of the R pixel and the single-class pixel gain of the B pixel in the entire image, and calculating the combined pixel gain of the Gr pixel and the Gb pixel.

[0012] Further, calculating the local gain of the sub-module based on the local pixel mean specifically includes: determining whether the deviation between the single-class pixel mean of the R pixel and the single-class pixel mean of the B pixel in the sub-module and the single-class pixel mean of the R pixel and the single-class pixel mean of the B pixel in the whole image exceeds a threshold; if the deviation exceeds the threshold, then calculating the single-class pixel gain of the R pixel and the single-class pixel gain of the B pixel in the sub-module.

[0013] In addition, the present invention also provides a Bayer format image white balance correction system based on FPGA. The system is integrated into a field-programmable gate array and includes: a pixel classification module for classifying Bayer format image data according to type labels; a statistics module for calculating the global pixel mean and local pixel mean of the entire image corresponding to the type labels after pixel classification; a gain compensation module for performing gain compensation on the entire image based on the global pixel mean and the local pixel mean; and a demosaic module for performing demosaic on the gain-compensated Bayer format image data to generate RGB image data.

[0014] Furthermore, the present invention also provides an electronic device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and the machine-readable instructions are executed by the processor to perform the steps of the Bayer format image white balance correction method described above.

[0015] Compared with existing technologies, the present invention provides an FPGA-based Bayer format image white balance correction method. This method classifies Bayer format image data into pixels based on type labels; statistically analyzes the global and local pixel mean values ​​corresponding to the type labels in the entire image after pixel classification; performs gain compensation on the entire image based on the global and local pixel mean values; and performs de-mosaicing on the gain-compensated Bayer format image data to generate RGB image data. The FPGA-based Bayer format image white balance correction method provided by this invention can process RGGB Bayer format raw image data directly and automatically in real time, completing white balance correction before de-mosaicing, avoiding the additional overhead of converting to RGB format. Simultaneously, through parallel processing and pipelined design, it ensures white balance calculation accuracy, meets real-time requirements, reduces data buffering and calculation steps, significantly reduces processing latency and resource consumption, and improves computational efficiency. Furthermore, through algorithm optimization and hardware resource reuse, it reduces the occupation of FPGA logic units and storage resources, making it particularly suitable for resource-constrained embedded systems. In addition, gain compensation can dynamically adjust the gain according to different scenarios and perform white balance correction in real time based on image features (such as brightness, color distribution, etc.), thereby improving the flexibility and adaptability of the algorithm. Attached Figure Description

[0016] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 A schematic flowchart of a Bayer format image white balance correction method provided for embodiments of the present invention; Figure 2 for Figure 1 The diagram illustrates the Bayer format image in the Bayer format image white balance correction method shown. Figure 3 This is a schematic diagram of the structure of an electronic device provided for an embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and do not constitute a limitation thereof. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of the invention. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, some operations related to the invention are not shown or described in the specification. This is to avoid obscuring the core parts of the invention with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0018] It should be noted that, unless otherwise specified, the embodiments and features described in this invention can be combined to form various implementations. Furthermore, the order of the steps or actions in the method description can be changed or adjusted in a manner readily apparent to those skilled in the art. Therefore, the various orders in the specification and drawings are merely for the clear description of a particular embodiment and do not imply a mandatory order, unless otherwise stated that a particular order must be followed.

[0019] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0020] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0021] The invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0022] like Figure 1 As shown, this invention provides an FPGA-based Bayer format image white balance correction method, comprising the following steps: classifying Bayer format image data by pixel according to type labels; statistically analyzing the global pixel mean and local pixel mean of the corresponding type label in the entire image after pixel classification; performing gain compensation on the entire image based on the global pixel mean and local pixel mean; and performing demosaicing on the gain-compensated Bayer format image data to generate RGB image data. The FPGA-based Bayer format image white balance correction method proposed in this invention can automatically and directly process RGGB Bayer format raw image data in real time, completing white balance correction before demosaicing, achieving gain compensation for Bayer format image data, and avoiding secondary processing after demosaicing. RGB image data is generated using bilinear interpolation and output in real time through a standard interface (such as HDMI).

[0023] The resulting technical benefits are: 1. Optimized processing of Bayer format image data, enabling direct white balance calculations on RGGBBayer format data, avoiding the overhead of converting to RGB format, and improving computational efficiency through parallel processing and pipelined design. 2. Optimized algorithm implementation by leveraging the parallel computing capabilities and pipelined design of FPGAs, ensuring white balance accuracy while meeting real-time requirements. 3. Reduced FPGA logic unit and storage resource usage through algorithm optimization and hardware resource reuse, making it suitable for resource-constrained embedded systems and improving applicability. 4. Dynamically and automatically adjusting gain compensation in real time according to different application scenarios, adjusting white balance parameters in real time based on image characteristics (such as brightness and color distribution), and automatically performing real-time gain compensation, improving the algorithm's flexibility and adaptability.

[0024] Furthermore, in the FPGA-based Bayer format image white balance correction method provided in this embodiment of the invention, pixel classification of Bayer format image data based on type labels specifically includes: determining the type label corresponding to the pixel based on the pixel's coordinates; and generating a type label at the coordinates of the corresponding pixel. Unprocessed RAW image data is received through a sensor interface, and the type label of the pixel can be determined based on the parity of the row and column coordinates of the pixel in the entire image. The pixel type label can be R, Gr, Gb, or B. Wherein: R represents the row and column that are odd-numbered; Gr is located in an odd-numbered row and an even-numbered column; Gb is located in an even-numbered row and an odd-numbered column; B is located in an even-numbered row and an even-numbered column.

[0025] Furthermore, in the FPGA-based Bayer format image white balance correction method provided in this embodiment of the invention, the global pixel mean and local pixel mean of the corresponding type label in the entire image after pixel classification specifically include: calculating the global pixel mean of the corresponding type label in the entire image based on the type label; dividing the entire image into multiple sub-modules; and calculating the local pixel mean of the corresponding type label in each sub-module. Specifically, the global pixel mean of the type labels R, Gr, Gb, and B in the entire image are calculated respectively, and then the local pixel mean of the corresponding type label in each sub-module is calculated.

[0026] This Bayer format image white balance correction method uses a combination of global and local pixel mean values ​​to dynamically adjust the gain coefficient. The gain coefficient α is dynamically adjusted based on the local illumination uniformity. 1. Uniformity determination: If the deviation (Δ) between the single-class pixel mean of R pixel and the single-class pixel mean of B pixel in the submodule and the global mean is less than 15%, it is determined to be uniform lighting; otherwise, it is non-uniform lighting.

[0027] 2. Rules for choosing the value of α:

[0028] Specifically, it is necessary to ensure that α=0.85 when Δ=15% and α=0.5 when Δ=30%, balancing smoothness and correction strength.

[0029] Furthermore, in the FPGA-based Bayer format image white balance correction method provided in this embodiment of the invention, gain compensation is performed on the entire image based on the global pixel mean and the local pixel mean, specifically including: calculating dynamic gain based on the global pixel mean and the local pixel mean; and compensating the entire image with dynamic gain based on the dynamic gain.

[0030] Furthermore, in the FPGA-based Bayer format image white balance correction method provided in this embodiment of the invention, dynamic gain is calculated based on global pixel mean and local pixel mean. Specifically, this includes: calculating the global gain of the entire image based on the global pixel mean; calculating the local gain of the sub-module based on the local pixel mean; and performing gain fusion based on the global gain and local gain to obtain the dynamic gain. The global gain can be generated based on the global pixel mean using an improved gray-world assumption. The improved gray-world assumption calculates the mean of Gr pixels (odd-numbered rows G) and the mean of Gb pixels (even-numbered rows G) in the entire image separately, and then takes the average, avoiding noise sensitivity introduced by statistical merging. The gain coefficient is optimized by judging the deviation between global and local pixels, as detailed in the following description.

[0031] Furthermore, in the FPGA-based Bayer format image white balance correction method provided in this embodiment of the invention, the rows and columns of the pixel coordinates that are odd-numbered represent the type labels for R pixels, the rows and columns of the pixel coordinates that are odd-numbered represent the type labels for Gr pixels, the rows and columns of the pixel coordinates that are even-numbered represent the type labels for Gb pixels, and the rows of the pixel coordinates that are even-numbered represent the type labels for B pixels. The global pixel mean of the entire image is calculated based on the type labels, specifically including: calculating the single-class pixel mean for each type label in the entire image; merging and calculating the merged pixel mean of Gr pixels and Gb pixels in the entire image; calculating the average of the single-class pixel mean of R pixels, the single-class pixel mean of B pixels, and the merged pixel mean to obtain the grayscale pixel mean of the entire image.

[0032] Furthermore, in the FPGA-based Bayer format image white balance correction method provided in this embodiment of the invention, the global gain of the entire image is calculated based on the global pixel mean, specifically including: calculating the single-class pixel gain of R pixels and the single-class pixel gain of B pixels in the entire image, and merging and calculating the merged pixel gain of Gr pixels and Gb pixels.

[0033] Furthermore, in the FPGA-based Bayer format image white balance correction method provided in this embodiment of the invention, the local gain of the sub-module based on the local pixel mean is specifically included as follows: determining whether the deviation between the single-class pixel mean of R pixel and the single-class pixel mean of B pixel in the sub-module and the single-class pixel mean of R pixel and the single-class pixel mean of B pixel in the whole image exceeds a threshold; if the deviation exceeds the threshold, then calculating the single-class pixel gain of R pixel and the single-class pixel gain of B pixel in the sub-module.

[0034] Since the G channel (Gr pixels, Gb pixels) is usually used as the reference channel for white balance correction, its gain is generally directly adopted from the global merged pixel gain. Therefore, only the single-class pixel mean of R pixels and the single-class pixel mean of B pixels in the sub-module are calculated, and the deviation between the single-class pixel mean of R pixels and the single-class pixel mean of B pixels in the sub-module and the whole image is judged. For sub-modules that exceed the threshold, the single-class pixel gain of R pixels and the single-class pixel gain of B pixels in that sub-module need to be calculated. It should be noted that the sub-module division can remain unchanged, but due to the real-time nature of Bayer format image data, the data in the sub-module changes in real time. Noise and other outliers may cause the data deviation in a certain sub-module to exceed the threshold at a certain moment. If the noise is eliminated in the next moment and the data in the sub-module returns to normal, then the local gain adjustment can be stopped in the next moment. Therefore, judging the deviation threshold can distinguish between true color cast and noise and small fluctuations, avoiding over-correction. The human eye's perception threshold for color difference is usually about 10-20%, and a deviation threshold greater than 15% will show obvious color cast. Therefore, this threshold can be set to 15%.

[0035] The following detailed description is provided with reference to specific embodiments: Step 1: Image Acquisition and Pixel Classification: Unprocessed RAW image data stream in RGGB Bayer format output from the camera sensor, such as 12-bit raw image pixels. Pixel Classification: Based on the pixel's coordinates (odd or even of row and column numbers), its type label (R, Gr, Gb, B) is determined in real time, and a corresponding type label is generated at the pixel's coordinates. R: Located in an odd-numbered row and an odd-numbered column; Gr: Located in odd-numbered rows and even-numbered columns; Gb: Located in even-numbered rows and odd-numbered columns; B: Located in an even-numbered row and an even-numbered column.

[0036] The Bayer format image with type labels generated at the coordinates of the pixels is as follows: Figure 2 As shown, the Bayer format image data is stored in the BRAM cache.

[0037] Step 2: Calculate the global pixel mean and the local pixel mean. Global statistics: Calculate the global pixel mean (R_mean, Gr_mean, Gb_mean, B_mean) for the four classification labels (R, Gr, Gb, B) of the entire image, and combine the global pixel mean of Gr and Gb in the entire image to obtain the grayscale pixel mean (Gray_global): G_global_mean = (Gr_mean + Gb_mean) / 2 Gray_global = (R_mean + G_global_mean + B_mean) / 3 Local statistics: The entire image is divided into 8×8 sub-modules, and the local pixel mean values ​​of R, Gr, Gb, and B in each sub-module are calculated independently for dynamic adjustment of the gain coefficient.

[0038] Step 3: Calculate the dynamic gain Global gain calculation: R_gain_global = Gray_global / R_mean G_gain_global = Gray_global / G_global_mean B_gain_global = Gray_global / B_mean Local gain adjustment: If the deviation between the local pixel mean values ​​of R and B in a local submodule and the global pixel mean values ​​of R and B in the entire image exceeds a threshold (e.g., ±15%), then the single-class pixel gain of R pixels and the single-class pixel gain of B pixels in the submodule are calculated based on local pixel statistics. R_gain_local = Gray_local / R_local_mean B_gain_local = Gray_local / B_local_mean Gain fusion: A weighted average method is used to fuse global and local gains to suppress color shift caused by sudden changes in local illumination.

[0039] Step 4: Gain compensation for Bayer image data Application of gain compensation: Applying the corresponding gain directly to Bayer format image data based on the pixel type label (before performing demosaic), and implementing multiplication operations using an FPGA DSP array: Gain compensation for R: R_corrected = R_raw × R_gain Gain compensation in Gr / Gb: G_corrected = G_raw × G_gain Gain compensation for B: B_corrected = B_raw × B_gain Where R_gain, G_gain, and B_gain are all final gains, and the final gain = α × global gain + (1-α) × local gain, where α is a weighting coefficient. R_raw, G_raw, and B_raw are the pixel values ​​of the unprocessed RAW image data stream.

[0040] Data truncation and saturation protection: A 16-bit fixed-point multiplier is used to retain the high 12 bits of valid data to avoid overflow.

[0041] Step 5: Remove mosaic Bilinear interpolation: De-mosaic is performed on the gain-compensated Bayer format image data to generate a complete RGB image. The output interface can output the gain-compensated RGB image data via HDMI or MIPI.

[0042] In addition, according to embodiments of the present invention, a Bayer format image white balance correction system based on FPGA is also provided. The system is integrated into a field-programmable gate array and includes: a pixel classification module for classifying Bayer format image data according to type labels; a statistics module for calculating the global pixel mean and local pixel mean of the corresponding type label in the entire image after pixel classification; a gain compensation module for performing gain compensation on the entire image based on the global pixel mean and local pixel mean; and a demosaic module for performing demosaic on the gain-compensated Bayer format image data to generate RGB image data.

[0043] The Bayer format image white balance correction system provided in this invention is integrated into a Field Programmable Gate Array (FPGA). Gain compensation is performed before de-mosaicing, reducing data buffering by requiring only 8 lines of raw data and lowering data latency. The FPGA ensures parallel processing through a 7-stage pipeline, processing one pixel per clock cycle for zero latency and enhanced real-time performance. Furthermore, using an FPGA allows for DSP multiplexing, reducing resource consumption by sharing multipliers and adders. This Bayer format image white balance correction system can separately calculate the global and local pixel mean values ​​for corresponding type labels in the entire image after pixel classification. In particular, it can dynamically adjust gain compensation according to different application scenarios, such as shadow or mixed lighting, reducing chromatic aberration and enhancing applicability. In addition, this Bayer format image white balance correction system has stronger compatibility and scalability. By configuring pixel classification label logic, it can adapt to image formats such as RGGB, BGGR, and GRBG, supporting various Bayer format image data. The parameters of dynamic gain compensation can be adjusted in real time, such as the block size of the submodule (e.g., 4×4 or 16×16), and the gain fusion weight (α) can be dynamically configured through registers. For example, in scenarios with high-resolution images or low-end FPGAs (resource-constrained), the block size of the submodule can be changed from 8×8 to 16×16 to reduce the resource consumption of the statistics module.

[0044] Furthermore, embodiments of the present invention also provide an electronic device, a readable storage medium, and a computer program product. These include a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate via the bus, and the machine-readable instructions are executed by the processor to perform the steps of the Bayer format image white balance correction method described above. Figure 3 This invention provides a computer device, a readable storage medium, and a computer program product in its embodiments.

[0045] Figure 3 This is a schematic diagram of the structure of a computer device 12 provided in an embodiment of the present invention. Figure 3 A block diagram of an exemplary computer device 12 suitable for implementing embodiments of the present invention is shown. Figure 3 The computer device 12 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0046] As shown in Figure N, computer device 12 is represented in the form of a general-purpose computing device. Computer device 12 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0047] The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0048] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0049] Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0050] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory. Computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 3 Not shown; usually referred to as a "hard drive"). Although Figure 3 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0051] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0052] Computer device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with computer device 12, and / or with any device that enables computer device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, computer device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0053] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the Bayer format image white balance correction method provided in the embodiments of the present invention.

[0054] This invention also provides a non-transitory computer-readable storage medium storing computer instructions, on which a computer program is stored, wherein the program, when executed by a processor, is the Bayer format image white balance correction method provided in all embodiments of this application.

[0055] The computer storage medium of this invention can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0056] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0057] The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. The computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0058] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the Bayer format image white balance correction method described above.

[0059] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0060] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A Bayer format image white balance correction method based on FPGA, characterized in that: Includes the following steps: Classify Bayer format image data by pixel based on type labels; After classifying pixels, calculate the global pixel mean and local pixel mean of the corresponding type label in the entire image; Gain compensation is performed on the entire image based on the global pixel mean and the local pixel mean; Demosaic is performed on the Bayer format image data after gain compensation to generate RGB image data.

2. The Bayer format image white balance correction method according to claim 1, characterized in that: Pixel classification of Bayer format image data based on type labels, specifically including: The type label corresponding to the pixel is determined based on the coordinates of the pixel; The type label is generated at the coordinates of the pixel.

3. The Bayer format image white balance correction method according to claim 2, characterized in that: After classifying pixels, the global pixel mean and local pixel mean of the corresponding type label in the entire image are calculated, specifically including: Calculate the global pixel mean value corresponding to the type label in the entire image based on the type label; The entire graph is divided into multiple sub-modules; Calculate the local pixel mean value corresponding to the type label in the submodule.

4. The Bayer format image white balance correction method according to claim 3, characterized in that: Gain compensation is performed on the entire image based on the global pixel mean and the local pixel mean, specifically including: The dynamic gain is calculated based on the global pixel mean and the local pixel mean; The dynamic gain is used to compensate the dynamic gain of the entire graph.

5. The Bayer format image white balance correction method according to claim 4, characterized in that: The dynamic gain is calculated based on the global pixel mean and the local pixel mean, specifically including: The global gain of the entire image is calculated based on the global pixel mean. The local gain of the submodule is calculated based on the local pixel mean. The dynamic gain is obtained by performing gain fusion based on the global gain and the local gain.

6. The Bayer format image white balance correction method according to any one of claims 3 to 5, characterized in that: The pixel coordinates are categorized into types: odd-numbered rows and columns represent R-pixels; odd-numbered rows and even-numbered columns represent Gr-pixels; even-numbered rows and odd-numbered columns represent Gb-pixels; and even-numbered rows and even-numbered columns represent B-pixels. The global pixel mean is calculated based on these type labels, specifically including: Calculate the average pixel value of each type label in the entire image; The average value of the merged pixels of Gr pixels and Gb pixels in the entire image is calculated. The average value of the single-class pixel mean of the R pixel, the single-class pixel mean of the B pixel, and the average value of the merged pixel are calculated to obtain the average grayscale pixel value of the entire image.

7. The Bayer format image white balance correction method according to claim 6, characterized in that: The global gain of the entire image is calculated based on the global pixel mean, specifically including: Calculate the single-class pixel gain of the R pixel and the single-class pixel gain of the B pixel in the entire image, and calculate the combined pixel gain of the Gr pixel and the Gb pixel.

8. The Bayer format image white balance correction method according to claim 6, characterized in that: The local gain of the sub-module is calculated based on the local pixel mean, specifically including: Determine whether the deviation between the average value of a single type of pixel of the R pixel and the average value of a single type of pixel of the B pixel in the submodule and the average value of a single type of pixel of the R pixel and the average value of a single type of pixel of the B pixel in the whole image exceeds a threshold. If the deviation exceeds the threshold, then the single-class pixel gain of the R pixel and the single-class pixel gain of the B pixel in the submodule are calculated.

9. A Bayer format image white balance correction system based on FPGA, characterized in that, This system is integrated into a field-programmable gate array (FPGA), including: The pixel classification module classifies Bayer format image data into pixels based on type labels. The statistics module calculates the global and local pixel mean values ​​for the corresponding type labels in the entire image after pixel classification. The gain compensation module performs gain compensation on the entire image based on the global pixel mean and the local pixel mean; The demosaic module performs demosaicing on the Bayer format image data after gain compensation to generate RGB image data.

10. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. The machine-readable instructions are executed by the processor to perform the steps of the Bayer format image white balance correction method as described in any one of claims 1 to 8.

Images