A RAW image reconstruction method, system, computer device and storage medium

Abstract

This invention provides a RAW image reconstruction method, system, computer device, and storage medium, belonging to the field of image processing. The invention establishes red, green, and blue sampling planes for the RAW image based on the pixel sampling positions of the Bayer array, and constructs color statistical samples with resolutions lower than a set value based on neighborhood. It calculates the brightness, saturation, and local gradient of each sample, filters candidate background regions, and then estimates the white balance gain of the red and blue channels. Using the green channel as the brightness benchmark, it adaptively recovers the missing green value. Using the white balance gain and the recovered green value, it calculates the normalized gain color difference at the original red and blue positions, thereby reconstructing the complete RGB channels and outputting a corrected image. This method embeds white balance constraints into the entire de-mosaic process, uses channel ratio constraints to constrain color difference propagation, effectively suppresses color shift and false color, and preserves the edge and texture details of the green channel, achieving a color-balanced, detailed, and artifact-free RGB image output.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a RAW image reconstruction method, system, computer equipment, and storage medium. Background Technology

[0002] Bright-field microscopy imaging systems typically employ a CMOS image sensor combined with a Bayer color filter array to acquire optical information from the sample. Each pixel in the Bayer array acquires only one of the three color components: red, green, or blue. Microscopic RAW (RAW Image Format) refers to the raw image data directly output from the image sensor (such as CMOS) without any processing, and it requires a demosaicing algorithm to recover the complete RGB image. Furthermore, due to differences in the type of light source in the microscopic illumination system, the transmission characteristics of the microscopic optical path, the objective lens spectral response, and the sensor channel response, significant response deviations exist between different color channels, necessitating white balance correction.

[0003] Current image processing workflows typically treat white balance and demosaicing as independent, sequential steps: one approach is to directly multiply the white balance gain by the RAW sample value before performing demosaicing; another approach is to perform demosaicing first, and then perform white balance correction on the complete RGB image. While these methods are simple to implement, when the white balance gain is directly applied to sparse RAW samples, noise and local outliers in the red and blue channels may be amplified simultaneously and propagate during subsequent interpolation. Furthermore, the demosaicing process cannot utilize the channel response relationships contained in the white balance parameters, making it difficult to maintain cross-channel consistency during color restoration. This can easily damage the main brightness structure carried by the green channel, reducing image detail clarity. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides a RAW image reconstruction method. This method ensures that white balance parameters are no longer merely used as pre-processing or post-processing coefficients, but rather participate as constraint variables in color difference modeling, propagation, and complete color channel recovery. This approach improves color consistency and suppresses false color while maintaining the original Bayer sampling relationship.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A method for reconstructing RAW images includes the following steps: Acquire RAW images with Bayer array data and estimate the white balance gain parameters of the RAW images; Using the green channel as the brightness structure benchmark, the missing green channel value is adaptively restored at the red and blue sampling positions of the RAW image based on horizontal and vertical gradients, adjacent green sampling values, and the low-frequency ratio constraint of the same color. According to the white balance gain parameter and the restored green channel, the initial normalized gain color difference is calculated at the original red and original blue sampling positions respectively. The horizontal and vertical gradients are the composite gradients at the red or blue sampling positions where the green channel value to be restored is obtained by weighted summation of the directional differences between adjacent green samples and the directional differences between two pixel sampling points of the same color channel. The adjacent green sampling values ​​are the sensor's native green pixels directly adjacent to the red and blue sampling points in all directions. The low-frequency ratio constraint of the same color estimates the local brightness baseline using the original RAW pixel values ​​of the same color space between the current red and blue sampling points to obtain the brightness ratio, which is used to correct candidate green values. By combining the white balance gain parameter, the restored green channel, and the initial normalized gain color difference to reconstruct the RGB channels, the corrected RGB image is output.

[0006] Preferably, estimating the white balance gain parameter of the RAW image specifically includes the following steps: The red, green, and blue sampling planes of the RAW image are established based on the pixel sampling positions of the Bayer array; Based on the red, green, and blue sampling planes, color statistical samples with resolutions lower than a set value are constructed. Sample sets that meet the conditions of brightness, saturation, and local gradient threshold are selected to form a candidate background domain. The white balance gain parameters of the red, green, and blue channels in the candidate background domain are calculated.

[0007] Preferably, establishing the red sampling plane, green sampling plane, and blue sampling plane of the RAW image based on the pixel sampling positions of the Bayer array includes the following steps: Obtain the width, height, and bit depth of the RAW image, and determine the Bayer array layout rules based on the width, height, and bit depth; Initialize the red, green, and blue sampling planes, which are the same size as the RAW image, respectively, and initialize all pixels in the three planes to invalid marker values; Iterate through each pixel coordinate (x, y) of the RAW image and determine the color channel corresponding to the current coordinate based on the Bayer array arrangement rules; If the current coordinates (x, y) are the red sampling position, the pixel value of that position in the RAW image is assigned to the (x, y) coordinate position of the red sampling plane. The green sampling plane and the blue sampling plane retain invalid label values ​​at the same coordinates (x, y). If the current coordinates (x, y) are the green sampling position, the pixel value of that position in the RAW image is assigned to the (x, y) coordinate position of the green sampling plane, while the red sampling plane and the blue sampling plane retain invalid label values ​​at the same coordinates (x, y). If the current coordinates (x, y) are the blue sampling position, the pixel value of that position in the RAW image is assigned to the (x, y) coordinate position of the blue sampling plane, while the red sampling plane and the green sampling plane retain invalid label values ​​at the same coordinates (x, y). After traversing all pixels and assigning values, we obtain a red sampling plane, a green sampling plane, and a blue sampling plane with invalid padding.

[0008] Preferably, the step of using the green channel as the brightness structure reference and adaptively recovering the missing green channel values ​​at the red and blue sampling positions of the RAW image based on horizontal and vertical gradients, adjacent green sampling values, and the constraint of the proportion of low frequencies of the same color specifically includes the following steps: Read the constructed green sampling plane, red sampling plane and blue sampling plane, and configure preset parameters, including proportional compensation intensity, same color gradient contribution coefficient, proportional threshold and gradient judgment threshold; Traverse all pixel coordinates of the RAW image, skipping the original green sampling positions, and determine the red and blue sampling positions as the target positions to be restored in the green channel; for each target position, obtain the adjacent green sampling values ​​above, below, left, and right, as well as the original sampling values ​​of the same color two pixels apart, and calculate the low-frequency estimates of the same color in the horizontal and vertical directions respectively; combine the original sampling values ​​of the target position with the ratio threshold to perform a limiting operation, and solve for the horizontal and vertical ratios corresponding to the target position; Based on adjacent green sample values, horizontal ratio, vertical ratio, and ratio compensation intensity, calculate horizontal green candidate values ​​and vertical green candidate values, and then limit the amplitude of the candidate values ​​by combining the maximum and minimum values ​​of the adjacent green samples. By combining the sampling differences between adjacent green samples, the sampling differences of the same color at a distance, and the gradient contribution coefficient of the same color, the horizontal and vertical gradients of the target position are calculated, and the horizontal and vertical weights are calculated based on the horizontal and vertical gradients. The weights are the reciprocal of the square of the corresponding gradient plus one. By comparing the horizontal gradient and the vertical gradient, when the difference between the two is greater than the gradient judgment threshold, the green candidate value in the direction with the smaller gradient is selected as the green output of the current position; when the difference is not greater than the gradient judgment threshold, the two sets of green candidate values ​​are weighted and fused using horizontal and vertical weights to obtain the green channel value restored for the current position. Write the recovered green channel value to the corresponding coordinates of the green sampling plane. After traversing all target positions, output the completed green channel.

[0009] Preferably, the step of calculating the initial normalized gain color difference at the original red sampling position and the original blue sampling position based on the white balance gain parameter and the recovered green channel specifically includes the following steps: Obtain the white balance gain parameters of the red channel Green channel white balance gain parameters Blue channel white balance gain parameters Read the completed green channel value and the original red and blue sample values ​​of the RAW image. B ; Iterate through all pixel coordinates of the RAW image and filter out all native red sampling locations; For each native red sampling location, extract the original red sampling value at that location. R The green channel value that has been restored at the same coordinate position G According to the formula Calculate the initial red normalized gain color difference at this location, and assign the calculation result to the corresponding coordinates in the red normalized gain color difference plane; Set the coordinates of the red normalized gain chromatic difference plane corresponding to the non-native red sampling positions as invalid markers; traverse all pixel coordinates of the image and filter out all native blue sampling positions; For each native blue sampling location, extract the original blue sampling value at that location. B Retrieve the restored green channel value at the same coordinate position. G According to the formula Calculate the initial blue normalized gain color difference at this location, and assign the calculation result to the corresponding coordinates in the blue normalized gain color difference plane; Set the coordinates of the blue normalized gain chromatic difference plane corresponding to the non-native blue sampling positions as invalid markers; Complete the calculation of all native red and native blue sampling positions, and output the initialized red normalized gain color difference and blue normalized gain color difference.

[0010] Preferably, before reconstructing the RGB channels by combining white balance gain, the recovered green channel, and the initial normalized gain color difference, the process further includes optimizing the initial normalized gain color difference, specifically including the following optimization steps: Based on the spatial relationship between the green sampling position and the adjacent red and blue sampling positions in the Bayer array, the normalized gain color difference is propagated along the corresponding direction at the green sampling position, and the adaptive weights of the adjacent color differences are determined by using the local gradient of the green channel. The normalized gain color difference of blue is recovered at the red sampling position, and the normalized gain color difference of red is recovered at the blue sampling position. Candidate color differences are calculated for the two diagonal directions respectively, and weighted fusion is performed according to the diagonal gradient of the green channel to complete the cross color difference compensation at the different color sampling positions and obtain the complete normalized gain color difference plane. The complete normalized gain color difference plane is repaired and the local median consistency constraint is optimized. The optimization is applied only to the color difference plane and does not directly smooth the green channel, resulting in the optimized normalized gain color difference.

[0011] Preferably, the RGB channels are reconstructed by combining the white balance gain parameter, the recovered green channel, and the optimized normalized gain color difference, and the corrected RGB image is output. This specifically includes the following steps: Obtain white balance gain parameters , , Read the red normalized gain color difference after propagation, cross-compensation, and local consistency optimization. Planar and Blue Normalized Gain Color Difference Plane and complete greenway plan; Iterate through each pixel coordinate of the RGB image and extract the optimized red normalized gain color difference at the current coordinate. Blue normalized gain color difference And the green channel value at the same coordinate. G ; According to the formula Calculate the red channel value after white balance correction at the current coordinates. ; According to the formula Calculate the green channel value after white balance correction for the current coordinates. ; According to the formula Calculate the blue channel value after white balance correction at the current coordinates. ; Determine the upper and lower limits of pixel values ​​based on the target bit depth of the output image, and then adjust the red channel values ​​accordingly. Green channel value Blue channel value Perform a truncation operation to limit the pixel values ​​of each channel to within the target dynamic range; The truncated red, green, and blue pixel values ​​are sequentially assigned to the corresponding coordinate positions in the RGB image. After traversing all pixels, output an RGB image with white balance correction.

[0012] This invention also proposes a RAW image reconstruction system, comprising: The white balance gain parameter acquisition module is used to acquire RAW images with Bayer array data and estimate the white balance gain parameters of the RAW images. The green channel recovery and normalized gain color difference initialization module is used to adaptively recover missing green channel values ​​at the red and blue sampling positions of the RAW image, based on horizontal and vertical gradients, adjacent green sampling values, and low-frequency ratio constraints of the same color, using the green channel as the brightness structure reference. It calculates the initialized normalized gain color difference at the original red and original blue sampling positions according to the white balance gain parameters and the recovered green channel. The horizontal and vertical gradients are composite gradients at the red or blue sampling positions where the green channel value to be recovered is obtained by weighted summation of the directional differences between adjacent green samples and the directional differences between two pixel sampling points of the same color. The adjacent green sampling values ​​are the sensor's native green pixels directly adjacent to the red and blue sampling points in all directions. The low-frequency ratio constraint of the same color estimates the local brightness baseline using the original RAW pixel values ​​of the same color separated by the current red and blue sampling points to obtain the brightness ratio, which is used to correct candidate green values. The image reconstruction module combines the white balance gain parameter, the restored green channel, and the initialized normalized gain color difference to reconstruct the RGB channels and output the corrected RGB image.

[0013] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement any of the steps in the RAW image reconstruction method.

[0014] The present invention also provides a computer-readable storage medium storing a computer program, which, when loaded by a processor, is capable of executing any of the steps in the RAW image reconstruction method.

[0015] The RAW image reconstruction method provided by this invention has the following beneficial effects: This invention departs from the conventional approach of sequential processing of white balance and de-mosaic. Instead of pre-multiplying the white balance gain by the original RAW sample, it transforms the white balance gain into a constraint for color restoration and embeds it into the RAW domain processing flow. First, this invention uses the green channel as the luminance structure benchmark, combining directional gradient, adjacent green samples, and low-frequency ratio constraints of the same color to adaptively restore missing green pixels. This constraint introduces local luminance information from the red and blue samples themselves to assist in green reconstruction, avoiding the loss of detail caused by simply relying on neighboring green interpolation. It fully preserves the main luminance and edge structure carried by the green channel, avoiding the problem of luminance detail loss. Furthermore, it utilizes the white balance gain and the restored green channel to construct a normalized gain color difference, encapsulating the channel response relationship into a color difference variable. The white balance gain participates in the color difference normalization calculation, rather than directly scaling the original red and blue RAW data, fundamentally preventing red and blue channel noise and outliers from being amplified by the gain and then propagating with interpolation.

[0016] White balance parameters are no longer independent pre-processing or post-processing coefficients, but rather constrain the definition and propagation of color difference, ensuring that the color restoration process in demosaicing always follows the channel response relationship and guarantees cross-channel color consistency. The entire optimization operation operates on the color difference plane without destroying the luminance information of the green channel, taking into account noise suppression, false color suppression, and detail preservation, achieving joint optimization of RAW domain white balance correction and demosaicing. Attached Figure Description

[0017] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is an overall flowchart of the RAW image reconstruction method proposed in this invention; Figure 2 This is a schematic diagram of the Bayer color filter array and the native sampling positions of red, green, and blue pixels; among them... Figure 2 (a) is a Bayer RGGB array structure; Figure 2 (b) represents the original red sampling location; Figure 2 (c) represents the original green sampling location; Figure 2 (d) represents the original blue sampling location; Figure 3 This is the original Bayer RAW image; Figure 4 This is a schematic diagram of neighborhood sampling; Figure 5 This is a diagram showing the restored green channel; Figure 6This is a schematic diagram illustrating the cross-compensation of the position difference between the red and blue sampling positions; where, Figure 6 (a) is a schematic diagram of blue normalized gain color difference cross-compensation at the red sampling position. Figure 6 (b) is a schematic diagram of red normalized gain color difference cross-compensation at the blue sampling position; Figure 7 This is a schematic diagram of the software interface of the joint reconstruction system of the present invention; wherein, Figure 7 (a) is a preview of the input image - the BayerRAW raw image. Figure 7 (b) is a preview of the output image - the RGB image; Figure 8 Visual comparison of white balance correction results from different white balance methods; Figure 9 Comparison of reconstruction results and magnified details of different demosaic methods. Detailed Implementation

[0019] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0020] Traditional methods typically treat white balance and demosaicing as two independent processing stages, with white balance gain not participating in the color restoration process. This invention, however, does not alter the original Bayer sampling structure. Instead, based on the restoration of the green channel, it establishes a normalized gain color difference according to the white balance gain parameters, and uses this normalized gain color difference as the red and blue color information propagation variable. This ensures that the color restoration process simultaneously satisfies spatial structure constraints and channel response constraints. This invention aims to address the issues of inconsistent color restoration, loss of detail structure, and false color in edge regions caused by the independent processing of white balance correction and demosaicing in bright-field microscopy Bayer RAW imaging. Specifically, it proposes a RAW domain joint white balance and demosaicing reconstruction method based on white balance gain constraint color difference propagation.

[0021] This invention does not pre-multiply the white balance gain directly to all sampled values ​​of the Bayer RAW image. Unlike traditional methods of white balance followed by demosaic or demosaic followed by white balance, this invention introduces the white balance gain parameter into the RAW domain color residual space, ensuring that color recovery during the demosaic process always satisfies the channel gain constraint relationship. First, the green reference channel is recovered. Then, a normalized gain color difference model is established based on the white balance gain. Directional propagation, cross compensation, and local consistency optimization are performed within this color difference domain. Finally, the white balance RGB image is recovered based on the gain color difference model.

[0022] like Figure 1 As shown, the RAW image reconstruction method proposed in this invention includes eight main steps: RAW image input, white balance gain estimation, green channel restoration, normalized gain chromatic difference construction, directional chromatic difference propagation, cross-chromatic difference compensation, chromatic difference consistency optimization, and white balance RGB image restoration, as detailed below: S1: Acquire a RAW image with Bayer array data, determine the image size, bit depth, and color filter array arrangement, and establish red, green, and blue sampling planes according to the sampling positions, specifying the native color sampling type corresponding to each coordinate (x, y). The RAW image with Bayer array data is raw color sampling data acquired by a CMOS image sensor configured with a Bayer color filter array, and has not yet undergone color interpolation reconstruction (de-mosaic) or color correction processing such as white balance.

[0023] S2: Based on the red, green, and blue sampling planes, low-resolution color (red, green, and blue) statistical samples are constructed using Bayer neighborhood sampling. Brightness, saturation, and local gradient are used to filter candidate background regions, estimating the white balance gain parameters for the red, green, and blue channels of the RAW image. The white balance gain parameters can be obtained from existing automatic white balance methods, camera calibration parameters, or user-defined methods; this embodiment uses a Bayer neighborhood-based statistical approach for estimation. Candidate background regions are selected by calculating the brightness, saturation, and local gradient of the RGB statistical samples constructed based on Bayer neighborhood sampling, and filtering out a set of samples that meet the threshold conditions.

[0024] In this embodiment, the estimation of the white balance gain parameter of the RAW image specifically includes the following steps: The red, green, and blue sampling planes of the RAW image are established based on the pixel sampling positions of the Bayer array.

[0025] Based on the red, green, and blue sampling planes, color statistical samples with resolutions lower than a set value are constructed. Sample sets that meet the conditions of brightness, saturation, and local gradient threshold are selected to form a candidate background domain. The white balance gain parameters of the red, green, and blue channels in the candidate background domain are calculated.

[0026] Furthermore, establishing the red, green, and blue sampling planes of the RAW image based on the pixel sampling positions of the Bayer array includes the following steps: Obtain the width, height, and bit depth of the RAW image, and determine the Bayer array layout rules based on the width, height, and bit depth.

[0027] Initialize the red, green, and blue sampling planes, which are the same size as the RAW image, respectively, and initialize all pixels in the three planes to invalid marker values.

[0028] Iterate through each pixel coordinate (x, y) of the RAW image and determine the color channel corresponding to the current coordinate based on the Bayer array arrangement rules.

[0029] If the current coordinates (x, y) are the red sampling position, the pixel value of that position in the RAW image is assigned to the (x, y) coordinate position of the red sampling plane. The green sampling plane and the blue sampling plane retain invalid label values ​​at the same coordinates (x, y).

[0030] If the current coordinates (x, y) are the green sampling position, the pixel value of that position in the RAW image is assigned to the (x, y) coordinate position of the green sampling plane. The red sampling plane and the blue sampling plane retain invalid label values ​​at the same coordinates (x, y).

[0031] If the current coordinates (x, y) are the blue sampling position, the pixel value of that position in the RAW image is assigned to the (x, y) coordinate position of the blue sampling plane. The red sampling plane and the green sampling plane retain invalid label values ​​at the same coordinates (x, y).

[0032] After traversing all pixels and assigning values, we obtain a red sampling plane, a green sampling plane, and a blue sampling plane with invalid padding.

[0033] S3: Using the green channel as the brightness structure benchmark, at the red and blue sampling positions of the RAW image, the missing green channel values ​​are adaptively recovered based on the horizontal and vertical gradients, adjacent green sample values, and the low-frequency ratio constraint of the same color. The horizontal and vertical gradients are the composite gradients at the red or blue sampling positions of the green channel values ​​to be recovered, obtained by weighted summation of the directional differences between adjacent green samples and the directional differences between two pixel sampling points of the same color channel. Adjacent green sample values ​​are the sensor's native green pixels directly adjacent to the red and blue sampling points in all directions. The low-frequency ratio constraint of the same color estimates the local brightness baseline using the original RAW pixel values ​​of the same color space between the current red and blue sampling points to obtain the brightness ratio, which is used to correct the candidate green values.

[0034] The adaptive recovery of missing green channel values ​​specifically includes the following steps: Read the constructed green sampling plane, red sampling plane, and blue sampling plane, and configure the preset parameters, including the proportional compensation intensity, the same-color gradient contribution coefficient, the proportional threshold, and the gradient judgment threshold.

[0035] Traverse all pixel coordinates of the RAW image, skipping the original green sampling positions, and determine the red and blue sampling positions as the target positions to be restored in the green channel; for each target position, obtain the adjacent green sampling values ​​above, below, left, and right, as well as the original sampling values ​​of the same color two pixels apart, and calculate the low-frequency estimates of the same color in the horizontal and vertical directions respectively; combine the original sampling values ​​of the target position with the scale threshold to perform a limiting operation, and solve for the horizontal and vertical scale corresponding to the target position.

[0036] Based on adjacent green sample values, horizontal ratio, vertical ratio, and ratio compensation intensity, horizontal green candidate values ​​and vertical green candidate values ​​are calculated. Then, the maximum and minimum values ​​of the adjacent green samples are combined to limit the amplitude of the candidate values.

[0037] By combining the sampling differences between adjacent green samples, the sampling differences of the same color samples at a distance, and the gradient contribution coefficient of the same color, the horizontal and vertical gradients of the target location are calculated. Based on the horizontal and vertical gradients, the horizontal and vertical weights are calculated, and the weights are the reciprocal of the square of the corresponding gradient plus one.

[0038] By comparing the horizontal gradient and the vertical gradient, when the difference between the two is greater than the gradient judgment threshold, the green candidate value in the direction with the smaller gradient is selected as the green output of the current position; when the difference is not greater than the gradient judgment threshold, the two sets of green candidate values ​​are weighted and fused using horizontal and vertical weights to obtain the green channel value restored for the current position.

[0039] Write the recovered green channel value to the corresponding coordinates of the green sampling plane. After traversing all target positions, output the completed green channel.

[0040] S4: Based on the white balance gain parameters and the recovered green channel, calculate the initial normalized gain color difference at the original red sampling position and the original blue sampling position, respectively. In this invention, the normalized gain color difference differs from the direct color difference in the traditional RGB space. It constrains the response scale of the red and blue channels through the white balance gain, ensuring that the subsequent color difference propagation process remains consistent with the white balance color restoration model.

[0041] Specifically, let the white balance gains of the red channel, green channel, and blue channel be respectively... , and The original red sample value is The restored green value The original blue sample value is To avoid the problem that traditional red-green difference (RG) and blue-green difference (BG) methods do not consider the gain scale of different color channels, this invention uses white balance gain parameters to normalize and constrain the color residuals. The red normalized gain color difference is defined as follows: and blue normalized gain color difference They are respectively: ; ; in, and This represents the initial normalized gain color difference calculated from the original sampling location. During subsequent propagation, cross-compensation, and consistency optimization processes, the color difference variable is updated to... and These two are not equivalent. The final restoration stage utilizes the optimized normalized gain color difference to restore the white balance RGB image. In this embodiment, the green channel is selected as the brightness reference, therefore... =1, only using and Describe the response compensation of the red and blue channels relative to the green channel; in other implementations, It can also participate in unified normalization as an independent gain. As can be seen from the above model, the white balance gain participates in both the definition of color difference and the final color restoration, thus placing the color difference propagation process and the white balance output under a unified mathematical constraint. Therefore, in this invention, the white balance gain is no longer a post-processing parameter independent of the demosaic process, but becomes a constraint variable in the color restoration process.

[0042] S5: Optimize normalized gain color difference. Specifically, based on the spatial relationship between the green sampling position and the adjacent red and blue sampling positions in the Bayer array, the normalized gain color difference is propagated along the corresponding direction at the green sampling position, and the adaptive weights of adjacent color differences are determined using the local gradient of the green channel. By using the green channel as the main structural reference and performing directional color difference propagation in the normalized gain color difference space, the propagation process is simultaneously constrained by the Bayer spatial adjacency relationship and the color channel response ratio, reducing the color shift caused by channel gain differences in the traditional color difference propagation process.

[0043] The normalized gain color difference of blue is recovered at the red sampling position, and the normalized gain color difference of red is recovered at the blue sampling position. Candidate color differences are calculated for the two diagonal directions respectively, and weighted fusion is performed based on the diagonal gradient of the green channel to complete the cross-color difference compensation at the different color sampling positions. The purpose of the cross-color difference compensation is to address the problem that the red and blue sampling spatial positions do not coincide in the Bayer array. By selecting a more continuous diagonal propagation path through the green structural gradient, the color shift in high-frequency texture areas is reduced.

[0044] The invention performs effective repair and local median consistency constraint optimization on the complete normalized gain chromatic aberration plane. This optimization only applies to the chromatic aberration plane and does not directly smooth the green channel. Since the green channel contains key brightness and edge structure information, this invention avoids smoothing the green channel and only optimizes the color residual to achieve a balance between color noise suppression and spatial detail preservation.

[0045] S6: Based on the white balance gain parameters, the restored green channel, and the optimized normalized gain color difference, reconstruct the white balance red, green, and blue channels, and output the corrected RGB image, as follows: Obtain white balance gain parameters , , Read the red normalized gain color difference after propagation, cross-compensation, and local consistency optimization. Planar and Blue Normalized Gain Color Difference Plane and complete greenway plan.

[0046] Iterate through each pixel coordinate of the RGB image and extract the optimized red normalized gain color difference at the current coordinate. Blue normalized gain color difference And the green channel value at the same coordinate. G .

[0047] According to the formula Calculate the red channel value after white balance correction at the current coordinates. According to the formula Calculate the green channel value after white balance correction for the current coordinates. According to the formula Calculate the blue channel value after white balance correction at the current coordinates. .

[0048] Determine the upper and lower limits of pixel values ​​based on the target bit depth of the output image, and then adjust the red channel values ​​accordingly. Green channel value Blue channel value Perform a truncation operation to limit the pixel values ​​of each channel to within the target dynamic range.

[0049] The truncated red, green, and blue pixel values ​​are sequentially assigned to the corresponding coordinate positions in the RGB image.

[0050] After traversing all pixels, output an RGB image with white balance correction.

[0051] The method of the present invention will be further described below through specific embodiments.

[0052] Example 1: RAW Image Input and Bayer Sampling Plane Establishment like Figure 2 As shown, for a Bayer color filter array using the RGGB arrangement, each pixel position of the sensor only samples one of the three color components: red, green, and blue. The red, green, and blue sampling positions are arranged according to a fixed spatial period. For image coordinates (x, y), three sparse sampling planes for red, green, and blue are established according to the RGGB Bayer arrangement.

[0053] Figure 2 Taking the RGGB arrangement as an example: when both x and y are even numbers, it is the red sampling position; when both x and y are odd numbers, it is the blue sampling position; and the remaining positions are green sampling positions. Each pixel only samples one color component, and de-mosaic processing is needed to recover the complete RGB image. Different Bayer arrangements (BGGR, GRBG, GBRG) only differ in color position, but the principle is the same.

[0054] Red sampling positions are those where both x and y are even; blue sampling positions are those where both x and y are odd; and green sampling positions are the remaining positions. For 10-bit, 12-bit, 14-bit, or 16-bit RAW images, the upper limit of the dynamic range and the correlation threshold can be adjusted proportionally to the bit depth.

[0055] Example 2: White Balance Gain Estimation This invention first constructs low-resolution color statistical samples based on Bayer neighborhood sampling, and then performs joint screening using brightness, saturation, and regional gradient. At this stage, the gain parameter is only estimated and stored, without being directly pre-multiplied to the original RAW sample value. Brightness quantiles are used to select candidate points close to white regions, saturation constraints are used to exclude obviously colored regions, and gradient constraints are used to exclude edge texture regions, thereby improving the stability of light source estimation. Brightness is calculated for each statistical sample. Y saturation S and local gradient Grad The brightness can be expressed as: ; In the formula, the subscript i The first term constructed based on Bayer neighborhood sampling is represented as... i Each RGB statistical sample number.

[0056] The saturation formula is: ; Candidate background samples with high brightness but not yet saturated are selected based on brightness quantiles, and textured and edge regions are excluded using a local gradient threshold. The local gradient formula is: ; In the formula, Represents the pixel value function. s This is the sampling step size.

[0057] Then based on the set gradient threshold And the luminance quantiles P70 and P95 are used to select candidate background areas: ; When the number of candidate samples is insufficient, the saturation threshold and gradient threshold can be relaxed step by step.

[0058] Then, the median of the red, green, and blue statistics in the candidate background samples were calculated respectively. , and And obtain the white balance gain as follows: ; ; ; in, This indicates that the gain will be limited to a preset range. This is the lower limit threshold for gain calculation during white balance gain. This is the upper limit threshold for the gain calculated during white balance gain. For example, 0.40 is acceptable. A value of 2.80 is acceptable. The above values ​​are merely examples and do not constitute a limitation.

[0059] Example 3: Adaptive recovery of the direction of the green channel Figure 3 For Bayer RAW raw images, this invention uses the green channel as the brightness structure benchmark and constructs horizontal green candidate values ​​at the red and blue sampling positions, respectively. and vertical green candidate values The missing green information is recovered by combining directional gradient constraints and low-frequency ratio constraints of the same color. At the original green sampling position, G(x,y) is directly written into the green plane. In this embodiment, directional gradient constraints are used to maintain the edge direction and brightness structure, and the low-frequency change trend of the current color sampling is maintained by the same color ratio constraints. Missing green values ​​are only recovered at the red or blue sampling positions.

[0060] Let the original red or blue sample value at the center be c, and the green sample values ​​to its left and right be c. , The adjacent green sample values ​​are , The sampled values ​​of two pixels of the same color with a spacing of two pixels are respectively , , and Neighborhood sampling diagram as shown Figure 4 As shown.

[0061] Then the horizontal and vertical low-frequency estimates of point c are the same color. and The calculation formula is: ; ; By preset Threshold to obtain horizontal proportion Vertical proportions : ; ; Finally, the green candidate values ​​in the horizontal direction were obtained. and vertical green candidate values for: ; ; in, The strength is proportional to the compensation. This is a proportional threshold used to limit the estimation of low-frequency colors. and The lower limit. For example, 0.5 is acceptable. 0.5 is acceptable. A value of 2.0 is acceptable. To avoid candidate green values ​​deviating from the local true green range, further adjustments can be made based on... , , Minimum, maximum and preset margin and Limit the amplitude.

[0062] The horizontal gradient and the vertical gradient can be expressed as follows: ; ; in This is the contribution coefficient of the same-color gradient, used to adjust the impact of long-distance same-color sampling on direction determination.

[0063] when and If the difference exceeds a preset threshold, the direction with the smaller gradient is selected; otherwise, the weights of the inverse square of the gradient are used. and To merge: ; in and for: ; The boundary region is then restored by means of existing adjacent native green sample values. The restored green channel is as follows: Figure 5 As shown.

[0064] Example 4: Normalized Gain Chromatic Aberration Initialization and Directional Propagation This invention first establishes a normalized gain color difference model using white balance gain parameters, and then performs directional color difference propagation at the green sampling position based on the spatial relationship between the red, green, and blue sampling positions in the Bayer array. After completing the green channel restoration, it does not directly restore RGB, but instead performs directional color difference propagation at the original red sampling position according to... Initialize the red normalized gain color difference, and then sample the original blue color difference according to... Initialize blue normalized gain color difference.

[0065] Because green samples in the RGGB Bayer array have two spatial types, different propagation strategies are used for different green positions to match the actual spatial adjacency of red and blue samples. For a green sample position located in the red row, the red normalized gain color difference propagates from the red sample positions to its left and right, while the blue normalized gain color difference propagates from the blue sample positions above and below it. Taking the red normalized gain color difference as an example: ; ; ; For the green sampling position located in the blue row, the red normalized gain chromatic aberration propagates vertically, while the blue normalized gain chromatic aberration propagates horizontally. Thus, the chromatic aberration at the green position is recovered using the native sampling geometry of the Bayer array.

[0066] Example 5: Cross-color difference compensation at different color sampling locations like Figure 6 As shown, at the red and blue sampling positions, since the other color channel lacks native sampling, this invention utilizes a diagonal propagation method guided by the green structural gradient to recover the heterochromatic normalized gain color difference. At the red sampling position, the blue normalized gain color difference needs to be recovered; at the blue sampling position, the red normalized gain color difference needs to be recovered. Taking the blue normalized gain color difference at the red sampling position as an example, two candidate values ​​are constructed along the main diagonal and the secondary diagonal, respectively: ; ; 1; 1; ; In this embodiment, candidate color differences are formed in two diagonal directions to avoid structural mismatch caused by simple diagonal averaging. Weighted fusion using the diagonal gradient of the green channel helps reduce color fringing, color misalignment, and false color. The normalized gain color difference of the red sample location is calculated in the same way. This cross-compensation step changes the restoration of dissimilar color channels from simple diagonal averaging to diagonal fusion guided by the green structural gradient.

[0067] Example 6: Optimization of Color Difference Effectiveness and Local Consistency After completing the full-image color difference propagation, this invention only performs effectiveness repair and median consistency constraint optimization on the normalized gain color difference plane, without directly processing the green luminance channel. After completing the full-image color difference propagation, the boundary color difference is extended using neighboring internal values. or When the value is not finite or is an outlier, the average finite color difference in its four neighboring regions is preferentially used for repair. This effective repair is used to avoid invalid color difference propagation caused by boundary extension, floating-point calculation anomalies, or extreme noise. For example: ; If no valid value exists in the four neighboring domains, the value will revert to the default value.

[0068] For each non-boundary location, establish a 3×3 color difference neighborhood, and calculate the median med of the normalized gain color difference for red and blue respectively. With med : ; The color difference after local median consistency constraint is: ; ; in, The median constraint strength can be 0.60, for example.

[0069] Subsequently, local weighted uniform smoothing is performed on the normalized gain color difference. For example, the center weight is 4, the weights of horizontally and vertically adjacent positions are 2, and the weights of diagonally opposite positions are 1. For example: ; Finally, the low-pass result is blended with the original color difference in the following way: ; in, The low-pass intensity for chromatic aberration can be 0.45, for example. Neither the median constraint nor the low-pass optimization directly affects the green channel, which helps to balance color depth stability and image detail. The optimized normalized gain chromatic aberration is output for subsequent white balance RGB reconstruction.

[0070] Example 7: White Balance RGB Restoration and Output Finally, this invention reconstructs the complete white balance RGB image based on the optimized normalized gain color difference of red and blue and the restored green channel, following a unified mathematical model. Based on the optimized normalized gain color difference of red and blue, and according to the color recovery relationship established by the normalized gain color difference model, the complete white balance RGB image is reconstructed. ; ; ; For 8-bit output, the recovered value is limited to the range of 0 to 255 before quantization: ; ; ; For other output bit depths, the dynamic range is limited accordingly. Further color correction matrix, gamma correction, tone mapping, or display space conversion can be performed after this step. These subsequent processing steps do not affect the core technical solution of this invention in the RAW domain joint reconstruction stage.

[0071] Example 8 This invention also provides a software system for joint reconstruction of the RAW domain based on white balance gain-constrained chromatic aberration propagation. Figure 7 This is a preview image of the input image—a Bayer RAW original image—and the output image—an RGB image—displayed in the software interface of the joint reconstruction system of the present invention. The software system is used to read the Bayer RAW image, set white balance parameters, perform joint reconstruction processing, preview the image, and save the results.

[0072] In one specific implementation, the software system is developed using C++, employs the Qt framework to construct the graphical user interface, and uses OpenCV to perform image preview, format conversion, and saving of processing results. The software system includes a RAW input and parameter setting area, an image preview area, and a running status display area.

[0073] 1. RAW file input and size setting module: Used to select the RAW file to be processed, set the image width and image height, and check whether the data length of the RAW file matches the current image size.

[0074] 2. Camera Connection and Acquisition Module: Used to connect to CMOS / Bayer image acquisition devices with RAW data output capability and acquire the current frame RAW data.

[0075] 3. White Balance Parameter Module: Used for automatic estimation, manual setting, and resetting of the white balance gain of the red, green, and blue channels. After the white balance gain is estimated or set, it is only saved as a constraint parameter in the joint reconstruction process and is not directly pre-multiplied to all sampled values ​​of the input RAW image.

[0076] 4. Joint Reconstruction Processing Module: Used to perform green channel restoration, normalized gain color difference propagation, and white balance RGB restoration.

[0077] 5. Image Preview Module: Used to display the RAW input preview and the RGB output result after joint reconstruction, respectively; 6. Result saving and status prompt module: Used to save the processing results and display the file status, white balance status, and error prompts.

[0078] The software system described above is a specific implementation of the joint reconstruction method of the present invention. Its graphical interface layout, button names, and display methods can be adjusted according to actual application needs without affecting the technical essence of the present invention to complete the joint reconstruction of the RAW domain based on white balance gain constraint color difference propagation.

[0079] Algorithm performance verification To verify the effectiveness of the RAW domain joint reconstruction method based on white balance gain constraint color difference propagation proposed in this invention, experimental tests were conducted on both white balance correction performance and demosaic reconstruction performance.

[0080] This experiment uses actual Bayer RAW images as input and compares them with traditional white balance methods and classic demosaic methods. Evaluation metrics include color restoration accuracy, structure preservation ability, and edge restoration ability.

[0081] 1. White balance correction performance test: The white balance experiment used Bayer RAW images acquired by the Mingmei MS60 microscope camera and a Basler industrial camera as test data. Since differences in light source, optical path, and sensor response in the microscope imaging system can cause significant shifts in the RGB channels, a white balance correction experiment was first performed on the RAW images. Comparison methods included: color temperature estimation, Gray World algorithm, and Perfect Reflection algorithm. Objective comparisons were conducted using the average color difference C_ab, ΔAE, BE, and information entropy as evaluation results. Smaller values ​​for C_ab, ΔAE, and BE indicate lower color deviation, while higher information entropy indicates richer image information.

[0082] Subjective effects such as Figure 8 As shown, different methods yield significant differences in white balance results. Images without white balance correction exhibit noticeable color shifts, with imbalanced red and blue channel responses, resulting in an overall color tone that is either too cool or too warm. The gray-world method is easily affected by large stained areas in dyed samples, misinterpreting the sample's own color as a light source color cast, leading to a noticeable color cast in the image. The perfect reflection method relies on bright areas in the image, and when there is local overexposure, dust, or bubble interference in the background area, it is prone to unstable gain estimation. The color temperature estimation method can correct the overall color cast to some extent, but its adaptability to complex staining conditions in microscopic images is insufficient. In contrast, the method of this invention constrains color difference propagation through white balance gain, ensuring that the color recovery process remains consistent with the channel response relationship, achieving a more natural and stable color performance while maintaining the tissue texture structure. Objective indicators are shown in Table 1: Table 1 Objective Indicators 1

[0083] Table 1 shows that the algorithm of this invention achieves optimal results in the three main indicators: C_ab, ΔAE, and BE. Specifically, C_ab decreased from 12.2334 in the unbalanced image to 9.6237, indicating that the algorithm effectively reduces color cast in the background region; ΔAE decreased from 6.4823 to 3.9233, indicating that the algorithm estimates the illumination color more accurately; and BE decreased from 0.2634 to 0.1606, indicating that the RGB channel response is more balanced after white balance. Experimental results show that the method of this invention can utilize white balance gain to constrain the color recovery process, preserving the structural information of the microscopic image while reducing color errors.

[0084] 2. Performance test of mosaic reconstruction To further verify the joint demosaicing reconstruction capability, this experiment used microscopic image data from Olympus Life Sciences' brightfield microscope digital image library. The test RAW data was generated from a reference RGB image through Bayer sampling simulation. This data contains a large number of cellular, tissue, and microstructural regions with rich edges and high-frequency textures, effectively evaluating the demosaicing algorithm's ability to restore details and suppress false colors. Comparison methods included: Bilinear interpolation, Edge Adaptive (EA), Malvar-He-Cutler (MHC), and Variable Number of Gradients (VNG).

[0085] like Figure 9As shown, different algorithms exhibit significant differences in cell edges, tissue textures, and high-frequency structural regions. While bilinear interpolation is computationally fast, it easily produces blurring and jagged edges at cell boundaries and texture junctions because it doesn't consider image edge direction. The Malvar-He-Cutler algorithm achieves good overall reconstruction results, but slight false-color edges may still appear in high-frequency texture regions. The VNG and EA algorithms can utilize gradient information to select the interpolation direction, but they are prone to unstable direction judgment in areas with fine textures and rapidly changing directions in microscopic images, resulting in local artifacts. The algorithm of this invention performs better in terms of edge sharpness, texture continuity, and false-color suppression. Objective indicators are shown in Table 2: Table 2 Objective Indicators II

[0086] Table 2 shows that the PSNR of the algorithm of this invention is 26.8396 dB, SSIM is 0.937263, and ΔE76 is 5.3632. All three metrics are superior to other comparative algorithms, indicating that the algorithm of this invention performs well in terms of overall pixel reconstruction accuracy, structure preservation capability, and color restoration accuracy. Compared with bilinear interpolation, the algorithm of this invention significantly improves PSNR and SSIM and reduces color difference, indicating an effective reduction in edge blurring and color crosstalk caused by simple average interpolation. Compared with EA and VNG, the algorithm of this invention performs more stably in complex texture regions and reduces artifacts caused by orientation misjudgment. The Edge RMSE metric shows that the method of this invention achieves better overall performance in complex edge restoration. Although the MHC method has a lower error in this metric, its color error ΔE76 is significantly higher than that of the method of this invention, indicating that simply optimizing the edge response may lead to a decrease in color restoration capability.

[0087] Based on the above experimental results, the method of this invention demonstrates good color recovery and structure preservation capabilities on actual Bayer RAW microscopic data. Compared to traditional methods that separate white balance and demosaic processing, this method utilizes white balance gain to establish a color constraint relationship, allowing white balance parameters to participate in the RAW domain color recovery process, rather than being treated as independent post-processing parameters. Experimental results prove that the joint reconstruction method proposed in this invention can effectively reduce color deviation and false color while preserving the detailed structure of microscopic images, making it suitable for bright-field microscopy and other Bayer CMOS imaging systems.

[0088] Compared with the prior art, the present invention has at least the following beneficial effects: (1) The white balance gain is not directly premultiplied by all RAW sample values, which can avoid indiscriminate amplification of noise and outliers in the red and blue channels before de-mosaicing.

[0089] (2) By constructing a normalized gain color difference constrained by white balance gain, the white balance parameter is introduced from the traditional color correction stage to the RAW domain color residual propagation stage, so that the color restoration process considers both spatial structure information and channel response relationship.

[0090] (3) The green channel adopts the combined constraint of directional gradient and low frequency ratio of the same color, which is conducive to maintaining the main brightness structure and edge direction.

[0091] (4) The directional color difference propagation at the green sampling position and the cross-color difference compensation at the red and blue sampling positions help to reduce color fringing, color misalignment and false color. It is particularly suitable for color restoration of cell edges, tissue texture and microstructure regions in brightfield microscopy, and can improve the color authenticity and structure preservation of microscopic images.

[0092] (5) Local median constraint and weighted low-pass only apply to the normalized color difference plane and do not directly blur the green brightness plane, which is beneficial to balance color stability and image details.

[0093] (6) The scheme can be implemented by software, hardware logic, image signal processor, embedded processor or a combination thereof, and is applicable to various Bayer arrangement methods and RAW images with different bit depths.

[0094] Based on the same inventive concept, this invention also proposes a RAW image reconstruction system, comprising: The white balance gain parameter acquisition module is used to acquire RAW images with Bayer array data and estimate the white balance gain parameters of the RAW images.

[0095] The green channel recovery and normalized gain color difference initialization module is used to adaptively recover missing green channel values ​​at the red and blue sampling positions of the RAW image, based on horizontal and vertical gradients, adjacent green sampling values, and the same-color low-frequency ratio constraint, using the green channel as the brightness structure reference. It calculates the initialized normalized gain color difference at the original red and original blue sampling positions according to the white balance gain parameters and the recovered green channel. The horizontal and vertical gradients are the composite gradients at the red or blue sampling positions where the green channel value to be recovered is obtained by weighted summing of the directional differences between adjacent green samples and the directional differences between two pixel sampling points of the same color channel. Adjacent green sampling values ​​are the sensor's native green pixels directly adjacent to the red and blue sampling points in all directions. The same-color low-frequency ratio constraint estimates the local brightness baseline using the same-color original RAW pixel values ​​separated by the current red and blue sampling points to obtain the brightness ratio, which is used to correct candidate green values.

[0096] The image reconstruction module combines the white balance gain parameter, the restored green channel, and the initialized normalized gain color difference to reconstruct the RGB channels and output the corrected RGB image.

[0097] The modules in the aforementioned RAW image reconstruction system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0098] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps in the embodiments of the RAW image reconstruction method. Specific implementation methods can be found in the method embodiments, and will not be repeated here.

[0099] Furthermore, the present invention also provides a non-transitory computer-readable storage medium containing instructions, on which a computer program is stored. For example, a memory containing instructions that can be executed by a processor of a computer device to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc. When the computer program is executed by the processor, it can implement the steps in the RAW image reconstruction method embodiments. Specific implementation methods can be found in the method embodiments, which will not be repeated here.

[0100] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail in this specification and embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention are covered within the protection scope of the present invention patent. No reference numerals in the claims should be construed as limiting the scope of the claims. Any simple variations or equivalent substitutions of technical solutions that can be readily obtained by those skilled in the art within the scope of the technology disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A method for reconstructing RAW images, characterized in that, Includes the following steps: Acquire RAW images with Bayer array data and estimate the white balance gain parameters of the RAW images; Using the green channel as the brightness structure benchmark, the missing green channel values ​​are adaptively recovered at the red and blue sampling positions of the RAW image based on the horizontal and vertical gradients, adjacent green sampling values, and the low-frequency ratio of the same color. Based on the white balance gain parameters and the recovered green channel, the initial normalized gain color difference is calculated at the original red sampling position and the original blue sampling position, respectively. The horizontal and vertical gradients are the composite gradients at the red or blue sampling positions of the green channel value to be recovered, which are obtained by weighted summation of the directional differences between adjacent green samples and the directional differences between two pixel sampling points of the same color channel. The adjacent green sampling values ​​are the sensor's original green pixels that are directly adjacent to the red and blue sampling points in the top, bottom, left, and right directions. The same color low-frequency ratio constraint uses the same color original RAW pixel values ​​of the current red and blue sampling points to estimate the local brightness baseline to obtain the brightness ratio, which is used to correct the candidate green value. By combining the white balance gain parameter, the restored green channel, and the initial normalized gain color difference to reconstruct the RGB channels, the corrected RGB image is output.

2. The RAW image reconstruction method according to claim 1, characterized in that, The estimation of the white balance gain parameter of the RAW image specifically includes the following steps: The red, green, and blue sampling planes of the RAW image are established based on the pixel sampling positions of the Bayer array; Based on the red, green, and blue sampling planes, color statistical samples with resolutions lower than a set value are constructed. Sample sets that meet the conditions of brightness, saturation, and local gradient threshold are selected to form a candidate background domain. The white balance gain parameters of the red, green, and blue channels in the candidate background domain are calculated.

3. The RAW image reconstruction method according to claim 2, characterized in that, The process of establishing the red, green, and blue sampling planes of the RAW image based on the pixel sampling positions of the Bayer array includes the following steps: Obtain the width, height, and bit depth of the RAW image, and determine the Bayer array layout rules based on the width, height, and bit depth; Initialize the red, green, and blue sampling planes, which are the same size as the RAW image, respectively, and initialize all pixels in the three planes to invalid marker values; Iterate through each pixel coordinate (x, y) of the RAW image and determine the color channel corresponding to the current coordinate based on the Bayer array arrangement rules; If the current coordinates (x, y) are the red sampling position, the pixel value of that position in the RAW image is assigned to the (x, y) coordinate position of the red sampling plane. The green sampling plane and the blue sampling plane retain invalid label values ​​at the same coordinates (x, y). If the current coordinates (x, y) are the green sampling position, the pixel value of that position in the RAW image is assigned to the (x, y) coordinate position of the green sampling plane, while the red sampling plane and the blue sampling plane retain invalid label values ​​at the same coordinates (x, y). If the current coordinates (x, y) are the blue sampling position, the pixel value of that position in the RAW image is assigned to the (x, y) coordinate position of the blue sampling plane, while the red sampling plane and the green sampling plane retain invalid label values ​​at the same coordinates (x, y). After traversing all pixels and assigning values, we obtain a red sampling plane, a green sampling plane, and a blue sampling plane with invalid padding.

4. The RAW image reconstruction method according to claim 3, characterized in that, The method, which uses the green channel as the brightness structure benchmark, adaptively recovers the missing green channel values ​​at the red and blue sampling positions of the RAW image based on horizontal and vertical gradients, adjacent green sampling values, and the constraint of the proportion of low frequencies of the same color. Specifically, this includes the following steps: Read the constructed green sampling plane, red sampling plane and blue sampling plane, and configure preset parameters, including proportional compensation intensity, same color gradient contribution coefficient, proportional threshold and gradient judgment threshold; Traverse all pixel coordinates of the RAW image, skipping the original green sampling positions, and determine the red and blue sampling positions as the target positions to be restored in the green channel; for each target position, obtain the adjacent green sampling values ​​above, below, left, and right, as well as the original sampling values ​​of the same color two pixels apart, and calculate the low-frequency estimates of the same color in the horizontal and vertical directions respectively; combine the original sampling values ​​of the target position with the ratio threshold to perform a limiting operation, and solve for the horizontal and vertical ratios corresponding to the target position; Based on adjacent green sample values, horizontal ratio, vertical ratio, and ratio compensation intensity, calculate horizontal green candidate values ​​and vertical green candidate values, and then limit the amplitude of the candidate values ​​by combining the maximum and minimum values ​​of the adjacent green samples. By combining the sampling differences between adjacent green samples, the sampling differences of the same color at a distance, and the gradient contribution coefficient of the same color, the horizontal and vertical gradients of the target position are calculated, and the horizontal and vertical weights are calculated based on the horizontal and vertical gradients. The weights are the reciprocal of the square of the corresponding gradient plus one. By comparing the horizontal gradient and the vertical gradient, when the difference between the two is greater than the gradient judgment threshold, the green candidate value in the direction with the smaller gradient is selected as the green output of the current position; when the difference is not greater than the gradient judgment threshold, the two sets of green candidate values ​​are weighted and fused using horizontal and vertical weights to obtain the green channel value restored for the current position. Write the recovered green channel value to the corresponding coordinates of the green sampling plane. After traversing all target positions, output the completed green channel.

5. The RAW image reconstruction method according to claim 4, characterized in that, The initial normalized gain color difference is calculated at the original red sampling position and the original blue sampling position based on the white balance gain parameter and the recovered green channel, specifically including the following steps: Obtain the white balance gain parameters of the red channel Green channel white balance gain parameters and blue channel white balance gain parameters Read the completed green channel value and the original red sample value of the RAW image. R Native blue sample value B ; Iterate through all pixel coordinates of the RAW image and filter out all native red sampling locations; For each native red sampling location, extract the original red sampling value at that location. R The green channel value that has been restored at the same coordinate position G According to the formula Calculate the initial red normalized gain color difference at this location, and assign the calculation result to the corresponding coordinates in the red normalized gain color difference plane; Set the coordinates of the red normalized gain chromatic difference plane corresponding to the non-native red sampling positions as invalid markers; traverse all pixel coordinates of the image and filter out all native blue sampling positions; For each native blue sampling location, extract the original blue sampling value at that location. B Retrieve the restored green channel value at the same coordinate position. G According to the formula Calculate the initial blue normalized gain color difference at this location, and assign the calculation result to the corresponding coordinates in the blue normalized gain color difference plane; Set the coordinates of the blue normalized gain chromatic difference plane corresponding to the non-native blue sampling positions as invalid markers; Complete the calculation of all native red and native blue sampling positions, and output the initialized red normalized gain color difference and blue normalized gain color difference.

6. The RAW image reconstruction method according to claim 5, characterized in that, Before reconstructing the RGB channels by combining white balance gain, the restored green channel, and the initial normalized gain color difference, the process also includes optimizing the initial normalized gain color difference. The specific optimization steps are as follows: Based on the spatial relationship between the green sampling position and the adjacent red and blue sampling positions in the Bayer array, the normalized gain color difference is propagated along the corresponding direction at the green sampling position, and the adaptive weights of the adjacent color differences are determined by using the local gradient of the green channel. The normalized gain color difference of blue is recovered at the red sampling position, and the normalized gain color difference of red is recovered at the blue sampling position; Candidate color differences are calculated for the two diagonal directions respectively, and weighted fusion is performed based on the diagonal gradient of the green channel to complete the cross color difference compensation at the different color sampling positions, and obtain the complete normalized gain color difference plane. The complete normalized gain color difference plane is repaired and the local median consistency constraint is optimized. The optimization is applied only to the color difference plane and does not directly smooth the green channel, resulting in the optimized normalized gain color difference.

7. The RAW image reconstruction method according to claim 6, characterized in that, The RGB channels are reconstructed by combining white balance gain parameters, the recovered green channel, and optimized normalized gain color difference, and the corrected RGB image is output. The specific steps include: Obtain white balance gain parameters , , Read the red normalized gain color difference after propagation, cross-compensation, and local consistency optimization. Planar and Blue Normalized Gain Color Difference Plane and complete greenway plan; Iterate through each pixel coordinate of the RGB image and extract the optimized red normalized gain color difference at the current coordinate. Blue normalized gain color difference And the green channel value at the same coordinate. G ; According to the formula Calculate the red channel value after white balance correction at the current coordinates. ; According to the formula Calculate the green channel value after white balance correction for the current coordinates. ; According to the formula Calculate the blue channel value after white balance correction at the current coordinates. ; Determine the upper and lower limits of pixel values ​​based on the target bit depth of the output image, and then adjust the red channel values ​​accordingly. Green channel value Blue channel value Perform a truncation operation to limit the pixel values ​​of each channel to within the target dynamic range; The truncated red, green, and blue pixel values ​​are sequentially assigned to the corresponding coordinate positions in the RGB image. After traversing all pixels, output an RGB image with white balance correction.

8. A RAW image reconstruction system, characterized in that, include: The white balance gain parameter acquisition module is used to acquire RAW images with Bayer array data and estimate the white balance gain parameters of the RAW images. The green channel recovery and normalized gain color difference initialization module is used to adaptively recover the missing green channel values ​​at the red and blue sampling positions of the RAW image, based on the horizontal and vertical gradients, adjacent green sampling values, and the low-frequency ratio constraint of the same color, using the green channel as the brightness structure reference. Based on the white balance gain parameters and the recovered green channel, the initial normalized gain color difference is calculated at the original red sampling position and the original blue sampling position, respectively. The horizontal and vertical gradients are the composite gradients at the red or blue sampling positions of the green channel value to be recovered, which are obtained by weighted summation of the directional differences between adjacent green samples and the directional differences between two pixel sampling points of the same color channel. The adjacent green sampling values ​​are the sensor's original green pixels that are directly adjacent to the red and blue sampling points in the top, bottom, left, and right directions. The same color low-frequency ratio constraint uses the same color original RAW pixel values ​​of the current red and blue sampling points to estimate the local brightness baseline to obtain the brightness ratio, which is used to correct the candidate green value. The image reconstruction module combines the white balance gain parameter, the restored green channel, and the initialized normalized gain color difference to reconstruct the RGB channels and output the corrected RGB image.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is loaded by a processor, it is able to execute the steps of the method according to any one of claims 1 to 7.

Images