Full-color imaging enhancement method and device, computing equipment and storage medium

Abstract

The invention provides a full-color imaging enhancement method and device, computing equipment and a storage medium, which are used for minimizing color reconstruction distortion after decoding under real-time constraints. The method comprises the following steps: performing neighborhood chrominance up-sampling analysis on a current pixel block in original image frame data according to a traversal sequence to obtain predicted chrominance data for reconstructing the current pixel block, and performing linear transformation of RGB space and reconstruction error analysis on the current pixel block according to the predicted chrominance data to obtain a distortion measure function of the current pixel block, and performing global optimal solution on the distortion measure function to obtain an optimal imaging adjustment parameter group of the current pixel block, and performing one-by-one optimization of the pixel blocks on the original image frame data according to the optimal imaging adjustment parameter group to generate full-color enhanced image data.

Description

Technical Field

[0001] This application relates to the field of image vision technology, and in particular to a full-color imaging enhancement method, apparatus, computing device, and storage medium. Background Technology

[0002] Devices such as dashcams, which have high requirements for the timeliness of image data, impose stringent constraints on full-color image enhancement: the processing must be completed within a strict frame-time budget (low latency, pipelined parallelization), and run under limited computing power and power consumption. At the same time, under bandwidth / storage constraints, it must ensure visual fidelity (especially the preservation of color and detail of key information such as license plates and road signs), robustness under motion and lighting changes, and work in conjunction with standard video encoders (QP, quantization, loop filtering, etc.) without introducing uncontrollable fluctuations in bit rate or time.

[0003] Currently, most commonly used full-color enhancement methods fall into two categories: one is traditional / empirical chroma subsampling and upsampling strategies (such as various implementations of 4:2:0—average, column / row sampling, MPEG-B filtering, BRIGHT / BRIGHT_MEAN, etc.), and the other is independent image enhancement preprocessing (noise suppression, color temperature correction, Retinex / adaptive histogram, guided / bilateral filtering, edge interpolation, etc.), as well as several improved upsampling methods based on interpolation or edge guidance.

[0004] The existing methods described above treat encoding / subsampling decisions, decoding reconstruction capabilities, and the degree of freedom for brightness adjustment compensation separately. They lack the design concept of taking the "final RGB reconstruction error after decoding" as the decision target on the encoding side and solving it jointly using a closed-loop method that can be calculated in real time. Therefore, in the actual encoding-decoding process, color reconstruction distortion is likely to occur or the bit rate may be forced to be increased in order to improve quality. Furthermore, it is difficult to achieve a balance between quality, bit rate, and real-time performance under embedded real-time constraints. Summary of the Invention

[0005] This application provides a full-color imaging enhancement method, apparatus, computing device, and storage medium for minimizing color reconstruction distortion after decoding under real-time constraints.

[0006] In a first aspect, this application provides a full-color image enhancement method, the method comprising: Step S1: Perform neighborhood chromaticity upsampling analysis on the current pixel block in the original image frame data according to the preset traversal order to obtain the predicted chromaticity data for reconstructing the current pixel block; Step S2: Based on the predicted chromaticity data, perform a linear transformation in RGB space and reconstruction error analysis on the current pixel block to obtain the distortion measurement function of the current pixel block; Step S3: Perform a global optimal solution on the distortion measurement function to obtain the optimal imaging adjustment parameter set for the current pixel block; Step S4: Based on the optimal imaging adjustment parameter group, optimize the original image frame data pixel by pixel to generate full-color enhanced image data.

[0007] In one possible implementation, the step of performing neighborhood chromaticity upsampling analysis on the current pixel block in the original image frame data according to a preset traversal order to obtain the predicted chromaticity data for reconstructing the current pixel block includes: The original image frame data is divided according to the preset pixel block size to obtain pixel block groups; According to the preset traversal order, the chromaticity values ​​of the adjacent pixel blocks of the current pixel block in the pixel block group are extracted to obtain the downsampled chromaticity reference value set corresponding to the current pixel block; Based on the relative coordinate distance between the current pixel block and the adjacent pixel blocks in the pixel block group and the downsampled chromaticity reference value set, bilinear interpolation is performed on the chromaticity values ​​in the current pixel block to obtain the predicted chromaticity data for reconstructing the current pixel block.

[0008] In one possible implementation, the step of extracting the chromaticity values ​​of adjacent pixel blocks from the current pixel block in the pixel block group according to a preset traversal order to obtain the downsampled chromaticity reference value set corresponding to the current pixel block includes: Based on a preset traversal order and a preset neighborhood threshold, the set of neighboring pixel blocks corresponding to the current pixel block in the pixel block group is determined; wherein, the set of neighboring pixel blocks includes a first subset of neighboring pixel blocks whose traversal order is before the current pixel block, and a second subset of neighboring pixel blocks whose traversal order is after the current pixel block; Obtain the neighborhood reference data and pixel coordinates of each pixel block in the first adjacent pixel block subset to form the first reference value subset corresponding to the current pixel block; The mean chromaticity value of each pixel block in the second adjacent pixel block subset is calculated to obtain the mean chromaticity value of each pixel block. Based on the pixel coordinates of each pixel block in the second adjacent pixel block subset and the pixel coordinates of each pixel block in the first adjacent pixel block subset, the mean chromaticity value and the neighborhood reference data of each pixel block in the first adjacent pixel block subset are summarized into the downsampled chromaticity reference value set corresponding to the current pixel block, wherein the neighborhood reference data is the optimized chromaticity value of each pixel block in the first adjacent pixel block subset.

[0009] In one possible implementation, the step of performing a linear transformation in RGB space and reconstruction error analysis on the current pixel block based on the predicted chromaticity data to obtain a distortion metric function for the current pixel block includes: The predicted chromaticity data is linearly transformed in RGB space to obtain the predicted RGB pixel data of the current pixel block; The deviation between the RGB pixel data of the current pixel block in the original image frame data and the predicted RGB pixel data in each color channel is calculated to obtain the distortion data of the current pixel block; Based on preset brightness variables and preset chromaticity downsampling variables, error fluctuation trend analysis is performed on the distortion data to construct a distortion measurement function for the current pixel block.

[0010] In one possible implementation, the step of performing a global optimal solution on the distortion metric function to obtain the optimal imaging adjustment parameter set for the current pixel block includes: Based on the predicted chromaticity data and the preset conversion standard offset, the RGB pixel data is linearly combined with the RGB channels to obtain the observation response vector of the current pixel block; According to the preset least squares criterion, the distortion metric function is converted into an overdetermined linear equation system; The unknown variables are identified and coefficients are extracted from the overdetermined linear equation system to obtain the design parameter set; The design parameter set is reorganized according to a preset variable combination method to generate an adjustment design matrix; The optimal imaging adjustment parameter set is obtained by performing pseudo-inverse solving on the adjustment design matrix based on the observed response vector.

[0011] In one possible implementation, the optimal imaging adjustment parameter set includes brightness adjustment parameter values ​​and chromaticity downsampling parameter values. The step of optimizing the original image frame data pixel by pixel based on the optimal imaging adjustment parameter set to generate full-color enhanced image data includes: Adjust the original brightness data of the current pixel block according to the brightness adjustment parameter value to obtain the optimized brightness component of the current pixel block; The original chromaticity components of the current pixel block are adjusted according to the chromaticity downsampling parameter value to obtain the optimized chromaticity value of the current pixel block; According to the preset storage method, the optimized chromaticity value is recorded as the neighborhood reference data corresponding to the current pixel block, and the pixel block whose traversal order is one position after the current pixel block in the traversal order is determined as the new current pixel block; Based on the traversal order and the neighborhood reference data, repeat steps S1 to S4 until all pixel blocks in the original image frame data complete chroma and brightness adjustment one by one to generate full-color enhanced image data.

[0012] Secondly, this application provides a full-color image enhancement device, the device comprising: The reconstruction prediction module is used to perform neighborhood chromaticity upsampling analysis on the current pixel block in the original image frame data according to a preset traversal order to obtain the predicted chromaticity data for reconstructing the current pixel block. The error analysis module is used to perform linear transformation in RGB space and reconstruction error analysis on the current pixel block based on the predicted chromaticity data, and obtain the distortion measurement function of the current pixel block. The optimization and solution module is used to perform a global optimal solution for the distortion measurement function to obtain the optimal imaging adjustment parameter set for the current pixel block; The collaborative optimization module is used to optimize the original image frame data pixel by pixel according to the optimal imaging adjustment parameter group to generate full-color enhanced image data.

[0013] In summary, this application includes at least the following beneficial technical effects: 1. By predicting and upsampling at the encoding side and jointly optimizing luminance and chrominance, the RGB reconstruction deviation caused by chrominance subsampling is greatly reduced, especially at edges and key targets (such as license plates) to restore more accurate colors and details.

[0014] 2. The pseudo-inverse of the overdetermined linear system is pre-computed offline / initialized, and only small-scale matrix multiplication is performed at runtime, which facilitates pipelined parallel implementation and is suitable for embedded high frame rate scenarios such as dashcams.

[0015] 3. By adopting the traversal order and recursively referencing the reference values ​​of the processed blocks, and by using conservative estimation for the unprocessed neighborhood, the intra-frame optimization results are consistent and resistant to quantization / QP fluctuations and error accumulation in motion scenes. Attached Figure Description

[0016] Figure 1 This is a schematic flowchart of a full-color imaging enhancement method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a full-color imaging enhancement device provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of the computing device provided in the embodiments of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. With the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0018] The terms "first," "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the description of embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to those processes, methods, products, or apparatuses.

[0019] like Figure 1 The diagram shown is a flowchart illustrating the full-color imaging enhancement method provided in this application embodiment. It should be understood that this application is applicable to imaging equipment systems with high timeliness requirements (e.g., body cameras, dashcams, etc.). The following uses a dashcam system as an example to explain the technical solution of this application in detail. The full-color imaging enhancement method provided in this application embodiment includes the following steps.

[0020] Step S1: Perform neighborhood chromaticity upsampling analysis on the current pixel block in the original image frame data according to the preset traversal order to obtain the predicted chromaticity data for reconstructing the current pixel block.

[0021] It should be understood that the raw image frame data comes directly from the output of the dashcam's built-in image signal processor. The image signal processor performs a series of preprocessing steps on the raw Bayer array data acquired by the sensor, such as de-mosaicing, white balance, color correction, and gamma correction, to generate uncompressed intermediate format data, where each pixel has independent and complete luminance and chrominance components—that is, frame-by-frame data in YUV-4:4:4 format. The raw image frame data, as the initial input to the technical solution of this application, contains precise color information that the optimization algorithm strives to retain as much as possible after final compression and reconstruction.

[0022] After obtaining the raw image frame data, it is first necessary to divide it into basic units that are easy to process independently. This application uses a preset fixed pixel block size, such as the widely used 2x2 pixel square, to perform regular raster division of the entire frame data, thereby obtaining an ordered group of pixel blocks. The division is based on the chroma subsampling unit commonly used in subsequent video coding standards, ensuring that the optimization operation is aligned with the encoder's data organization method. By decomposing the entire complex global image optimization problem into a large number of local optimization subproblems that can be solved independently or sequentially, the high real-time requirements of the dashcam are met. In order to perform high-quality chroma prediction on the current block to be processed in the pixel block group, the chroma information of its surrounding area must be obtained. This application uses a preset, determined traversal order (raster scanning order from left to right and from top to bottom) to identify all pixel blocks spatially adjacent to the current pixel block, forming its neighboring pixel block set. Since the processing is sequential, these adjacent blocks are naturally divided into two categories: one category consists of pixel blocks that have been traversed before the current block and have completed all processing steps of this enhancement method, referred to as the first subset of adjacent pixel blocks; the other category consists of pixel blocks that have been traversed after the current block and have not yet been processed, referred to as the second subset of adjacent pixel blocks. For the pixel blocks in the first subset of adjacent pixel blocks, their processing results already include the neighborhood reference data determined and stored by the complete optimization operation of the technical solution of this application—that is, the final representative optimized chroma value of the block. By querying the storage area, these known exact values ​​and their pixel coordinates in the image are directly obtained and aggregated into the first subset of reference values. For the pixel blocks in the second subset of adjacent pixel blocks, since they have not yet been processed, their optimized chroma values ​​are unknown. In order not to interrupt the calculation of the current block and to maintain the causality and feasibility of the algorithm, the system adopts a fast and low-complexity prediction strategy. A typical strategy adopted in the embodiments of this application is to calculate the arithmetic mean of the original chroma components of all pixels in the unprocessed block and use this mean as the estimated value of its downsampled chroma value. Similarly, by combining the pixel coordinates of these unprocessed blocks, these estimated values ​​are aggregated into a second subset of reference values. Merging the first and second subsets of reference values ​​constitutes the complete set of downsampled chromaticity reference values ​​upon which neighborhood analysis of the current pixel block depends. This classification process cleverly solves the common forward dependency problem in image processing, enabling the processing of each block to be based on a context that is both partially accurate and partially estimated, thus achieving information coupling and collaboration between blocks.

[0023] After obtaining a series of downsampled chromaticity reference points and their precise coordinates distributed around the current block, interpolation is needed to estimate the chromaticity value that each pixel position within the current block should have. This application employs a computationally efficient and smooth bilinear interpolation algorithm, the core of which lies in a distance weighting mapping mechanism based on relative coordinate distance. Specifically, the horizontal and vertical weighting coefficients need to be determined for the interpolation calculation. Taking the estimation of the chromaticity value of a specific pixel P within the current block as an example, the four nearest neighbor reference points surrounding point P are first found, which typically form a rectangular grid. Let the normalized relative coordinate distance of point P within this rectangular grid be (x, y), where x represents the relative distance from point P to the left reference point in the horizontal direction, and y represents the relative distance from point P to the upper reference point in the vertical direction, with both x and y values ​​between 0 and 1. The set of horizontal weight coefficients used for bilinear interpolation can be represented as (1-x) and (x), corresponding to the contribution weights of the left and right reference points in the horizontal direction, respectively; the set of vertical weight coefficients can be represented as (1-y) and (y), corresponding to the contribution weights of the upper and lower reference points in the vertical direction, respectively. These coefficients are entirely determined by geometric position and can be pre-calculated or obtained through simple subtraction, with minimal computational overhead.

[0024] Finally, the distance weighting mapping mechanism is combined with the downsampled chromaticity reference value set to perform bilinear interpolation calculation. Specifically, the predicted chromaticity value Cp of point P is obtained through two linear interpolations: First, using the vertical weight coefficient sets (1-y) and (y), the chromaticity values ​​of the two upper reference points and the two lower reference points are weighted and averaged respectively to obtain two intermediate horizontal interpolation points; then, using the horizontal weight coefficient sets (1-x) and (x), the two intermediate horizontal interpolation points are weighted and averaged to obtain the final predicted chromaticity value of point P. Mathematically, this can be expressed as: Cp = (1-x) × (1-y)Q(upper left) + x × (1-y)Q(upper right) + (1-x) × yQ(lower left) + x × yQ(lower right), where Q(*) represents the chromaticity value of the reference point corresponding to the relative position of point P. For the four pixels within the current 2x2 pixel block, the bilinear interpolation calculation process described above is repeated based on their respective relative coordinate distances to obtain all the predicted chroma data required to reconstruct the current pixel block. This operation simulates the precise process of upsampling and reconstruction performed by the video decoder after receiving chroma downsampling data. By performing this simulation in advance on the encoding side, the optimization algorithm can accurately predict the reconstruction results caused by different downsampling decisions.

[0025] Step S1 of this application transforms the global image enhancement problem into a series of local problems that can be solved quickly through block division, sequential traversal, context extraction and efficient interpolation. While ensuring the effectiveness of the algorithm, it strictly meets the stringent time constraints of dashcams on processing efficiency.

[0026] Step S2: Based on the predicted chromaticity data, perform a linear transformation in RGB space and reconstruction error analysis on the current pixel block to obtain the distortion measurement function of the current pixel block.

[0027] It should be understood that the predicted chromaticity data of the current pixel block obtained in step S1 exists in the form of chromaticity components in the YUV color space. In order to evaluate the quality of the final reconstructed image under different processing parameters and to establish an optimizable mathematical model, the analysis must be anchored to the color space directly corresponding to human visual perception and image display terminals—the RGB color space.

[0028] First, the system obtains the original, uncompressed, and unmodified true color value corresponding to the current pixel block. Based on the known pixel coordinates of the current pixel block within the image frame, the system reads its original RGB pixel data from the original image frame data. In this application, for RGB space comparison, the luminance and chrominance components of the original image frame data need to be pre-converted to their corresponding RGB values ​​using a standard, reversible color space conversion formula, or the ISP pipeline itself provides RGB format output. RGB pixel data represents the absolute color standard that should be presented under ideal, lossless conditions; for example, the accurate chromaticity of red in a traffic light, or the depth of blue on a license plate under specific lighting. Obtaining the original RGB pixel data provides an unshakeable reference benchmark for error calculation, ensuring that the optimization process always prioritizes restoring visual realism.

[0029] Next, it is necessary to calculate the RGB color values ​​that the current pixel block may reconstruct under a certain parameter combination. It should be understood that the predicted chromaticity data obtained in step S1 above only contains chromaticity information—that is, a linear function of the luminance variable and a linear function of the chromaticity downsampling variable. A complete pixel color requires both luminance and chromaticity to define it. Therefore, two key variables to be optimized are introduced here: one is the luminance variable (denoted as Y') representing a slight adjustment to the original luminance value of the current block, and the other is the unique chromaticity downsampling variable (denoted as Us, Vs) selected for this block. Taking four pixels in a 2×2 pixel block of this application embodiment as an example, these four luminance variables (denoted as Y'1, Y'2, Y'3, Y'4) and the predicted chromaticity values ​​calculated through the chromaticity variables are substituted into the standard linear inverse transformation formula from YUV to RGB. This transformation formula is a set of linear equations with fixed coefficients, and its mathematical expression is as follows:

[0030] in, , This represents the U chromaticity component (corresponding to the Cb component, i.e., the blue component, in the YCbCr color space) and the V chromaticity component (corresponding to the Cr component, i.e., the red component, in the YCbCr color space) in the predicted chromaticity data calculated from the chromaticity downsampling variables and known neighborhood offsets. a, b, c, d, e, f, g, h, i, and j are constants defined by color space standards (such as ITU-R BT.601 or BT.709). After performing this linear transformation, the predicted RGB pixel data of the current pixel block under a specific parameter combination is obtained. The above linear transformation operation maps the abstract parameter decisions located in the YUV space to the concrete, directly perceptible RGB color space.

[0031] After obtaining the original RGB pixel data and the predicted RGB pixel data, a precise comparison can be performed pixel-by-pixel and color channel-by-color comparison. The numerical deviation between the two in the red, green, and blue channels is calculated by simply subtracting the original value from the predicted value. For a 2x2 block containing four pixels, a total of twelve deviation values ​​will be generated. These deviation values ​​constitute the distortion data of the current pixel block, visually reflecting the specific magnitude and direction of the reconstructed color's deviation from the original color in each fundamental color component, given the parameters. For example, a positive red channel deviation means the reconstructed red is redder than the original, and a negative blue channel deviation means the reconstructed blue is paler than the original.

[0032] While directly examining these deviation values ​​allows us to perceive the error, it doesn't provide a single, comprehensive criterion for evaluating quality for automated optimization. Therefore, it's necessary to mathematically model these discrete distortion data—constructing a distortion metric function. This distortion metric function captures the overall trend and pattern of distortion as a function of luminance and chrominance downsampling variables. Since the transformation from YUV to RGB is linear, and the predicted chrominance is a linear function of the chrominance downsampling variables, the predicted RGB value for each color channel of each pixel can ultimately be expressed as a linear combination of six unknowns (i.e., four luminance variables Y'1, Y'2, Y'3, Y'4, and two chrominance downsampling variables Us, Vs), plus a fixed offset calculated from known constants. This means that each previously calculated deviation value is itself a linear function of these six variables. This application's embodiment employs a comprehensive error metric with excellent mathematical properties—the sum of squared errors. Specifically, all twelve deviation values ​​of the current block (four pixels multiplied by three channels) are squared, and then these twelve squared values ​​are summed. This sum, denoted as D, allows the constructed distortion metric function to be expressed as: D(Y'1, Y'2, Y'3, Y'4, Us, Vs). A smaller value for the distortion metric function indicates a smaller overall color deviation between the predicted reconstructed image and the original image under that set of parameters, signifying higher quality. This function construction elevates the image from a specific color difference to an abstract mathematical model, transforming an image quality optimization problem into a well-defined problem of finding the minimum value of a multivariate quadratic function with six variables. This paves the way for the application of efficient and deterministic mathematical optimization algorithms (such as the least squares method), ensuring that the entire enhancement method can find the theoretically optimal or near-optimal processing parameters within the strict time constraints required by dashcams.

[0033] Step S3: Perform a global optimal solution on the distortion metric function to obtain the optimal imaging adjustment parameter set for the current pixel block.

[0034] To obtain a set of specific parameters from the above function that minimizes the distortion of the predicted reconstruction, this application first requires preprocessing the known image data to construct an observation response vector. The observation response vector is a twelve-dimensional mathematical column vector, where each component is calculated from the original RGB pixel data of the current pixel block, the constant offset contained in the predicted chromaticity data obtained in step S1 above, and the fixed offset defined in the color space conversion standard. Specifically, for each pixel in a 2×2 pixel block, a specific linear combination calculation is performed for its red, green, and blue channels. Taking the red channel as an example, the formula for calculating the observed value of this channel is: the original red value minus a product of the V channel conversion coefficient and the predicted chromaticity constant offset ΔV, plus a composite constant derived from the conversion coefficient and the standard offset. The calculations for the green and blue channels follow similar rules, but the combinations of coefficients and offsets involved are different. By separating the "desired original color target of the final image" from the "fixed influence determined by the known neighborhood chromaticity background" through the above linear combination operation, all known quantities are merged to one side of the equation, thereby constructing a pure mathematical vector (i.e., the observation response vector) representing the ideal observation target. The observation response vector serves as the absolute benchmark in the function solution process, and its calculation depends entirely on the determined image information and standard parameters, without containing any unknown variables to be solved.

[0035] After obtaining the observed response vector, a clear mathematical constraint needs to be established for the solution process. Based on the pre-defined least squares criterion, the distortion metric function defined in step S2, presented as a sum of squares, is equivalently transformed into an overdetermined linear equation system. This transformation is based on an intuitive mathematical idea: to minimize the distortion metric function, ideally, the predicted RGB values ​​should be forced to be exactly equal to the original RGB values. For a block containing four pixels, each pixel has three RGB channels, thus twelve equations can be listed. The left-hand side of the equations represents the original RGB values ​​(adjusted with the same constant term as the observed response vector), while the right-hand side is the result obtained by substituting the predicted chromaticity data, luminance variable, and chromaticity downsampling variable into the inverse color space transformation formula. Since the predicted chromaticity data itself can be expressed as a linear function of the chromaticity downsampling variable, the right-hand side of the equation system can ultimately be fully expanded into a linear expression with six unknown variables. This forms a linear system containing twelve equations and only six unknowns, the so-called overdetermined linear equation system. By transforming the system of equations, a quadratic function optimization problem that requires minimization through differentiation or iteration is transformed into a more structured problem form that can be solved directly using linear algebra tools.

[0036] Next, the unknown variables and their coefficients are systematically extracted from the overdetermined linear equation system. Each of the twelve equations is analyzed individually, identifying the numerical coefficients multiplied by the six unknown variables in each equation. For example, in the equation corresponding to the red channel of the first pixel, the coefficients multiplied by variable Y'1 and variable Us are extracted. These coefficients are entirely determined by the fixed elements of the color space transformation matrix and the bilinear interpolation weights determined in step S1, and are independent of the image content. Arranging the coefficients corresponding to the same unknown variable in all equations according to the equation order yields the coefficient sequence for each variable. This set of six coefficient sequences constitutes the design parameter set, which fully describes how each unknown variable linearly affects the twelve observation equations.

[0037] After obtaining the design parameter set, it is reorganized into a structured mathematical entity—the adjustment design matrix—according to a preset variable combination method. This matrix is ​​a constant matrix with twelve rows and six columns. Each row of the matrix corresponds to an equation in the overdetermined system of equations, and each column corresponds to a specific unknown variable. The element in the i-th row and j-th column is the coefficient of the j-th unknown variable in the i-th equation. Therefore, the first column of the matrix is ​​filled entirely with the coefficients affecting the first luminance variable Y'1, the second column corresponds to Y'2, and so on. The last two columns correspond to the chromaticity variables Us and Vs, respectively, because it concisely and accurately expresses the overdetermined system of equations in the standard matrix multiplication form: the adjustment design matrix multiplied by the column vector consisting of the six unknown variables equals the observed response vector. The fixed nature of the adjustment design matrix means that as long as the color conversion standard and interpolation method are selected, this matrix can be pre-calculated and stored in the memory of the dashcam system, without needing to be recalculated for each image block processed, which greatly ensures the high timeliness of the algorithm.

[0038] Finally, the pseudo-inverse technique from linear algebra is used to process this matrix equation. Since the adjustment design matrix is ​​a rectangular full-rank matrix, its standard inverse matrix does not exist. The pseudo-inverse provides a mathematical tool for finding the optimal approximate solution in the least squares sense. Specifically, the Moore-Penrose pseudo-inverse matrix of the adjustment design matrix is ​​calculated. In practical engineering implementation, this pseudo-inverse matrix can also be pre-calculated offline based on a fixed adjustment design matrix, resulting in a six-row, twelve-column matrix with constant coefficients. In real-time processing, the solution process simplifies to a single efficient matrix-vector multiplication: multiplying the pre-calculated pseudo-inverse matrix by the observation response vector specific to the current pixel block, the result is directly a six-dimensional column vector. The six elements of this column vector, in a predetermined order, are the four luminance adjustment parameter values ​​and two chrominance downsampling parameter values ​​that globally minimize the predicted distortion, i.e., the optimal imaging adjustment parameter set. The above pseudo-inverse solution operation mathematically guarantees that the obtained parameters are the unique globally optimal solution that minimizes the sum of squared differences between the predicted RGB values ​​and the original RGB values.

[0039] Step S4: Based on the optimal imaging adjustment parameter group, optimize the original image frame data pixel by pixel to generate full-color enhanced image data.

[0040] It should be understood that the optimal imaging adjustment parameter set contains two sets of core data: one set is the brightness adjustment parameter value calculated for each of the four pixel positions in the current pixel block, and the other set is a pair of chromaticity downsampling parameter values ​​uniformly determined for the entire current pixel block.

[0041] To transform the aforementioned mathematical optimization results into actual image data and drive the entire processing flow systematically until the final execution task is completed, the first step is the actual modification of the luminance component. Based on the unique identifier of the current pixel block, its original luminance data matrix is ​​extracted from the original image frame data. Subsequently, the four specific luminance adjustment parameter values ​​output in step S3 are used to replace the luminance values ​​at the corresponding pixel positions in the original matrix. For example, for a pixel block containing a region where bright light and shadow meet, the optimization algorithm may calculate parameters that slightly reduce the luminance of highlight pixels while slightly brightening the luminance of shadow pixels to better coordinate with the upcoming chroma downsampling and preserve the color details of the edges. After the replacement adjustment, the original luminance data is updated with the optimized luminance component. The above adjustment operation directly changes the brightness of the pixels, but its modification amount is subject to strict distortion minimization constraints, aiming to achieve the most effective compensation for the loss of chroma information at the cost of minimal luminance change.

[0042] Next, the chroma components are adjusted. Unlike the luminance components, which are adjusted independently for each pixel, chroma adjustment acts on the overall representative value of the entire pixel block. By reading the two chroma downsampling parameter values ​​output in step S3, these directly constitute the unique pair of chroma values ​​used by the pixel block in subsequent video encoding. The original chroma components (U and V) of the four pixels corresponding to this pixel block in the original image frame data are discarded. These two parameter values ​​are the optimized chroma values ​​for the block. For example, for a small area containing alternating red and green textures, traditional downsampling may lead to color blurring due to simple averaging, while the optimized chroma values ​​here are obtained by solving a global distortion minimization model, which theoretically can reduce such blurring to the greatest extent. Through the above chroma subsampling decision, high-resolution chroma information is compressed into a low-resolution representation, but this decision is the optimal choice obtained after complex optimization.

[0043] Subsequently, the processing results must be properly saved to affect the processing of subsequent pixel blocks. This application associates the newly obtained optimized chroma values ​​(i.e., the downsampling parameters) with the physical coordinates of the pixel block in the image frame, packaging and recording them as a data unit called the neighborhood reference data corresponding to that pixel block. This data unit is typically sent to a specially managed buffer or lookup table. The purpose of recording this data is crucial: when the processing stream proceeds sequentially to the next pixel block, step S1 requires obtaining the chroma information of the processed adjacent blocks. At this time, the optimized, determined chroma values ​​of these processed blocks can be quickly retrieved from this buffer based on their coordinate indices, serving as reliable known references.

[0044] After updating and storing the data for the current block, the process controller, according to a preset traversal order (e.g., raster scan order), marks the next pixel block to be processed immediately following the current pixel block as the new current pixel block. The system state is updated accordingly, including the data pointer pointing to the new block and the contents of the neighborhood reference cache that may be updated. Finally, the entire processing loop continues. The system focuses on the new current pixel block and restarts execution from step S1. Step S1 uses the neighborhood reference data of all processed blocks in the cache (now including the block that has just been processed) to build more accurate and context-aware predicted chroma data for the new current block. Then, the modeling in step S2, the solving in step S3, and the application and storage in step S4 are repeated. This loop is like a scrolling window, scanning and optimizing the entire frame image block by block. The processing of each subsequent block is based on the optimal decisions of all previously processed blocks, allowing the optimization effect to accumulate and propagate along the traversal direction. The loop terminates when the last pixel block in the original image frame data has also completed this series of chroma and brightness adjustments. At this point, the system integrates all outputs: a complete luminance plane composed of all optimized luminance components (i.e., the modified Y value of each pixel), and a subsampled chrominance plane composed of all optimized chrominance values ​​(i.e., the Us, Vs pairs corresponding to each 2x2 block, arranged in a 4:2:0 format). These two parts together form the final full-color enhanced image data. While maintaining high compatibility and being directly fed into a standard video encoder, this image data has undergone globally coordinated optimization of its inherent luminance and chrominance information. Theoretically, after compression and decompression, it can present color fidelity and visual detail closer to the original scene than traditional processing methods, thus meeting the core requirement of dashcams to improve image quality under stringent real-time constraints.

[0045] Please see Figure 2 , Figure 2 This is a schematic diagram of a full-color imaging enhancement device provided in an embodiment of this application. Figure 2As shown, the full-color imaging enhancement device 2 includes: a reconstruction prediction module 21, an error analysis module 22, an optimization solution module 23, and a collaborative optimization module 24.

[0046] The reconstruction prediction module 21 is used to perform neighborhood chromaticity upsampling analysis on the current pixel block in the original image frame data according to a preset traversal order to obtain the predicted chromaticity data for reconstructing the current pixel block. Error analysis module 22 is used to perform linear transformation in RGB space and reconstruction error analysis on the current pixel block based on the predicted chromaticity data, and obtain the distortion measurement function of the current pixel block; The optimization and solution module 23 is used to perform a global optimal solution for the distortion measurement function to obtain the optimal imaging adjustment parameter set for the current pixel block; The collaborative optimization module 24 is used to optimize the original image frame data pixel by pixel according to the optimal imaging adjustment parameter group to generate full-color enhanced image data.

[0047] In one possible implementation, the reconstruction prediction module 21 is specifically used for: The original image frame data is divided according to the preset pixel block size to obtain pixel block groups; According to the preset traversal order, the chromaticity values ​​of the adjacent pixel blocks of the current pixel block in the pixel block group are extracted to obtain the downsampled chromaticity reference value set corresponding to the current pixel block; Based on the relative coordinate distance between the current pixel block and the adjacent pixel blocks in the pixel block group and the downsampled chromaticity reference value set, bilinear interpolation is performed on the chromaticity values ​​in the current pixel block to obtain the predicted chromaticity data for reconstructing the current pixel block.

[0048] In one possible implementation, the error analysis module 22 is specifically used for: The predicted chromaticity data is linearly transformed in RGB space to obtain the predicted RGB pixel data of the current pixel block; The deviation between the RGB pixel data of the current pixel block in the original image frame data and the predicted RGB pixel data in each color channel is calculated to obtain the distortion data of the current pixel block; Based on preset brightness variables and preset chromaticity downsampling variables, error fluctuation trend analysis is performed on the distortion data to construct a distortion measurement function for the current pixel block.

[0049] In one possible implementation, the optimization solution module 23 is specifically used for: Based on the predicted chromaticity data and the preset conversion standard offset, the RGB pixel data is linearly combined with the RGB channels to obtain the observation response vector of the current pixel block; According to the preset least squares criterion, the distortion metric function is converted into an overdetermined linear equation system; The unknown variables are identified and coefficients are extracted from the overdetermined linear equation system to obtain the design parameter set; The design parameter set is reorganized according to a preset variable combination method to generate an adjustment design matrix; The optimal imaging adjustment parameter set is obtained by performing pseudo-inverse solving on the adjustment design matrix based on the observed response vector.

[0050] In one possible implementation, the collaborative optimization module 24 is specifically used for: Adjust the original brightness data of the current pixel block according to the brightness adjustment parameter value to obtain the optimized brightness component of the current pixel block; The original chromaticity components of the current pixel block are adjusted according to the chromaticity downsampling parameter value to obtain the optimized chromaticity value of the current pixel block; According to the preset storage method, the optimized chromaticity value is recorded as the neighborhood reference data corresponding to the current pixel block, and the pixel block whose traversal order is one position after the current pixel block in the traversal order is determined as the new current pixel block; Based on the traversal order and the neighborhood reference data, repeat steps S1 to S4 until all pixel blocks in the original image frame data complete chroma and brightness adjustment one by one to generate full-color enhanced image data.

[0051] In one possible implementation, the reconstruction prediction module 21 is further configured to: Based on a preset traversal order and a preset neighborhood threshold, the set of neighboring pixel blocks corresponding to the current pixel block in the pixel block group is determined; wherein, the set of neighboring pixel blocks includes a first subset of neighboring pixel blocks whose traversal order is before the current pixel block, and a second subset of neighboring pixel blocks whose traversal order is after the current pixel block; Obtain the neighborhood reference data and pixel coordinates of each pixel block in the first adjacent pixel block subset to form the first reference value subset corresponding to the current pixel block; The mean chromaticity value of each pixel block in the second adjacent pixel block subset is calculated to obtain the mean chromaticity value of each pixel block. Based on the pixel coordinates of each pixel block in the second adjacent pixel block subset and the pixel coordinates of each pixel block in the first adjacent pixel block subset, the mean chromaticity value and the neighborhood reference data of each pixel block in the first adjacent pixel block subset are summarized into the downsampled chromaticity reference value set corresponding to the current pixel block, wherein the neighborhood reference data is the optimized chromaticity value of each pixel block in the first adjacent pixel block subset.

[0052] The reconstruction prediction module 21, error analysis module 22, optimization solution module 23, and collaborative optimization module 24 can all be implemented in software or hardware. For example, the implementation of the reconstruction prediction module 21 will be described below. Similarly, the implementation of the error analysis module 22, optimization solution module 23, and collaborative optimization module 24 can refer to the implementation of the reconstruction prediction module 21.

[0053] As an example of a software functional unit, the reconstruction prediction module 21 may include code running on a compute instance. The compute instance may include at least one of a physical host (computing device), a virtual machine, or a container. Further, the aforementioned compute instance may be one or more. For example, the reconstruction prediction module 21 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed within the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed within the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0054] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0055] As an example of a hardware functional unit, the reconstruction prediction module 21 may include at least one computing device, such as a server. Alternatively, the reconstruction prediction module 21 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0056] The reconstruction prediction module 21 includes multiple computing devices that can be distributed within the same region or in different regions. Similarly, the multiple computing devices in the reconstruction prediction module 21 can be distributed within the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices in the reconstruction prediction module 21 can be distributed within the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0057] See Figure 3 As shown, Figure 3 This is a schematic diagram of a computing device provided in this application. The computing device 100 includes: a processor 104, a communication interface 108, a bus 102, and a memory 106. The processor 104, the communication interface 108, and the memory 106 communicate via the bus 102. In practical applications, communication can also be achieved through other means such as wireless transmission; however, this is not limited here.

[0058] The computing device 100 may be a server or a terminal device. It should be understood that this application does not limit the number of processors and memory in the computing device 100.

[0059] The processor 104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0060] The communication interface 108 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 100 and other devices or communication networks.

[0061] Bus 102 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 3 The bus 102 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 102 may include a path for transmitting information between various components of the computing device 100 (e.g., memory 106, processor 104, communication interface 108).

[0062] Memory 106 may include volatile memory, such as random access memory (RAM). Memory 106 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0063] The memory 106 stores executable program code, and the processor 104 executes the executable program code to implement the functions of the aforementioned reconstruction prediction module 21, error analysis module 22, optimization solution module 23 and co-optimization module 24 respectively, thereby realizing the full-color imaging enhancement method. That is, the memory 106 stores instructions for executing the full-color imaging enhancement method.

[0064] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to perform a full-color image enhancement method, or instruct the computing device to perform a full-color image enhancement method.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A full-color imaging enhancement method, characterized in that, The method includes: Step S1: Perform neighborhood chromaticity upsampling analysis on the current pixel block in the original image frame data according to the preset traversal order to obtain the predicted chromaticity data for reconstructing the current pixel block; Step S2: Based on the predicted chromaticity data, perform a linear transformation in RGB space and reconstruction error analysis on the current pixel block to obtain the distortion measurement function of the current pixel block; Step S3: Perform a global optimal solution on the distortion measurement function to obtain the optimal imaging adjustment parameter set for the current pixel block; Step S4: Based on the optimal imaging adjustment parameter group, optimize the original image frame data pixel by pixel to generate full-color enhanced image data.

2. The full-color imaging enhancement method according to claim 1, characterized in that, The step of performing neighborhood chromaticity upsampling analysis on the current pixel block in the original image frame data according to a preset traversal order to obtain the predicted chromaticity data for reconstructing the current pixel block includes: The original image frame data is divided according to the preset pixel block size to obtain pixel block groups; According to the preset traversal order, the chromaticity values ​​of the adjacent pixel blocks of the current pixel block in the pixel block group are extracted to obtain the downsampled chromaticity reference value set corresponding to the current pixel block; Based on the relative coordinate distance between the current pixel block and the adjacent pixel blocks in the pixel block group and the downsampled chromaticity reference value set, bilinear interpolation is performed on the chromaticity values ​​in the current pixel block to obtain the predicted chromaticity data for reconstructing the current pixel block.

3. The full-color imaging enhancement method according to claim 2, characterized in that, The step of extracting the chromaticity values ​​of adjacent pixel blocks from the current pixel block in the pixel block group according to a preset traversal order to obtain the downsampled chromaticity reference value set corresponding to the current pixel block includes: Based on a preset traversal order and a preset neighborhood threshold, the set of neighboring pixel blocks corresponding to the current pixel block in the pixel block group is determined; wherein, the set of neighboring pixel blocks includes a first subset of neighboring pixel blocks whose traversal order is before the current pixel block, and a second subset of neighboring pixel blocks whose traversal order is after the current pixel block; Obtain the neighborhood reference data and pixel coordinates of each pixel block in the first adjacent pixel block subset to form the first reference value subset corresponding to the current pixel block; The mean chromaticity value of each pixel block in the second adjacent pixel block subset is calculated to obtain the mean chromaticity value of each pixel block. Based on the pixel coordinates of each pixel block in the second adjacent pixel block subset and the pixel coordinates of each pixel block in the first adjacent pixel block subset, the mean chromaticity value and the neighborhood reference data of each pixel block in the first adjacent pixel block subset are summarized into the downsampled chromaticity reference value set corresponding to the current pixel block, wherein the neighborhood reference data is the optimized chromaticity value of each pixel block in the first adjacent pixel block subset.

4. The full-color imaging enhancement method according to claim 1, characterized in that, The step of performing a linear transformation in RGB space and reconstruction error analysis on the current pixel block based on the predicted chromaticity data to obtain the distortion metric function of the current pixel block includes: The predicted chromaticity data is linearly transformed in RGB space to obtain the predicted RGB pixel data of the current pixel block; The deviation between the RGB pixel data of the current pixel block in the original image frame data and the predicted RGB pixel data in each color channel is calculated to obtain the distortion data of the current pixel block; Based on preset brightness variables and preset chromaticity downsampling variables, error fluctuation trend analysis is performed on the distortion data to construct a distortion measurement function for the current pixel block.

5. The full-color imaging enhancement method according to claim 4, characterized in that, The step of performing a global optimal solution on the distortion metric function to obtain the optimal imaging adjustment parameter set for the current pixel block includes: Based on the predicted chromaticity data and the preset conversion standard offset, the RGB pixel data is linearly combined with the RGB channels to obtain the observation response vector of the current pixel block; According to the preset least squares criterion, the distortion metric function is converted into an overdetermined linear equation system; The unknown variables are identified and coefficients are extracted from the overdetermined linear equation system to obtain the design parameter set; The design parameter set is reorganized according to a preset variable combination method to generate an adjustment design matrix; The optimal imaging adjustment parameter set is obtained by performing pseudo-inverse solving on the adjustment design matrix based on the observed response vector.

6. The full-color imaging enhancement method according to claim 1, characterized in that, The optimal imaging adjustment parameter set includes brightness adjustment parameter values ​​and chromaticity downsampling parameter values. The step of optimizing the original image frame data pixel by pixel based on the optimal imaging adjustment parameter set to generate full-color enhanced image data includes: Adjust the original brightness data of the current pixel block according to the brightness adjustment parameter value to obtain the optimized brightness component of the current pixel block; The original chromaticity components of the current pixel block are adjusted according to the chromaticity downsampling parameter value to obtain the optimized chromaticity value of the current pixel block; According to the preset storage method, the optimized chromaticity value is recorded as the neighborhood reference data corresponding to the current pixel block, and the pixel block whose traversal order is one position after the current pixel block in the traversal order is determined as the new current pixel block; Based on the traversal order and the neighborhood reference data, repeat steps S1 to S4 until all pixel blocks in the original image frame data complete chroma and brightness adjustment one by one to generate full-color enhanced image data.

7. A full-color image enhancement device, applied to the full-color image enhancement method according to claim 1, characterized in that, The device includes: The reconstruction prediction module is used to perform neighborhood chromaticity upsampling analysis on the current pixel block in the original image frame data according to a preset traversal order to obtain the predicted chromaticity data for reconstructing the current pixel block. The error analysis module is used to perform linear transformation in RGB space and reconstruction error analysis on the current pixel block based on the predicted chromaticity data, and obtain the distortion measurement function of the current pixel block. The optimization and solution module is used to perform a global optimal solution for the distortion measurement function to obtain the optimal imaging adjustment parameter set for the current pixel block; The collaborative optimization module is used to optimize the original image frame data pixel by pixel according to the optimal imaging adjustment parameter group to generate full-color enhanced image data.

8. A computing device, characterized in that, The computing device includes: At least one processor; and, A memory and a communication interface that are communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor implements the full-color imaging enhancement method according to any one of claims 1 to 6 by executing the instructions stored in the memory.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the full-color imaging enhancement method according to any one of claims 1 to 6.

Images