Image enhancement method, electronic device, storage medium, and program product
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SPREADTRUM COMMUNICATION (SHANGHAI) CO LTD
- Filing Date
- 2026-03-04
- Publication Date
- 2026-06-09
AI Technical Summary
During image processing, the downsampling and upsampling of chroma information during the YUV444 to YUV420 conversion can easily cause artifacts at the image edges, affecting image quality.
By acquiring pixel difference data of the first chromaticity plane, the second chromaticity plane, and the luminance plane of the image to be processed, the filtering weights are determined and the luminance plane is subjected to weighted filtering. The pixel difference features of the chromaticity plane are then fused to match the adjustment of the luminance plane.
It effectively reduces distortion phenomena such as false edges and false dots at the boundaries of high-saturation and high-brightness areas in the image, thereby improving image quality.
Smart Images
Figure CN122175802A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to an image enhancement method, electronic device, storage medium, and program product. Background Technology
[0002] In image processing, the YUV color space (luminance Y, chromaticity UV) is widely used due to its efficient encoding characteristics for human visual perception.
[0003] In practical applications, it is usually necessary to downsample the YUV444 format (both luminance Y and chrominance UV are fully sampled) to YUV420 (each chrominance UV is reduced to 1 / 4 sampling rate) to reduce bandwidth and storage requirements. However, when displaying the final product, the YUV420 format needs to be upsampled back to the YUV444 format.
[0004] However, the downsampling and upsampling of chromaticity information (UV) during this process can easily lead to artifacts at the image edges. Therefore, how to suppress the image quality degradation caused by the loss of UV information during the YUV444 to YUV420 conversion has become an urgent problem to be solved in current image processing technology. Summary of the Invention
[0005] This application provides an image enhancement method, electronic device, storage medium, and program product for improving image quality when performing YUV444 to YUV420 conversion on an image.
[0006] In a first aspect, embodiments of this application provide an image enhancement method, including:
[0007] Pixel difference data of the first chromaticity plane, the second chromaticity plane, and the luminance plane of the image to be processed are obtained. The pixel difference data is used to characterize the difference between the pixel value of each pixel in the sliding window and the pixel value of the center pixel. The first chromaticity plane and the second chromaticity plane are obtained by performing resolution transformation processing on the first chromaticity initial plane and the second chromaticity initial plane respectively.
[0008] Based on the pixel difference data, the filtering weights are determined. The filtering weights are used to characterize the contribution of each pixel value in the current sliding window to the pixel value of the center pixel.
[0009] The target brightness plane is obtained by performing weighted filtering on the brightness plane according to the filtering weights.
[0010] The target image is determined based on the target luminance plane, the first initial chromaticity plane, and the second initial chromaticity plane.
[0011] In one possible implementation, the pixel difference data includes first pixel difference data, second pixel difference data, and third pixel difference data. Acquiring pixel difference data for the first chromaticity plane, the second chromaticity plane, and the luminance plane of the image to be processed includes:
[0012] On the first chromaticity plane, the second chromaticity plane, and the luminance plane, respectively, a sliding window of a preset size is determined with each pixel as the center.
[0013] For each sliding window in the first chromaticity plane, calculate the difference between the pixel value of each pixel in the current sliding window and the pixel value of the center pixel to obtain the first pixel difference data;
[0014] For each sliding window of the second chromaticity plane, calculate the difference between the pixel value of each pixel in the current sliding window and the pixel value of the center pixel to obtain the second pixel difference data;
[0015] For each sliding window of the brightness plane, calculate the difference between the pixel value of each pixel in the current sliding window and the pixel value of the center pixel to obtain the third pixel difference data.
[0016] In one possible implementation, determining the filtering weights based on pixel difference data includes:
[0017] Based on the first pixel difference data and the Gaussian function, the first weight factor corresponding to each pixel in the sliding window is determined. The first weight factor is used to characterize the similarity between the current pixel and the center pixel in the first chromaticity.
[0018] Based on the second pixel difference data and the Gaussian function, the second weighting factor corresponding to each pixel in the sliding window is determined. The second weighting factor is used to characterize the similarity between the current pixel and the center pixel in the second chromaticity.
[0019] Based on the third pixel difference data and the Gaussian function, the third weight factor corresponding to each pixel in the sliding window is determined. The third weight factor is used to characterize the similarity in brightness between the current pixel and the center pixel.
[0020] The first weight factor, the second weight factor, and the third weight factor are combined to obtain the filter weight.
[0021] In one possible implementation, the brightness plane is weighted and filtered according to the filtering weights to obtain the target brightness plane, including:
[0022] For each sliding window, the filter weights in the sliding window are summed to obtain the total filter weights;
[0023] For each sliding window, in the brightness plane, calculate the product between the pixel value of each pixel and its corresponding filter weight, and sum the products to obtain the cumulative pixel sum;
[0024] The total filtering weights and the accumulated pixel sum are weighted and averaged to obtain the filtering result of the center pixel of the current sliding window;
[0025] The fusion weight is determined based on the initial brightness value of the center pixel. The initial brightness value is the pixel value of the image to be processed on the brightness plane. The fusion weight is used to control the mixing ratio between the filtering result and the initial brightness value.
[0026] Based on the fusion weights, the filtering results and the initial brightness values are weighted and fused to obtain the target brightness value corresponding to the center pixel.
[0027] The target brightness values of each sliding window are combined to obtain the target brightness plane.
[0028] In one possible implementation, determining the fusion weight based on the initial brightness value of the center pixel includes:
[0029] Obtain the preset brightness threshold;
[0030] When the initial brightness value is less than the brightness threshold, the fusion weight is determined as the first fusion weight, where the first fusion weight is positively correlated with the initial brightness value;
[0031] When the initial brightness value is greater than or equal to the brightness threshold, the fusion weight is determined as the second fusion weight, where the second fusion weight is the preset maximum weight value.
[0032] In one possible implementation, determining the target image based on the target luminance plane, the first initial chromaticity plane, and the second initial chromaticity plane includes:
[0033] The first initial chromaticity plane and the second initial chromaticity plane are downsampled respectively to obtain the first downsampled chromaticity plane and the second downsampled chromaticity plane;
[0034] The target image is obtained by combining the target brightness plane with the first chromaticity downsampling plane and the second chromaticity downsampling plane.
[0035] In one possible implementation, before acquiring pixel difference data for the first chromaticity plane, the second chromaticity plane, and the luminance plane of the image to be processed, the method further includes:
[0036] The first chromaticity plane of the image to be processed is downsampled and upsampled sequentially to obtain the first chromaticity plane;
[0037] The second chromaticity initial plane of the image to be processed is subjected to downsampling and upsampling processes in sequence to obtain the second chromaticity plane.
[0038] Secondly, embodiments of this application provide an image enhancement apparatus, comprising:
[0039] The acquisition module is used to acquire pixel difference data of the first chromaticity plane, the second chromaticity plane, and the luminance plane of the image to be processed. The pixel difference data is used to characterize the difference between the pixel value of each pixel point in the sliding window and the pixel value of the center pixel. The first chromaticity plane and the second chromaticity plane are obtained by performing resolution transformation processing on the first chromaticity initial plane and the second chromaticity initial plane respectively.
[0040] The first determining module is used to determine the filtering weights based on the pixel difference data. The filtering weights are used to characterize the contribution of the pixel values of each pixel in the current sliding window to the pixel value of the center pixel.
[0041] The processing module is used to perform weighted filtering on the brightness plane according to the filtering weights to obtain the target brightness plane;
[0042] The second determining module is used to determine the target image based on the target brightness plane, the first initial chromaticity plane, and the second initial chromaticity plane.
[0043] In one possible implementation, the acquisition module is specifically used for:
[0044] On the first chromaticity plane, the second chromaticity plane, and the luminance plane, respectively, a sliding window of a preset size is determined with each pixel as the center.
[0045] For each sliding window in the first chromaticity plane, calculate the difference between the pixel value of each pixel in the current sliding window and the pixel value of the center pixel to obtain the first pixel difference data;
[0046] For each sliding window of the second chromaticity plane, calculate the difference between the pixel value of each pixel in the current sliding window and the pixel value of the center pixel to obtain the second pixel difference data;
[0047] For each sliding window of the brightness plane, calculate the difference between the pixel value of each pixel in the current sliding window and the pixel value of the center pixel to obtain the third pixel difference data.
[0048] In one possible implementation, the first determining module is specifically used for:
[0049] Based on the first pixel difference data and the Gaussian function, the first weight factor corresponding to each pixel in the sliding window is determined. The first weight factor is used to characterize the similarity between the current pixel and the center pixel in the first chromaticity.
[0050] Based on the second pixel difference data and the Gaussian function, the second weighting factor corresponding to each pixel in the sliding window is determined. The second weighting factor is used to characterize the similarity between the current pixel and the center pixel in the second chromaticity.
[0051] Based on the third pixel difference data and the Gaussian function, the third weight factor corresponding to each pixel in the sliding window is determined. The third weight factor is used to characterize the similarity in brightness between the current pixel and the center pixel.
[0052] The first weight factor, the second weight factor, and the third weight factor are combined to obtain the filter weight.
[0053] In one possible implementation, the processing module is specifically used for:
[0054] For each sliding window, the filter weights in the sliding window are summed to obtain the total filter weights;
[0055] For each sliding window, in the brightness plane, calculate the product between the pixel value of each pixel and its corresponding filter weight, and sum the products to obtain the cumulative pixel sum;
[0056] The total filtering weights and the accumulated pixel sum are weighted and averaged to obtain the filtering result of the center pixel of the current sliding window;
[0057] The fusion weight is determined based on the initial brightness value of the center pixel. The initial brightness value is the pixel value of the image to be processed on the brightness plane. The fusion weight is used to control the mixing ratio between the filtering result and the initial brightness value.
[0058] Based on the fusion weights, the filtering results and the initial brightness values are weighted and fused to obtain the target brightness value corresponding to the center pixel.
[0059] The target brightness values of each sliding window are combined to obtain the target brightness plane.
[0060] In one possible implementation, the processing module is further configured to:
[0061] Obtain the preset brightness threshold;
[0062] When the initial brightness value is less than the brightness threshold, the fusion weight is determined as the first fusion weight, where the first fusion weight is positively correlated with the initial brightness value;
[0063] When the initial brightness value is greater than or equal to the brightness threshold, the fusion weight is determined as the second fusion weight, where the second fusion weight is the preset maximum weight value.
[0064] In one possible implementation, the second determining module is specifically used for:
[0065] The first initial chromaticity plane and the second initial chromaticity plane are downsampled respectively to obtain the first downsampled chromaticity plane and the second downsampled chromaticity plane;
[0066] The target image is obtained by combining the target brightness plane with the first chromaticity downsampling plane and the second chromaticity downsampling plane.
[0067] In one possible implementation, the image enhancement device is also used for:
[0068] The first chromaticity plane of the image to be processed is downsampled and upsampled sequentially to obtain the first chromaticity plane;
[0069] The second chromaticity initial plane of the image to be processed is subjected to downsampling and upsampling processes in sequence to obtain the second chromaticity plane.
[0070] Thirdly, this application provides a chip module on which a computer program is stored. When the computer program is executed by the chip module, it implements the image enhancement method as described in any of the first aspects.
[0071] Fourthly, embodiments of this application provide an electronic device, including: a memory and a processor;
[0072] The memory stores instructions that the computer executes;
[0073] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.
[0074] Fifthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.
[0075] In a sixth aspect, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.
[0076] The image enhancement method, electronic device, storage medium, and program product provided in this application acquire pixel difference data of the first chromaticity plane, the second chromaticity plane, and the luminance plane after resolution transformation, determine the filtering weights, and perform weighted filtering on the luminance plane. Finally, the target image is obtained by combining the initial chromaticity plane. The pixel difference features of the chromaticity plane are integrated into the filtering process of the luminance plane, so that the adjustment of the luminance plane matches the pixel features of the chromaticity plane. From the perspective of luminance plane optimization, this method compensates for the problem of luminance and chromaticity information mismatch caused by the loss of chromaticity information during the downsampling and upsampling process of traditional YUV444. It effectively reduces the distortion phenomena such as false edges and false points at the boundaries of high-saturation and high-luminance areas of the image, and improves the image quality. Attached Figure Description
[0077] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0078] Figure 1 A schematic diagram of an image conversion result provided in this application;
[0079] Figure 2 A flowchart illustrating an image enhancement method provided in this application. Figure 1 ;
[0080] Figure 3 for Figure 2 The flowchart illustrating the specific implementation method of step S101 in the embodiment shown is as follows;
[0081] Figure 4 for Figure 2 The flowchart illustrating the specific implementation method of step S102 in the embodiment shown is as follows;
[0082] Figure 5 for Figure 2 The flowchart illustrating the specific implementation method of step S103 in the embodiment shown is as follows;
[0083] Figure 6 A schematic diagram illustrating a method for determining fusion weights provided in this application;
[0084] Figure 7 A flowchart illustrating an image enhancement method provided in this application. Figure 2 ;
[0085] Figure 8 A schematic diagram of the image enhancement device provided in this application;
[0086] Figure 9 A schematic diagram of the structure of the electronic device provided in this application.
[0087] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0088] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0089] In the field of digital image processing, image signal processing technology is the core component for achieving high-quality image acquisition and display. As users' demands for image quality increase, especially in high dynamic range and high saturation scenes, higher requirements are being placed on the preservation of image details and the ability to reproduce colors.
[0090] In image processing, the YUV color space is widely used due to its efficient encoding characteristics for human visual perception. Among them, the YUV444 format retains the most complete image information due to its lack of chroma compression, but its large data size is unfavorable for storage and transmission. Therefore, in practical applications, it is usually necessary to downsample the YUV444 format to YUV420 to reduce bandwidth and storage requirements, while the final display requires upsampling the YUV420 back to the YUV444 format. See also Figure 1 , Figure 1 This is a schematic diagram of an image conversion result provided in this application, wherein the left side is the original YUV444 image, and the right side is the image after downsampling the original YUV444 image to YUV420 and then upsampling it to YUV444.
[0091] In related technologies, the conversion from YUV444 to YUV420 is usually achieved through simple chroma downsampling methods, such as 2×2 pixel block averaging based on bilinear interpolation, or more complex bicubic interpolation and Lanczos interpolation algorithms. The core of these methods is to downsample the UV plane to reduce the amount of data, while the Y plane data remains unchanged.
[0092] However, the chromaticity (UV) downsampling and upsampling operations in the above process can easily lead to artifacts at image edges (such as false edges and false dots), especially in high-saturation and high-brightness areas. This problem is particularly noticeable and seriously affects the naturalness of the image and the visual experience. Therefore, how to achieve efficient conversion from YUV444 to YUV420 under low-power and low-cost hardware conditions, and effectively suppress the image quality degradation caused by UV information loss, has become a key problem that urgently needs to be solved in current image processing technology.
[0093] Based on this, this application provides an image enhancement method. By acquiring pixel difference data of the first chromaticity plane, the second chromaticity plane, and the luminance plane after resolution transformation, the method determines the filtering weights and performs weighted filtering on the luminance plane. Finally, the target image is obtained by combining the initial chromaticity plane. The pixel difference features of the chromaticity plane are integrated into the filtering process of the luminance plane, so that the adjustment of the luminance plane matches the pixel features of the chromaticity plane. From the perspective of luminance plane optimization, this method compensates for the problem of luminance and chromaticity information mismatch caused by the loss of chromaticity information during the downsampling and upsampling process of traditional YUV444. It effectively reduces the distortion phenomena such as false edges and false points at the boundaries of high-saturation and high-brightness areas of the image, and improves the image quality.
[0094] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.
[0095] Figure 2 A flowchart illustrating an image enhancement method provided in this application. Figure 1 ,like Figure 2 As shown, the method includes:
[0096] S101. Obtain pixel difference data of the first chromaticity plane, the second chromaticity plane, and the luminance plane of the image to be processed.
[0097] The execution subject of this method can be a computing device or a processor located in the computing device.
[0098] The image to be processed is in YUV444 format. The first chromaticity plane refers to the U-plane, the second chromaticity plane refers to the V-plane, and the luma plane refers to the Y-plane. The first and second chromaticity planes are obtained by performing resolution transformation on the first and second initial chromaticity planes respectively. The first initial chromaticity plane is the U-plane of the unprocessed image to be processed, and the second initial chromaticity plane is the V-plane of the unprocessed image to be processed. The resolution transformation includes downsampling and upsampling. The first and second chromaticity planes are obtained by performing resolution transformation on the first and second initial chromaticity planes respectively.
[0099] In this embodiment, a sliding window is used to process each pixel in the image to be processed sequentially. The size of the sliding window is, for example, 3×3. The pixel difference data is the difference between the pixel value of each pixel within the current sliding window and the pixel value of the center pixel, where the center pixel is the pixel located in the middle position within the current sliding window. The pixel difference data includes pixel difference data in the U-plane, pixel difference data in the V-plane, and pixel difference data in the Y-plane.
[0100] Optionally, the pixel difference data includes first pixel difference data, second pixel difference data, and third pixel difference data. See also Figure 3 , Figure 3 for Figure 2 The flowchart illustrating the specific implementation of step S101 in the illustrated embodiment includes:
[0101] S1011. On the first chromaticity plane, the second chromaticity plane, and the luminance plane, respectively, a sliding window of a preset size is determined sequentially with each pixel as the center.
[0102] A sliding window of a preset size, such as 3×3, can be used to fill in the pixel values within the sliding window for boundary pixels by methods such as mirroring or flipping the edges. For example, for the first row of pixels, the second row of pixels can be mirrored and flipped to the top of the first row of pixels to form a negative row of pixels, so that the pixel values within the sliding window are complete when processing the first row of pixels.
[0103] S1012. For each sliding window of the first chromaticity plane, calculate the difference between the pixel value of each pixel in the current sliding window and the pixel value of the center pixel to obtain the first pixel difference data.
[0104] For any sliding window in the first chromaticity plane, the pixel in the sliding window can be represented as U(i,j), where (i,j) are the coordinates of the pixel in the sliding window.
[0105] The first pixel difference data can be determined by the following formula:
[0106]
[0107] in, This is the difference data for the first pixel. This is the center pixel of the current sliding window.
[0108] S1013. For each sliding window of the second chromaticity plane, calculate the difference between the pixel value of each pixel in the current sliding window and the pixel value of the center pixel to obtain the second pixel difference data.
[0109] For any sliding window in the second chromaticity plane, the pixel in the sliding window can be represented as V(i,j), where (i,j) are the coordinates of the pixel in the sliding window.
[0110] The second pixel difference data can be determined by the following formula:
[0111]
[0112] in, This is the difference data for the second pixel. This is the center pixel of the current sliding window.
[0113] S1014. For each sliding window of the brightness plane, calculate the difference between the pixel value of each pixel in the current sliding window and the pixel value of the center pixel to obtain the third pixel difference data.
[0114] For any sliding window in the brightness plane, the pixel in the sliding window can be represented as Y(i,j), where (i,j) are the coordinates of the pixel in the sliding window.
[0115] The third pixel difference data can be determined by the following formula:
[0116]
[0117] in, This is the difference data for the third pixel. This is the center pixel of the current sliding window.
[0118] In the above method, by determining sliding windows centered on pixels on the first chromaticity plane, the second chromaticity plane, and the luminance plane respectively, and calculating the pixel value difference between each pixel and the center pixel window by window to obtain the corresponding pixel difference data, the method achieves refined and region-by-region extraction of pixel feature differences on the three planes. This lays a data foundation for the effectiveness of subsequent luminance plane weighted filtering and further ensures the accuracy of luminance and chromaticity information matching.
[0119] S102. Determine the filtering weights based on the pixel difference data.
[0120] The filter weights are normalized weight values generated based on pixel difference data, with a value range of [0,1]. They are used to characterize the contribution of each pixel value within the sliding window to the pixel value of the center pixel. The larger the weight value, the higher the proportion and the greater the contribution of that pixel to the brightness calculation of the center pixel, and vice versa.
[0121] Based on the pixel difference data of the Y, U, and V planes, the weight of each single plane can be calculated for each pixel in the sliding window, and then the weights of the three planes can be fused to obtain the final filtering weight. The magnitude of the filtering weight is negatively correlated with the degree of pixel difference; the greater the difference, the smaller the filtering weight.
[0122] Optional, see Figure 4 , Figure 4 for Figure 2 The flowchart illustrating the specific implementation of step S102 in the illustrated embodiment includes:
[0123] S1021. Based on the first pixel difference data and the Gaussian function, determine the first weight factor corresponding to each pixel point within the sliding window.
[0124] The first weighting factor is used to characterize the similarity between the current pixel and the center pixel in the first chromaticity.
[0125] For any sliding window, the first weighting factor can be determined using a Gaussian function, as shown below:
[0126]
[0127] in, As the first weighting factor, The standard deviation of the Gaussian function is used to adjust the decay rate of the weights. The smaller the value, the faster the weight decays, and only pixels extremely close to the center pixel can obtain a higher weight, resulting in a stronger suppression effect on differing pixels. The larger the value, the slower the weight decay, allowing more pixels with some difference from the center pixel to receive higher weights, resulting in a smoother filtering effect. In practical applications, The value range is generally [1, 40], for example, 5.
[0128] Furthermore, it can adaptively adjust according to the actual scenario. For example, adjustments can be made based on brightness information, flatness, contrast, saturation, etc., where low-brightness areas can be enlarged. Suppress noise in dark areas and reduce noise in high-brightness areas. Preserve details and edges in highlighted areas; flat areas can be enlarged. To eliminate minor undulations, the textured area can be reduced. Protect edges and textures; low-contrast areas can be enlarged. Smoothing out subtle changes, high-contrast areas can be reduced. Protect edges to prevent blurring; low-saturation areas can reduce... To prevent color diffusion, high-saturation areas can be enlarged. To suppress artifacts.
[0129] S1022. Based on the second pixel difference data and the Gaussian function, determine the second weight factor corresponding to each pixel point within the sliding window.
[0130] The second weighting factor is used to characterize the similarity between the current pixel and the center pixel in the second chromaticity.
[0131] For any sliding window, the second weighting factor can be determined using a Gaussian function, as shown below:
[0132]
[0133] in, It is the second weighting factor.
[0134] S1023. Based on the third pixel difference data and the Gaussian function, determine the third weighting factor corresponding to each pixel point within the sliding window.
[0135] The third weighting factor is used to characterize the similarity in brightness between the current pixel and the center pixel.
[0136] For any sliding window, the third weighting factor can be determined using a Gaussian function, as shown below:
[0137]
[0138] in, It is the third weighting factor.
[0139] S1024. The first weight factor, the second weight factor, and the third weight factor are fused to obtain the filter weights.
[0140] Filter weights It can be determined using the following formula:
[0141]
[0142] In the above method, based on the pixel difference data of each plane, the corresponding weight factors are determined by combining the Gaussian function and fused to obtain the filtering weights. The characteristics of the Gaussian function are used to make the weight factors accurately represent the similarity between each pixel and the center pixel in the dimensions of chroma and luminance. The higher the similarity, the greater the weight contribution. This allows the fused filtering weights to comprehensively reflect the correlation of pixel features of the three planes, making the subsequent weighted filtering of the luminance plane more in line with the actual pixel distribution characteristics of the image. This avoids the distortion of luminance information caused by indiscriminate filtering and makes the adjustment of the luminance plane more suitable for the pixel features of the chroma plane. This effectively strengthens the matching between luminance and chroma information and alleviates the image boundary problem caused by the loss of chroma information.
[0143] S103. Perform weighted filtering on the brightness plane according to the filtering weights to obtain the target brightness plane.
[0144] Weighted filtering is an adaptive pixel value adjustment operation on the luminance plane based on filtering weights. Unlike traditional fixed-weight mean filtering, it only allows pixels with high contribution to participate in the luminance value calculation of the center pixel, and finally outputs the center pixel filtered value that adapts to the chroma plane. The target luminance plane refers to the new luminance plane obtained after weighted filtering, which is composed of the filtered luminance values of all center pixels in the image arranged according to their original coordinates.
[0145] Optional, see Figure 5 , Figure 5 for Figure 2 The flowchart illustrating the specific implementation of step S103 in the illustrated embodiment includes:
[0146] S1031. For each sliding window, sum the filter weights in the sliding window to obtain the total filter weights.
[0147] The total filter weights can be determined by the following formula:
[0148]
[0149] in, is the total filtering weight, and m and n are the sizes of the sliding window.
[0150] S1032. For each sliding window, in the brightness plane, calculate the product between the pixel value of each pixel and its corresponding filter weight, and sum the products to obtain the cumulative pixel sum.
[0151] The sum of pixels can be determined by the following formula:
[0152]
[0153] in, This is for accumulating pixel sums.
[0154] S1033. Perform a weighted average of the total filtering weights and the accumulated pixel sum to obtain the filtering result of the center pixel of the current sliding window.
[0155] The filtering result of the center pixel of the current sliding window can be determined by the following formula:
[0156]
[0157] in, This is the filtered result of the center pixel of the current sliding window.
[0158] S1034. Determine the fusion weight based on the initial brightness value of the center pixel.
[0159] The initial brightness value is the pixel value of the image to be processed on the brightness plane, and the fusion weight is used to control the mixing ratio between the filtering result and the initial brightness value.
[0160] For example, determining the fusion weight includes: obtaining a preset brightness threshold; when the initial brightness value is less than the brightness threshold, determining the fusion weight as a first fusion weight, wherein the first fusion weight is positively correlated with the initial brightness value; when the initial brightness value is greater than or equal to the brightness threshold, determining the fusion weight as a second fusion weight, wherein the second fusion weight is a preset maximum weight value.
[0161] The brightness threshold is preset; for 8-bit images, its value ranges from [0, 255], for example, 140. See also... Figure 6 , Figure 6 This diagram illustrates a method for determining fusion weights according to this application. The fusion weights are represented as follows: The horizontal axis represents the initial brightness value. Taking an 8-bit image as an example, the initial brightness value ranges from [0, 255]. The vertical axis represents the fusion weight, expressed as follows: .exist Within the initial brightness value, the fusion weight is positively correlated with the initial brightness value; the larger the initial brightness value, the larger the fusion weight. Within, the fusion weight is the maximum weight value, which is 1.
[0162] Furthermore, the brightness threshold can be adaptively adjusted according to the actual scene. For example, it can be adjusted based on brightness information, such as lowering the brightness threshold in low-brightness areas and increasing the brightness threshold in high-brightness areas.
[0163] By adjusting the contribution ratio of the filtering result according to the actual brightness characteristics of the pixels, the fusion degree of the filtering result is improved in high brightness areas (areas prone to false edges and false dots), enhancing the matching effect of brightness and color information. In low brightness areas, the original brightness information is moderately preserved, avoiding the loss of details caused by unnecessary filtering adjustments. This specifically solves the boundary distortion problem in high saturation and high brightness areas in traditional processing, making the image enhancement effect more in line with the visual needs of different brightness areas.
[0164] S1035. Based on the fusion weight, the filtering result and the initial brightness value are weighted and fused to obtain the target brightness value corresponding to the center pixel.
[0165] The target brightness value can be determined by the following formula:
[0166]
[0167] in, Target brightness value, This is the initial brightness value.
[0168] S1036. Combine the target brightness values of each sliding window to obtain the target brightness plane.
[0169] The target brightness values corresponding to each sliding window are arranged and combined according to the original coordinates to obtain the target brightness plane.
[0170] In the above method, by fusing the filtering results with the initial brightness value, the original effective information of the brightness plane is preserved, avoiding the loss of image details caused by over-filtering. While matching the brightness plane with the chromaticity information, the brightness level and detail features of the image itself are maintained. This not only solves the problem of brightness and chromaticity mismatch caused by the loss of chromaticity information, but also ensures the visual sense of hierarchy after image enhancement, thus improving the overall display effect.
[0171] S104. Determine the target image based on the target brightness plane, the first initial chromaticity plane, and the second initial chromaticity plane.
[0172] The target image is a combination of the target luminance plane, the plane obtained by downsampling the first initial chromaticity plane, and the plane obtained by downsampling the second initial chromaticity plane. The image format of the target image is YUV420.
[0173] For example, determining the target image includes: performing downsampling processing on the first initial chromaticity plane and the second initial chromaticity plane respectively to obtain the first downsampled chromaticity plane and the second downsampled chromaticity plane; and combining the target luminance plane with the first downsampled chromaticity plane and the second downsampled chromaticity plane to obtain the target image.
[0174] The downsampling process performed on the first and second initial chromaticity planes includes, for example, downsampling the first initial chromaticity plane by 1 / 2 × 1 / 2 on the U plane, reducing it to 1 / 4 of its original value, thus obtaining the first downsampled chromaticity plane; and downsampling the second initial chromaticity plane by 1 / 2 × 1 / 2 on the V plane, reducing it to 1 / 4 of its original value, thus obtaining the second downsampled chromaticity plane.
[0175] The image enhancement method provided in this application obtains pixel difference data of the first chromaticity plane, the second chromaticity plane, and the luminance plane after resolution transformation, determines the filtering weights, performs weighted filtering on the luminance plane, and finally combines the initial chromaticity plane to obtain the target image. By incorporating the pixel difference features of the chromaticity plane into the filtering process of the luminance plane, the adjustment of the luminance plane matches the pixel features of the chromaticity plane. From the perspective of luminance plane optimization, this method compensates for the problem of luminance and chromaticity information mismatch caused by the loss of chromaticity information during the downsampling and upsampling process of traditional YUV444. It effectively reduces distortion phenomena such as false edges and false points at the boundaries of high-saturation and high-brightness areas of the image, and improves image quality.
[0176] Figure 7 A flowchart illustrating an image enhancement method provided in this application. Figure 2 ,like Figure 7 As shown, the method includes:
[0177] S201. Perform downsampling on the first initial chromaticity plane of the image to be processed to obtain the first downsampled chromaticity plane. Perform downsampling on the second initial chromaticity plane of the image to be processed to obtain the second downsampled chromaticity plane.
[0178] For example, in the U-plane, the first initial chromaticity plane is downsampled by 1 / 2 × 1 / 2, reducing it to 1 / 4 of its original size, resulting in the first downsampled chromaticity plane. Similarly, in the V-plane, the second initial chromaticity plane is downsampled by 1 / 2 × 1 / 2, reducing it to 1 / 4 of its original size, resulting in the second downsampled chromaticity plane. The downsampling process can be a commonly used algorithm in this field, such as average downsampling or bilinear downsampling.
[0179] S202. Upsample the first chromaticity downsampling plane to obtain the first chromaticity plane; upsample the second chromaticity downsampling plane to obtain the second chromaticity plane.
[0180] For example, in the U-plane, the first chroma downsampling plane is upsampled 2×2 to restore it to its original size, resulting in the first chroma plane; in the V-plane, the second chroma downsampling plane is upsampled 2×2 to restore it to its original size, resulting in the second chroma plane. The upsampling process can be using commonly used nearest neighbor interpolation algorithms, bilinear interpolation algorithms, etc.
[0181] S203. Obtain pixel difference data of the first chromaticity plane, the second chromaticity plane, and the luminance plane of the image to be processed.
[0182] S204. Determine the filtering weights based on the pixel difference data.
[0183] S205. Perform weighted filtering on the brightness plane according to the filtering weights to obtain the target brightness plane.
[0184] S206. Determine the target image based on the target brightness plane, the first chromaticity downsampling plane, and the second chromaticity downsampling plane.
[0185] The target luminance plane obtained in step S205 is merged with the first chromaticity downsampling plane and the second chromaticity downsampling plane obtained in step S201 according to the YUV color space format to obtain the target image.
[0186] The image enhancement method provided in this application obtains pixel difference data of the first chromaticity plane, the second chromaticity plane, and the luminance plane after resolution transformation, determines the filtering weights, performs weighted filtering on the luminance plane, and finally combines the initial chromaticity plane to obtain the target image. By incorporating the pixel difference features of the chromaticity plane into the filtering process of the luminance plane, the adjustment of the luminance plane matches the pixel features of the chromaticity plane. From the perspective of luminance plane optimization, this method compensates for the problem of luminance and chromaticity information mismatch caused by the loss of chromaticity information during the downsampling and upsampling process of traditional YUV444. It effectively reduces distortion phenomena such as false edges and false points at the boundaries of high-saturation and high-brightness areas of the image, and improves image quality.
[0187] Figure 8 A schematic diagram of the image enhancement device provided in this application is shown below. Figure 8 As shown, the image enhancement device 80 provided in this embodiment includes:
[0188] The acquisition module 801 is used to acquire pixel difference data of the first chromaticity plane, the second chromaticity plane and the luminance plane of the image to be processed. The pixel difference data is used to characterize the difference between the pixel value of each pixel in the sliding window and the pixel value of the center pixel. The first chromaticity plane and the second chromaticity plane are obtained by performing resolution transformation processing on the first chromaticity initial plane and the second chromaticity initial plane respectively.
[0189] The first determining module 802 is used to determine the filtering weight based on the pixel difference data. The filtering weight is used to characterize the contribution of the pixel value of each pixel in the current sliding window to the pixel value of the center pixel.
[0190] Processing module 803 is used to perform weighted filtering on the brightness plane according to the filtering weights to obtain the target brightness plane;
[0191] The second determining module 804 is used to determine the target image based on the target brightness plane, the first initial chromaticity plane, and the second initial chromaticity plane.
[0192] In one possible implementation, the acquisition module 801 is specifically used for:
[0193] On the first chromaticity plane, the second chromaticity plane, and the luminance plane, respectively, a sliding window of a preset size is determined with each pixel as the center.
[0194] For each sliding window in the first chromaticity plane, calculate the difference between the pixel value of each pixel in the current sliding window and the pixel value of the center pixel to obtain the first pixel difference data;
[0195] For each sliding window of the second chromaticity plane, calculate the difference between the pixel value of each pixel in the current sliding window and the pixel value of the center pixel to obtain the second pixel difference data;
[0196] For each sliding window of the brightness plane, calculate the difference between the pixel value of each pixel in the current sliding window and the pixel value of the center pixel to obtain the third pixel difference data.
[0197] In one possible implementation, the first determining module 802 is specifically used for:
[0198] Based on the first pixel difference data and the Gaussian function, the first weight factor corresponding to each pixel in the sliding window is determined. The first weight factor is used to characterize the similarity between the current pixel and the center pixel in the first chromaticity.
[0199] Based on the second pixel difference data and the Gaussian function, the second weighting factor corresponding to each pixel in the sliding window is determined. The second weighting factor is used to characterize the similarity between the current pixel and the center pixel in the second chromaticity.
[0200] Based on the third pixel difference data and the Gaussian function, the third weight factor corresponding to each pixel in the sliding window is determined. The third weight factor is used to characterize the similarity in brightness between the current pixel and the center pixel.
[0201] The first weight factor, the second weight factor, and the third weight factor are combined to obtain the filter weight.
[0202] In one possible implementation, the processing module 803 is specifically used for:
[0203] For each sliding window, the filter weights in the sliding window are summed to obtain the total filter weights;
[0204] For each sliding window, in the brightness plane, calculate the product between the pixel value of each pixel and its corresponding filter weight, and sum the products to obtain the cumulative pixel sum;
[0205] The total filtering weights and the accumulated pixel sum are weighted and averaged to obtain the filtering result of the center pixel of the current sliding window;
[0206] The fusion weight is determined based on the initial brightness value of the center pixel. The initial brightness value is the pixel value of the image to be processed on the brightness plane. The fusion weight is used to control the mixing ratio between the filtering result and the initial brightness value.
[0207] Based on the fusion weights, the filtering results and the initial brightness values are weighted and fused to obtain the target brightness value corresponding to the center pixel.
[0208] The target brightness values of each sliding window are combined to obtain the target brightness plane.
[0209] In one possible implementation, the processing module 803 is further configured to:
[0210] Obtain the preset brightness threshold;
[0211] When the initial brightness value is less than the brightness threshold, the fusion weight is determined as the first fusion weight, where the first fusion weight is positively correlated with the initial brightness value;
[0212] When the initial brightness value is greater than or equal to the brightness threshold, the fusion weight is determined as the second fusion weight, where the second fusion weight is the preset maximum weight value.
[0213] In one possible implementation, the second determining module 804 is specifically used for:
[0214] The first initial chromaticity plane and the second initial chromaticity plane are downsampled respectively to obtain the first downsampled chromaticity plane and the second downsampled chromaticity plane;
[0215] The target image is obtained by combining the target brightness plane with the first chromaticity downsampling plane and the second chromaticity downsampling plane.
[0216] In one possible implementation, the image enhancement device 80 is further used for:
[0217] The first chromaticity plane of the image to be processed is downsampled and upsampled sequentially to obtain the first chromaticity plane;
[0218] The second chromaticity initial plane of the image to be processed is subjected to downsampling and upsampling processes in sequence to obtain the second chromaticity plane.
[0219] The image enhancement device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.
[0220] Figure 9 A schematic diagram of the structure of the electronic device provided in this application. Figure 9 As shown, the electronic device 90 provided in this embodiment includes at least one processor 901 and a memory 902. The processor 901 and the memory 902 are connected via a bus 903.
[0221] In a specific implementation, at least one processor 901 executes computer execution instructions stored in memory 902, causing at least one processor 901 to perform the above-described method.
[0222] The specific implementation process of processor 901 can be found in the above-described image enhancement method embodiment, which has a similar implementation principle and technical effect, and will not be repeated here.
[0223] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.
[0224] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.
[0225] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.
[0226] This application also provides a chip on which a computer program is stored. When the computer program is executed by the chip, it implements the above-described image enhancement method. The corresponding content and effects can be referred to the embodiment section of the image enhancement method, and will not be repeated here.
[0227] This application also provides a chip module on which a computer program is stored. When the computer program is executed by the chip module, it implements the above-mentioned image enhancement method. The corresponding content and effects can be referred to the embodiment section of the image enhancement method, which will not be repeated here.
[0228] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described image enhancement method.
[0229] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described image enhancement method.
[0230] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.
[0231] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.
[0232] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.
[0233] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0234] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0235] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0236] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.
[0237] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.
Claims
1. An image enhancement method, characterized in that, The method includes: Pixel difference data of the first chromaticity plane, the second chromaticity plane, and the luminance plane of the image to be processed are obtained. The pixel difference data is used to characterize the difference between the pixel value of each pixel point in the sliding window and the pixel value of the center pixel. The first chromaticity plane and the second chromaticity plane are obtained by performing resolution transformation processing on the first chromaticity initial plane and the second chromaticity initial plane respectively. Based on the pixel difference data, a filtering weight is determined. The filtering weight is used to characterize the contribution of the pixel value of each pixel in the current sliding window to the pixel value of the center pixel. The brightness plane is weighted and filtered according to the filtering weights to obtain the target brightness plane. The target image is determined based on the target luminance plane, the first initial chromaticity plane, and the second initial chromaticity plane.
2. The image enhancement method according to claim 1, characterized in that, The pixel difference data includes first pixel difference data, second pixel difference data, and third pixel difference data. The step of obtaining pixel difference data for the first chromaticity plane, the second chromaticity plane, and the luminance plane of the image to be processed includes: On the first chromaticity plane, the second chromaticity plane, and the luminance plane, respectively, a sliding window of a preset size is sequentially determined with each pixel as the center; For each sliding window of the first chromaticity plane, the difference between the pixel value of each pixel in the current sliding window and the pixel value of the center pixel is calculated to obtain the first pixel difference data; For each sliding window of the second chromaticity plane, the difference between the pixel value of each pixel in the current sliding window and the pixel value of the center pixel is calculated to obtain the second pixel difference data; For each sliding window of the brightness plane, the difference between the pixel value of each pixel in the current sliding window and the pixel value of the center pixel is calculated to obtain the third pixel difference data.
3. The image enhancement method according to claim 2, characterized in that, The step of determining the filtering weights based on the pixel difference data includes: Based on the first pixel difference data and the Gaussian function, the first weight factor corresponding to each pixel in the sliding window is determined. The first weight factor is used to characterize the similarity between the current pixel and the center pixel in the first chromaticity. Based on the second pixel difference data and the Gaussian function, a second weighting factor is determined for each pixel within the sliding window. The second weighting factor is used to characterize the similarity between the current pixel and the center pixel in the second chromaticity. Based on the third pixel difference data and the Gaussian function, the third weight factor corresponding to each pixel in the sliding window is determined. The third weight factor is used to characterize the similarity in brightness between the current pixel and the center pixel. The first weight factor, the second weight factor, and the third weight factor are fused to obtain the filter weight.
4. The image enhancement method according to claim 2, characterized in that, The step of performing weighted filtering on the brightness plane according to the filtering weights to obtain the target brightness plane includes: For each sliding window, the filter weights in the sliding window are summed to obtain the total filter weights; For each sliding window, in the brightness plane, the product between the pixel value of each pixel and its corresponding filter weight is calculated, and the product is summed to obtain the cumulative pixel sum; The total filtering weights and the accumulated pixel sum are weighted and averaged to obtain the filtering result of the center pixel of the current sliding window; The fusion weight is determined based on the initial brightness value of the center pixel, where the initial brightness value is the pixel value of the image to be processed on the brightness plane, and the fusion weight is used to control the mixing ratio of the filtering result and the initial brightness value. Based on the fusion weight, the filtering result and the initial brightness value are weighted and fused to obtain the target brightness value corresponding to the center pixel; The target brightness values of each sliding window are combined to obtain the target brightness plane.
5. The image enhancement method according to claim 4, characterized in that, The step of determining the fusion weight based on the initial brightness value of the center pixel includes: Obtain the preset brightness threshold; When the initial brightness value is less than the brightness threshold, the fusion weight is determined as the first fusion weight, wherein the first fusion weight is positively correlated with the initial brightness value; When the initial brightness value is greater than or equal to the brightness threshold, the fusion weight is determined as the second fusion weight, wherein the second fusion weight is a preset maximum weight value.
6. The image enhancement method according to claim 1, characterized in that, Determining the target image based on the target luminance plane, the first initial chromaticity plane, and the second initial chromaticity plane includes: The first initial chromaticity plane and the second initial chromaticity plane are downsampled respectively to obtain the first downsampled chromaticity plane and the second downsampled chromaticity plane. The target luminance plane is combined with the first chroma downsampling plane and the second chroma downsampling plane to obtain the target image.
7. The image enhancement method according to claim 1, characterized in that, Before acquiring the pixel difference data of the first chromaticity plane, the second chromaticity plane, and the luminance plane of the image to be processed, the method further includes: The first chromaticity plane of the image to be processed is subjected to downsampling and upsampling processes in sequence to obtain the first chromaticity plane. The second chromaticity initial plane of the image to be processed is subjected to downsampling and upsampling processes in sequence to obtain the second chromaticity plane.
8. An image enhancement device, characterized in that, include: The acquisition module is used to acquire pixel difference data of the first chromaticity plane, the second chromaticity plane, and the luminance plane of the image to be processed. The pixel difference data is used to characterize the difference between the pixel value of each pixel point in the sliding window and the pixel value of the center pixel. The first chromaticity plane and the second chromaticity plane are obtained by performing resolution transformation processing on the first chromaticity initial plane and the second chromaticity initial plane respectively. The first determining module is used to determine the filtering weight based on the pixel difference data. The filtering weight is used to characterize the contribution of the pixel value of each pixel in the current sliding window to the pixel value of the center pixel. The processing module is used to perform weighted filtering on the brightness plane according to the filtering weights to obtain the target brightness plane; The second determining module is used to determine the target image based on the target brightness plane, the first initial chromaticity plane, and the second initial chromaticity plane.
9. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-7.