Image processing method, electronic device, storage medium, and program product

Abstract

Embodiments of the present application provide an image processing method, an electronic device, a storage medium and a program product. Relate to computer field. The method comprises: obtaining an original image to be processed, a first corrected image and a second corrected image, the first corrected image and the second corrected image are obtained after the original image is subjected to shadow correction processing, the pixel value range of the first corrected image is less than that of the second corrected image; determining a first weight set of each pixel point in the first corrected image according to the first corrected image; performing layered processing on the original image to obtain an original flat layer and an original detail layer corresponding to the original image, and performing layered processing on the second corrected image to obtain a corrected flat layer and a corrected detail layer corresponding to the second corrected image; and generating a target image processed according to the original flat layer, the original detail layer, the corrected flat layer, the corrected detail layer and the first weight set. Improve the effect of image processing.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly to an image processing method, electronic device, storage medium, and program product. Background Technology

[0002] Due to the optical characteristics of a lens, when light passes through a convex lens, the light concentration occurs in the central area and the light attenuation occurs in the edge area, thus producing the lens shadow effect.

[0003] In related technologies, lens shadow correction algorithms can be used to increase the brightness of the image edge areas, making the overall image brightness more uniform.

[0004] However, the above method can cause pixel values ​​in the originally highlighted areas of the image to be truncated after correction because they exceed the data storage limit, resulting in the complete loss of highlight details and thus poor image processing results. Summary of the Invention

[0005] This application provides an image processing method, electronic device, storage medium, and program product to improve the image processing effect.

[0006] In a first aspect, embodiments of this application provide an image processing method, including:

[0007] Obtain the original image to be processed, the first corrected image, and the second corrected image. The first corrected image and the second corrected image are obtained after the original image has undergone shadow correction processing. The pixel value range of the first corrected image is smaller than that of the second corrected image.

[0008] Based on the first corrected image, a first weight set for each pixel in the first corrected image is determined, and the first weight set is used to indicate the color saturation of each pixel in the first corrected image.

[0009] The original image is processed into layers to obtain the original flattening layer and the original detail layer corresponding to the original image. The second corrected image is also processed into layers to obtain the corrected flattening layer and the corrected detail layer corresponding to the second corrected image.

[0010] The processed target image is generated based on the original flattening layer, the original detail layer, the corrected flattening layer, the corrected detail layer, and the first weight set.

[0011] In one possible implementation, determining a first weight set for each pixel within the first corrected image, based on the first corrected image, includes:

[0012] The first corrected image is subjected to white balance processing to obtain the first preprocessed image;

[0013] The first preprocessed image is de-mosaiced to obtain the second preprocessed image;

[0014] The first weight set is determined based on the second preprocessed image.

[0015] In one possible implementation, determining a first weight set based on the second preprocessed image includes:

[0016] Determine the maximum and minimum brightness values ​​of the three channels for each pixel in the second preprocessed image;

[0017] Determine the difference between the maximum and minimum brightness values ​​of the three channels for each pixel;

[0018] The first weight of each pixel is determined based on the difference between the corresponding pixels.

[0019] The first weight set is determined based on the first weight of each pixel.

[0020] In one possible implementation, the processed target image is generated based on the original flattening layer, the original detail layer, the corrected flattening layer, the corrected detail layer, and the first weight set, including:

[0021] Based on the original image and the first weight set, the second weight set and the third weight set of the original image are determined. The second weight set and the third weight set are determined based on the brightness values ​​of each pixel in the original image.

[0022] The original flattening layer and the corrected flattening layer are weighted according to the second weight set to obtain the target flattening layer;

[0023] The original detail layer and the corrected detail layer are weighted according to the third weight set to obtain the target detail layer;

[0024] Based on the target flattening layer and the target detail layer, the processed target image is obtained.

[0025] In one possible implementation, determining a second weight set and a third weight set for the original image based on the original image and a first weight set includes:

[0026] Based on the original image and the first weight set, determine the brightness set of each pixel in the original image;

[0027] Gaussian filtering is applied to the brightness set to obtain a flat brightness set of the original image;

[0028] Based on the brightness set and the flat brightness set, determine the detail brightness set of the original image;

[0029] Based on the flatness set, determine the second weight set corresponding to the flat layer, and based on the detail brightness set, determine the third weight set corresponding to the detail layer.

[0030] In one possible implementation, the processed target image is obtained based on the target flattening layer and the target detail layer, including:

[0031] The target flattening layer and the target detail layer are overlaid to obtain the pre-target image;

[0032] Determine the range of pixel values ​​for the original image;

[0033] The target image is stored based on the pixel value range of the original image;

[0034] The stored pre-target image is truncated to obtain the processed target image.

[0035] Secondly, embodiments of this application provide an image processing apparatus, including: an acquisition module, a determination module, a layer processing module, and a generation module, wherein,

[0036] The acquisition module is used to acquire the original image to be processed, the first corrected image, and the second corrected image. The first corrected image and the second corrected image are obtained after the original image has undergone shadow correction processing. The pixel value range of the first corrected image is smaller than that of the second corrected image.

[0037] The determining module is used to determine a first weight set for each pixel in the first corrected image based on the first corrected image, wherein the first weight set is used to indicate the color saturation of each pixel in the first corrected image.

[0038] The layer processing module is used to perform layer processing on the original image to obtain the original flat layer and the original detail layer corresponding to the original image, and to perform layer processing on the second corrected image to obtain the corrected flat layer and the corrected detail layer corresponding to the second corrected image.

[0039] The generation module is used to generate the processed target image based on the original flattening layer, the original detail layer, the corrected flattening layer, the corrected detail layer, and the first weight set.

[0040] In one possible implementation, the module is specifically used for,

[0041] The first corrected image is subjected to white balance processing to obtain the first preprocessed image;

[0042] The first preprocessed image is de-mosaiced to obtain the second preprocessed image;

[0043] The first weight set is determined based on the second preprocessed image.

[0044] In one possible implementation, the module is specifically used for,

[0045] Determine the maximum and minimum brightness values ​​of the three channels for each pixel in the second preprocessed image;

[0046] Determine the difference between the maximum and minimum brightness values ​​of the three channels for each pixel;

[0047] The first weight of each pixel is determined based on the difference between the corresponding pixels.

[0048] The first weight set is determined based on the first weight of each pixel.

[0049] In one possible implementation, the generation module is specifically used for,

[0050] Based on the original image and the first weight set, the second weight set and the third weight set of the original image are determined. The second weight set and the third weight set are determined based on the brightness values ​​of each pixel in the original image.

[0051] The original flattening layer and the corrected flattening layer are weighted according to the second weight set to obtain the target flattening layer;

[0052] The original detail layer and the corrected detail layer are weighted according to the third weight set to obtain the target detail layer;

[0053] Based on the target flattening layer and the target detail layer, the processed target image is obtained.

[0054] In one possible implementation, the generation module is specifically used for,

[0055] Based on the original image and the first weight set, determine the brightness set of each pixel in the original image;

[0056] Gaussian filtering is applied to the brightness set to obtain a flat brightness set of the original image;

[0057] Based on the brightness set and the flat brightness set, determine the detail brightness set of the original image;

[0058] Based on the flatness set, determine the second weight set corresponding to the flat layer, and based on the detail brightness set, determine the third weight set corresponding to the detail layer.

[0059] In one possible implementation, the generation module is specifically used for,

[0060] The target flattening layer and the target detail layer are overlaid to obtain the pre-target image;

[0061] Determine the range of pixel values ​​for the original image;

[0062] The target image is stored based on the pixel value range of the original image;

[0063] The stored pre-target image is truncated to obtain the processed target image.

[0064] Thirdly, embodiments of this application provide an electronic device, including: at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, causing the at least one processor to perform the image processing method as described in the first aspect and various possible designs of the first aspect.

[0065] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the image processing method described in the first aspect and various possible designs of the first aspect.

[0066] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the image processing method described in the first aspect and various possible designs of the first aspect.

[0067] In a sixth aspect, embodiments of this application provide a chip, the chip including at least one processor, the processor being configured to execute program instructions to implement the image processing method as described in the first aspect above and various possible designs of the first aspect.

[0068] This application provides an image processing method, electronic device, storage medium, and program product. When processing an original image, the electronic device can acquire the original image to be processed, a first corrected image, and a second corrected image. The first and second corrected images are obtained after shadow correction processing of the original image, and the pixel value range of the first corrected image is smaller than that of the second corrected image. Based on the first corrected image, a first weight set for each pixel in the first corrected image is determined. The original image is processed in layers to obtain an original flat layer and an original detail layer corresponding to the original image. The second corrected image is also processed in layers to obtain a corrected flat layer and a corrected detail layer corresponding to the second corrected image. Based on the original flat layer, the original detail layer, the corrected flat layer, the corrected detail layer, and the first weight set, a processed target image is generated. In this way, the weights can be determined based on the image features after shadow correction, making the weight allocation more consistent with the image content characteristics. At the same time, by dividing the image into flat and detail layers through layer processing, the weighting process can be adjusted for the image characteristics of different layers, thereby improving the image processing effect. Attached Figure Description

[0069] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0070] Figure 1 This is a schematic diagram of the system architecture provided for an embodiment of this application;

[0071] Figure 2 A schematic flowchart of an image processing method provided in an embodiment of this application;

[0072] Figure 3 A schematic diagram illustrating the process of determining the first weight set provided in an embodiment of this application;

[0073] Figure 4 A schematic diagram illustrating the process of generating the processed target image provided in an embodiment of this application;

[0074] Figure 5 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application;

[0075] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0076] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0077] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0078] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation access points for users to choose to authorize or refuse.

[0079] Furthermore, the technical solution involved in this application, which involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.) and the use of artificial intelligence technology for automated decision-making, and makes decisions that have a significant impact on personal rights based on the results of automated decision-making, provides users with corresponding operation entry points for users to choose to agree to or reject the results of automated decision-making; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0080] It should be noted that the image processing method, electronic device, storage medium, and program product provided in this application can be used in the field of image processing, or in any field other than image processing. The application field of the image processing method, electronic device, storage medium, and program product in this application is not limited.

[0081] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0082] To facilitate understanding, the following will be combined with... Figure 1 The system architecture applicable to the embodiments of this application will be described.

[0083] Figure 1 This is a schematic diagram of the system architecture provided for an embodiment of this application. Please refer to [link / reference]. Figure 1 This includes electronic devices, such as cameras equipped with CMOS (Complementary Metal-Oxide-Semiconductor Image Sensor) image sensors, like mobile phones, tablets, camcorders, and video-enabled smartwatches. Electronic devices can capture raw images through a lens, but due to lens shading, the central area of ​​the raw image is brighter than the edge areas. To improve image quality, electronic devices can perform image processing on the raw image to obtain a target image with uniform brightness and higher image quality.

[0084] In related technologies, lens shading correction algorithms can enhance the brightness of image edge areas to make the overall image brightness more uniform. However, in these methods, the correction process typically applies a higher gain to edge pixel values, for example, 4-5 times at the edges, while the gain in the center area is only 1. While this approach can improve the image brightness distribution, it causes pixel values ​​in originally bright areas to exceed the data storage limit after correction, resulting in truncation and complete loss of highlight details. For example, in backlit or high-contrast scenes, images taken by users may lose crucial details due to overexposure correction.

[0085] To address the aforementioned technical problems, in this embodiment, when processing of the original image is required, the electronic device can acquire the original image to be processed, a first corrected image, and a second corrected image. The first and second corrected images are obtained after shadow correction processing of the original image, and the pixel value range of the first corrected image is smaller than that of the second corrected image. Based on the first corrected image, a first weight set for each pixel within the first corrected image is determined. The original image is then processed in layers to obtain an original flat layer and an original detail layer corresponding to the original image. The second corrected image is also processed in layers to obtain a corrected flat layer and a corrected detail layer corresponding to the second corrected image. Based on the original flat layer, the original detail layer, the corrected flat layer, the corrected detail layer, and the first weight set, a processed target image is generated. Thus, through the above method, weights can be determined based on the image features after shadow correction, making the weight allocation more closely aligned with the image content characteristics. Simultaneously, by dividing the image into flat and detail layers through layer processing, the weighting process can be adjusted for the image characteristics of different layers, thereby improving the image processing effect.

[0086] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0087] Figure 2 This is a schematic flowchart illustrating an image processing method provided in an embodiment of this application. Please refer to [link / reference]. Figure 2 As shown, the method may include the following steps:

[0088] S201. Obtain the original image to be processed, the first corrected image, and the second corrected image.

[0089] The execution subject in this application embodiment can be an electronic device, which can be a camera equipped with a CMOS (Complementary Metal-Oxide-Semiconductor Image Sensor), such as a mobile phone, tablet, camera, or video-enabled smartwatch. The execution subject can also be an image processing device located within the electronic device. The image processing device can be implemented through software or a combination of software and hardware.

[0090] The original image can refer to the image directly output by the CMOS image sensor without any correction processing. Understandably, due to the lens shading effect, the brightness of the surrounding areas of the original image is lower, while the brightness of the central area is higher. At the same time, since no correction processing has been performed, the original image retains all the details.

[0091] The first corrected image can be the image obtained after performing shadow correction processing on the original image. The pixel value range of the first corrected image can be the same as the pixel value range of the original image.

[0092] The second corrected image can be the image obtained after applying shadow correction to the original image. The pixel value range of the first corrected image is smaller than that of the second corrected image. Understandably, the pixel value range of the second corrected image is larger; that is, the second corrected image can accommodate the enlarged pixel values ​​resulting from the shadow correction process.

[0093] For example, the pixel value range of the original image can be 14 bits, the pixel value range of the first corrected image can be 14 bits, and the pixel value range of the second corrected image can be 16 bits.

[0094] Shadow correction processing can refer to the shadow correction algorithm (LSC). That is, shadow correction processing can refer to assigning a corresponding gain value to each pixel of the original image, and the gain of the pixels located at the edge of the original image is higher than that of the pixels located at the center of the original image, thereby increasing the brightness of the edge areas of the original image and making the overall image brightness more uniform.

[0095] For example, a gain of 4x can be assigned to pixels at the edges of the original image, and a gain of 1x can be assigned to pixels at the center of the original image.

[0096] The pixel value range refers to the minimum to maximum brightness value that a single pixel can store. The pixel value range is determined by the number of bits used for storage; for example, a 14-bit range is 0 to 16383, and a 16-bit range is 0 to 65535. Understandably, a larger pixel value range means that a pixel can accommodate more brightness levels, making overflow and truncation less likely.

[0097] S202. Based on the first corrected image, determine the first weight set of each pixel in the first corrected image.

[0098] The first weight set can be used to indicate the color saturation of each pixel in the first corrected image. The first weight set can refer to the set composed of the weights corresponding to each pixel in the first corrected image. It can be understood that each pixel can correspond to a first weight, and the value of the first weight can be from 0 to 1.

[0099] In one possible implementation, for any given pixel, the first weight corresponding to that pixel can be used to indicate the color saturation of that pixel. That is, the greater the color saturation of the pixel, the greater the first weight, and the smaller the color saturation of the pixel, the smaller the first weight.

[0100] Understandably, the larger the first weight (closer to 1), the greater the probability that the pixel is located in a detailed region. Conversely, the smaller the first weight (closer to 0), the greater the probability that the pixel is located in a flat region. Detailed regions can refer to details, edges, and highlighted areas in an image, while flat regions can refer to areas in an image that lack detail and are less prone to overexposure.

[0101] For example, suppose the first corrected image includes pixel 1, pixel 2, pixel 3 and pixel 4, the first weight of pixel 1 is 0.4, the first weight of pixel 2 is 0.6, the first weight of pixel 3 is 0.5 and the first weight of pixel 4 is 0.4, then the first weight set is (0.4, 0.6, 0.5, 0.4).

[0102] Understandably, if the original image is used to determine the first weight set, the lens shading effect of the original image will result in lower brightness around the edges and higher brightness in the center area, which will make the calculated weights inaccurate. If the second corrected image is used to determine the first weight set, the pixel value range of the second corrected image is relatively high, and using high-precision data to calculate the weights will lead to a waste of computing resources. However, if the first weight set is determined based on the first corrected image, since the first corrected image has already completed shadow correction and the brightness of the image is uniform, it can better and more realistically reflect whether each pixel is a detail area.

[0103] S203. Perform layer processing on the original image to obtain the original flattening layer and the original detail layer corresponding to the original image. Perform layer processing on the second corrected image to obtain the corrected flattening layer and the corrected detail layer corresponding to the second corrected image.

[0104] Layered processing can refer to dividing an image into a flat layer and a detail layer; that is, an image is equal to the superposition of the flat layer and the detail layer.

[0105] Specifically, the original image can be divided into an original flattening layer and an original detail layer, and the second corrected image can be divided into a corrected flattening layer and a corrected detail layer.

[0106] The original flat layer can refer to the image obtained after applying Gaussian filtering to the original image. The original flat layer can be considered as an image without details, edges, or noise, containing only the large area brightness distribution of the original image.

[0107] The original detail layer can refer to the image obtained by subtracting the original flat layer from the original image. The original detail layer can be considered as containing only the details, edges, textures, and high-frequency information of the original image, without containing large-area brightness distribution, and without being amplified by shadow correction. In other words, the original detail layer is not overexposed and has complete details.

[0108] The correction flattening layer can refer to the image obtained after applying Gaussian filtering to the second correction image. The correction flattening layer can be considered as an image without details, edges, or noise, containing only the large area brightness distribution of the second correction image.

[0109] The correction detail layer can refer to the image obtained by subtracting the correction flattening layer from the second correction image. The correction detail layer can be considered as containing only the details, edges, textures, and high-frequency information of the second correction image, without containing large-area brightness distribution, and without being amplified by shadow correction. In other words, the correction detail layer has no overexposure and complete details.

[0110] Understandably, the original image has complete details and no overexposure, but the brightness around the edges is low. The second corrected image has uniform brightness and no vignetting, but the details may be blown out due to magnification. Based on the characteristics of the original image and the second corrected image, by performing layered processing on the original image and the second corrected image, we can obtain the complete details of the original image and the complete flat area of ​​the second corrected image, thereby improving the image processing effect.

[0111] S204. Generate the processed target image based on the original flattening layer, the original detail layer, the corrected flattening layer, the corrected detail layer, and the first weight set.

[0112] The target image can refer to the final high-quality image, that is, the target image has uniform brightness without vignetting, and complete details without overexposure or truncation.

[0113] In one possible implementation, the processed target image can be generated as follows: Based on the original image and a first weight set, a second weight set and a third weight set are determined for the original image, wherein the second and third weight sets are determined based on the brightness values ​​of each pixel in the original image; the original flattening layer and the corrected flattening layer are weighted according to the second weight set to obtain the target flattening layer; the original detail layer and the corrected detail layer are weighted according to the third weight set to obtain the target detail layer; and the processed target image is obtained based on the target flattening layer and the target detail layer.

[0114] The brightness value of each pixel in the original image can refer to the brightness of each pixel in the original image.

[0115] The second weight set corresponds to the fusion weight of the flattening layer, which is a fusion of the first weight and the brightness values ​​of each pixel in the original image. It can be understood that the lower the brightness value of the original image, the more the second weight is biased towards correcting the flattening layer, that is, prioritizing brightening and eliminating vignetting. The higher the brightness value of the original image, the more the second weight is biased towards the original flattening layer, avoiding over-correction that leads to brightness distortion.

[0116] The third weight set corresponds to the fusion weight of the detail layer, which is a fusion of the first weight and the brightness values ​​of each pixel in the original image. It can be understood that the lower the brightness value of the original image, the more the third weight is biased towards the original detail layer, that is, it prioritizes the protection of the original details that have not been magnified and avoids overexposure. The higher the brightness value of the original image, the more the third weight is biased towards the correction detail layer, supplementing the corrected details and improving the clarity.

[0117] In this embodiment, when image processing is required, the electronic device can acquire the original image to be processed, a first corrected image, and a second corrected image. The original image may refer to the image directly output by the CMOS image sensor without any correction processing. The first and second corrected images may be images obtained after shadow correction processing of the original image. The pixel value range of the first corrected image is smaller than that of the second corrected image. Based on the first corrected image, a first weight set for each pixel in the first corrected image is determined. The first weight set may refer to the set composed of the weights corresponding to each pixel in the first corrected image. The original image is then processed in layers to obtain the original image. The original flattening layer and original detail layer of the image are used to perform layered processing on the second corrected image, resulting in a corrected flattening layer and a corrected detail layer corresponding to the second corrected image. Layered processing can refer to dividing the image into flattening and detail layers. The original flattening layer can refer to the image obtained after Gaussian filtering the original image, and the original detail layer can refer to the image obtained by subtracting the original flattening layer from the original image. The corrected flattening layer can refer to the image obtained after Gaussian filtering the second corrected image, and the corrected detail layer can refer to the image obtained by subtracting the corrected flattening layer from the second corrected image. Based on the original flattening layer, original detail layer, corrected flattening layer, corrected detail layer, and the first weight set, the processed target image is generated. In this way, the weights can be determined based on the image features after shadow correction, making the weight allocation more in line with the image content characteristics. At the same time, dividing the image into flattening and detail layers through layered processing allows the weighting process to be adjusted for the image characteristics at different levels, thereby improving the image processing effect.

[0118] Based on any of the above embodiments, the following, in conjunction with Figure 3 The process of determining the first weight set ( Figure 2 The embodiment of S202 will be described in detail.

[0119] Figure 3 This is a schematic diagram illustrating the process of determining the first weight set provided in an embodiment of this application. Please refer to... Figure 3 The method may include:

[0120] S301. Perform white balance processing on the first corrected image to obtain the first preprocessed image.

[0121] White balance processing refers to adjusting the brightness ratio of the three channels (red (R), green (G), and blue (B)) in the first calibrated image to eliminate color temperature shifts caused by different light sources (e.g., sunlight, artificial light, fluorescent light), thereby making the white in the first calibrated image appear as true white, and thus ensuring that the brightness values ​​of each color channel can truly reflect the color differences of the pixels.

[0122] In one possible implementation, white balance processing may include: detecting neutral color regions, such as white regions and gray regions, in the first calibrated image; calculating the gain coefficient of each channel based on the R, G, and B brightness ratios of the neutral color regions, for example, increasing the gain of the B channel when the B channel is too dark, so that R=G=B in the neutral color regions; applying the gain coefficients to each pixel of the first calibrated image to adjust the brightness of the R, G, and B channels of each pixel, thereby obtaining the first preprocessed image.

[0123] Understandably, the first calibrated image has inherent color temperature cast. This means that images captured by a CMOS sensor will exhibit brightness imbalances in the R, G, and B channels due to variations in the color temperature of the light source (e.g., warm light appears yellowish, cool light appears bluish). For instance, under warm light, the R channel brightness might be higher than the B channel brightness. Directly calculating pixel brightness differences in this case would be affected by color cast interference, failing to accurately reflect pixel detail. Meanwhile, white balance processing does not alter image detail or brightness uniformity. It only adjusts the proportions of the R, G, and B channels, without amplifying or reducing pixel values ​​or changing image detail contours and brightness distribution. This maximizes the preservation of brightness uniformity in the first calibrated image while eliminating color cast interference, ensuring the effectiveness of subsequent image processing.

[0124] S302. Perform de-mosaic processing on the first preprocessed image to obtain the second preprocessed image.

[0125] De-mosaic processing can refer to Bayer interpolation processing, which converts the original photosensitive RAW image in Bayer filter array format into a three-channel (red, green, and blue) RGB image.

[0126] Understandably, the first preprocessed image is a Bayer format RAW image, meaning that each pixel contains only one color from R (red channel), Gr (green-red channel), Gb (green-blue channel), and B (blue channel). The de-mosaic processing uses interpolation algorithms, such as bilinear interpolation and adaptive interpolation, to restore the missing two color channel values ​​of each pixel based on the color information of the neighboring pixels around it, so that each pixel has complete R, G, and B channel brightness values.

[0127] In this way, after the first preprocessed image is de-mosaiced to obtain the second preprocessed image, the second preprocessed image can inherit the characteristics of the first preprocessed image, such as true color, uniform brightness, and no dark corners. At the same time, each pixel of the second preprocessed image has complete R, G, and B channel brightness values, which ensures the effect of subsequent image processing.

[0128] S303. Determine the first weight set based on the second preprocessed image.

[0129] The first weight set can refer to the set composed of the weights corresponding to each pixel in the second preprocessed image. The first weight set can be used to distinguish between flat areas and detail areas in the second preprocessed image. The larger the weight, the more likely the pixel is to be a detail / edge / overexposed area; the smaller the weight, the more likely the pixel is to be a flat area.

[0130] In one possible implementation, the first weight set can be determined as follows: determine the maximum and minimum brightness values ​​of the three channels corresponding to each pixel in the second preprocessed image; determine the difference between the maximum and minimum brightness values ​​of the three channels corresponding to each pixel; determine the first weight of each pixel based on the difference; and determine the first weight set based on the first weight of each pixel.

[0131] In this process, the difference between each pixel can be normalized to obtain the first weight of each pixel, so that the value of the first weight is between 0 and 1.

[0132] Understandably, the larger the first weight of a pixel, the more it conforms to the characteristics of a detailed area; the smaller the first weight of a pixel, the more it conforms to the characteristics of a flat area. At the same time, if the first weight is larger and the maximum brightness value of the three channels is larger, it means that the area belongs to a detailed area that is prone to overexposure.

[0133] For example, for any pixel, assuming the pixel's RGB = (60, 200, 150), we can determine that the pixel's maximum luminance value for the three channels is max(r, g, b) = 200, and the minimum luminance value for the three channels is min(r, g, b) = 60. Then, the difference between the maximum and minimum luminance values ​​for the three channels of this pixel is 140. If the pixel value range is 8 bits, and the pixel value range corresponding to 8 bits is 0 to 255, then we can divide the difference by the maximum value of the pixel value range to obtain the first weight. That is, the first weight of this pixel is 0.55.

[0134] For example, suppose the second preprocessed image includes pixel 1, pixel 2, pixel 3 and pixel 4, and the first weight corresponding to pixel 1 is 0.2, the first weight corresponding to pixel 2 is 0.4, the first weight corresponding to pixel 1 is 0.5 and the first weight corresponding to pixel 4 is 0.6, then the first weight set is (0.2, 0.4, 0.5, 0.6).

[0135] exist Figure 3In the illustrated embodiment, when it is necessary to determine the first weight set, white balance processing can be performed on the first corrected image to obtain a first pre-processed image. White balance processing can refer to adjusting the brightness ratio of the R, G, and B channels in the first corrected image to eliminate color temperature shifts caused by different light sources. De-mosaic processing can be performed on the first pre-processed image to obtain a second pre-processed image. De-mosaic processing can refer to converting the Bayer format RAW image to an RGB image. Based on the maximum and minimum brightness values ​​of the three channels corresponding to each pixel in the second pre-processed image, the difference between the maximum and minimum brightness values ​​of the three channels corresponding to each pixel is determined as the first weight of each pixel, thereby determining the first weight set. In this way, by first eliminating interference factors such as color cast and channel loss through white balance processing and de-mosaic processing, and then determining the weights based on the second pre-processed image, the final first weight set can accurately reflect the detail intensity of each pixel, avoiding misjudgment of detail areas or flat areas, thereby improving the effect of subsequent image processing.

[0136] Based on any of the above embodiments, the following, in conjunction with Figure 4 The process of generating the processed target image ( Figure 2 The embodiment of S204 will be described in detail.

[0137] Figure 4 This is a schematic diagram illustrating the process of generating the processed target image provided in an embodiment of this application. Please refer to... Figure 4 The method may include:

[0138] S401. Based on the original image and the first weight set, determine the second weight set and the third weight set of the original image.

[0139] The second and third weight sets are determined based on the brightness values ​​of each pixel in the original image.

[0140] In one possible implementation, the second and third weight sets can be determined as follows: based on the original image and the first weight set, determine the brightness set of each pixel in the original image; perform Gaussian filtering on the brightness set to obtain the flat brightness set of the original image; based on the brightness set and the flat brightness set, determine the detail brightness set of the original image; based on the flat brightness set, determine the second weight set corresponding to the flat layer; and based on the detail brightness set, determine the third weight set corresponding to the detail layer.

[0141] Specifically, determining the brightness set of each pixel in the original image based on the original image and the first weight set may include: performing demosaic processing on the original image to obtain a third preprocessed image with complete brightness values ​​of the R, G, and B channels; and determining the brightness set of each pixel based on the brightness values ​​of the R, G, and B channels of each pixel and the first weight set.

[0142] Specifically, for any pixel, assuming the red channel brightness value of the pixel is represented by R, the green channel brightness value by G, the blue channel brightness value by B, the first weight corresponding to the pixel in the first weight set is represented by weight, and the maximum brightness value of the pixel's three channels is represented by max(r, g, b), then the brightness Y of the pixel can be represented as:

[0143]

[0144] For example, for any pixel, assuming the brightness values ​​of the three channels R, G, and B of the pixel are (R=400, G=300, B=500), and the first weight corresponding to the pixel in the first weight set is 0.5, then the brightness Y of the pixel is 850.

[0145] The detailed brightness set can be obtained by subtracting the flat brightness set from the brightness set.

[0146] Specifically, based on the flat brightness set, the second weight set corresponding to the flat layer is determined. This can be obtained by normalizing the flat brightness set. Similarly, based on the detail brightness set, the third weight set corresponding to the detail layer is determined. This can also be obtained by normalizing the detail brightness set.

[0147] S402. The original flattening layer and the corrected flattening layer are weighted according to the second weight set to obtain the target flattening layer.

[0148] Specifically, assuming the original flattening layer is represented as sensor1, the corrected flattening layer as LSC1, and the second weight set as weight1, then the target flattening layer new1 can be represented as:

[0149] new1= sensor1* weight1+ LSC1* (1- weight1)

[0150] S403. The original detail layer and the corrected detail layer are weighted according to the third weight set to obtain the target detail layer.

[0151] Specifically, assuming the original detail layer is represented as sensor2, the corrected detail layer as LSC2, and the third weight set as weight2, then the target detail layer new2 can be represented as:

[0152] new2= sensor2* weight2+ LSC2* (1- weight2)

[0153] S404. Based on the target flattening layer and the target detail layer, the processed target image is obtained.

[0154] In one possible implementation, the processed target image can be obtained by: overlaying the target flattening layer and the target detail layer to obtain a pre-target image; determining the pixel value range of the original image; storing the pre-target image according to the pixel value range of the original image; and truncating the stored pre-target image to obtain the processed target image.

[0155] Among them, the overlay process can refer to adding the target flat layer and the target detail layer pixel by pixel to obtain a pre-target image that retains brightness and detail.

[0156] In one possible implementation, the target flattening layer and the target detail layer can be superimposed as follows: Assuming the target flattening layer is represented as new1, the target detail layer as new2, and the control coefficient as k, the pre-target image new can be represented as:

[0157] new = new1 + new2 * k

[0158] Among them, the control coefficient k can be a value preset by the user. k can be used to indicate the richness of the details of the target image. It can be understood that if k is less than 1, it means that the target image has less detail but also less noise. If k is greater than 1, it means that the target image has more detail but also more noise.

[0159] Understandably, if the pixel value range of the original image is 14 bits, then the target image is stored according to 14 bits, and the part exceeding 14 bits is truncated to obtain the processed target image.

[0160] For example, the range of 14-bit pixel values ​​is 0 to 16383. If the pixel value of any pixel in the target image exceeds 16383, for example, if the pixel value is 17000, then the pixel will be truncated, that is, the pixel value of the pixel will be modified to 16383.

[0161] exist Figure 4In the illustrated embodiment, when a processed target image needs to be generated, the electronic device can determine a second weight set and a third weight set for the original image based on the original image and a first weight set. The second and third weight sets are determined based on the brightness values ​​of each pixel in the original image. The original flattening layer and the corrected flattening layer are weighted according to the second weight set to obtain a target flattening layer. The original detail layer and the corrected detail layer are weighted according to the third weight set to obtain a target detail layer. The processed target image is then obtained from the target flattening layer and the target detail layer. Thus, by using the above method, the second and third weight sets can be determined based on the brightness values ​​of the pixels. Simultaneously, combined with the detail differentiation function of the first weight set, targeted weighting is applied to the original flattening layer and the corrected flattening layer, as well as the original detail layer and the corrected detail layer. This eliminates vignetting in the original image, avoids overexposure in bright areas, and preserves detail authenticity, ultimately resulting in a target image with uniform brightness, no vignetting, and complete detail, thereby improving the image processing effect.

[0162] Figure 5 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application. Please refer to... Figure 5 The image processing device 10 includes: an acquisition module 11, a determination module 12, a layer processing module 13, and a generation module 14, wherein,

[0163] The acquisition module 11 is used to acquire the original image to be processed, the first corrected image and the second corrected image. The first corrected image and the second corrected image are obtained after the original image has undergone shadow correction processing. The pixel value range of the first corrected image is smaller than that of the second corrected image.

[0164] The determining module 12 is used to determine a first weight set for each pixel in the first corrected image based on the first corrected image.

[0165] The layer processing module 13 is used to perform layer processing on the original image to obtain the original flat layer and the original detail layer corresponding to the original image, and to perform layer processing on the second corrected image to obtain the corrected flat layer and the corrected detail layer corresponding to the second corrected image.

[0166] The generation module 14 is used to generate the processed target image based on the original flattening layer, the original detail layer, the corrected flattening layer, the corrected detail layer, and the first weight set.

[0167] The image processing apparatus provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0168] In one possible implementation, the determining module 12 is specifically used for,

[0169] The first corrected image is subjected to white balance processing to obtain the first preprocessed image;

[0170] The first preprocessed image is de-mosaiced to obtain the second preprocessed image;

[0171] The first weight set is determined based on the second preprocessed image.

[0172] In one possible implementation, the determining module 12 is specifically used for,

[0173] Determine the maximum and minimum brightness values ​​of the three channels for each pixel in the second preprocessed image;

[0174] Determine the difference between the maximum and minimum brightness values ​​of the three channels for each pixel;

[0175] The first weight of each pixel is determined based on the difference between the corresponding pixels.

[0176] The first weight set is determined based on the first weight of each pixel.

[0177] In one possible implementation, the generation module 14 is specifically used for,

[0178] Based on the original image and the first weight set, the second weight set and the third weight set of the original image are determined. The second weight set and the third weight set are determined based on the brightness values ​​of each pixel in the original image.

[0179] The original flattening layer and the corrected flattening layer are weighted according to the second weight set to obtain the target flattening layer;

[0180] The original detail layer and the corrected detail layer are weighted according to the third weight set to obtain the target detail layer;

[0181] Based on the target flattening layer and the target detail layer, the processed target image is obtained.

[0182] In one possible implementation, the generation module 14 is specifically used for,

[0183] Based on the original image and the first weight set, determine the brightness set of each pixel in the original image;

[0184] Gaussian filtering is applied to the brightness set to obtain a flat brightness set of the original image;

[0185] Based on the brightness set and the flat brightness set, determine the detail brightness set of the original image;

[0186] Based on the flatness set, determine the second weight set corresponding to the flat layer, and based on the detail brightness set, determine the third weight set corresponding to the detail layer.

[0187] In one possible implementation, the generation module 14 is specifically used for,

[0188] The target flattening layer and the target detail layer are overlaid to obtain the pre-target image;

[0189] Determine the range of pixel values ​​for the original image;

[0190] The target image is stored based on the pixel value range of the original image;

[0191] The stored pre-target image is truncated to obtain the processed target image.

[0192] The image processing apparatus provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0193] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 6 As shown, the electronic device 20 may include: a transceiver 21, a processor 22, and a memory 23.

[0194] Processor 22 executes computer execution instructions stored in memory, causing processor 22 to perform the scheme in the above embodiments. Processor 22 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0195] The memory 23 is connected to the processor 22 via the system bus and completes communication between them. The memory 23 is used to store computer program instructions.

[0196] Transceiver 21 can be used to obtain the task to be run and the configuration information of the task to be run.

[0197] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. Transceivers are used to enable communication between database access devices and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.

[0198] The electronic device provided in this application embodiment can be the terminal device described in the above embodiments.

[0199] This application also provides a chip for executing instructions, which is used to execute the image processing method described in the above embodiments.

[0200] This application also provides a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the technical solution of the image processing method described above.

[0201] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, it can implement the technical solution of the image processing method in the above embodiments.

[0202] This application provides a chip, which includes at least one processor. The processor is used to execute program instructions to implement the image processing method described in the first aspect above and various possible designs of the first aspect.

[0203] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0204] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0205] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit composed of the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0206] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0207] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0208] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0209] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0210] The aforementioned storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0211] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic control unit or main control device.

[0212] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0213] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An image processing method, characterized in that, include: Obtain the original image to be processed, the first corrected image, and the second corrected image. The first corrected image and the second corrected image are obtained by performing shadow correction processing on the original image. The pixel value range of the first corrected image is smaller than that of the second corrected image. Based on the first corrected image, a first weight set for each pixel in the first corrected image is determined, and the first weight set is used to indicate the color saturation of each pixel in the first corrected image. The original image is processed into layers to obtain the original flat layer and the original detail layer corresponding to the original image. The second corrected image is processed into layers to obtain the corrected flat layer and the corrected detail layer corresponding to the second corrected image. The processed target image is generated based on the original flattening layer, the original detail layer, the corrected flattening layer, the corrected detail layer, and the first weight set.

2. The method according to claim 1, characterized in that, Based on the first corrected image, a first weight set for each pixel within the first corrected image is determined, including: The first corrected image is subjected to white balance processing to obtain the first preprocessed image; The first preprocessed image is de-mosaiced to obtain the second preprocessed image; The first weight set is determined based on the second preprocessed image.

3. The method according to claim 2, characterized in that, Based on the second preprocessed image, the first weight set is determined, including: Determine the maximum and minimum brightness values ​​of the three channels for each pixel in the second preprocessed image; Determine the difference between the maximum brightness value and the minimum brightness value of the three channels corresponding to each pixel; The first weight of each pixel is determined based on the difference between the pixels. The first weight set is determined based on the first weight of each pixel.

4. The method according to any one of claims 1-3, characterized in that, Based on the original flattening layer, the original detail layer, the corrected flattening layer, the corrected detail layer, and the first weight set, a processed target image is generated, including: Based on the original image and the first weight set, a second weight set and a third weight set of the original image are determined, wherein the second weight set and the third weight set are determined based on the brightness values ​​of each pixel in the original image; The original flattening layer and the corrected flattening layer are weighted according to the second weight set to obtain the target flattening layer; The original detail layer and the corrected detail layer are weighted according to the third weight set to obtain the target detail layer; The processed target image is obtained based on the target flattening layer and the target detail layer.

5. The method according to claim 4, characterized in that, Based on the original image and the first weight set, a second weight set and a third weight set for the original image are determined, including: Based on the original image and the first weight set, determine the brightness set of each pixel in the original image; The brightness set is subjected to Gaussian filtering to obtain a flat brightness set of the original image; The detail brightness set of the original image is determined based on the brightness set and the flat brightness set; Based on the set of flat brightness, a second weight set corresponding to the flat layer is determined, and based on the set of detailed brightness, a third weight set corresponding to the detailed layer is determined.

6. The method according to claim 4, characterized in that, Based on the target flattening layer and the target detail layer, the processed target image is obtained, including: The target flattening layer and the target detail layer are overlaid to obtain a pre-target image; Determine the range of pixel values ​​for the original image; The pre-target image is stored according to the pixel value range of the original image; The stored pre-target image is truncated to obtain the processed target image.

7. An image processing apparatus, characterized in that, include: The module consists of an acquisition module, a determination module, a hierarchical processing module, and a generation module. The acquisition module is used to acquire the original image to be processed, the first corrected image, and the second corrected image. The first corrected image and the second corrected image are obtained after the original image has undergone shadow correction processing. The pixel value range of the first corrected image is smaller than that of the second corrected image. The determining module is used to determine a first weight set for each pixel in the first corrected image based on the first corrected image, wherein the first weight set is used to indicate the color saturation of each pixel in the first corrected image. The layer processing module is used to perform layer processing on the original image to obtain the original flat layer and the original detail layer corresponding to the original image, and to perform layer processing on the second corrected image to obtain the corrected flat layer and the corrected detail layer corresponding to the second corrected image. The generation module is used to generate a processed target image based on the original flattening layer, the original detail layer, the corrected flattening layer, the corrected detail layer, and the first weight set.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 6.

Images