Image processing method, apparatus, device, storage medium and product

Abstract

The present disclosure relates to an image processing method, device, equipment, storage medium and product. The method comprises: in response to the existence of a target region in a first image, and the deviation value between the initial white balance gain value determined based on the pixel value of the target region and the reference white balance gain value being greater than a preset threshold value, determining the distribution information of the target region in the first image; based on the distribution information, adjusting the pixel value of the target region, and determining a target white balance gain value based on the adjusted pixel value; and adjusting the first image based on the target white balance gain value to obtain a first target image, thereby reducing the misjudgment of the environment color temperature by the white balance algorithm, and further making the image color temperature of the first target image closer to the environment color temperature, and improving the color restoration degree of the first target image.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, and more particularly to an image processing method, apparatus, device, storage medium, and product. Background Technology

[0002] In the field of image processing, when the light source in the shooting environment changes, the image sensor captures different color information under the light source, resulting in color cast in the image, which requires white balance processing. When the environmental information of the shooting environment is complex or the camera's white balance algorithm is imperfect, the image color temperature adjustment is prone to inaccurate, resulting in a mismatch between the color temperature of the image captured by the camera and the color temperature of the shooting environment. Summary of the Invention

[0003] To overcome the problems in related technologies, this disclosure provides an image processing method, apparatus, device, storage medium, and product, thereby reducing the misjudgment of ambient color temperature by white balance algorithms, and thus making the color temperature of the first target image closer to the ambient color temperature, thereby improving the color reproduction of the first target image.

[0004] According to a first aspect of the present disclosure, an image processing method is provided, comprising:

[0005] In response to the presence of a target region in the first image, and the deviation between the initial white balance gain value determined based on the pixel value of the target region and the reference white balance gain value being greater than a preset threshold, the distribution information of the target region in the first image is determined;

[0006] Based on the distribution information, the pixel values ​​of the target region are adjusted, and based on the adjusted pixel values, the target white balance gain value is determined.

[0007] The first image is adjusted based on the target white balance gain value to obtain the first target image.

[0008] According to a second aspect of the present disclosure, an image processing apparatus is provided, comprising:

[0009] The first determining module is configured to determine the distribution information of the target region in the first image in response to the presence of a target region in the first image and the deviation between the initial white balance gain value determined based on the pixel value of the target region and the reference white balance gain value being greater than a preset threshold.

[0010] The execution module is configured to adjust the pixel values ​​of the target region based on the distribution information, and determine the target white balance gain value based on the adjusted pixel values;

[0011] The first acquisition module is configured to adjust the first image based on the target white balance gain value to obtain the first target image.

[0012] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0013] processor;

[0014] Memory used to store computer programs or instructions;

[0015] The processor executes computer programs or instructions to implement the steps in any of the image processing methods in the first aspect described above.

[0016] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, comprising:

[0017] When a computer program or instruction in a storage medium is executed by a processor, the steps in any of the image processing methods in the first aspect described above are implemented.

[0018] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program or instructions, which, when executed by a processor, implement the steps of any of the image processing methods in the first aspect described above.

[0019] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0020] In this embodiment of the present disclosure, in response to the presence of a target region in the first image, and the deviation between the initial white balance gain value determined based on the pixel values ​​of the target region and the reference white balance gain value is greater than a preset threshold, the distribution information of the target region in the first image is determined; based on the distribution information, the pixel values ​​of the target region are adjusted, and based on the adjusted pixel values, a target white balance gain value is determined; the first image is adjusted based on the target white balance gain value to obtain the first target image.

[0021] In this embodiment of the disclosure, when the deviation between the initial white balance gain value and the reference white balance gain value is large, it is determined that the white balance algorithm misjudges the ambient color temperature, resulting in low accuracy of the determined initial white balance gain value. Then, by using the distribution information of the target area in the first image, the pixel value of the target area is adjusted to balance the color distribution in the first image, so that the white balance algorithm can accurately capture the relationship between the pixel value of the target area and the ambient color temperature, thereby reducing the situation where the white balance algorithm misjudges the ambient color temperature, and making the color temperature of the first target image closer to the ambient color temperature, thus improving the color reproduction of the first target image.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0024] Figure 1 This is a flowchart illustrating an image processing method according to an exemplary embodiment. Figure 1 .

[0025] Figure 2 This is a schematic diagram of the sensitivity curve of a human eye cone cell according to an exemplary embodiment.

[0026] Figure 3a This is a schematic diagram illustrating the structure of a classification model according to an exemplary embodiment.

[0027] Figure 3b This is a flowchart illustrating a training method for a classification model according to an exemplary embodiment.

[0028] Figure 4a This is a schematic diagram illustrating a preset color space according to an exemplary embodiment.

[0029] Figure 4b This is a flowchart illustrating an image processing method according to an exemplary embodiment. Figure 2 .

[0030] Figure 4c This is a schematic image illustrating an exemplary embodiment. Figure 1 ;

[0031] Figure 4d This is a schematic image illustrating an exemplary embodiment. Figure 2 ;

[0032] Figure 4e This is a schematic diagram of an image according to an exemplary embodiment;

[0033] Figure 4f This is a schematic diagram of an image according to an exemplary embodiment;

[0034] Figure 4g This is a schematic image illustrating an exemplary embodiment. Figure 5 ;

[0035] Figure 5 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment.

[0036] Figure 6 This is a structural block diagram of an electronic device 600 according to an exemplary embodiment. Detailed Implementation

[0037] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0038] Figure 1 This is a flowchart illustrating an image processing method according to an exemplary embodiment. Figure 1 ,like Figure 1 As shown, this image processing method mainly includes the following steps:

[0039] In step 101, in response to the presence of a target region in the first image and the deviation between the initial white balance gain value determined based on the pixel value of the target region and the reference white balance gain value being greater than a preset threshold, the distribution information of the target region in the first image is determined.

[0040] In step 102, the pixel values ​​of the target region are adjusted based on the distribution information, and the target white balance gain value is determined based on the adjusted pixel values.

[0041] In step 103, the first image is adjusted based on the target white balance gain value to obtain the first target image.

[0042] It should be noted that the image processing method proposed in this disclosure can be applied to electronic devices. Here, electronic devices may include terminal devices, such as mobile terminals or fixed terminals. Mobile terminals may include mobile phones, tablets, laptops, wearable electronic devices, etc. Fixed terminals may include desktop computers, smart TVs, in-vehicle systems, etc. In other embodiments, the image processing method can also be applied to applications installed on electronic devices.

[0043] In other embodiments, the image processing method described in this disclosure can be configured in an image processing device, which can be located in an electronic device; this disclosure does not limit this. It should be noted that the execution entity of this disclosure can be a central processing unit (CPU) in the electronic device in hardware, and a related background service in the electronic device in software; this is not limited.

[0044] In some embodiments, Figure 2 This is a schematic diagram illustrating the sensitivity curve of a human eye's cone cells according to an exemplary embodiment, such as... Figure 2As shown, the human eye has three types of photoreceptors: L (long), M (medium), and S (short) cone cells. These cone cells have different spectral sensitivities (BGR in the diagram). Therefore, the human eye can respond to light and perceive color. Here, B represents the sensitivity range of S-type cone cells (short wavelength light), G represents the sensitivity range of M-type cone cells (medium wavelength light), and R represents the sensitivity range of L-type cone cells (long wavelength light).

[0045] When shooting in a special scene (i.e., when there are no white or gray objects in the scene), the human eye can adapt to the environment and identify objects in the environment, ensuring that all objects in the environment are under standard lighting conditions. The automatic white balance algorithm restores white objects under various light sources to white and restores objects with color casts to the color under standard lighting (i.e., the D65 light source with a color temperature of 6500K as defined by the International Commission on Illumination (CIE)).

[0046] Here, white and gray are neutral colors, containing no obvious color bias. White and gray objects can reflect most or all of the incident light, making their color performance relatively stable under different light sources and less susceptible to the influence of ambient color temperature. When white objects are present in an image, observing their color allows for a rough assessment of the light source's color temperature, enabling appropriate corrections and ensuring the image remains unaffected by the light source, resulting in a picture with normal color temperature. However, when white objects are absent or constitute a small proportion of the image, the camera's white balance algorithm may fail to accurately identify and adapt to the light source's color temperature, potentially leading to color deviations in the image.

[0047] In some embodiments, the white balance algorithm includes at least: a statistical algorithm and a learning-based algorithm. The statistical algorithm is based on certain statistical assumptions about natural scenes and obtains color constancy patterns through statistical analysis of a large number of images, thereby achieving color constancy. The learning-based algorithm uses machine learning techniques to enable a neural network to learn color constancy patterns from training data, obtaining a neural network model for estimating the color of scene light sources, thus achieving color constancy.

[0048] Here, statistical algorithms can include the Gray World Assumption and the Perfect Reflector Assumption.

[0049] In some embodiments, the gray world algorithm assumes that the average reflectance of light by objects in nature is a constant value, which can be approximated as "gray". This assumption holds that for an image with a large number of color variations, the average values ​​of the R, G, and B color channels tend to the same gray value. The white balance algorithm applies this assumption to the image to be processed and weakens the influence of the light source on the overall image to accurately estimate the ambient color temperature of the shooting scene in order to obtain a target image that matches the ambient color temperature.

[0050] In other embodiments, the perfect reflection method assumes that the brightest points in the image (i.e., the points with the largest pixel values) perfectly reflect external light, and these points should be considered white. By traversing the points with the largest RGB values ​​in the image to obtain white pixels, the RGB values ​​of other pixels in the image are stretched or scaled based on the RGB values ​​of these white pixels, so that the white balance algorithm accurately estimates the ambient color temperature of the shooting scene to obtain a target image that matches the ambient color temperature.

[0051] Taking a scene containing only blue objects and capturing a pure blue image as an example, the average values ​​of the R and G channels in the image will be much lower than the value of the B channel. The grayscale world hypothesis assumes that the average values ​​of the R, G, and B channels tend to converge to the same grayscale value. However, in a pure blue image, it's impossible to find a suitable grayscale value that makes the average values ​​of the three channels equal. Simultaneously, the perfect reflection hypothesis assumes that the maximum response of the RGB channels in the image is caused by white surfaces in the scene, while the maximum response of the B channel is caused by blue areas, not white surfaces. This makes the perfect reflection hypothesis inaccurate in determining the ambient color temperature. Therefore, for pure blue images, white balance algorithms are prone to misjudging the ambient color temperature.

[0052] Based on this, in this embodiment of the disclosure, a target area is preset, and when the target area is detected in the first image, an initial white balance gain value is determined based on the pixel value of the target area, and the initial white balance gain value is matched with a reference white balance gain value to determine whether the white balance algorithm accurately judges the ambient color temperature.

[0053] Here, the first image can be an image captured in real time when the electronic device receives the shooting command, or it can be any image stored in the database, that is, an image not captured in real time by the electronic device.

[0054] In some embodiments, the target area may consist of one area or multiple identical or different areas. The target area may be a solid color area, such as a solid blue area or a solid yellow area, or it may be a non-solid color area, such as a wood grain area or a patterned area. This disclosure does not limit the specific color of the target area.

[0055] Here, the reference white balance gain value matches the true color temperature of the first image. By adjusting the pixel values ​​of the first image based on the reference white balance gain value, a first image with a normal color temperature can be obtained. The reference white balance gain value can be a white balance gain value determined based on a preset image obtained from a shooting scene similar to the first image and containing white objects, or it can be a white balance gain value set based on experience or experimental data. This embodiment does not limit the specific white balance gain value.

[0056] Here, the pixel value of the target area can be the original pixel value of the target area, or it can be the pixel value after pre-adjustment of the original pixel value. This disclosure does not limit it in this way.

[0057] In some embodiments, during automatic white balance processing, a preset weight value for each pixel can be determined based on the brightness, color distribution, etc., of each pixel within the target area. The original pixel values ​​of the target area are then weighted based on these preset weight values ​​to obtain pre-adjusted pixel values. This allows the white balance algorithm to identify white areas or key color areas in the image, laying the foundation for color temperature correction. For example, pixels within the target area with moderate brightness—neither overexposed nor underexposed—reflect true lighting conditions and are assigned higher preset weight values. Furthermore, a higher ambient color temperature results in a more bluish light emission from the light source, while a lower ambient color temperature results in a more reddish or yellowish light emission. When the ambient color temperature is higher, blue pixels are assigned higher preset weight values, while yellow pixels are assigned lower preset weight values.

[0058] Understandably, to improve the accuracy of determining the initial white balance gain value, a preset threshold can be set in advance, and the deviation between the initial white balance gain value and the reference white balance gain value can be determined. When the deviation value is greater than the preset threshold, the white balance algorithm has misjudged the ambient color temperature or the color balance in the image, making the determined initial white balance gain value inaccurate. When the deviation value is less than or equal to the preset threshold, the white balance algorithm has high accuracy in determining the ambient color temperature or the color balance in the image, making the determined initial white balance gain value accurate.

[0059] Here, the preset threshold can be set arbitrarily according to requirements, and this embodiment does not limit it.

[0060] In some embodiments, the white balance gain value includes a red gain value for adjusting the brightness or intensity of the red channel in the image, a green gain value for adjusting the brightness or intensity of the green channel in the image, and a blue gain value for adjusting the brightness or intensity of the blue channel in the image.

[0061] It should be noted that by analyzing the distribution information of the target region in the first image, the positional information, quantity information, density information, and / or relative positional relationship of the pixels in the target region with non-target regions can be determined. This information is related to the white balance algorithm's determination of the ambient color temperature. For example, based on positional and quantity information, the ambient color temperature and color tendency can be determined; based on density information, color differences and transitions in the first image can be identified.

[0062] Therefore, after adjusting the pixel values ​​of the target region based on its distribution information in the first image, the white balance algorithm can accurately simulate and reproduce the color performance of the target region's pixels under different color temperatures based on the adjusted pixel values. At the same time, adjusting the pixel values ​​of the target region is essentially a process of color correction of the first image. By enhancing or reducing the intensity of specific components, the color distribution in the first image is balanced, enabling the white balance algorithm to accurately capture the relationship between the pixel values ​​of the target region and the ambient color temperature, thereby reducing the possibility of the white balance algorithm misjudging the ambient color temperature.

[0063] Here, the adjustment amount of each pixel value in the target area can be the same or different.

[0064] In some embodiments, an initial weight value is determined based on the distribution information of the target region in the first image; the pixel values ​​of the target region are weighted based on the initial weight value to obtain the adjusted pixel values, wherein density information is negatively correlated with the weight value, and position information is negatively correlated with the weight value.

[0065] In other embodiments, a first mapping relationship between the target pixel value of the target region and a preset range is established in advance. After obtaining the proportion, the preset range in which the proportion is located is determined. Based on the preset range and the first mapping relationship, the target pixel value of the target region is determined, and the pixel value of the target region is adjusted to the target pixel value.

[0066] In this embodiment of the present disclosure, after obtaining the adjusted pixel values, a target white balance gain value is determined based on the adjusted pixel values; and the first image is adjusted based on the target white balance gain value to obtain a first target image, so that the color temperature of the first target image is closer to the ambient color temperature, thereby improving the color reproduction of the first target image.

[0067] Understandably, since the adjusted pixel values ​​can accurately reflect the ambient color temperature and improve the situation where the white balance algorithm misjudges the ambient color temperature, the deviation between the target white balance gain value and the reference gain value is less than or equal to the preset threshold, making it less likely for the first target image adjusted based on the target white balance gain value to have color cast problems.

[0068] In some embodiments, adjusting the first image based on the target white balance gain value can be done by adjusting some pixel values ​​of the first image based on the target white balance gain value, or by adjusting all pixel values ​​of the first image based on the target white balance gain value, as long as the color temperature of the first image can be corrected. This disclosure does not limit this.

[0069] Here, the target white balance gain value can be determined based on the gray-world method or the perfect reflection method, and this disclosure does not limit it.

[0070] For example, the target white balance gain value is determined based on the gray world algorithm, and then the first image is adjusted based on the target white balance gain value to obtain the first target image, which is achieved by the following formulas (1)-(3):

[0071] First, determine the grayscale value:

[0072]

[0073] In formula (1), These are the adjusted mean values ​​of the red, green, and blue channels for all pixels. This is the grayscale value.

[0074] Then the gain values ​​used on the R, G, and B channels are as follows:

[0075]

[0076] In formula (2), gain r gain g gain b These are the gain values ​​for the red channel, green channel, and blue channel, respectively.

[0077] Finally, the pixels of the first image are adjusted based on the target white balance gain value to obtain the target pixel value:

[0078]

[0079] Another example is that the target white balance gain value is determined based on the perfect reflection method, and then the first image is adjusted based on the target white balance gain value to obtain the first target image, which is achieved by the following formulas (1)-(3):

[0080] Iterate through all pixel values ​​in the target area, comparing the R, G, and B channels of each pixel value sequentially, and then finding the maximum value among the red, green, and blue channels:

[0081]

[0082] In formula (4), I (x,y) I represents the pixel value of a point in the target region. R(x,y) I G(x,y) I B(x,y) These are the channel values ​​for the red, green, and blue channels, respectively.

[0083] Then, based on the ratio between the maximum dynamic range and the maximum values ​​of each color channel, determine the gain values ​​for the red channel, green channel, and blue channel:

[0084]

[0085] Finally, the pixels of the first image are adjusted based on the target white balance gain value to obtain the target pixel value:

[0086]

[0087] In this embodiment of the present disclosure, in response to the presence of a target region in the first image, and the deviation between the initial white balance gain value determined based on the pixel values ​​of the target region and the reference white balance gain value is greater than a preset threshold, the distribution information of the target region in the first image is determined; based on the distribution information, the pixel values ​​of the target region are adjusted, and based on the adjusted pixel values, a target white balance gain value is determined; the first image is adjusted based on the target white balance gain value to obtain the first target image.

[0088] In this embodiment of the disclosure, when the deviation between the initial white balance gain value and the reference white balance gain value is large, it is determined that the white balance algorithm misjudges the ambient color temperature, resulting in low accuracy of the determined initial white balance gain value. Then, by using the distribution information of the target area in the first image, the pixel value of the target area is adjusted to balance the color distribution in the first image, so that the white balance algorithm can accurately capture the relationship between the pixel value of the target area and the ambient color temperature, thereby reducing the situation where the white balance algorithm misjudges the ambient color temperature, and making the color temperature of the first target image closer to the ambient color temperature, thus improving the color reproduction of the first target image.

[0089] In some embodiments, the pixel values ​​of the target region are adjusted based on distribution information, and a target white balance gain value is determined based on the adjusted pixel values, including:

[0090] Based on distribution information, a first weight value for the pixel values ​​of the target region is determined; wherein, the first weight value is negatively correlated with the proportion of the target region in the first image;

[0091] The pixel values ​​of the target region are weighted based on the first weight value, and the target white balance gain value is determined based on the weighted pixel values.

[0092] Understandably, in order to improve the accuracy of adjusting the pixel values ​​of the target area, a first weight value can be determined based on the distribution information, and then the pixel values ​​of the target area can be weighted based on the first weight value to improve the accuracy of correcting the color of the first image, thereby balancing the color distribution in the first image, so that the white balance algorithm can accurately capture the relationship between the pixel values ​​of the target area and the ambient color temperature.

[0093] Here, the first weight value is negatively correlated with the proportion of the target region in the first image. When the proportion is larger, the first weight value is smaller, and when the proportion is smaller, the first weight value is larger.

[0094] In some embodiments, a second mapping relationship between the Miss Leading Correction (MLC) value and the proportion is established in advance. After determining the proportion of the target region in the first image, the correction weight value corresponding to the proportion is determined based on the proportion and the second mapping relationship. Then, a first weight value is obtained based on the correction weight value and a standard value. Here, the standard value is 1.

[0095] For example, the second mapping relationship can be shown in Table 1:

[0096] Table 1. Correspondence between the proportion K of the target region in the first image and the correction weight value.

[0097] The proportion K of the target region in the first image Correcting weight values 0≤K<0.3 0.1 0.3≤K<0.6 0.35 0.6≤K<0.9 0.65 0.9≤K<1 1

[0098] In some embodiments, taking a wood grain area as an example, when the proportion of the wood grain area in the first image is determined to be greater than a preset proportion threshold, the red gain value in the initial white balance gain value is determined to be smaller and the blue gain value to be larger, predicting that the color temperature of the first image is biased towards blue. At this point, the proportion of the trigger correction weight value can be increased to adjust the pixel values ​​of the wood grain area, so that based on the adjusted pixel values, the red gain value in the determined target white balance gain value is larger and the blue gain value is smaller, thereby correcting the color temperature of the first image and ensuring the color reproduction of the first image.

[0099] In this embodiment, a first weight value is first determined based on distribution information; then, the pixel values ​​of the target region are weighted based on the first weight value, and the target white balance gain value is determined based on the weighted pixel values. This improves the accuracy of adjusting the pixel values ​​of the target region, and consequently improves the accuracy of determining the target white balance gain value.

[0100] In some embodiments, the method further includes:

[0101] Based on the pixel values ​​of the target area, determine the coordinates of the pixels in the target area within the preset color space;

[0102] A second weight value for the pixel value of the target region is determined based on the relative position between the coordinate point and the gray block region in the preset color space; wherein, the second weight value is negatively correlated with the relative distance indicated by the relative position.

[0103] The pixel values ​​of the target region are weighted based on the first weight value, including:

[0104] The target weight value is determined based on the first weight value and the second weight value;

[0105] The pixel values ​​of the target region are weighted based on the target weight value.

[0106] Understandably, in automatic white balance processing, a second weight value can be determined first to adjust the pixel values ​​of the target area, thereby pre-correcting the color of the first image. Therefore, in this embodiment of the present disclosure, by converting different color channels of the pixel values ​​of the target area according to a preset conversion rule, a first conversion value and a second conversion value are obtained; and based on the first conversion value and the second conversion value, the coordinate points of the pixels in the target area in a preset color space are determined; and then based on the coordinate points, the second weight value is determined.

[0107] In some embodiments, the R / GB / G color space is a way of representing image colors using specific color ratios. This color space is created by calculating the ratio of the red channel to the green channel (R / G) and the ratio of the blue channel to the green channel (B / G), resulting in two ratio channels that form a new color space. This representation helps to highlight color deviations in the first image and makes color correction more intuitive and accurate. Therefore, the preset color space here can be the R / GB / G color space.

[0108] Here, the gray area is a specific region in a preset color space, representing the ideal distribution of gray colors. The gray area can be determined by a preset threshold or boundary.

[0109] Understandably, we can first determine the relative position between the coordinate point and the gray area in the preset color space. Based on the relative position, we can improve the accuracy of determining the second weight value, thereby improving the accuracy of color correction.

[0110] Here, the different coordinates of the pixels in the target area in the preset color space make the relative positions between the coordinates and the gray area different, thus the second weight value of the pixel value in the target area is different.

[0111] In some embodiments, the second weight value includes at least weight W1 and weight W2. Weight W1 is assigned by determining whether the coordinate point is located within the gray area; and when the coordinate point is located within the gray area, weight W2 is assigned by determining the distance of the coordinate point from the center of the gray area. For example, by calculating the distance from the coordinate point to the boundary of the gray area, if the distance is less than a first distance threshold, the coordinate point is determined to be within the gray area, and weight W1 is set to 1; if the distance is greater than or equal to the first distance threshold, the coordinate point is determined to be outside the gray area, and weight W1 is set to 0.1-0.5. In another example, by calculating the distance from the coordinate point to the center of the gray area boundary, if the distance is less than a second distance threshold, the coordinate point is determined to be close to the center, and weight W1 is set to 1; if the distance is greater than or equal to the second distance threshold, the coordinate point is determined to be close to the center, and weight W1 is set to 0.7-0.9.

[0112] In other embodiments, the color temperature region of the coordinate point can be determined based on its relative position to a gray area in a preset color space. Besides determining the second weight value based on the relative distance indicated by the relative position, the second weight value can also be determined based on the color temperature region of the coordinate point. For example, a larger second weight value can be assigned when the coordinate point is in a high color temperature region, and a smaller second weight value can be assigned when the coordinate point is in a low color temperature region.

[0113] It is understandable that after determining the second weight value, the target weight value can be determined based on the first and second weight values; and the pixel values ​​of the target area can be weighted based on the target weight value. In this way, compared with the method of weighting the pixel values ​​based on the second weight value and then weighting the pixel values ​​again based on the first weight value, this solution can reduce the number of weighting processes and reduce the power consumption of electronic devices.

[0114] Here, the target weight value is the product of the first weight value and the second weight value.

[0115] In some embodiments, when there are other regions in the first image besides the target region, the coordinates of the first pixels of the other regions in a preset color space are determined based on the first pixel values ​​of the other regions; and a second weight value of the first pixel value is determined based on the relative position between the coordinates of the first pixel and the gray block region in the preset color space; then, the first pixel value is weighted based on the determined second weight value to obtain a weighted first pixel value; finally, the first image after color correction is obtained based on the weighted first pixel value and the pixel value of the target region after weighting.

[0116] In this embodiment of the disclosure, the pixels of the target area are converted to a preset color space to obtain the coordinates of the pixels in the preset color space; and a second weight value of the pixel value is determined based on the relative position between the coordinates and the gray block area, thereby improving the accuracy of determining the second weight value; then a target weight value is determined based on the first weight value and the second weight value, and the pixel values ​​of the target area are weighted based on the target weight value, thereby improving the accuracy of the first image color correction and reducing the situation where the white balance algorithm misjudges the ambient color temperature.

[0117] In some embodiments, determining a first weight value for the pixel values ​​of the target region based on distribution information includes:

[0118] The first weight value is determined based on the proportion of the target region in the first image and the preset mapping relationship;

[0119] The preset mapping relationship represents the correspondence between the proportion and the first weight value.

[0120] Understandably, a pre-defined mapping relationship between the representation proportion and the first weight value can be established firstly so that after determining the proportion of the target region in the first image, the corresponding first weight value can be determined in a timely manner, thereby improving the accuracy and efficiency of adjusting the pixel values ​​of the target region.

[0121] In this embodiment of the disclosure, a preset mapping relationship is established in advance to represent the correspondence between the proportion and the first weight value. After determining the proportion of the target region in the first image, the first weight value can be accurately and quickly determined based on the preset mapping relationship and the proportion, thereby improving the accuracy of adjusting the pixel value of the target region.

[0122] In some embodiments, the method further includes:

[0123] The first image is input into the classification model to obtain the classification result of the first image; wherein, the classification model is used to identify whether there is a target region in the image;

[0124] In response to the classification result indicating the presence of a target region in the first image, an initial white balance gain value is determined based on the pixel values ​​of the target region.

[0125] It should be noted that, in order to improve the efficiency and accuracy of determining whether a target region exists in the first image, the first image can be input into a pre-trained classification model, which will then identify the first image and obtain the classification result.

[0126] In some embodiments, to improve the accuracy of the classification model in recognizing the first image, the first image can be preprocessed, such as denoising, image scaling, or image enhancement. For example, the denoising process is multi-scale bilateral filtering denoising. By filtering the image at different scales, noise of different sizes can be better handled while preserving image details. Specifically, the first image is first decomposed into pyramids to obtain images at different scales. Bilateral filtering is then performed on each scale image to remove noise. Smaller scale images are given higher weights to retain more details. Finally, a weighted average is calculated for each scale image according to the set weights to obtain the final denoised image.

[0127] It is understandable that when the classification result indicates that a target region exists in the first image, the initial white balance gain value can be determined based on the pixel value of the target region; while when the classification result indicates that a target region does not exist in the first image, it is not necessary to determine the initial white balance gain value based on the pixel value of the target region. This can reduce the situation of invalid determination of the initial white balance gain value, which is beneficial to reducing the power consumption of electronic devices.

[0128] In this embodiment, a first image is input into a classification model to obtain a classification result for the first image, improving the efficiency and accuracy of determining whether a target region exists in the first image. When the classification result indicates the presence of a target region in the first image, an initial white balance gain value is determined based on the pixel values ​​of the target region. On one hand, this reduces the likelihood of invalidally determining the initial white balance gain value, which helps reduce the power consumption of electronic devices. On the other hand, by directly using the classification result from the classification model, it reduces the likelihood of the white balance algorithm misjudging the ambient color temperature, eliminating the need to set automatic white balance parameters separately for each preset scene, thereby reducing the human and time resources consumed in adjusting automatic white balance parameters.

[0129] In some embodiments, the method further includes:

[0130] Obtain the sample image and its label. The label of the sample image is used to indicate whether the target region exists in the sample image.

[0131] The feature extraction network in the classification model is used to extract image features from the sample images.

[0132] By using the classification network in the classification model, image features are identified to obtain predicted labels;

[0133] Based on the difference between the predicted label and the label of the sample image, the network parameters of the classification network are adjusted until the preset convergence condition is met, and the classification model is obtained.

[0134] It is understandable that, given that the task of identifying the presence of a target region in an image is relatively simple, directly fine-tuning the model parameters of the classification model can lead to overfitting. That is, the classification model has high accuracy in recognizing images during training, but low accuracy in recognizing other images, and therefore lacks generalization ability.

[0135] Therefore, in this embodiment of the disclosure, during the training process, only the network parameters of the classification network are adjusted, while the network parameters of the feature extraction network remain unchanged, so as to achieve the effect of training only the classification network, thereby improving the generalization ability of the classification model.

[0136] Here, the feature extraction network can be a Convolutional Neural Network (CNN) or a Transformer model, etc.; the feature extraction network includes at least convolutional layers and pooling layers. The number of convolutional and pooling layers within the feature extraction layer can be arbitrarily set according to requirements, and this embodiment does not limit this.

[0137] The classification network may include a fully connected layer and a softmax layer, or it may include a fully connected layer and a classifier (Adaptive Boosting). This disclosure does not limit the scope of the network.

[0138] For example, Figure 3a This is a schematic diagram illustrating the structure of a classification model according to an exemplary embodiment, such as... Figure 3a As shown, the classification model 30 consists of an input module 31, a feature extraction network 32, and a classification network 33. The feature extraction network 32 includes 5 convolutional layers and 5 pooling layers, while the classification network 33 includes multiple fully connected layers of 3 layers each and a classifier.

[0139] Specifically, the sample image is input into the feature extraction network, which extracts the image features of the sample image. The image features output by the feature extraction network are then input into the classification network. The classification network is used to identify the image features and obtain the predicted label. Based on the difference between the predicted label and the label of the sample image, the network parameters of the classification network are adjusted until the preset convergence condition is met. Finally, based on the feature extraction network and the trained classification network, a classification model is obtained.

[0140] Here, the sample images can be images synthesized from a database or images acquired by electronic devices.

[0141] In some embodiments, a large number of sample images with a repetition rate below a threshold, along with the labels of each sample image, can be acquired to train the classification model. In addition, image enhancement processing can be performed on the sample images, such as random cropping, random rotation, or random flipping, which helps to improve the generalization ability of the classification model.

[0142] In some embodiments, a loss function (e.g., cross-entropy loss) is used to calculate the difference information between the predicted label and the label of the sample image. When the difference information is less than the difference threshold, it is determined that the number of iterations of training the classification network has reached a preset number, and further, it is determined that the convergence condition has been met.

[0143] For example, Figure 3b This is a flowchart illustrating a method for training a classification model according to an exemplary embodiment. The method for training the classification model includes at least the following steps:

[0144] In step 301, the sample image and the label of the sample image are obtained.

[0145] In some embodiments, the electronic device’s autofocus, auto exposure and auto white balance modes are enabled in advance, and an exchangeable image file format is set to store the acquired images. Then, a large number of sample images containing special scenes such as wood grain scenes and solid color scenes are acquired, and labels are set for each sample image.

[0146] In step 302, the sample images are preprocessed.

[0147] In some embodiments, the sample image is preprocessed, such as noise reduction, image scaling, or image enhancement.

[0148] In step 303, the feature extraction network in the classification model is used to extract the image features of the sample images.

[0149] In step 304, the classification network in the classification model is used to identify image features and obtain predicted labels.

[0150] In some embodiments, image features are progressively extracted through multiple convolutional and pooling layers, and then these features are mapped to different categories through fully connected layers. There are 13 convolutional layers, each using a 3x3 kernel, and each pooling layer using a 2x2 pooling layer. After the convolutional layers, three fully connected layers are connected, linking the IR ratio and CCT data from the flicker sensor with the image features extracted by the convolutional layers. Following the three fully connected layers, a classifier is connected to output the predicted label.

[0151] In step 305, based on the difference information between the predicted label and the label of the sample image, the network parameters of the classification network are adjusted until the preset convergence condition is met, and the classification model is obtained.

[0152] In this embodiment of the disclosure, during the training process of the classification model, the classification network in the classification model is trained only using the sample images and their labels. That is, the training process does not affect the feature extraction network, so that the feature extraction network does not learn whether the sample image is a target region. Therefore, the overfitting of the trained classification model is reduced, thereby ensuring the generalization ability of the classification model and improving the accuracy of using the classification model to identify whether an image has a target region.

[0153] In some embodiments, the method further includes:

[0154] Acquire at least two frames of the second image; wherein the second image is sequentially continuous with the first image;

[0155] The first image is input into the classification model to obtain the classification result of the first image, including:

[0156] Each second image and the first image are input into the classification model to obtain the classification results of each second image and the initial classification result of the first image.

[0157] Based on the classification results of each second image and the initial classification result of the first image, the classification result of the first image is determined.

[0158] It should be explained that, in order to improve the accuracy of the classification result of the first image, a fusion mechanism of the classification results of historical frame images can be used. After determining the classification results of each second image and the initial classification result of the first image, the classification result of the first image is then determined. This reduces the possibility of inaccurate classification results of the first image due to the instability of the classification model in recognizing the first image.

[0159] Here, the second image is sequentially continuous with the first image. This can be understood as follows: in an ordered image sequence, the first image follows the second image, and there are no jumps or missing other image frames between them, maintaining the coherence and smoothness between the images. Thus, based on the classification results of each second image and the initial classification result of the first image, the accuracy of determining the classification result of the first image can be improved.

[0160] It is understandable that after acquiring multiple frames of second images, each second image can be input into a classification model, which will then identify each second image and obtain the classification result for each second image, thereby improving the accuracy and efficiency of determining the classification result for each second image.

[0161] In some embodiments, the similarity between the classification results of each second image and the initial classification result of the first image is determined; then, based on each similarity, a total similarity is obtained; when the total similarity is greater than or equal to a similarity threshold, the reliability of determining the initial classification result of the first image is high, and the initial classification result of the first image is determined as the classification result of the first image; when the total similarity is less than a similarity threshold, the reliability of determining the initial classification result of the first image is low, and the classification result of the first image is determined based on the classification results of each second image.

[0162] In this embodiment, multiple frames of second images that are sequentially continuous with the first image are acquired, and each frame of the second image and the first image are input into a classification model to obtain the classification results of each second image and the initial classification result of the first image. Then, based on the classification results of each second image and the initial classification result of the first image, the classification result of the first image is determined. Thus, by fusing the classification results of historical frames, the error rate of the classification result of the first image is reduced, thereby reducing the possibility of invalid white balance gain values ​​being determined due to reliance on the classification result of the first image output by the classification model.

[0163] In some embodiments, the classification result of the first image is determined based on the classification results of each second image and the initial classification result of the first image, including:

[0164] Based on the acquisition time of the second image, a third weight value for the classification result of the second image is determined; wherein, the acquisition time is negatively correlated with the third weight value;

[0165] The classification result of the second image is weighted based on the third weight value to obtain the target classification result;

[0166] Based on the classification results of each target and the initial classification result of the first image, the classification result of the first image is determined.

[0167] In some embodiments, in time series data, data points closer to the current time are considered to reflect the current state or trend, while data points farther from the current time are more susceptible to unpredictable factors. In other words, data points closer to the current time are more important than data points farther from the current time.

[0168] Therefore, in this embodiment of the present disclosure, by obtaining the acquisition time of each frame of the second image, the correlation between the classification result of each second image and the classification result of the first image is determined; and based on the correlation, a third weight value of the classification result of the second image is determined; and then the classification result of the second image is weighted based on the third weight value to obtain the target classification result, thereby improving the accuracy of determining the classification result of the first image.

[0169] Here, the closer the acquisition time of the second image is to the acquisition time of the first image, the higher the correlation between the classification result of the second image and the classification result of the first image, and a larger third weight value is assigned; the farther the acquisition time of the second image is from the acquisition time of the first image, the lower the correlation between the classification result of the second image and the classification result of the first image, and a smaller third weight value is assigned.

[0170] For example, after acquiring three frames of the second image, a third weight value confi is assigned to each frame based on the acquisition time of each frame, i.e., confi1 > confi2 > confi3. Then, the classification results of the second images are weighted based on the third weight value to obtain the target classification result. Finally, based on each target classification result, the fusion classification result of the historical frame images is determined. Here, confi1 + confi2 + confi3 = 1.

[0171] In some embodiments, the initial classification result of the first image indicates that a target region exists in the first image, the classification result of the second image A indicates that a target region exists in the second image, the classification result of the second image B indicates that no target region exists in the second image, and the classification result of the second image C indicates that no target region exists in the second image. The third weight values ​​are set to 0.3, 0.6, and 0.9 respectively. The classification results of the second images are weighted based on the corresponding third weight values ​​to obtain the target classification results. Finally, based on each target classification result and the initial classification result of the first image, it is determined that no target region exists in the first image.

[0172] Taking the first image as a wood grain image as an example, after determining that there is a wood grain region in the first image based on the classification results of each target and the initial classification results of the first image, it is necessary to adjust the pixel value of the wood grain pixel. Figure 4a This is a schematic diagram illustrating a preset color space according to an exemplary embodiment, such as... Figure 4a As shown, the wood grain pixels are converted to a preset color space to obtain their coordinates. The relative positions of these coordinates with the gray area (Figure 400) are also obtained. The wood grain pixels in the gray area are mainly concentrated in the lower right (Figure 41), i.e., the low color temperature region. The white balance algorithm assigns a smaller second weight value to the wood grain pixels, resulting in a decrease in the red channel gain and an increase in the blue channel gain, causing the wood grain image to appear bluish and cool. By increasing the proportion of the trigger correction weight value, the wood grain pixels are readjusted based on their proportion. This results in a larger red gain and a smaller blue gain in the target white balance gain value determined based on the adjusted pixel values, thereby correcting the color temperature of the wood grain image and ensuring its color reproduction.

[0173] In this embodiment, a third weight value for the classification result of the second image is determined based on the acquisition time of the second image. Then, the classification result of the second image is weighted based on the third weight value to obtain the target classification result. Finally, the classification result of the first image is determined based on each target classification result and the initial classification result of the first image. This can improve the accuracy of determining the classification result of the first image and reduce the situation where the white balance gain value is invalidally determined due to reliance on the classification result of the first image output by the classification model.

[0174] In some embodiments, the method further includes:

[0175] In response to the absence of a target region in the first image, the first image is adjusted based on a reference white balance gain value to obtain a second target image; or,

[0176] In response to the deviation between the initial white balance gain value and the reference white balance gain value being less than or equal to a preset threshold, the first image is adjusted based on the initial white balance gain value to obtain the third target image.

[0177] It is understandable that when there is no target area in the first image, it is determined that there is a white or gray object in the first image, so that the accuracy of the ambient color temperature or color balance in the image determined by the white balance algorithm is high. It can adjust the first image based on the reference white balance gain value to obtain a second target image with normal image color temperature.

[0178] Here, the reference white balance gain value is obtained from the pixel values ​​corresponding to white or gray objects in the first image.

[0179] When the deviation between the initial white balance gain value and the reference white balance gain value is less than or equal to a preset threshold, the accuracy of the ambient color temperature or color balance in the image determined by the white balance algorithm is high, making the determined initial white balance gain value accurate. Based on the initial white balance gain value, the first image can be adjusted to obtain a third target image with normal color temperature.

[0180] In this embodiment of the disclosure, when there is no target area in the first image, or when the deviation between the initial white balance gain value determined based on the pixel value of the target area and the reference white balance gain value is less than or equal to a preset threshold, the white balance determination algorithm can accurately determine the ambient color temperature, thereby enabling the image color temperature of the second target image or the third target image to be closer to the ambient color temperature and improving the color reproduction of the second target image or the third target image.

[0181] Figure 4b This is a flowchart illustrating an image processing method according to an exemplary embodiment. Figure 2 ,like Figure 4b As shown, this image processing method mainly includes the following steps:

[0182] In step 401, each of the second images and the first image is input into the classification model.

[0183] In some embodiments, at least two frames of second images are acquired; wherein the second images are sequentially continuous with the first images; and each second image and the first image are respectively input into a classification model to obtain the classification result of each second image and the initial classification result of the first image;

[0184] In step 402, the classification result of the second image is weighted based on the third weight value to obtain the target classification result.

[0185] In some embodiments, a third weight value for the classification result of the second image is determined based on the acquisition time of the second image; wherein the acquisition time is negatively correlated with the third weight value; and the classification result of the second image is weighted based on the third weight value to obtain the target classification result.

[0186] In step 403, the initial classification result of the first image is determined.

[0187] In step 404, it is determined whether a target region exists in the first image.

[0188] In some embodiments, based on the classification results of each target and the initial classification result of the first image, the classification result of the first image is determined, and further based on the classification result of the first image, it is determined whether there is a target region in the first image.

[0189] In some embodiments, if it is determined that a target region exists in the first image, step 405 is performed.

[0190] In other embodiments, step 406 is performed if it is determined that no target region exists in the first image.

[0191] In step 405, a first weight value for the pixel values ​​of the target region is determined based on the proportion of the target region in the first image.

[0192] Here, the first weight value is negatively correlated with the proportion of the target region in the first image;

[0193] In step 406, the first image is adjusted based on the reference white balance gain value to obtain the second target image.

[0194] In some embodiments, when the target region is not present in the first image, it is determined that a white or gray object exists in the first image. This ensures that the accuracy of the ambient color temperature or color balance determined by the white balance algorithm is high, enabling the adjustment of the first image based on a reference white balance gain value to obtain a second target image with a normal color temperature. Here, the reference white balance gain value is obtained from the pixel values ​​corresponding to the white or gray objects in the first image.

[0195] In step 407, a target weight value is determined based on the first weight value and the second weight value, and the pixel values ​​of the target region are weighted based on the target weight value.

[0196] In some embodiments, the coordinates of the pixels in the target region in a preset color space are determined based on the pixel values ​​of the target region; a second weight value of the pixel values ​​in the target region is determined based on the relative position between the coordinates and the gray block region in the preset color space; wherein the second weight value is negatively correlated with the relative distance indicated by the relative position.

[0197] In step 408, the target white balance gain value is determined based on the weighted pixel values.

[0198] In step 409, the first image is adjusted based on the target white balance gain value to obtain the first target image.

[0199] When a target area is present in the first image, the white balance algorithm may misjudge the ambient color temperature, resulting in low accuracy of the initial white balance gain value determined based on the pixel values ​​of the target area. By adjusting the pixel values ​​of the target area according to its proportion in the first image, the color distribution in the first image can be balanced. This allows the white balance algorithm to accurately capture the relationship between the pixel values ​​of the target area and the ambient color temperature, thereby reducing the possibility of misjudging the ambient color temperature and making the color temperature of the first target image closer to the ambient color temperature, thus improving the color reproduction of the first target image.

[0200] For example, taking a camera shooting an indoor wooden door scene as an example, the white balance processing method proposed in this embodiment is compared with the white balance processing method in related technologies, specifically as follows:

[0201] The wooden door is a warm-toned wood color. Figure 4c This is a schematic image illustrating an exemplary embodiment. Figure 1 ,like Figure 4c As shown, in the current shooting scene, there is a white wall 41 and a wooden door 42. After acquiring the first image, the white balance algorithm in the relevant technology can determine the current ambient color temperature based on the white wall to ensure the imaging effect of the first image.

[0202] Figure 4dThis is a schematic image illustrating an exemplary embodiment. Figure 2 , Figure 4e This is a schematic diagram of an image according to an exemplary embodiment, such as... Figures 4d-4e As shown, when the camera lens moves closer to the wooden door, the white walls on both sides of the door appear smaller and smaller in the lens as the camera moves, causing the wooden door 42 to occupy the entire frame. At this point, the white balance algorithm in the relevant technology misjudges the ambient color temperature, resulting in a bluish color temperature in the first image.

[0203] Figure 4f This is a schematic diagram of an image according to an exemplary embodiment. Figure 4g This is a schematic image illustrating an exemplary embodiment. Figure 5 ,like Figure 4f-4g As shown, the white balance processing method proposed in this embodiment determines the proportion of the wooden door in the first image and adjusts the pixel value of the wooden door to balance the color distribution in the first image. This enables the white balance algorithm to accurately capture the relationship between the pixel value of the wooden door and the ambient color temperature, thereby reducing the possibility of the white balance algorithm misjudging the ambient color temperature and making the color temperature of the first image closer to the ambient color temperature.

[0204] Figure 5 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment, such as... Figure 5 As shown, the image processing apparatus 500 includes:

[0205] The first determining module 501 is configured to determine the distribution information of the target region in the first image in response to the presence of a target region in the first image and the deviation between the initial white balance gain value determined based on the pixel value of the target region and the reference white balance gain value being greater than a preset threshold.

[0206] The execution module 502 is configured to adjust the pixel values ​​of the target region based on the distribution information, and determine the target white balance gain value based on the adjusted pixel values.

[0207] The first acquisition module 503 is configured to adjust the first image based on the target white balance gain value to obtain the first target image.

[0208] In some embodiments, the execution module 502 is specifically configured as follows:

[0209] Based on the distribution information, a first weight value for the pixel values ​​of the target region is determined; wherein the first weight value is negatively correlated with the proportion of the target region in the first image.

[0210] The pixel values ​​of the target region are weighted based on the first weight value, and the target white balance gain value is determined based on the weighted pixel values.

[0211] In some embodiments, the device 500 further includes:

[0212] The second determining module is configured to determine the coordinates of the pixels in the target region in a preset color space based on the pixel values ​​of the target region;

[0213] The third determining module is configured to determine a second weight value of the pixel value of the target region based on the relative position between the coordinate point and the gray block region in the preset color space; wherein the second weight value is negatively correlated with the relative distance indicated by the relative position;

[0214] The execution module 502 is further configured as follows:

[0215] Based on the first weight value and the second weight value, determine the target weight value;

[0216] The pixel values ​​of the target region are weighted based on the target weight value.

[0217] In some embodiments, the execution module 502 is further configured to:

[0218] The first weight value is determined based on the proportion of the target region in the first image and the preset mapping relationship;

[0219] The preset mapping relationship represents the correspondence between the proportion and the first weight value.

[0220] In some embodiments, the device 500 further includes:

[0221] The recognition module is configured to input the first image into a classification model to obtain a classification result for the first image; wherein the classification model is used to identify whether the target region exists in the image;

[0222] The fourth determining module is configured to determine the initial white balance gain value based on the pixel value of the target region in response to the classification result indicating that the target region exists in the first image.

[0223] In some embodiments, the device 500 further includes:

[0224] The training module is configured to acquire sample images and labels for the sample images, wherein the labels of the sample images are used to indicate whether the target region exists in the sample images;

[0225] The feature extraction network in the classification model is used to extract the image features of the sample image;

[0226] The image features are identified using the classification network in the classification model to obtain predicted labels;

[0227] Based on the difference information between the predicted label and the label of the sample image, the network parameters of the classification network are adjusted until a preset convergence condition is reached to obtain the classification model.

[0228] In some embodiments, the device 500 further includes:

[0229] The second acquisition module is configured to acquire at least two frames of the second image; wherein the second image is sequentially continuous with the first image.

[0230] The identification module is specifically configured as follows:

[0231] Each of the second images and the first image is input into the classification model to obtain the classification result of each of the second images and the initial classification result of the first image;

[0232] The classification result of the first image is determined based on the classification results of each of the second images and the initial classification result of the first image.

[0233] In some embodiments, the identification module is further configured to:

[0234] Based on the acquisition time of the second image, a third weight value for the classification result of the second image is determined; wherein, the acquisition time is negatively correlated with the third weight value;

[0235] The classification result of the second image is weighted based on the third weight value to obtain the target classification result;

[0236] Based on the classification results of each target and the initial classification result of the first image, the classification result of the first image is determined.

[0237] In some embodiments, the method apparatus further includes:

[0238] The third acquisition module is configured to adjust the first image based on the reference white balance gain value to obtain a second target image in response to the absence of a target region in the first image; or,

[0239] The fourth acquisition module is configured to adjust the first image based on the initial white balance gain value to obtain the third target image in response to the deviation between the initial white balance gain value and the reference white balance gain value being less than or equal to the preset threshold.

[0240] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0241] Figure 6 This is a structural block diagram illustrating an electronic device 600 according to an exemplary embodiment. For example, the electronic device 600 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0242] Reference Figure 6 The electronic device 600 may include one or more of the following components: processing component 602, memory 604, power supply component 606, multimedia component 608, audio component 610, input / output (I / O) interface 612, sensor component 614, and communication component 616.

[0243] Processing component 602 typically controls the overall operation of electronic device 600, such as operations associated with at least one of display, telephone call, data communication, camera operation, and recording operation. Processing component 602 may include one or more processors 620 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 602 may include one or more modules to facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602.

[0244] Memory 604 is configured to store various types of data to support the operation of electronic device 600. Examples of such data include at least one of the following: instructions for any application or method operating on electronic device 600, contact data, phonebook data, messages, pictures, and videos. Memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0245] Power supply component 606 provides power to various components of electronic device 600. Power supply component 606 may include at least one of the following: a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 600.

[0246] Multimedia component 608 includes a screen that provides an output interface between electronic device 600 and user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen may be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 608 includes a front-facing camera and / or a rear-facing camera. When electronic device 600 is in an operating mode, such as a shooting mode or video mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0247] Audio component 610 is configured to output and / or input audio signals. For example, audio component 610 includes a microphone (MIC) configured to receive external audio signals when electronic device 600 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 604 or transmitted via communication component 616. In some embodiments, audio component 610 also includes a speaker for outputting audio signals.

[0248] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, and buttons. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0249] Sensor assembly 614 includes one or more sensors for providing state assessments of various aspects of electronic device 600. For example, sensor assembly 614 may detect the on / off state of electronic device 600, the relative positioning of components such as the display and keypad of electronic device 600, changes in position of electronic device 600 or one of its components, the presence or absence of user contact with electronic device 600, orientation or acceleration / deceleration of electronic device 600, and temperature changes of electronic device 600. Sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 614 may also include an optical sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, sensor assembly 614 may also include, but is not limited to, at least one of the following: an accelerometer, a gyroscope, a magnetometer, a pressure sensor, and a temperature sensor.

[0250] Communication component 616 is configured to facilitate wired or wireless communication between electronic device 600 and other devices. Electronic device 600 can access wireless networks based on communication standards, such as Wi-Fi, 4G, 5G, or combinations thereof. In one exemplary embodiment, communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 616 also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wide Band (UWB), Bluetooth (BT), and other technologies.

[0251] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components.

[0252] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including executable instructions or a computer program, which can be executed by a processor 620 of an electronic device 600 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0253] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform any of the image processing methods described above in the embodiments of this disclosure. For example, the image processing method includes:

[0254] In response to the presence of a target region in the first image, and the deviation between the initial white balance gain value determined based on the pixel value of the target region and the reference white balance gain value being greater than a preset threshold, the distribution information of the target region in the first image is determined;

[0255] Based on the distribution information, the pixel values ​​of the target area are adjusted, and the target white balance gain value is determined based on the adjusted pixel values.

[0256] The first target image is obtained by adjusting the first image based on the target white balance gain value.

[0257] This disclosure provides a computer program product comprising a computer program or executable instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer program or executable instructions from the computer-readable storage medium and executes the computer program or executable instructions, causing the computer device to perform any of the image processing methods described above in this disclosure.

[0258] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0259] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that, The method includes: In response to the presence of a target region in the first image, and the deviation between the initial white balance gain value determined based on the pixel value of the target region and the reference white balance gain value being greater than a preset threshold, the distribution information of the target region in the first image is determined; Based on the distribution information, the pixel values ​​of the target region are adjusted, and based on the adjusted pixel values, the target white balance gain value is determined. The first image is adjusted based on the target white balance gain value to obtain the first target image.

2. The method according to claim 1, characterized in that, The step of adjusting the pixel values ​​of the target region based on the distribution information, and determining the target white balance gain value based on the adjusted pixel values, includes: Based on the distribution information, a first weight value for the pixel values ​​of the target region is determined; wherein the first weight value is negatively correlated with the proportion of the target region in the first image. The pixel values ​​of the target region are weighted based on the first weight value, and the target white balance gain value is determined based on the weighted pixel values.

3. The method according to claim 2, characterized in that, The method further includes: Based on the pixel values ​​of the target region, determine the coordinates of the pixels in the target region in a preset color space; Based on the relative position between the coordinate point and the gray block region in the preset color space, a second weight value for the pixel value of the target region is determined; wherein, the second weight value is negatively correlated with the relative distance indicated by the relative position; The weighting process of the pixel values ​​of the target region based on the first weight value includes: Based on the first weight value and the second weight value, determine the target weight value; The pixel values ​​of the target region are weighted based on the target weight value.

4. The method according to claim 2, characterized in that, The step of determining a first weight value for the pixel values ​​of the target region based on the distribution information includes: The first weight value of the pixel value of the target region is determined based on the proportion of the target region in the first image. The first weight value is determined based on the proportion of the target region in the first image and the preset mapping relationship; The preset mapping relationship represents the correspondence between the proportion and the first weight value.

5. The method according to claim 1, characterized in that, The method further includes: The first image is input into a classification model to obtain a classification result for the first image; wherein, the classification model is used to identify whether the target region exists in the image; In response to the classification result indicating the presence of the target region in the first image, the initial white balance gain value is determined based on the pixel values ​​of the target region.

6. The method according to claim 5, characterized in that, The method further includes: Obtain a sample image and its label, wherein the label indicates whether the target region exists in the sample image; The feature extraction network in the classification model is used to extract the image features of the sample image; The image features are identified using the classification network in the classification model to obtain predicted labels; Based on the difference information between the predicted label and the label of the sample image, the network parameters of the classification network are adjusted until a preset convergence condition is reached to obtain the classification model.

7. The method according to claim 5, characterized in that, The method further includes: Acquire at least two frames of the second image; wherein the second image is sequentially continuous with the first image; The step of inputting the first image into the classification model to obtain the classification result of the first image includes: Each of the second images and the first image is input into the classification model to obtain the classification result of each of the second images and the initial classification result of the first image; The classification result of the first image is determined based on the classification results of each of the second images and the initial classification result of the first image.

8. The method according to claim 7, characterized in that, The step of determining the classification result of the first image based on the classification results of each of the second images and the initial classification result of the first image includes: Based on the acquisition time of the second image, a third weight value for the classification result of the second image is determined; wherein, the acquisition time is negatively correlated with the third weight value; The classification result of the second image is weighted based on the third weight value to obtain the target classification result; Based on the classification results of each target and the initial classification result of the first image, the classification result of the first image is determined.

9. The method according to any one of claims 1 to 8, characterized in that, The method further includes: In response to the absence of a target region in the first image, the first image is adjusted based on the reference white balance gain value to obtain a second target image; or, In response to the deviation between the initial white balance gain value and the reference white balance gain value being less than or equal to the preset threshold, the first image is adjusted based on the initial white balance gain value to obtain the third target image.

10. An image processing apparatus, characterized in that, The device includes: The first determining module is configured to determine the distribution information of the target region in the first image in response to the presence of a target region in the first image and the deviation between the initial white balance gain value determined based on the pixel value of the target region and the reference white balance gain value being greater than a preset threshold. The execution module is configured to adjust the pixel values ​​of the target region based on the distribution information, and determine the target white balance gain value based on the adjusted pixel values; The first acquisition module is configured to adjust the first image based on the target white balance gain value to obtain the first target image.

11. An electronic device, characterized in that, include: processor; Memory used to store computer programs or instructions; The processor executes a computer program or instructions to implement the steps of the method of any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium storing a computer program or instructions, characterized in that, When a computer program or instruction in a storage medium is executed by a processor, it implements the steps of the method of any one of claims 1 to 9.

13. A computer program product, comprising a computer program or instructions, characterized in that, When a computer program or instruction is executed by a processor, it implements the steps of the method of any one of claims 1 to 9.

Images