Image processing method, electronic device, and storage medium

Abstract

This application provides an image processing method, electronic device, and storage medium, relating to the field of image processing. The method includes: launching a camera application in response to a first operation; displaying a first interface, the first interface including a preview area; determining the shooting scene corresponding to the image to be processed; determining a high-confidence white point corresponding to the image to be processed based on the shooting scene, the high-confidence white point being used to characterize the true light source color of the shooting environment corresponding to the image to be processed; performing white balance processing on the image to be processed using the high-confidence white point to obtain a preview image, and displaying the preview image in the preview area. This method can determine the high-confidence white point corresponding to the image to be processed under different shooting scenes, and perform white balance processing on the image to be processed based on the high-confidence white point, which can effectively improve the white balance effect, enabling the preview image obtained after white balance processing to accurately restore the true colors of the image, and significantly improving the overall image quality.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly to an image processing method, electronic device, and storage medium. Background Technology

[0002] With the widespread use of electronic devices with camera functions in daily life, taking pictures with these devices has become a common practice. To ensure the accuracy of image colors, electronic devices perform white balance processing on the image during the process of capturing and displaying it on a screen, in order to reproduce the true colors of the image.

[0003] Currently, image white balance processing generally relies on a defined white point in the image. However, the defined white point in traditional methods is not accurate enough, resulting in poor white balance processing of the image, failing to accurately reproduce the true colors of the image, and affecting the overall image quality. Summary of the Invention

[0004] This application provides an image processing method, electronic device, and storage medium that can detect the shooting scene of the image to be processed and determine the high-confidence white points corresponding to the image to be processed under different shooting scenes. Based on the high-confidence white points, white balance processing is performed on the image to be processed, which can effectively improve the white balance effect, so that the preview image obtained after white balance processing accurately restores the true color of the image and significantly improves the overall image quality.

[0005] In a first aspect, this application provides an image processing method applied to an electronic device, the electronic device including a camera. The image processing method includes: responding to a first operation to launch a camera application; displaying a first interface, the first interface including a preview area; determining a shooting scene corresponding to an image to be processed; the image to be processed is an image captured in real time by the camera; the shooting scene includes any one of a solid color scene, a patchwork color scene, a distracting color scene, a black and white gray scene, and a color-rich scene; determining a high-confidence white point corresponding to the image to be processed based on the shooting scene; performing white balance processing on the image to be processed using the high-confidence white point to obtain a preview image, and displaying the preview image in the preview area.

[0006] Among them, the high-confidence white point is used to characterize the true light source color of the shooting environment corresponding to the image to be processed.

[0007] Optionally, the black-and-white gray scenes in the embodiments of this application may include a rich set of black-and-white gray scenes.

[0008] In this implementation, the shooting scene of the image to be processed can be detected, and the high-confidence white points corresponding to the image to be processed under different shooting scenes can be determined. Then, based on the high-confidence white points, white balance processing is performed on the image to be processed to obtain a preview image. Compared with related technologies that all use the same method to determine white points, the image processing method provided in this application takes into account the influence of the shooting scene when determining the white points of the image to be processed. For different shooting scenes (such as solid color scenes, spliced ​​color scenes, interference color scenes, black and white gray scenes, and color-rich scenes), the method of determining high-confidence white points is different, so that the white points corresponding to the image to be processed are highly confident, effective, and accurate.

[0009] For example, in colorful scenes, the white point of the image to be processed in that scene is directly used as the high-confidence white point. For other scenes (such as solid color scenes), historical high-confidence white points stored in a preset queue are used as the high-confidence white points of the image to be processed. Then, white balance processing is performed on the image to be processed based on these high-confidence white points. This greatly improves the white balance effect, effectively corrects color deviations under different lighting conditions, and ensures that the colors in the preview image presented to the user are as close as possible to the colors of the actual objects. In other words, it more accurately reproduces the true colors of the image, improves the overall image quality, and provides a better user experience.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, determining the shooting scene corresponding to the image to be processed includes: determining the statistical information of the image to be processed; the statistical information includes multiple statistical points, which are used to indicate multiple processed pixel blocks corresponding to the image to be processed; determining the feature information of the image to be processed based on the statistical information; the feature information includes the number of statistical points located in the gray area, and / or the two-dimensional distribution pattern of multiple statistical points; and determining the shooting scene corresponding to the image to be processed based on the feature information.

[0011] Alternatively, a pixel block refers to a region consisting of multiple pixels; that is, a pixel block can include multiple pixels.

[0012] In this implementation, feature information is determined based on the statistical information of the image to be processed, so that the feature information can accurately reflect the characteristics of the image to be processed under different shooting scenarios. Based on this, the shooting scenario corresponding to the image to be processed can be accurately identified, which makes it easier to accurately identify high-confidence white points in the image to be processed based on the shooting scenario. Then, based on the high-confidence white points, the image to be processed can be processed more accurately for white balance, which greatly improves the white balance processing effect and effectively restores the true color of the image.

[0013] In conjunction with the first aspect, in some implementations of the first aspect, determining the statistical information of the image to be processed includes: performing correction processing on the image to be processed to obtain a corrected image; performing block processing on the corrected image to obtain multiple pixel blocks; and determining the multiple statistical points based on the multiple pixel blocks.

[0014] Optionally, the correction process may include black level correction (BLC) and / or lens shading correction (LSC) processes.

[0015] It should be understood that the more pixel blocks there are, the more statistical points are obtained, and the more accurate the subsequent statistical and feature information is. Correspondingly, the more accurate the shooting scene of the image to be processed is determined based on the feature information. The fewer pixel blocks there are, the less resources the electronic device consumes to process the image, and the faster the processing speed is.

[0016] In this implementation, the image to be processed is corrected to improve its quality. Then, statistical points are determined based on the corrected image, which ensures that the two-dimensional and three-dimensional distribution patterns of the image to be processed can be accurately represented using the statistical points.

[0017] In conjunction with the first aspect, in some implementations of the first aspect, the feature information of the image to be processed is determined based on statistical information, including: determining the RGB values ​​corresponding to multiple statistical points respectively; determining a first ratio and a second ratio based on the RGB value of each statistical point; counting the number of statistical points located in the gray area based on the first ratio and the second ratio; and / or drawing the two-dimensional distribution pattern of multiple statistical points.

[0018] The first ratio represents the ratio of the R channel value to the G channel value for each statistical point, and the second ratio represents the ratio of the B channel value to the G channel value for each statistical point.

[0019] Optionally, in the embodiments of this application, the first ratio is denoted as R / G, and the second ratio is denoted as B / G.

[0020] Optionally, the two-dimensional distribution pattern may include a dispersed distribution pattern, a linear distribution pattern, or a clustered distribution pattern.

[0021] In this implementation, feature information (i.e., the number of statistical points located in the gray area and / or the two-dimensional distribution of multiple statistical points) is determined based on the R / G and B / G of the statistical points. This feature information can accurately reflect the characteristics of the image to be processed under different shooting scenarios. Based on this, the shooting scenario corresponding to the image to be processed can be accurately identified. This makes it easier to accurately identify high-confidence white points in the image to be processed based on the shooting scenario. Then, based on the high-confidence white points, more accurate white balance processing is performed on the image to be processed, which greatly improves the white balance processing effect and effectively restores the true color of the image.

[0022] In conjunction with the first aspect, in some implementations of the first aspect, the shooting scene corresponding to the image to be processed is determined based on feature information, including: when the number of statistical points located in the gray area is less than a preset number and / or the two-dimensional distribution pattern is a linear distribution pattern, the shooting scene corresponding to the image to be processed is determined to be a solid color scene.

[0023] In this implementation, when the distribution characteristics of multiple statistical points in the two-dimensional coordinate system are detected and meet the distribution characteristics of multiple statistical points in a solid color scene, the shooting scene corresponding to the image to be processed is determined to be a solid color scene. This improves the accuracy of identifying the shooting scene of the image to be processed and facilitates the accurate determination of high-confidence white points based on the solid color scene.

[0024] In conjunction with the first aspect, in some implementations of the first aspect, the shooting scene corresponding to the image to be processed is determined based on feature information, including: when the number of statistical points located in the gray area is less than a preset number and / or the two-dimensional distribution pattern is a linear distribution pattern, and there are gray points in the image to be processed, the shooting scene corresponding to the image to be processed is determined to be an interference color scene.

[0025] In this implementation, when multiple statistical points are detected to satisfy the distribution characteristics of multiple statistical points under the shooting scene as a interference color scene, and the image to be processed satisfies the image characteristics under the shooting scene as a interference color scene, the shooting scene corresponding to the image to be processed is determined to be a interference color scene. This improves the accuracy of identifying the shooting scene of the image to be processed and facilitates the accurate determination of high-confidence white points based on the interference color scene.

[0026] In conjunction with the first aspect, in some implementations of the first aspect, the shooting scene corresponding to the image to be processed is determined based on feature information, including: if the two-dimensional distribution pattern is a clustered distribution pattern, the three-dimensional distribution pattern corresponding to the image to be processed is determined based on multiple statistical points; if the three-dimensional distribution pattern is a clustered distribution pattern, the shooting scene corresponding to the image to be processed is determined to be a solid color scene.

[0027] In this implementation, when multiple statistical points only satisfy the two-dimensional distribution pattern as a clustered distribution pattern, it is further determined whether the three-dimensional distribution pattern of multiple statistical points satisfies the three-dimensional distribution characteristics under a solid color scene. This can avoid confusing solid color scenes with black, white and gray scenes and improve the accuracy of identifying the shooting scene of the image to be processed.

[0028] In conjunction with the first aspect, in some implementations of the first aspect, the shooting scene corresponding to the image to be processed is determined based on feature information, including: if the two-dimensional distribution pattern is a clustered distribution pattern, the three-dimensional distribution pattern is a clustered distribution pattern, and there are gray points in the image to be processed, the shooting scene corresponding to the image to be processed is determined to be an interference color scene.

[0029] This implementation avoids confusing solid color scenes with black, white, and gray scenes, thus improving the accuracy of identifying the shooting scene of the image to be processed.

[0030] In conjunction with the first aspect, in some implementations of the first aspect, after determining the three-dimensional distribution pattern corresponding to the image to be processed based on multiple statistical points, the image processing method provided in this application embodiment may further include: if the three-dimensional distribution pattern is a dispersed distribution pattern, and the dispersed distribution pattern belongs to a block-dispersed pattern, determining that the shooting scene corresponding to the image to be processed is a spliced ​​color scene; or, if the three-dimensional distribution pattern is a dispersed distribution pattern, and the dispersed distribution pattern belongs to a random dispersed pattern, determining that the shooting scene corresponding to the image to be processed is a black-and-white gray scene.

[0031] This implementation effectively avoids confusing spliced ​​color scenes with black, white, and gray scenes, improving the accuracy of identifying the shooting scene of the image to be processed.

[0032] In conjunction with the first aspect, in some implementations of the first aspect, the shooting scene corresponding to the image to be processed is determined based on feature information, including: if the two-dimensional distribution pattern is a dispersed distribution pattern and the three-dimensional distribution pattern is a random dispersed pattern, the shooting scene corresponding to the image to be processed is determined to be a color-rich scene.

[0033] In this implementation, when multiple statistical points are detected to satisfy the distribution characteristics of statistical points in a color-rich scene, the shooting scene corresponding to the image to be processed is determined to be a color-rich scene. This improves the accuracy of identifying the shooting scene of the image to be processed and facilitates the accurate determination of high-confidence white points based on the color-rich scene.

[0034] In conjunction with the first aspect, in some implementations of the first aspect, determining the high-confidence white point corresponding to the image to be processed based on the shooting scene includes: when the shooting scene corresponding to the image to be processed is any one of a color-rich scene, a distracting color scene, and a black-and-white scene, determining the white point corresponding to the image to be processed; determining the white point as the high-confidence white point corresponding to the image to be processed; and storing the high-confidence white point corresponding to the image to be processed in a preset queue.

[0035] In this embodiment of the application, the high-confidence white points stored in the preset queue can be called historical high-confidence white points, which facilitates the use of historical high-confidence white points for white balance processing of images to be processed in other shooting scenarios (such as solid color scenes), thereby improving the white balance effect of images to be processed in other shooting scenarios (such as solid color scenes).

[0036] In conjunction with the first aspect, in some implementations of the first aspect, the high-confidence white point corresponding to the image to be processed is determined according to the shooting scene, including: when the shooting scene corresponding to the image to be processed is a solid color scene or a spliced ​​color scene, the historical high-confidence white point corresponding to the color-rich scene is obtained from the preset queue; and the historical high-confidence white point is determined as the high-confidence white point corresponding to the image to be processed.

[0037] In this implementation, a preset queue stores high-confidence white points from images captured before they are taken in solid-color or multi-color scenes. With the shooting light source unchanged, these high-confidence white points are used to perform white balance processing on the images to be processed in either solid-color or multi-color scenes. Since the high-confidence white points in the preset queue are determined from images in colorful scenes, these white points are accurate, effective, and have low interference. White balance processing based on these white points can greatly restore the true colors of the image, ensuring that the color of the photographed object displayed on the screen matches the color of the photographed object in the real world. This significantly improves the image white balance processing effect and enhances image quality.

[0038] In conjunction with the first aspect, in some implementations of the first aspect, the image processing method provided in this application embodiment further includes: determining the first correlated color temperature corresponding to the image to be processed based on the high confidence white point corresponding to the image to be processed; and performing color correction processing on the image to be processed based on the first correlated color temperature to obtain a color-corrected image.

[0039] In this implementation, the correlated color temperature is adjusted according to the high-confidence white point to make the determined correlated color temperature more accurate, thereby making the colors of the image after color correction based on the correlated color temperature more accurate and natural.

[0040] In conjunction with the first aspect, in some implementations of the first aspect, the image processing method provided in this application further includes: determining the second correlated color temperature corresponding to the adjacent images of the image to be processed; the acquisition time of the adjacent images is before the acquisition time of the image to be processed; if the difference between the first correlated color temperature and the second correlated color temperature is detected to be greater than a preset difference, the high confidence white point corresponding to the image to be processed is not used to perform white balance processing on the image to be processed.

[0041] In this implementation, by calculating the difference in correlated color temperature between consecutive frames, the reliability of the correlated color temperature of the current image to be processed can be determined. This allows for a decision on whether to use the high-confidence white point of the image to be processed for white balance processing. This avoids the problem of storing a high-confidence white point under light source A and then using the high-confidence white point stored under light source A under light source B.

[0042] Secondly, this application provides an apparatus included in an electronic device, which has the function of implementing the behaviors of the electronic device in the first aspect and possible implementations thereof. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above-described functions. For example, a receiving module or unit, a processing module or unit, etc.

[0043] Thirdly, this application provides an electronic device, which includes a processor, a memory, and an interface; the processor, memory, and interface cooperate with each other to enable the electronic device to execute any one of the methods in the first aspect of the technical solution.

[0044] Fourthly, this application provides a chip system including a processor. The processor is used to read and execute a computer program stored in a memory to perform the methods in the first aspect and any possible implementation thereof.

[0045] Optionally, the chip system may also include memory, which is connected to the processor via circuitry or wires.

[0046] Alternatively, the chip system may also include a communication interface.

[0047] Fifthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform any one of the methods in the first aspect of the technical solution.

[0048] Sixthly, this application provides a computer program product, which includes computer program code that, when executed on an electronic device, causes the electronic device to perform any one of the methods in the first aspect of the technical solution.

[0049] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of a scene for capturing an image, illustrating an exemplary embodiment of this application.

[0051] Figure 2 A schematic diagram of a scene for capturing another image, as illustrated in an exemplary embodiment of this application;

[0052] Figure 3 This is a schematic diagram of the structure of an electronic device illustrated in an exemplary embodiment of this application;

[0053] Figure 4 This is a software structure block diagram of an electronic device illustrated in an exemplary embodiment of this application;

[0054] Figure 5 This is a schematic flowchart of an image processing method shown in an exemplary embodiment of this application;

[0055] Figure 6 This is a schematic flowchart illustrating a method for determining a shooting scene according to an exemplary embodiment of this application;

[0056] Figure 7 This is a schematic diagram of a gray area shown in an exemplary embodiment of this application;

[0057] Figure 8 This is a schematic flowchart illustrating a method for determining multiple statistical points according to an exemplary embodiment of this application;

[0058] Figure 9 This is a schematic flowchart illustrating a method for determining feature information according to an exemplary embodiment of this application;

[0059] Figure 10 This is a schematic diagram illustrating the distribution pattern of statistical points in a solid color scene according to an exemplary embodiment of this application;

[0060] Figure 11 This is a schematic diagram illustrating another distribution pattern of statistical points in a solid color scene, as shown in an exemplary embodiment of this application;

[0061] Figure 12 This is a schematic diagram illustrating another distribution pattern of statistical points in a solid color scene, as shown in an exemplary embodiment of this application;

[0062] Figure 13 This is a schematic diagram illustrating another distribution pattern of statistical points in a solid color scene, as shown in an exemplary embodiment of this application;

[0063] Figure 14This is a schematic diagram illustrating another distribution pattern of statistical points in a solid color scene, as shown in an exemplary embodiment of this application;

[0064] Figure 15 This is a schematic diagram illustrating another distribution pattern of statistical points in a solid color scene, as shown in an exemplary embodiment of this application;

[0065] Figure 16 This is a schematic flowchart illustrating a method for determining high-confidence white points, as shown in an exemplary embodiment of this application.

[0066] Figure 17 This is a schematic diagram illustrating a light source switching method according to an exemplary embodiment of this application;

[0067] Figure 18 This is a schematic diagram of the structure of an image processing apparatus shown in an exemplary embodiment of this application;

[0068] Figure 19 This is a schematic diagram of the structure of a chip provided in an embodiment of this application. Detailed Implementation

[0069] In the embodiments of this application, the terms "first", "second", "third" and "fourth" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0070] To facilitate understanding of the technical solutions in the embodiments of this application, some terms used in the embodiments of this application will be explained before introducing the technical solutions in the examples of this application.

[0071] 1. White spots

[0072] The white dot is used to represent the color of the light source in the shooting environment corresponding to the image.

[0073] 2. White Balance

[0074] White balance refers to adjusting the color settings of electronic devices to ensure that white objects photographed under different light sources appear pure white, thereby accurately reproducing the true colors of other objects.

[0075] 3. Correlated Color Temperature (CCT)

[0076] CCT is a standard for measuring the color of a light source. It is determined by comparing the color of light emitted by a light source with the color of light emitted by a blackbody at different temperatures. For example, if the color of light emitted by a light source matches the color of light emitted by a blackbody at a certain temperature, the temperature of the blackbody at that time is expressed as the color temperature of the light source.

[0077] CCT is an absolute temperature, expressed in Kelvin (K).

[0078] 4. RGB (Red, Green, Blue) color space

[0079] The RGB color space, also known as the RGB domain, refers to a color model related to the structure of the human visual system. Based on the structure of the human eye, all colors are treated as different combinations of red, green, and blue.

[0080] 5. RGB (Red, Green, Blue) channel values

[0081] RGB channel values, also known as RGB values, refer to the intensity values ​​of the three color channels (red, green, and blue) used to define a specific color in the RGB color model. These intensity values ​​together determine the final displayed color.

[0082] 6. YUV color space

[0083] YUV color space refers to a color encoding method, where Y represents luminance (or luma), and U and V represent chroma (or chroma). RGB color space focuses on how the human eye perceives color, while YUV color space focuses on the visual sensitivity to luminance. RGB and YUV color spaces can be converted to each other.

[0084] 7. Brightness

[0085] Luminance is the luminous flux emitted by a light source per unit area per unit solid angle in a given direction. The symbol for luminance is L, and the unit is nit.

[0086] The above is a brief introduction to the terms used in the embodiments of this application, and will not be repeated below.

[0087] When an electronic device captures an image, it performs white balance processing on the image during the process of displaying the image on the screen. This process corrects color deviations under different lighting conditions, ensuring that the colors in the image presented to the user are as close as possible to the colors of the actual objects, thus restoring the true colors of the image.

[0088] Currently, image white balance processing is typically performed using the Automatic White Balance (AWB) algorithm. As one of the most basic and commonly used AWB algorithms, the gray-world method is based on the assumption that in a color-rich (or color-varying) image, the average values ​​of its corresponding RGB color channels should tend towards the same gray value. This assumption is based on the premise that light distribution in the natural environment is usually uniform, and that the various color channels in an image should have similar brightness levels overall.

[0089] Therefore, in practical applications, the AWB algorithm is typically used to determine statistical points in an image, and a weighted average is applied to these statistical points to obtain white points. White balance processing is then performed on the image based on these white points. Statistical points can be understood as a set of sample points in the image used to calculate white balance parameters.

[0090] However, the method for determining the white point described above is only suitable for images with rich colors, or images captured in scenes with rich colors. For images captured in scenes with solid colors, patchwork colors, or interfering colors, the average values ​​of the RGB color channels do not converge to the same grayscale value. Therefore, the white point determined using the traditional method is inaccurate or ineffective for these images. If the white point determined using the traditional method is still used for white balance processing, it will interfere with the white balance results, failing to effectively correct color deviations under different lighting conditions. This makes it impossible to ensure that the colors presented to the user are as close as possible to the actual colors of the objects, thus failing to reproduce the true colors of the image, severely impacting the overall image quality and providing a poor user experience.

[0091] In view of this, embodiments of this application provide an image processing method applied to an electronic device including a camera. The method includes: responding to a first operation, launching a camera application; displaying a first interface, the first interface including a preview area; determining a shooting scene corresponding to an image to be processed; the shooting scene includes any one of a solid color scene, a patchwork color scene, a distracting color scene, a black-and-white gray scene, and a color-rich scene; determining a high-confidence white point corresponding to the image to be processed based on the shooting scene, the high-confidence white point being used to characterize the true light source color of the shooting environment corresponding to the image to be processed; performing white balance processing on the image to be processed using the high-confidence white point to obtain a preview image, and displaying the preview image in the preview area.

[0092] In this implementation, the shooting scene of the image to be processed can be detected, and the high-confidence white points corresponding to the image to be processed under different shooting scenes can be determined. Then, based on the high-confidence white points, white balance processing is performed on the image to be processed to obtain a preview image. Compared with related technologies that all use the same method to determine white points, the image processing method provided in this application takes into account the influence of the shooting scene when determining the white points of the image to be processed. For different shooting scenes (such as solid color scenes, spliced ​​color scenes, interference color scenes, black and white gray scenes, and color-rich scenes), the method of determining high-confidence white points is different, so that the white points corresponding to the image to be processed are highly confident, effective, and accurate.

[0093] For example, in colorful scenes, the white point of the image to be processed in that scene is directly used as the high-confidence white point. For other scenes (such as solid color scenes), historical high-confidence white points stored in a preset queue are used as the high-confidence white points of the image to be processed. Then, white balance processing is performed on the image to be processed based on these high-confidence white points. This greatly improves the white balance effect, effectively corrects color deviations under different lighting conditions, and ensures that the colors in the preview image presented to the user are as close as possible to the colors of the actual objects. In other words, it more accurately reproduces the true colors of the image, improves the overall image quality, and provides a better user experience.

[0094] Before introducing the image processing method provided in the embodiments of this application, the application scenarios of the image processing method provided in the embodiments of this application will be illustrated with reference to the accompanying drawings.

[0095] Application Scenario 1: Scenarios for capturing images

[0096] The image processing method provided in this application can be applied to the scene where images are captured. Please refer to [link / reference]. Figure 1 , Figure 1 This is a schematic diagram of a scene for capturing an image, illustrating an exemplary embodiment of this application.

[0097] In this embodiment, the electronic device 100 is described as a mobile phone. Exemplarily, the mobile phone's display screen shows the main interface, such as... Figure 1 The main interface UI1 is shown. UI1 includes icons for several applications, such as the "Video" application icon, the "Music" application icon, and the "Camera" application icon. When a user wants to take a picture, they can tap the "Camera" application icon. The phone responds to this tap, runs the camera application, and simultaneously, the phone's display switches from the main interface UI1 to the shooting interface UI2.

[0098] like Figure 1The shooting interface UI2 shown includes a preview area 102 and a shooting control 103. The preview area 102 is used to display a preview image; the shooting control 103 is used to instruct the phone to take an image when a shooting operation is received (such as clicking the shooting control 103).

[0099] For example, if the light source in the shooting environment where the subject 101 is located is sunlight, and the user wants to take a picture of the subject 101, they point their phone's camera at the subject 101. The camera captures the image to be processed from the subject 101, the phone performs white balance processing on the image, and displays the white balance processed preview image in the preview area 102. Because the relevant technology does not consider the shooting scene corresponding to the subject 101 and uses a traditional method to determine the white point, the determined white point is inaccurate. That is, the white point cannot accurately represent the true light source color of the shooting environment where the subject 101 is located, resulting in poor white balance processing effects based on this white point. Figure 1 As shown, the color of the subject 101 presented in the preview image displayed in preview area 102 is inconsistent with the color of the subject 101 in the real world.

[0100] The image processing method provided in this application determines the shooting scene corresponding to the image to be processed, and determines a precise and effective white point, i.e., a high-confidence white point, corresponding to the image to be processed based on the shooting scene. This high-confidence white point can accurately represent the true light source color of the shooting environment in which the subject 101 is located. White balance processing based on this high-confidence white point can greatly restore the true color of the image, so that the color of the subject 101 presented in the preview image displayed in the final preview area is consistent with the color of the subject 101 in the real world.

[0101] Please see Figure 2 , Figure 2 This is a schematic diagram illustrating another scene of image capture, as shown in an exemplary embodiment of this application. It is worth noting that... Figure 2 The scene depicted in the captured image uses the image processing method provided in this application embodiment. The preview image displayed in the preview area 104 of the shooting interface UI3 is a preview image processed by the image processing method provided in this application embodiment. Obviously, the color of the shooting object 101 presented in the preview image displayed in the preview area 104 is consistent with the color of the shooting object 101 in the real world, greatly restoring the true color of the shooting object 101 and providing users with a better shooting experience.

[0102] Optionally, when the user is satisfied with the effect of the preview image, they can click the shooting control 103. The mobile phone responds to the user's click on the shooting control 103, captures the preview image displayed in the current preview area 104, processes it, saves it as a photo, and stores it in the mobile phone's gallery.

[0103] Application Scenario 2: Video Call Scenarios

[0104] The image processing method provided in this application can also be applied to video call scenarios. For example, when a user makes a video call using an electronic device, regardless of whether a front-facing or rear-facing camera is used, images are captured in real time and displayed on the device's screen. During this process, the image processing method provided in this application can determine the precise and effective white point corresponding to the real-time captured image, i.e., the high-confidence white point. White balance processing of the real-time captured image based on this high-confidence white point can greatly restore the true colors of the image, ensuring that the color of the photographed object displayed on the screen is consistent with the color of the photographed object in the real world. This significantly improves the image white balance processing effect, enhances image quality, and provides users with a better video call experience.

[0105] Application Scenario 3: Live Video Streaming

[0106] The image processing method provided in this application can also be applied to live video streaming scenarios. For example, when a user uses an electronic device for live video streaming, whether using a front-facing camera or a rear-facing camera, images are captured in real time and displayed on the electronic device's screen. Similarly, in this process, the image processing method provided in this application can determine the precise and effective white point corresponding to the real-time captured image, i.e., the high-confidence white point. White balance processing of the real-time captured image based on this high-confidence white point can greatly restore the true colors of the image, ensuring that the color of the live-streamed object displayed on the screen is consistent with the color of the live-streamed object in the real world. This significantly improves the image white balance processing effect, enhances image quality, and provides users with a better live video streaming experience.

[0107] It should be understood that the application scenarios of the above image processing methods are merely illustrative and do not limit the actual application scenarios of this application. The image processing methods provided in the embodiments of this application can be applied to any scenario that requires white balance processing of images, such as, but not limited to, the following scenarios: white balance processing of images in video conferencing applications, white balance processing of images in long and short video applications, white balance processing of images in intelligent camera movement applications, and white balance processing of images in other applications, etc.

[0108] The structure of the electronic devices involved in the embodiments of this application will be briefly described below with reference to the accompanying drawings.

[0109] In the embodiments of this application, the aforementioned electronic device may also be referred to as a terminal, terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), etc.

[0110] The electronic device can be a mobile phone, smart screen, tablet computer, computer with wireless transceiver function, wearable device, in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), handheld or laptop device, media player, smart projector, smart TV, desktop computer, Internet of Things (IoT) device, in-vehicle infotainment system, wireless terminal in industrial control, wireless terminal in self-driving, wireless terminal in remote medical surgery, wireless terminal in smart city, wireless terminal in smart home, etc., and is a device with image display function. The embodiments of this application do not limit the specific type and form of the electronic device.

[0111] Please see Figure 3 , Figure 3This is a schematic diagram of the structure of an electronic device illustrated in an exemplary embodiment of this application. The electronic device 200 may include a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a headphone jack 270D, a sensor module 280, buttons 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a subscriber identification module (SIM) card interface 295, etc. The sensor module 280 may include a pressure sensor 280A, a gyroscope sensor 280B, a barometric pressure sensor 280C, a magnetic sensor 280D, an accelerometer sensor 280E, a distance sensor 280F, a proximity sensor 280G, a fingerprint sensor 280H, a temperature sensor 280J, a touch sensor 280K, an ambient light sensor 280L, a bone conduction sensor 280M, etc.

[0112] It should be noted that, Figure 3 The structure shown does not constitute a specific limitation on the electronic device 200. In other embodiments of this application, the electronic device 200 may include more than Figure 3 The components shown may include more or fewer components, or the electronic device 200 may include... Figure 3 The components shown may be a combination of certain components, or the electronic device 200 may include... Figure 3 Sub-components of some of the components shown. Figure 3 The components shown can be implemented using hardware, software, or a combination of hardware and software.

[0113] Processor 210 may include one or more processing units. For example, processor 210 may include at least one of the following processing units: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and neural network processing unit (NPU). These different processing units may be independent devices or integrated devices.

[0114] Optionally, the processor 210 may run the software code of the image processing method provided in the embodiments of this application, and perform the following steps in response to the first operation: launching a camera application; displaying a first interface, the first interface including a preview area; determining the shooting scene corresponding to the image to be processed; the shooting scene includes any one of a solid color scene, a spliced ​​color scene, a distracting color scene, a black and white gray scene, and a color-rich scene; determining a high-confidence white point corresponding to the image to be processed based on the shooting scene, the high-confidence white point being used to characterize the true light source color of the shooting environment corresponding to the image to be processed; performing white balance processing on the image to be processed using the high-confidence white point to obtain a preview image, and displaying the preview image in the preview area.

[0115] It is understood that the interface connection relationships between the modules illustrated in this embodiment are merely illustrative and do not constitute a structural limitation on the electronic device 200. In other embodiments, the electronic device 200 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0116] Electronic device 200 can implement display functions through a GPU, a display screen 294, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 294 and the application processor. The GPU can also be used to perform mathematical and pose calculations, and for graphics rendering, etc. Processor 210 may include one or more GPUs, whose executed program instructions can generate or modify display information.

[0117] Display screen 294 can be used to display preview images, videos, etc. For example, after the GPU receives the TE signal, it sends the next frame preview image to display screen 294, and display screen 294 displays the next frame preview image.

[0118] Optionally, the display screen 294 may also display various operable controls provided by the electronic device 200 for the user, such as clickable buttons / options and slidable sliders.

[0119] The display screen 294 in this embodiment is a touch screen. The display screen 294 may integrate a touch sensor 280K. The touch sensor 280K can also be referred to as a "touch panel." That is, the display screen 294 may include a display panel and a touch panel; the touch sensor 280K and the display screen 294 together form a touch screen, also called a "touchscreen." The touch sensor 280K is used to detect touch operations applied to or near it. After the touch sensor 280K detects a touch operation, it can be passed to the upper layer by the kernel layer driver (such as the TP driver) to determine the touch event type, thereby realizing different functions.

[0120] For example, the electronic device 200 can implement the shooting function through an ISP, a camera 293, a video codec, a GPU, a display screen 294, and an application processor.

[0121] For example, the ISP is used to process data fed back by the camera 293. For instance, when taking a picture, the shutter is opened, and light is transmitted through the camera to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits this electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can perform algorithmic optimization of the image's noise, brightness, and color. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be integrated into the camera 293.

[0122] For example, camera 293 (also referred to as a lens) is used to capture still images or videos. It can be activated via application commands to enable photo-taking functionality, such as capturing images of any scene. The camera may include components such as an imaging lens, a filter, and an image sensor. Light emitted or reflected by an object enters the imaging lens, passes through the filter, and is ultimately converged onto the image sensor. The imaging lens is primarily used to converge and image the light emitted or reflected by all objects within the shooting field of view (which can be referred to as the object to be photographed, the target object, or the object the user intends to photograph); the filter is primarily used to filter out excess light waves (such as infrared light waves other than visible light) from the light.

[0123] Image sensors can be either charge-coupled devices (CCDs) or complementary metal-oxide-semiconductor (CMOS) phototransistors. The primary function of an image sensor is to convert received light signals into electrical signals via photoelectric conversion. These electrical signals are then transmitted to an image sensor (ISP) for conversion into digital image signals. The ISP outputs the digital image signals to a digital signal processing unit (DSP). The DSP then converts the digital image signals into standard image signals in formats such as RGB and YUV.

[0124] For example, a digital signal processor is used to process digital signals, including digital image signals and other digital signals. For instance, when the electronic device 200 selects a frequency point, the digital signal processor performs a Fourier transform on the frequency energy, etc.

[0125] For example, a video codec is used to compress or decompress digital video. Electronic device 200 may support one or more video codecs. Thus, electronic device 200 can play or record video in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, and MPEG 4.

[0126] For example, the gyroscope sensor 280B can be used to determine the motion posture of the electronic device 200. In some embodiments, the gyroscope sensor 280B can determine the angular velocity of the electronic device 200 around three axes (i.e., the X-axis, Y-axis, and Z-axis). The gyroscope sensor 280B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 280B detects the angle of the shake of the electronic device 200, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the electronic device 200 by moving in the opposite direction, thus achieving image stabilization. The gyroscope sensor 280B can also be used in scenarios such as navigation and motion-sensing games.

[0127] For example, the ambient light sensor 280L is used to sense the ambient light intensity. The electronic device 200 can adaptively adjust the brightness of the display screen 294 according to the sensed ambient light intensity. The ambient light sensor 280L can also be used to automatically adjust the white balance when taking a picture, or to adjust the white balance using the image processing method provided in the embodiments of this application. The ambient light sensor 280L can also cooperate with the proximity sensor 280G to detect whether the electronic device 200 is in a pocket to prevent accidental touch.

[0128] The charging management module 240 receives charging input from the charger. The power management module 241 connects to the battery 242, and the charging management module 240 connects to the processor 210. The wireless communication function of the electronic device 200 can be implemented through devices such as antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modem processor, and baseband processor.

[0129] The mobile communication module 250 can provide wireless communication solutions, including 2G / 3G / 2G / 5G / 6G, for use on electronic devices 200.

[0130] Similar to the mobile communication module 250, the wireless communication module 260 can also provide a wireless communication solution for use on the electronic device 200, such as at least one of the following: wireless local area networks (WLAN), Bluetooth (BT), Bluetooth Low Energy (BLE), ultra-wideband (UWB), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technology.

[0131] The external storage interface 220 can be used to connect an external memory card, such as a secure digital (SD) card, to expand the storage capacity of the electronic device 200. The external memory card communicates with the processor 210 through the external storage interface 220 to perform data storage functions. For example, music, images, and video files can be stored on the external memory card.

[0132] Internal memory 221 can be used to store computer executable program code, which includes instructions. Internal memory 221 may include a program storage area and a data storage area.

[0133] The fingerprint sensor 280H is used to collect fingerprints. The electronic device 200 can use the collected fingerprint characteristics to perform functions such as unlocking, accessing application locks, taking photos, and answering calls.

[0134] Buttons 290 include a power button and volume buttons. Buttons 290 can be mechanical buttons or touch buttons. Electronic device 200 can receive button input signals and implement functions related to the button input signals, such as taking a picture.

[0135] In addition, various types of operating systems run on these components. Examples include Android, iOS, Symbian, BlackBerry, Linux, and Windows. This is merely illustrative and not intended to be limiting. Different applications can be installed and run on these operating systems.

[0136] The image processing method provided in this application embodiment can be implemented in an electronic device 200 having the above-described hardware structure.

[0137] The structure of the electronic device 200 involved in the embodiments of this application has been briefly described above. The software structure involved in the embodiments of this application will be briefly described below. Please refer to... Figure 4 , Figure 4 This is a software structure block diagram of an electronic device illustrated in an exemplary embodiment of this application. For example... Figure 4 As shown, electronic devices can adopt a layered architecture, dividing the software into several layers, each with a clear role and division of labor, and communicating with each other through software interfaces.

[0138] In some embodiments, taking the electronic device 100 as an example of an Android system, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the system library, and the kernel layer.

[0139] The application layer can include a series of application packages. For example... Figure 4 As shown, the application package may include applications such as camera, calendar, maps, wireless local area networks (WLAN), music, text messaging, gallery, calling, navigation, Bluetooth, video, and live streaming.

[0140] The application framework layer provides an Application Programming Interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions. For example... Figure 4 As shown, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0141] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0142] Content providers are used to store and retrieve data and make that data accessible to applications. This data can include videos, any images, audio, phone calls made and received, browsing history and bookmarks, phone books, etc.

[0143] A view system can include visual controls, such as controls for displaying text and controls for displaying images. A view system can be used to build the display interface of an application, which can consist of one or more views, such as a view for displaying the application icon, a view for displaying text, and a view for displaying images.

[0144] The phone manager is used to provide communication functions for electronic devices 200, such as managing call status (including connection, hang-up, etc.).

[0145] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, etc.

[0146] The notification manager allows applications to display notification information in the status bar. It can be used to convey informational messages and can disappear automatically after a short time without user interaction.

[0147] The system library can include multiple functional modules, such as: surface manager, media libraries, 3D graphics processing libraries (such as OpenGL ES), 2D graphics engines (such as SGL), etc.

[0148] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0149] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0150] 3D graphics processing libraries are used to implement 3D graphics drawing, image rendering, compositing, and layer processing. 2D graphics engines are drawing engines for 2D graphics.

[0151] The kernel layer is the layer between hardware and software. The kernel layer may include the touch panel driver, which is used to collect touch events generated after the user touches the touch panel of the electronic device.

[0152] The kernel layer may also include a display driver, which can be used to drive the display screen to show different preview images, videos, etc.

[0153] The kernel layer can also include camera drivers, audio drivers, sensor drivers, etc.

[0154] The software structure involved in the embodiments of this application has been briefly introduced above. The image processing method provided by the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0155] Please see Figure 5 , Figure 5 This is a schematic flowchart illustrating an image processing method according to an exemplary embodiment of this application. The method includes:

[0156] S101, In response to the first operation, launch the camera application.

[0157] The first action is used to instruct the camera application to be launched, or to instruct the camera application to be run.

[0158] For example, the first operation may include a click operation. For instance, a user can click the "Camera" application icon on the main interface, and the electronic device responds to this click by running the camera application, subsequently capturing images in real time. Another example is when the electronic device is locked, a user can double-click the volume down button, and the electronic device responds to this double-click by running the camera application. Yet another example is when the electronic device is locked, and the lock screen includes a camera application icon; the user clicks the camera application icon, and the electronic device responds to this click by running the camera application. Alternatively, when the electronic device is running another application with permission to access the camera application, the user clicks the corresponding control in that other application, and the electronic device responds to this click by running the camera application. Or, if the electronic device is running an instant messaging application, the user can click the camera control within the instant messaging application, and the electronic device responds to this click by running the camera application, and so on.

[0159] It should be understood that the above is merely an example of the operation of running a camera application. In the embodiments of this application, the first operation may also include voice operation, gesture operation, or other operations that instruct the electronic device to run a camera application. This application does not impose any limitations on this.

[0160] S102. Display the first interface, which includes a preview area.

[0161] After the camera application runs, the electronic device displays a first interface on the screen. This first interface may include shooting controls, setting options, shooting mode options, a thumbnail display area, and a preview area. The shooting controls are used to indicate whether to capture an image, GIF, or video; the setting options are used to indicate settings for shooting parameters such as photo aspect ratio, video resolution, guide lines, level, and timer shooting; the shooting mode options are used to indicate the selection of different shooting modes; the thumbnail display area is used to display thumbnail images; and the preview area is used to display preview images.

[0162] Understandably, upon launching the camera app, the initial screen may display shooting controls, settings options, shooting mode options, and thumbnail display area. This is because the camera is initializing and preparing to capture image data, so the preview area may not yet display an image, or it may display a black background. In that instant, after the electronic device's camera completes initialization, it begins capturing the image to be processed, determines the shooting scene corresponding to the image, identifies the high-confidence white point corresponding to the image based on the shooting scene, performs white balance processing on the preview image using the high-confidence white point, obtains the preview image, and displays the preview image in the preview area. Since the entire process is completed very quickly—in the blink of an eye for the user—it does not affect the user's shooting experience.

[0163] S103. Determine the shooting scene corresponding to the image to be processed.

[0164] The image to be processed is the image captured by the camera in real time, which can also be understood as the raw image captured by the camera in real time. For example, when an electronic device captures an image, the light signal is transmitted through the lens to the camera's image sensor, so that the image of the object to be captured is focused onto the camera's image sensor. The image sensor converts the light signal into an electrical signal, and the electronic device processes the electrical signal to convert it into the image to be processed.

[0165] For example, the image to be processed can be a Raw image. Since Raw images are typically stored in Bayer format, in this embodiment, the Bayer format can be converted to RGB format according to actual needs.

[0166] Optionally, the image to be processed can be stored in the electronic device in matrix form, array form, or other storage form.

[0167] The shooting scene is used to characterize the shooting environment and background in which the subject is photographed, as well as the color representation of the shooting environment and background. Shooting scenes can include any of the following: solid color scenes, multi-colored scenes, distracting color scenes, black-and-white / gray scenes, and richly colored scenes.

[0168] This can be understood as the acquired image to be processed containing features of the shooting scene. For example, the image to be processed may contain any one of the following: solid color, spliced ​​color, interfering color, black, white, gray, or multiple colors.

[0169] Among them, a solid color scene refers to a shooting environment with a large area of ​​a single color, with little or no interference from other colors; the image to be processed corresponding to a solid color scene also has a large area of ​​a single color, with little or no interference from other colors.

[0170] A color-stitched scene refers to a shooting environment containing different pure color areas, with boundaries between each area; the image to be processed in a color-stitched scene also contains different pure color blocks, with boundaries between each block.

[0171] A scene with interfering colors refers to a shooting environment in which there are at least two color regions, and one of the color regions interferes with the other color regions; the image to be processed in the scene with interfering colors also contains at least two color regions, and one of the color regions interferes with the other color regions.

[0172] Optionally, the interfering color scene may also include a large-area interfering color scene. A large-area interfering color scene refers to a shooting environment with at least two color regions, one of which occupies a large area and interferes with other color regions. The image to be processed corresponding to a large-area interfering color scene also contains at least two color regions, one of which occupies a large area and interferes with other color regions.

[0173] A black-and-white scene refers to a shooting environment where the main tones are black, white, and gray; the corresponding image to be processed in a black-and-white scene also mainly consists of black, white, and gray tones.

[0174] Optionally, a black-and-white gray scene can also include a rich black-and-white gray scene, which refers to a shooting environment that contains black, white and various shades of gray; the image to be processed corresponding to a rich black-and-white gray scene also contains black, white and various shades of gray.

[0175] A color-rich scene refers to a shooting environment that includes a variety of vivid, high-contrast colors, which can display rich color levels; the image to be processed corresponding to a color-rich scene also contains a variety of vivid, high-contrast colors.

[0176] For example, the image to be processed can be input into a pre-trained shooting scene recognition model for processing. The shooting scene recognition model extracts image features of the image to be processed, such as color features, texture features, shape features, brightness features, etc., matches the image features with known shooting scene categories, and outputs the shooting scene corresponding to the image to be processed.

[0177] The scene recognition model is trained using machine learning on a sample training set. The sample training set can include multiple sample images to be processed, as well as the corresponding scene label for each sample image.

[0178] S104. Determine the high-confidence white point corresponding to the image to be processed based on the shooting scene.

[0179] A high-confidence white point refers to a pixel that is identified as a valid, highly reliable white or near-white pixel during the white balance processing of an image to be processed. In this embodiment, the high-confidence white point is used to characterize the true light source color of the shooting environment corresponding to the preview image, and is also used to determine the white balance of the image to be processed. That is, the high-confidence white point is subsequently used to adjust the white balance of the image to be processed to ensure color accuracy.

[0180] Understandably, when the light source remains unchanged, the methods for determining high-confidence white points differ depending on the shooting scenario. The light source refers to the light source used to illuminate the subject, which can include natural light (such as sunlight) and artificial light (such as flash, incandescent lamps, and LED lights).

[0181] For example, during an indoor photoshoot using an incandescent light source, if the user points the electronic device's camera at colorful books, the scene captured by the camera will be a colorful scene; conversely, if the user points the camera at solid-colored curtains, the scene captured by the camera will be a solid-color scene. In this embodiment, the method for determining the high-confidence white point in the image to be processed differs depending on whether the scene is colorful or solid-color.

[0182] For example, a mapping relationship can be pre-established between different shooting scenarios and different high-confidence white points, or a mapping relationship can be pre-established between different shooting scenarios and the methods for determining high-confidence white points. Once the shooting scenario corresponding to the image to be processed is determined, the high-confidence white points corresponding to that shooting scenario or the methods for determining high-confidence white points corresponding to that shooting scenario can be found according to the mapping relationship.

[0183] Of these two implementation methods, the former can directly find the high-confidence white points corresponding to the image to be processed in the shooting scene by using the mapping relationship, thus improving the efficiency of determining high-confidence white points. The latter does not need to store too much data to cover all possible shooting scenes, but relies on a dynamic determination process, making it more flexible to adapt to various changing shooting scenes.

[0184] S105. Perform white balance processing on the image to be processed using high-confidence white points to obtain a preview image, and display the preview image in the preview area.

[0185] For example, the gain factors of the image to be processed in the R, G, and B channels are calculated using high-confidence white points. That is, the gain values ​​of the pixels in the image to be processed in the R channel, the G channel, and the B channel. The gain factors are used to adjust the brightness of the R, G, and B channels in the image to be processed. A gain matrix is ​​constructed based on the gain factors of the R, G, and B channels. The matrix formed by the R channel values, G channel values, and B channel values ​​of each pixel in the image to be processed is multiplied by the gain matrix to adjust the R channel values, G channel values, and B channel values ​​of each pixel in the image to be processed.

[0186] In one example, the image after the white balance processing described above is the preview image, which is displayed in the preview area of ​​the first interface for easy viewing by the user. When the user is satisfied with the preview image, they can click the shooting control in the first interface. The electronic device responds to the user's click, captures the preview image displayed in the current preview area, processes it, saves it as a photo, and stores it in the electronic device's gallery.

[0187] In another example, the electronic device can use a Color Correction Matrix (CCM) to perform color restoration processing on the white-balanced image, thereby converting the white-balanced RAW image from the RAW domain to the RGB color space. The electronic device can also process the RGB color space image in the YUV domain to obtain a YUV domain image, and then perform Joint Photographic Experts Group (JPEG) processing on the YUV domain image to obtain a preview image. The preview image is then displayed in the preview area of ​​the first interface for easy viewing by the user.

[0188] In this implementation, the shooting scene of the image to be processed can be detected, and the high-confidence white points corresponding to the image under different shooting scenes can be determined. Then, white balance processing is performed on the image to be processed based on the high-confidence white points to obtain a preview image. Compared with related technologies that use the same method to determine white points, the image processing method provided in this application takes into account the influence of the shooting scene when determining the white points of the image to be processed, so that the determined white points corresponding to the image to be processed have high confidence, strong effectiveness, and high accuracy. Therefore, white balance processing of the image to be processed based on the high-confidence white points can greatly improve the white balance effect, effectively correct color deviations under different lighting conditions, and thus effectively ensure that the colors in the preview image presented to the user are as close as possible to the colors of the actual objects, that is, to a higher degree of restoration of the true colors of the image, improve the overall image quality, and bring a better visual experience to the user.

[0189] Optionally, in one possible implementation, step S103 may include steps S1031 to S1033, see [link to relevant documentation]. Figure 6 , Figure 6 This is a schematic flowchart illustrating a method for determining a shooting scene according to an exemplary embodiment of this application, as detailed below.

[0190] S1031. Determine the statistical information of the image to be processed.

[0191] The statistical information may include multiple statistical points, which indicate multiple processed pixel blocks corresponding to the image to be processed. A pixel block refers to a region composed of multiple pixels; that is, a pixel block may include multiple pixels.

[0192] In one example, multiple pixel blocks corresponding to the image to be processed are identified; based on the multiple pixel blocks, multiple statistical points are determined.

[0193] In another example, the image to be processed is corrected to obtain a corrected image; multiple pixel blocks corresponding to the corrected image are determined; and multiple statistical points are determined based on the multiple pixel blocks.

[0194] The correction process may include Black Level Correction (BLC) and / or Lens Shading Correction (LSC).

[0195] Black level correction is used to correct the black level output of an image sensor. Black level refers to the video signal level on a calibrated display device where no line of light is output. The reason for performing black level correction is twofold: firstly, due to manufacturing processes and environmental factors, dark current may exist in the image sensor, leading to voltage output from pixels even in the absence of light; secondly, the image sensor's analog-to-digital conversion accuracy is insufficient, and black level correction can overcome this problem.

[0196] Lens Shading Correction (LSC) refers to the correction of uneven image brightness caused by lens characteristics. LSC eliminates inconsistencies in color and brightness around the edges of the image compared to the center, resulting in a more uniform brightness distribution across the entire image.

[0197] S1032. Based on statistical information, determine the feature information of the image to be processed.

[0198] The feature information includes the number of statistical points located in the gray area, and / or the two-dimensional distribution pattern of multiple statistical points.

[0199] For example, obtain the camera module information (such as model, manufacturer information, etc.) corresponding to the camera, and determine the gray area based on the camera module information. For instance, calibrate the camera using a standard color chart or a known color temperature light source, establish a color temperature curve for the camera module based on the calibration data, and determine the range of the gray area based on the color temperature curve.

[0200] It should be understood that different camera module information corresponds to different gray areas. Therefore, for the same electronic device, if the camera module information is determined, its corresponding gray area is also fixed.

[0201] Please see Figure 7 , Figure 7 This is a schematic diagram of a gray area shown in an exemplary embodiment of this application. For example... Figure 7 As shown, the area enclosed by the lines is the gray area, and the points distributed within the gray area are statistical points. The distribution pattern of multiple statistical points within the gray area is called the two-dimensional distribution pattern.

[0202] S1033. Based on the feature information, determine the shooting scene corresponding to the image to be processed.

[0203] For example, the feature information may include the number of statistical points located within the gray area, and / or the two-dimensional distribution pattern of multiple statistical points. The two-dimensional distribution pattern of the multiple statistical points may include a clustered distribution pattern or a dispersed distribution pattern. It should be understood that different feature information corresponds to different shooting scenarios of the image to be processed.

[0204] For example, a mapping relationship between different feature information and different shooting scenes can be established in advance. Once the feature information of the image to be processed is determined, the shooting scene corresponding to that feature information can be found according to the mapping relationship, thus obtaining the shooting scene corresponding to the image to be processed.

[0205] In this implementation, feature information is determined based on the statistical information of the image to be processed, so that the feature information can accurately reflect the characteristics of the image to be processed under different shooting scenarios. Based on this, the shooting scenario corresponding to the image to be processed can be accurately identified, which makes it easier to accurately identify high-confidence white points in the image to be processed based on the shooting scenario. Then, based on the high-confidence white points, the image to be processed can be processed more accurately for white balance, which greatly improves the white balance processing effect and effectively restores the true color of the image.

[0206] Optionally, in one possible implementation, step S1031 may include steps S10311 to S10313, see [link to relevant documentation]. Figure 8 , Figure 8 This is a schematic flowchart illustrating a method for determining multiple statistical points according to an exemplary embodiment of this application, as detailed below.

[0207] S10311. Perform correction processing on the image to be processed to obtain a corrected image.

[0208] For example, black level correction and / or lens shading correction are performed on the image to be processed to obtain a corrected image. For instance, black level correction is first performed on the image to be processed, which can be achieved by subtracting a preset black level offset value from the pixel value of each pixel in the image to be processed; then lens shading correction is performed on the image after black level correction, which can be achieved through a lens shading correction map, which contains the brightness value that needs to be increased for each pixel. The image after lens shading correction is the corrected image.

[0209] S10312. The corrected image is divided into blocks to obtain multiple pixel blocks.

[0210] For example, the image to be processed is a Raw image. The original pixel size of the Raw image is H*W. After BLC and LSC processing, the corrected image is obtained. The corrected image is then divided into blocks, for example, 64*48 pixel blocks, or 3072 pixel blocks. The number of pixels in each pixel block is H / 64*W / 48. This is only an example; the number of blocks can be adjusted according to the actual situation and is not limited thereto.

[0211] S10313. Determine multiple statistical points based on multiple pixel blocks.

[0212] For example, for each pixel block, the RGB values ​​of the corresponding Raw image are averaged to obtain the RGB value of that pixel block, which is the statistical point. For instance, for a certain pixel block, the number of pixels contained in the pixel block is counted, the sum of the R channel values ​​of all pixels in the pixel block is calculated, and then the quotient of the sum of the R channel values ​​and the number of pixels is calculated to obtain the R channel mean. Similarly, the sum of the G channel values ​​of all pixels in the pixel block is calculated, and then the quotient of the sum of the G channel values ​​and the number of pixels is calculated to obtain the G channel mean. The sum of the B channel values ​​of all pixels in the pixel block is calculated, and then the quotient of the sum of the B channel values ​​and the number of pixels is calculated to obtain the B channel mean. The R channel mean, G channel mean, and B channel mean are the RGB values ​​of that pixel block.

[0213] The above processing is performed on all pixel blocks to obtain multiple statistical points. For example, if the above processing is performed on 64*48 pixel blocks, the RGB values ​​of these 64*48 blocks will be the multiple statistical points.

[0214] In this implementation, the image to be processed is corrected to improve its quality. Then, statistical points are determined based on the corrected image, which ensures that the two-dimensional and three-dimensional distribution patterns of the image to be processed can be accurately represented using the statistical points.

[0215] Optionally, in one possible implementation, step S1032 may include steps S10321 to S10323, see [link to relevant documentation]. Figure 9 , Figure 9 This is a schematic flowchart illustrating a method for determining feature information according to an exemplary embodiment of this application, as detailed below.

[0216] S10321. Obtain the RGB values ​​corresponding to multiple statistical points respectively.

[0217] It should be understood that for each statistical point, the RGB values ​​of the three channels corresponding to each pixel block are averaged to obtain the RGB value of that pixel block. The RGB value of that pixel block is recorded as the statistical point. Therefore, the RGB value of each pixel block can be directly obtained, thus obtaining the RGB value corresponding to each statistical point. The RGB value can include the R channel value, G channel value, and B channel value.

[0218] S10322. Determine the first ratio and the second ratio based on the RGB values.

[0219] The first ratio represents the ratio of the R channel value to the G channel value at each statistical point, and the second ratio represents the ratio of the B channel value to the G channel value at each statistical point.

[0220] For example, based on the R channel value and the G channel value, a first ratio between the R channel value and the G channel value is calculated, and in this embodiment, the first ratio is denoted as R / G; based on the B channel value and the G channel value, a second ratio between the B channel value and the G channel value is calculated, and in this embodiment, the second ratio is denoted as B / G.

[0221] S10323. Based on the first ratio and the second ratio, count the number of statistical points located in the gray area, and / or draw the two-dimensional distribution pattern of multiple statistical points.

[0222] For example, all statistical points are iterated through, and it is checked whether the R / G and B / G of each statistical point fall within the gray area. For each statistical point whose R / G and B / G fall within the gray area, the number of statistical points in the gray area is increased by one, until all statistical points have been checked, and the number of statistical points located in the gray area is obtained.

[0223] For example, a two-dimensional coordinate system is pre-established, with the horizontal axis being R / G and the vertical axis being B / G. Based on the R / G and B / G of each statistical point, each statistical point is plotted in the two-dimensional coordinate system. When all statistical points are plotted, a two-dimensional distribution of multiple statistical points is obtained. For example... Figure 7 As shown, the horizontal axis is R / G and the vertical axis is B / G. The two-dimensional distribution of multiple statistical points is a dispersed distribution.

[0224] In this implementation, feature information (i.e., the number of statistical points located in the gray area and / or the two-dimensional distribution of multiple statistical points) is determined based on the R / G and B / G of the statistical points. This feature information can accurately reflect the characteristics of the image to be processed under different shooting scenarios. Based on this, the shooting scenario corresponding to the image to be processed can be accurately identified. This makes it easier to accurately identify high-confidence white points in the image to be processed based on the shooting scenario. Then, based on the high-confidence white points, more accurate white balance processing is performed on the image to be processed, which greatly improves the white balance processing effect and effectively restores the true color of the image.

[0225] The following description, in conjunction with the accompanying drawings, details the specific process of determining different shooting scenarios based on different feature information.

[0226] Shooting Scene 1: Solid Color Scene

[0227] In one possible implementation, when the number of statistical points located in the gray area is less than a preset number and / or the two-dimensional distribution is a linear distribution, the shooting scene corresponding to the image to be processed is determined to be a solid color scene.

[0228] For example, when multiple statistical points corresponding to the image to be processed meet at least one of the following conditions: the number of statistical points located in the gray area is less than a preset number, and the two-dimensional distribution pattern is a linear distribution pattern, the shooting scene corresponding to the image to be processed is determined to be a solid color scene. The preset number can be set and adjusted according to actual conditions and is not limited thereto.

[0229] Please see Figure 10 , Figure 10 This is a schematic diagram illustrating the distribution of statistical points in a solid-color scene, as shown in an exemplary embodiment of this application. For example... Figure 10 As shown, the horizontal axis of the two-dimensional coordinate system is R / G, and the vertical axis is B / G. Most of the statistical points are located outside the gray area, meaning that the number of statistical points located inside the gray area is less than the preset number.

[0230] Please see Figure 11 , Figure 11 This is a schematic diagram illustrating another distribution pattern of statistical points in a solid-color scene, as shown in an exemplary embodiment of this application. For example... Figure 11 As shown, due to the influence of light and shadow, multiple statistical points are distributed in a straight line in the two-dimensional coordinate system, that is, the two-dimensional distribution of multiple statistical points is a straight line distribution.

[0231] In this implementation, when the distribution characteristics of multiple statistical points in the two-dimensional coordinate system are detected and meet the distribution characteristics of multiple statistical points in a solid color scene, the shooting scene corresponding to the image to be processed is determined to be a solid color scene. This improves the accuracy of identifying the shooting scene of the image to be processed and facilitates the accurate determination of high-confidence white points based on the solid color scene.

[0232] Alternatively, in another possible implementation, if the two-dimensional distribution of multiple statistical points corresponding to the image to be processed is a clustered distribution, and the number of statistical points located in the gray area is less than a preset number, the shooting scene corresponding to the image to be processed is determined to be a solid color scene.

[0233] Please see Figure 12 , Figure 12 This is a schematic diagram illustrating another distribution pattern of statistical points in a solid-color scene, as shown in an exemplary embodiment of this application. For example... Figure 12 As shown, multiple statistical points are clustered in the two-dimensional coordinate system, that is, the two-dimensional distribution of multiple statistical points is a clustered distribution. At the same time, most of the multiple statistical points are located outside the gray area, that is, the number of statistical points located in the gray area is less than the preset number. Therefore, it is determined that the shooting scene corresponding to the image to be processed is a solid color scene.

[0234] Alternatively, in another possible implementation, if the two-dimensional distribution pattern is a clustered distribution pattern, the three-dimensional distribution pattern corresponding to the image to be processed is determined based on multiple statistical points; if the three-dimensional distribution pattern is a clustered distribution pattern, the shooting scene corresponding to the image to be processed is determined to be a solid color scene.

[0235] Since the two-dimensional distribution of multiple statistical points corresponding to the image to be processed in a black-and-white gray scene is also a clustered distribution, if multiple statistical points only satisfy the two-dimensional distribution of clustered distribution, in order to avoid confusing the solid color scene with the black-and-white gray scene, we continue to judge whether the three-dimensional distribution of multiple statistical points satisfies the three-dimensional distribution characteristics of the solid color scene.

[0236] For example, by substituting the RGB channel values ​​corresponding to multiple statistical points into a preset brightness conversion formula, the brightness corresponding to each statistical point can be obtained. In this embodiment of the application, brightness can be represented by Y.

[0237] A three-dimensional coordinate system is established based on Y, R / G, and B / G, with the X-axis representing R / G, the Y-axis representing B / G, and the Z-axis representing Y. Each statistical point is plotted in the three-dimensional coordinate system according to its Y, R / G, and B / G values. Once all statistical points are plotted, the three-dimensional distribution of multiple statistical points is obtained.

[0238] Please see Figure 13 , Figure 13 This is a schematic diagram illustrating another distribution pattern of statistical points in a solid color scene, as shown in an exemplary embodiment of this application. For example... Figure 13 As shown, the horizontal axis represents the R / G label, the vertical axis represents the B / G label, and the Z-axis represents the Y label. The three-dimensional distribution of multiple statistical points is a clustered distribution.

[0239] In other words, when multiple statistical points corresponding to the image to be processed have a clustered distribution pattern in both two-dimensional and three-dimensional dimensions, the shooting scene corresponding to the image to be processed is determined to be a solid color scene.

[0240] Optionally, after determining the Y, R / G, and B / G values ​​for each statistical point, a Top-K strategy can be used to analyze the 3D distribution pattern of multiple statistical points. For example, the Top-K strategy is used to process the Y, R / G, and B / G values ​​of all statistical points, outputting a numerical value. When this value is greater than a first preset threshold, the shooting scene corresponding to the image to be processed is determined to be a solid color scene. For example, the first preset threshold can be set to 0.6, and the value output by the Top-K strategy is 0.68098958, indicating that the shooting scene corresponding to the image to be processed is a black-and-white scene.

[0241] In this implementation, when multiple statistical points only satisfy the two-dimensional distribution pattern as a clustered distribution pattern, it is further determined whether the three-dimensional distribution pattern of multiple statistical points satisfies the three-dimensional distribution characteristics under a solid color scene. This can avoid confusing solid color scenes with black, white and gray scenes and improve the accuracy of identifying the shooting scene of the image to be processed.

[0242] Shooting Scene 2: Interference Color Scene

[0243] In one possible implementation, when the number of statistical points located in the gray area is less than a preset number and / or the two-dimensional distribution is a linear distribution, and there are gray points in the image to be processed, the shooting scene corresponding to the image to be processed is determined to be an interference color scene.

[0244] Gray spots refer to the midtones in an image to be processed, indicating that a portion of the image has a color close to neutral gray, which is more common in large-area solid color scenes.

[0245] For example, a deep neural network can be trained using deep learning to detect the presence of gray points in the image to be processed. Another example is calculating the grayscale index mapping for each pixel in the image to identify the pixels most likely to be gray points. This is merely an illustrative example, and the comparison is not intended to be limiting.

[0246] If the number of statistical points located in the gray area of ​​the image to be processed is less than the preset number and / or the two-dimensional distribution is a linear distribution, and gray points are detected in the image to be processed, then the shooting scene corresponding to the image to be processed is determined to be an interference color scene.

[0247] Optionally, if the two-dimensional distribution of multiple statistical points corresponding to the image to be processed is an aggregated distribution, and the number of statistical points located in the gray area is less than a preset number, and gray points are detected in the image to be processed, then the shooting scene corresponding to the image to be processed is determined to be an interference color scene.

[0248] Optionally, if multiple statistical points corresponding to the image to be processed have a clustered distribution pattern in both two-dimensional and three-dimensional dimensions, and gray points are detected in the image to be processed, then the shooting scene corresponding to the image to be processed is determined to be an interference color scene.

[0249] In this implementation, when multiple statistical points are detected to satisfy the distribution characteristics of multiple statistical points under the shooting scene as a interference color scene, and the image to be processed satisfies the image characteristics under the shooting scene as a interference color scene, the shooting scene corresponding to the image to be processed is determined to be a interference color scene. This improves the accuracy of identifying the shooting scene of the image to be processed and facilitates the accurate determination of high-confidence white points based on the interference color scene.

[0250] Optionally, when the two-dimensional distribution of multiple statistical points corresponding to the image to be processed is a linear distribution, a Top-K strategy can be used to output a linearity value. When the linearity value is greater than or equal to a preset value, the shooting scene corresponding to the image to be processed is determined to be a solid color scene; when the linearity value is less than the preset value, the shooting scene corresponding to the image to be processed is determined to be an interfering color scene.

[0251] For example, the preset level value can be set to 0.9. If the level value of the straight line output by the Top-K strategy is 0.94, the shooting scene corresponding to the image to be processed is determined to be a solid color scene; if the level value of the straight line output by the Top-K strategy is 0.8, the shooting scene corresponding to the image to be processed is determined to be a distracting color scene.

[0252] Shooting Scene 3: Color-blocked Scene

[0253] In everyday shooting scenarios, besides solid color scenes, patchwork color scenes are also common, such as when transitioning from a non-solid color scene to a solid color scene. If patchwork color scenes cannot be accurately identified, the identified white points will have an interference effect, thus affecting the white balance. Therefore, this application embodiment also provides an image processing method that, after determining the three-dimensional distribution pattern of the image to be processed based on multiple statistical points, if the two-dimensional distribution pattern is a clustered distribution pattern, may further include: if the three-dimensional distribution pattern is a dispersed distribution pattern, and the dispersed distribution pattern belongs to a block-dispersed pattern, then determine that the shooting scene corresponding to the image to be processed is a patchwork color scene.

[0254] For example, the multiple statistical points corresponding to the color-blocking scene and the multiple statistical points corresponding to the black-and-white / gray scene both exhibit a clustered distribution pattern in two dimensions and a roughly dispersed distribution pattern in three dimensions. To accurately distinguish between the color-blocking scene and the black-and-white / gray scene, the distribution pattern under the dispersed distribution pattern is further determined. Since the color-blocking scene contains at least two pure colors, its dispersed distribution pattern specifically presents as a block-based dispersed pattern.

[0255] Please see Figure 14 , Figure 14 This is a schematic diagram illustrating another distribution pattern of statistical points in a solid color scene, as shown in an exemplary embodiment of this application. For example... Figure 14 As shown, the horizontal axis represents the R / G label, the vertical axis represents the B / G label, and the Z-axis represents the Y label. The three-dimensional distribution of multiple statistical points is a dispersed distribution, and the dispersed distribution is a block-dispersed distribution, which determines that the shooting scene corresponding to the image to be processed is a spliced ​​color scene.

[0256] Optionally, a Top-K strategy is used to process the Y, R / G, and B / G values ​​of all statistical points, outputting a numerical value. When this value is greater than a second preset threshold, the scene corresponding to the image to be processed is determined to be a mosaic color scene. For example, the second preset threshold can be set to 0.23, and the value output by the Top-K strategy is 0.2327474, thus determining that the scene corresponding to the image to be processed is a mosaic color scene.

[0257] This implementation effectively avoids confusing spliced ​​color scenes with black, white, and gray scenes, improving the accuracy of identifying the shooting scene of the image to be processed.

[0258] Shooting Scene 4: Black, White, and Gray Scene

[0259] Since the two-dimensional distribution of multiple statistical points corresponding to the color-blocking scene and the black-and-white / gray scene are both clustered, and their three-dimensional distribution is roughly dispersed, in order to avoid confusion between the color-blocking scene and the black-and-white / gray scene, this application embodiment also provides an image processing method. After determining the three-dimensional distribution of the image to be processed based on multiple statistical points, when the two-dimensional distribution is clustered, the method may further include: if the three-dimensional distribution is dispersed and the dispersed distribution is random, determining that the shooting scene corresponding to the image to be processed is a black-and-white / gray scene.

[0260] Since the color-blocked scene contains at least two solid colors, its distribution pattern is specifically presented as a block-based distribution pattern. In contrast, the distribution pattern of the black, white, and gray scene is completely random. The distribution pattern under the distribution pattern can be used to accurately identify the black, white, and gray scene.

[0261] Please see Figure 15 , Figure 15 This is a schematic diagram illustrating another distribution pattern of statistical points in a solid-color scene, as shown in an exemplary embodiment of this application. For example... Figure 15As shown, the horizontal axis represents the R / G label, the vertical axis represents the B / G label, and the Z-axis represents the Y label. The three-dimensional distribution of multiple statistical points is a dispersed distribution, and the dispersed distribution is a random dispersion, which determines that the shooting scene corresponding to the image to be processed is a black and white scene.

[0262] Optionally, a value is output using a Top-K strategy. When this value is less than a second preset threshold, the scene corresponding to the image to be processed is determined to be a black-and-white scene. For example, the second preset threshold can be set to 0.23, and the value output by the Top-K strategy is 0.22949219, thus determining that the scene corresponding to the image to be processed is a black-and-white scene.

[0263] Optionally, in this example, the black-and-white scene can be a rich black-and-white scene.

[0264] This implementation effectively avoids confusing spliced ​​color scenes with black, white, and gray scenes, improving the accuracy of identifying the shooting scene of the image to be processed.

[0265] Shooting Scene 5: Colorful Scene

[0266] When multiple statistical points corresponding to the image to be processed exhibit a dispersed two-dimensional distribution and a randomly dispersed three-dimensional distribution, it is determined that the shooting scene corresponding to the image to be processed is a color-rich scene. For example... Figure 7 As shown, the horizontal axis is R / G and the vertical axis is B / G. The two-dimensional distribution of multiple statistical points is a dispersed distribution.

[0267] In this implementation, when multiple statistical points are detected to satisfy the distribution characteristics of statistical points in a color-rich scene, the shooting scene corresponding to the image to be processed is determined to be a color-rich scene. This improves the accuracy of identifying the shooting scene of the image to be processed and facilitates the accurate determination of high-confidence white points based on the color-rich scene.

[0268] The following describes the specific process of determining the high-confidence white points in the image to be processed under different shooting scenarios. Please refer to [link / reference]. Figure 16 , Figure 16 This is a schematic flowchart illustrating a method for determining high-confidence white points, as shown in an exemplary embodiment of this application, as detailed below.

[0269] S201. When the shooting scene corresponding to the image to be processed is any one of the following scenes: a scene with rich colors, a scene with interfering colors, or a black and white scene, determine the white point corresponding to the image to be processed.

[0270] For example, the white points of the image to be processed can be calculated using an automatic white balance algorithm. This automatic white balance algorithm may include the gray-world method, total internal reflection theory method, color temperature estimation method, etc. In this embodiment, the gray-world method is used as an example for illustration.

[0271] For example, the white points of the image to be processed can be calculated using the following formula.

[0272]

[0273] In equation (1) above, (rg,bg) represents the white points of the image to be processed.

[0274]

[0275] Among them, R i G represents the R channel value of the i-th pixel in the image to be processed. i B represents the G channel value of the i-th pixel in the image to be processed. i This represents the B channel value of the i-th pixel in the image to be processed. M represents the width of the image to be processed, N represents the height of the image to be processed, and the unit of M and N is pixels. i represents any pixel in the image to be processed, and the value of i can be 0, 1, 2...M*N-1.

[0276] S202. Identify the white points as high-confidence white points corresponding to the image to be processed.

[0277] In one example, when the scene corresponding to the image to be processed is a colorful scene, it proves that the image to be processed has rich colors. In this case, the white point of the image to be processed calculated through the above step S201 is accurate, effective, has low interference, and high confidence. Therefore, this white point can be identified as the high-confidence white point corresponding to the image to be processed. Based on this high-confidence white point, white balance processing of the image to be processed can greatly improve the white balance effect, effectively correct color deviations under different lighting conditions, and effectively ensure that the colors in the image presented to the user are as close as possible to the colors of the actual objects, that is, to restore the true colors of the image to a higher degree, improve the overall image quality, and bring a good experience to the user.

[0278] Alternatively, when the shooting scene corresponding to the image to be processed is a colorful scene, the white balance processing of the image to be processed can also be performed in a traditional way.

[0279] In another example, when the shooting scene corresponding to the image to be processed is a scene with interfering colors, the white point calculated in step S201 above can be determined as the high confidence white point corresponding to the image to be processed, and then the image to be processed is white balanced using the high confidence white point; alternatively, the image to be processed can be white balanced using a traditional method, such as determining the white point corresponding to the image to be processed and then using the white point to perform white balance processing on the image to be processed.

[0280] In another example, when the shooting scene corresponding to the image to be processed is a black and white scene, the white point calculated in step S201 above can be determined as the high confidence white point corresponding to the image to be processed, and then the image to be processed is white balanced using the high confidence white point; alternatively, the image to be processed can be white balanced using the traditional method.

[0281] S203. Store the high-confidence white points corresponding to the image to be processed into a preset queue.

[0282] When the scene corresponding to the image to be processed is a colorful scene, the calculated white point of the image to be processed is accurate and effective, with low interference and high confidence. In this case, the high-confidence white point can also be stored in a preset queue. In the embodiments of this application, the high-confidence white points stored in the preset queue are called historical high-confidence white points, which facilitate the use of historical high-confidence white points for white balance processing of images to be processed in other shooting scenes (such as solid color scenes), thereby improving the white balance effect of images to be processed in other shooting scenes (such as solid color scenes).

[0283] For example, a fixed-length queue (i.e., a preset queue) can be pre-established and initialized each time the camera application is launched. When the camera captures an image to be processed, and it is determined that the scene in which the image to be processed was captured is a colorful scene, the white points in the image to be processed are calculated, and these white points are identified as high-confidence white points and stored in the fixed-length queue.

[0284] Optionally, the fixed-length queue may also store the image identification information, acquisition time, correlated color temperature, lighting value (LV), infrared (IR) information from the multispectral sensor, CCM, etc.

[0285] Optionally, when the shooting scene corresponding to the image to be processed is a scene with interfering colors or a black and white scene, the white points of the image to be processed may not be stored in the fixed-length queue. This ensures that the white points stored in the fixed-length queue are all high-confidence white points, thereby improving the effect of subsequent images using historical high-confidence white points for white balance processing.

[0286] Optionally, in one possible implementation, the image processing method provided in this application embodiment may also involve responding to a first operation by launching a camera application; displaying a first interface, the first interface including a preview area; determining the white point corresponding to the image to be processed; determining the shooting scene corresponding to the image to be processed; determining the high-confidence white point corresponding to the image to be processed based on the white point and the shooting scene; performing white balance processing on the image to be processed using the high-confidence white point to obtain a preview image, and displaying the preview image in the preview area.

[0287] For example, when the scene corresponding to the image to be processed is a color-rich scene, the color-rich scene is determined to be a high-confidence scene. The white points of the determined image to be processed are stored as high-confidence white points in a preset queue. Optionally, the white points of the determined image to be processed or the high-confidence white points can be used to perform white balance processing on the image to be processed to obtain a preview image, and the preview image is displayed in the preview area.

[0288] For example, when the scene corresponding to the image to be processed is a solid color scene, the solid color scene is determined to be a low-confidence scene. In this case, the white points of the determined image to be processed are not stored in the preset queue. Instead, historical high-confidence white points are searched in the preset queue, and the white balance of the image to be processed is performed using the historical high-confidence white points to obtain a preview image, which is then displayed in the preview area.

[0289] For example, when the scene corresponding to the image to be processed is a scene with interfering colors, the scene with interfering colors is determined to be a low-confidence scene. In this case, the white points of the image to be processed will not be stored in the preset queue. Optionally, in this scenario, historical high-confidence white points may not be used for white balance processing of the image to be processed.

[0290] Optionally, in one possible implementation, the method for determining high-confidence white points provided in this application embodiment may further include steps S301 and S302, as detailed below.

[0291] S301. When the shooting scene corresponding to the image to be processed is a solid color scene or a spliced ​​color scene, obtain the historical high confidence white points corresponding to the color-rich scene from the preset queue.

[0292] S302. Identify historical high-confidence white points as the high-confidence white points corresponding to the image to be processed.

[0293] When the scene depicted by the image to be processed is a solid color scene or a scene with patchwork colors, the white points of the image to be processed obtained directly are subject to interference, have low confidence, and are inaccurate; they can be considered invalid. In this case, historical high-confidence white points corresponding to scenes with rich colors are obtained from a preset queue, and these historical high-confidence white points are determined as the high-confidence white points corresponding to the current image to be processed. Then, white balance processing of the image to be processed is performed using these high-confidence white points.

[0294] Optionally, when the shooting scene corresponding to the image to be processed is a solid color scene or a spliced ​​color scene, the historical high confidence white points corresponding to the color-rich scene are obtained from the preset queue, and the historical high confidence white points are used to perform white balance processing on the image to be processed.

[0295] Optionally, the historical high-confidence white points can be any one or more frames of images to be processed in a color-rich scene from the preset queue. When the historical high-confidence white points are high-confidence white points of multiple frames of images to be processed in a color-rich scene from the preset queue (such as all currently cached high-confidence white points in the preset queue), the high-confidence white points of multiple frames of images to be processed can be weighted and fused, and the weighted fusion result can be used as the high-confidence white point corresponding to the current image to be processed.

[0296] For example, if the current image to be processed is the 4th frame, and the scene captured in the 1st frame was detected as a colorful scene, the white points in the 1st frame are calculated, identified as high-confidence white points, and stored in a preset queue. Optionally, the confidence level (e.g., confidence level 1) corresponding to the high-confidence white point can also be determined, and the confidence level 1 corresponding to the high-confidence white point can also be associated and stored in the preset queue. Assuming that the subsequent 2nd and 3rd frames are both captured in colorful scenes, the white points in the 2nd and 3rd frames are identified as high-confidence white points and stored in the preset queue in the same way. Simultaneously, the confidence levels (e.g., confidence level 2 and confidence level 3) corresponding to the high-confidence white points in the 2nd and 3rd frames can also be associated and stored in the preset queue.

[0297] Subsequently, when the fourth frame of the image to be processed is acquired, it is detected that the shooting scene of the fourth frame is a solid color scene or a spliced ​​color scene. High-confidence white points corresponding to the first, second, and third frames of the image to be processed, along with the confidence level corresponding to each high-confidence white point, are extracted from the preset queue. Based on the confidence level corresponding to each high-confidence white point, these high-confidence white points are weighted and fused, and the weighted fusion result is used as the high-confidence white point corresponding to the fourth frame of the image to be processed. For example, the product of the high-confidence white point of the first frame and confidence level 1 is calculated, the product of the high-confidence white point of the second frame and confidence level 2 is calculated, and the product of the high-confidence white point of the third frame and confidence level 3 is calculated. The sum of the three products is then divided by 3 to obtain the high-confidence white point of the fourth frame.

[0298] Understandably, if the scenes captured in the subsequent 5th, 6th, 7th, and 8th frames of the image to be processed are all solid-color scenes, then the historical high-confidence white points in the preset queue will not be updated. This means that the high-confidence white points in the 5th, 6th, 7th, and 8th frames of the image to be processed are the same as the high-confidence white points in the 4th frame of the image to be processed. Conversely, if the scenes captured in the subsequent images of the image to be processed are rich in color, then the historical high-confidence white points in the preset queue will be updated. This means that the high-confidence white points in the subsequent solid-color scenes of the image to be processed will also change accordingly.

[0299] In this implementation, a preset queue stores high-confidence white points from images captured before they are taken in solid-color or multi-color scenes. With the shooting light source unchanged, these high-confidence white points are used to perform white balance processing on the images to be processed in either solid-color or multi-color scenes. Since the high-confidence white points in the preset queue are determined from images in colorful scenes, these white points are accurate, effective, and have low interference. White balance processing based on these white points can greatly restore the true colors of the image, ensuring that the color of the photographed object displayed on the screen matches the color of the photographed object in the real world. This significantly improves the image white balance processing effect and enhances image quality.

[0300] Optionally, when the shooting scene corresponding to the image to be processed is a scene with interfering colors, the accuracy of the white balance processing of images in interfering color scenes by the camera module can also be determined. For example, if the accuracy is greater than or equal to the accuracy threshold (such as 80%), the image to be processed is white-balanced using the traditional method to ensure the white balance processing effect.

[0301] For example, if the accuracy is less than the accuracy threshold (e.g., 80%), historical high-confidence white points corresponding to color-rich scenes are obtained from a preset queue, and white balance processing is performed on the image to be processed using these historical high-confidence white points. It should be understood that the accuracy threshold can be set and adjusted according to the actual situation, and there are no restrictions on it.

[0302] In this implementation, a pre-set queue stores high-confidence white points from images captured before the arrival of the interfering color scene. With the shooting light source unchanged, the high-confidence white points in the pre-set queue are used to perform white balance processing on the image to be processed under the interfering color scene. Because the high-confidence white points in the pre-set queue are accurate, effective, have low interference levels, and high confidence, white balance processing based on these points can greatly restore the true colors of the image and improve image quality.

[0303] Optionally, in one possible implementation, the image processing method provided in this application embodiment further includes: determining the first correlated color temperature corresponding to the image to be processed based on the high confidence white point corresponding to the image to be processed; and performing color correction processing on the image to be processed based on the first correlated color temperature to obtain a color-corrected image.

[0304] There is a mapping relationship between white points and correlated color temperatures. If only high-confidence white points are determined without adjusting the corresponding correlated color temperature, image color distortion may still occur. Therefore, in this embodiment, the first correlated color temperature corresponding to the image to be processed is determined based on the high-confidence white points. For example, a pre-established mapping relationship table between white points and correlated color temperatures is obtained, and the corresponding first correlated color temperature is looked up in the mapping relationship table based on the high-confidence white points.

[0305] For example, a color calibration curve (CCM) is constructed using a determined first correlated color temperature and a high-confidence white point. The CCM is then applied to each pixel of the image to be processed to obtain a color-corrected image, thus achieving color correction. Afterward, the color-corrected image is subjected to white balance processing in the aforementioned manner to obtain a preview image, which is then displayed in the preview area of ​​the first interface.

[0306] Optionally, CCM can be applied to each pixel of the white balance processed image (i.e., the preview image) to obtain a color-corrected preview image, which is then displayed in the preview area of ​​the first interface.

[0307] In this implementation, the correlated color temperature is adjusted according to the high-confidence white point to make the determined correlated color temperature more accurate, thereby making the colors of the image after color correction based on the correlated color temperature more accurate and natural.

[0308] Optionally, in one possible implementation, the first correlated color temperature corresponding to the image to be processed can be determined using the aforementioned method for determining the high-confidence white point of the image to be processed. For example, when the shooting scene corresponding to the image to be processed is any of the following: a color-rich scene, a distracting color scene, or a black-and-white / gray scene, the correlated color temperature corresponding to the image to be processed is determined using the method described in related technologies. The correlated color temperature of the current image to be processed is then determined as the high-confidence correlated color temperature (i.e., the first correlated color temperature) and stored in a preset queue. As another example, when the shooting scene corresponding to the image to be processed is a solid color scene or a patchwork color scene, the high-confidence correlated color temperature corresponding to the color-rich scene is obtained from the preset queue, and the historical high-confidence white point is determined as the high-confidence correlated color temperature of the image to be processed. It should be understood that this is only an illustrative example; the specific implementation method can refer to the aforementioned method for determining the high-confidence white point of the image to be processed, and will not be elaborated further here.

[0309] Optionally, in one possible implementation, the image processing method provided in this application embodiment further includes: determining the second correlated color temperature corresponding to the adjacent images of the image to be processed; if the difference between the first correlated color temperature and the second correlated color temperature is detected to be greater than a preset difference, the high confidence white point corresponding to the image to be processed is not used to perform white balance processing on the image to be processed.

[0310] Accurately determining the CCT of the ambient light source during shooting is typically challenging, especially when electronic devices can only acquire the reflectance spectrum. For example, in scenes with interfering colors, the acquired CCT can be affected to varying degrees, leading to inaccurate CCT determinations. In this embodiment, the reliability of the first correlated color temperature (CCT) of the image to be processed can be determined by the difference between the second CCT of adjacent images and the first CCT of the image to be processed. Furthermore, since there is a mapping relationship between CCT and white point, the reliability of the first CCT of the image to be processed can be used to determine whether to apply a high-confidence white point to the image for white balance processing.

[0311] For example, the acquisition time of the adjacent images of the image to be processed is before the acquisition time of the image to be processed. For example, the image to be processed is the 6th frame, and the adjacent image of the image to be processed is the 5th frame.

[0312] Optionally, adjacent images of the image to be processed can also be spaced out by multiple frames. For example, the image to be processed is the 6th frame, and the adjacent image is the 3rd frame.

[0313] Retrieve the previously stored correlated color temperature (i.e., the second correlated color temperature) from the preset queue. Calculate the difference between the first and second correlated color temperatures. If the difference is greater than a preset difference, it indicates that the first correlated color temperature of the image to be processed has low confidence, and the high-confidence white point corresponding to the image to be processed is not used for white balance processing. If the difference is less than or equal to the preset difference, it indicates that the first correlated color temperature of the image to be processed has high confidence, and the high-confidence white point corresponding to the image to be processed is used for white balance processing.

[0314] In this implementation, by calculating the difference in correlated color temperature between consecutive frames, the reliability of the correlated color temperature of the current image to be processed can be determined. This allows for a decision on whether to use the high-confidence white point of the image to be processed for white balance processing. This avoids the problem of storing a high-confidence white point under light source A and then using the high-confidence white point stored under light source A under light source B.

[0315] Optionally, in one possible implementation, a multispectral sensor can be used to calculate the CCT and infrared (IR) values ​​of each frame of the image to be processed, and to determine whether the CCT and IR values ​​of the image to be processed differ significantly from those of adjacent images. If the differences in both CCT and IR values ​​are relatively large (e.g., the difference in CCT is greater than a preset difference, and the difference in IR values ​​is greater than a preset IR difference), the high-confidence white point corresponding to the image to be processed is not used for white balance processing. If the differences in both CCT and IR values ​​are relatively small (e.g., the difference in CCT is less than or equal to a preset difference, and the difference in IR values ​​is less than or equal to a preset IR difference), the high-confidence white point corresponding to the image to be processed is used for white balance processing. This effectively avoids the problem of storing a high-confidence white point under light source A and then using the high-confidence white point stored under light source A under light source B.

[0316] Please see Figure 17 , Figure 17 This is a schematic diagram illustrating a light source switching method according to an exemplary embodiment of this application. For example... Figure 17As shown, the first, second, and third frames of the image to be processed all use sunlight as the shooting light source. Assuming the third frame was captured in a colorful scene, the white point of that frame is identified as a high-confidence white point and stored in a preset queue. The fourth frame of the image to be processed uses incandescent lamps as the shooting light source. Assuming the fourth frame was captured in a solid-color scene, if the high-confidence white point of the third frame in the preset queue is used for white balance processing, the high-confidence white point of the third frame, determined based on sunlight, cannot correct for color deviations under incandescent lamps, resulting in poor white balance performance. However, the method provided in this embodiment effectively avoids the problem of storing high-confidence white points under sunlight and using them under incandescent lamps.

[0317] Optionally, in one possible implementation, if a change in the shooting scene corresponding to the image to be processed is detected, the preset queue is cleared.

[0318] For example, the preset queue can store high-confidence white point, correlated color temperature, lighting value (LV), infrared (IR) information from the multispectral sensor, CCM, etc. of the image to be processed. This information from neighboring images of the image to be processed can be obtained from the preset queue and compared with this information of the current image to be processed. If the differences in this information are all relatively large, it is determined that the shooting scene from the neighboring images has changed to the shooting scene of the current image to be processed. For example, switching from a solid color scene to a special scene, at this time the preset queue is cleared.

[0319] Optionally, special scenes may include blue sky scenes, night scene scenes, natural scenes (such as forest scenes, grassland scenes, beach scenes, ocean scenes, mountain scenes, etc.), architectural scenes, etc.

[0320] In these special scenarios, specific white points are designed, allowing for the use of these white points for white balance processing of the image. Clearing the preset queue can reduce cache pressure on electronic devices, freeing up more storage space; it can also prevent the misuse of historically high-confidence white points from the preset queue for white balance processing in special scenarios.

[0321] Alternatively, in one possible implementation, the preset queue is cleared when the camera application is detected to be closed / stopped from running.

[0322] The foregoing has detailed examples of the image processing methods provided in the embodiments of this application. The following description, in conjunction with the accompanying drawings, describes the image processing apparatus provided in the embodiments of this application.

[0323] Please refer to Figure 18 , Figure 18 This is a schematic diagram of the structure of an image processing apparatus illustrated in an exemplary embodiment of this application. Figure 18 As shown, the image processing device may include a startup unit 410, a display unit 420, a first determination unit 430, a second determination unit 440, and a processing unit 450.

[0324] The startup unit 410 is used to launch the camera application in response to the first operation;

[0325] Display unit 420 is used to display a first interface, the first interface including a preview area;

[0326] The first determining unit 430 is used to determine the shooting scene corresponding to the image to be processed; the image to be processed is an image captured by the camera in real time; the shooting scene includes any one of the following: solid color scene, spliced ​​color scene, interference color scene, black and white gray scene, and color rich scene.

[0327] The second determining unit 440 is used to determine the high confidence white point corresponding to the image to be processed based on the shooting scene. The high confidence white point is used to characterize the real light source color of the shooting environment corresponding to the image to be processed.

[0328] The processing unit 450 is used to perform white balance processing on the image to be processed using a high-confidence white point, obtain a preview image, and display the preview image in the preview area.

[0329] It is understood that these units included in the image processing apparatus can also be used to perform other steps in the image processing method provided above, which will not be elaborated here.

[0330] Since the image processing apparatus provided in this application embodiment is used to execute the corresponding image processing method provided above, the beneficial effects it can achieve can be referred to the beneficial effects in the corresponding image processing method provided above, and will not be repeated here.

[0331] The foregoing has detailed examples of image processing methods provided in the embodiments of this application. It is understood that, in order to achieve the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application.

[0332] This application embodiment can divide the electronic device into functional modules according to the above method example. For example, each function can be divided into a separate functional module, such as a startup unit, a display unit, a first determining unit, a second determining unit, a processing unit, etc., or two or more functions can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0333] It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0334] The electronic device provided in this embodiment is used to execute the above-described image processing method, and therefore can achieve the same effect as the above-described implementation method.

[0335] When using integrated units, the electronic device may further include a processing module, a storage module, and a communication module. The processing module is used to control and manage the operation of the electronic device. The storage module supports the execution of stored program code and data. The communication module supports communication between the electronic device and other devices.

[0336] The processing module can be a processor or a controller. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a digital signal processor (DSP), and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, or other devices that interact with other electronic devices.

[0337] In one embodiment, when the processing module is a processor and the storage module is a memory, the electronic device involved in this embodiment can be a device having... Figure 3 The device with the structure shown.

[0338] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the image processing method of any of the above embodiments.

[0339] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the image processing method described above.

[0340] This application also provides a chip. Please refer to [link / reference]. Figure 19 , Figure 19 This is a schematic diagram of the structure of a chip provided in an embodiment of this application. Figure 19 The chip shown can be a general-purpose processor or a special-purpose processor. The chip includes a processor 510. The processor 510 is used to execute the communication method of any of the above embodiments.

[0341] Optionally, the chip also includes a transceiver 520, which is used to receive control from the processor and to support the communication device in executing the aforementioned technical solution.

[0342] Optional, Figure 19 The chip shown may also include: storage medium 530.

[0343] It should be noted that, Figure 19 The chip shown can be implemented using one or more field programmable gate arrays (FPGAs), programmable logic devices (PLDs), controllers, state machines, gate logic, discrete hardware components, any other suitable circuits, or any combination of circuits capable of performing the various functions described throughout this application.

[0344] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component, or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the chip to execute the image processing methods in the above-described method embodiments.

[0345] In this embodiment, the electronic device, computer-readable storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0346] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0347] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0348] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0349] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0350] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the image processing methods provided in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0351] It is understood that the above description is merely a specific embodiment provided in this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, Applied to an electronic device including a camera, the method includes: In response to the first action, launch the camera app; Display a first interface, which includes a preview area; Determine the shooting scene corresponding to the image to be processed; the image to be processed is an image captured in real time by the camera; the shooting scene includes any one of the following: solid color scene, spliced ​​color scene, interference color scene, black and white gray scene, and color rich scene. The high-confidence white point corresponding to the image to be processed is determined based on the shooting scene. The high-confidence white point is used to characterize the true light source color of the shooting environment corresponding to the image to be processed. The image to be processed is subjected to white balance processing using the high-confidence white point to obtain a preview image, which is then displayed in the preview area.

2. The method according to claim 1, characterized in that, Determining the shooting scene corresponding to the image to be processed includes: Determine the statistical information of the image to be processed; the statistical information includes multiple statistical points, which are used to indicate multiple processed pixel blocks corresponding to the image to be processed; Based on the statistical information, the feature information of the image to be processed is determined; the feature information includes the number of statistical points located in the gray area, and / or the two-dimensional distribution pattern of the multiple statistical points; Based on the feature information, the shooting scene corresponding to the image to be processed is determined.

3. The method according to claim 2, characterized in that, The determination of statistical information for the image to be processed includes: The image to be processed is corrected to obtain a corrected image; The corrected image is divided into blocks to obtain multiple pixel blocks; The plurality of statistical points are determined based on the plurality of pixel blocks.

4. The method according to claim 2 or 3, characterized in that, The step of determining the feature information of the image to be processed based on the statistical information includes: Obtain the RGB values ​​corresponding to the multiple statistical points respectively; Based on the RGB values ​​of each statistical point, a first ratio and a second ratio are determined; the first ratio represents the ratio of the R channel value to the G channel value of each statistical point, and the second ratio represents the ratio of the B channel value to the G channel value of each statistical point. Based on the first ratio and the second ratio, count the number of statistical points located in the gray area, and / or draw the two-dimensional distribution pattern of the multiple statistical points.

5. The method according to any one of claims 2 to 4, characterized in that, Determining the shooting scene corresponding to the image to be processed based on the feature information includes: When the number of statistical points located in the gray area is less than the preset number and / or the two-dimensional distribution pattern is a linear distribution pattern, the shooting scene corresponding to the image to be processed is determined to be a solid color scene.

6. The method according to claim 5, characterized in that, Determining the shooting scene corresponding to the image to be processed based on the feature information includes: When the number of statistical points located in the gray area is less than the preset number and / or the two-dimensional distribution pattern is a linear distribution pattern, and there are gray points in the image to be processed, the shooting scene corresponding to the image to be processed is determined to be an interference color scene.

7. The method according to any one of claims 2 to 4, characterized in that, Determining the shooting scene corresponding to the image to be processed based on the feature information includes: If the two-dimensional distribution pattern is an aggregated distribution pattern, the three-dimensional distribution pattern corresponding to the image to be processed is determined based on the multiple statistical points. If the three-dimensional distribution pattern is a clustered distribution pattern, the shooting scene corresponding to the image to be processed is determined to be a solid color scene.

8. The method according to claim 7, characterized in that, Determining the shooting scene corresponding to the image to be processed based on the feature information includes: If the two-dimensional distribution pattern is a clustered distribution pattern, the three-dimensional distribution pattern is a clustered distribution pattern, and there are gray points in the image to be processed, then the shooting scene corresponding to the image to be processed is determined to be an interference color scene.

9. The method according to claim 7, characterized in that, After determining the three-dimensional distribution pattern corresponding to the image to be processed based on the plurality of statistical points, the method further includes: If the three-dimensional distribution pattern is a dispersed distribution pattern, and the dispersed distribution pattern belongs to a block-dispersed pattern, then the shooting scene corresponding to the image to be processed is determined to be a spliced ​​color scene. If the three-dimensional distribution pattern is a dispersed distribution pattern, and the dispersed distribution pattern is a random dispersion pattern, then the shooting scene corresponding to the image to be processed is determined to be a black-and-white scene.

10. The method according to claim 7, characterized in that, Determining the shooting scene corresponding to the image to be processed based on the feature information includes: If the two-dimensional distribution pattern is a dispersed distribution pattern and the three-dimensional distribution pattern is a random dispersed distribution pattern, then the shooting scene corresponding to the image to be processed is determined to be a color-rich scene.

11. The method according to any one of claims 1 to 10, characterized in that, The step of determining the high-confidence white point corresponding to the image to be processed based on the shooting scene includes: When the shooting scene corresponding to the image to be processed is any one of the following scenes: a color-rich scene, a distracting color scene, and a black-and-white gray scene, the white point corresponding to the image to be processed is determined. The white points are identified as high-confidence white points corresponding to the image to be processed; Store the high-confidence white points corresponding to the image to be processed into a preset queue.

12. The method according to claim 11, characterized in that, The step of determining the high-confidence white point corresponding to the image to be processed based on the shooting scene includes: When the shooting scene corresponding to the image to be processed is a solid color scene or a spliced ​​color scene, the historical high confidence white points corresponding to the color-rich scene are obtained from the preset queue. The historical high-confidence white points are identified as the high-confidence white points corresponding to the image to be processed.

13. The method according to any one of claims 1 to 12, characterized in that, The method further includes: Based on the high-confidence white point corresponding to the image to be processed, determine the first correlated color temperature corresponding to the image to be processed; The image to be processed is color-corrected based on the first correlated color temperature to obtain a color-corrected image.

14. The method according to claim 13, characterized in that, The method further includes: Determine the second correlated color temperature corresponding to the neighboring images of the image to be processed; the acquisition time of the neighboring images is before the acquisition time of the image to be processed; If the difference between the first correlated color temperature and the second correlated color temperature is detected to be greater than a preset difference, the high confidence white point corresponding to the image to be processed will not be used to perform white balance processing on the image to be processed.

15. An electronic device, characterized in that, The electronic device includes: one or more processors, and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 14.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 14.

17. A computer program product, characterized in that, The computer program product includes computer program code that, when run, causes the method as described in any one of claims 1 to 14 to be performed.

Images