Portrait image white balance optimization method, control device, mobile phone and storage medium

Abstract

The invention relates to an image processing technology, and discloses a white balance optimization method of a portrait image, a control device, a mobile phone and a storage medium, and the method comprises the steps: obtaining an illumination parameter of the mobile phone when the mobile phone shoots the portrait image in a current environment; comparing the illumination parameters with corresponding preset conditions to judge whether the current environment belongs to a weak white balance environment or not; if yes, based on the currently shot portrait image, matching a pre-stored image matched with the human face from mobile phone storage images; selecting a pre-stored image shot based on the strong white balance environment as a reference image; comparing the face color difference between the current portrait image and the reference image; and performing white balance calibration optimization on the current portrait image based on the face color difference, wherein the optimization range comprises a portrait part and a background part. The invention aims to realize precise adjustment of the white balance during mobile phone portrait shooting under a complex illumination condition.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a white balance optimization method for portrait images, a control device, a mobile phone, and a computer-readable storage medium. Background Art

[0002] White balance is a concept in photography and image processing. It refers to adjusting the colors in an image so that photos taken under different light sources can show colors close to the real world. Different light sources have different color temperatures, that is, the light emitted by the light source has different color tendencies. For example, daylight is usually cold (high color temperature), while tungsten light is warm (low color temperature). If the white balance is not adjusted correctly when shooting, the photos taken may be blue or yellow, which is inconsistent with the colors in reality.

[0003] In current mobile phone cameras, the CMOS sensor of the mobile phone is needed to capture the three colors of red, green and blue, and then more colors are inferred based on this information to give the white balance. In other words, the white balance of a photo needs to be "guessed" by the mobile phone. But the problem is that the traditional RGB array signal can only cover part of the colors that the human eye can perceive. Once the light is complex, the lack of color information can easily lead to inaccurate white balance.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide a white balance optimization method for portrait images, a control device, a mobile phone and a computer-readable storage medium, aiming to achieve precise adjustment of the white balance when taking portrait photos with a mobile phone under complex lighting conditions.

[0006] To achieve the above object, the present application provides a method for optimizing white balance of a portrait image, comprising the following steps:

[0007] Acquire lighting parameters when the mobile phone takes a portrait image in the current environment, where the lighting parameters include at least one of color temperature, light intensity, and chromaticity;

[0008] Compare the lighting parameters with the corresponding preset conditions to determine whether the current environment is a weak white balance environment;

[0009] If so, based on the currently captured portrait image, a pre-stored image that matches the face is matched from the images stored in the mobile phone;

[0010] Selecting a pre-stored image shot in a strong white balance environment as a reference image;

[0011] Compare the facial difference between the current portrait image and the reference image;

[0012] The white balance calibration of the current portrait image is optimized based on the face color difference, and the optimization range includes the portrait part and the background part.

[0013] To achieve the above object, the present application also provides a control device, comprising:

[0014] An acquisition module, used to acquire lighting parameters when the mobile phone takes a portrait image in a current environment, where the lighting parameters include at least one of color temperature, light intensity and chromaticity;

[0015] A comparison module is used to compare the illumination parameters with corresponding preset conditions to determine whether the current environment belongs to a weak white balance environment;

[0016] A query module, for matching a pre-stored image that matches the face of the person from images stored in the mobile phone based on the currently captured portrait image;

[0017] A selection module, used for selecting a pre-stored image shot based on a strong white balance environment as a reference image;

[0018] A calculation module, used for comparing the facial difference between the current portrait image and the reference image;

[0019] The calibration module is used to optimize the white balance calibration of the current portrait image based on the human face difference, and the optimization range includes the portrait part and the background part.

[0020] To achieve the above objectives, the present application also provides a mobile phone, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the white balance optimization method for portrait images as described above.

[0021] To achieve the above objectives, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the white balance optimization method for portrait images as described above are implemented.

[0022] The white balance optimization method, control device, mobile phone and computer-readable storage medium for portrait images provided in the present application perform face color difference calibration for portrait images through intelligent white balance environment detection and reference image comparison, and extend the optimization to the background part, thereby achieving accurate white balance adjustment under complex lighting conditions, thereby improving the overall color restoration quality of the portrait and background, and ensuring the authenticity and aesthetics of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A schematic diagram of the steps of a method for optimizing white balance of a portrait image in one embodiment of the present application;

[0024] Figure 2 This is a schematic diagram of a control device in an embodiment of the present application;

[0025] Figure 3 A schematic diagram of the internal structure of a mobile phone according to an embodiment of the present application.

[0026] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0027] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0028] In addition, if the description of "first", "second", etc. is involved in this application, it is only used for descriptive purposes (such as for distinguishing the same or similar features), and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in this field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0029] Reference Figure 1 In one embodiment, the white balance optimization method of the portrait image includes:

[0030] Step S10, obtaining lighting parameters when the mobile phone takes a portrait image in the current environment, where the lighting parameters include at least one of color temperature, light intensity and chromaticity;

[0031] Step S20: comparing the illumination parameters with corresponding preset conditions to determine whether the current environment is a weak white balance environment;

[0032] Step S30: If yes, then based on the currently captured portrait image, a pre-stored image matching the face is matched from the images stored in the mobile phone;

[0033] Step S40, selecting a pre-stored image shot in a strong white balance environment as a reference image;

[0034] Step S50, comparing the facial color difference between the current portrait image and the reference image;

[0035] Step S60: Optimize the white balance calibration of the current portrait image based on the face color difference, and the optimization range includes the portrait part and the background part.

[0036] In this embodiment, the execution terminal of the embodiment may be a mobile phone, or may be other equipment or devices (such as a control device) that controls the mobile phone.

[0037] As described in step S10, the illumination parameters are mainly obtained through the sensors and algorithms of the mobile phone, and the specific sources include:

[0038] Light source sensor: used to detect the color temperature and light intensity of ambient light;

[0039] Camera sensor (CMOS / CCD): Analyzes the color distribution of the scene and calculates the color temperature and chromaticity information;

[0040] Ambient light sensor (ALS): used to measure the brightness of ambient light (light intensity);

[0041] Algorithm-assisted: Combine sensor data and image processing algorithms to further optimize the accuracy of lighting parameters.

[0042] Color temperature is a physical quantity that describes the color of a light source, and its unit is Kelvin (K). Different light sources have different color temperatures, for example: daylight is about 5500K (cold tones), tungsten lamps are about 2800K (warm tones), and cloudy days are about 6500K (cold tones).

[0043] The color temperature can be directly measured by the light source sensor; it can also be inferred by analyzing the color distribution of white or gray areas in the image.

[0044] Light intensity is a physical quantity that describes the brightness of ambient light, and its unit is lux.

[0045] The light intensity can be directly measured by an ambient light sensor; or the light intensity can be estimated by analyzing the brightness distribution of the image.

[0046] Chromaticity is another dimension that describes the color of a light source and is usually expressed as chromaticity coordinates (x, y).

[0047] The chromaticity can be obtained by analyzing the color distribution of the light source through a light source sensor or a camera sensor; or, the chromaticity coordinates of the light source can be determined using a chromaticity diagram (such as the CIE 1931 chromaticity diagram).

[0048] Optionally, when taking a portrait image, the mobile phone automatically starts the light source sensor, the ambient light sensor and the camera sensor to start collecting ambient light information.

[0049] Optionally, the light source sensor collects color temperature and chromaticity data; the ambient light sensor collects light intensity data; the camera sensor analyzes the color distribution in the image by shooting the scene to assist in estimating the color temperature and chromaticity.

[0050] Optionally, the raw data collected by the sensor is processed, for example, the color temperature data is smoothed to reduce noise; the light intensity data is calibrated to ensure accuracy; the chromaticity data is standardized and converted into chromaticity coordinates.

[0051] The processed color temperature, light intensity and chromaticity data are output as input for subsequent steps.

[0052] Assume that the current environment is an indoor scene, illuminated by tungsten lamps: the sensor detects a color temperature of 2800K (warm tone); the ambient light sensor detects a light intensity of 200 lux; the camera sensor analyzes the image and determines the chromaticity coordinates to be (0.45, 0.41). These data will be output for judgment and optimization in subsequent steps.

[0053] As described in step S20, the main task is to compare the acquired illumination parameters with the preset conditions to determine whether the current environment is a weak white balance environment. This judgment is the key to determining whether further optimization of white balance is required.

[0054] Preset conditions are a series of thresholds or standards set to determine whether the current environment is a weak white balance environment. These conditions are defined based on lighting parameters (color temperature, light intensity, chromaticity). Optional preset conditions include:

[0055] (1) Color temperature threshold: If the color temperature is lower than 3000K or higher than 7000K, it may indicate that the ambient light color temperature is extreme, making it difficult to accurately adjust the white balance;

[0056] (2) Light intensity threshold: If the light intensity is lower than 100 lux or higher than 10,000 lux, it may mean that the ambient light is too dark or too bright, affecting the accuracy of white balance;

[0057] (3) Chromaticity range: If the chromaticity coordinates exceed a certain range, it means that the ambient light color shift is serious and requires special treatment.

[0058] These thresholds and ranges can be determined based on actual application scenarios and experimental data to ensure accurate identification of weak white balance environments.

[0059] By comparing the acquired lighting parameters with the preset conditions, you can check whether the color temperature, light intensity and chromaticity exceed their respective thresholds or ranges. You can also combine multiple lighting parameters for a comprehensive judgment. For example, when the color temperature is lower than a certain value and the light intensity is lower than a certain value, it is judged as a weak white balance environment.

[0060] Optionally, different weights are assigned to each lighting parameter, and a comprehensive score is calculated based on the parameter values. If a certain threshold is exceeded, it is determined to be a weak white balance environment.

[0061] Based on the comparison results, determine whether the current environment is a weak white balance environment. The specific logic may include:

[0062] If the color temperature is lower than the preset lower limit or higher than the preset upper limit, it is judged as a weak white balance environment;

[0063] If the light intensity is lower than the preset lower limit or higher than the preset upper limit, it is judged as a weak white balance environment;

[0064] If the chromaticity coordinates exceed the preset range, it is judged as a weak white balance environment;

[0065] When multiple conditions are met at the same time, it is more certain to be judged as a weak white balance environment.

[0066] Assume that the preset conditions are as follows: color temperature <3000K or >7000K, light intensity <100lux or >10000lux, and chromaticity coordinates (x, y) are not in the range of (0.3, 0.3) to (0.5, 0.5).

[0067] If the lighting parameters obtained are: color temperature is 800K, illumination intensity is 80lux, and chromaticity coordinates are (0.42, 0.41).

[0068] According to the preset conditions, the color temperature is lower than 3000K, the light intensity is lower than 100lux, and the chromaticity coordinates are within the range. Therefore, it can be judged that the current environment belongs to a weak white balance environment.

[0069] Optionally, the threshold of the preset condition may be dynamically adjusted according to different application scenarios or user preferences to adapt to different environmental requirements.

[0070] As described in step S30, after determining that the current environment is a weak white balance environment, based on the currently captured portrait image, a pre-stored image that matches the facial features of the current portrait is matched from images stored in the mobile phone.

[0071] Optionally, perform face detection on the currently captured portrait image and extract the face area. Use a face recognition algorithm (such as MTCNN, FaceNet) to extract key features of the face, such as the facial contour, the position of the eyes, nose, and mouth, etc. Generate a face feature vector (Face Feature Vector) for the current portrait image.

[0072] Perform face detection and feature extraction on the pre-stored images stored in the mobile phone one by one. For each pre-stored image, extract its face feature vector.

[0073] Compare the facial feature vector of the current portrait with the facial feature vectors of all pre-stored images. Use feature distance measurement methods (such as Euclidean distance, cosine similarity) to calculate the similarity of facial features of the two images.

[0074] All pre-stored images are sorted according to the similarity scores, and one or more pre-stored images that are most similar to the current portrait are selected as candidates.

[0075] To improve matching accuracy, candidate images can be further screened:

[0076] Check whether the lighting conditions (such as color temperature and light intensity) of the candidate image are close to the current environment, and select it if so; or check whether the background of the candidate image is similar to the background of the current portrait image (if so, select it) to ensure the comprehensiveness of the match.

[0077] As described in step S40, after matching the pre-stored image similar to the current portrait in step S30, a pre-stored image shot in a strong white balance environment is further selected as a reference image. The reference image will be used as a reference for subsequent white balance optimization to ensure that the optimization result is closer to the real color performance.

[0078] Make sure that the pre-stored images are taken in a strong white balance environment, for example, photos taken by the user under strong lighting conditions.

[0079] Alternatively, the phone's built-in standard reference images (usually high-quality images taken under standard light sources). The white balance of these images has been verified to be accurate and can provide a reliable reference for white balance optimization of the current portrait.

[0080] From the candidate pre-stored images matched in step S30, images shot in a strong white balance environment are further screened out to ensure that the lighting conditions (such as color temperature and light intensity) of these images meet the standards of a strong white balance environment.

[0081] Optionally, a pre-stored image that does not belong to a weak white balance environment may be defined as a pre-stored image of a strong white balance environment.

[0082] Through screening and evaluation, a high-quality image that meets the strong white balance environment is selected from the matched candidate pre-stored images as the reference image. This step provides an accurate and reliable reference for subsequent white balance optimization, ensuring that the optimization result is more natural and realistic.

[0083] As described in step S50, after selecting the reference image, the color difference between the current portrait image and the reference image is compared to determine the deviation of the current portrait image in white balance. This step is the key to white balance optimization. By comparing the color difference, accurate data support can be provided for subsequent white balance calibration.

[0084] Optionally, use a face detection algorithm (such as MTCNN, Dlib) to locate the face area in the current portrait image. Extract key feature points of the face, such as eyes, nose, mouth, etc.

[0085] Perform the same face detection and feature point extraction on the reference image to ensure that the same face area is compared.

[0086] Select a suitable color space for color feature extraction. Common color spaces include RGB, Lab, XYZ, etc. Extract the color features of the face area in the current portrait image and the reference image, including the average color value, color distribution, etc. Extract the Lab color value of each pixel in the face area.

[0087] The color difference formula is used to calculate the color difference of the corresponding face areas in the two images. The optional color difference formulas include CIE76, CIE94, CIEDE2000, etc.

[0088] Optionally, the color difference is calculated for each pixel in the face area of ​​the current portrait image and the reference image to generate a color difference map indicating the color difference value of each pixel.

[0089] Optionally, the face region is divided into multiple sub-regions (such as eyes, nose, mouth, cheeks, etc.), and the average color difference of each sub-region is calculated to obtain local color difference features.

[0090] Optionally, the average color difference of the entire face area is calculated as a global color difference index.

[0091] Optionally, the color difference of each sub-area is analyzed to determine which area has a larger color difference and may need focused calibration.

[0092] Optionally, draw a color difference histogram to analyze the distribution of color difference and further understand the characteristics of color difference.

[0093] By calculating the color difference, the color difference between the current portrait image and the reference image can be accurately determined, providing accurate data support for subsequent white balance optimization. By calculating the local color difference, it is possible to find which area has the largest color difference, which helps to make targeted white balance adjustments. The color difference results provide a reference for the white balance optimization algorithm, ensuring that the optimized image color is closer to the real scene and improving the visual effect of the image.

[0094] Step S50 accurately compares the face difference between the current portrait image and the reference image by locating the face area, extracting color features, calculating color difference, and evaluating comprehensive color difference. This step provides detailed data support for subsequent white balance calibration, ensuring that the optimization result is more natural and realistic.

[0095] As described in step S60 , the main goal is to adjust the white balance of the current image by analyzing the difference in facial color between the portrait image and the reference image, so that the color of the overall image is more accurate and natural.

[0096] First, perform a global white balance adjustment to correct the color temperature deviation of the entire image. This adjustment is based on global color difference data and its main purpose is to make the overall color of the image closer to the standard white balance state.

[0097] Optionally, use the gray world algorithm: Assume that the average color in the image is neutral gray, and adjust the gain of the RGB channels so that the average R, G, and B values ​​of the image are equal. Calculate the average value of the R, G, and B channels of the current image. Calculate the gain coefficient and apply the gain to adjust the image.

[0098] Optionally, use the white point method: Identify the white points in the image and adjust the RGB values ​​of these points to pure white (255,255,255) to correct the color temperature. Find the white points in the image (assuming these points are white in the real world) and calculate the RGB average of these white points. Calculate the gain factor and apply the gain to adjust the image.

[0099] Optionally, the color temperature of the current image is estimated based on the color difference calculation result and adjusted to a standard color temperature (such as 5000K or 6500K). The color of the image is adjusted using a color temperature conversion matrix.

[0100] After the global adjustment, local white balance adjustment is performed for the face area to ensure the accuracy and naturalness of the skin tone. The color difference of the face area is calculated, especially the deviation in skin tone. According to the color difference of the face area, the gain of the RGB channel is fine-tuned. The average value of R, G, and B in the face area is calculated and compared with the average value of the face area in the reference image. The gain is adjusted to reduce this difference.

[0101] While adjusting the portrait part, you also need to optimize the background part to ensure color consistency and natural transition of the entire image. Analyze the color difference of the background area to avoid overexposure or underexposure of the background. According to the color difference of the background area, adjust the gain of the RGB channel to ensure that the background color is coordinated with the portrait part.

[0102] Optionally, use image fusion technology to ensure that the color transition between the portrait and the background is natural and there is no obvious dividing line.

[0103] Optionally, the color difference with the reference image is gradually reduced by adjusting the white balance parameters through multiple iterations, and the effect is evaluated after each adjustment. A certain number of iterations is set, and the color difference is recalculated after each adjustment. When the color difference is less than a preset threshold or the maximum number of iterations is reached, the adjustment is stopped.

[0104] Optionally, a color difference formula (such as ΔE) is used to evaluate the difference between the adjusted image and the reference image.

[0105] In one embodiment, through intelligent white balance environment detection and reference image comparison, face color difference calibration is performed on portrait images, and the optimization is extended to the background part, so as to achieve accurate white balance adjustment under complex lighting conditions, thereby improving the overall color restoration quality of the portrait and background, and ensuring the authenticity and aesthetics of the image.

[0106] In one embodiment, based on the above embodiment, the step of selecting a pre-stored image shot based on a strong white balance environment as a reference image includes:

[0107] If there are multiple pre-stored images shot in a strong white balance environment, the image with the shooting time closest to the current time is selected as the reference image.

[0108] In this embodiment, pre-stored images taken under a strong white balance environment are selected from images stored in the mobile phone, and these images usually have good color reproduction and white balance effects.

[0109] If there are multiple pre-stored images that meet the strong white balance conditions, the phone will further compare the shooting times of these images.

[0110] Among all pre-stored images that meet the strong white balance condition, the one whose shooting time is closest to the current shooting time is selected as the reference image.

[0111] Among them, the images stored in the mobile phone are initially screened to identify which images were taken under strong white balance conditions. This can be achieved by analyzing the metadata of the image (such as lighting parameters, color temperature, etc.).

[0112] For the pre-stored images in the selected strong white balance environment, the phone will extract the shooting time of each image, which is usually recorded in the metadata of the image, and compare these shooting times with the current shooting time to calculate each time difference.

[0113] Among all the time differences, the image with the smallest one is selected. The shooting time of this image is closest to the current shooting time, so it is considered to be the best reference image.

[0114] Selecting the image with the closest shooting time as the reference image can ensure that the reference image is closer to the lighting conditions of the current environment, thereby improving the accuracy and reliability of white balance calibration. Images taken by users are usually in similar environments, and selecting the most recent image as the reference can better adapt to the current shooting environment and improve the overall image quality.

[0115] In one embodiment, not only can the pre-stored images that meet the strong white balance conditions be screened out, but the image closest to the current shooting time can also be further selected as the reference image, thereby improving the effect of white balance optimization and ensuring that the final generated image has good color reproduction and visual effects.

[0116] In one embodiment, based on the above embodiment, the optimization algorithm used in the white balance calibration optimization is any one of a gray world algorithm, a perfect reflection algorithm and a retinex algorithm.

[0117] In this embodiment, the algorithm used for white balance calibration optimization is any one of a gray world algorithm, a perfect reflection algorithm or a Retinex algorithm.

[0118] Gray World Algorithm: The basic assumption is that the average color in natural scenes is medium gray; the implementation method is to calculate the average value of the brightness or RGB channels of the image, and then adjust the color balance of the image to make these average values ​​reach the level of medium gray. The process is simple and fast and is applicable to many natural scenes.

[0119] Perfect reflection algorithm: It basically assumes that there are perfect reflection points in the image, which reflect the entire spectrum of the light source. It is implemented by identifying high-brightness areas in the image (which may be perfect reflection points) and estimating the color of the light source based on the RGB values ​​of these points, thereby performing white balance correction. This can better process images with bright spots.

[0120] Retinex algorithm: The basic principle is to simulate the visual system of the human eye and separate the illumination component and reflection component of the image through multi-scale analysis to achieve color restoration. The implementation methods include multi-scale processing, such as Single Scale Retinex (SSR), MultiScale Retinex (MSR) and MultiScale Retinex with Color Restoration (MSRCR). This algorithm can effectively handle the shadow and uneven illumination problems in the image and provide more natural color restoration.

[0121] Optionally, different white balance calibration algorithms may be selected according to specific usage scenarios and requirements.

[0122] In practical applications, it is also possible to combine the advantages of multiple algorithms, or automatically select the most appropriate algorithm based on the specific characteristics of the image to achieve the best white balance optimization effect.

[0123] In one embodiment, based on the above embodiment, the white balance optimization method of the portrait image further includes:

[0124] Based on the degree of difference in people's facial expressions, a corresponding optimization algorithm is selected.

[0125] In this embodiment, the color difference between the current portrait image and the reference image has been calculated through step S50. The color difference can be measured by comparing the RGB value, chromaticity (such as ΔE in CIE Lab* space) of the face area in the two images.

[0126] According to the calculated color difference value, the color difference is divided into different levels (such as low, medium, and high) to reflect the degree of color distortion of the current image.

[0127] For example, low color difference means that the color difference value is small, the image color is close to the reference image, and the distortion degree is low; medium color difference means that the color difference value is moderate, and there is a certain deviation between the image color and the reference image; high color difference means that the color difference value is large, and the image color deviates significantly from the reference image.

[0128] Optionally, based on the classification of the degree of color difference, a most suitable optimization algorithm is selected to perform white balance calibration.

[0129] Optionally, when the color difference is small, it means that the color deviation of the current image is small, and the gray world algorithm can be used for rapid calibration. The gray world algorithm is simple and efficient, and can quickly correct slight color deviations while maintaining the natural color of the image.

[0130] Optionally, when the color difference reaches a moderate level, it means that the color deviation of the image is more obvious, but it can still be calibrated by identifying the bright spots (such as reflection points). The perfect reflection algorithm can use the high brightness areas in the image (which may be reflection points) to quickly adjust the white balance, which is suitable for images with medium deviation.

[0131] Optionally, when the color difference is large, it means that the color deviation of the image is very significant, and there may be complex lighting unevenness or shadow problems. The Retinex algorithm can separate the illumination component and reflection component of the image, providing more refined color calibration, which is suitable for high color difference scenes.

[0132] In this way, different optimization algorithms are selected according to the degree of color difference, which can better adapt to scenes with different lighting conditions and image quality. For images with small color difference, a simple and efficient gray world algorithm is used; for images with large color difference, a more sophisticated Retinex algorithm is used to avoid unnecessary waste of computing resources.

[0133] By selectively selecting the optimization algorithm, the white balance effect of the image can be significantly improved, especially for portrait images in complex lighting environments.

[0134] In one embodiment, during the white balance optimization process, by determining the degree of difference in facial color difference between people and dynamically selecting the corresponding optimization algorithm (gray world algorithm, perfect reflection algorithm or Retinex algorithm), the flexibility and effect of white balance calibration can be significantly improved. This can better adapt to different lighting conditions and image features, ensuring that the final portrait image has a high-quality color reproduction effect.

[0135] In one embodiment, based on the above embodiment, after the step of optimizing the white balance calibration of the current portrait image based on the face color difference, and the optimization range includes the portrait part and the background part, the step further includes:

[0136] After receiving a confirmation instruction of the optimized portrait image, saving the optimized white balance parameters;

[0137] When the mobile phone continues to shoot images based on the current environment, the white balance of the mobile phone camera is optimized based on the optimized white balance parameters.

[0138] In this embodiment, after viewing the optimized portrait image, the user can confirm the optimization effect by confirming the instruction (such as clicking a "save" button).

[0139] After receiving the confirmation command, the mobile phone will save the white balance parameters (such as color temperature, chromaticity, light intensity, etc.) used in the current optimization process.

[0140] The optimized white balance parameters will be stored in the phone's settings or cache as a white balance reference for the current environment. These parameters can be used for subsequent image capture in the same or similar environments to quickly achieve white balance optimization.

[0141] When the phone continues to take images in the current environment, the system will automatically detect the lighting conditions of the current environment (such as color temperature, light intensity, etc.). If the lighting conditions of the current environment are similar to those during the previous optimization, the previously saved optimization parameters will be directly applied.

[0142] In subsequent shooting, the mobile phone camera module will automatically adjust the white balance settings according to the saved optimization parameters. This can avoid repeated complex lighting parameter detection and algorithm optimization, and quickly generate images with good color reproduction.

[0143] If the lighting conditions of the current environment change significantly, the system will restart the white balance optimization process (such as re-detecting the lighting parameters and selecting a new algorithm). Otherwise, the saved optimization parameters are directly applied to ensure shooting efficiency and image quality.

[0144] By saving the optimized white balance parameters, there is no need to repeat the complex white balance optimization calculations when shooting in the same or similar environment, saving system resources and processing time. Users can take high-quality images more quickly and improve the shooting experience.

[0145] The saved optimization parameters can ensure that images taken in the same environment have consistent color reproduction effects, avoiding color deviation caused by lighting changes.

[0146] If the ambient lighting changes significantly, the system will re-optimize the white balance to ensure high-quality color calibration under different lighting conditions.

[0147] In one embodiment, by saving the white balance parameters after optimization and directly applying these parameters in subsequent shooting, the shooting efficiency and image color consistency can be significantly improved. This is particularly important for users to shoot continuously in the same or similar environment, and can quickly generate high-quality images with good color reproduction.

[0148] In one embodiment, based on the above embodiment, after receiving the confirmation instruction of the optimized portrait image, after saving the optimized white balance parameters, the method further includes:

[0149] Based on the optimized white balance parameters, adjust the white balance adaptive algorithm pre-stored in the mobile phone.

[0150] In this embodiment, after saving the optimized white balance parameters, the system will extract these parameters (such as color temperature, chromaticity, light intensity, etc.) These parameters are calculated based on the lighting conditions of the current environment and the optimization effect of the portrait image.

[0151] The system will analyze the white balance adaptive algorithm currently stored in the phone (such as the default gray world algorithm, perfect reflection algorithm or Retinex algorithm). If the adaptive algorithm does not have specific optimization parameters for the current environment, or the parameter settings of the existing algorithm are not ideal, adjustments need to be made.

[0152] Based on the optimized white balance parameters, the system will adjust the pre-stored white balance adaptive algorithm.

[0153] The specific methods of adjustment include:

[0154] (1) Update algorithm parameters: embed optimized white balance parameters (such as color temperature and chromaticity) into the pre-stored algorithm so that the algorithm can better adapt to the lighting conditions of the current environment;

[0155] (2) Dynamically adjust weights: According to the optimized parameters, adjust the weights of different modules in the algorithm (such as the priority of the gray world algorithm, perfect reflection algorithm or Retinex algorithm);

[0156] (3) Generate a new adaptive mode: If the lighting conditions of the current environment are significantly different from those of the pre-stored algorithm, the system can also generate a new adaptive mode specifically for the current environment.

[0157] After the adjustment is completed, the system will save the adjusted adaptive algorithm to the phone as the default white balance calibration mode in the current environment. In this way, in subsequent shooting, the phone can directly use the updated adaptive algorithm to quickly generate high-quality images.

[0158] By adjusting the pre-stored algorithm based on the optimized white balance parameters, the algorithm's adaptability to the current environment can be significantly improved; the adjusted algorithm can better deal with color deviation problems under specific lighting conditions and improve the overall quality of the image; after the user confirms the optimized portrait image, the system will automatically adjust the algorithm without manual intervention by the user. This dynamic optimization method can significantly improve the user experience, and users can more easily capture images with good color reproduction.

[0159] The adjusted adaptive algorithm can be directly applied to subsequent shooting, avoiding repeated complex white balance optimization calculations. This not only improves shooting efficiency, but also saves system resources.

[0160] In one embodiment, by adjusting the pre-stored white balance adaptive algorithm of the mobile phone based on the optimized white balance parameters, the adaptability of the algorithm to the current environment can be significantly improved, and an image with good color restoration can be quickly generated. At the same time, this dynamic adjustment method can also save system resources and improve user experience, which is very suitable for application in smart photography devices.

[0161] In one embodiment, based on the above embodiment, after receiving the confirmation instruction of the optimized portrait image, after saving the optimized white balance parameters, the method further includes:

[0162] The optimized white balance parameters are associated with the lighting parameters of the current environment and uploaded to the cloud server to provide the cloud server with training samples for the white balance adaptive algorithm.

[0163] In this embodiment, the optimized white balance parameters (such as color temperature, chromaticity, light intensity, etc.) are associated with the lighting parameters of the current environment to generate a data packet containing the optimized parameters and the lighting parameters.

[0164] The generated data packet is uploaded to the cloud server. After receiving the uploaded data packet, the cloud server performs the following processing:

[0165] (1) Store the data packet in the database as part of the training sample;

[0166] (2) Analyze the uploaded samples and extract features for subsequent white balance adaptive algorithm training and optimization;

[0167] (3) The collected samples are used to train or optimize the white balance adaptive algorithm to improve the accuracy and adaptability of the algorithm.

[0168] After completing the algorithm training, the cloud server can push the updated white balance adaptive algorithm model or parameters to the mobile phone. After receiving the update, the mobile phone can automatically apply the new algorithm model to further optimize the subsequent white balance calibration effect.

[0169] By associating the optimized white balance parameters with the lighting parameters of the current environment and uploading them to the cloud server, more diverse training samples can be collected. This helps improve the generalization ability of the white balance adaptive algorithm, enabling it to perform well under a variety of lighting conditions.

[0170] The cloud server can continuously train and optimize the white balance adaptive algorithm based on the uploaded sample data. This continuous optimization method can dynamically adjust the algorithm to adapt to the changing lighting environment.

[0171] By uploading optimized parameters, users can obtain more accurate and consistent white balance calibration results, which not only improves the user's shooting experience, but also enhances the user's trust and satisfaction with the device.

[0172] In one embodiment, by associating the optimized white balance parameters with the lighting parameters of the current environment and uploading them to the cloud server, more diverse training samples can be collected, further improving the generalization ability and accuracy of the white balance adaptive algorithm. At the same time, this continuous optimization method ensures that the algorithm can perform well under a variety of lighting conditions, improving the user's shooting experience and satisfaction.

[0173] In addition, refer to Figure 2 In an embodiment of the present application, a control device Z10 is further provided, comprising:

[0174] An acquisition module Z11 is used to acquire illumination parameters when the mobile phone takes a portrait image in a current environment, where the illumination parameters include at least one of color temperature, illumination intensity and chromaticity;

[0175] The comparison module Z12 is used to compare the illumination parameters with the corresponding preset conditions to determine whether the current environment belongs to a weak white balance environment;

[0176] A query module Z13 is used to match a pre-stored image that matches the face of the person from images stored in the mobile phone based on the currently captured portrait image.

[0177] A selection module Z14 is used to select a pre-stored image shot based on a strong white balance environment as a reference image;

[0178] A computing module Z15 is used to compare the facial color difference between the current portrait image and the reference image;

[0179] The calibration module Z16 is used to perform white balance calibration optimization on the current portrait image based on the human face difference, and the optimization range includes the portrait part and the background part.

[0180] Optionally, the control device Z10 may be a virtual control device (such as a virtual machine) or a physical device (such as a physical device other than a mobile phone that can execute the corresponding method).

[0181] In addition, a mobile phone is also provided in an embodiment of the present application. The internal structure of the mobile phone can be as follows: Figure 3 As shown, it includes a processor, a memory, a communication interface and an input interface connected through a mobile phone bus. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating mobile phone, a computer program and a database. The internal memory provides an environment for the operation of the operating mobile phone and the computer program in the non-volatile storage medium. The database is used to store data called by the computer program. The communication interface is used to communicate data with an external terminal. The input interface is used to receive a signal input by an external device. When the computer program is executed by the processor, a white balance optimization method for a portrait image as described in the above embodiment is implemented.

[0182] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the present application scheme, and does not constitute a limitation on the mobile phone to which the present application scheme is applied. For example, in some optional embodiments, the mobile phone may also include an output interface (not shown in the figure), and the output interface is also connected to the mobile phone bus and is used to output corresponding signals to the peripheral device.

[0183] In addition, the present application also proposes a computer-readable storage medium, the computer-readable storage medium including a computer program, and when the computer program is executed by a processor, the steps of the white balance optimization method for a portrait image as described in the above embodiment are implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0184] In summary, the white balance optimization method, control device, mobile phone and computer-readable storage medium for portrait images provided in the embodiments of the present application perform face color difference calibration for portrait images through intelligent white balance environment detection and reference image comparison, and extend the optimization to the background part, thereby achieving accurate white balance adjustment under complex lighting conditions, thereby improving the overall color restoration quality of the portrait and background, and ensuring the authenticity and aesthetics of the image.

[0185] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0186] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0187] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for optimizing white balance of a portrait image, characterized in that: include: Acquire lighting parameters when the mobile phone takes a portrait image in the current environment, where the lighting parameters include at least one of color temperature, light intensity, and chromaticity; Compare the lighting parameters with the corresponding preset conditions to determine whether the current environment is a weak white balance environment; If so, based on the currently captured portrait image, a pre-stored image that matches the face is matched from the images stored in the mobile phone; Selecting a pre-stored image shot in a strong white balance environment as a reference image; Compare the facial difference between the current portrait image and the reference image; The white balance calibration of the current portrait image is optimized based on the face color difference, and the optimization range includes the portrait part and the background part.

2. The white balance optimization method for a portrait image according to claim 1, characterized in that: The step of selecting a pre-stored image shot based on a strong white balance environment as a reference image comprises: If there are multiple pre-stored images shot in a strong white balance environment, the image with the shooting time closest to the current time is selected as the reference image.

3. The white balance optimization method for a portrait image according to claim 1, wherein: The optimization algorithm used in the white balance calibration optimization is any one of a gray world algorithm, a perfect reflection algorithm and a retinex algorithm.

4. The white balance optimization method for a portrait image according to claim 3, characterized in that: The white balance optimization method for the portrait image also includes: Based on the degree of difference in people's facial expressions, a corresponding optimization algorithm is selected.

5. The white balance optimization method for a portrait image according to claim 1, wherein: After the step of optimizing the white balance calibration of the current portrait image based on the face difference of the human face, and the optimization range includes the portrait part and the background part, the step further includes: After receiving a confirmation instruction of the optimized portrait image, saving the optimized white balance parameters; When the mobile phone continues to shoot images based on the current environment, the white balance of the mobile phone camera is optimized based on the optimized white balance parameters.

6. The white balance optimization method for a portrait image according to claim 5, characterized in that: After receiving the confirmation instruction of the optimized portrait image, after saving the optimized white balance parameters, the method further includes: Based on the optimized white balance parameters, adjust the white balance adaptive algorithm pre-stored in the mobile phone.

7. The white balance optimization method for a portrait image according to claim 5 or 6, characterized in that: After receiving the confirmation instruction of the optimized portrait image, after saving the optimized white balance parameters, the method further includes: The optimized white balance parameters are associated with the lighting parameters of the current environment and uploaded to the cloud server to provide the cloud server with training samples for the white balance adaptive algorithm.

8. A control device, characterized in that: include: An acquisition module, used to acquire lighting parameters when the mobile phone takes a portrait image in a current environment, where the lighting parameters include at least one of color temperature, light intensity and chromaticity; A comparison module is used to compare the illumination parameters with corresponding preset conditions to determine whether the current environment belongs to a weak white balance environment; A query module, for matching a pre-stored image that matches the face of the person from images stored in the mobile phone based on the currently captured portrait image; A selection module, used for selecting a pre-stored image shot based on a strong white balance environment as a reference image; A calculation module, used for comparing the facial difference between the current portrait image and the reference image; The calibration module is used to optimize the white balance calibration of the current portrait image based on the human face difference, and the optimization range includes the portrait part and the background part.

9. A mobile phone, characterized in that: The mobile phone includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the white balance optimization method for a portrait image as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the white balance optimization method for a portrait image according to any one of claims 1 to 7 are implemented.

Images