Image processing method and device, electronic equipment, storage medium and program product
By acquiring target color temperature feature data and performing weighted summation, and combining it with feature data from multiple sensors for time synchronization and smoothing, the problem of large measurement errors of color temperature sensors due to reflected light from objects is solved, achieving high-precision white balance adjustment and image quality improvement.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING XIAOMI MOBILE SOFTWARE CO LTD
- Filing Date
- 2024-10-29
- Publication Date
- 2026-05-01
AI Technical Summary
In the existing technology, when electronic devices capture images, the reflected light from objects affects the measurement of the color temperature sensor, resulting in a large error in color temperature measurement and low white balance accuracy.
By acquiring target color temperature feature data, the target color temperature level is determined by weighted summation of the prior probability of color temperature and the conditional probability of feature values. White balance is then adjusted based on the target color temperature level. Furthermore, time synchronization and smoothing processing are performed by combining feature data from multiple sensors to improve the accuracy of color temperature estimation.
It improves the accuracy of color temperature estimation and white balance precision, thereby enhancing image quality.
Smart Images
Figure CN121967907A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of image processing technology, specifically to an image processing method, apparatus, electronic device, storage medium, and program product. Background Technology
[0002] In image capture scenarios, electronic devices such as mobile phones that use cameras typically need to adjust the white balance of images. Using relevant technologies, these devices usually determine the current color temperature using a color temperature sensor and then adjust the white balance accordingly.
[0003] However, since reflected light from objects can affect the color temperature sensor's measurements, the measured color temperature usually contains errors, which in turn leads to lower white balance accuracy. Summary of the Invention
[0004] The purpose of this disclosure is to provide an image processing method, apparatus, electronic device, storage medium, and program product to improve color temperature accuracy and white balance precision.
[0005] On one hand, this disclosure provides an image processing method, the method comprising:
[0006] Obtain target color temperature feature data; the target color temperature feature data includes at least one feature associated with color temperature and its corresponding feature value;
[0007] The target color temperature probability for each color temperature level is determined based on the prior color temperature probability of each color temperature level and the conditional probability of the feature value of at least one feature under each color temperature level. Both the prior color temperature probability and the conditional probability are determined based on multiple color temperature feature sample data and their corresponding color temperature levels.
[0008] Based on the target color temperature probability of each color temperature level, the target color temperature level is obtained by weighted summation of the color temperature levels.
[0009] Based on the target color temperature level, the white balance of the image to be processed is adjusted to obtain the target image.
[0010] In one embodiment, before acquiring the target color temperature feature data, the method further includes:
[0011] For each feature, perform the following steps:
[0012] Obtain the initial values of the features at multiple sampling times;
[0013] The initial values of the feature at multiple sampling times are smoothed to obtain the feature values.
[0014] In one embodiment, before determining the target color temperature probability for each color temperature level based on the prior color temperature probability of each color temperature level and the conditional probability of the eigenvalue of at least one feature at each color temperature level, the method further includes:
[0015] Obtain multiple color temperature feature sample data and their corresponding color temperature levels; the color temperature feature sample data includes at least one feature associated with color temperature and its corresponding feature value;
[0016] Count the total number of samples for multiple color temperature feature data;
[0017] For each color temperature level, perform the following steps:
[0018] Based on the total number of samples and the first sample number of the color temperature feature sample data corresponding to the color temperature level, determine the color temperature prior probability of the color temperature level.
[0019] Based on the color temperature feature sample data corresponding to the color temperature level, determine the conditional probability of each feature value under the color temperature level.
[0020] In one implementation, based on color temperature feature sample data corresponding to color temperature levels, the conditional probability of each feature value at each color temperature level is determined, including:
[0021] Obtain the correction coefficients and the total number of eigenvalues for each of at least one feature; the total number of eigenvalues for a feature is the total number of eigenvalues contained in the eigenvalue set corresponding to that feature.
[0022] Based on the color temperature feature sample data corresponding to the color temperature level, the correction coefficient, and the total number of feature values for at least one feature, determine the conditional probability of each feature value under the color temperature level; the conditional probability is negatively correlated with the total number of feature values and the correction coefficient.
[0023] In one implementation, the target color temperature probability for each color temperature level is determined based on the prior color temperature probability of each color temperature level and the conditional probability of the eigenvalue of at least one feature at each color temperature level, including:
[0024] For each color temperature level, perform the following steps:
[0025] The feature probability is obtained by multiplying the conditional probabilities of the feature values of at least one feature under the color temperature level.
[0026] The product of the prior probability and the feature probability of the color temperature level is determined to obtain the target color temperature probability of the color temperature level.
[0027] In one implementation, the target color temperature level is obtained by weighted summation of the color temperature levels based on their respective target color temperature probabilities, including:
[0028] Obtain the initial weights corresponding to each color temperature level;
[0029] Based on the target color temperature probability of each color temperature level, the probability weight of each color temperature level is determined; the probability weight is positively correlated with the target color temperature probability.
[0030] The target weights for each color temperature level are obtained by multiplying the initial weights and probability weights for each color temperature level.
[0031] Based on the weights of each target, the color temperature levels are weighted and summed to obtain the target color temperature level.
[0032] In one implementation, before obtaining the target weight corresponding to each color temperature level based on the product of the initial weight and the probability weight of each color temperature level, the method further includes:
[0033] Obtain the adjustment coefficients corresponding to each probability weight;
[0034] Each probability weight is adjusted according to its corresponding adjustment coefficient.
[0035] In one aspect, this disclosure provides an image processing apparatus, comprising:
[0036] The acquisition unit is used to acquire target color temperature feature data; the target color temperature feature data includes at least one feature associated with color temperature and its corresponding feature value;
[0037] The determining unit is used to determine the target color temperature probability of each color temperature level based on the prior color temperature probability of each color temperature level and the conditional probability of the feature value of at least one feature under each color temperature level; the prior color temperature probability and the conditional probability are both determined based on multiple color temperature feature sample data and their corresponding color temperature levels.
[0038] The obtaining unit is used to perform a weighted summation of each color temperature level based on the target color temperature probability of each color temperature level, and obtain the target color temperature level.
[0039] The adjustment unit is used to adjust the white balance of the image to be processed according to the target color temperature level to obtain the target image.
[0040] In one embodiment, the acquisition unit is further configured to:
[0041] For each feature, perform the following steps:
[0042] Obtain the initial values of the features at multiple sampling times;
[0043] The initial values of the feature at multiple sampling times are smoothed to obtain the feature values.
[0044] In one embodiment, the determining unit is further configured to:
[0045] Obtain multiple color temperature feature sample data and their corresponding color temperature levels; the color temperature feature sample data includes at least one feature associated with color temperature and its corresponding feature value;
[0046] Count the total number of samples for multiple color temperature feature data;
[0047] For each color temperature level, perform the following steps:
[0048] Based on the total number of samples and the first sample number of the color temperature feature sample data corresponding to the color temperature level, determine the color temperature prior probability of the color temperature level.
[0049] Based on the color temperature feature sample data corresponding to the color temperature level, determine the conditional probability of each feature value under the color temperature level.
[0050] In one embodiment, the determining unit is further configured to:
[0051] Obtain the correction coefficients and the total number of eigenvalues for each of at least one feature; the total number of eigenvalues for a feature is the total number of eigenvalues contained in the eigenvalue set corresponding to that feature.
[0052] Based on the color temperature feature sample data corresponding to the color temperature level, the correction coefficient, and the total number of feature values for at least one feature, determine the conditional probability of each feature value under the color temperature level; the conditional probability is negatively correlated with the total number of feature values and the correction coefficient.
[0053] In one implementation, the determining unit is used to:
[0054] For each color temperature level, perform the following steps:
[0055] The feature probability is obtained by multiplying the conditional probabilities of the feature values of at least one feature under the color temperature level.
[0056] The product of the prior probability and the feature probability of the color temperature level is determined to obtain the target color temperature probability of the color temperature level.
[0057] In one embodiment, the obtaining unit is used to:
[0058] Obtain the initial weights corresponding to each color temperature level;
[0059] Based on the target color temperature probability of each color temperature level, the probability weight of each color temperature level is determined; the probability weight is positively correlated with the target color temperature probability.
[0060] The target weights for each color temperature level are obtained by multiplying the initial weights and probability weights for each color temperature level.
[0061] Based on the weights of each target, the color temperature levels are weighted and summed to obtain the target color temperature level.
[0062] In one embodiment, the obtaining unit is further configured to:
[0063] Obtain the adjustment coefficients corresponding to each probability weight;
[0064] Each probability weight is adjusted according to its corresponding adjustment coefficient.
[0065] In one aspect, this disclosure provides an electronic device, including:
[0066] Processor; and
[0067] The memory stores computer instructions that cause the processor to perform the steps of the methods provided in the various alternative implementations of any of the image processing described above.
[0068] In one aspect, this disclosure provides a computer-readable storage medium storing computer instructions for causing a computer to perform the steps of the methods provided in various alternative implementations of any of the above-described image processing methods.
[0069] On one hand, this disclosure provides a computer program product including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the steps of the method provided in various alternative implementations of any of the above-described image processing methods.
[0070] The image processing method in this embodiment includes acquiring target color temperature feature data; the target color temperature feature data includes at least one feature associated with color temperature and its corresponding feature value; determining the target color temperature probability for each color temperature level based on the prior probability of each color temperature level and the conditional probability of the feature value of each of the at least one feature under each color temperature level; both the prior probability and the conditional probability are determined based on multiple color temperature feature sample data and their corresponding color temperature levels; weighted summation of each color temperature level based on its target color temperature probability to obtain the target color temperature level; and white balance adjustment of the image to be processed based on the target color temperature level to obtain the target image. In this way, by combining environmental feature data to determine the target color temperature probability for each color temperature level, and by weighted fusion of each color temperature level using the target color temperature probability to obtain the target color temperature level, and then adjusting the white balance, the accuracy of color temperature estimation and white balance is improved, thereby improving image quality. Attached Figure Description
[0071] Figure 1 This is a flowchart of an image processing method according to an embodiment of the present disclosure.
[0072] Figure 2 This is a flowchart of an image imaging method according to an embodiment of the present disclosure.
[0073] Figure 3 This is an image of a fire hydrant in an embodiment of this disclosure.
[0074] Figure 4 This is a schematic diagram of another image inpainting system in an embodiment of this disclosure.
[0075] Figure 5 This is a structural block diagram of an image processing apparatus according to an embodiment of the present disclosure.
[0076] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. Detailed Implementation
[0077] The technical solutions of this disclosure will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure. Furthermore, the technical features involved in the different embodiments of this disclosure described below can be combined with each other as long as they do not conflict with each other.
[0078] The cone cells in the human eye can generate different stimulation values for different wavelengths of light, enabling humans to perceive various colors. Because non-luminous objects have different absorption and reflectivity for different colors of light, the color an object appears to be is usually affected by the color of the light source. Due to the color constancy of the human visual system, even under different lighting conditions, the color of the same object is perceived as the same.
[0079] However, because digital cameras lack the color reproduction capabilities of the human eye during shooting, the colors in images captured by digital cameras will deviate to some extent from the true colors of the scene under different lighting conditions. In order for people to obtain accurate image colors when taking pictures, cameras usually need to dynamically detect the color temperature of the ambient light and compensate for its influence, that is, to perform automatic white balance (AWB).
[0080] In practical applications, the AWB module in electronic devices typically measures the current color temperature using a color temperature sensor and adjusts the white balance accordingly, making the image appear as if it were taken under achromatic lighting. However, because reflected light from objects can affect the color temperature sensor's measurement, the measured color temperature usually contains errors, resulting in lower white balance accuracy.
[0081] Based on the deficiencies of the aforementioned related technologies, this disclosure provides an image processing method, apparatus, electronic device, storage medium, and program product, aiming to improve color temperature accuracy and white balance precision.
[0082] This disclosure provides an image processing method that can be applied to electronic devices. This disclosure does not limit the type of electronic device, which can be any suitable type of device, such as terminal devices and servers, etc. This disclosure will not elaborate further.
[0083] See Figure 1 The diagram shown is a flowchart of an image processing method according to an embodiment of this disclosure. The following is a description of the method in conjunction with... Figure 1 The method is described below, and the specific implementation process is as follows:
[0084] Step 101: Obtain target color temperature feature data; the target color temperature feature data includes at least one feature associated with color temperature and its corresponding feature value.
[0085] In one implementation, for each feature, the following steps are performed: obtaining the initial value of the feature at multiple sampling times, smoothing the initial value of the feature at multiple sampling times, and obtaining the feature value of the feature.
[0086] Optionally, the features may include, but are not limited to, at least one of the following: ambient statistical color temperature, ambient measured color temperature, foreground distance, rotational displacement, motion speed, and motion displacement. The initial values may be directly acquired by the sensor or obtained after preprocessing the acquired values. The sensor may include, but is not limited to, at least one of the following: a color temperature sensor, a ranging sensor, a gyroscope sensor, and a camera device. In practical applications, the sensor and features can be selected according to the actual application scenario, and no restrictions are imposed here.
[0087] For example, images can be captured using a camera device, and the ambient statistical color temperature can be determined based on the image information. As an example, a statistical white balance algorithm can be used. For each frame of the image, a gray-world algorithm or a white block algorithm is applied to estimate the color temperature and output the ambient statistical color temperature. In practical applications, the algorithm for determining the ambient statistical color temperature can be selected based on the specific application scenario; no restrictions are placed here.
[0088] For example, a color temperature sensor can be used to detect the spectrum of ambient light sources and output the measured ambient color temperature. Furthermore, preprocessing operations such as filtering and removing outliers in the measured ambient color temperature can be performed.
[0089] For example, a range sensor can be used to measure the distance between the camera and the foreground, i.e., the foreground distance. Optionally, the range sensor can be a red-green-blue (RGB) camera, a binocular camera, a structured light radar, or a time-of-flight (TOF) radar, etc.
[0090] Among these, the RGB camera can obtain distance using a monocular ranging model. Both binocular and structured light radars can obtain distance using a triangulation ranging model. Structured light radar and Time-of-Flight (TOF) radar can obtain distance using time-of-flight ranging.
[0091] For example, a gyroscope sensor can be used to measure the rotational displacement and acceleration per unit time, and the velocity and displacement can be obtained by integral calculation based on the acceleration.
[0092] Optionally, the data acquisition command can be issued when the image acquisition function is activated (e.g., when the mobile phone turns on the camera function), or it can be issued by the user.
[0093] The color temperature sensor is used for color temperature detection, which can obtain the ambient color temperature. The gyroscope sensor is used to detect the rotational displacement of the camera device. The images captured by the camera device can be used to calculate the distance between the camera device and the foreground.
[0094] Furthermore, time synchronization processing can be performed on each feature value.
[0095] In one implementation, the feature values of each feature can be linearly interpolated based on the timestamp, so that various types of information can be synchronized in time.
[0096] This is because the information acquisition and processing frequencies of different sensors are usually asynchronous. For example, there is no time synchronization between color temperature sensors and distance sensors. Therefore, this method can be used to achieve time synchronization.
[0097] In this way, more information can be obtained through multiple sensors, thereby improving the accuracy of color temperature estimation in subsequent steps.
[0098] Step 102: Determine the target color temperature probability for each color temperature level based on the prior probability of each color temperature level and the conditional probability of the feature value of at least one feature under each color temperature level. The prior probability and conditional probability of color temperature are determined based on multiple color temperature feature sample data and their corresponding color temperature levels.
[0099] In this embodiment of the disclosure, for ease of processing, the color temperature is divided into multiple color temperature ranges, and a corresponding color temperature level is set for each color temperature range.
[0100] For example, color temperature can be divided into the following nine color temperature ranges: LOW (0-1500K), H (1600-2400K), A (2500-3200K), CWF (3300-3700K), TL84 (3800-4400K), D50 (4500-5500K), D65 (5600-6800K), D75 (6900-7800K), and High (8000-12000K).
[0101] In one implementation, the following steps can be used to determine the prior probability and conditional probability of color temperature:
[0102] S1021: Obtain multiple color temperature feature sample data and their corresponding color temperature levels. The color temperature feature sample data includes at least one feature associated with color temperature and its corresponding feature value.
[0103] If there are multiple color temperature feature sample data, the index of the color temperature feature sample data can be represented as i, and the i-th color temperature feature sample data can be represented as x. i The color temperature level corresponding to the i-th color temperature feature sample data can be represented as y. i A sample data T containing multiple color temperature feature samples and their corresponding color temperature levels can be represented as T = {(x1, y1), (x2, y2), ..., (x...}. N y N )}, where N is the total number of samples, and i and N are positive integers.
[0104] The j-th feature in the i-th color temperature feature sample data can be represented as: For example, the i-th color temperature feature sample data n is the number of features, and j and n are both positive integers.
[0105] This represents the range of values for the j-th feature in the i-th color temperature feature sample data, i.e., the set of feature values. jt S represents the t-th eigenvalue in the set of eigenvalues. j Let t and S be the total number of eigenvalues in the eigenvalue set of the j-th feature. j All are positive integers.
[0106] y i ∈(c1, c2, ..., c M ), representing the range of color temperature levels to which the color temperature level corresponding to the i-th color temperature feature sample data belongs, i.e., the set of color temperature levels. c k This represents the k-th color temperature level, and M represents the number of color temperature levels, such as 9. Both M and k are positive integers.
[0107] S1022: Count the total number N of multiple color temperature feature sample data.
[0108] S1023: For each color temperature level, perform the following steps:
[0109] S1023-1: Determine the prior probability of color temperature for each color temperature level based on the total number of samples and the first sample number of the color temperature feature sample data corresponding to the color temperature level.
[0110] In one implementation, the prior probability of color temperature level ck is determined. When this is the case, the following formula can be used:
[0111]
[0112] Where I() represents the quantity.
[0113] S1023-2: Based on the color temperature feature sample data corresponding to the color temperature level, determine the conditional probability of each feature value under the color temperature level.
[0114] In one implementation, the first sample size of color temperature feature sample data corresponding to the color temperature level is determined, and statistics are performed. And = c k The second sample size of the color temperature feature sample data, and the conditional probability obtained based on the ratio of the first sample size to the second sample size.
[0115] Optional, confirm The eigenvalue is a jt At color temperature level c k When calculating the conditional probability under certain circumstances, the following formula can be used:
[0116]
[0117] Furthermore, to prevent information carried by other attributes from being erased by attribute values that do not appear in the sample data, conditional probabilities can be corrected using Laplacian correction.
[0118] In one implementation, when correcting the conditional probability, the following steps can be performed separately for each color temperature level:
[0119] S1023-21: Obtain the correction coefficient λ and the total number of eigenvalues S for each of at least one feature. j The total number of eigenvalues of a feature is the total number of eigenvalues contained in the eigenvalue set corresponding to that feature.
[0120] S1023-22: Based on the color temperature feature sample data corresponding to the color temperature level, the correction coefficient, and the total number of feature values for at least one feature, determine the conditional probability of each feature value under the color temperature level; the conditional probability is negatively correlated with the total number of feature values and the correction coefficient.
[0121] Optionally, the optimized conditional probability can be expressed using the following formula:
[0122]
[0123] Where λ is the correction coefficient, S j Let be the total number of feature values for the j-th feature.
[0124] In one implementation, when performing step 102, the following steps can be performed separately for each color temperature level:
[0125] The feature probability is obtained by multiplying the conditional probabilities of the feature values of at least one feature under the color temperature level; the target color temperature probability of the color temperature level is obtained by multiplying the prior color temperature probability of the color temperature level with the feature probability.
[0126] For example, in an application scenario, the feature value of the j-th feature in the target color temperature feature data can be represented as d. (j) The target color temperature feature data can be represented as x = (d (1) d (2) , ..., d (n) ). Optional, determine the color temperature level c. k When determining the probability of the target color temperature, the following formula can be used:
[0127]
[0128] Furthermore, since different sensors typically have different accuracies, different feature weights can be set for different features, and for each feature, the product of the feature weight and the conditional probability can be used as the adjusted conditional probability.
[0129] In this embodiment of the disclosure, multiple features are used as conditions to estimate the corresponding conditional probabilities. Different features are independent of each other and will not interfere with each other. Therefore, the target color temperature probability of each color temperature level can be estimated by combining the conditional probabilities of each feature.
[0130] Step 103: Based on the target color temperature probability of each color temperature level, perform a weighted summation of each color temperature level to obtain the target color temperature level.
[0131] In one implementation, step 103 may be performed using the following steps:
[0132] S1031: Obtain the initial weights corresponding to each color temperature level.
[0133] Optionally, initial weights can be set manually based on experience, or statistical points for each color temperature level can be collected using a statistical white balance method, and the initial weights can be set based on the statistical results. In practical applications, the initial weights can be set according to the actual application scenario, and there are no restrictions here.
[0134] S1032: Determine the probability weight of each color temperature level based on the target color temperature probability of each color temperature level; the probability weight is positively correlated with the target color temperature probability.
[0135] Furthermore, the probability weights can be adjusted. In one implementation, the adjustment coefficient corresponding to each probability weight is obtained; and each probability weight is adjusted separately according to its corresponding adjustment coefficient.
[0136] Optional, adjusted color temperature level c k The probability weights can be w 2k *b k , where w 2k Color temperature level c k The probability weights, b k Color temperature level c k Adjustment coefficient.
[0137] This allows for fine-tuning of the probability weights to improve the accuracy of subsequent color temperature estimation.
[0138] S1033: Obtain the target weight corresponding to each color temperature level based on the product of the initial weight and the probability weight of each color temperature level.
[0139] In one implementation, the color temperature level c is determined. k Target weight W k When this is the case, the following formula can be used:
[0140] W k =w 1k *w 2k ;
[0141] Among them, w 1k Color temperature level c k The initial weights, w 2k Color temperature level c k The probability weights.
[0142] In this way, the initial weights set by human experience and the probability weights obtained based on the target color temperature feature data can be combined.
[0143] S1034: Based on the target weights, perform a weighted summation of each color temperature level to obtain the target color temperature level.
[0144] In one implementation, the following formula can be used to determine the target color temperature level (CCT):
[0145]
[0146] Furthermore, steps 101-103 can be executed repeatedly to obtain multiple target color temperature levels. Based on these multiple target color temperature levels, a new target color temperature level can be calculated. In this way, by treating ambient color temperature as a continuous state variable, and by continuously collecting external color temperature characteristic data and smoothing this data, the ambient color temperature can be dynamically corrected to approximate its true value. This results in more continuous and smooth color temperature changes, reducing sudden color temperature fluctuations.
[0147] Step 104: Adjust the white balance of the image to be processed according to the target color temperature level to obtain the target image.
[0148] In one implementation, a target color temperature range corresponding to the target color temperature level can be obtained, and a target color temperature can be selected based on the target color temperature range. Furthermore, white balance adjustments can be made to the image to be processed based on the target color temperature to obtain the target image. For example, the target color temperature can be the average or median value of the target color temperature range.
[0149] This disclosure applies to the AWB decision-making stage in the image imaging process, as described below. Figure 2 This section explains the image imaging process. (See also...) Figure 2 The diagram shows a flowchart of an image imaging method. The implementation process of this method includes:
[0150] Step 201: Perform image acquisition to obtain the acquired image.
[0151] One implementation scheme allows for image acquisition using a sensor with a color filter array.
[0152] Step 202: Perform gain processing on the image.
[0153] One implementation group can perform International Organization for Standardization (ISO) gain on the image to be processed, as well as raw-image processing.
[0154] Step 203: Perform RGB mosaic removal on the image.
[0155] Demoasicing is a technique in digital image processing, primarily used to convert monochrome channel image data into a full-color image.
[0156] Step 204: Perform model recognition and processing on the image.
[0157] Pattern recognition and processing (PRC) is an important branch of computer science and artificial intelligence, primarily involving the identification of meaningful patterns or regularities from data and their further analysis and processing. This field has a wide impact in many practical applications, such as image recognition, speech recognition, and natural language processing. The core concepts, key technologies, and applications of PRC will be detailed below. Pattern recognition: refers to the automatic identification of patterns and regularities in data through algorithms and techniques. For example, in image recognition, the algorithm needs to learn how to distinguish different image categories. Data processing: involves the process of collecting, cleaning, transforming, and analyzing data so that pattern recognition algorithms can work more effectively.
[0158] Step 205: Adjust the white balance and convert the color space of the image.
[0159] AWB (Auto-Aspect Ratio) is a method for adjusting image colors to adapt to different lighting conditions. Its purpose is to ensure that whites or other neutral tones in an image remain white or neutral under different light sources, thereby correcting overall color deviations. Color space transformation is the process of converting an image from one color space to another.
[0160] When performing AWB on the image, the specific steps are as described in steps 101-104 above, and there are no restrictions here.
[0161] Step 206: Perform color manipulation on the image.
[0162] Color manipulation is a key technology in digital image processing. It involves changing and adjusting the color attributes of an image to achieve specific visual effects or extract image features.
[0163] Step 207: Map the image to standard RGB output.
[0164] Mapping to standard sRGB output typically involves using a color matrix or lookup table to achieve accurate color mapping, ensuring that images are rendered correctly on various display devices. sRGB is a standard color space widely used in monitors, cameras, and internet images. It defines how digital signals are converted into visible light, resulting in images with consistent visual quality across different devices. This usually involves using a color matrix or lookup table to achieve accurate color mapping.
[0165] Step 208: Compress the image.
[0166] Optionally, the image can be compressed using the Joint Photographic Experts Group (JPEG) image file format.
[0167] Step 209: Save the compressed file.
[0168] The following is combined Figure 3 and Figure 4 The above embodiments are illustrated by examples. See also... Figure 3 The image shown is of a fire hydrant. (See attached image.) Figure 4 The image shown is of another type of fire hydrant. Figure 3 This is an image of a fire hydrant taken outdoors, partially encompassing the land. Using relevant techniques, a statistical white balance estimation method is typically employed, determining the ambient color temperature based on the color of the land. When photographing the fire hydrant, moving the lens upwards will capture an image that does not include the aforementioned land. Figure 4.because Figure 4 Since the aforementioned land is not included in the data, statistical white balance estimation methods may incorrectly determine the color temperature of the current environment, resulting in poor image white balance. However, the method described in this embodiment can accurately estimate the white balance. Figure 4 By adjusting the color temperature level of the ambient environment, the white balance can be accurately adjusted to obtain high-quality images and achieve better results.
[0169] In related technologies, statistical white balance estimation methods or color temperature sensors are typically used to determine the ambient color temperature. However, these methods often result in significant color temperature errors. In this embodiment, a Bayesian approach is used to estimate the probability of color temperature levels, which is then converted into probability weights. These probability weights, combined with initial weights, are used to estimate the target color temperature level, thereby optimizing the white balance of the camera device and improving the accuracy of color temperature estimation. Furthermore, the use of multiple sensors for feature data acquisition provides richer information sources, further enhancing the accuracy of color temperature estimation. Moreover, by setting different weights for the feature data from different sensors, the weight of feature data from high-precision sensors can be enhanced, thereby improving the practicality and accuracy of color temperature estimation. This improves the accuracy, stability, and robustness of color temperature estimation, enabling real-time adjustment and optimization of white balance, and ultimately improving image quality.
[0170] Based on the same inventive concept, this disclosure also provides an image processing apparatus. Since the principle of the above-described apparatus and device in solving the problem is similar to that of an image processing method, the implementation of the above-described apparatus can refer to the implementation of the method, and repeated details will not be elaborated further. This apparatus can be applied to electronic devices. This disclosure does not limit the type of electronic device; it can be any suitable type of device, such as terminal devices and servers, etc., which will not be elaborated further in this disclosure. The apparatus embodiment can be implemented by software, or by hardware, or a combination of software and hardware. Taking software implementation as an example, as a logically defined apparatus, it is formed by the processor of the electronic device loading the corresponding computer program instructions from non-volatile memory into memory for execution.
[0171] See Figure 5 The diagram shown is a structural block diagram of an image processing apparatus according to an embodiment of this disclosure. In some embodiments, the image processing apparatus of this disclosure includes:
[0172] The acquisition unit 501 is used to acquire target color temperature feature data; the target color temperature feature data includes at least one feature associated with color temperature and its corresponding feature value;
[0173] The determining unit 502 is used to determine the target color temperature probability of each color temperature level based on the prior color temperature probability of each color temperature level and the conditional probability of the feature value of at least one feature under each color temperature level; the prior color temperature probability and the conditional probability are both determined based on multiple color temperature feature sample data and their corresponding color temperature levels.
[0174] The obtaining unit 503 is used to perform a weighted summation of each color temperature level based on the target color temperature probability of each color temperature level to obtain the target color temperature level;
[0175] The adjustment unit 504 is used to adjust the white balance of the image to be processed according to the target color temperature level to obtain the target image.
[0176] In one embodiment, the acquisition unit 501 is further configured to:
[0177] For each feature, perform the following steps:
[0178] Obtain the initial values of the features at multiple sampling times;
[0179] The initial values of the feature at multiple sampling times are smoothed to obtain the feature values.
[0180] In one embodiment, the determining unit 502 is further configured to:
[0181] Obtain multiple color temperature feature sample data and their corresponding color temperature levels; the color temperature feature sample data includes at least one feature associated with color temperature and its corresponding feature value;
[0182] Count the total number of samples for multiple color temperature feature data;
[0183] For each color temperature level, perform the following steps:
[0184] Based on the total number of samples and the first sample number of the color temperature feature sample data corresponding to the color temperature level, determine the color temperature prior probability of the color temperature level.
[0185] Based on the color temperature feature sample data corresponding to the color temperature level, determine the conditional probability of each feature value under the color temperature level.
[0186] In one embodiment, the determining unit 502 is further configured to:
[0187] Obtain the correction coefficients and the total number of eigenvalues for each of at least one feature; the total number of eigenvalues for a feature is the total number of eigenvalues contained in the eigenvalue set corresponding to that feature.
[0188] Based on the color temperature feature sample data corresponding to the color temperature level, the correction coefficient, and the total number of feature values for at least one feature, determine the conditional probability of each feature value under the color temperature level; the conditional probability is negatively correlated with the total number of feature values and the correction coefficient.
[0189] In one embodiment, the determining unit 502 is used to:
[0190] For each color temperature level, perform the following steps:
[0191] The feature probability is obtained by multiplying the conditional probabilities of the feature values of at least one feature under the color temperature level.
[0192] The product of the prior probability and the feature probability of the color temperature level is determined to obtain the target color temperature probability of the color temperature level.
[0193] In one embodiment, the obtaining unit 503 is used for:
[0194] Obtain the initial weights corresponding to each color temperature level;
[0195] Based on the target color temperature probability of each color temperature level, the probability weight of each color temperature level is determined; the probability weight is positively correlated with the target color temperature probability.
[0196] The target weights for each color temperature level are obtained by multiplying the initial weights and probability weights for each color temperature level.
[0197] Based on the weights of each target, the color temperature levels are weighted and summed to obtain the target color temperature level.
[0198] In one embodiment, the obtaining unit 503 is further configured to:
[0199] Obtain the adjustment coefficients corresponding to each probability weight;
[0200] Each probability weight is adjusted according to its corresponding adjustment coefficient.
[0201] The image processing method in this embodiment includes acquiring target color temperature feature data; the target color temperature feature data includes at least one feature associated with color temperature and its corresponding feature value; determining the target color temperature probability for each color temperature level based on the prior probability of each color temperature level and the conditional probability of the feature value of each of the at least one feature under each color temperature level; both the prior probability and the conditional probability are determined based on multiple color temperature feature sample data and their corresponding color temperature levels; weighted summation of each color temperature level based on its target color temperature probability to obtain the target color temperature level; and white balance adjustment of the image to be processed based on the target color temperature level to obtain the target image. In this way, by combining environmental feature data to determine the target color temperature probability for each color temperature level, and by weighted fusion of each color temperature level using the target color temperature probability to obtain the target color temperature level, and then adjusting the white balance, the accuracy of color temperature estimation and white balance is improved, thereby improving image quality.
[0202] In this embodiment of the disclosure, an electronic device is also provided, including:
[0203] Processor; and
[0204] The memory stores computer instructions that cause the processor to execute the methods of any of the above-described embodiments.
[0205] In this embodiment of the disclosure, a computer-readable storage medium is provided, storing computer instructions for causing a computer to perform the methods of any of the above embodiments.
[0206] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the method described in any of the above embodiments.
[0207] Figure 6 A schematic diagram of the structure of an electronic device 6000 is shown. (See also...) Figure 6 As shown, the electronic device 6000 includes a processor 6010 and a memory 6020, and optionally may also include a power supply 6030, a display unit 6040, and an input unit 6050.
[0208] The processor 6010 is the control center of the electronic device 6000. It connects various components through various interfaces and lines, and performs various functions of the electronic device 6000 by running or executing software programs and / or data stored in the memory 6020, thereby performing overall monitoring of the electronic device 6000.
[0209] In this embodiment of the present disclosure, the processor 6010 executes the steps in the above embodiments when it calls the computer program stored in the memory 6020.
[0210] Optionally, the processor 6010 may include one or more processing units; preferably, the processor 6010 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 6010. In some embodiments, the processor and memory may be implemented on a single chip; in some embodiments, they may also be implemented separately on independent chips.
[0211] The memory 6020 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, various applications, etc.; the data storage area may store data created based on the use of the electronic device 6000, etc. In addition, the memory 6020 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device, etc.
[0212] Electronic device 6000 also includes a power supply 6030 (such as a battery) that supplies power to various components. The power supply can be logically connected to processor 6010 through a power management system, thereby enabling the management of charging, discharging, and power consumption.
[0213] The display unit 6040 can be used to display information input by the user or information provided to the user, as well as various menus of the electronic device 6000. In this embodiment, it is mainly used to display the display interfaces of various applications in the electronic device 6000, as well as text, images, and other objects displayed on the display interfaces. The display unit 6040 may include a display panel 6041. The display panel 6041 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.
[0214] The input unit 6050 can be used to receive information such as numbers or characters input by the user. The input unit 6050 may include a touch panel 6051 and other input devices 6052. The touch panel 6051, also known as a touch screen, can collect touch operations on or near the touch panel 6051 by the user (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 6051).
[0215] Specifically, the touch panel 6051 can detect user touch operations and the signals generated by these operations, convert them into touch point coordinates, send them to the processor 6010, and receive and execute commands from the processor 6010. Furthermore, the touch panel 6051 can be implemented using various types of touch technologies, including resistive, capacitive, infrared, and surface acoustic wave. Other input devices 6052 can include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.
[0216] Of course, the touch panel 6051 can cover the display panel 6041. When the touch panel 6051 detects a touch operation on or near it, it transmits the information to the processor 6010 to determine the type of touch event. Subsequently, the processor 6010 provides corresponding visual output on the display panel 6041 based on the type of touch event. Although in Figure 6 In this embodiment, the touch panel 6051 and the display panel 6041 are two separate components to realize the input and output functions of the electronic device 6000. However, in some embodiments, the touch panel 6051 and the display panel 6041 can be integrated to realize the input and output functions of the electronic device 6000.
[0217] The electronic device 6000 may also include one or more sensors, such as a pressure sensor, a gravity acceleration sensor, a proximity sensor, etc. Of course, depending on the specific application, the electronic device 6000 may also include other components such as a camera. Since these components are not the focus of this disclosure, they will not be discussed further. Figure 6 It is not shown in the text and will not be described in detail here.
[0218] Those skilled in the art will understand that Figure 6 This is merely an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than shown, or a combination of certain components, or different components.
[0219] For ease of description, the above sections are divided into modules (or units) according to their functions and described separately. Of course, in implementing this disclosure, the functions of each module (or unit) can be implemented in one or more software or hardware components.
Claims
1. An image processing method, characterized in that, The method includes: Obtain target color temperature feature data; the target color temperature feature data includes at least one feature associated with color temperature and its corresponding feature value; The target color temperature probability for each color temperature level is determined based on the prior color temperature probability of each color temperature level and the conditional probability of the feature value of each of the at least one feature under each color temperature level; the prior color temperature probability and the conditional probability are both determined based on multiple color temperature feature sample data and their corresponding color temperature levels. Based on the target color temperature probability of each color temperature level, the target color temperature level is obtained by weighted summation of the color temperature levels. Based on the target color temperature level, the white balance of the image to be processed is adjusted to obtain the target image.
2. The method according to claim 1, characterized in that, Before acquiring the target color temperature feature data, the method further includes: For each feature, perform the following steps: Obtain the initial values of the feature at multiple sampling times; The initial values of the feature at multiple sampling times are smoothed to obtain the feature values of the feature.
3. The method according to claim 1 or 2, characterized in that, Before determining the target color temperature probability for each color temperature level based on the prior color temperature probability of each color temperature level and the conditional probability of the eigenvalue of each of the at least one feature at each color temperature level, the method further includes: Obtain multiple color temperature feature sample data and their corresponding color temperature levels; the color temperature feature sample data includes at least one feature associated with color temperature and its corresponding feature value; Count the total number of samples for multiple color temperature feature data; For each color temperature level, perform the following steps: Based on the total number of samples and the first sample number of the color temperature feature sample data corresponding to the color temperature level, the color temperature prior probability of the color temperature level is determined. Based on the color temperature feature sample data corresponding to the color temperature level, determine the conditional probability of each feature value under the given color temperature level.
4. The method according to claim 3, characterized in that, The step of determining the conditional probability of each feature value under the given color temperature level based on the color temperature feature sample data corresponding to the color temperature level includes: Obtain the correction coefficient and the total number of feature values for each of the at least one feature; the total number of feature values for a feature is the total number of feature values contained in the feature value set corresponding to that feature. Based on the color temperature feature sample data corresponding to the color temperature level, the correction coefficient, and the total number of feature values for each of the at least one feature, the conditional probability corresponding to each feature value under the color temperature level is determined; the conditional probability is negatively correlated with both the total number of feature values and the correction coefficient.
5. The method according to claim 1 or 2, characterized in that, Based on the prior color temperature probability of each color temperature level and the conditional probability of the eigenvalue of each of the at least one feature at each color temperature level, the target color temperature probability of each color temperature level is determined, including: For each color temperature level, perform the following steps: The feature probability is obtained by multiplying the conditional probabilities of the feature values of the at least one feature under the color temperature level. The product of the prior probability of the color temperature at the given color temperature level and the feature probability is determined to obtain the target color temperature probability of the given color temperature level.
6. The method according to claim 1 or 2, characterized in that, The step of obtaining the target color temperature level by weighted summation of each color temperature level based on its respective target color temperature probability includes: Obtain the initial weights corresponding to each color temperature level; Based on the target color temperature probability of each color temperature level, a probability weight for each color temperature level is determined; the probability weight is positively correlated with the target color temperature probability. The target weights for each color temperature level are obtained by multiplying the initial weights and probability weights for each color temperature level. Based on the target weights, the color temperature levels are weighted and summed to obtain the target color temperature level.
7. The method according to claim 6, characterized in that, Before obtaining the target weight corresponding to each color temperature level based on the product of the initial weight and the probability weight of each color temperature level, the method further includes: Obtain the adjustment coefficients corresponding to each probability weight; Each probability weight is adjusted according to its corresponding adjustment coefficient.
8. An image processing apparatus, characterized in that, The device includes: An acquisition unit is used to acquire target color temperature feature data; the target color temperature feature data includes at least one feature associated with color temperature and its corresponding feature value; The determining unit is used to determine the target color temperature probability of each color temperature level based on the prior color temperature probability of each color temperature level and the conditional probability of the feature value of each of the at least one feature under each color temperature level; the prior color temperature probability and the conditional probability are both determined based on multiple color temperature feature sample data and their corresponding color temperature levels. The obtaining unit is used to perform a weighted summation of each color temperature level based on the target color temperature probability of each color temperature level, and obtain the target color temperature level. The adjustment unit is used to adjust the white balance of the image to be processed according to the target color temperature level to obtain the target image.
9. An electronic device, characterized in that, include: processor; as well as A memory storing computer instructions for causing the processor to perform the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer contains computer instructions for causing the computer to perform the method according to any one of claims 1 to 7.
11. A computer program product, characterized in that, Includes computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is executed in a processor of an electronic device, the processor in the electronic device is the method according to any one of claims 1 to 7.