Image processing method, image processing device and storage medium

Abstract

The present disclosure relates to an image processing method, an image processing device, and a storage medium. The image processing method is applied to a terminal, and the image processing method includes: when determining that a target color temperature scene exists in an image to be processed, using a first gain to perform color temperature compensation on the image to be processed to obtain a first compensated image, and using a second gain to perform color temperature compensation on the image to be processed to obtain a second compensated image, the first gain corresponding to a first color temperature, and the second gain corresponding to a second color temperature; determining a color deviation value of the target compensated image, the target compensated image being the first compensated image or the second compensated image; based on the color deviation value of the target compensated image, performing a fusion process on the first compensated image and the second compensated image to obtain a fused image. The present disclosure can improve the transition effect and compensation accuracy at the boundary between high and low color temperatures when performing color temperature compensation.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, an image processing device, and a storage medium. Background Art

[0002] When a camera captures an object of the same color, the object's color will appear different under different lighting conditions. Therefore, automatic white balance (AWB) processing is required to address color distortion. Currently, the white balance algorithms of most computing platforms are still limited to global red, green, and blue (RGB) gain compensation, which is limited to mixed light sources, especially high and low color temperature scenes. Engineers are forced to sacrifice compensation for lower-priority color temperatures based on their preferred priorities. Summary of the Invention

[0003] To overcome the problems existing in the related art, the present disclosure provides an image processing method, an image processing device, and a storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, including:

[0005] When it is determined that there is a target color temperature scene in the image to be processed, a first gain is used to perform color temperature compensation on the image to be processed to obtain a first compensated image, and a second gain is used to perform color temperature compensation on the image to be processed to obtain a second compensated image, the first gain corresponds to a first color temperature, and the second gain corresponds to a second color temperature, wherein the existence of a color temperature conflict includes that the image to be processed has a first color temperature and a second color temperature, both of which have color temperature proportions greater than a first threshold, and the difference between the color temperature proportion of the first color temperature and the color temperature proportion of the second color temperature is less than a second threshold, and the first threshold is greater than the second threshold; the color deviation value of the target compensated image is determined, and the target compensated image is the first compensated image or the second compensated image; based on the color deviation value of the target compensated image, the first compensated image and the second compensated image are fused to obtain a fused image.

[0006] In one embodiment, determining the color deviation value of the target compensation image includes:

[0007] Determine the color gamut data of the target compensation image in the HSV color gamut space; call a model for calculating the color deviation value; input the color gamut data of the target compensation image in the HSV color gamut space into the model to obtain the color deviation value of the target compensation image.

[0008] In one embodiment, the model is pre-trained in the following manner, including:

[0009] Acquire sample images with different degrees of color deviation, the sample images being obtained by adjusting red gain or blue gain, and at least one of the red gain and blue gain corresponding to different sample images being different; mark the sample images with different degrees of color deviation with color deviation values ​​to obtain marked color deviation values; determine color gamut data of the sample images with different degrees of color deviation in an HSV color gamut space to obtain color gamut sample data in the HSV color gamut space;

[0010] The color gamut sample data is used as input features, and the marked color deviation value is used as an output feature, and a model for calculating the color deviation value is obtained through training.

[0011] In one embodiment, the color gamut sample data is used as an input feature, and the marked color deviation value is used as an output feature to train a model for calculating the color deviation value, including:

[0012] Based on the regression model, the color gamut sample data and the marked color deviation value are trained until the regression model converges, and the converged regression model is determined as a model for calculating the color deviation value.

[0013] In one embodiment, the fusing of the first compensation image and the second compensation image based on the color deviation value of the target compensation image includes:

[0014] The color deviation value of the target compensation image is used as the first weight corresponding to the target compensation image; based on the first weight, a second weight is determined, and the sum of the second weight and the first weight is 1; based on the first weight and the second weight, the first compensation image and the second compensation image are fused.

[0015] In one embodiment, determining whether the image to be processed has a target color temperature scene includes:

[0016] Obtaining color temperature statistical information of the image to be processed; and determining, based on the color temperature statistical information, whether the image to be processed has the target color temperature scene.

[0017] In one embodiment, a color temperature difference between the first color temperature and the second color temperature is greater than a third threshold.

[0018] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing apparatus, including:

[0019] A compensation unit is used to determine that when a target color temperature scene exists in an image to be processed, perform color temperature compensation on the image to be processed using a first gain to obtain a first compensated image, and perform color temperature compensation on the image to be processed using a second gain to obtain a second compensated image, the first gain corresponding to the first color temperature, and the second gain corresponding to the second color temperature, wherein the existence of a color temperature conflict includes the presence of a first color temperature and a second color temperature in the image to be processed, both of which have color temperature proportions greater than a first threshold, and the difference between the color temperature proportions of the first color temperature and the color temperature proportions of the second color temperature is less than a second threshold, and the first threshold is greater than the second threshold; a processing unit is used to determine a color deviation value of a target compensated image, the target compensated image being the first compensated image or the second compensated image; a fusion unit is used to fuse the first compensation image and the second compensation image based on the color deviation value of the target compensated image to obtain a fused image.

[0020] In one embodiment, the processing unit determines the color deviation value of the target compensation image in the following manner:

[0021] Determine the color gamut data of the target compensation image in the HSV color gamut space; call a model for calculating the color deviation value; input the color gamut data of the target compensation image in the HSV color gamut space into the model to obtain the color deviation value of the target compensation image.

[0022] In one embodiment, the image processing apparatus further includes a training unit, which pre-trains the model in the following manner:

[0023] Sample images with different degrees of color deviation are obtained, where the sample images are obtained by adjusting red gain or blue gain, and at least one of the red gain and blue gain corresponding to different sample images is different; the sample images with different degrees of color deviation are marked with color deviation values ​​to obtain marked color deviation values; color gamut data of the sample images with different degrees of color deviation in an HSV color gamut space are determined to obtain color gamut sample data in the HSV color gamut space; and a model for calculating color deviation values ​​is trained using the color gamut sample data as input features and the marked color deviation values ​​as output features.

[0024] In one embodiment, the training unit uses the color gamut sample data as input features and the marked color deviation values ​​as output features to train a model for calculating color deviation values ​​in the following manner:

[0025] Based on the regression model, the color gamut sample data and the marked color deviation value are trained until the regression model converges, and the converged regression model is determined as a model for calculating the color deviation value.

[0026] In one embodiment, the fusion unit fuses the first compensation image and the second compensation image based on the color deviation value of the target compensation image in the following manner:

[0027] The color deviation value of the target compensation image is used as the first weight corresponding to the target compensation image; based on the first weight, a second weight is determined, and the sum of the second weight and the first weight is 1; based on the first weight and the second weight, the first compensation image and the second compensation image are fused.

[0028] In one embodiment, the image processing apparatus further includes a determining unit, which determines whether the image to be processed has a target color temperature scene in the following manner:

[0029] Obtaining color temperature statistical information of the image to be processed; and determining, based on the color temperature statistical information, whether the image to be processed has the target color temperature scene.

[0030] In one embodiment, a color temperature difference between the first color temperature and the second color temperature is greater than a third threshold.

[0031] According to a third aspect of an embodiment of the present disclosure, there is provided an image processing apparatus, including:

[0032] processor;

[0033] a memory for storing processor-executable instructions;

[0034] The processor is configured to: execute the method described in the first aspect or any one of the embodiments of the first aspect.

[0035] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein instructions are stored in the storage medium. When the instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to execute the method described in the first aspect or any one of the embodiments of the first aspect.

[0036] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: based on the acquired color temperature statistical information of the image to be processed, after determining that there is a color temperature conflict, the image is compensated, the color deviation value of the target compensated image is determined, and the image is fused to achieve improved transition effects and compensation accuracy at the boundaries of high and low color temperatures, as well as resolve color temperature conflict scenarios.

[0037] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0039] Figure 1 The figure is a flowchart of an image processing method according to an exemplary embodiment.

[0040] Figure 2 The figure is a flowchart of an image processing method according to an exemplary embodiment.

[0041] Figure 3 The figure is a flow chart of a method for training a model for calculating color deviation values ​​according to an exemplary embodiment.

[0042] Figure 4 The figure is a flowchart of an image processing method according to an exemplary embodiment.

[0043] Figure 5 is a schematic diagram showing an image processing method according to an exemplary embodiment.

[0044] Figure 6 is a block diagram of an image processing apparatus according to an exemplary embodiment.

[0045] Figure 7 The figure is a block diagram of an image processing apparatus according to an exemplary embodiment. DETAILED DESCRIPTION

[0046] Exemplary embodiments are described in detail herein, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numerals in different drawings represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present disclosure.

[0047] The image processing method provided by the present disclosure is applicable to scenarios where white balance algorithms are used. For example, the image processing method provided by the present disclosure can be applied to local automatic white balance (AWB) scenarios. In local AWB scenarios, the method is mainly based on segmentation and post-segmentation block compensation. However, segmentation can result in errors, leading to deviations in color temperature compensation. Furthermore, even if segmentation is accurate, compensation discontinuities are likely to occur at color temperature boundaries.

[0048] In view of this, an embodiment of the present disclosure provides an image processing method, in which, when performing white balance processing, high color temperature scenes and low color temperature scenes in scenes with color temperature conflicts are compensated separately, and then the compensated images are fused based on the degree of color deviation to achieve continuity of compensation at the color temperature boundary.

[0049] Figure 1 is a flowchart of an image processing method according to an exemplary embodiment. Figure 1 As shown, the image processing method is used in a terminal and includes the following steps.

[0050] In step S11 , when it is determined that the image to be processed has a target color temperature scene, the image to be processed is color-temperature compensated using a first gain to obtain a first compensated image, and the image to be processed is color-temperature compensated using a second gain to obtain a second compensated image.

[0051] The image to be processed in the present disclosure can be understood as an image that needs to be white balanced. The image to be white balanced can be an image to be white balanced taken by a terminal camera.

[0052] In the present disclosure, color temperature statistical information of an image to be processed is obtained. Based on the color temperature statistical information, it is determined that the image to be processed contains a target color temperature scene. A first gain is used to perform color temperature compensation on the image to be processed to obtain a first compensated image. A second gain is used to perform color temperature compensation on the image to be processed to obtain a second compensated image. Two images with different color temperature compensations are obtained to achieve local white balance processing of the image. The color temperature statistical information of the image to be processed can be statistical landing point data of the RGB values ​​of each segmented area after the image to be processed is segmented, within a chromaticity plane.

[0053] In the present disclosure, a frame of an image to be processed is taken and divided into M×N segmented areas. The R / G value and the B / G value are calculated for the RGB value of each segmented area. The R / G value and the B / G value are regarded as a whole as the coordinates of a color temperature statistical point of the segmented area on the chromaticity plane. The position of the segmented area on the chromaticity plane is determined, and then the color temperature of each segmented area is determined according to the color temperature curve on the chromaticity plane.

[0054] In the present disclosure, the presence of a target color temperature scene includes the presence of a first color temperature ratio and a second color temperature ratio in the image to be processed, wherein the sum of the color temperature ratios of the first color temperature and the second color temperature ratio is greater than a first threshold, the difference between the color temperature ratios of the first color temperature and the second color temperature ratio is less than a second threshold, and the first threshold is greater than the second threshold. The first threshold is 80%, and the second threshold is 40%.

[0055] The color temperature difference between the first color temperature and the second color temperature is greater than a third threshold value, and the third threshold value can be set according to specific practical application conditions.

[0056] Among them, the first color temperature can be understood as falling within the first color temperature range in the chromaticity plane, such as a low color temperature with a chromaticity of red below 3300K. The second color temperature can be understood as falling within the second color temperature range in the chromaticity plane, such as a high color temperature with a chromaticity of bluish above 5000K. The color temperature proportion refers to the proportion of the number of color temperature statistical points on the chromaticity plane within the same color temperature range to the number of color temperature statistical points on the chromaticity plane of the entire image to be processed.

[0057] In one embodiment of the present disclosure, the third threshold may be the difference between the minimum value of the low color temperature in the chromaticity plane and the maximum value of the high color temperature in the chromaticity plane. For example, in the chromaticity plane, the minimum value of the high color temperature is 5000K and the maximum value of the low color temperature is 3300K. The third threshold may be 5000-3300=1700K. Only when the difference between the first color temperature and the second color temperature is greater than 1700K, the image to be processed may contain the target color temperature scene.

[0058] The first color temperature and the second color temperature can be determined based on the standard light source category present in the image to be processed. For example, if the light source category present in the image to be processed includes A light and / or H light, the color temperature is determined to be the first color temperature (low color temperature); if the light source category present in the image to be processed includes DF light, the color temperature is determined to be the second color temperature (high color temperature). It is understood that the standard light source category present in the image to be processed may include at least A light, and the present disclosure does not specifically limit the standard light source category present in the image to be processed.

[0059] In one embodiment of the present disclosure, color temperature compensation is performed on the high color temperature area and the low color temperature area of ​​the image to be processed, respectively. The blue gain (B Gain) is added to the high color temperature area of ​​the image to be processed, and the red gain (R Gain) is added to the low color temperature area of ​​the image to be processed, so as to obtain two images with different color temperature compensations, thereby achieving local white balance processing of the image.

[0060] In step S12 , a color deviation value of a target compensation image is determined, where the target compensation image is the first compensation image or the second compensation image.

[0061] In the disclosed embodiment, the degree of color shift is calculated based on one of the first and second compensated images to obtain a color shift value for the compensated image. Color shift refers to the deviation between the color of the image after adjusting the RB gain and the color of the original image. A color shift of 0 represents the original image color, while a color shift of 1 represents the image color that is the bluest / reddest.

[0062] In step S13 , based on the color deviation value of the target compensation image, the first compensation image and the second compensation image are fused to obtain a fused image.

[0063] In the present disclosure, based on the color temperature statistical information obtained from the image to be processed, the existence of a target color temperature scene is determined, color temperature compensation is performed on the image, the color deviation value of the target compensated image is determined, and the images are fused based on the color deviation value of the target compensated image to achieve continuity of compensation at the color temperature boundary and solve the compensation deviation caused by segmenting the image to be processed.

[0064] In the embodiment of the present disclosure, when calculating the degree of color deviation to obtain the color deviation value of the compensated image, a pre-trained model for calculating the color deviation value can be called, and one of the images is input into the model after obtaining HSV data. After the color deviation degree is calculated by the machine learning model, the color deviation value of the target compensated image is output.

[0065] In one implementation, since the HSV (Hue Saturation Value) color gamut space is more suitable for identifying color shift information, the color shift value is calculated based on the color gamut data of the HSV color gamut space in the embodiments of the present disclosure.

[0066] Figure 2 is a flowchart of an image processing method according to an exemplary embodiment. Figure 2 As shown, the image processing method is used in a terminal and includes the following steps.

[0067] In step S21 , the color gamut data of the target compensated image in the HSV color gamut space is determined.

[0068] In step S22, a model for calculating color deviation values ​​is called.

[0069] In step S23 , the color gamut data of the target compensated image in the HSV color gamut space is input into the model to obtain the color deviation value of the target compensated image.

[0070] Among them, the color gamut data in the HSV color gamut space can be converted from RGB data. This color model can better identify color deviation.

[0071] In this disclosure, the RGB data of the target compensation image is first determined. Using the RGB-HSV correspondence, the color gamut data of the target compensation image in the HSV color space is determined. Color gamut data includes hue, saturation, and lightness. Furthermore, the color gamut data in the HSV color space is used as input features to invoke a model for color deviation calculation, outputting the color deviation value.

[0072] In the embodiment of the present disclosure, the model used to calculate the color deviation value may be pre-trained.

[0073] Figure 3 FIG. 1 is a flow chart of a method for training a color deviation value calculation model according to an exemplary embodiment. Figure 3The training method for calculating the color deviation value model shown includes the following steps.

[0074] In step S31 , sample images with different degrees of color cast are acquired.

[0075] The sample image is obtained by adjusting the red gain or the blue gain, and at least one of the red gain and the blue gain corresponding to different sample images is different.

[0076] In step S32 , color deviation values ​​are marked on sample images with different color deviation degrees to obtain marked color deviation values.

[0077] In step S33 , the color gamut data of the sample images with different color cast degrees in the HSV color gamut space are determined to obtain the color gamut sample data in the HSV color gamut space.

[0078] In step S34, the color gamut sample data is used as input features, and the marked color deviation value is used as output features, and a model for calculating the color deviation value is obtained through training.

[0079] In the present disclosure, a color card image is collected, and sample images with different degrees of color deviation are obtained by adjusting the red gain and blue gain of the sample image. Furthermore, the sample images with different degrees of color deviation are marked with color deviation values ​​to obtain marked color deviation values. For example, the blue gain of the color card image is adjusted so that the color card image presents different degrees of blue. For example, the marked values ​​are 1, 0.8, 0.3, 0.15, and 0, respectively. The closer to 0, the more the color presented by the color card image is biased towards a reddish image with a low color temperature, and the closer to 1, the more the color presented by the color card image is biased towards a bluish image with a high color temperature.

[0080] Furthermore, in this disclosure, the RGB data of the color card image is converted to HSV because HSV is more easily used to identify color shift information. As shown in Tables 1 and 2, Table 1 shows the RGB values ​​and label values ​​of the color card image, and Table 2 shows the HSV values ​​and label values ​​of the color card image.

[0081] Table 1

[0082]

[0083]

[0084] Table 2

[0085]

[0086]

[0087] In the disclosed embodiment, a color chart image is captured, color deviation adjustment is performed on the acquired color chart image, and the degree of deviation is marked. The HSV values ​​and marked values ​​of the deviation color chart image are input into a prediction model for model training until the prediction model converges, resulting in a final model for calculating the deviation value.

[0088] In an exemplary embodiment of the present disclosure, a regression model may be used for model training. Based on the regression model, color gamut sample data and marked color deviation values ​​are trained until the regression model converges, and the converged regression model is determined as the model for calculating the color deviation value.

[0089] Among them, the regression model in the embodiment of the present disclosure can be a logistic regression model, a support vector machine SVM, a linear regression model, etc.

[0090] In an exemplary embodiment of the present disclosure, a Gaussian process regression (GPR) model is used for training.

[0091] For example: using the Gaussian process regression model: P(yi|f(xi),xi)~N(yi|h(xi)Tβ+f(xi),σ 2 ), train until the Gaussian process regression model converges, and determine the converged regression model as the model for calculating the color deviation value. Where f(x) is the Gaussian prediction model and h(x) is the conversion basis function.

[0092] In the embodiment of the present disclosure, the color gamut data of the first compensated image or the second compensated image in the HSV color gamut space is input into the trained model to obtain a color deviation value, and the first compensated image and the second compensated image are fused based on the obtained color deviation value.

[0093] Figure 4 is a flowchart of an image processing method according to an exemplary embodiment. Figure 4 As shown, the image processing method is used in a terminal and includes the following steps.

[0094] In step S41 , the color deviation value of the target compensation image is used as a first weight corresponding to the target compensation image.

[0095] In step S42 , a second weight is determined based on the first weight.

[0096] In step S43 , the first compensation image and the second compensation image are fused based on the determined first weight and second weight.

[0097] The sum of the second weight and the first weight is 1.

[0098] In the present disclosure, the color deviation value of the target compensation image is used as the first weight corresponding to the target compensation image. For example, if the first weight corresponding to the target compensation image is α, then the second weight is 1-α. Based on the first weight and the second weight, the two compensation images are fused pixel by pixel: Img_RGB = Img_RGB1*α + Img_RGB2*(1-α). Among them, Img_RGB1 and Img_RGB2 are two compensation images, and α is the predicted degree of color deviation, with a value between 0 and 1. The two compensation images are fused according to the ratio of the first weight and the second weight to obtain the final image.

[0099] The image processing method provided by the disclosed embodiments resolves color temperature conflicts in processed images by compensating for both high and low color temperatures. The method then fuses the images based on the color deviation values, resolving the conflict and improving the transition effect at the boundary between high and low color temperatures. Furthermore, image fusion based on color deviation values ​​improves compensation accuracy and mitigates poor image compensation caused by segmentation issues.

[0100] Figure 5 FIG2 shows a schematic diagram of an image processing process provided in an exemplary embodiment of the present disclosure. Figure 5 As shown, in the present disclosure, the color temperature statistical information of the image to be processed is extracted, and the color temperature statistical information includes: high color temperature and low color temperature. High color temperature makes the image appear blue, and low color temperature makes the image appear red. Based on the ratio of high and low color temperature areas, it is determined whether there is a color temperature conflict in the image to be processed. If there is no high and low color temperature in the image to be processed, the automatic white balance algorithm in traditional technology is used to process the image to obtain an output image. If there is a high and low color temperature conflict in the image to be processed, the high color temperature area and the low color temperature area are compensated respectively. The high color temperature area is compensated using blue gain, and the low color temperature area is compensated using red gain to obtain two compensated images. A color chart image is collected, the color deviation of the color chart image is adjusted, the RGB value of the color deviation color chart image is obtained, and the RGB value of the color deviation color chart image is converted into HSV value. The degree of color deviation of the color deviation color chart image is marked using project experience, for example, 1, 0.8, 0.3, 0.15, and 0 respectively. The HSV values ​​and label values ​​of the color-shifted color chart image are used as input data for the regression model training to obtain a converged regression model. The HSV values ​​and label values ​​of one of the two compensated images are input into the converged regression model to obtain the color shift degree of that image, which can be set as α. Based on the two compensated images and the color shift degree values, the pixels of the two compensated images are fused according to the color shift degree values ​​to obtain the final image.

[0101] It should be noted that those skilled in the art will appreciate that the various implementation methods / embodiments involved in the embodiments of the present disclosure can be used in conjunction with the aforementioned embodiments or can be used independently. Whether used alone or in conjunction with the aforementioned embodiments, the implementation principles are similar. In the implementation of the present disclosure, some embodiments are described in terms of implementation methods used together; of course, those skilled in the art will appreciate that such examples are not limitations on the embodiments of the present disclosure.

[0102] Based on the same concept, an embodiment of the present disclosure also provides an image processing device.

[0103] It is understandable that the image processing device provided by the embodiment of the present disclosure includes hardware structures and / or software modules corresponding to the execution of each function in order to realize the above functions. In combination with the units and algorithm steps of the various examples disclosed in the embodiment of the present disclosure, the embodiment of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present disclosure.

[0104] Figure 6 FIG. 1 is a block diagram of an image processing apparatus according to an exemplary embodiment. Figure 6 The device 100 can be provided as the terminal involved in the above embodiment, including a compensation unit 101, a processing unit 102 and a fusion unit 103.

[0105] The compensation unit 101 is used to determine that the image to be processed has a target color temperature scene, and then use a first gain to perform color temperature compensation on the image to be processed to obtain a first compensated image, and use a second gain to perform color temperature compensation on the image to be processed to obtain a second compensated image. The processing unit 102 is used to determine the color deviation value of the target compensated image. The fusion unit 103 is used to fuse the first compensation image and the second compensation image based on the color deviation value of the target compensated image to obtain a fused image.

[0106] In one embodiment, the processing unit 102 determines the color deviation value of the target compensation image in the following manner:

[0107] Determine the color gamut data of the target compensation image in the HSV color gamut space; call a model for calculating the color deviation value; input the color gamut data of the target compensation image in the HSV color gamut space into the model to obtain the color deviation value of the target compensation image.

[0108] In one embodiment, the image processing apparatus 100 further includes a training unit, which pre-trains the model in the following manner:

[0109] Sample images with different degrees of color deviation are obtained, where the sample images are color card images after adjusting red gain and blue gain; color deviation values ​​are marked on the sample images with different degrees of color deviation to obtain marked color deviation values; color gamut data of the sample images with different degrees of color deviation in the HSV color gamut space are determined to obtain color gamut sample data in the HSV color gamut space; the color gamut sample data are used as input features and the marked color deviation values ​​are used as output features to train a model for calculating color deviation values.

[0110] In one embodiment, the training unit uses the following method to take the color gamut sample data as input features and the marked color deviation value as output features to train a model for calculating the color deviation value:

[0111] Based on the regression model, the color gamut sample data and the marked color deviation values ​​are trained until the regression model converges, and the converged regression model is determined as the model for calculating the color deviation value.

[0112] In one embodiment, the fusion unit 103 fuses the first compensation image and the second compensation image based on the color deviation value of the target compensation image in the following manner:

[0113] The color deviation value of the target compensation image is used as the first weight corresponding to the target compensation image; based on the first weight, a second weight is determined, and the sum of the second weight and the first weight is 1; based on the first weight and the second weight, the first compensation image and the second compensation image are fused.

[0114] In one embodiment, it is determined that the image to be processed has a target color temperature scene:

[0115] Obtain color temperature statistical information of the image to be processed; and determine whether the image to be processed has a target color temperature scene based on the color temperature statistical information.

[0116] In one embodiment, the difference between the first color temperature and the second color temperature is greater than a third threshold.

[0117] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0118] Figure 7 2 is a block diagram illustrating an apparatus for application control according to an exemplary embodiment. For example, apparatus 200 may be provided as a terminal as described in the above embodiments. For example, it may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0119] Reference Figure 7, apparatus 200 may include one or more of the following components: a processing component 202 , a memory 204 , a power component 206 , a multimedia component 208 , an audio component 210 , an input / output (I / O) interface 212 , a sensor component 214 , and a communication component 216 .

[0120] The processing component 202 generally controls the overall operation of the device 200, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 202 may include one or more modules to facilitate interaction between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

[0121] The memory 204 is configured to store various types of data to support operations on the device 200. Examples of such data include instructions for any application or method operating on the device 200, contact data, phone book data, messages, pictures, videos, etc. The memory 204 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0122] The power component 206 provides power to the various components of the device 200. The power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 200.

[0123] The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0124] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC) that is configured to receive external audio signals when the device 200 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 204 or transmitted via the communication component 216. In some embodiments, the audio component 210 further includes a speaker for outputting audio signals.

[0125] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0126] The sensor assembly 214 includes one or more sensors for providing various aspects of the status assessment of the device 200. For example, the sensor assembly 214 can detect the open / closed state of the device 200, the relative positioning of components, such as the display and keypad of the device 200. The sensor assembly 214 can also detect changes in the position of the device 200 or a component of the device 200, the presence or absence of user contact with the device 200, the orientation or acceleration / deceleration of the device 200, and temperature changes of the device 200. The sensor assembly 214 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 214 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0127] The communication component 216 is configured to facilitate wired or wireless communication between the device 200 and other devices. The device 200 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0128] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method.

[0129] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as the memory 204 including instructions, which can be executed by the processor 220 of the apparatus 200 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0130] It is further understood that although operations are described in a particular order in the drawings in the embodiments of the present disclosure, this should not be construed as requiring that the operations be performed in the particular order shown or in a serial order, or that all of the operations shown be performed to obtain the desired results. In certain circumstances, multitasking and parallel processing may be advantageous.

[0131] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0132] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the scope of the appended claims.

Claims

1. An image processing method, characterized in that: include: When it is determined that the image to be processed exists in a target color temperature scene, color temperature compensation is performed on the image to be processed using a first gain to obtain a first compensated image, and color temperature compensation is performed on the image to be processed using a second gain to obtain a second compensated image, the first gain corresponding to a first color temperature, and the second gain corresponding to a second color temperature, wherein the existence of the target color temperature scene includes that a sum of a color temperature proportion of the first color temperature and a color temperature proportion of the second color temperature in the image to be processed is greater than a first threshold, a difference between a color temperature proportion of the first color temperature and a color temperature proportion of the second color temperature is less than a second threshold, and the first threshold is greater than the second threshold; determining a color deviation value of a target compensation image, wherein the target compensation image is the first compensation image or the second compensation image; Based on the color deviation value of the target compensation image, the first compensation image and the second compensation image are fused to obtain a fused image.

2. The image processing method according to claim 1, wherein: Determining the color deviation value of the target compensation image includes: Determining color gamut data of the target compensated image in the HSV color gamut space; Calling a model for calculating color deviation values, wherein the input of the model is color gamut data in the HSV color gamut space, and the output is the color deviation value; The color gamut data of the target compensation image in the HSV color gamut space is input into the model to obtain the color deviation value of the target compensation image.

3. The image processing method according to claim 2, wherein: The model is pre-trained using the following methods, including: Acquire sample images with different degrees of color shift, wherein the sample images are obtained by adjusting a red gain or a blue gain, and at least one of the red gain and the blue gain corresponding to different sample images is different; Marking the sample images with different color deviation degrees with color deviation values ​​to obtain marked color deviation values; Determine the color gamut data of the sample images with different color cast degrees in the HSV color gamut space, and obtain color gamut sample data in the HSV color gamut space; The color gamut sample data is used as input features, and the marked color deviation value is used as an output feature, and a model for calculating the color deviation value is obtained through training.

4. The image processing method according to claim 3, wherein: The color gamut sample data is used as an input feature, the marked color deviation value is used as an output feature, and a model for color deviation value calculation is trained, including: Based on the regression model, the color gamut sample data and the marked color deviation value are trained until the regression model converges, and the converged regression model is determined as a model for calculating the color deviation value.

5. The image processing method according to any one of claims 1 to 3, characterized in that: The fusing process of the first compensation image and the second compensation image based on the color deviation value of the target compensation image includes: Using the color deviation value of the target compensation image as the first weight corresponding to the target compensation image; Determine a second weight based on the first weight, where the sum of the second weight and the first weight is 1; Based on the first weight and the second weight, the first compensation map and the second compensation image are fused.

6. The image processing method according to claim 1, wherein: Determining whether the image to be processed has a target color temperature scene includes: Obtaining color temperature statistical information of the image to be processed; Based on the color temperature statistical information, it is determined that the image to be processed has the target color temperature scene.

7. The image processing method according to claim 1, wherein: The color temperature difference between the first color temperature and the second color temperature is greater than a third threshold.

8. An image processing device, characterized in that: include: a compensation unit, configured to, upon determining that a target color temperature scene exists in an image to be processed, perform color temperature compensation on the image to be processed using a first gain to obtain a first compensated image, and perform color temperature compensation on the image to be processed using a second gain to obtain a second compensated image, wherein the first gain corresponds to a first color temperature, and the second gain corresponds to a second color temperature, wherein the presence of the target color temperature scene comprises that a sum of a color temperature proportion of the first color temperature and a color temperature proportion of the second color temperature in the image to be processed is greater than a first threshold, a difference between a color temperature proportion of the first color temperature and a color temperature proportion of the second color temperature is less than a second threshold, and the first threshold is greater than the second threshold; a processing unit, configured to determine a color deviation value of a target compensation image, wherein the target compensation image is the first compensation image or the second compensation image; A fusion unit is used to fuse the first compensation image and the second compensation image based on the color deviation value of the target compensation image to obtain a fused image.

9. The image processing device according to claim 8, wherein The processing unit determines the color deviation value of the target compensation image in the following manner: Determining color gamut data of the target compensated image in the HSV color gamut space; Call the model for color deviation calculation; The color gamut data of the target compensation image in the HSV color gamut space is input into the model to obtain the color deviation value of the target compensation image.

10. The image processing device according to claim 9, wherein The model is pre-trained in the following manner: Acquire sample images with different degrees of color shift, wherein the sample images are obtained by adjusting a red gain or a blue gain, and at least one of the red gain and the blue gain corresponding to different sample images is different; Marking the sample images with different color deviation degrees with color deviation values ​​to obtain marked color deviation values; Determine the color gamut data of the sample images with different color cast degrees in the HSV color gamut space, and obtain color gamut sample data in the HSV color gamut space; The color gamut sample data is used as input features, and the marked color deviation value is used as an output feature, and a model for calculating the color deviation value is obtained through training.

11. The image processing device according to claim 10, wherein The training unit uses the color gamut sample data as input features and the marked color deviation value as output features to train a model for calculating the color deviation value in the following manner: Based on the regression model, the color gamut sample data and the marked color deviation value are trained until the regression model converges, and the converged regression model is determined as a model for calculating the color deviation value.

12. The image processing device according to any one of claims 8 to 10, characterized in that: The fusion unit fuses the first compensation image and the second compensation image based on the color deviation value of the target compensation image in the following manner: Using the color deviation value of the target compensation image as the first weight corresponding to the target compensation image; Determine a second weight based on the first weight, where the sum of the second weight and the first weight is 1; Based on the first weight and the second weight, the first compensation map and the second compensation image are fused.

13. The image processing device according to claim 8, wherein Determining whether the image to be processed has a target color temperature scene includes: Obtaining color temperature statistical information of the image to be processed; Based on the color temperature statistical information, it is determined that the target color temperature scene exists in the image to be processed.

14. The image processing device according to claim 8, wherein The color temperature difference between the first color temperature and the second color temperature is greater than a third threshold.

15. An image processing device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that The storage medium stores instructions, and when the instructions in the storage medium are executed by the processor of the mobile terminal, the mobile terminal is enabled to execute the method according to any one of claims 1 to 7.

Images