Image processing method and device, equipment and storage medium

Abstract

The invention relates to an image processing method and device, equipment and a storage medium. The image processing method comprises the following steps: acquiring a current color temperature acquired by a color temperature sensor from a current shooting environment; comparing the current color temperature with a plurality of reference color temperatures to determine a target reference color temperature, and determining a first color gain value under the current color temperature according to a target reference color gain value under the target reference color temperature; obtaining the current brightness collected by a brightness sensor from the current shooting environment; determining a target color interval where the current brightness is located, and adjusting the first color gain value according to a compensation adjustment coefficient and a set adjustment weight of the target color interval to obtain a second color gain value; and performing image processing on a current image acquired in the current shooting environment according to the second color gain value. According to the method provided by the invention, the image processing flow is simplified, the debugging flow is simpler and more convenient, the time cost is reduced to a certain extent, and the processing efficiency is further improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, device, equipment and storage medium. Background Art

[0002] White Balance (WB) is a key step in image processing, used to ensure that photos or videos taken under light sources of different color temperatures can present natural colors.

[0003] At present, most white balance calculation schemes perform image quality processing such as optical fitting and lens shading correction on the sensor's original image, perform statistical data sampling, and then process the sampled data with some special mechanisms such as the advanced grayscale world. For example, changing the weight of the statistical data, etc., in order to determine the optimal image processing parameters to make the image look more realistic and natural.

[0004] However, the existing solution processing process is cumbersome and involves knowledge and technology in multiple fields. It also requires a lot of experiments and data analysis to determine the optimal debugging parameters. Since the debugging process is complicated and time-consuming, it also increases the overall time cost. Summary of the invention

[0005] In order to solve the above technical problems, the embodiments of the present disclosure provide an image processing method, apparatus, device and storage medium.

[0006] In a first aspect, an embodiment of the present disclosure provides an image processing method, the method comprising:

[0007] Get the current color temperature collected by the color temperature sensor from the current shooting environment;

[0008] Compare the current color temperature with a plurality of reference color temperatures to determine a target reference color temperature, and determine a first color gain value at the current color temperature according to a target reference color gain value at the target reference color temperature; wherein the reference color gain value is determined by photographing a gray card at the plurality of reference color temperatures;

[0009] Acquire the current brightness collected by the brightness sensor from the current shooting environment;

[0010] Determine the target color interval where the current brightness is located, and adjust the first color gain value according to the compensation adjustment coefficient and the set adjustment weight of the target color interval to obtain a second color gain value; wherein the target color interval is a preferred color interval divided according to the set brightness;

[0011] Image processing is performed on a current image collected in the current shooting environment according to the second color gain value.

[0012] Optionally, comparing the current color temperature with a plurality of reference color temperatures to determine a target reference color temperature includes:

[0013] Compare the current color temperature with a plurality of reference color temperatures to determine whether there is a first reference color temperature that is the same as the current color temperature;

[0014] If the first reference color temperature exists, determining the first reference color temperature as the target reference color temperature; or,

[0015] If the first reference color temperature does not exist, then when there are two similar reference color temperatures close to the current color temperature among the multiple reference color temperatures, the two similar reference color temperatures are determined as the target reference color temperature; wherein the current color temperature is within the color temperature range of the two similar reference color temperatures; or,

[0016] If the two similar reference color temperatures do not exist, a second reference color temperature close to the current color temperature is determined from the multiple reference color temperatures, and the second reference color temperature is determined as the target reference color temperature; wherein the second reference color temperature is the maximum reference color temperature or the minimum reference color temperature from the multiple reference color temperatures.

[0017] Optionally, the two similar reference color temperatures include a first similar reference color temperature and a second similar reference color temperature, and determining the first color gain value at the current color temperature according to the target reference color gain value at the target reference color temperature includes:

[0018] Calculate a first color temperature difference between the current color temperature and the first similar reference color temperature; wherein the first similar reference color temperature is less than the current color temperature;

[0019] Calculate a second color temperature difference between the second similar reference color temperature and the first similar reference color temperature;

[0020] Calculating a gain difference between a first reference color gain value at the first similar reference color temperature and a second reference color gain value at the second similar reference color temperature;

[0021] Calculating a gain adjustment value according to the first color temperature difference, the second color temperature difference and the gain difference;

[0022] A sum of the gain adjustment value and the first reference color gain value is determined as a first color gain value at the current color temperature.

[0023] Optionally, before determining the target color interval in which the current brightness is located, the method further includes:

[0024] Divide the set preferred color into intervals according to the set brightness to obtain the preferred color intervals;

[0025] The gray card is photographed under different light sources with the set brightness to set compensation adjustment coefficients of the preferred color range for different light sources.

[0026] Optionally, the adjusting the first color gain value according to the compensation adjustment coefficient of the target color interval and setting the adjustment weight to obtain the second color gain value includes:

[0027] Calculate the product of the compensation adjustment coefficient of the target color interval, set the adjustment weight and the first color gain value to obtain a first part of the color gain value;

[0028] Calculating a difference between a set threshold and the set adjustment weight, and calculating a product of the difference and the first color gain value to obtain a second part of the color gain value;

[0029] A sum of the first portion of color gain values ​​and the second portion of color gain values ​​is calculated to obtain a second color gain value.

[0030] Optionally, the first color gain value includes a first red gain value and a first blue gain value;

[0031] The reference color gain value includes a reference red gain value and a reference blue gain value;

[0032] The compensation adjustment coefficient includes a red gain compensation adjustment coefficient and a blue gain compensation adjustment coefficient;

[0033] The second color gain value includes a second red gain value and a second blue gain value.

[0034] Optionally, before comparing the current color temperature with a plurality of reference color temperatures, the method further includes:

[0035] Photographing gray cards at multiple reference color temperatures, and calibrating reference points in the captured gray card images; wherein the reference points refer to pixels in a neutral gray area in the gray card images;

[0036] The color average value of the neutral gray area is calculated, and the color gain value is adjusted based on the color average value to obtain a reference color gain value at each reference color temperature.

[0037] In a second aspect, an embodiment of the present disclosure provides an image processing device, the device comprising:

[0038] A first acquisition unit, used to acquire a current color temperature collected by a color temperature sensor from a current shooting environment;

[0039] A color gain value determination unit, configured to compare the current color temperature with a plurality of reference color temperatures to determine a target reference color temperature, and determine a first color gain value at the current color temperature according to a target reference color gain value at the target reference color temperature; wherein the reference color gain value is determined by photographing a gray card at the reference color temperature;

[0040] A second acquisition unit, used to acquire the current brightness collected by the brightness sensor from the current shooting environment;

[0041] A color gain value adjustment unit, used for determining a target color interval where the current brightness is located, and adjusting the first color gain value according to a compensation adjustment coefficient and a set adjustment weight of the target color interval to obtain a second color gain value; wherein the target color interval is a preferred color interval divided according to the set brightness;

[0042] An image processing unit is used to perform image processing on a current image collected in the current shooting environment according to the second color gain value.

[0043] In a third aspect, an embodiment of the present disclosure provides an electronic device, including:

[0044] Memory;

[0045] Processor; and

[0046] Computer programs;

[0047] The computer program is stored in the memory and is configured to be executed by the processor to implement the image processing method as described above.

[0048] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the image processing method described above are implemented.

[0049] The image processing method provided by the present disclosure includes: obtaining the current color temperature collected by the color temperature sensor from the current shooting environment; comparing the current color temperature with multiple reference color temperatures to determine the target reference color temperature, and determining the first color gain value under the current color temperature according to the target reference color gain value under the target reference color temperature; wherein the reference color gain value is determined by shooting gray cards under multiple reference color temperatures; obtaining the current brightness collected by the brightness sensor from the current shooting environment; determining the target color interval where the current brightness is located, and adjusting the first color gain value according to the compensation adjustment coefficient and the set adjustment weight of the target color interval to obtain the second color gain value; wherein the target color interval is a preferred color interval divided according to the set brightness; and performing image processing on the current image collected in the current shooting environment according to the second color gain value. The method provided by the present application simplifies the image processing process, makes the debugging process more convenient, reduces the time cost to a certain extent, and further improves the processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0052] Figure 1 A schematic diagram of a white balance calculation strategy for a Qualcomm series chip provided in an embodiment of the present disclosure;

[0053] Figure 2 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0054] Figure 3 A flowchart of another image processing method provided by an embodiment of the present disclosure;

[0055] Figure 4 A schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure;

[0056] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0057] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0058] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0059] Before explaining this application in detail, we will first explain the professional terms involved, including:

[0060] White Balance: The basic concept is "white objects can be restored to white regardless of the light source". The color cast that occurs when shooting under a specific light source is compensated by strengthening the corresponding complementary color. The camera's white balance setting can calibrate the deviation of color temperature. When shooting, you can achieve the desired picture effect by adjusting the white balance. The so-called white balance is a calculation method for digital image color processing that accurately restores the colors of other objects by restoring the color of white objects (producing a pure white color effect).

[0061] Color temperature: The light emitted by a certain light source has the same spectral components as the light emitted by a black body at a certain temperature, which is called a color temperature of a certain K. For example, if the color of the light emitted by a 100W bulb is the same as the color of an absolute black body at 2527℃, then the color temperature of the light emitted by this bulb is: (2527+273)K=2800K.

[0062] Specifically, the existing white balance calculation scheme is to sample the statistical data of the original image (raw image) transmitted by the sensor through the OB (Optical Black) and LSC (Lens Shading Correction) modules, and process the sampled stats (all points in the area) with some special mechanisms such as advanced grayscale world algorithms, including changing the weight of the statistical data, or assigning certain weights to R Gain and B Gain (color gain) according to the characteristic statistical data. Finally, the AWB (Auto White Balance Statistics) decision that can handle most scenes is obtained, which is far superior to the simple grayscale world.

[0063] For example, see Figure 1 , Figure 1A schematic diagram of a white balance calculation strategy for a Qualcomm series chip provided in an embodiment of the present disclosure. The white balance calculation strategy for Qualcomm series chips mainly relies on the automatic white balance algorithm in its image signal processor. The AWB algorithm automatically adjusts the gain values ​​of the red, green, and blue (RGB) channels by analyzing image data to adapt to different lighting conditions, ensuring that white objects in the image can appear neutral under different light sources. Figure 1 As can be seen from the process, the existing white balance solution is extremely complex and requires understanding of the principles of many mechanisms, as well as debugging parameters and methods. Although it can solve white balance problems in many scenarios, it requires more manpower to support algorithm upgrades and white balance parameter debugging, and it also occupies more chip operating memory and requires the chip to have better computing power. This solution is often only suitable for products such as consumer electronics that have better chips and higher white balance accuracy requirements.

[0064] In view of the above technical problems, the embodiments of the present disclosure provide an image processing method, which makes the image white balance more in line with the requirements of the human eye, and the process is more simplified, and the debugging is simpler and more convenient. Specifically, the initial R Gain and B Gain of the original image transmitted by the sensor are determined, and on this basis, the preferred color that is more in line with the perception of the human eye is added, and the R Gain and B Gain are further adjusted by the added preferred color, so as to achieve a white balance bias that is more in line with the requirements (preferences) of the human eye under various light sources, so that the image is more realistic and the image quality is higher. Detailed description is given by one or more of the following embodiments.

[0065] The image processing method provided by the embodiment of the present disclosure can be applied to the scene of white balance processing of the original image. The method can be performed by an image processing device, which can be implemented by software and / or hardware, and the device can be integrated in an electronic device. Among them, the electronic device can include but is not limited to mobile terminals such as smart phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Tablet Personal Computers, Tablet PCs), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), wearable devices, etc., and fixed terminals such as digital televisions, desktop computers, smart home devices, etc.

[0066] Figure 2A flowchart of an image processing method provided by an embodiment of the present disclosure is applied to a target device, wherein the target device includes an image sensor, a color temperature sensor, a processor, and a brightness sensor, wherein the image sensor is used to collect images, the color temperature sensor is used to collect color temperature, the brightness sensor is used to collect brightness, and the processor is used to perform white balance processing on the image according to the color temperature and brightness. The image processing method specifically includes the following steps: Figure 2 The following steps are shown:

[0067] S201: Acquire a current color temperature collected by a color temperature sensor from a current shooting environment.

[0068] It can be understood that the color temperature sensor senses the color temperature information from the shooting environment in real time, and converts the color temperature information into color temperature values ​​and transmits them to the processor. The processor obtains the color temperature transmitted by the color temperature sensor to perform image processing such as white balance on the original image captured by the image sensor, and outputs a target image that is more in line with the requirements of the human eye and has more realistic and natural image colors.

[0069] Optionally, before comparing the current color temperature with a plurality of reference color temperatures, the method further includes:

[0070] Photograph gray cards at multiple reference color temperatures, and calibrate reference points in the captured gray card images; wherein the reference points refer to pixels in a neutral gray area in the gray card images; calculate the color average of the neutral gray area, and adjust the color gain value based on the color average to obtain a reference color gain value at each reference color temperature.

[0071] It can be understood that by photographing gray cards at multiple reference color temperatures in a light box in a laboratory, reference points of different reference color temperatures can be calibrated, wherein the reference point refers to a pixel point in a neutral gray area in the captured gray card image. Specifically, the light emitted by different light sources of the light box has different reference color temperatures. For example, the light source can be a variety of light sources such as D75, D65, D50, TL84, CWF, A and H. The color temperature of the light emitted by D75 is 7500K, the color temperature of the D65 light source is 6500K, the color temperature of the D50 light source is 5000K, the color temperature of the TL84 light source is 4000K, the color temperature of the CWF light source is 4150K, the color temperature of the A light source is 2856K, and the color temperature of the H light source is 2300K. Other possible light sources and color temperatures are not limited here. Take an image of the same gray card at each color temperature to obtain the gray card image corresponding to each color temperature. Then, select the neutral gray area in the gray card image, that is, for the gray card image at each reference color temperature, find the brightest neutral gray area and calibrate the reference point in the neutral gray area, or directly determine the neutral gray area as the reference point. Neutral gray refers to gray with a reflectivity of 18%. The human eye perceives its brightness between black and white, with a brightness of 50%. After calibrating the reference point, calculate the RGB average value (color average) of the neutral gray area, and use the simple gray world algorithm to adjust the RGB gain value so that the three average values ​​are as close as possible. The adjusted gain value is the reference color gain value at the reference color temperature.

[0072] S202: Compare the current color temperature with a plurality of reference color temperatures to determine a target reference color temperature, and determine a first color gain value at the current color temperature according to the target reference color gain value at the target reference color temperature.

[0073] The reference color gain value is determined by photographing gray cards at multiple reference color temperatures; the reference color gain value includes a reference red gain value and a reference blue gain value; and the first color gain value includes a first red gain value and a first blue gain value.

[0074] It can be understood that, based on the above S201, the current color temperature is compared with multiple reference color temperatures to determine the target reference color temperature, and the target reference color temperature refers to at least one reference color temperature that is the same as or close to the color temperature value of the current color temperature. After determining the target reference color temperature, the first color gain value at the current color temperature is determined according to the target reference color gain value at the target reference color temperature. The target reference color gain value is determined by the gray card image taken by the light box, and the first color gain value can be understood as the initial color gain value determined based on the current color temperature. Among them, the reference color gain value includes a reference red gain value and a reference blue gain value. Accordingly, the first color gain value includes a first red gain value and a first blue gain value. The red gain value is represented by an RGain value, and the blue gain value is represented by a B Gain value.

[0075] Optionally, the above-mentioned comparison of the current color temperature with multiple reference color temperatures to determine the target reference color temperature can be specifically implemented through the following steps:

[0076] Compare the current color temperature with multiple reference color temperatures to determine whether there is a first reference color temperature that is the same as the current color temperature; if the first reference color temperature exists, determine the first reference color temperature as the target reference color temperature; or, if the first reference color temperature does not exist, then when there are two similar reference color temperatures close to the current color temperature among multiple reference color temperatures, determine the two similar reference color temperatures as the target reference color temperature; wherein the current color temperature is within the color temperature range of the two similar reference color temperatures; or, if there are no two similar reference color temperatures, determine a second reference color temperature close to the current color temperature among multiple reference color temperatures, and determine the second reference color temperature as the target reference color temperature; wherein the second reference color temperature is the maximum reference color temperature or the minimum reference color temperature among the multiple reference color temperatures.

[0077] It can be understood that the steps of determining the target reference color temperature among multiple reference color temperatures are as follows: compare the color temperature values ​​of the current color temperature with the color temperature values ​​of multiple reference color temperatures, determine whether there is a first reference color temperature with the same color temperature value as the current color temperature among the multiple reference color temperatures, and if there is a first reference color temperature with the same color temperature value, directly determine the first reference color temperature as the target reference color temperature. If there is no first reference color temperature, determine whether there are two similar reference color temperatures with a color temperature value close to the current color temperature among the multiple reference color temperatures, that is, retrieve whether the current color temperature is between the two reference color temperatures, and if there are two similar reference color temperatures, determine the two similar reference color temperatures as the target reference color temperature, wherein the color temperature value of the current color temperature is within the color temperature value range composed of the two similar reference color temperatures. If there are not two similar reference color temperatures, determine whether there is a second reference color temperature close to the current color temperature among the multiple reference color temperatures, that is, determine whether the current color temperature is greater than the maximum reference color temperature or whether the current color temperature is less than the minimum reference color temperature, where the second reference color temperature refers to the maximum reference color temperature with the largest color temperature value or the minimum reference color temperature with the smallest color temperature value among the multiple reference color temperatures.

[0078] It is understandable that after determining the target reference color temperature, the first color gain value at the current color temperature is determined according to the target reference color gain value at the target reference color temperature. Specifically, when the target reference color temperature is the first reference color temperature, the first color gain value is directly determined by the reference color gain value at the first reference color temperature, that is, the R Gain and B Gain at the reference color temperature are directly used to whiten the image. If the color temperature value of the current color temperature is different from the color temperature value of any reference color temperature, it is searched whether there are two similar reference color temperatures that are similar to the color temperature value of the current color temperature. If the current color temperature is between the two similar reference color temperatures, the first color gain value is calculated by interpolation according to the R Gain and B Gain at the two similar reference color temperatures. For the specific calculation process, see the following embodiment. If the current color temperature is not between two similar reference color temperatures, for example, the current color temperature is greater than 7500K (D75 light source, maximum reference color temperature) or less than 2300K (H light source, minimum reference color temperature), the R Gain value and B Gain value under the D75 light source or the H light source are directly used as the first color gain value.

[0079] The two similar reference color temperatures include a first similar reference color temperature and a second similar reference color temperature.

[0080] It is understandable that the two similar reference color temperatures include a first similar reference color temperature and a second similar reference color temperature, wherein the difference between the two similar reference color temperatures and the current color temperature may be different or the same, the color temperature value of the first similar reference color temperature is less than the current color temperature, and the color temperature value of the second similar reference color temperature is greater than the current color temperature. Alternatively, the difference between each reference color temperature and the current color temperature may be calculated, and multiple similar reference color temperatures may be determined based on the difference, and two similar reference color temperatures with the smallest difference and located on both sides of the current color temperature may be determined from the multiple similar reference color temperatures.

[0081] Optionally, when there are two similar reference color temperatures, determining the first color gain value at the current color temperature according to the target reference color gain value at the target reference color temperature can be specifically implemented by the following steps:

[0082] Calculate a first color temperature difference between a current color temperature and a first close reference color temperature, wherein the first close reference color temperature is less than the current color temperature; calculate a second color temperature difference between a second close reference color temperature and the first close reference color temperature; calculate a gain difference between a first reference color gain value at the first close reference color temperature and a second reference color gain value at the second close reference color temperature; calculate a gain adjustment value according to the first color temperature difference, the second color temperature difference and the gain difference; and determine a sum of the gain adjustment value and the first reference color gain value as the first color gain value at the current color temperature.

[0083] It can be understood that in the case where there are two similar reference color temperatures, the specific steps of calculating the first color gain value by interpolation are as follows: Calculate the difference between the current color temperature and the color temperature value of the first similar reference color temperature to obtain the first color temperature difference, wherein the first similar reference color temperature is less than the current color temperature. Calculate the difference between the color temperature values ​​of the second similar reference color temperature and the first similar reference color temperature to obtain the second color temperature difference, wherein the second similar reference color temperature is greater than the current color temperature. Calculate the difference between the gain values ​​of the first reference color gain value under the first similar reference color temperature and the second reference color gain value under the second similar reference color temperature to obtain the gain difference. Calculate the gain adjustment value according to the first color temperature difference, the second color temperature difference and the gain difference. Determine the sum of the gain adjustment value and the first reference color gain value as the first color gain value under the current color temperature. The calculation process of each color gain value (R Gain and B Gain) in the first color gain value is the same as the above steps, which will not be repeated here, and is specifically shown in formula (1).

[0084]

[0085] In the formula, R Gain is used as an example for explanation, and the same is true for B Gain. R0 is the first red gain value, R1 is the first reference red gain value, R2 is the second reference red gain value, C0 is the current color temperature, C1 is the first similar reference color temperature, C2 is the second similar reference color temperature, C0-C1 is the first color temperature difference, C2-C1 is the second color temperature difference, R2-R1 is the gain difference, is the gain adjustment value.

[0086] S203: Acquire the current brightness collected by the brightness sensor from the current shooting environment.

[0087] It can be understood that, based on the above S202, the brightness sensor senses the brightness information from the shooting environment in real time, and converts the brightness information into a brightness value and transmits it to the processor. The processor obtains the current brightness transmitted by the brightness sensor to determine the white balance gain value based on the current color temperature and the current brightness, and performs white balance processing on the original image captured by the image sensor.

[0088] S204: Determine the target color interval where the current brightness is located, and adjust the first color gain value according to the compensation adjustment coefficient of the target color interval and the set adjustment weight to obtain a second color gain value.

[0089] The target color range is a preferred color range divided according to the set brightness; the compensation adjustment coefficient includes a red gain compensation adjustment coefficient and a blue gain compensation adjustment coefficient; and the second color gain value includes a second red gain value and a second blue gain value.

[0090] It is understandable that, based on the above S203, after determining the first color gain value, a preferred color is added, or a pre-set preferred color is used, and the compensation adjustment coefficient of each preferred color area is obtained. After obtaining the compensation adjustment coefficient of the preferred color interval, the target color interval where the current brightness is located is determined. The target color interval is a preferred color interval divided according to the set brightness. The preferred color refers to the user's preferred color. For example, if the user prefers red, the red gain value may be increased accordingly, and the blue gain value may be decreased accordingly. After determining the target color interval corresponding to the current brightness, the first color gain value is adjusted according to the compensation adjustment coefficient of the target color interval and the set adjustment weight to obtain the second color gain value. The set adjustment weight can be determined according to the user's needs. The specific color gain value adjustment steps refer to the following embodiment.

[0091] Optionally, before determining the target color interval in which the current brightness is located, the method further includes:

[0092] The set preferred color is divided into intervals according to the set brightness to obtain the preferred color intervals; the gray card is photographed under different light sources with the set brightness to set compensation adjustment coefficients of the preferred color intervals for different light sources.

[0093] It is understandable that the preferred color is divided into an interval according to the set brightness to obtain at least one preferred color interval. For example, the value range of the brightness value is 0-255, and 200-255 is divided into a preferred color interval. After completing the division of the preferred color interval, by photographing the gray card under different light sources of the light box, after observing the gray card image obtained by the photographing, the gain value compensation adjustment coefficient of the preferred color interval is set for different light sources to adjust the color gain value (Gain value), wherein the different light sources are light sources corresponding to the brightness value of the above-mentioned set brightness. It is understandable that the compensation adjustment coefficient is a product operation of the R Gain value and the B Gain value obtained by complete whitening under CCT (Correlated Color Temperature), and CCT refers to the measurement unit of color temperature.

[0094] Optionally, the first color gain value is adjusted according to the compensation adjustment coefficient of the target color interval and the set adjustment weight to obtain the second color gain value, which can be specifically implemented by the following steps:

[0095] The compensation adjustment coefficient of the target color interval, the product of the set adjustment weight and the first color gain value are calculated to obtain the first color gain value; the difference between the set threshold and the set adjustment weight is calculated, and the product of the difference and the first color gain value is calculated to obtain the second color gain value; the sum of the first color gain value and the second color gain value is calculated to obtain the second color gain value.

[0096] It can be understood that the adjustment steps of a single color gain value are as follows: calculate the product of the compensation adjustment coefficient, the set adjustment weight and the first color gain value to obtain the first color gain value. Calculate the difference between the set threshold and the set adjustment weight, and calculate the product of the difference and the first color gain value to obtain the second color gain value. Calculate the sum of the first color gain value and the second color gain value to obtain the second color gain value. Specifically, as shown in formula (2).

[0097] R Gain 0=β*R Gain 1*α+(1-β)*R Gain1 formula (2)

[0098] In the formula, R Gain is used as an example for explanation, and the same is true for B Gain. R Gain 0 is the second red gain value, β is the set adjustment weight, R Gain 1 is the first red gain value, α is the compensation adjustment coefficient, 1-β is the difference, β*R Gain 1*α is the first part of the color gain value, and (1-β)*R Gain1 is the second part of the color gain value.

[0099] In a possible embodiment, both R Gain and B Gain are adjusted, and an adjustment can also be made for a certain color gain value, and the compensation adjustment coefficient is [1.1, 0.9], wherein the compensation adjustment coefficient includes multiple sub-coefficients for each color gain value, namely, a first sub-coefficient for a red gain value and a second sub-coefficient for a blue gain value. In the case of complete whitening, R Gain is multiplied by 1.1 for an amplification operation, and B Gain is multiplied by 0.9 for a reduction operation. At the same time, the adjustment weight β is set to 0.8, that is, the influence of the compensation adjustment coefficient α on the completely whitened R Gain value and B Gain value changes from 100% to 80%. Based on the above example, R Gain 0 = 0.8*R Gain 1*1.1 + 0.2*R Gain 1; BGain 0 = 0.8*B Gain 1*0.9 + 0.2*B Gain 1, wherein B Gain 0 is the second blue gain value, and B Gain 1 is the first blue gain value.

[0100] S205: Perform image processing on the current image collected in the current shooting environment according to the second color gain value.

[0101] It can be understood that, on the basis of the above S204, the second color gain value is applied to the current image collected by the image sensor in the shooting environment, and a target image after image processing is output, wherein the current image refers to the original image.

[0102] An image processing method provided by an embodiment of the present disclosure adopts a preferred color adding technology triggered by brightness and color temperature. In a scene where complete whitening is not required, an initial color gain value is determined according to the real-time color temperature of the shooting scene, and a preferred color gain compensation adjustment coefficient is determined according to the real-time brightness of the shooting scene. The initial color gain value is adjusted based on the compensation adjustment coefficient to obtain a final color gain value. The initial gain value is determined by color temperature, and the gain value is adjusted by brightness, thereby simplifying the white balance processing flow, and can be flexibly adjusted according to the shooting scene. The entire processing process also reduces the dependence on chip computing power, while also reducing the workload of debugging personnel, reducing time costs, and further improving processing efficiency.

[0103] Based on the above embodiments, Figure 3 A flowchart of another image processing method provided by an embodiment of the present disclosure specifically includes the following steps:

[0104] 1) The color temperature sensor collects the current color temperature of the shooting environment; 2) The reference point is calibrated through the light box, and the reference color gain value under different reference color temperatures is determined; 3) According to the current color temperature and the calibration reference point, the initial color gain value in the completely white scene is determined; 4) The compensation adjustment coefficient of the preferred color range is set through the light box; 5) The preferred color is added; 6) The brightness sensor collects the current brightness of the shooting environment; 7) The initial color gain value that meets the preferred color is fine-tuned according to the current brightness and current color temperature; 8) The final color gain value is applied to the original image and the processed image is output.

[0105] Figure 4 The structure diagram of an image processing device provided by an embodiment of the present disclosure is shown in FIG. The image processing device provided by an embodiment of the present disclosure can execute the processing flow provided by an embodiment of the image processing method, such as Figure 4 As shown, the image processing device 400 includes a first acquisition unit 401, a color gain value determination unit 402, a second acquisition unit 403, a color gain value adjustment unit 404 and an image processing unit 405, wherein:

[0106] A first acquisition unit 401 is used to acquire a current color temperature collected by a color temperature sensor from a current shooting environment;

[0107] A color gain value determining unit 402 is used to compare the current color temperature with a plurality of reference color temperatures to determine a target reference color temperature, and determine a first color gain value at the current color temperature according to the target reference color gain value at the target reference color temperature; wherein the reference color gain value is determined by photographing a gray card at the reference color temperature;

[0108] The second acquisition unit 403 is used to acquire the current brightness collected by the brightness sensor from the current shooting environment;

[0109] The color gain value adjustment unit 404 is used to determine the target color interval where the current brightness is located, and adjust the first color gain value according to the compensation adjustment coefficient of the target color interval and the set adjustment weight to obtain a second color gain value; wherein the target color interval is a preferred color interval divided according to the set brightness;

[0110] The image processing unit 405 is configured to perform image processing on a current image collected in a current shooting environment according to the second color gain value.

[0111] Optionally, the color gain value determining unit 402 is used to:

[0112] Compare the current color temperature with a plurality of reference color temperatures to determine whether there is a first reference color temperature that is the same as the current color temperature;

[0113] If there is a first reference color temperature, the first reference color temperature is determined as the target reference color temperature; or,

[0114] If the first reference color temperature does not exist, then when there are two similar reference color temperatures close to the current color temperature among the multiple reference color temperatures, the two similar reference color temperatures are determined as the target reference color temperature; wherein the current color temperature is within the color temperature range of the two similar reference color temperatures; or,

[0115] If there are no two similar reference color temperatures, a second reference color temperature close to the current color temperature is determined from among the multiple reference color temperatures, and the second reference color temperature is determined as the target reference color temperature; wherein the second reference color temperature is the maximum reference color temperature or the minimum reference color temperature among the multiple reference color temperatures.

[0116] The two similar reference color temperatures include a first similar reference color temperature and a second similar reference color temperature.

[0117] Optionally, the color gain value determining unit 402 is used to:

[0118] Calculating a first color temperature difference between the current color temperature and a first similar reference color temperature; wherein the first similar reference color temperature is less than the current color temperature;

[0119] Calculate a second color temperature difference between the second similar reference color temperature and the first similar reference color temperature;

[0120] Calculating a gain difference between a first reference color gain value at a first similar reference color temperature and a second reference color gain value at a second similar reference color temperature;

[0121] Calculate a gain adjustment value according to the first color temperature difference, the second color temperature difference and the gain difference;

[0122] The sum of the gain adjustment value and the first reference color gain value is determined as the first color gain value at the current color temperature.

[0123] Optionally, the image processing device 400 is further configured to:

[0124] Divide the set preferred color into intervals according to the set brightness to obtain the preferred color intervals;

[0125] Photograph the gray card under different light sources with set brightness to set compensation adjustment coefficients for preferred color ranges for different light sources.

[0126] Optionally, the color gain value adjustment unit 404 is used to:

[0127] Calculate the product of the compensation adjustment coefficient of the target color interval, the set adjustment weight and the first color gain value to obtain the first part of the color gain value;

[0128] Calculating a difference between a set threshold and a set adjustment weight, and calculating a product of the difference and the first color gain value to obtain a second color gain value;

[0129] The sum of the first color gain value and the second color gain value is calculated to obtain a second color gain value.

[0130] Wherein, the first color gain value includes a first red gain value and a first blue gain value;

[0131] The reference color gain value includes a reference red gain value and a reference blue gain value;

[0132] The compensation adjustment coefficient includes a red gain compensation adjustment coefficient and a blue gain compensation adjustment coefficient;

[0133] The second color gain values ​​include a second red gain value and a second blue gain value.

[0134] Optionally, the image processing device 400 is further configured to:

[0135] Photographing gray cards at multiple reference color temperatures, and calibrating reference points in the captured gray card images; wherein the reference points refer to pixels in a neutral gray area in the gray card images;

[0136] The color average value of the neutral gray area is calculated, and the color gain value is adjusted based on the color average value to obtain a reference color gain value under each reference color temperature.

[0137] Figure 4 The image processing device of the illustrated embodiment can be used to execute the technical solution of the above-mentioned method embodiment, and its implementation principle and technical effect are similar and will not be described in detail here.

[0138] Figure 5 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 5, which shows a schematic diagram of the structure of an electronic device 500 suitable for implementing the embodiment of the present disclosure. The electronic device 500 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), wearable electronic devices, etc., and fixed terminals such as digital TVs, desktop computers, smart home devices, etc. Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0139] like Figure 5 As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 to a random access memory (RAM) 503 to implement the image processing method of the embodiment described in the present disclosure. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0140] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 5 The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0141] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart, thereby implementing the image processing method as described above. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0142] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0143] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0144] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0145] Optionally, when the above one or more programs are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0146] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0147] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0148] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0149] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0150] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0151] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or gateway that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or gateway. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or gateway that includes the elements.

[0152] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, characterized in that: The method comprises: Get the current color temperature collected by the color temperature sensor from the current shooting environment; Compare the current color temperature with a plurality of reference color temperatures to determine a target reference color temperature, and determine a first color gain value at the current color temperature according to a target reference color gain value at the target reference color temperature; wherein the reference color gain value is determined by photographing a gray card at the plurality of reference color temperatures; Acquire the current brightness collected by the brightness sensor from the current shooting environment; Determine the target color interval where the current brightness is located, and adjust the first color gain value according to the compensation adjustment coefficient and the set adjustment weight of the target color interval to obtain a second color gain value; wherein the target color interval is a preferred color interval divided according to the set brightness; Image processing is performed on a current image collected in the current shooting environment according to the second color gain value.

2. The method according to claim 1, characterized in that The step of comparing the current color temperature with a plurality of reference color temperatures to determine a target reference color temperature comprises: Compare the current color temperature with a plurality of reference color temperatures to determine whether there is a first reference color temperature that is the same as the current color temperature; If the first reference color temperature exists, determining the first reference color temperature as the target reference color temperature; or, If the first reference color temperature does not exist, then when there are two similar reference color temperatures close to the current color temperature among the multiple reference color temperatures, the two similar reference color temperatures are determined as the target reference color temperature; wherein the current color temperature is within the color temperature range of the two similar reference color temperatures; or, If the two similar reference color temperatures do not exist, a second reference color temperature close to the current color temperature is determined from the multiple reference color temperatures, and the second reference color temperature is determined as the target reference color temperature; wherein the second reference color temperature is the maximum reference color temperature or the minimum reference color temperature from the multiple reference color temperatures.

3. The method according to claim 2, characterized in that The two similar reference color temperatures include a first similar reference color temperature and a second similar reference color temperature, and determining the first color gain value at the current color temperature according to the target reference color gain value at the target reference color temperature includes: Calculate a first color temperature difference between the current color temperature and the first similar reference color temperature; wherein the first similar reference color temperature is less than the current color temperature; Calculate a second color temperature difference between the second similar reference color temperature and the first similar reference color temperature; Calculating a gain difference between a first reference color gain value at the first similar reference color temperature and a second reference color gain value at the second similar reference color temperature; Calculating a gain adjustment value according to the first color temperature difference, the second color temperature difference and the gain difference; A sum of the gain adjustment value and the first reference color gain value is determined as a first color gain value at the current color temperature.

4. The method according to claim 1, characterized in that: Before determining the target color interval where the current brightness is located, the method further includes: Divide the set preferred color into intervals according to the set brightness to obtain the preferred color intervals; The gray card is photographed under different light sources with the set brightness to set compensation adjustment coefficients of the preferred color range for different light sources.

5. The method according to claim 1, characterized in that The step of adjusting the first color gain value according to the compensation adjustment coefficient of the target color interval and setting the adjustment weight to obtain the second color gain value includes: Calculate the product of the compensation adjustment coefficient of the target color interval, set the adjustment weight and the first color gain value to obtain a first part of the color gain value; Calculating a difference between a set threshold and the set adjustment weight, and calculating a product of the difference and the first color gain value to obtain a second part of the color gain value; A sum of the first portion of color gain values ​​and the second portion of color gain values ​​is calculated to obtain a second color gain value.

6. The method according to claim 1, characterized in that The first color gain value includes a first red gain value and a first blue gain value; The reference color gain value includes a reference red gain value and a reference blue gain value; The compensation adjustment coefficient includes a red gain compensation adjustment coefficient and a blue gain compensation adjustment coefficient; The second color gain value includes a second red gain value and a second blue gain value.

7. The method according to claim 1, characterized in that Before comparing the current color temperature with a plurality of reference color temperatures, the method further includes: Photographing gray cards at multiple reference color temperatures, and calibrating reference points in the captured gray card images; wherein the reference points refer to pixels in a neutral gray area in the gray card images; The color average value of the neutral gray area is calculated, and the color gain value is adjusted based on the color average value to obtain a reference color gain value at each reference color temperature.

8. An image processing device, characterized in that: The device comprises: A first acquisition unit, used to acquire a current color temperature collected by a color temperature sensor from a current shooting environment; A color gain value determination unit, configured to compare the current color temperature with a plurality of reference color temperatures to determine a target reference color temperature, and determine a first color gain value at the current color temperature according to a target reference color gain value at the target reference color temperature; wherein the reference color gain value is determined by photographing a gray card at the reference color temperature; A second acquisition unit, used to acquire the current brightness collected by the brightness sensor from the current shooting environment; A color gain value adjustment unit, used for determining a target color interval where the current brightness is located, and adjusting the first color gain value according to a compensation adjustment coefficient and a set adjustment weight of the target color interval to obtain a second color gain value; wherein the target color interval is a preferred color interval divided according to the set brightness; An image processing unit is used to perform image processing on a current image collected in the current shooting environment according to the second color gain value.

9. An electronic device, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the image processing method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 7 are implemented.

Images