Image processing method and device, equipment and medium

Abstract

The invention discloses an image processing method and device, equipment and a medium, and the method comprises the steps: obtaining a short-exposure image and a long-exposure image of a camera module in a shooting process, and enabling the long-exposure image to be an image obtained through the continuous exposure on the basis of the short-exposure image; determining a first R / G value and a first B / G value of the short exposure image, and determining a second R / G value and a second B / G value of the long exposure image; determining a first gain of the short exposure image according to the first R / G value and the first B / G value, and determining a second gain of the long exposure image according to the second R / G value and the second B / G value; mixing the first gain and the second gain according to a preset proportion, and determining a target gain; and performing white balance reduction on the long exposure image according to the target gain to obtain a target image. According to the invention, the long exposure gain and the short exposure gain are mixed for white balance reduction, so that the reduction effect of white balance is greatly improved, and the color cast degree of a highlight area is greatly reduced.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, device, equipment and medium. Background Art

[0002] With the development of smart electronic devices, people have higher and higher requirements for the image quality of smart electronic devices. The camera module of smart electronic devices may encounter shooting scenes with large brightness differences. At present, the white balance restoration effect of images with large brightness differences is poor, and there is often a color cast in the highlight area. Therefore, how to improve the white balance restoration effect of images with large brightness differences is an urgent problem to be solved. Summary of the invention

[0003] The embodiments of the present application provide an image processing method, device, equipment and medium to solve the technical problems in the prior art of poor white balance restoration effect for images with large brightness differences and color cast in the highlight area, thereby achieving the technical effect of improving the white balance restoration effect for images with large brightness differences and reducing color cast in the highlight area.

[0004] In a first aspect, the present application provides an image processing method, the method comprising:

[0005] Acquire the short-exposure image and the long-exposure image of the camera module during the shooting process, wherein the long-exposure image is an image obtained by continuing to expose the short-exposure image;

[0006] Determine a first R / G value and a first B / G value of the short exposure image, and determine a second R / G value and a second B / G value of the long exposure image;

[0007] Determine a first gain of the short exposure image according to the first R / G value and the first B / G value, and determine a second gain of the long exposure image according to the second R / G value and the second B / G value;

[0008] Mix the first gain and the second gain according to a preset ratio to determine a target gain;

[0009] The long exposure image is white-balanced according to the target gain to obtain the target image.

[0010] Further, determining a first R / G value and a first B / G value of the short exposure image includes:

[0011] Divide the short exposure image into M first sub-images; M is a positive integer;

[0012] Determine the first sub color temperature of each first sub image according to the RGBY data of each first sub image;

[0013] Determine, according to the first sub-color temperatures corresponding to the M first sub-images, the N light source types included in the short exposure image; N is a positive integer;

[0014] Based on a first association relationship between a grayscale value corresponding to the short exposure image and a weight, determining first weights corresponding to each first sub-image according to first grayscale values ​​corresponding to each first sub-image;

[0015] Determine the sum of the first components corresponding to the RGB three channels respectively corresponding to the various light source types based on the first weights of the first sub-images respectively corresponding to the various light source types among the N light source types and the first components respectively corresponding to the RGB three channels;

[0016] According to the sum of the first components corresponding to the three RGB channels corresponding to the various light source types, the first R / G value and the first B / G value corresponding to each light source type in the short exposure image are determined.

[0017] Further, determining a first gain of the short exposure image according to the first R / G value and the first B / G value includes:

[0018] Determine the first light source gains corresponding to the various light source types according to the first R / G values ​​and the first B / G values ​​corresponding to the various light source types;

[0019] Determine first light source brightness levels corresponding to the various light source types according to first actual exposure times and first actual brightness gains corresponding to the various light source types;

[0020] Determine the first light source adjustment gain corresponding to each light source type according to the first light source gain corresponding to each light source type, the first adjustment ratio corresponding to the first light source brightness level, and the user's RGB preference setting parameter;

[0021] The first light source adjustment gains corresponding to the various light source types are combined to obtain a first gain corresponding to the short exposure image.

[0022] Further, determining a first gain of the short exposure image according to the first R / G value and the first B / G value includes:

[0023] A first gain of the short exposure image is determined according to a first difference between the first R / G value and a preset first target R / G value, and a second difference between the first B / G value and a preset first target B / G value.

[0024] Further, determining a first gain of the short exposure image according to the first R / G value and the first B / G value includes:

[0025] Determine the total brightness level corresponding to the short exposure image according to the total exposure time and the total brightness gain corresponding to the short exposure image;

[0026] A first gain of the short exposure image is determined according to the first R / G value and the first B / G value, the total adjustment ratio corresponding to the total brightness level, and the total RGB preference setting parameters of the user.

[0027] Further, the first gain and the second gain are mixed according to a preset ratio to determine the target gain, including:

[0028] Determining a first adjustment gain according to a first preset ratio and a first gain;

[0029] Determining a second adjustment gain according to a second preset ratio and a second gain; the preset ratio includes a first preset ratio and a second preset ratio, and the sum of the first preset ratio and the second preset ratio is 1;

[0030] The sum of the first adjustment gain and the second adjustment gain is determined as the target gain.

[0031] Further, the long exposure image is white-balanced according to the target gain to obtain a target image, including:

[0032] Determining a first starting gain according to a first starting ratio and a first gain; wherein the first starting ratio is less than a first preset ratio;

[0033] Determine the second starting gain according to the second starting ratio and the second gain; the second starting ratio is greater than the second preset ratio; the sum of the first starting ratio and the second starting ratio is 1;

[0034] Controlling the first starting gain to gradually change to the first adjustment gain according to a preset increasing rule, to obtain a plurality of first intermediate gains arranged in sequence, wherein the plurality of first intermediate gains arranged in sequence include the first starting gain and the first adjustment gain;

[0035] The second starting gain is controlled to gradually change to the second adjustment gain according to a preset decreasing rule, so as to obtain a plurality of second intermediate gains arranged in sequence; the plurality of second intermediate gains arranged in sequence include the second starting gain and the second adjustment gain; the preset increasing rule and the preset decreasing rule have the same change amount and opposite change directions; the plurality of first intermediate gains arranged in sequence correspond to the plurality of second intermediate gains arranged in sequence one by one;

[0036] determining a plurality of target intermediate gains arranged in order based on the sum of the respective corresponding first intermediate gains and second intermediate gains, the target intermediate gains comprising a target gain;

[0037] The long exposure image is controlled to perform white balance restoration in sequence according to the target intermediate gains arranged in sequence, so as to obtain a target image corresponding to the target gain.

[0038] In a second aspect, the present application provides an image processing device, the device comprising:

[0039] An image acquisition module is used to acquire a short-exposure image and a long-exposure image of the camera module during shooting. The long-exposure image is an image obtained by continuing to expose the short-exposure image.

[0040] a component determination module, configured to determine a first R / G value and a first B / G value of the short exposure image, and determine a second R / G value and a second B / G value of the long exposure image;

[0041] a gain determination module, configured to determine a first gain of the short exposure image according to the first R / G value and the first B / G value, and to determine a second gain of the long exposure image according to the second R / G value and the second B / G value;

[0042] A gain mixing module, used for mixing the first gain and the second gain according to a preset ratio to determine a target gain;

[0043] The white balance module is used to restore the white balance of the long exposure image according to the target gain to obtain the target image.

[0044] Furthermore, the component determination module is used to:

[0045] Divide the short exposure image into M first sub-images; M is a positive integer;

[0046] Determine the first sub color temperature of each first sub image according to the RGBY data of each first sub image;

[0047] Determine, according to the first sub-color temperatures corresponding to the M first sub-images, the N light source types included in the short exposure image; N is a positive integer;

[0048] Based on a first association relationship between a grayscale value corresponding to the short exposure image and a weight, determining first weights corresponding to each first sub-image according to first grayscale values ​​corresponding to each first sub-image;

[0049] Determine the sum of the first components corresponding to the RGB three channels respectively corresponding to the various light source types based on the first weights of the first sub-images respectively corresponding to the various light source types among the N light source types and the first components respectively corresponding to the RGB three channels;

[0050] According to the sum of the first components corresponding to the three RGB channels corresponding to the various light source types, the first R / G value and the first B / G value corresponding to each light source type in the short exposure image are determined.

[0051] Furthermore, the gain determination module is used to:

[0052] Determine the first light source gains corresponding to the various light source types according to the first R / G values ​​and the first B / G values ​​corresponding to the various light source types;

[0053] Determine first light source brightness levels corresponding to the various light source types according to first actual exposure times and first actual brightness gains corresponding to the various light source types;

[0054] Determine the first light source adjustment gain corresponding to each light source type according to the first light source gain corresponding to each light source type, the first adjustment ratio corresponding to the first light source brightness level, and the user's RGB preference setting parameter;

[0055] The first light source adjustment gains corresponding to the various light source types are combined to obtain a first gain corresponding to the short exposure image.

[0056] Furthermore, the gain determination module is used to:

[0057] A first gain of the short exposure image is determined according to a first difference between the first R / G value and a preset first target R / G value, and a second difference between the first B / G value and a preset first target B / G value.

[0058] Furthermore, the gain determination module is used to:

[0059] Determine the total brightness level corresponding to the short exposure image according to the total exposure time and the total brightness gain corresponding to the short exposure image;

[0060] A first gain of the short exposure image is determined according to the first R / G value and the first B / G value, the total adjustment ratio corresponding to the total brightness level, and the total RGB preference setting parameters of the user.

[0061] Furthermore, the gain mixing module is used to:

[0062] Determining a first adjustment gain according to a first preset ratio and a first gain;

[0063] Determining a second adjustment gain according to a second preset ratio and a second gain; the preset ratio includes a first preset ratio and a second preset ratio, and the sum of the first preset ratio and the second preset ratio is 1;

[0064] The sum of the first adjustment gain and the second adjustment gain is determined as the target gain.

[0065] Furthermore, the white balance module is used to:

[0066] Determining a first starting gain according to a first starting ratio and a first gain; wherein the first starting ratio is less than a first preset ratio;

[0067] Determine the second starting gain according to the second starting ratio and the second gain; the second starting ratio is greater than the second preset ratio; the sum of the first starting ratio and the second starting ratio is 1;

[0068] Controlling the first starting gain to gradually change to the first adjustment gain according to a preset increasing rule, to obtain a plurality of first intermediate gains arranged in sequence, wherein the plurality of first intermediate gains arranged in sequence include the first starting gain and the first adjustment gain;

[0069] The second starting gain is controlled to gradually change to the second adjustment gain according to a preset decreasing rule, so as to obtain a plurality of second intermediate gains arranged in sequence; the plurality of second intermediate gains arranged in sequence include the second starting gain and the second adjustment gain; the preset increasing rule and the preset decreasing rule have the same change amount and opposite change directions; the plurality of first intermediate gains arranged in sequence correspond to the plurality of second intermediate gains arranged in sequence one by one;

[0070] determining a plurality of target intermediate gains arranged in order based on the sum of the respective corresponding first intermediate gains and second intermediate gains, the target intermediate gains comprising a target gain;

[0071] The long exposure image is controlled to perform white balance restoration in sequence according to the target intermediate gains arranged in sequence, so as to obtain a target image corresponding to the target gain.

[0072] In a third aspect, the present application provides an electronic device, including:

[0073] processor;

[0074] a memory for storing processor-executable instructions;

[0075] The processor is configured to execute to implement an image processing method as provided in the first aspect.

[0076] In a fourth aspect, the present application provides a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to implement an image processing method as provided in the first aspect.

[0077] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0078] This embodiment separately obtains the short exposure image and the long exposure image of the camera module during the shooting process, and then determines the R / G value and the B / G value of the short exposure image and the long exposure image; then determines the gains corresponding to the short exposure image and the long exposure image respectively, mixes the gains corresponding to the two respectively to obtain the target gain, and finally performs white balance restoration on the long exposure image according to the target gain to obtain the target image. It can be seen that this embodiment collects the long exposure data and the short exposure data separately when collecting statistical data, and mixes the gain used for the long exposure with the gain for the short exposure to perform white balance restoration on images with large brightness differences, which greatly improves the white balance restoration effect, greatly reduces the degree of color cast in the highlight area, improves the picture quality, and ensures a better viewing experience for users. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0080] Figure 1 A flowchart of an image processing method provided in an embodiment of the present application;

[0081] Figure 2 A schematic diagram of a short exposure image provided in an embodiment of the present application;

[0082] Figure 3 A schematic diagram of a long exposure image provided in an embodiment of the present application;

[0083] Figure 4 A schematic diagram of a light source type coordinate system provided in an embodiment of the present application;

[0084] Figure 5 The first sub-color temperature corresponding to each first sub-image provided in the embodiment of the present application falls within Figure 4 Schematic diagram of the distribution behind the coordinate system shown;

[0085] Figure 6 A schematic diagram of the relationship between grayscale value and brightness value provided in the embodiment of the present application;

[0086] Figure 7 A schematic diagram of the relationship between the brightness value and the weight value corresponding to the short exposure image provided in an embodiment of the present application;

[0087] Figure 8 A schematic diagram of highlighting a brightness area by weighting provided in an embodiment of the present application;

[0088] Fig. 9A schematic diagram of the relationship between the brightness value and the weight value corresponding to the long exposure image provided in an embodiment of the present application;

[0089] Fig.10 A schematic diagram of the relationship between the light source type, brightness level and adjustment ratio provided in an embodiment of the present application;

[0090] Fig.11 A schematic diagram of the RGB preference setting parameters of a user provided in an embodiment of the present application;

[0091] Fig.12 A comparison diagram before and after white balance processing using the image processing method provided in an embodiment of the present application;

[0092] Fig.13 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0093] Fig.14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0094] The embodiment of the present application provides an image processing method to solve the technical problems in the prior art of poor white balance restoration effect for images with large brightness differences and color cast in highlight areas.

[0095] The technical solution of the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows:

[0096] An image processing method, the method comprising: obtaining a short exposure image and a long exposure image of a camera module during shooting, wherein the long exposure image is an image obtained after continued exposure on the basis of the short exposure image; determining a first R / G value and a first B / G value of the short exposure image, and determining a second R / G value and a second B / G value of the long exposure image; determining a first gain of the short exposure image according to the first R / G value and the first B / G value, and determining a second gain of the long exposure image according to the second R / G value and the second B / G value; mixing the first gain and the second gain according to a preset ratio to determine a target gain; and performing white balance restoration on the long exposure image according to the target gain to obtain a target image.

[0097] This embodiment separately obtains the short exposure image and the long exposure image of the camera module during the shooting process, and then determines the R / G value and the B / G value of the short exposure image and the long exposure image; then determines the gains corresponding to the short exposure image and the long exposure image respectively, mixes the gains corresponding to the two respectively to obtain the target gain, and finally performs white balance restoration on the long exposure image according to the target gain to obtain the target image. It can be seen that this embodiment collects the long exposure data and the short exposure data separately when collecting statistical data, and mixes the gain used for the long exposure with the gain for the short exposure to perform white balance restoration on images with large brightness differences, which greatly improves the white balance restoration effect, greatly reduces the degree of color cast in the highlight area, improves the picture quality, and ensures a better viewing experience for users.

[0098] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0099] First of all, the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0100] This embodiment provides Figure 1 An image processing method is shown, the method comprising steps S11 to S15.

[0101] Step S11, obtaining a short exposure image and a long exposure image of the camera module during shooting, wherein the long exposure image is an image obtained by continuing to expose the short exposure image;

[0102] Step S12, determining a first R / G value and a first B / G value of the short exposure image, and determining a second R / G value and a second B / G value of the long exposure image;

[0103] Step S13, determining a first gain of the short exposure image according to the first R / G value and the first B / G value, and determining a second gain of the long exposure image according to the second R / G value and the second B / G value;

[0104] Step S14, mixing the first gain and the second gain according to a preset ratio to determine a target gain;

[0105] Step S15, performing white balance restoration on the long exposure image according to the target gain to obtain a target image.

[0106] The image processing method provided in this embodiment (which may be referred to as a multi-gain method) is mainly used to perform white balance processing on an image, and is particularly suitable for white balance processing in a high dynamic mixed light source scene. The image processing method provided in this embodiment may be executed by a control chip in a camera module, or by other controllers or other platforms that communicate with the camera module, and this embodiment does not limit this.

[0107] The image processing method provided in this embodiment is mainly aimed at images captured in scenes with large brightness differences in the shooting environment. For example, if the shooting picture includes both indoor and outdoor pictures, the brightness difference between the indoor and outdoor pictures will be relatively large. The high-brightness area (i.e., the outdoor area) of such a picture is easy to be exposed, while the low-brightness area (i.e., the indoor area) is easy to lack details. For another example, in the picture recording state, when the picture turns from the outdoor picture to the indoor picture, or from the indoor picture to the outdoor picture, due to the large change in the ambient brightness in the picture, the picture is easy to be exposed and color cast is also easy to occur.

[0108] A white balance method (called X gamma method) is provided in the related art, which mainly collects the data of long exposure and short exposure together. The image data obtained by short exposure is linearly compressed to 12 bits to be consistent with the 12 bits of the image data of long exposure. The RGBY information (including R channel, G channel, B channel, and Y brightness value data) in the image data is used for prediction calculation to estimate the color temperature of the current environment, and then the white balance gain is adjusted to make the white restoration in the image more accurate.

[0109] However, X gamma is only suitable for white balance processing in static scenes or low-dynamic scenes, and has a poor effect on white balance processing in high-dynamic scenes, and cannot compensate for the color temperatures of high and low brightness areas respectively. In order to solve this problem, this embodiment provides the following Figure 1 An image processing method is shown to improve the white balance restoration effect of images in high dynamic scenes.

[0110] In actual application scenarios, the camera module has various shooting modes, including static shooting (photographing), low dynamic shooting, high dynamic shooting, etc. The shooting environment may also be diverse, including indoor, outdoor, indoor and outdoor mixed, single light source, multiple light sources, etc. In fact, indoor and outdoor mixed is a case of multiple light source scene, indoor may include artificial active light source, outdoor may include natural light source.

[0111] Therefore, during the actual shooting process of the camera module, the current shooting environment type can be detected through the preview image, and the white balance processing method currently used by the camera module can be determined in combination with the shooting mode currently used by the camera module.

[0112] When the shooting mode of the camera module is high dynamic shooting and the shooting environment is a mixture of indoor and outdoor, the camera can be controlled to use the image processing method provided in this embodiment to perform white balance processing, and the color temperature of high and low brightness areas can be compensated respectively to achieve a relatively good white balance restoration effect, that is, execute steps S11 to S15.

[0113] When the shooting mode of the camera module is static shooting or low-dynamic shooting, and the shooting environment is a mixture of indoor and outdoor, the camera can be controlled to use the X gamma method to perform white balance restoration processing. Although the image processing method provided in this embodiment can also be used to perform white balance processing in this case, it occupies more computing resources. Using X gamma can meet the current white balance requirements and achieve a certain energy saving effect.

[0114] When the shooting mode of the camera module is static shooting and the shooting environment is indoor shooting, the multi-gain mode provided in this embodiment may not be adopted, and the X gamma mode may not be adopted, thereby achieving a certain energy saving effect.

[0115] It can be seen that the multi-gain mode provided in this embodiment is more suitable for high dynamic range scenes, and can solve more white balance problems in high dynamic range scenes. The most critical thing is that it can take into account the white balance state of the highlight area, while the Xgamma solution cannot take into account the white balance state of the highlight area. Retaining these two algorithms in the camera module at the same time can provide users with choices. Users can decide whether to use the multi-gain mode or Xgamma according to the actual use environment and image quality requirements. In addition, the multi-gain mode provided in this embodiment is to calculate the short exposure data and the long exposure data separately and independently, and the two must be integrated, so the power consumption of the chip of the camera module is relatively high. Therefore, the multi-gain mode provided in this embodiment can be used when necessary, which can effectively save energy.

[0116] The image processing method provided in this embodiment is now described as follows.

[0117] Regarding step S11, a short exposure image and a long exposure image of the camera module during the shooting process are obtained, and the long exposure image is an image obtained by continuing to expose the short exposure image.

[0118] The shooting process can be a process of static shooting, low-dynamic shooting, and high-dynamic shooting. A short-exposure image refers to an image captured by the camera module at a shorter exposure time, and a long-exposure image is an image obtained by continuing to expose the short-exposure image. In other words, during the exposure process of the camera module, an image data is obtained before the entire exposure action is completed, which is a short-exposure image, and an image data is obtained after the entire exposure action is completed, which is a long-exposure image.

[0119] In this embodiment, short exposure data and long exposure data are collected separately to obtain corresponding short exposure images and long exposure images.

[0120] The short exposure mode (which can be recorded as SE mode) is the image captured by the chip at the moment of exposure, that is, when the time is very short. It can be understood that the shutter is very fast, so that the highlight area in the picture can be well presented. For example, in a high dynamic range scene, the data collected by the short exposure mode is all in the highlight area, and other dark areas are basically black. Short exposure can make the outdoor scene clear, so that the image RGBY data of the highlight area can be well collected, such as Figure 2 Shown is a short exposure image.

[0121] The long exposure mode (also called LE mode) can be understood as the normal exposure mode. For example, in a high dynamic range scene, the indoor part is dark. Although long exposure can increase the indoor brightness, the highlight area will be overexposed. Figure 3 It should be noted that short exposure and long exposure are relative, and there is no specific exposure time limit to distinguish short exposure from long exposure.

[0122] Regarding step S12 , a first R / G value and a first B / G value of the short exposure image are determined, and a second R / G value and a second B / G value of the long exposure image are determined.

[0123]

Method 1 for determining R / G and B / G

[0124] For a short exposure image, the RGBY data in the short exposure image may be counted to determine the first R / G value and the first B / G value. For a long exposure image, the RGBY data in the long exposure image may be counted to determine the second R / G value and the second B / G value.

[0125] The first method is to regard the entire image of the short exposure image and the long exposure image as a whole, and then determine the R / G value and B / G value of the entire image. This method is simple and can quickly calculate the R / G value and B / G value corresponding to the short exposure image and the long exposure image, respectively, and is relatively efficient. However, due to the low brightness area in the short exposure image and the high brightness area in the long exposure image, the accuracy of the R / G value and B / G value calculated by each is relatively low. Although the final white balance effect is improved, the degree of improvement is not much.

[0126]

Method 2 for determining R / G and B / G

[0127] In order to further improve the accuracy of the R / G value and the B / G value, this embodiment provides a second method, including steps S1211 to S1216.

[0128] Step S1211, dividing the short exposure image into M first sub-images; M is a positive integer;

[0129] Step S1212, determining the first sub color temperature of each first sub image according to the RGBY data of each first sub image;

[0130] Step S1213, determining N light source types included in the short exposure image according to the first sub color temperatures corresponding to the M first sub images, where N is a positive integer;

[0131] Step S1214, based on the first correlation relationship between the grayscale value corresponding to the short exposure image and the weight, determining the first weight corresponding to each first sub-image according to the first grayscale value corresponding to each first sub-image;

[0132] Step S1215, determining the sum of the first components corresponding to the RGB channels respectively corresponding to the N light source types based on the first weights of the first sub-images respectively corresponding to the N light source types and the first components respectively corresponding to the RGB channels;

[0133] Step S1216, determining first R / G values ​​and first B / G values ​​corresponding to each light source type in the short exposure image according to the sum of the first components corresponding to the three RGB channels corresponding to each light source type.

[0134] Regarding step S1211, the short exposure image is divided into M first sub-images. When M=1, the aforementioned method is to regard the entire image of the short exposure image and the long exposure image as a whole, and then determine the R / G value and B / G value of each of the entire image. Steps S1211 to S1216 are mainly described for the case where M>1.

[0135] M can be selected according to actual conditions, and the shapes and sizes of the first sub-images can be the same or different. When M is larger, the data for subsequent calculations are more accurate and the white balance effect is better, but the computing resources occupied are relatively more. On the contrary, when M is smaller, the data for subsequent calculations are less accurate and the white balance effect is worse, but the computing resources occupied are relatively less. When the shapes and sizes of the first sub-images are the same, the division method is simpler and the calculation method is also simpler. When the shapes and sizes of the first sub-images are different, the division method is more complicated and the calculation method is also more complicated. The shapes and sizes of the first sub-images have little effect on the subsequent white balance effect. Therefore, the short exposure image can be divided into first sub-images with the same shape and size.

[0136] For example, the short exposure image may be equally divided into 120 parts horizontally and 96 parts vertically to form a 120*96 square matrix, then M is 11520. This embodiment will only take this as an example for subsequent description.

[0137] Regarding step S1212, for each first sub-image, the corresponding first sub-R / G value and the first sub-B / G value are determined based on the RGBY data in the first sub-image, and the Planck curve is queried based on the first sub-R / G value and the first sub-B / G value, so as to determine the first sub-color temperature corresponding to the first sub-image. The color temperature corresponds to the light source, and the color temperature may include H light, TL84, D65, D75, A, CWF, DF, etc.

[0138] Regarding step S1213, based on the first sub-color temperatures corresponding to the M first sub-images, the N light source types contained in the short exposure image are determined. In actual operation, the light source chip of the camera module can be calibrated in a standard light box to obtain the following: Figure 4 The color temperature coordinate system shown, Figure 4 The color temperature coordinate system shown is equivalent to an RGB color space. Figure 4 The boxes of different colors correspond to different light source types.

[0139] The first sub color temperature corresponding to each first sub image is matched with the light source type corresponding to each box, and the first sub color temperature corresponding to each first sub image falls into the box corresponding to the matched light source type, forming the following Figure 5 The distribution diagram of the falling points shown in FIG. 1 can further determine the N light source types, and the M first sub-images and the N light source types can also correspond to each other. For example, when M is 11520, then Figure 5 The number of landing points in is also 11520. This embodiment is described below by taking H light, TL84, D65, D75, A, CWF, and DF as examples, that is, N is 7 as an example.

[0140] It should be noted that Figure 4 The size of the box can be adjusted according to the actual situation. For example, when you want to increase the proportion of light sources corresponding to the green box, you can enlarge the green box. Then the number of points falling into the green box is greater, and the proportion of light sources corresponding to the green box is also increased.

[0141] Based on the relationship between the first sub color temperatures corresponding to the M first sub images and the N light source types, the light source ratio of the N light source types in the short exposure image can be determined, and the light source ratio will affect the subsequent white balance gain compensation.

[0142] Regarding step S1214, the first association relationship between the gray value corresponding to the short exposure image and the weight is mainly composed of two sub-associations. The first sub-association relationship is the association relationship between the gray value and the brightness value, such as Figure 6 As shown in , the horizontal axis is the gray value and the vertical axis is the brightness value. The second sub-association relationship is the association relationship between the brightness value and the weight value, such as Figure 7 As shown, the horizontal axis is the brightness value and the vertical axis is the weight value.

[0143] It should be noted that Figure 6 and Figure 7 There are 17 points set on the curve shown. These 17 points belong to the standard gamma data points and are a fixed set of values. Some points are selected from the curve to simplify the calculation and reduce the chip computing resources. If the continuous points on the curve are used for calculation, the amount of data is too large, and the exponential operation will consume a lot of computing resources. Therefore, the gamma curve is actually fitted with 17 line segments, so that only the data of 17 points need to be saved, and the gamma correction is performed by linear transformation or table lookup, and then the brightness value and weight value are quickly determined.

[0144] The grayscale value of the image data can be calculated using floating point arithmetic, integer method, shift method, average method, etc. mentioned in the related art. This embodiment only uses the average method as an example for explanation. Figure 6 As shown, the grayscale value of the horizontal axis is determined based on the average value of the RGB three-channel data, wherein Ravg, Gavg, and Bavg represent the average values ​​of the RGB single channel of the first sub-image.

[0145] First, the average value method is used to determine the first grayscale value corresponding to each first sub-image, and then query Figure 6 and Figure 7 The first association relationship shown determines the first weights corresponding to the first sub-images.

[0146] Regarding step S1215, the M first sub-images correspond to the N light source types, that is, the M first sub-images are respectively divided into the light source types that match them. The first components corresponding to the RGB three channels of the multiple first sub-images corresponding to each light source type include a first R component, a first G component, and a first B component.

[0147] For multiple first sub-images corresponding to each light source type, the first R component, the first G component and the first B component of each first sub-image are respectively multiplied by the corresponding first weights to obtain the corresponding first R adjustment component, the first G adjustment component and the first B adjustment component. Then the adjustment components corresponding to the same channels of the multiple first sub-images are added to obtain the sum of the first components corresponding to the corresponding channels, that is, the first R adjustment components of the multiple first sub-images are added to obtain the sum of the first R component, the first G adjustment components of the multiple first sub-images are added to obtain the sum of the first G component, and the first B adjustment components of the multiple first sub-images are added to obtain the sum of the first B component. It should be noted that in this embodiment, by setting weights, the image in the highlighted area of ​​the image can be highlighted for subsequent processing, such as Figure 8 As shown, the picture is formed by highlighting the highlight area of ​​the short exposure image, wherein the black area is the low brightness area, and the blue, yellow and other color areas are the high brightness areas.

[0148] Regarding step S1216, for each light source type, the sum of the corresponding first R components is divided by the sum of the first G components to obtain the first R / G value corresponding to the light source type, and the sum of the corresponding first B components is divided by the sum of the first G components to obtain the first B / G value corresponding to the light source type.

[0149] It can be seen that this embodiment divides the short exposure image into M first sub-images, analyzes the light source type corresponding to the short exposure image, and then performs image processing on each first sub-image according to the different light source types to obtain the first R / G values ​​and first B / G values ​​corresponding to the different light source types.

[0150] The principle of determining the second R / G value and the second B / G value of the long exposure image is the same as the principle of determining the first R / G value and the first B / G value of the short exposure image, and any one of the above two methods may be used. When the above method 2 is used to determine the second R / G value and the second B / G value of the long exposure image, reference may be made to steps S1221 to S1226.

[0151] Step S1221, dividing the long exposure image into P second sub-images; P is a positive integer; P can be much smaller than M, for example, when M is 11520, P can be 30*24=720.

[0152] Step S1222, determining the second sub color temperature of each second sub image according to the RGBY data of each second sub image;

[0153] Step S1223, determining Q types of light sources included in the long exposure image according to the second sub color temperatures corresponding to the P second sub images, where Q is a positive integer;

[0154] Step S1224, based on the second association relationship between the grayscale value corresponding to the long exposure image and the weight, determine the second weight corresponding to each second sub-image according to the second grayscale value corresponding to each second sub-image;

[0155] Step S1225, based on the second weights of the second sub-images corresponding to the Q light source types and the second components corresponding to the RGB three channels, the sum of the second components corresponding to the RGB three channels corresponding to the various light source types is determined;

[0156] Step S1226, determining the second R / G value and the second B / G value corresponding to each light source type in the long exposure image according to the sum of the second components corresponding to the three RGB channels corresponding to each light source type.

[0157] The principles of steps S1221 to S1226 are the same as those of steps S1211 to S1216, except that in step S1224, the second association relationship is mainly composed of two sub-associations, and the first sub-association relationship is the association relationship between the gray value and the brightness value, such as Figure 6 As shown in , the horizontal axis is the gray value and the vertical axis is the brightness value. The second sub-association relationship is the association relationship between the brightness value and the weight value, such as Fig. 9 As shown, the horizontal axis is the brightness value and the vertical axis is the weight value. Figure 7 Replaced with Fig. 9 Since the highlight weight needs to be reduced in long exposure images, the brighter the area, the smaller the weight. Fig. 9 The brightness value 190 is used as the boundary, and brightness values ​​greater than or equal to 190 are considered as highlight areas. The brightness value of 190 corresponds to the starting weight, for example, 32, and the brightness value of 256 corresponds to the ending weight, for example, 4. When the brightness value changes from 190 to 256, the weight also decreases from the starting weight to the ending weight according to a certain slope.

[0158] That is to say, when executing step S1224, it is necessary to query the brightness value Fig. 9 The corresponding second weight is obtained.

[0159] Regarding step S13 , a first gain of the short exposure image is determined according to the first R / G value and the first B / G value, and a second gain of the long exposure image is determined according to the second R / G value and the second B / G value.

[0160] [Method 3] corresponds to the first R / G value and the first B / G value calculated in the above-mentioned method 1.

[0161] A first gain of the short exposure image is determined according to a first difference between the first R / G value and a preset first target R / G value, and a second difference between the first B / G value and a preset first target B / G value.

[0162] The first target R / G value and the first target B / G value can take any positive value less than or equal to 1. When the first target R / G value and the first target B / G value are closer to 1, the white balance effect is better. A first gain of the short exposure image can be determined based on the first difference and the second difference.

[0163] On this basis, the influence of the total brightness level and the user's RGB preference factors on the gain of the short exposure image may also be considered, including steps S1311 and S1312.

[0164] Step S1311 , determining the total brightness level corresponding to the short exposure image according to the total exposure time and the total brightness gain corresponding to the short exposure image.

[0165] Step S1312: determining a first gain of the short exposure image according to the first R / G value and the first B / G value, the total adjustment ratio corresponding to the total brightness level, and the total RGB preference setting parameters of the user.

[0166] Regarding step S1311, the chip of the camera module will record the total exposure time and total brightness gain corresponding to the short exposure image, and then determine the total brightness level.

[0167] Regarding step S1312, the corresponding total adjustment ratio can be determined by querying the correspondence between the brightness level and the adjustment ratio according to the total brightness level. The user's RGB preference setting total parameter is the user's adjustment of the RGB component value. Combining the first R / G value and the first B / G value, the total adjustment ratio corresponding to the total brightness level, and the user's RGB preference setting total parameter, the first gain of the short exposure image can be obtained more accurately.

[0168] [Method 4] corresponds to the first R / G value and the first B / G value calculated in the aforementioned method 2, including steps S1321 to S1324.

[0169] Step S1321, determining first light source gains corresponding to various light source types according to first R / G values ​​and first B / G values ​​corresponding to various light source types respectively;

[0170] Step S1322, determining first light source brightness levels corresponding to the various light source types according to the first actual exposure times and first actual brightness gains corresponding to the various light source types;

[0171] Step S1323, determining the first light source adjustment gain corresponding to each light source type according to the first light source gain corresponding to each light source type, the first adjustment ratio corresponding to the first light source brightness level, and the user's RGB preference setting parameter;

[0172] Step S1324: Combine the first light source adjustment gains corresponding to the various light source types to obtain a first gain corresponding to the short exposure image.

[0173] Regarding step S1321, the first light source gain corresponding to each light source type is determined according to the first R / G value and the first B / G value corresponding to each light source type, combined with the target R / G value and the target B / G value. Among them, the target R / G value and the target B / G value can take any positive value less than or equal to 1. When the target R / G value and the target B / G value are closer to 1, the white balance effect is better. The first light source gain of the short exposure image can be determined based on the difference between the first R / G value corresponding to each light source type and the target R / G value, and the difference between the first B / G value and the target B / G value.

[0174] Regarding step S1322, the chip of the camera module will record the first actual exposure time and the first actual brightness gain corresponding to each light source type, and then determine the first light source brightness level corresponding to each light source type.

[0175] Regarding step S1323, based on the first light source brightness level corresponding to each light source type, query Fig.10 The first adjustment ratios corresponding to different light source types and different brightness levels are shown. Fig.10 The first row represents the brightness level, the first column represents the light source type, and the data in the remaining area represents the adjustment ratio in percentage.

[0176] The user's RGB preference setting parameters are specifically the user's preference parameters for the RGB channels, such as Fig.11 The following is an example of RGB preference setting parameters.

[0177] The first light source gain corresponding to each light source type is multiplied by the first adjustment ratio to obtain an adjusted first light source gain, and then the adjusted first light source gain is adjusted based on the user's RGB preference setting parameters to obtain a first light source adjustment gain.

[0178] Regarding step S1324, the first light source adjustment gains corresponding to the N light source types are added together to obtain the first gain corresponding to the short exposure image.

[0179] The calculation principle of the second gain for the long exposure image is similar to the calculation principle of the first gain for the short exposure image. For details, please refer to the aforementioned method 3 and method 4, which will not be described in detail in this embodiment.

[0180] Regarding step S14, the first gain and the second gain are mixed according to a preset ratio to determine a target gain.

[0181] Step S14 can be decomposed into step S1411 to step S1413.

[0182] Step S1411, determining a first adjustment gain according to a first preset ratio and a first gain;

[0183] Step S1412, determining a second adjustment gain according to a second preset ratio and a second gain; the preset ratio includes a first preset ratio and a second preset ratio, and the sum of the first preset ratio and the second preset ratio is 1;

[0184] Step S1413: determine the sum of the first adjustment gain and the second adjustment gain as the target gain.

[0185] Theoretically, the first adjustment gain is used to adjust the short exposure image to obtain the high brightness area in the short exposure image. The second adjustment gain is used to adjust the long exposure image to obtain the details of the low brightness area in the long exposure image. However, in order to make the final target image more realistic, it is necessary to combine the first adjustment gain and the second adjustment gain to obtain the target gain, and use one target gain to adjust the long exposure image, so as to achieve a better white balance effect while ensuring that the target image is relatively more realistic.

[0186] Regarding step S15, white balance restoration is performed on the long exposure image according to the target gain to obtain a target image.

[0187] The target gain can be used to directly restore the white balance of the long exposure image to obtain the final target image. The white balance effect of the target image is better, such as Fig.12 As shown, the left picture is the picture before white balance, and the right picture is the picture after white balance. It can be seen that the color cast phenomenon after white balance has been effectively improved, and the white balance restoration effect has been effectively improved.

[0188] However, when the long exposure image directly uses the target gain to achieve white balance to obtain the target image, the naked eye can clearly see the change from the long exposure image to the target image, causing a jump in image quality. If the camera module is in the recording state, the sudden jump in image quality will result in a large image quality and a poor viewing experience. To solve this problem, this embodiment provides steps S1511 to S1516.

[0189] Step S1511, determining a first starting gain according to a first starting ratio and a first gain; the first starting ratio is less than a first preset ratio;

[0190] Step S1512, determining a second starting gain according to the second starting ratio and the second gain; the second starting ratio is greater than the second preset ratio; the sum of the first starting ratio and the second starting ratio is 1;

[0191] Step S1513, controlling the first starting gain to gradually change to the first adjustment gain according to a preset increasing rule, to obtain a plurality of first intermediate gains arranged in sequence, wherein the plurality of first intermediate gains arranged in sequence include the first starting gain and the first adjustment gain;

[0192] Step S1514, controlling the second starting gain to gradually change to the second adjustment gain according to a preset decreasing rule, and obtaining a plurality of second intermediate gains arranged in sequence; the plurality of second intermediate gains arranged in sequence include the second starting gain and the second adjustment gain; the preset increasing rule and the preset decreasing rule have the same amount of change and opposite directions of change; the plurality of first intermediate gains arranged in sequence correspond to the plurality of second intermediate gains arranged in sequence one by one;

[0193] Step S1515, determining a plurality of target intermediate gains arranged in order based on the sum of the first intermediate gains and the second intermediate gains corresponding to each other, the target intermediate gains including the target gain;

[0194] Step S1516, controlling the long exposure image to perform white balance restoration in sequence according to the target intermediate gains arranged in sequence, to obtain a target image corresponding to the target gain.

[0195] Regarding steps S1511 - S1512 , a first starting gain and a second starting gain are determined.

[0196] Regarding steps S1513-S1515, the preset increasing rule can be a rule of continuous change, for example, the preset increasing rule conforms to a straight line change rule with a certain slope; as shown in example 1, it can also be the rule of an arithmetic progression, for example, gradually changing from 10% to 80% in steps of 10%. The preset decreasing rule is the same size as the preset increasing rule but in the opposite direction. The first gain is denoted as G1, and the second gain is denoted as G2, where 10%*G1 is the first starting gain, 80%*G1 is the first adjustment gain, 90%*G1 is the second starting gain, and 20%*G1 is the second adjustment gain. The first intermediate gain and the second intermediate gain of the same row are added to obtain the corresponding target intermediate gain.

[0197] Table 1

[0198]

[0199]

[0200] Regarding step S1516, the target intermediate gain obtained is shown in the rightmost column of Table 1, and the order is from top to bottom. The long exposure image is restored to white balance in sequence according to the target intermediate gain from top to bottom. Then the long exposure image screen will gradually change from the screen corresponding to Output1 to the screen corresponding to Output8 in sequence. During this change process, due to the limited dynamic vision of the human eye, it will not be seen that the screen is changing, but what is finally presented to the user is the target screen after white balance restoration. In this way, the image quality will not suddenly jump, thereby improving the imaging quality and ensuring a good user experience.

[0201] In summary, this embodiment separately obtains the short exposure image and the long exposure image of the camera module during the shooting process, and then determines the R / G value and the B / G value of the short exposure image and the long exposure image; then determines the gains corresponding to the short exposure image and the long exposure image respectively, mixes the gains corresponding to the two respectively to obtain the target gain, and finally performs white balance restoration on the long exposure image according to the target gain to obtain the target image. It can be seen that this embodiment collects the long exposure data and the short exposure data separately when collecting statistical data, and mixes the gain used for the long exposure with the gain for the short exposure to perform white balance restoration on images with large brightness differences, which greatly improves the white balance restoration effect, greatly reduces the degree of color cast in the highlight area, improves the picture quality, and ensures a better viewing experience for users.

[0202] Based on the same inventive concept, this embodiment provides Fig.13 An image processing device as shown, the device comprises:

[0203] The image acquisition module 131 is used to acquire a short-exposure image and a long-exposure image of the camera module during shooting, wherein the long-exposure image is an image obtained by further exposure based on the short-exposure image;

[0204] A component determination module 132, configured to determine a first R / G value and a first B / G value of the short exposure image, and determine a second R / G value and a second B / G value of the long exposure image;

[0205] a gain determination module 133, configured to determine a first gain of the short exposure image according to the first R / G value and the first B / G value, and to determine a second gain of the long exposure image according to the second R / G value and the second B / G value;

[0206] A gain mixing module 134, configured to mix the first gain and the second gain according to a preset ratio to determine a target gain;

[0207] The white balance module 135 is used to restore the white balance of the long exposure image according to the target gain to obtain the target image.

[0208] Furthermore, the component determination module 132 is used to:

[0209] Divide the short exposure image into M first sub-images; M is a positive integer;

[0210] Determine the first sub color temperature of each first sub image according to the RGBY data of each first sub image;

[0211] Determine, according to the first sub-color temperatures corresponding to the M first sub-images, the N light source types included in the short exposure image; N is a positive integer;

[0212] Based on a first association relationship between a grayscale value corresponding to the short exposure image and a weight, determining first weights corresponding to each first sub-image according to first grayscale values ​​corresponding to each first sub-image;

[0213] Determine the sum of the first components corresponding to the RGB three channels respectively corresponding to the various light source types based on the first weights of the first sub-images respectively corresponding to the various light source types among the N light source types and the first components respectively corresponding to the RGB three channels;

[0214] According to the sum of the first components corresponding to the three RGB channels corresponding to the various light source types, the first R / G value and the first B / G value corresponding to each light source type in the short exposure image are determined.

[0215] Furthermore, the gain determination module 133 is used to:

[0216] Determine the first light source gains corresponding to the various light source types according to the first R / G values ​​and the first B / G values ​​corresponding to the various light source types;

[0217] Determine first light source brightness levels corresponding to the various light source types according to first actual exposure times and first actual brightness gains corresponding to the various light source types;

[0218] Determine the first light source adjustment gain corresponding to each light source type according to the first light source gain corresponding to each light source type, the first adjustment ratio corresponding to the first light source brightness level, and the user's RGB preference setting parameter;

[0219] The first light source adjustment gains corresponding to the various light source types are combined to obtain a first gain corresponding to the short exposure image.

[0220] Furthermore, the gain determination module 133 is used to:

[0221] A first gain of the short exposure image is determined according to a first difference between the first R / G value and a preset first target R / G value, and a second difference between the first B / G value and a preset first target B / G value.

[0222] Furthermore, the gain determination module 133 is used to:

[0223] Determine the total brightness level corresponding to the short exposure image according to the total exposure time and the total brightness gain corresponding to the short exposure image;

[0224] A first gain of the short exposure image is determined according to the first R / G value and the first B / G value, the total adjustment ratio corresponding to the total brightness level, and the total RGB preference setting parameters of the user.

[0225] Furthermore, the gain mixing module 134 is used to:

[0226] Determining a first adjustment gain according to a first preset ratio and a first gain;

[0227] Determining a second adjustment gain according to a second preset ratio and a second gain; the preset ratio includes a first preset ratio and a second preset ratio, and the sum of the first preset ratio and the second preset ratio is 1;

[0228] The sum of the first adjustment gain and the second adjustment gain is determined as the target gain.

[0229] Furthermore, the white balance module 135 is used to:

[0230] Determining a first starting gain according to a first starting ratio and a first gain; wherein the first starting ratio is less than a first preset ratio;

[0231] Determine the second starting gain according to the second starting ratio and the second gain; the second starting ratio is greater than the second preset ratio; the sum of the first starting ratio and the second starting ratio is 1;

[0232] Controlling the first starting gain to gradually change to the first adjustment gain according to a preset increasing rule, to obtain a plurality of first intermediate gains arranged in sequence, wherein the plurality of first intermediate gains arranged in sequence include the first starting gain and the first adjustment gain;

[0233] The second starting gain is controlled to gradually change to the second adjustment gain according to a preset decreasing rule, so as to obtain a plurality of second intermediate gains arranged in sequence; the plurality of second intermediate gains arranged in sequence include the second starting gain and the second adjustment gain; the preset increasing rule and the preset decreasing rule have the same change amount and opposite change directions; the plurality of first intermediate gains arranged in sequence correspond to the plurality of second intermediate gains arranged in sequence one by one;

[0234] determining a plurality of target intermediate gains arranged in order based on the sum of the respective corresponding first intermediate gains and second intermediate gains, the target intermediate gains comprising a target gain;

[0235] The long exposure image is controlled to perform white balance restoration in sequence according to the target intermediate gains arranged in sequence, so as to obtain a target image corresponding to the target gain.

[0236] Based on the same inventive concept, this embodiment provides Fig.14 An electronic device as shown includes:

[0237] Processor 141;

[0238] A memory 142 for storing instructions executable by the processor 141;

[0239] The processor 141 is configured to execute to implement an image processing method as provided above.

[0240] Based on the same inventive concept, this embodiment provides a non-transitory computer-readable storage medium. When the instructions in the storage medium are executed by the processor 141 of the electronic device, the electronic device can execute an image processing method as provided above.

[0241] Since the electronic device introduced in this embodiment is an electronic device used to implement the information processing method in the embodiment of the present application, based on the information processing method introduced in the embodiment of the present application, a person skilled in the art can understand the specific implementation of the electronic device of the present embodiment and its various variations, so how the electronic device implements the method in the embodiment of the present application is not described in detail here. As long as a person skilled in the art implements the electronic device used by the information processing method in the embodiment of the present application, it belongs to the scope of protection of this application.

[0242] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0243] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0244] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0245] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0246] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0247] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a short exposure image and a long exposure image of the camera module during shooting, wherein the long exposure image is an image obtained by further exposing the short exposure image; Determining a first R / G value and a first B / G value of the short exposure image, and determining a second R / G value and a second B / G value of the long exposure image; determining a first gain of the short-exposure image according to the first R / G value and the first B / G value, and determining a second gain of the long-exposure image according to the second R / G value and the second B / G value; Mixing the first gain and the second gain according to a preset ratio to determine a target gain; White balance restoration is performed on the long exposure image according to the target gain to obtain a target image.

2. The method according to claim 1, characterized in that The determining a first R / G value and a first B / G value of the short exposure image includes: Divide the short exposure image into M first sub-images; M is a positive integer; Determine the first sub color temperature of each of the first sub images respectively according to the RGBY data of each of the first sub images; Determine, according to the first sub-color temperatures corresponding to the M first sub-images, N light source types included in the short-exposure image, where N is a positive integer; Based on a first association relationship between a grayscale value corresponding to the short exposure image and a weight, determining first weights corresponding to each of the first sub-images according to first grayscale values ​​corresponding to each of the first sub-images; Determine, based on the first weights of the first sub-images respectively corresponding to the various light source types among the N light source types and the first components respectively corresponding to the RGB three channels; The first R / G value and the first B / G value respectively corresponding to each light source type in the short-exposure image are determined according to the sum of the first components respectively corresponding to the three RGB channels respectively corresponding to the various light source types.

3. The method according to claim 2, characterized in that The determining a first gain of the short exposure image according to the first R / G value and the first B / G value includes: Determine the first light source gains corresponding to the various light source types according to the first R / G values ​​and the first B / G values ​​corresponding to the various light source types; Determine first light source brightness levels corresponding to the various light source types according to first actual exposure times and first actual brightness gains corresponding to the various light source types; Determine the first light source adjustment gain corresponding to each light source type according to the first light source gain corresponding to each light source type, the first adjustment ratio corresponding to the brightness level of the first light source, and the user's RGB preference setting parameter; The first light source adjustment gains corresponding to various light source types are combined to obtain the first gain corresponding to the short exposure image.

4. The method according to claim 1, characterized in that The determining a first gain of the short exposure image according to the first R / G value and the first B / G value includes: The first gain of the short exposure image is determined according to a first difference between the first R / G value and a preset first target R / G value, and a second difference between the first B / G value and a preset first target B / G value.

5. The method according to claim 1, characterized in that The determining a first gain of the short exposure image according to the first R / G value and the first B / G value includes: Determining a total brightness level corresponding to the short exposure image according to a total exposure time and a total brightness gain corresponding to the short exposure image; The first gain of the short exposure image is determined according to the first R / G value and the first B / G value, the total adjustment ratio corresponding to the total brightness level, and the total RGB preference setting parameter of the user.

6. The method according to claim 1, characterized in that The mixing the first gain and the second gain according to a preset ratio to determine a target gain includes: Determining a first adjustment gain according to a first preset ratio and the first gain; Determining a second adjustment gain according to a second preset ratio and the second gain; the preset ratio includes a first preset ratio and the second preset ratio, and the sum of the first preset ratio and the second preset ratio is 1; A sum of the first adjustment gain and the second adjustment gain is determined as the target gain.

7. The method according to claim 6, characterized in that The performing white balance restoration on the long exposure image according to the target gain to obtain a target image includes: Determining a first starting gain according to a first starting ratio and the first gain; the first starting ratio is less than the first preset ratio; Determining a second starting gain according to a second starting ratio and the second gain; the second starting ratio is greater than the second preset ratio; the sum of the first starting ratio and the second starting ratio is 1; Controlling the first starting gain to gradually change to the first adjustment gain according to a preset increasing rule, to obtain a plurality of first intermediate gains arranged in sequence, wherein the plurality of first intermediate gains arranged in sequence include the first starting gain and the first adjustment gain; Control the second starting gain to gradually change to the second adjustment gain according to a preset decreasing rule, so as to obtain a plurality of second intermediate gains arranged in sequence; the plurality of second intermediate gains arranged in sequence include the second starting gain and the second adjustment gain; the preset increasing rule and the preset decreasing rule have the same change amount and opposite change directions; the plurality of first intermediate gains arranged in sequence correspond to the plurality of second intermediate gains arranged in sequence one by one; determining a plurality of target intermediate gains arranged in order based on the sum of the first intermediate gains and the second intermediate gains of the respective corresponding ones, the target intermediate gains including the target gain; The long exposure image is controlled to perform white balance restoration in sequence according to the target intermediate gains arranged in sequence, so as to obtain the target image corresponding to the target gain.

8. An image processing device, characterized in that: The device comprises: An image acquisition module, used to acquire a short-exposure image and a long-exposure image of the camera module during shooting, wherein the long-exposure image is an image obtained by continuing to expose the short-exposure image; a component determination module, configured to determine a first R / G value and a first B / G value of the short exposure image, and determine a second R / G value and a second B / G value of the long exposure image; a gain determination module, configured to determine a first gain of the short exposure image according to the first R / G value and the first B / G value, and to determine a second gain of the long exposure image according to the second R / G value and the second B / G value; a gain mixing module, configured to mix the first gain and the second gain according to a preset ratio to determine a target gain; The white balance module is used to restore the white balance of the long exposure image according to the target gain to obtain a target image.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute to implement an image processing method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to implement an image processing method as claimed in any one of claims 1 to 7.

Images