Image processing method, device and storage medium

Abstract

The embodiment of the present application discloses an image processing method, device and storage medium, wherein the method comprises: obtaining an image to be processed, the image to be processed is a monochrome scene image; determining a first 3D histogram corresponding to the image to be processed; obtaining a target reference image, and determining a second 3D histogram corresponding to the target reference image; determining a target correlation between the image to be processed and the target reference image according to the first 3D histogram and the second 3D histogram; determining a target color temperature value according to the target correlation; determining a target gain value of the image to be processed according to the target color temperature value; adjusting the image to be processed according to the target gain value to obtain an output image. The embodiment of the present application can improve image quality for monochrome scenes.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device and storage medium. Background Art

[0002] With the rapid development of electronic technology, taking pictures has become a standard technology for electronic devices (such as mobile phones, tablets, etc.). Taking pictures inevitably involves white balance technology. At present, general white balance algorithms are only effective for scenes containing white spots (gray spots, neutral gray). However, how to improve image quality for scenes without white spots needs to be solved urgently. Summary of the invention

[0003] The embodiments of the present application provide an image processing method, device, and storage medium, which can improve image quality for monochrome scenes.

[0004] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0005] Acquire an image to be processed, where the image to be processed is a monochrome scene image;

[0006] Determining a first 3D histogram corresponding to the image to be processed;

[0007] Acquire a target reference image, and determine a second 3D histogram corresponding to the target reference image;

[0008] Determining a target correlation between the image to be processed and the target reference image according to the first 3D histogram and the second 3D histogram;

[0009] Determining a target color temperature value according to the target correlation;

[0010] Determining a target gain value of the image to be processed according to the target color temperature value;

[0011] The image to be processed is adjusted according to the target gain value to obtain an output image.

[0012] In a second aspect, an embodiment of the present application provides an image processing device, the device comprising: an acquisition unit, a determination unit and an adjustment unit, wherein:

[0013] The acquisition unit is used to acquire the image to be processed, where the image to be processed is a monochrome scene image;

[0014] The determining unit is used to determine a first 3D histogram corresponding to the image to be processed;

[0015] The acquisition unit is further used to acquire a target reference image;

[0016] The determination unit is further configured to determine a second 3D histogram corresponding to the target reference image; determine a target correlation between the image to be processed and the target reference image according to the first 3D histogram and the second 3D histogram; determine a target color temperature value according to the target correlation; and determine a target gain value of the image to be processed according to the target color temperature value;

[0017] The adjustment unit is used to adjust the image to be processed according to the target gain value to obtain an output image.

[0018] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for executing the steps of any method in the first aspect of the embodiment of the present application.

[0019] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps described in any method of the first aspect of the embodiment of the present application.

[0020] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in any method of the first aspect of the embodiment of the present application. The computer program product may be a software installation package.

[0021] The embodiments of the present application have the following beneficial effects:

[0022] It can be seen that the image processing method, device and storage medium described in the embodiments of the present application are applied to an electronic device to obtain an image to be processed, where the image to be processed is a monochrome scene image, determine a first 3D histogram corresponding to the image to be processed, obtain a target reference image, and determine a second 3D histogram corresponding to the target reference image, determine a target correlation between the image to be processed and the target reference image based on the first 3D histogram and the second 3D histogram, determine a target color temperature value based on the target correlation, determine a target gain value of the image to be processed based on the target color temperature value, adjust the image to be processed based on the target gain value, and obtain an output image. In this way, a color temperature suitable for a monochrome scene can be determined, and the gain value can be adjusted based on the color temperature, which helps to improve image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1A It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0025] Figure 1B It is a flowchart of an image processing method provided in an embodiment of the present application;

[0026] Figure 1C is a flowchart of another scene recognition method provided in an embodiment of the present application;

[0027] Figure 1D is a flowchart of another scene recognition method provided in an embodiment of the present application;

[0028] Figure 1E is a schematic diagram of a process for adjusting image gain provided in an embodiment of the present application;

[0029] Figure 1F is a flowchart of another image processing method provided in an embodiment of the present application;

[0030] Figure 2 is a flowchart of another image processing method provided in an embodiment of the present application;

[0031] Figure 3 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0032] Figure 4 It is a block diagram of the functional units of an image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0034] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0035] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0036] The electronic device involved in the embodiment of the present application may be an electronic device with communication capability or an electronic device without communication capability. The electronic device may include various handheld devices with wireless communication function, vehicle-mounted devices, wearable devices (smart glasses, smart bracelets, smart watches, etc.), computing devices or other processing devices connected to wireless modems, and various forms of user equipment (UE), mobile stations (MS), terminal devices, etc. In the embodiment of the present application, a monochrome scene can be understood as a scene where a color appears in a large area, or it can be described as an area ratio of a color in an image exceeding a preset area threshold. The preset area threshold can be set by the user or by the system default, for example, the preset area threshold can be 80%, 90%, etc. A monochrome scene can be a single color scene, for example, pure green, pure blue, etc. Of course, a monochrome scene can also be defined as a scene where the number of white spots in the image is less than a preset number threshold, which can be set by the user or by the system default, for example, the preset number threshold can be 0, 1, 2, etc. A white spot scene can be defined as a scene where the number of white spots in the image is greater than or equal to the preset number threshold. In a specific implementation, monochrome scenes and white spot scenes can be recognized according to the white spot detection related algorithm mentioned in the embodiment of the present application. The embodiment of the present application is described in detail below.

[0037] See also Figure 1A , Figure 1A1 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device includes a processor, a memory, a signal processor, a transceiver, a display screen, a speaker, a microphone, a random access memory (RAM), a camera, a sensor, and a communication module, etc. Among them, the memory, the signal processor, the display screen, the speaker, the microphone, the RAM, the camera, the sensor, and the communication module are connected to the processor, and the transceiver is connected to the signal processor.

[0038] The display screen may be a liquid crystal display (LCD), an organic or inorganic light-emitting diode (OLED), an active matrix organic light-emitting diode panel (AMOLED), etc.

[0039] The camera may be a common camera or an infrared camera, which is not limited here. The camera may be a front camera or a rear camera, which is not limited here.

[0040] Among them, the sensor includes at least one of the following: a light sensor, a gyroscope, an infrared proximity sensor, a fingerprint sensor, a pressure sensor, and the like. Among them, the light sensor, also known as an ambient light sensor, is used to detect the brightness of ambient light. The light sensor may include a photosensitive element and an analog-to-digital converter. Among them, the photosensitive element is used to convert the collected light signal into an electrical signal, and the analog-to-digital converter is used to convert the above electrical signal into a digital signal. Optionally, the light sensor may also include a signal amplifier, which may amplify the electrical signal converted by the photosensitive element and output it to the analog-to-digital converter. The above-mentioned photosensitive element may include at least one of a photodiode, a phototransistor, a photoresistor, and a silicon photocell.

[0041] Among them, the processor is the control center of the electronic device, which uses various interfaces and lines to connect the various parts of the entire electronic device, and performs various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory, and calling data stored in the memory, thereby monitoring the electronic device as a whole.

[0042] The processor may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor, wherein the processor may be at least one of the following: ISP, CPU, GPU, NPU, etc., which are not limited here.

[0043] The memory is used to store software programs and / or modules, and the processor executes various functional applications and data processing of the electronic device by running the software programs and / or modules stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a software program required for at least one function, etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0044] Among them, the communication module can be used to realize the communication function, and the communication module can be at least one of the following: infrared module, Bluetooth module, mobile communication module, NFC module, Wi-Fi module, etc., which is not limited here.

[0045] See also Figure 1B , Figure 1B The present application provides a flowchart of an image processing method, which is applied to Figure 1A The electronic device shown in the figure, as shown in the figure, the image processing method includes the following operations.

[0046] 101. Acquire an image to be processed, where the image to be processed is a monochrome scene image.

[0047] The image to be processed may be an RGB image or a Bayer image. The image to be processed may be a partial image or the entire image of an original image.

[0048] In a possible example, before the above step 101, the following steps may also be included:

[0049] A1. Acquire a color temperature curve of the image to be processed, and determine the number of target white points of the image to be processed based on the color temperature curve.

[0050] A2. Perform scene recognition based on the number of target white spots to obtain a scene recognition result.

[0051] The electronic device can obtain the color temperature curve of the image to be processed, and determine the number of target white points of the image to be processed based on the color temperature curve. The scene recognition result can be a white point scene or a monochrome scene. The electronic device can perform scene recognition based on the number of target white points to obtain a scene recognition result. Different scene recognition results can be processed using different image processing algorithms to better improve image quality.

[0052] In a possible example, the above step A1, obtaining the color temperature curve of the image to be processed, and determining the number of target white points of the image to be processed based on the color temperature curve, may include the following steps:

[0053] A11. Obtain a first environmental parameter;

[0054] A12. Determine the color temperature curve corresponding to the first environmental parameter;

[0055] A13, converting the image to be processed into a YCbCr space to obtain a Cb component and a Cr component;

[0056] A14. Determine the target white point quantity of the image to be processed according to the Cb component and the Cr component of the color temperature curve.

[0057] In the embodiment of the present application, the environmental parameter may be at least one of the following: ambient brightness, geographical location, weather, temperature, humidity, magnetic field interference coefficient, etc., which are not limited here. The electronic device may obtain the environmental parameter through an environmental detection sensor, which may be at least one of the following: an ambient light sensor, a positioning sensor, a meteorological sensor, a temperature sensor, a humidity sensor, a magnetic field detection sensor, etc., which are not limited here.

[0058] In a specific implementation, the electronic device may pre-store a mapping relationship between environmental parameters and color temperature curves. For example, by adopting a "white point detection method based on color temperature curve", a test card image is first collected in a color temperature light box according to the characteristics of the sensor to fit a color temperature curve function. Specifically, the electronic device may obtain a first environmental parameter, and determine a color temperature curve corresponding to the first environmental parameter according to the mapping relationship, and may convert the image to be processed into a YCbCr space to obtain a Cb component and a Cr component. Furthermore, the number of target white points of the image to be processed may be determined according to the color temperature curve, the Cb component and the Cr component. Pixels near the color temperature curve are white points. The white points and their RGB values ​​are counted. Then, white point detection is performed on the counted pixels to count the number of white points Cnt1.

[0059] In a possible example, the above step A2, performing scene recognition based on the number of target white points to obtain a scene recognition result, may include the following steps:

[0060] A21, when the number of target white points is less than or equal to a first preset threshold, confirming that the scene recognition result is a monochrome scene;

[0061] A22. When the number of target white dots is greater than the first preset threshold, obtaining a proportion of white dots of the image to be processed in a three-dimensional histogram domain;

[0062] A23. Perform scene recognition according to the white point proportion to obtain the scene recognition result.

[0063] The first preset threshold value can be set by the user or by the system default. When the number of target white dots is less than or equal to the first preset threshold value, the electronic device can confirm that the scene recognition result is a monochrome scene. When the number of target white dots is greater than the first preset threshold value, the white dot ratio of the image to be processed in the three-dimensional histogram domain is obtained, and the scene recognition is performed based on the white dot ratio to obtain the scene recognition result. In this way, the scene can be preliminarily recognized based on the number of target white dots, and the white dot scene or monochrome scene can be further recognized through the white dot ratio in the three-dimensional histogram.

[0064] In a possible example, the above step A23, performing scene recognition according to the white point proportion to obtain the scene recognition result, may include the following steps:

[0065] A231, when the white spot ratio is greater than a second preset threshold, confirming that the scene recognition result is a white spot scene;

[0066] A232, when the white point proportion is less than or equal to a second preset threshold, determining the number of bins of the image to be processed in a three-dimensional histogram domain;

[0067] A233. Perform scene recognition according to the bin number to obtain the scene recognition result.

[0068] Among them, the second preset threshold can be set by the user or by the system default. When the white point proportion is greater than the second preset threshold, the electronic device can confirm that the scene recognition result is a white point scene. When the white point proportion is less than or equal to the second preset threshold, the electronic device can determine the number of bins of the image to be processed in the three-dimensional histogram domain, and perform scene recognition based on the number of bins to obtain a scene recognition result. In this way, the scene can be further accurately identified.

[0069] In a possible example, the above step A233, performing scene recognition according to the bin number to obtain the scene recognition result, may include the following steps:

[0070] A2331. When the number of bins is less than or equal to a third preset threshold, confirming that the scene recognition result is a monochrome scene;

[0071] A2332: When the number of bins is greater than the third preset threshold, confirm that the scene recognition result is a white spot scene.

[0072] The third preset threshold value can be set by the user or by the system default. In a specific implementation, the electronic device can confirm that the scene recognition result is a monochrome scene when the number of bins is less than or equal to the third preset threshold value, and vice versa, when the number of bins is greater than the third preset threshold value, confirm that the scene recognition result is a white spot scene. In this way, the scene recognition result can be accurately obtained, which is conducive to selecting an image processing algorithm suitable for the scene to process the image to be processed, and helps to improve the image quality.

[0073] For example, in a specific implementation, an electronic device can use three characteristic parameters to determine whether there are white spots in a scene. The parameters are: the number of white spots Cnt1 obtained by "based on the color temperature curve algorithm and the white spot detection algorithm", the white spot ratio Cnt2 of the image obtained by "based on 3D_hist" statistics, and the number of bins Cnt3 of the image. The process of scene type recognition is as follows: Figure 1C shown.

[0074] exist Figure 1C In the example, T1 is the first preset threshold, T2 is the second preset threshold, and T3 is the third preset threshold. The scene type determination method is as follows:

[0075] First, count the number of white dots Cnt1. When the number of white dots Cnt1 ≤ T1, it is a monochrome scene. When Cnt1> Thr1, it may be a scene with white dots or a monochrome scene.

[0076] Secondly, the white spot ratio Cnt2 is counted. When Cnt2>T2, it is a scene with white spots. When Cnt2≤T2, it may be a scene with white spots, but a small number of images (such as low-light images at the camera mount) may be monochrome scenes. Further judgment needs to be made in combination with the bin number Cnt3.

[0077] Finally, the number of bins of the image is counted Cnt3. When Cnt3>T3, it is a scene with white spots, and when Cnt3≤Thr3, it is a monochrome scene.

[0078] In a specific implementation, the electronic device may pre-process the image to be processed, convert the RGB image into the 3D_hist space domain, and calculate the proportion Cnt2 of the white points in the image and the number Cnt3 of the image bins, such as Figure 1C shown.

[0079] Further, in Figure 1D In the process, it mainly includes preset white point Bin coordinate range, color space conversion (RGB2YCbCr), 3D_hist generation, 3D_hist preprocessing, statistical white point Bin value Sum_bin_w, statistical image Bin value Sum_bin, calculation of white point proportion, and detection of image Bin number. Each part is described as follows:

[0080] Color space conversion (RGB2YCbCr), specifically converting the RGB image to the YCbCr space to obtain the Cb and Cr components;

[0081] Generate 3D_hist, where an xy coordinate plane is established with Cr as the x-axis and Cb as the y-axis, and its range is preset, and the cumulative distribution value of the image pixel points in the CbCr color domain is counted, that is, 3D_hist is generated;

[0082] 3D_hist preprocessing can be used to process the generated 3D_hist to remove the interference of noise and unsaturated pixels.

[0083] The preset white point Bin coordinates bin_w(i,j) can be obtained through neural network algorithm training. Specifically, the 3D_hist data of the image to be processed can be input into the trained neural network model, that is, it can be obtained that the test card image is collected at different color temperatures, 3D_hist is generated, and the Bin(i,j) coordinates of the gray point are recorded. The set of these Bin(i,j) coordinates is called the white point Bin coordinates bin_w(i,j).

[0084] Count the white point Bin value Sum_bin_w. When a pixel in the image falls into bin_w(i,j), count the cumulative distribution value Sum_bin_w of each bin. Count the image Bin value Sum_bin. Count the cumulative distribution value Sum_bin of all pixels in the image that fall into bin(i,j).

[0085] Furthermore, the white point ratio is calculated, wherein the white point ratio is the percentage Cnt2 of the white point Bin value to the image Bin value, and is calculated as follows:

[0086]

[0087] Count the number of image bins Cnt3. The number of bins can reflect the richness of the scene colors. Count the number of bins in 3D_hist as follows:

[0088] Cnt3=Sum(ind(Bin(i,j)>Thr3_0))

[0089] Among them, “find(Bin(i,j)>Thr)” means counting the number of Bins in the image whose Bin values ​​are greater than the threshold Thr, and removing a small amount of false colors or interfering colors.

[0090] 102. Determine a first 3D histogram corresponding to the image to be processed.

[0091] The electronic device may perform color space conversion on the image to be processed, that is, convert it into a YCbCr color space to obtain Cb and Cr color components, and construct a 3D histogram with the Cb and Cr color components to obtain a first 3D histogram.

[0092] 103. Acquire a target reference image, and determine a second 3D histogram corresponding to the target reference image.

[0093] The target reference image can be preset, or the system defaults, or can be selected from a reference image set according to the characteristics of the image to be processed, and the reference image set can include multiple reference images. Then, the electronic device can perform color space conversion on the target reference image, that is, convert it into a YCbCr color space to obtain Cb and Cr color components, and the Cb and Cr color components construct a 3D histogram to obtain a second 3D histogram.

[0094] In a possible example, the above step 103, obtaining the target reference image, may include the following steps:

[0095] B31. Obtain target environment parameters;

[0096] B32. Determine the target reference image corresponding to the target environmental parameter according to the mapping relationship between the preset environmental parameter and the reference image.

[0097] In the embodiment of the present application, the environmental parameter may be at least one of the following: ambient brightness, geographical location, weather, temperature, humidity, magnetic field interference coefficient, etc., which are not limited here. The electronic device may obtain the environmental parameter through an environmental detection sensor, which may be at least one of the following: an ambient light sensor, a positioning sensor, a meteorological sensor, a temperature sensor, a humidity sensor, a magnetic field detection sensor, etc., which are not limited here.

[0098] In a specific implementation, the electronic device may pre-store a mapping relationship between preset environmental parameters and reference images, and then the electronic device may obtain target environmental parameters and determine a target reference image corresponding to the target environmental parameters according to the mapping relationship between the preset environmental parameters and reference images.

[0099] In a possible example, the above step 31, obtaining the target reference image, may include the following steps:

[0100] C311, dividing the target reference image into multiple regions;

[0101] C312, identifying the number of white spot areas and the number of non-white spot areas in the plurality of areas;

[0102] C313, determining a proportion of white spot areas according to the number of white spot areas and the number of non-white spot areas;

[0103] C314, determining a target interval corresponding to the white spot area ratio;

[0104] C315. Determine the target reference image corresponding to the target interval according to a mapping relationship between a preset interval and a reference image.

[0105] Among them, the electronic device can divide the target reference image into multiple areas, the area size of each area is equal, and can also identify the number of white dot areas and the number of non-white dot areas in the multiple areas. Some areas in the multiple areas are white dot areas, and some areas are non-white dot areas. The proportion of white dot areas is determined according to the number of white dot areas and the number of non-white dot areas. The proportion of white dot areas = the number of white dot areas / (total number of areas), the total number of areas = the number of white dot areas and the number of non-white dot areas, and the proportion of white dot areas is between 0 and 1. Then, the target interval corresponding to the proportion of white dot areas can be determined. The electronic device can pre-store the mapping relationship between the preset interval and the reference image, and then, according to the mapping relationship between the preset interval and the reference image, the target reference image corresponding to the target interval can be determined.

[0106] 104. Determine a target correlation between the image to be processed and the target reference image according to the first 3D histogram and the second 3D histogram.

[0107] The electronic device may use a correlation calculation algorithm and determine the target correlation between the image to be processed and the target reference image according to the first 3D histogram and the second 3D histogram. The correlation calculation algorithm includes at least one of the following: histogram intersection method, Bhattacharyya distance, Euclidean distance, Hamming distance, histogram weighted distance method, etc., which are not limited here.

[0108] 105. Determine a target color temperature value according to the target correlation.

[0109] In the embodiment of the present application, a mapping relationship between the correlation and the color temperature value may be pre-stored in the electronic device, and then, the target color temperature value corresponding to the correlation may be determined based on the mapping relationship.

[0110] For example, the electronic device can arrange the obtained correlation coefficients Corr in descending order and calculate the color temperature value by weighted statistics. If the n color temperatures of an input image and a reference image are T1, T2, T3, ..., Tn (7500K), the correlation coefficients are arranged in descending order as Corr(1), Corr(2), Corr(3), ..., Corr(n); the three highest correlation coefficients are taken to calculate the color temperature value of the image, and the weighted coefficient is calculated as:

[0111]

[0112]

[0113]

[0114] Then the target color temperature value of the image to be processed is:

[0115]

[0116] Where T is the target color temperature.

[0117] 106. Determine a target gain value of the image to be processed according to the target color temperature value.

[0118] Among them, in the embodiment of the present application, different color temperature values ​​can correspond to different gain values.

[0119] In a possible example, the above step 106, determining the target gain value of the image to be processed according to the target color temperature value, can be implemented as follows:

[0120] According to a preset mapping relationship between color temperature values ​​and gain values, the target gain value corresponding to the target color temperature value is determined.

[0121] The electronic device may pre-store a mapping relationship between a preset color temperature value and a gain value, and then, the target gain value corresponding to the target color temperature value may be determined according to the mapping relationship.

[0122] 107. Adjust the image to be processed according to the target gain value to obtain an output image.

[0123] Among them, the electronic device can adjust the white balance effect of the image to be processed according to the target enhancement value to obtain an output image, so that the white balance processing effect can be improved.

[0124] In the specific implementation, Figure 1E In the process, the electronic device can first input an image (bayer), and adjust the bayer gain according to the gain value of the white balance statistics module. White balance correction mainly includes bayer image merging, gain value of white balance statistics, and adjustment of bayer gain.

[0125] When adjusting the gain value in the Bayer domain, the gain multiple of the G component is the same, that is, if the white balance statistics module obtains the gain values ​​Gain_r, Gain_g, and Gain_b of RGB, the Bayer adjustment gain values ​​are Gain_r, Gain_gr, Gain_gb, and Gain_b. Among them, Gain_g=Gain_gb=Gain_gr=1.

[0126] If the input image is Bayer, the calculation of white balance correction is as follows:

[0127] R'=R*Gain_r

[0128] Gr'=Gr*Gain_gr

[0129] Gb'=Gb*Gain_gb

[0130] B'=B*Gain_b

[0131] Among them, R, Gr, Gb, and B are the bayer components before white balance, and R', Gr', Gb', and B' are the bayer components after white balance adjustment.

[0132] If the input image is RGB, the calculation is as follows:

[0133] R'=R*Gain_r

[0134] G'=G*Gain_g

[0135] B'=B*Gain_b

[0136] Among them, R, G, B are RGB components before white balance, and R', G', B' are RGB components after white balance adjustment.

[0137] For example, Figure 1F As shown, the electronic device can collect a reference image and preprocess the reference image. The preprocessing can be at least one of the following: noise reduction, elimination of oversaturated pixels (pixel values ​​greater than a certain threshold), elimination of unsaturated pixels (pixel values ​​less than a certain threshold), for example, the RGB pixels and brightness Y of the image are preprocessed, and the maximum threshold RGB_max (eliminating oversaturated pixels) and the minimum threshold RGB_min (eliminating unsaturated pixels and noise) of the image RGB pixels are set. Further, the preprocessed reference image RGB image is converted into YCbCr space, and 3D_hist is generated based on Cb and Cr color components. Based on the same method, 3D_hist is generated from the input rgb image, and the correlation between the input image and the reference image is calculated based on the two 3D_hist, the color temperature value is determined based on the correlation, and the image gain value is determined based on the color temperature value, and the gain is adjusted to obtain the output image.

[0138] Furthermore, when the image to be processed is a white spot scene, the following steps may also be included:

[0139] D1, dividing the image to be processed into multiple regions;

[0140] D2. determining the white spot distribution density of each of the plurality of regions to obtain a plurality of target white spot distribution densities;

[0141] D3, obtaining algorithm control parameters of the target image processing algorithm of the image to be processed;

[0142] D4. According to a preset mapping relationship between white point distribution density and adjustment factor, determining the adjustment factor corresponding to each target white point distribution density among the multiple target white point distribution densities to obtain multiple adjustment factors;

[0143] D5. Determine the target algorithm control parameters of each region according to the multiple adjustment factors and the algorithm control parameters to obtain multiple target algorithm control parameters;

[0144] D6. Process the image to be processed according to the multiple target algorithm control parameters and the target image processing algorithm to obtain a result image.

[0145] In the embodiment of the present application, the target image processing algorithm may be at least one of the following: image enhancement algorithm, white balance algorithm, smoothing algorithm, wavelet algorithm, neural network algorithm, etc., which are not limited here.

[0146] In a specific implementation, the electronic device can divide the image to be processed into multiple areas, and the area size of each area can be the same or different. In addition, the white point distribution density of each area in the multiple areas can be determined to obtain multiple target white point distribution densities, where the target white point distribution density = the total number of white points in each area / area size. The algorithm control parameters of the target image processing algorithm can also be obtained. The algorithm control parameters are used to control the processing degree of the algorithm. For example, for a deblurring algorithm, the algorithm control parameters can control the degree of deblurring.

[0147] Further, the electronic device may pre-store a mapping relationship between a preset white point distribution density and an adjustment factor. Then, the adjustment factor corresponding to each target white point distribution density in a plurality of target white point distribution densities may be determined according to the mapping relationship between the preset white point distribution density and the adjustment factor, and a plurality of adjustment factors may be obtained. The value range of the adjustment factor may be between -1 and 1. For example, the value of the adjustment factor may be between -0.0134 and 0.0134. The algorithm control parameter may be adjusted by the adjustment factor. Then, the target algorithm control parameter of each region may be determined according to the plurality of adjustment factors and the algorithm control parameter, and a plurality of target algorithm control parameters may be obtained. The specific formula may be as follows:

[0148] Target algorithm control parameter = (1 + adjustment factor) * algorithm control parameter

[0149] Finally, the electronic device can process the image to be processed according to multiple target algorithm control parameters and the target image processing algorithm to obtain a result image. In this way, the image to be processed can be processed according to the differences in different regions, which helps to improve the image quality in a targeted manner.

[0150] It can be seen that the image processing method described in the embodiment of the present application is applied to an electronic device, and an image to be processed is obtained, where the image to be processed is a monochrome scene image, a first 3D histogram corresponding to the image to be processed is determined, a target reference image is obtained, and a second 3D histogram corresponding to the target reference image is determined, and a target correlation between the image to be processed and the target reference image is determined based on the first 3D histogram and the second 3D histogram, a target color temperature value is determined based on the target correlation, a target gain value of the image to be processed is determined based on the target color temperature value, and the image to be processed is adjusted based on the target gain value to obtain an output image. In this way, a color temperature suitable for a monochrome scene can be determined, and the gain value can be adjusted based on the color temperature, which helps to improve image quality.

[0151] With the above Figure 1B For details on the embodiments shown in the drawings, please refer to Figure 2 , Figure 2 It is a flowchart of an image processing method provided in an embodiment of the present application, which is applied to an electronic device. As shown in the figure, the image processing method includes the following steps.

[0152] 201. Acquire an image to be processed, acquire a color temperature curve of the image to be processed, and determine the number of target white points of the image to be processed based on the color temperature curve.

[0153] 202. Perform scene recognition based on the target white point quantity to obtain a scene recognition result.

[0154] 203. When the image to be processed is a monochrome scene image, determine a first 3D histogram corresponding to the image to be processed.

[0155] 204. Acquire a target reference image, and determine a second 3D histogram corresponding to the target reference image.

[0156] 205. Determine a target correlation between the image to be processed and the target reference image according to the first 3D histogram and the second 3D histogram.

[0157] 206. Determine a target color temperature value according to the target correlation.

[0158] 207. Determine a target gain value of the image to be processed according to the target color temperature value.

[0159] 208. Adjust the image to be processed according to the target gain value to obtain an output image.

[0160] The detailed description of the above steps 201 to 208 can be found in the above Figure 1B The corresponding steps of the described image processing method will not be repeated here.

[0161] It can be seen that the image processing method described in the embodiment of the present application is applied to an electronic device, obtains an image to be processed, obtains a color temperature curve of the image to be processed, and determines the number of target white points of the image to be processed based on the color temperature curve, performs scene recognition based on the target number of white points, and obtains a scene recognition result. When the image to be processed is a monochrome scene image, the image to be processed is obtained, the image to be processed is a monochrome scene image, a first 3D histogram corresponding to the image to be processed is determined, a target reference image is obtained, and a second 3D histogram corresponding to the target reference image is determined, a target correlation between the image to be processed and the target reference image is determined based on the first 3D histogram and the second 3D histogram, a target color temperature value is determined based on the target correlation, a target gain value of the image to be processed is determined based on the target color temperature value, and the image to be processed is adjusted based on the target gain value to obtain an output image. In this way, a color temperature suitable for a monochrome scene can be determined for a monochrome scene, and the gain value can be adjusted based on the color temperature, which helps to improve image quality.

[0162] With the above Figure 1B , Figure 2 For details on the embodiments shown in the drawings, please refer to Figure 3 , Figure 3 It is a structural diagram of an electronic device 300 provided in an embodiment of the present application. As shown in the figure, the electronic device 300 includes a processor 310, a memory 320, a communication interface 330 and one or more programs 321, wherein the one or more programs 321 are stored in the above-mentioned memory 320 and are configured to be executed by the above-mentioned processor 310, and the one or more programs 321 include instructions for executing any step in the above-mentioned method embodiment.

[0163] Acquire an image to be processed, where the image to be processed is a monochrome scene image;

[0164] Determining a first 3D histogram corresponding to the image to be processed;

[0165] Acquire a target reference image, and determine a second 3D histogram corresponding to the target reference image;

[0166] Determining a target correlation between the image to be processed and the target reference image according to the first 3D histogram and the second 3D histogram;

[0167] Determining a target color temperature value according to the target correlation;

[0168] Determining a target gain value of the image to be processed according to the target color temperature value;

[0169] The image to be processed is adjusted according to the target gain value to obtain an output image.

[0170] It can be seen that the electronic device described in the embodiment of the present application obtains an image to be processed, which is a monochrome scene image, determines a first 3D histogram corresponding to the image to be processed, obtains a target reference image, and determines a second 3D histogram corresponding to the target reference image, determines a target correlation between the image to be processed and the target reference image based on the first 3D histogram and the second 3D histogram, determines a target color temperature value based on the target correlation, determines a target gain value of the image to be processed based on the target color temperature value, adjusts the image to be processed based on the target gain value, and obtains an output image. In this way, it is possible to determine a color temperature suitable for a monochrome scene, and adjust the gain value based on the color temperature, which helps to improve image quality.

[0171] In a possible example, in determining the target gain value of the image to be processed according to the target color temperature value, the instructions in the program are further specifically used to perform the following operations:

[0172] According to a preset mapping relationship between color temperature values ​​and gain values, the target gain value corresponding to the target color temperature value is determined.

[0173] In a possible example, in terms of acquiring the target reference image, the instructions in the program are further specifically used to perform the following operations:

[0174] Get target environment parameters;

[0175] According to the mapping relationship between the preset environmental parameters and the reference image, the target reference image corresponding to the target environmental parameters is determined.

[0176] In a possible example, in terms of acquiring the target reference image, the instructions in the program are further specifically used to perform the following operations:

[0177] Dividing the target reference image into a plurality of regions;

[0178] identifying the number of white spot areas and the number of non-white spot areas in the plurality of areas;

[0179] Determine the proportion of white spot areas according to the number of white spot areas and the number of non-white spot areas;

[0180] Determine a target interval corresponding to the white spot area ratio;

[0181] According to a mapping relationship between a preset interval and a reference image, the target reference image corresponding to the target interval is determined.

[0182] In a possible example, when the correlation includes multiple correlations, each correlation corresponds to a color temperature value, and in determining the target color temperature value according to the correlation, the instructions in the program are further specifically used to perform the following operations:

[0183] Determine a weighting coefficient corresponding to each of the multiple correlations to obtain multiple weighting coefficients;

[0184] A weighted operation is performed according to the plurality of weighting coefficients and the color temperature value corresponding to each of the plurality of correlations to obtain the target color temperature value.

[0185] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the execution process on the method side. It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0186] The embodiment of the present application can divide the electronic device into functional units according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0187] Figure 4 4 is a functional unit block diagram of an image processing device 400 involved in an embodiment of the present application. The image processing device 400 is applied to an electronic device, and the image processing device 400 includes an acquisition unit 401, a determination unit 402, and an adjustment unit 403. The following is a detailed description, specifically as follows:

[0188] The acquisition unit 401 is used to acquire an image to be processed, where the image to be processed is a monochrome scene image;

[0189] The determining unit 402 is used to determine a first 3D histogram corresponding to the image to be processed;

[0190] The acquisition unit 401 is further used to acquire a target reference image;

[0191] The determining unit 402 is further configured to determine a second 3D histogram corresponding to the target reference image; determine a target correlation between the image to be processed and the target reference image according to the first 3D histogram and the second 3D histogram; determine a target color temperature value according to the target correlation; and determine a target gain value of the image to be processed according to the target color temperature value;

[0192] The adjustment unit 403 is used to adjust the image to be processed according to the target gain value to obtain an output image.

[0193] It can be seen that the image processing device described in the embodiment of the present application is applied to an electronic device, obtains an image to be processed, the image to be processed is a monochrome scene image, determines a first 3D histogram corresponding to the image to be processed, obtains a target reference image, and determines a second 3D histogram corresponding to the target reference image, determines a target correlation between the image to be processed and the target reference image based on the first 3D histogram and the second 3D histogram, determines a target color temperature value based on the target correlation, determines a target gain value of the image to be processed based on the target color temperature value, adjusts the image to be processed based on the target gain value, and obtains an output image. In this way, it is possible to determine a color temperature suitable for a monochrome scene, and adjust the gain value based on the color temperature, which helps to improve image quality.

[0194] In a possible example, in determining the target gain value of the to-be-processed image according to the target color temperature value, the determining unit 402 is specifically configured to:

[0195] According to a preset mapping relationship between color temperature values ​​and gain values, the target gain value corresponding to the target color temperature value is determined.

[0196] In a possible example, in terms of acquiring the target reference image, the acquiring unit 401 is specifically configured to:

[0197] Get target environment parameters;

[0198] According to the mapping relationship between the preset environmental parameters and the reference image, the target reference image corresponding to the target environmental parameters is determined.

[0199] The image processing device 400 may further include a storage unit 404 for storing program codes and data of the electronic device. The acquisition unit 401, the determination unit 402 and the adjustment unit 403 may be processors, and the storage unit 404 may be a memory.

[0200] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps of any method described in the above method embodiments, and the above computer includes an electronic device.

[0201] The embodiment of the present application also provides a computer program product, the computer program product includes a non-transitory computer-readable storage medium storing a computer program, the computer program is operable to cause a computer to execute some or all of the steps of any method described in the method embodiment. The computer program product may be a software installation package, and the computer includes an electronic device.

[0202] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0203] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0204] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above-mentioned units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0205] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0206] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0207] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the above-mentioned methods of each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or CD-ROM and other media that can store program codes.

[0208] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a magnetic disk or an optical disk, etc.

[0209] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire an image to be processed, acquire a color temperature curve of the image to be processed, and determine the number of target white points of the image to be processed based on the color temperature curve; Performing scene recognition based on the number of target white spots to obtain a scene recognition result; When the image to be processed is a monochrome scene image, a first 3D histogram corresponding to the image to be processed is determined; the first 3D histogram is to convert the image to be processed into a YCbCr space to obtain Cb and Cr components, and generate 3D_hist based on the Cb and Cr components, specifically: establish an xy coordinate plane with Cr as the x-axis and Cb as the y-axis, preset its range to count the cumulative distribution values ​​of image pixels in the CbCr color domain, that is, generate 3D_hist; Acquire a target reference image, and determine a second 3D histogram corresponding to the target reference image; Determining a target correlation between the image to be processed and the target reference image according to the first 3D histogram and the second 3D histogram; Determining a target color temperature value according to the target correlation; Determining a target gain value of the image to be processed according to the target color temperature value; Adjusting the image to be processed according to the target gain value to obtain an output image; Wherein, when the image to be processed is a white spot scene, the following steps are also included: Dividing the image to be processed into a plurality of regions; Determine the white point distribution density of each of the multiple areas to obtain multiple target white point distribution densities; Acquiring algorithm control parameters of a target image processing algorithm for the image to be processed; According to a preset mapping relationship between white point distribution density and adjustment factor, determining the adjustment factor corresponding to each target white point distribution density among the multiple target white point distribution densities to obtain multiple adjustment factors; Determining a target algorithm control parameter for each region according to the multiple adjustment factors and the algorithm control parameter to obtain multiple target algorithm control parameters; The image to be processed is processed according to the multiple target algorithm control parameters and the target image processing algorithm to obtain a result image.

2. The method according to claim 1, characterized in that The step of determining a target gain value of the image to be processed according to the target color temperature value includes: According to a preset mapping relationship between color temperature values ​​and gain values, the target gain value corresponding to the target color temperature value is determined.

3. The method according to claim 1 or 2, characterized in that: The step of acquiring a target reference image comprises: Get target environment parameters; According to the mapping relationship between the preset environmental parameters and the reference image, the target reference image corresponding to the target environmental parameters is determined.

4. The method according to claim 1 or 2, characterized in that: The step of acquiring a target reference image comprises: Dividing the target reference image into a plurality of regions; identifying the number of white spot areas and the number of non-white spot areas in the plurality of areas; Determine the proportion of white spot areas according to the number of white spot areas and the number of non-white spot areas; Determine a target interval corresponding to the white spot area ratio; According to a mapping relationship between a preset interval and a reference image, the target reference image corresponding to the target interval is determined.

5. The method according to claim 1 or 2, characterized in that: When the correlation includes multiple correlations, each correlation corresponds to a color temperature value, and determining the target color temperature value according to the correlation includes: Determine a weighting coefficient corresponding to each of the multiple correlations to obtain multiple weighting coefficients; A weighted operation is performed according to the plurality of weighting coefficients and the color temperature value corresponding to each of the plurality of correlations to obtain the target color temperature value.

6. An image processing device, characterized in that: The device comprises: an acquisition unit, a determination unit and an adjustment unit, wherein: The acquisition unit is used to acquire the image to be processed; The device is also specifically used to obtain a color temperature curve of the image to be processed, and determine the number of target white points of the image to be processed based on the color temperature curve; perform scene recognition based on the number of target white points to obtain a scene recognition result; The determining unit is used to determine a first 3D histogram corresponding to the image to be processed when the image to be processed is a monochrome scene image; the first 3D histogram is to convert the image to be processed into a YCbCr space to obtain Cb and Cr components, and generate 3D_hist based on the Cb and Cr components, specifically: establish an xy coordinate plane with Cr as the x-axis and Cb as the y-axis, preset its range to count the cumulative distribution values ​​of image pixels in the CbCr color domain, that is, generate 3D_hist; The acquisition unit is further used to acquire a target reference image; The determining unit is further configured to determine a second 3D histogram corresponding to the target reference image; and determine a target correlation between the image to be processed and the target reference image according to the first 3D histogram and the second 3D histogram; Determining a target color temperature value according to the target correlation; and determining a target gain value of the image to be processed according to the target color temperature value; The adjustment unit is used to adjust the image to be processed according to the target gain value to obtain an output image; Wherein, when the image to be processed is a white spot scene, the device is further specifically used for: Dividing the image to be processed into a plurality of regions; Determine the white point distribution density of each of the multiple areas to obtain multiple target white point distribution densities; Acquiring algorithm control parameters of a target image processing algorithm for the image to be processed; According to a preset mapping relationship between white point distribution density and adjustment factor, determining the adjustment factor corresponding to each target white point distribution density among the multiple target white point distribution densities to obtain multiple adjustment factors; Determining a target algorithm control parameter for each region according to the multiple adjustment factors and the algorithm control parameter to obtain multiple target algorithm control parameters; The image to be processed is processed according to the multiple target algorithm control parameters and the target image processing algorithm to obtain a result image.

7. The device according to claim 6, characterized in that In determining the target gain value of the image to be processed according to the target color temperature value, the determining unit is specifically used to: According to a preset mapping relationship between color temperature values ​​and gain values, the target gain value corresponding to the target color temperature value is determined.

8. The device according to claim 6 or 7, characterized in that In terms of acquiring the target reference image, the acquiring unit is specifically used to: Get target environment parameters; According to the mapping relationship between the preset environmental parameters and the reference image, the target reference image corresponding to the target environmental parameters is determined.

9. An electronic device, characterized in that: The method comprises a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for executing the steps in the method according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: A computer program for electronic data exchange is stored, wherein the computer program enables a computer to execute the method according to any one of claims 1 to 5.

Images