Image color correction method and device

Abstract

The embodiments of the present application provide an image color correction method and device, which relate to the field of electronic technology. The method and device can calculate the probability of the light source color of the current image based on the prior probability of the light source color provided by the image data of the image training set and the likelihood provided by the automatic white balance technology according to the Bayesian algorithm, and then estimate the light source color and correct the color of the current image, thereby improving the accuracy of image color correction. The specific scheme is as follows: the electronic device obtains a first original raw image, determines the probability distribution of the light source color of the initial raw image based on the first raw image and the prior probability distribution of the light source color, and performs color correction on the initial raw image based on the probability distribution of the light source color of the initial raw image. The embodiment of the present application is used in the process of image color correction.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of electronic technology, and in particular to a method and device for image color correction. Background Art

[0002] While the human eye perceives the same color similarly under different light sources, the color of images captured by electronic devices is often affected by the light source. For example, images captured by electronic devices under indoor tungsten lighting appear yellowish, while those captured in daylight shade appear bluish. Electronic devices can use automatic white balance (AWB) technology to correct the color of captured images, avoiding color casts caused by light sources and ensuring that the colors of images captured by electronic devices match those seen by the human eye.

[0003] In traditional automatic white balance processing (such as image color correction based on the Gray World algorithm and Gray Edge algorithm), image color correction is typically performed based on certain constraints. For example, the constraint can be that a certain image statistic (such as the global average color or the global average edge color) is linearly related to the color of the light source. However, image color correction based on traditional automatic white balance technology has low accuracy and cannot meet user visual requirements. Summary of the Invention

[0004] An embodiment of the present application provides an image color correction method and device, which can calculate the probability of the light source color of the current image based on the prior probability of the light source color provided by the image training set and the likelihood provided by AWB according to the Bayesian algorithm, and then estimate the light source color and correct the color of the current image, thereby improving the accuracy of image color correction, making the color of the image captured by the electronic device consistent with the color seen by the human eye, thereby avoiding the influence of the light source color on the image color and improving the user experience.

[0005] To achieve the above objectives, the present invention adopts the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides an image color correction method, which is applied to an electronic device, the method comprising: obtaining a first original raw image; determining a probability distribution of a light source color of the initial raw image based on the first raw image and a prior probability distribution of a light source color, wherein the first raw image is the initial raw image or an image obtained by preprocessing the initial raw image; and performing color correction on the initial raw image based on the probability distribution of the light source color of the initial raw image.

[0007] In this scheme, the electronic device corrects the color of the initial original image based on the prior probability distribution of the light source color, thereby improving the accuracy of image color correction and making the color of the image after image color correction consistent with the color seen by the human eye, thereby avoiding the influence of the light source color on the image color and improving the user experience.

[0008] In one possible implementation, the method further includes determining a priori probability distribution of the light source color. Here, the electronic device can use various methods to determine the priori probability distribution of the light source color, such as a gray world method, a gray edge method, a machine learning method, etc.

[0009] In one possible implementation, determining the prior probability distribution of the light source color includes: obtaining a UV label for each training image based on the RGB label of each training image in a plurality of training images included in an image training set; obtaining a first matrix based on the UV label of each training image; wherein the first matrix is ​​a two-dimensional matrix, and the value of each item in the first matrix is ​​used to represent the number of UV labels whose corresponding matrix coordinates are the matrix coordinates of the item; and determining the prior probability distribution of the light source color based on the second matrix.

[0010] In this solution, the prior probability distribution of light source color is determined using multiple training images in the image training set. Image color is then corrected based on this prior probability distribution. This improves the accuracy of image color correction, ensuring that the color of the corrected image is consistent with what the human eye sees, thereby reducing the influence of light source color on image color and improving the user experience.

[0011] In one possible implementation, determining the probability distribution of the light source color of the initial raw image based on the first raw image and the prior probability distribution of the light source color includes: determining the probability distribution of the light source color of the initial raw image according to a two-dimensional chromaticity histogram of the first raw image, the probability distribution of the light source color of the first raw image, and the prior probability distribution of the light source color.

[0012] In one possible implementation, the method further includes: obtaining an edge image of the first raw image; wherein the edge image includes information about locations where pixel values ​​in the first raw image suddenly change; and determining the probability distribution of the light source color of the initial raw image based on the first raw image and the prior probability distribution of the light source color includes: determining the probability distribution of the light source color of the initial raw image based on the first raw image, the edge image, and the prior probability distribution of the light source color.

[0013] In this scheme, the edge image of the first original image is considered when determining the probability distribution of the light source color of the initial original image. Since the edge image can reflect more light source information in the pure color scene, the pure color scene can be better processed in combination with the edge image, and the probability distribution of the light source color of the initial original image is determined with higher accuracy.

[0014] In one possible implementation, determining the probability distribution of the light source color of the initial raw image based on the first raw image, the edge image, and the prior probability distribution of the light source color includes: determining the probability distribution of the light source color of the initial raw image based on a two-dimensional chromaticity histogram of the edge image, the probability distribution of the light source color of the edge image, the two-dimensional chromaticity histogram of the first raw image, the probability distribution of the light source color of the first raw image, and the prior probability distribution of the light source color.

[0015] In one possible implementation, the method further includes determining a probability distribution of the light source color of the first raw image. The probability distribution of the light source color of the first raw image can be determined using various methods, such as a gray-world method, a gray-edge method, or a machine learning method.

[0016] In one possible implementation, determining the probability distribution of the light source color in the first raw image includes: obtaining a two-dimensional chromaticity histogram of each training image among a plurality of training images included in an image training set; obtaining a second matrix based on the UV label of each training image and the two-dimensional chromaticity histogram of each training image; wherein the second matrix is ​​a four-dimensional matrix, and each item of the second matrix is ​​used to represent the two-dimensional chromaticity histogram of the training image corresponding to the UV label, and the matrix coordinates of each item in the second matrix are the same as the matrix coordinates of the corresponding UV label; obtaining a fourth matrix based on the second matrix and the third matrix; wherein the third matrix is ​​a four-dimensional matrix corresponding to the two-dimensional chromaticity histogram of the first raw image; and determining the probability distribution of the light source color in the first raw image according to the fourth matrix.

[0017] In this solution, the probability distribution of the light source color in the first raw image is determined by using multiple training images in the image training set, which can improve the accuracy of determining the probability distribution of the light source color in the first raw image.

[0018] In a second aspect, an embodiment of the present application provides an electronic device comprising: a camera; a display screen; one or more processors and one or more memories; the one or more memories are coupled to the one or more processors, the one or more memories being used to store computer program code, the computer program code comprising computer instructions, and when the one or more processors execute the computer instructions, the electronic device performs the image color correction method as described in any one of claims 1-8.

[0019] In a third aspect, an embodiment of the present application provides a computer storage medium comprising computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the image color correction method as described in any one of claims 1 to 8.

[0020] In a fourth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the image color correction method according to any one of claims 1 to 8.

[0021] For the beneficial effects corresponding to the above-mentioned other aspects, please refer to the description of the beneficial effects of the method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0023] Figure 2 A flowchart of an image color correction method provided in an embodiment of the present application;

[0024] Figure 3 A set of interface diagrams provided for the embodiments of this application;

[0025] Figure 4 A set of images provided for the embodiments of this application;

[0026] Figure 5 A set of two-dimensional histograms provided in an embodiment of the present application;

[0027] Figure 6 A two-dimensional histogram provided in an embodiment of the present application;

[0028] Figure 7 Another set of two-dimensional histograms provided in an embodiment of the present application;

[0029] Figure 8 Another two-dimensional histogram provided in an embodiment of the present application;

[0030] Figure 9 A schematic diagram of an image color correction process provided in an embodiment of the present application;

[0031] Figure 10 A schematic structural diagram of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships can exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0033] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0034] The human eye and brain automatically correct the color of objects under different light sources, ensuring that the colors of the images we see match the actual colors in the real scene. For example, a person will perceive a white object as white, regardless of whether they are viewing it in daylight or in a room illuminated by incandescent light, regardless of the color of the light source. However, the color of images captured by electronic devices is often affected by the color of the light source.

[0035] Images captured by electronic devices can be digitally represented using a variety of color models. For example, the red, yellow, and blue (RGB) model is a commonly used color model. Red, green, and blue light can be added together in various ways to produce a color spectrum. Typically, the light received by an electronic device passes through a filter layer and a color filter array (CFA) to obtain raw data. This raw data is then processed through a demosaic algorithm to obtain RGB data, resulting in an RGB image.

[0036] To remove the influence of light source color, electronic devices can use AWB to correct image color. AWB is an image color correction method that adjusts the color of captured images to remove the influence of light sources. Its purpose is to ensure that the white in images captured by electronic devices accurately reflects the actual white in real scenes. Specifically, it ensures that the white in images captured by electronic devices is consistent with the white seen by the human eye, avoiding color casts caused by light source color.

[0037] Traditional AWB techniques estimate the light source color of the current image based on the linear relationship between the global average color or the global average edge color of the image and the light source color, and then perform color correction on the current image based on the estimated light source color. This traditional AWB method for image color correction has low accuracy and is difficult to use alone due to limitations.

[0038] AWB learning technologies are based on machine learning techniques and rely on machine learning models such as support vector machines (SVMs), random forests, and neural networks. Using large training datasets, they learn the mapping equation between the features of images in those datasets and the color of the light source, thereby estimating the light source color and performing color correction on the current image. Compared to traditional AWB techniques, AWB learning technologies can improve the accuracy of image color correction. However, AWB learning technologies have a large number of parameters, high computational complexity, and typically require specialized hardware (such as GPUs and NPUs). This makes it difficult for electronic devices to perform real-time image color correction, impacting the user experience.

[0039] The embodiment of the present application provides a method for image color correction. The electronic device can provide prior knowledge of the light source color through an image training set, provide likelihood through AWB technology, and then calculate the probability of the light source color of the current image, that is, the posterior of the light source color. Based on the calculated probability of the light source color, the electronic device can estimate the light source color and correct the color of the current image, thereby improving the accuracy of image color correction, making the color of the image captured by the electronic device consistent with the color seen by the human eye, and avoiding the influence of the light source color on the image color. At the same time, this method has a small amount of calculation, does not need to rely on special hardware or additional restrictions, has a wider range of applications, and can also realize real-time correction of the image color, improving the user experience.

[0040] In the embodiment of the present application, the electronic device can be a mobile terminal such as a mobile phone, a tablet computer, a wearable device (such as a smart watch), an in-vehicle device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), or a professional camera or other device. The embodiment of the present application does not impose any restrictions on the specific type of the electronic device.

[0041] For example, Figure 1 1 shows a schematic structural diagram of an electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0042] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0043] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0044] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly retrieve it from the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0045] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0046] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0047] In an embodiment of the present application, the display screen 194 can be used to display content after image color correction according to the method provided in an embodiment of the present application in interfaces such as a preview interface and a shooting interface.

[0048] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0049] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and transformed into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0050] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0051] In an embodiment of the present application, the camera 193 may include one or more of the following cameras: a telephoto camera, a wide-angle camera, an ultra-wide-angle camera, a zoom camera, or a depth camera. A telephoto camera has a small shooting range and is suitable for shooting distant objects; a wide-angle camera has a larger shooting range; and an ultra-wide-angle camera has a larger shooting range than a wide-angle camera and is suitable for shooting panoramic and other large-scale scenes. A depth camera can be used to measure the object distance of the object to be photographed, that is, to measure the depth information of the object to be photographed. For example, it may include a three-dimensional (3D) depth camera, a time of flight (TOF) depth camera, or a binocular depth camera. The camera 193 may include a front camera and / or a rear camera.

[0052] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0053] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0054] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0055] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), a double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.

[0056] In an embodiment of the present application, the internal memory 121 can be used to cache an initial source image or a target image during a photo or video recording process. The processor 110 executes instructions stored in the internal memory 121 to perform color correction on the initial source image in preview mode or capture mode according to the color correction method provided in an embodiment of the present application, thereby obtaining a target image with colors consistent with those seen by the human eye.

[0057] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0058] The touch sensor 180K is also called a "touch panel." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, in a location different from that of the display screen 194.

[0059] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0060] In an embodiment of the present application, the camera 193 can be used to capture the initial original image and the training image; the internal memory 121 can be used to cache the initial original image and the target image after color correction during the preview process; the display screen 194 can be used to display the preview interface or the shooting interface, and can also be used to display the target image in the preview interface or the shooting interface for the user to view; the processor 110 corrects the color of the original image by running the instructions stored in the internal memory 121, so that the color of the target image after color correction is consistent with the color seen by the human eye, avoiding the influence of the light source color on the image color.

[0061] The following will be based on electronic equipment Figure 1 Taking the mobile phone with the structure shown in the figure as an example, the image color correction method provided by the embodiment of the present application is described. Figure 2 As shown, the method includes:

[0062] 200. Start the photo taking function on the mobile phone.

[0063] In an embodiment of the present application, when a user wants to use a mobile phone to capture an image, the mobile phone can be triggered to activate the camera function of the mobile phone. For example, after receiving an operation from the user to open the camera application or other applications with a camera function (such as AR applications such as TikTok or Hetu Cyberverse), the mobile phone can launch the corresponding application, thereby activating the application's camera function.

[0064] For example, the phone detects that the user clicks Figure 3 After the camera icon 301 shown in (a) is operated, the camera application's photo function is started and the following is displayed: Figure 3For example, the mobile phone displays the desktop or non-camera application interface, detects the user's voice command to open the camera application, starts the photo taking function, and displays the following Figure 3 The preview interface shown in (b) in the figure.

[0065] It should be noted that the mobile phone can also start the photo-taking function in response to other user touch operations, voice commands or quick gestures. The embodiment of the present application does not limit the operation of triggering the mobile phone to start the photo-taking function.

[0066] 201. The mobile phone obtains and displays the target image.

[0067] In some embodiments, after the camera function is activated on a mobile phone, the image color correction method provided in the embodiments of the present application can be used to perform color correction on the captured initial raw image to obtain a target image and display the target image on the preview interface. The target image is the image after color correction (white balance) processing, and the color of the target image is substantially consistent with the color seen by the user's human eye.

[0068] In other embodiments, after the mobile phone starts the camera function and enters the target camera mode, the target image can be displayed in the preview interface using the image color correction method provided in the embodiments of the present application.

[0069] In the target photography mode, the mobile phone can perform color correction (i.e., white balance) on each frame of the captured original image to obtain the target image and present the target image to the user on the preview interface. The target photography mode can be a white balance mode, an automatic white balance mode, or a color correction mode. The embodiments of this application do not limit the name of the target photography mode.

[0070] The following description will be made by taking the target shooting mode as the white balance mode as an example.

[0071] For example, after the mobile phone starts the camera function, if it detects that the user clicks Figure 3 After the operation of the control 302 shown in (b) in FIG. 1 , the white balance mode is entered and the display is as shown in FIG. Figure 3 The preview interface in white balance mode is shown in (c). Figure 3 The preview interface shown in (c) shows the target image after white balance processing. For another example, the mobile phone detects that the user clicks Figure 3 After the operation of the control 303 shown in (b) in FIG. 1 , the following is displayed: Figure 3 The interface shown in (d) in FIG; After the mobile phone detects that the user clicks the control 304, as shown in FIG. Figure 3 Enter the white balance mode as shown in (c).

[0072] For example, when the mobile phone is displaying the desktop or non-camera application interface, it detects the user's voice command to enter the white balance mode and starts the photo taking function. Figure 3 Enter the white balance mode as shown in (c).

[0073] It should be noted that the mobile phone can also start the photo-taking function and enter the white balance mode in response to other user touch operations, voice commands or quick gestures. The embodiment of the present application does not limit the specific operation of triggering the mobile phone to enter the white balance mode.

[0074] In some embodiments, the mobile phone can prompt the user of the function of the photo mode in the white balance mode by displaying information or voice broadcasting, etc. For example, the mobile phone can display a text prompt message on the preview interface: the color of the image in the white balance mode is consistent with the actual color.

[0075] It is understood that after activating the camera function, a mobile phone can enter a preview state. In this preview state, the mobile phone can perform color correction on the captured initial raw image to obtain a target image and display the target image in the preview interface. The target image is the image after color correction and has no color cast, that is, the colors in the target image are consistent with the colors in the actual scene.

[0076] In the embodiment of the present application, the process of converting the original image into the target image by the mobile phone can be considered as the process of color correction of the original image. Figure 2 , specifically describes the process of the mobile phone obtaining the target image.

[0077] In an embodiment of the present application, the mobile phone can obtain the probability p(L|I) of the light source color in the initial original image I based on the two-dimensional histogram p(c1) of the first processed image c1 (also referred to as the first original image c1) of the initial original image I, the two-dimensional histogram p(c2) of the edge image c2 of the first processed image c1, the prior probability p(L) of the light source color in the image training set, the probability p(L|c1) of the light source color in the first processed image c1, and the probability p(L|c2) of the light source color in the edge image c2. Furthermore, based on p(L|I), the light source color in the initial original image I is estimated, and image color correction is performed on the initial original image I to obtain the target image.

[0078] The following steps 301-308 specifically describe the process of determining the above-mentioned p(c1), p(c2), p(L), p(L|c1), p(L|c2) and p(L|I), estimating the light source color of the initial original image I based on p(L|I), and performing image color correction on the initial original image I.

[0079] 301. The mobile phone obtains an initial original image I.

[0080] When the mobile phone enters the preview state, the mobile phone can capture the initial original image through the camera according to the preset capture frame rate.

[0081] The initial raw image is an image that has not been color corrected. For example, if a mobile phone captures an initial raw image under indoor tungsten light, the initial raw image will be yellowish.

[0082] In some embodiments, the mobile phone can display the acquired initial original image on the preview interface. For example, the mobile phone can display the acquired initial original image on the preview interface. Figure 3 On the preview interface shown in (b).

[0083] In other embodiments, after the mobile phone obtains the initial original image, it may not display the initial original image, but directly display the image in the form of Figure 3 The final target image is displayed on the preview interface shown in (b).

[0084] That is to say, in the embodiment of the present application, the mobile phone may display the initial original image for the user to view, or may not display the initial original image, but directly display the processed target image for the user to view.

[0085] 302. The mobile phone pre-processes the initial original image I to obtain a first processed image c1.

[0086] In this embodiment of the present application, after the mobile phone acquires an initial raw image I, it may first preprocess the initial raw image I to obtain a first processed image c1. Here, the first processed image may also be referred to as the first raw image. Preprocessing can improve the accuracy of image color correction performed by the mobile phone and increase the speed of subsequent image color correction performed by the mobile phone.

[0087] The pre-processing includes black level correction (BLC), lens shading correction (LSC), and resizing.

[0088] In some embodiments, each time the mobile phone obtains a frame of initial raw image, it can immediately pre-process the frame of initial raw image.

[0089] In other embodiments, after the mobile phone obtains a preset number of frames of initial raw images, it may pre-process these initial raw images together.

[0090] 303. The mobile phone obtains an edge image c2 of the first processed image c1 based on an edge operator.

[0091] In some embodiments, the mobile phone can use an edge operator to obtain an edge image c2 of the first processed image c1. The edge image contains information about locations where pixel values ​​in the first processed image suddenly change. Because edge images can reflect more light source information in pure color scenes, using edge images can better process pure color scenes.

[0092] It should be noted that in some embodiments, the edge image c2 may not be obtained, and the probability distribution of the light source color of the initial original image I may be determined based only on the first processed image c1. It is understandable that the probability distribution of the light source color of the initial original image I determined based on c1 and c2 has higher accuracy.

[0093] Exemplarily, the mobile phone may obtain the edge image by using any edge operator such as a Canny operator, a Sobel operator, or a Prewitt operator.

[0094] The following uses the Canny operator as an example to describe how a mobile phone acquires an edge image c2 from the first processed image c1. The Canny operator is a multi-stage optimization operator with filtering, enhancement, and detection. When acquiring the edge image of the first processed image using the Canny operator, the first processed image is first smoothed using a Gaussian filter. The finite difference of the first-order partial derivative is then used to calculate the magnitude and direction of the gradient. Non-maximum suppression is then performed on the gradient magnitude. Finally, a dual-threshold algorithm is used to detect and connect edges, resulting in the edge image of the first processed image.

[0095] like Figure 4 As shown, Figure 4 (a) in the figure is the first processed image c1, Figure 4 (b) is the edge image c2 of the first processed image. Figure 4 In (b), we can see the edges in the first processed image, where the pixel values ​​suddenly change.

[0096] 304. The mobile phone performs first processing on the first processed image c1 and the edge image c2 respectively to obtain p(c1) and p(c2).

[0097] In the embodiment of the present application, the first process may include:

[0098] (1) According to the formula u=log(g / r) and the formula v=log(g / b), the RGB value of each pixel in the first processed image c1 is converted into a UV chromaticity value, and the RGB value of each pixel in the edge image c2 is converted into a UV chromaticity value;

[0099] (2) Based on the uv chromaticity value of each pixel in the first processed image c1 and the uv chromaticity value of each pixel in the edge image c2, a two-dimensional (2D) histogram of the first processed image c1 and a two-dimensional (2D) histogram of the edge image c2 are obtained, which are denoted as p(c1) and p(c2) respectively.

[0100] Here, p(c1) and p(c2) can reflect the probability of occurrence of each uv chromaticity value in c1 and c2, respectively.

[0101] Wherein, based on the UV chromaticity value of each pixel in the first processed image c1 and the UV chromaticity value of each pixel in the edge image c2, obtaining the two-dimensional histogram of the first processed image c1 and the two-dimensional histogram of the edge image c2 specifically includes:

[0102] The mobile phone obtains the pre-set two-dimensional histogram boundaries uv_start and uv_end, as well as the bin distance uv_bin of the two-dimensional histogram. Based on the UV chromaticity value of each pixel in the first processed image c1 and the UV chromaticity value of each pixel in the edge image c2, the mobile phone performs two-dimensional histogram statistics on the first processed image c1 and the edge image c2, respectively. The statistical results are then normalized to obtain the two-dimensional histogram p(c1) of the first processed image c1 and the two-dimensional histogram p(c2) of the edge image c2. In the embodiment of the present application, the boundaries of the x-axis and y-axis of the two-dimensional histogram are the same, both uv_start and uv_end.

[0103] It should be noted that the pre-set two-dimensional histogram boundaries uv_start and uv_end, as well as the bin interval uv_bin of the two-dimensional histogram, can be set by the user based on experience or needs. In some embodiments, the mobile phone can determine whether the values ​​of uv_start, uv_end, and uv_bin are appropriate by testing multiple images in advance.

[0104] It should also be noted that mobile phones usually set different uv_start, uv_end and uv_bin for different camera components. According to the camera component selected by the user, the pre-configured corresponding uv_start, uv_end and uv_bin can be called.

[0105] Among them, the greater the difference between uv_end and uv_start, and the smaller the uv_bin, the higher the accuracy of the two-dimensional histogram generated for the image based on uv_start, uv_end and uv_bin.

[0106] like Figure 5 As shown, Figure 5 (a) in the equation represents p(c1), Figure 5(b) in the figure represents p(c2). Here, the boundaries of the two-dimensional histogram are [-0.50, 1.50], and the bin interval is 0.25. For example, in p(c1), the height of the histogram corresponding to each uv chromaticity value represents the probability of that uv chromaticity value appearing in the first processed image c1.

[0107] 305. The mobile phone obtains the prior probability p(L) of the light source color.

[0108] In some embodiments, the mobile phone can calculate the prior probability p(L) of the light source color in advance and directly obtain p(L) when needed. In other embodiments, the mobile phone can also calculate p(L) in real time when p(L) is needed.

[0109] In an embodiment of the present application, the mobile phone can calculate the prior probability p(L) based on the image training set.

[0110] In some embodiments, the image training set includes multiple training images and RGB labels corresponding to the multiple training images, wherein each training image may correspond to an RGB label.

[0111] It is understood that the RGB label reflects the color of the light source in the corresponding training image. In some embodiments, the RGB label can be annotated using a color chart in the training image. For example, when capturing a training image, the mobile phone can place a 24-color chart next to the subject, remove the average color of the gray blocks from the captured training image, and use this as the RGB label for the training image.

[0112] The mobile phone can obtain the mapping relationship between the features of the training image and the color of the light source based on the training images and corresponding RGB labels in the image training set, so as to calculate p(L), p(L|c1) and p(L|c2).

[0113] It should be noted that, generally, the number of training images in the image training set is much smaller than the number of images required for the training data set in the AWB learning technology. The embodiment of the present application does not limit the number of training images in the image training set.

[0114] In some embodiments, the mobile phone can calculate p(L) by traversing and summing the training images and RGB labels in the image training set. The specific process is as follows:

[0115] Step 1: The mobile phone constructs a two-dimensional matrix, recorded as initial hist_L, and the shape of the matrix is ​​set to (h, w).

[0116] That is to say, the mobile phone constructs an initial matrix with a height of h and a width of w (a two-dimensional matrix of h×w).

[0117] In some embodiments, the initial two-dimensional matrix may be referred to as a two-dimensional summation matrix, which is not limited in this embodiment of the present application.

[0118] In some embodiments, the height and width of the initial two-dimensional matrix may be equal. In other embodiments, the height and width of the initial two-dimensional matrix may also be unequal.

[0119] It should be noted that the boundaries and bin spacing of each two-dimensional histogram generated in the embodiment of the present application are the same, so the height h and width w of the initial hist_L can be calculated based on the above uv_start, uv_end and uv_bin using the formula h = w = floor((uv_end - uv_start) / uv_bin). Where floor means rounding down.

[0120] For example, uv_start = 0.05, uv_end = 1.50, uv_bin = 0.25, then h = w = floor ((1.50 - (-0.50)) / 0.25) = 8. In other words, the mobile phone constructs an 8×8 two-dimensional matrix.

[0121] It can be understood that the initial hist_L constructed by the mobile phone is a two-dimensional zero matrix, that is, each item in the initial hist_L is 0.

[0122] Step 2: The mobile phone obtains hist_L based on the training images and RGB labels in the image training set.

[0123] In the embodiment of the present application, the mobile phone obtains hist_L based on the training images and RGB labels in the image training set, specifically including:

[0124] First, the mobile phone converts each RGB label in the image training set into a UV label according to the formula u = log (g / r) and the formula v = log (g / b), recorded as (u1, v1); then, based on the UV label, the mobile phone calculates label_x and label_y according to label_x = floor ((u1-uv_start) / uv_bin) and label_y = floor ((v1-uv_start) / uv_bin), respectively, where (label_x, label_y) represents the position coordinates corresponding to the RGB label in the initial hist_L; after that, the mobile phone adds 1 to the number at (label_x, label_y) in the two-dimensional matrix according to the formula hist_L[label_x, label_y] + = 1, thereby obtaining the statistical two-dimensional matrix hist_L based on the image training set, and hist_L can reflect the number of each RGB label in the image training set.

[0125] It can also be understood that hist_L counts the number of images corresponding to each RGB label in the image training set.

[0126] Step 3: The mobile phone normalizes hist_L to obtain the prior probability p(L) of the image training set.

[0127] In the embodiment of the present application, by normalizing the two-dimensional matrix hist_L obtained in step 2, the prior probability p(L) of the image training set can be obtained.

[0128] Among them, p(L) can be expressed as a two-dimensional histogram, and the probability that the light source color in the image training set is each uv chromaticity value can be obtained from p(L).

[0129] It should be noted that the boundaries and bin spacing of p(L) are also the above-mentioned uv_start, uv_end and uv_bin.

[0130] like Figure 6 As shown, the boundaries of p(L) are [-0.50, 1.50] and the class interval is 0.25. For example, in p(L), the height of the histogram corresponding to each uv chromaticity value represents the probability that the light source color in the image training set is the uv chromaticity value.

[0131] 306. The mobile phone calculates p(L|c1) based on the image training set and the first processed image, and calculates p(L|c2) based on the image training set and the edge image.

[0132] In the embodiment of the present application, the mobile phone can calculate p(L|c1) and p(L|c2) using the maximum a posterior probability estimation method. Taking the mobile phone calculating p(L|c1) based on the image training set and the first processed image as an example, the specific steps of the method are as follows:

[0133] Step 1: The mobile phone constructs a four-dimensional matrix, denoted as initial likelihood_matrix, and the shape of the matrix is ​​set to (h, w, h, w).

[0134] That is to say, the mobile phone constructs an initial four-dimensional matrix of h×w×h×w.

[0135] In some embodiments, the initial four-dimensional matrix may be referred to as a four-dimensional similarity matrix, which is not limited in this embodiment of the present application.

[0136] In some embodiments, h and w may be equal. In other embodiments, h and w may not be equal.

[0137] It should be noted that the dimensions h and w of the initial likelihood_matrix can also be calculated based on uv_start, uv_end, and uv_bin as above, using the formula h = w = floor((uv_end - uv_start) / uv_bin). Here, floor means rounding down.

[0138] For example, uv_start = 0.05, uv_end = 1.50, uv_bin = 0.25, then h = w = floor ((1.50 - (-0.50)) / 0.25) = 8. In other words, the mobile phone constructs an initial four-dimensional matrix of 8 × 8 × 8 × 8.

[0139] It can be understood that the initial likelihood_matrix constructed by the mobile phone is a four-dimensional zero matrix, that is, every item in the initial likelihood_matrix is ​​0.

[0140] Step 2: The mobile phone obtains the likelihood_matrix based on the training images and RGB labels in the image training set.

[0141] In the embodiment of the present application, the mobile phone obtains the likeness_matrix based on the training images and RGB labels in the image training set, specifically including:

[0142] First, the phone converts a training image in the image training set into a two-dimensional histogram of uv, and records the two-dimensional matrix corresponding to the two-dimensional histogram as the initial histogram. It can be considered that the initial histogram corresponds to a two-dimensional matrix with a shape of (h, w); then, the phone converts the corresponding rgb label into a uv label according to the formula u = log(g / r) and the formula v = log(g / b), recorded as (u2, v2); then, based on the uv label, the phone calculates label_x and label_y according to label_x = floor((u2-uv_start) / uv_bin) and label_y = floor((v2-uv_start) / uv_bin), respectively, where (label_x, la bel_y) is used to retrieve the position corresponding to the RGB label in the final four-dimensional matrix likelihood_matrix, thereby viewing the two-dimensional matrix of the training image corresponding to the RGB label; then, the phone performs a binary operation on the initial hist, that is, hist[hist>0]=1, to obtain hist, where hist is a 01 matrix; finally, the phone obtains the final four-dimensional matrix likelihood_matrix through the formula likeness_matrix[label_x, label_y,:,:]+=hist. Among them, the four-dimensional matrix likeness_matrix can be searched according to (label_x, label_y), and the corresponding hist obtained by the conversion of the corresponding training image can be queried at the coordinate corresponding to (label_x, label_y).

[0143] Step 3: The mobile phone calculates the first p(L|c1) for the first processed image c1.

[0144] In the embodiment of the present application, the mobile phone calculates the first p(L|c1) by performing a dot product between the likelihood_matrix and a four-dimensional matrix corresponding to a two-dimensional histogram converted from the first processed image c1.

[0145] Specifically, the phone converts the first processed image c1 into a two-dimensional histogram using a method similar to step 304 and will not be repeated here. The two-dimensional matrix corresponding to this two-dimensional histogram is denoted as hist_c1, and the shape of hist_c1 is (h, w). In order to enable the dot product of hist_c1 and likelihood_matrix, the shape of hist_c1 needs to be adjusted. The phone then adjusts the shape of hist_c1 using the reshape function, resulting in a four-dimensional matrix hist1_c1 with a shape of (1, 1, h, w).

[0146] Furthermore, the mobile phone can calculate the first p(L|c1) by performing a dot product of the likelihood_matrix and the hist1_c1 (the hist_c1 after the shape adjustment) according to the formula: first p(L|c1)=likehood_matrix*hist1_c1.

[0147] Step 4: The mobile phone performs a two-dimensional summation on the first p(L|c1) to obtain the second p(L|c1).

[0148] Since the first p(L|c1) is a four-dimensional matrix, in order to obtain a two-dimensional histogram p(L|c1), the first p(L|c1) needs to be converted into a two-dimensional matrix, which is recorded as the second p(L|c1).

[0149] In the embodiment of the present application, the mobile phone can perform a two-dimensional summation on the first p(L|c1) along the third axis and the fourth axis to obtain a two-dimensional matrix, namely the second p(L|c1).

[0150] Step 5: The mobile phone normalizes the second p(L|c1) to obtain p(L|c1).

[0151] In the embodiment of the present application, the final p(L|c1) can be obtained by normalizing the second p(L|c1).

[0152] like Figure 7 As shown in (a), in p(L|c1), the probability of the light source color in the first processed image c1 being each uv chromaticity value can be obtained.

[0153] In some embodiments, the mobile phone can also repeat the above steps 1 to 4 for the edge image c2 to calculate the probability p(L|c2) of the light source color in the edge image c2, such as Figure 7 This embodiment of the present application will not be described in detail.

[0154] 307. The mobile phone calculates p(L|I) based on p(c1), p(c2), p(L), p(L|c1) and p(L|c2).

[0155] In the embodiment of the present application, I represents the initial original image obtained by the mobile phone, L represents the light source color to be estimated, c1 represents the first processed image of the initial original image, and c2 represents the edge image of the first processed image, where c1 and c2 are the complete event sets of the initial original image I.

[0156] It is understandable that calculating the light source color in the initial original image obtained by the mobile phone can be considered as calculating the posterior probability of L in I, that is, calculating p(L|I).

[0157] p(L|I)∝p(I|L)p(L)

[0158]

[0159] ∝p(c1|L)p(c2|L)p(L)

[0160] ∝p(L|c1)p(c1)p(L|c2)p(c2)p(L)

[0161] Therefore, by performing dot multiplication on the above p(c1), p(c2), p(L), p(L|c1) and p(L|c2), we can get p(L|I), as shown in Figure 8 As shown. Figure 8 In the equation, p(L|I) is the calculated probability of the light source color in the initial original image I. The uv chromaticity value corresponding to the highest point is the estimated most likely light source color.

[0162] In an embodiment of the present application, since p(c1), p(c2), p(L), p(L|c1) and p(L|c2) are dot-multiplied when calculating p(L|I), the uv_start, uv_end and uv_bin of p(c1), p(c2), p(L), p(L|c1) and p(L|c2) are the same, and the shapes of the corresponding two-dimensional matrices are also the same.

[0163] In the above description, the probability distribution of the light source color of the initial original image I is determined based on the first processed image c1 and the edge image c2. It will be appreciated that, on this basis, the probability distribution of the light source color of the initial original image I can also be determined based on other images that include some of the features of the initial original image I. In other words, the probability distribution of the light source color of the initial original image I can also be determined based on the first processed image c1 and one or more other images that include some of the features of the initial original image I, thereby further improving the accuracy of the probability distribution of the light source color of the initial original image I.

[0164] 308. The mobile phone performs color correction on the initial original image to obtain and display the target image.

[0165] In the embodiment of the present application, if the RGB value of the initial original image obtained by the mobile phone in step 201 is recorded as (Ir, Ig, Ib), the light source color is recorded as (Lr, Lg, Lb), and the true color of the initial original image (the image color after color correction) is recorded as (Wr, Wg, Wb), since the RGB value of the initial original image is the product of the light source color and the true color of the initial original image, therefore:

[0166] I r =W r L r Formula (1)

[0167] I g =W g L g Formula (2)

[0168] I b =W b L b Formula (3)

[0169] It can be seen that the RGB value of the true color of the original original image can be calculated by W=I / L.

[0170] Furthermore, since u = log(g / r) and v = log(g / b), we have:

[0171] W u =I u –L u Formula (4)

[0172] W v =I v –L v Formula (5)

[0173] It can be seen that by subtracting the UV value of the light source color from the UV value of the initial original image, the UV value of the true color of the initial original image can be directly obtained.

[0174] In an embodiment of the present application, after the mobile phone obtains p(L|I), it can estimate the light source color of the scene of the initial original image captured by the mobile phone based on p(L|I). For example, the uv chromaticity value (Lu, Lv) corresponding to the highest point of p(L|I) in the figure is the estimated uv value of the light source color of the initial original image (in the current scene). Furthermore, the mobile phone can perform color correction on the initial original image according to Formula 4 and Formula 5 to obtain the uv value (Wu, Wv) of the true color of the initial original image, convert it into RGB value, and obtain the target image after color correction.

[0175] In other embodiments, the mobile phone may first convert the uv chromaticity value (Lu, Lv) corresponding to the highest point p(L|I) in the figure into an RGB value, and then perform color correction on the initial original image according to Formula 1-Formula 3 to obtain the RGB value of the true color of the initial original image, thereby obtaining the target image after color correction.

[0176] The colors in the target image are consistent with the colors in the actual scene. Finally, the mobile phone can display the target image in the preview interface for the user to view.

[0177] In summary, if Figure 9As shown, the mobile phone preprocesses the initial original image to obtain a first processed image c1 and an edge image c2; performs two-dimensional histogram statistics on c1 and c2 respectively to obtain p(c1) and p(c2); based on the image training set, the maximum a posterior method is used to calculate p(L|c1) and p(L|c2) for c1 and c2 respectively; finally, based on p(c1), p(c2), p(L|c1), p(L|c2) and the obtained prior probability p(L), the probability p(L|I) of the light source color in the initial original image is calculated, and then the light source color is estimated based on p(L|I), and the image color of the initial original image is corrected to obtain the target image.

[0178] It should be noted that the above process of performing image color correction on the initial original image and obtaining and displaying the target image can be implemented by software or designed into a dedicated ISP chip, and the embodiments of the present application do not limit this.

[0179] When performing image color correction using the above method, because it simultaneously considers the prior probability of the light source color provided by the image training set and the likelihood provided by AWB, the probability of the light source color of the current image calculated by this method is more accurate. Furthermore, estimating the light source color of the current image based on this probability and performing color correction on the current image can improve the accuracy of image color correction, ensuring that the colors of images captured by electronic devices are consistent with those seen by the human eye, thereby preventing the influence of light source color on image color and improving the user experience.

[0180] Furthermore, because the method provided by the embodiments of the present application does not rely on a machine learning model, the computational effort is relatively small and does not require specialized hardware. Thus, performing image color correction according to the method provided by the embodiments of the present application can reduce the computational effort while ensuring the accuracy of image color correction, thereby enabling real-time image color correction and expanding the scope of application of the method.

[0181] For example, in a standard academic set indicator test, as shown in Table 1, the image color correction method provided by the embodiments of this application can effectively improve the accuracy of image color correction compared to existing grayscale world methods, original Bayesian algorithms, and deep learning algorithms. It should be noted that the numbers in Table 1 represent angular errors. The smaller the number, the higher the precision, and the higher the accuracy of image color correction.

[0182] At the same time, compared with the deep learning algorithm, the image color correction method provided by the embodiment of the present application has fewer parameters and a small amount of calculation. In other words, the image color correction method provided by the embodiment of the present application can achieve accuracy comparable to that of the deep learning algorithm when the amount of calculation is small. Therefore, the image correction method provided by the embodiment of the present application does not need to rely on special hardware (such as GPU and NPU, etc.), and can achieve real-time correction of the color of the initial original image.

[0183] In addition, the image color correction method provided in the embodiment of the present application does not have any additional restrictions. Therefore, the image color correction method provided in the embodiment of the present application has a wider scope of application and can be used alone.

[0184] Table 1

[0185]

[0186] 202. After the mobile phone detects the user's photo-taking operation, it displays the target image on the shooting interface, or after the mobile phone detects the user's video-recording operation, it generates the target video.

[0187] If the phone detects a user's camera operation in the preview interface, it can display the target image in the shooting interface. The camera operation can be a user clicking a camera control or button, or other user touch operations, voice commands, or quick gestures.

[0188] In some embodiments, the mobile phone can also save the target image at the same time after detecting the user's photo-taking operation.

[0189] In other embodiments, the mobile phone enters the preview interface after detecting an operation to return to the preview interface in the capture interface. In still other embodiments, the mobile phone automatically enters the preview interface after the target image is displayed in the capture interface for a preset time period. The preset time period can be 30 seconds or 1 minute, etc., and is not limited in this embodiment of the present application. The target image is displayed in the gallery of the preview interface.

[0190] If the phone detects a user recording action in the preview interface, it can generate a target video. The target video is generated by the video encoder based on the target image saved during the phone's shooting process. The recording action can be a user clicking a recording control or button, or other user touch operations, voice commands, or quick gestures.

[0191] In some embodiments, after the target video is generated, the mobile phone re-enters the preview interface. Optionally, the generated target video is displayed in the gallery window of the preview interface.

[0192] In some embodiments, the target image in the target video is obtained by performing the above-mentioned processing of steps 301 to 308 on each frame of the acquired original image in real time. In other embodiments, the target image in the target video may be obtained by performing the above-mentioned processing of steps 301 to 308 on each frame of the acquired original image simultaneously. This embodiment of the present application is not limited to this.

[0193] It is understood that in some embodiments, after the mobile phone performs color correction (white balance processing) on ​​the acquired initial raw image using the image color correction method to obtain the target image, the target image can be first stored in the mobile phone's cache. For example, the target image can be stored in the preview cache or the video cache. In the preview scenario of shooting a video or taking a photo, the mobile phone can obtain the various initial raw images that have been color corrected, i.e., the target image, from the preview cache in real time and output them in the viewfinder window of the preview interface. During the video shooting process, the mobile phone can obtain the various initial raw images that have been color corrected, i.e., the target image, from the video cache in real time and encode these target images into the target video of this recording for storage.

[0194] It is also understood that, in some embodiments, the image color correction methods provided by the embodiments of the present application can also be applied to existing images. For example, for downloaded images that have not been color-corrected, or images previously taken with a mobile phone that have not been color-corrected, the methods provided by the embodiments of the present application can still be used to correct the colors of these images so that the colors of the corrected images are consistent with the actual colors.

[0195] It can be seen that no matter in the photo scene or video scene, no matter in the preview scene or recording scene, after the mobile phone obtains each frame of the initial original image, it can obtain the target image based on the prior knowledge of the image data in the image training set.

[0196] The method provided in the embodiment of the present application combines the prior knowledge in the image training set and the AWB algorithm, and can improve the accuracy of image color correction with a smaller amount of calculation, improve the color of photos and videos taken by electronic devices, avoid the influence of light source color on the color of images taken by electronic devices, make the color of images taken by electronic devices consistent with the color seen by the human eye, and improve user experience.

[0197] It is understandable that in order to implement the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of this application.

[0198] In this embodiment, the electronic device can be divided into functional modules according to the above-mentioned method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into a single processing module. The above-mentioned integrated modules can be implemented in the form of hardware. It should be noted that the module division in this embodiment is illustrative and is only a logical functional division. In actual implementation, other division methods may be used.

[0199] The present application also provides an electronic device including one or more processors and one or more memories. The one or more memories are coupled to the one or more processors and are configured to store computer program code. The computer program code includes computer instructions. When the one or more processors execute the computer instructions, the electronic device executes the above-described method steps to implement the image color correction method in the above-described embodiment.

[0200] The present application also provides an electronic device, such as Figure 10 As shown, it includes: a display screen 1001, one or more processors 1002, multiple cameras 1003, a memory 1004, and one or more computer programs 1005. The above-mentioned components can be connected via one or more communication buses 1006. The one or more computer programs 1005 are stored in the above-mentioned memory 1004 and are configured to be executed by the one or more processors 1002. The one or more computer programs 1005 include instructions, and the above-mentioned instructions can be used to perform the various steps in the above-mentioned embodiments. Among them, all relevant contents of each step involved in the above-mentioned method embodiment can be referred to the functional description of the corresponding physical device and will not be repeated here.

[0201] For example, the processor 1002 may be Figure 1 The processor 110 shown, the memory 1004 can be specifically Figure 1 The internal memory 121 shown, the camera 1003 can be specifically Figure 1 The camera 193 shown, the display screen 1001 can be specifically Figure 1Display screen 194 is shown.

[0202] An embodiment of the present application further provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the image color correction method in the above-mentioned embodiment.

[0203] An embodiment of the present application further provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the image color correction method executed by the electronic device in the above-mentioned embodiment.

[0204] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory to enable the chip to execute the image color correction method performed by the electronic device in the above-mentioned method embodiments.

[0205] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0206] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0207] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0208] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0209] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0210] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0211] The above content is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. An image color correction method, applied to electronic equipment, characterized in that: include: Get the first original raw image; determining a probability distribution of the light source color of an initial raw image based on the first original raw image and a priori probability distribution of light source color, wherein the first original raw image is the initial raw image or an image obtained by preprocessing the initial raw image; performing color correction on the initial raw image based on a probability distribution of a light source color of the initial raw image; Acquire an edge image of the first original raw image; wherein the edge image includes information about locations where pixel values ​​in the first original raw image suddenly change; Determining the probability distribution of the light source color of the initial raw image based on the first original raw image and the prior probability distribution of the light source color includes: determining the probability distribution of the light source color of the initial raw image based on the first original raw image, the edge image, and the prior probability distribution of the light source color.

2. The method according to claim 1, characterized in that The method further includes determining a priori probability distribution of the light source color.

3. The method according to claim 2, characterized in that Determining a priori probability distribution of the light source color includes: Based on the RGB label of each training image in the plurality of training images included in the image training set, obtaining the UV label of each training image; Based on the UV labels of each training image, a first matrix is ​​obtained; wherein the first matrix is ​​a two-dimensional matrix, and the value of each item in the first matrix is ​​used to represent the number of UV labels whose corresponding matrix coordinates are the matrix coordinates of the item; Based on the first matrix, a priori probability distribution of the light source color is determined.

4. The method according to any one of claims 1 to 3, characterized in that The determining, based on the first original raw image and the prior probability distribution of the light source color, of the probability distribution of the light source color of the initial raw image includes: The probability distribution of the light source color of the initial raw image is determined according to the two-dimensional chromaticity histogram of the first original raw image, the probability distribution of the light source color of the first original raw image, and the prior probability distribution of the light source color.

5. The method according to claim 1, wherein The determining, based on the first original raw image, the edge image, and the prior probability distribution of the light source color, comprises: Determine the probability distribution of the light source color of the initial raw image based on the two-dimensional chromaticity histogram of the edge image, the probability distribution of the light source color of the edge image, the two-dimensional chromaticity histogram of the first original raw image, the probability distribution of the light source color of the first original raw image, and the prior probability distribution of the light source color.

6. The method according to claim 5, characterized in that The method further comprises: Determine a probability distribution of light source color of the first original raw image.

7. The method according to claim 6, characterized in that Determining the probability distribution of light source color in the first original raw image includes: Obtaining a two-dimensional chromaticity histogram of each training image in a plurality of training images included in the image training set; Based on the UV label of each training image and the two-dimensional chromaticity histogram of each training image, obtaining a second matrix; wherein the second matrix is ​​a four-dimensional matrix, each item of the second matrix is ​​used to represent the two-dimensional chromaticity histogram of the training image corresponding to the UV label, and the matrix coordinates of each item in the second matrix are the same as the matrix coordinates of the corresponding UV label; Based on the second matrix and the third matrix, a fourth matrix is ​​obtained; wherein the third matrix is ​​a four-dimensional matrix corresponding to the two-dimensional chromaticity histogram of the first original raw image; Determine the probability distribution of light source color in the first original raw image according to the fourth matrix.

8. An electronic device, characterized in that: include: Camera; Display screen; one or more processors and one or more memories; The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, wherein the computer program codes include computer instructions. When the one or more processors execute the computer instructions, the electronic device performs the image color correction method according to any one of claims 1 to 7.

9. A computer storage medium, characterized in that The method comprises computer instructions, which, when executed on an electronic device, enable the electronic device to execute the image color correction method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is run on a computer, the computer is enabled to execute the image color correction method according to any one of claims 1 to 7.

Images