Image processing method, intelligent terminal and storage medium

Abstract

The invention provides an image processing method, an intelligent terminal and a readable storage medium, and the method comprises the following steps: S1, in response to a preset operation for a target image, calculating a white balance gain value of the target image according to a target color point RBG value of the target image in a light source scene, and carrying out white balance compensation processing on the target image. According to the method, the RGB value of the target color point in the light source scene where the target image is located can be obtained without existence of the target color point in the target image, and finally white balance compensation is performed on the target image based on the RGB value of the target color point, so that the problem of inaccurate image white balance compensation caused by absence of the target color point in the image in the prior art is solved, and the image quality is improved. The application range is wider, and the white balance effect is better.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, an intelligent terminal and a storage medium. Background Art

[0002] White balance adjustment is an important part of color imaging, color display and other equipment. White balance adjustment refers to the use of digital signal processing to adaptively correct the color temperature deviation of the image obtained by the image sensor or the color temperature deviation of the LCD according to the image data, so that it is as consistent as possible with the actual scene color temperature.

[0003] Among traditional AWB (automatic white balance) algorithms, the one based on the grayscale world principle is a widely used method. The core assumption of this algorithm is that for an image with sufficiently rich colors, the average values ​​of its three RGB color channels will tend to be equal, that is, the image as a whole will appear gray. Based on this principle, the algorithm adjusts the gain of the RGB channels in the image so that the average color of the image is close to gray, thereby achieving the effect of white balance.

[0004] In the process of conceiving and implementing the present application, the inventors discovered that there are at least the following problems: if there are no white or gray points in the scene, the AWB statistics will be wrong; if there are color points close to white or gray points, AWB errors will also occur, resulting in color cast in the final compensated color.

[0005] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the invention

[0006] In response to the above technical problems, the present application provides an image processing method, an intelligent terminal and a storage medium, which can calculate the RGB value of the target color point in the target image under the light source scene where the target image is located, and perform white balance compensation on the target image without the presence of white spots or gray spots in the image. This is more convenient and has a wide range of applications.

[0007] To solve the above technical problems, the present application provides an image processing method, which includes the following steps: S1, in response to a preset operation for a target image, calculating the white balance gain value of the target image according to the RBG value of the target color point under the light source scene of the target image, and performing white balance compensation processing on the target image.

[0008] Optionally, the method further includes the step: S2, obtaining the spectral intensity distribution of the light source scene of the target image, and calculating the RGB value of the target color point according to the spectral intensity distribution.

[0009] Optionally, calculating the RGB value of the target color point according to the spectral intensity distribution includes: obtaining the RGB value of the target color point using an RGB value calculation formula based on the spectral intensity distribution, reflectivity information of the target color point and a camera sensor sensitivity function.

[0010] Optionally, acquiring the spectral intensity distribution of the light source scene of the target image includes at least one of the following: detecting the spectral intensity distribution of the light source scene of the target image by using multiple spectrometers; identifying the light source scene in the target image, and predicting the spectral intensity distribution of the light source scene.

[0011] Optionally, identifying the light source scene in the target image and predicting the spectral intensity distribution of the light source scene include: identifying the light source scene in the target image and obtaining at least two vector groups of the light source scene through singular decomposition; identifying the target area in the target image and obtaining the image reflectivity of the target area; and predicting the spectral intensity distribution of the light source scene based on the image reflectivity and the vector group.

[0012] Optionally, calculating the white balance gain value of the target image includes: acquiring corresponding R, G and B three-channel data values ​​according to the target color point RBG value; and acquiring R channel gain and B channel gain based on the G channel data value.

[0013] Optionally, before S1, the method further includes: performing black compensation processing and / or demosaic interpolation on the target image.

[0014] Optionally, the target color point includes a white point and a gray point.

[0015] The present application also provides an intelligent terminal, comprising: a memory and a processor, wherein the memory stores an image processing program, and when the image processing program is executed by the processor, the steps of the above method are implemented.

[0016] The present application also provides a computer storage medium, wherein the computer storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0017] As described above, the image processing method of the present application includes the steps of: in response to a preset operation for a target image, calculating the white balance gain value of the target image according to the RBG value of the target color point in the light source scene of the target image, and performing white balance compensation processing on the target image. Through the above technical solution, the RGB value of the target color point in the light source scene where the target image is located is directly calculated, and finally the target image is white balanced based on the calculated RGB value of the target color point, which solves the problem of inaccurate image white balance compensation caused by the absence of white points or gray points in the image in the prior art, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0019] Figure 1 A schematic diagram of the hardware structure of a smart terminal for implementing each embodiment of the present application;

[0020] Figure 2 A communication network system architecture diagram provided for an embodiment of the present application;

[0021] Figure 3 is a flowchart of an image processing method according to the first embodiment.

[0022] Figure 4 is a schematic diagram of image processing according to the image processing method shown in the first embodiment;

[0023] Figure 5 is a flowchart of an image processing method according to a second embodiment;

[0024] Figure 6 is a schematic flow chart of an image processing method according to a third embodiment;

[0025] Figure 7 is a specific flowchart of the image processing method according to the third embodiment.

[0026] The realization of the purpose, functional features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. The above-mentioned drawings have shown clear embodiments of this application, which will be described in more detail later. These drawings and textual descriptions are not intended to limit the scope of the concept of this application in any way, but to illustrate the concept of this application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0027] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0028] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0029] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in this article, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination". Furthermore, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising" and "including" indicate that there are the described features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or", "and / or", "including at least one of the following" etc. used in this application can be interpreted as inclusive, or mean any one or any combination. For example, “comprising at least one of the following: A, B, C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”, and for another example, “A, B or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”. An exception to this definition will only occur when a combination of elements, functions, steps or operations are inherently mutually exclusive in some manner.

[0030] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are displayed in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and it can be performed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0031] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0032] It should be noted that, in this article, step codes such as S11, S12, etc. are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the sequence. When implementing it specifically, those skilled in the art may execute S12 first and then S11, etc., but these should all be within the scope of protection of this application.

[0033] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0034] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0035] The smart terminal may be implemented in various forms. For example, the smart terminal described in this application may include smart terminals such as mobile phones, tablet computers, laptop computers, PDAs, portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, etc., as well as fixed terminals such as digital TVs and desktop computers.

[0036] The following description will be made by taking a mobile terminal as an example, and those skilled in the art will understand that, in addition to components specifically used for mobile purposes, the construction according to the embodiments of the present application can also be applied to fixed-type terminals.

[0037] See also Figure 1 , which is a schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application, the mobile terminal 100 may include: RF (Radio Frequency, radio frequency) unit 101, WiFi module 102, audio output unit 103, A / V (audio / video) input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, processor 110, and power supply 111 and other components. Those skilled in the art can understand that Figure 1 The structure of the mobile terminal shown in the figure does not constitute a limitation on the mobile terminal. The mobile terminal may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0038] Combine the following Figure 1 The following is a detailed introduction to the various components of the mobile terminal:

[0039] The radio frequency unit 101 can be used for receiving and sending signals during information transmission or communication. Specifically, after receiving the downlink information of the base station, it is sent to the processor 110 for processing; in addition, the uplink data is sent to the base station. Generally, the radio frequency unit 101 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 101 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communications can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution) and 5G, etc.

[0040] WiFi is a short-range wireless transmission technology. Mobile terminals can help users send and receive emails, browse web pages, and access streaming media through WiFi module 102. It provides users with wireless broadband Internet access. Figure 1 The WiFi module 102 is shown, but it is understandable that it is not an essential component of the mobile terminal and can be omitted as needed without changing the essence of the invention.

[0041] The audio output unit 103 can convert the audio data received by the RF unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output it as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, etc. Moreover, the audio output unit 103 can also provide audio output related to a specific function performed by the mobile terminal 100 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0042] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processor (GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 106. The image frame processed by the graphics processor 1041 can be stored in the memory 109 (or other storage medium) or sent via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in a telephone call mode, a recording mode, a voice recognition mode, and other operating modes, and can process such sound into audio data. The processed audio (voice) data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 101 in the case of a telephone call mode. The microphone 1042 can implement various types of noise elimination (or suppression) algorithms to eliminate (or suppress) noise or interference generated in the process of receiving and sending audio signals.

[0043] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 1061 and / or the backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured on the mobile phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0044] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0045] The user input unit 107 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile terminal. Optionally, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 1071 or near the touch panel 1071 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 110, and can receive and execute the command sent by the processor 110. In addition, the touch panel 1071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 1071, the user input unit 107 may further include other input devices 1072. Optionally, the other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, a function key (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, etc., which are not specifically limited here.

[0046] Optionally, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. Then, the processor 110 provides corresponding visual output on the display panel 1061 according to the type of touch event. Figure 1 In the figure, the touch panel 1071 and the display panel 1061 are used as two independent components to implement the input and output functions of the mobile terminal. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to implement the input and output functions of the mobile terminal, which is not limited here.

[0047] The interface unit 108 serves as an interface through which at least one external device can be connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and an external device.

[0048] The memory 109 can be used to store software programs and various data. The memory 109 can mainly include a program storage area and a data storage area. Optionally, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 109 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0049] The processor 110 is the control center of the mobile terminal. It uses various interfaces and lines to connect various parts of the entire mobile terminal. It executes various functions of the mobile terminal and processes data by running or executing software programs and / or modules stored in the memory 109, and calling data stored in the memory 109, so as to monitor the mobile terminal as a whole. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor. Optionally, the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 110.

[0050] The mobile terminal 100 may also include a power supply 111 (such as a battery) for supplying power to various components. Preferably, the power supply 111 may be logically connected to the processor 110 via a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system.

[0051] although Figure 1 Not shown, the mobile terminal 100 may further include a Bluetooth module, etc., which will not be described in detail here.

[0052] In order to facilitate understanding of the embodiments of the present application, the communication network system on which the mobile terminal of the present application is based is described below.

[0053] See also Figure 2 , Figure 2 A communication network system architecture diagram is provided for an embodiment of the present application, wherein the communication network system is an LTE system of universal mobile communication technology, and the LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203 and an operator's IP service 204, which are connected in sequence for communication.

[0054] Optionally, UE201 may be the above-mentioned terminal 100, which will not be described in detail here.

[0055] E-UTRAN 202 includes eNodeB 2021 and other eNodeBs 2022 , etc. Optionally, eNodeB 2021 may be connected to other eNodeBs 2022 via a backhaul (eg, an X2 interface), and eNodeB 2021 is connected to EPC 203 , and eNodeB 2021 may provide UE 201 with access to EPC 203 .

[0056] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gate Way) 2034, PGW (PDN Gate Way) 2035 and PCRF (Policy and Charging Rules Function) 2036, etc. Optionally, MME 2031 is a control node that processes signaling between UE 201 and EPC 203, and provides bearer and connection management. HSS 2032 is used to provide some registers to manage functions such as home location register (not shown in the figure), and store some user-specific information such as service features and data rates. All user data can be sent through SGW2034, PGW2035 can provide IP address allocation and other functions for UE 201, PCRF2036 is the policy and charging control policy decision point for service data flow and IP bearer resources, which selects and provides available policy and charging control decisions for the policy and charging execution function unit (not shown in the figure).

[0057] The IP service 204 may include the Internet, an intranet, an IMS (IP Multimedia Subsystem) or other IP services.

[0058] Although the above introduction takes the LTE system as an example, those skilled in the art should know that the present application is not only applicable to the LTE system, but also to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA and future new network systems (such as 5G), etc., without limitation here.

[0059] Based on the above-mentioned mobile terminal hardware structure and communication network system, various embodiments of the present application are proposed.

[0060] First embodiment

[0061] See also Figure 3 , Figure 3 is a flowchart of an image processing method according to the first embodiment.

[0062] like Figure 3 As shown, the first embodiment of the present application provides an image processing method, comprising the following steps:

[0063] S1, in response to a preset operation for a target image, a white balance gain value of the target image is calculated according to the RBG value of a target color point under a light source scene of the target image, and a white balance compensation process is performed on the target image.

[0064] The target image is an image that needs to be white balance compensated, for example, an image whose color is inconsistent with the color of the image viewed by a normal person, an image that is bluish or yellowish, etc.

[0065] The preset operations include: spectral distribution intensity prediction operation of the target image, white balance compensation operation and image shooting operation, etc.

[0066] Light source scenes include: natural light source scenes (such as sunlight and moonlight) and artificial light sources (such as LED lights). If the same area is photographed in different light source scenes with other conditions being the same, the spectral intensity distribution in the target image will also be different. The basis vectors of the spectral intensity distribution corresponding to different light source scenes are also different, and the spectral intensity distribution finally calculated is also inconsistent, and the RGB value of the target color point finally calculated is also inconsistent.

[0067] The target color point includes but is not limited to a white point, and may also be a gray point or other color point whose R, G, and B values ​​in RGB are not 0. In this embodiment, the target color point is a white point.

[0068] The white balance gain value is a coefficient value used to adjust the three values ​​of R, G and B of the target image. In this embodiment, the white balance gain value is a coefficient value used to adjust the R and B values ​​in the target image, and when the R value and the B value in the target image are different, the white balance gain values ​​of the two are also different.

[0069] White balance (WB) is an important concept in photography and videography. It is used to adjust the color balance in an image and ensure that white objects can be correctly restored to white under different light sources. The goal of white balance is to correct white objects under different light sources to the white color perceived by the human eye, eliminate color cast in the image, and make the image color conform to the visual perception of the human eye.

[0070] Optionally, the method further includes the step: S2, obtaining the spectral intensity distribution of the light source scene of the target image, and calculating the RGB value of the target color point according to the spectral intensity distribution.

[0071] Spectral power distribution (SPD), a general light source is a complex light composed of a mixture of colored lights of different wavelengths. If the intensity of each colored light in its spectrum is measured by a sensor, the radiant energy of colored lights of different wavelengths can be obtained. The instrument used to measure radiant energy is called a spectroradiometer. Spectral density represents the magnitude of radiant energy within a unit wavelength interval. Usually, the radiant energy of colored lights of different wavelengths in a light source changes with the change of wavelength, so spectral density is a function of wavelength. The functional relationship between spectral density and wavelength is called spectral distribution. In practice, the spectral distribution is more often described by the functional relationship between the relative value of spectral density and wavelength, which is called relative spectral energy (power) distribution, denoted as S(λ). Relative spectral energy distribution can be expressed by any value, but it is usually obtained by taking the radiant energy at wavelength λ=555nm as 100 as a reference point for comparison. If the spectral wavelength λ is used as the horizontal coordinate and the relative spectral energy distribution S(λ) is used as the vertical coordinate, the relative spectral energy distribution curve of the light source can be drawn.

[0072] The spectral intensity distribution of the target image can be obtained by professional equipment such as a spectrometer, or can be predicted by other data in the target image, which is not limited here.

[0073] Optionally, calculating the RGB value of the target color point according to the spectral intensity distribution includes: obtaining the RGB value of the target color point using an RGB value calculation formula based on the spectral intensity distribution, reflectivity information of the target color point and a camera sensor sensitivity function.

[0074] The reflectance information of the target color point can be obtained through a large number of tests and stored in a database.

[0075] The camera sensor sensitivity function stores the sensitivity functions corresponding to all sensors in the database. Before obtaining the camera sensor sensitivity function, the sensor model or camera (or other terminal with camera) model can be identified to obtain the corresponding camera sensor sensitivity function from the database; when identifying the camera model, it is also necessary to confirm the sensor model carried by the camera through the camera model, and then obtain the camera sensor sensitivity function according to the sensor model.

[0076] The RGB value calculation formula is a formula for calculating the three values ​​of R, G and B of a target color point in a target image, and can be referred to in the following formulas (a) to (c).

[0077] Taking the white point as an example, at 400-700nm, the reflectivity of the white point is obtained through testing; the camera sensor sensitivity function is known; and the spectral intensity distribution has been obtained through professional equipment or other data.

[0078] Therefore, the RGB value of the white point can be calculated by the following basic formula:

[0079]

[0080] Where E(λ) is the spectral intensity distribution; r(λ) is the white point reflectance; S r (λ) is the sensor sensitivity function. The coefficient a is related to the light intensity and has nothing to do with the spectrum and color, that is, it does not affect the color of the target image.

[0081] Substitute the known data such as spectral intensity distribution, white point reflectance and sensor sensitivity function obtained in the above steps into the formula, and the a coefficient is independent of the spectrum and color and can be set to 1 to calculate the white point RGB value.

[0082] Optionally, acquiring the spectral intensity distribution of the light source scene of the target image includes at least one of the following: detecting the spectral intensity distribution of the light source scene of the target image by using multiple spectrometers; identifying the light source scene in the target image, and predicting the spectral intensity distribution of the light source scene.

[0083] A spectrometer (i.e., a device for detecting spectral intensity distribution) can directly obtain the spectral intensity distribution of the light source scene of the target image when shooting the target image.

[0084] In the absence of a spectrometer, the light source scene of the target image can be identified, thereby obtaining the basis vector of the spectral intensity distribution of the light source scene of the target image. The spectral intensity distribution is then predicted using the basis vector and other known quantities.

[0085] Optionally, identifying the light source scene in the target image and predicting the spectral intensity distribution of the light source scene include: identifying the light source scene in the target image and obtaining at least two vector groups of the light source scene through singular decomposition; identifying the target area in the target image and obtaining the image reflectivity of the target area; and predicting the spectral intensity distribution of the light source scene based on the image reflectivity and the vector group.

[0086] Light source scenes include artificial light source scenes and natural light source scenes (eg, light sources other than artificial ones such as sunlight / daylight, moonlight and starlight). It should be noted that in this embodiment, the natural light source scene generally refers to the sunlight / daylight scene.

[0087] In the absence of a spectrometer, the light source scene in the target image is identified to confirm whether the target image is taken under natural light or artificial light.

[0088] The basis vectors of the spectral intensity distribution corresponding to the natural light source and the artificial light source are obtained in advance by singular decomposition using a large amount of data. After the basis vectors are confirmed, the spectral intensity distribution is split into multiple vector groups. The number of basis vectors in the vector group can be any positive integer greater than or equal to 2.

[0089] A vector group consists of the product of the vector coefficients and the basis vectors.

[0090] E(λ)=α1E1(λ)+α2E2(λ)+α3E3(λ)

[0091] In the above formula there are 3 vector groups.

[0092] The target area refers to the area of ​​a certain characteristic object in the target image, such as a person, a wall, a tree, and water. It should be noted that in this embodiment, the target area is the area where the face of a person is located in the target image.

[0093] The target image is a person image.

[0094] The image reflectivity is obtained after a large number of tests in advance and is stored in a database. In this embodiment, the image reflectivity is the skin color reflectivity. By performing a large number of tests on skins of different skin colors, the image reflectivity of each color skin is finally obtained. Therefore, when obtaining the image reflectivity of the target area, it is necessary to first identify the skin color of the face, and then obtain the reflectivity of the corresponding face skin color from the database. The method for obtaining the image reflectivity of other feature objects can refer to the method for obtaining the face.

[0095] Optionally, S03 includes: jointly deriving various color expressions of the target image through the image reflectance, the vector group and the camera sensor sensitivity function; and calculating the spectral intensity distribution in combination with actual values ​​of each color in the target image.

[0096] Camera sensor sensitivity function. Generally speaking, different models of cameras use different sensors, so the corresponding camera sensor sensitivity functions are also different. However, if the same sensor is used, the corresponding camera sensor sensitivity functions are also the same.

[0097] Color expression refers to the matrix corresponding to R, G and B and the vector coefficients, see formula (6).

[0098] The actual value of each color, that is, the average camera signal of the target area obtained by the camera, etc.

[0099] Optionally, S0 further includes the step: S04, when it is identified that the light source direction of the light source scene is a backlight direction, correcting the spectral intensity distribution by a backlight ratio function.

[0100] The backlight ratio function is the brightness ratio of each pixel position when shooting in a backlight environment and a frontlight environment.

[0101] It should be noted that after identifying the light source scene of the target image, the light direction needs to be confirmed. Then, based on the light source direction, it is determined whether the predicted spectral intensity distribution needs to be corrected.

[0102] In this embodiment, only the light source direction of the natural light source scene needs to be determined. In other embodiments, the light source direction of the artificial light source scene can also be determined.

[0103] Optionally, before S1, the step further includes: performing black compensation processing and / or demosaic interpolation processing on the target image.

[0104] By performing black compensation and demosaic interpolation in advance on the target image, a three-channel RGB image can be obtained, that is, the R, G and B values ​​of the image can be obtained through black compensation processing and / or demosaic interpolation processing.

[0105] Optionally, the target color point includes a white point and a gray point.

[0106] It is relatively simpler to obtain the reflectance of white points and gray points.

[0107] Exemplarily, the spectral intensity distribution is obtained by the following formula:

[0108]

[0109] In the above formula, r(λ) is the skin color reflectance, E(λ) is the spectral intensity distribution (SPD) of the ambient light. R, G, B are the raw data obtained by demosaicing interpolation in the skin color area, and the coefficient κ is related to the light intensity but has nothing to do with color prediction and white balance parameters.

[0110] Therefore, the problem is transformed into a multiple solution problem for E(λ). In order to obtain E(λ), we can regard E(λ) as a combination of three basis vectors. [The number of basis vectors can be any positive integer greater than or equal to 2, that is, 4, 5, 6... are all OK], that is:

[0111] E(λ)=α1E1(λ)+α2E2(λ)+α3E3(λ)......(4)

[0112] Then, define λ i is a uniformly distributed wavelength, and Δλ is the distance between the wavelengths.

[0113]

[0114] Therefore, the formula will be converted into the form of (5), α1, α2, α3 can be taken out, and other known quantities can be calculated and multiplied into a large known quantity, and then the formula can be rewritten into the form of (6). j,p The matrix M is obtained by merging, and the discrete algebraic equation system obtained by combining equations (1)-(4) is given by the following equation:

[0115]

[0116] The image processing method provided in this embodiment splits the spectral intensity distribution of the light source scene through the basis vectors of the light source, and then predicts the spectral intensity distribution by combining the image reflectivity and the camera sensor sensitivity function. Finally, white balance compensation is performed on the target image according to the predicted result of the spectral intensity distribution, thereby improving the accuracy of the spectral intensity distribution and the white balance compensation effect.

[0117] Optionally, calculating the white balance gain value of the target image includes: acquiring corresponding R, G and B three-channel data values ​​according to the target color point RBG value; and acquiring R channel gain and B channel gain based on the G channel data value.

[0118] The RGB color value range refers to the intensity value range of the three color channels: red (R), green (G), and blue (B). In computers, the values ​​of these three color channels are usually represented by integers between 0 and 255, where 0 represents the minimum intensity of the color channel (i.e., no color component), and 255 represents the maximum intensity of the color channel (i.e., the color component is the strongest).

[0119] For example, the white point is (255, 255, 255). That is, under normal circumstances, R=255, G=255, and B=255.

[0120] Specifically, after calculating the RGB value of the white point in the target image (the value is most likely not (255, 255, 255)), the corresponding R, G and B three-channel data values ​​are obtained respectively; then the G channel data value is used to obtain the R channel gain and the B channel gain respectively.

[0121] For example, the R channel gain, the G channel gain value (which is 1), and the B channel gain are obtained by Rgain=G / R, Gain=G / G, and Bgain=G / B.

[0122] See also Figure 4 , Figure 4Schematic diagram of image processing according to the image processing method of the first embodiment. After the target image with yellowish color is processed in the above manner, a color closer to the color seen by human eyes can be obtained.

[0123] The image processing method provided in this embodiment calculates the RGB value of the target color point in the target image through spectral intensity distribution, thereby realizing white balance compensation processing of the target image. It does not require the target color point to exist in the image, has a wide range of applications, and improves the white balance compensation effect.

[0124] Second embodiment

[0125] This embodiment is proposed based on any of the above embodiments.

[0126] See also Figure 5 , Figure 5 is a flowchart of an image processing method according to the second embodiment.

[0127] like Figure 5 As shown, the image processing method provided by the second embodiment of the present application includes the following steps:

[0128] S11: Obtain the spectral intensity distribution of the target image.

[0129] The target image is an image collected by a camera after a camera device (such as a camera, a mobile phone, a computer, or other device carrying a camera) starts a camera function. In other embodiments, the target image can also be an image obtained by shooting.

[0130] Spectral power distribution (SPD) is the spectral intensity distribution of the ambient light corresponding to the target image. Visible white light is composed of a spectrum of light of various colors, ranging from 380nm (purple) to 760nm (red). It should be noted that the wavelength range of visible light is not a fixed value, because different people have different perceptions of visible light. The wavelength of electromagnetic waves that the human eye can perceive is between 400 and 700nm. SPD is a graph that shows the power (intensity) of each wavelength of light produced by a specific light source.

[0131] S12: Obtain the RGB value of the target color point based on the spectral intensity distribution.

[0132] The target color point can be a white point or a gray point. The RGB value of the white point in the target image is calculated by the reflectivity of the white point or the gray point and the above spectral intensity distribution. The reflectivity of the white point or the gray point is obtained in advance by calculating a large amount of data.

[0133] The RGB value of a white point or a gray point can be directly calculated, so there is no need to search for the white point or the gray point in the target image.

[0134] It should be noted that under normal circumstances, the three-channel data values ​​of R, G and B of the white point are equal, such as 255, 255, 255 (the gray point is 127, 127, 127). Therefore, the white balance can be adjusted by the RGB value of the white point.

[0135] A variety of colors can be obtained by changing the three color channels of red (R), green (G), and blue (B) and superimposing them on each other. RGB represents the colors of the three channels of red, green, and blue, which almost includes all the colors that the human eye can perceive.

[0136] S13: Obtaining a white balance gain of the target image according to the RGB values.

[0137] White balance is an important concept in photography and videography. It refers to adjusting the color temperature of the camera or video camera to make white objects appear true white under different light sources. Different light sources have different color temperatures, so it is necessary to adjust the white balance gain to achieve accurate color reproduction.

[0138] Exemplarily, when the calculated RGB value is the white point, the white balance gains of the R channel and the B channel are obtained by Rg ain=G / R, G ain=G / G, and Bg ain=G / B. In this case, the G channel data value is taken as a reference, the G channel data value remains unchanged, and the R channel data value and the B channel data value are adjusted respectively by the G channel data value.

[0139] It should be noted that if the RGB values ​​of other color points are calculated, when calculating the white balance gain, it is necessary to make adaptive adjustments according to the RGB values ​​of the corresponding color points under normal circumstances.

[0140] Under normal circumstances, color points with equal normal RGB values ​​(except black, and the R, G, B three-channel data values ​​of other color points cannot be 0) are calculated to facilitate white balance gain calculation.

[0141] S14: Perform white balance processing on the target image according to the white balance gain.

[0142] The white balance gain is actually an adjustment coefficient. Since the G channel gain is 1, the white balance gain is used to adjust the data values ​​of the R channel and the B channel up / down so that the adjusted color of the target image appears to be the real color.

[0143] In the above manner, the spectral intensity distribution of the light source and the white point reflectance are used to calculate the RGB value of the white point in the target image, and then the white balance compensation of the target image is obtained by Rgain=G / R, Gain=G / G, Bgain=G / B, and the target image is adjusted based on the white balance compensation so that the adjusted target image presents the true color.

[0144] Optionally, before the step of acquiring the RGB value of the target color point according to the spectral intensity distribution, the step includes: acquiring target reflectance information of the target color point, where the target color point includes a white point and a gray point.

[0145] The target color point may be, but is not limited to, a white point and a gray point, and may also be a color point other than red, green, blue, yellow, and black points whose RGB value does not contain 0.

[0146] The RGB values ​​of the white point (255, 255, 255) and the gray point (127, 127, 127) are 3 equal numbers and are not 0. They are easy to set and convenient for obtaining their corresponding reflectivity through a large number of tests. They are also convenient for white balance gain calculation.

[0147] Optionally, before the step of acquiring the RGB value of the target color point according to the spectral intensity distribution, the step also includes: acquiring identification information of the terminal; confirming the terminal model of the terminal according to the identification information; and acquiring a sensor sensitivity function according to the terminal model.

[0148] The sensor sensitivity function is related to the terminal model, and the sensitivity function that matches the terminal model needs to be used. In order to ensure the algorithm's adaptability to various terminal models and the accuracy of white balance gain calculation, the sensor sensitivity function needs to be replaced as a whole according to the terminal model during the calculation of the white point RGB value.

[0149] For example, when using mobile phone A to take photos / videos, it is necessary to obtain the sensor sensitivity function of the sensor model corresponding to mobile phone A, and then use the sensor sensitivity function to calculate the white point RGB value; when using mobile phone B (with a different sensor from A) to take photos / videos, use the sensor sensitivity function of the sensor model corresponding to mobile phone B to calculate the white point RGB value.

[0150] Optionally, the step of acquiring the RGB value of the target color point according to the spectral intensity distribution includes: acquiring the RGB value of the target color point using an RGB value calculation formula based on the spectral intensity distribution, target reflectivity information and a sensor sensitivity function.

[0151] Exemplarily, the RGB value of the white point is calculated by the following basic formula:

[0152]

[0153] Where E(λ) is the spectral intensity distribution; r(λ) is the white point reflectance; S r (λ) is the sensor sensitivity function. The a coefficient is related to the light intensity and has nothing to do with the spectrum and color.

[0154] Substitute the known data such as spectral intensity distribution, white point reflectance and sensor sensitivity function obtained in the above steps into the formula, and the a coefficient is independent of the spectrum and color and can be set as a preset parameter to calculate the white point RGB value.

[0155] Optionally, the step of obtaining the white balance gain of the target image according to the RGB values ​​includes: obtaining corresponding R, G and B three-channel data values ​​according to the RGB values; and obtaining R channel gain and B channel gain based on the G channel data value.

[0156] Because the R, G, and B channel data of the white point are equal under normal circumstances, after calculating the RGB value of the white point in the target image, the G channel data can be used as a reference value, and then the R channel gain and the B channel gain can be calculated separately through the G channel data.

[0157] Illustratively, Rgai n=G / R, Gai n=G / G, and Bgai n=G / B.

[0158] Optionally, the step of respectively obtaining the R channel gain and the B channel gain based on the G channel data value includes: obtaining corresponding R, G and B three-channel data values ​​according to the RGB image of the target image; and respectively obtaining the R channel data value and the B channel data value after white balance processing of the target image according to the R channel data value and the B channel data value of the target image and the R channel gain and the B channel gain.

[0159] Exemplarily, after obtaining the R channel gain and the B channel gain of the target image, the R, G and B channel data values ​​of the target image are obtained, and then the R channel gain is multiplied by the R channel data value, and the B channel gain is multiplied by the B channel data value, so as to obtain the R and B channel data values ​​of the target image after white balance compensation, so that the target image presents the true color (see Figure 4 ).

[0160] Optionally, the step of obtaining the spectral intensity distribution of the target image includes: splitting the spectral intensity distribution of the light source scene in which the target image is located into a combination of multiple combination coefficients and basis vector data; predicting the spectral intensity distribution based on the skin color reflectance, basis vector data and sensor sensitivity function of the skin color area in the target image.

[0161] In the above manner, the RGB value of the target color point in the image can be calculated through the spectral intensity distribution and the reflectance of the target color point, and then the image is white balanced according to the calculated RGB value of the target color point, so that the processed image presents true color without the presence of white points or gray points in the image.

[0162] Third embodiment

[0163] This embodiment is proposed based on any of the above embodiments.

[0164] See also Figure 6 , Figure 6 is a flowchart of an image processing method according to the third embodiment.

[0165] like Figure 6 and Figure 7 As shown, the image processing method provided in the third embodiment of the present application includes:

[0166] S21: Acquire a target area image in a target image.

[0167] The target image is an image collected by a camera after a camera device (such as a camera, a mobile phone, a computer, or other device carrying a camera) starts a camera function. In other embodiments, the target image can also be an image obtained by shooting.

[0168] The target area image is a region image including an object, target and color in the target image.

[0169] In this embodiment, the image processing is for human images, so the target area image includes the area where the human skin is located: the skin color area (the same as below), that is, the target area image is the face area image, or other area images containing human skin (such as hands and arms, etc. that are not blocked by clothes).

[0170] In other embodiments, the target area image is an area of ​​a specific color (eg, gray, white, red, and green) in the target image, thereby realizing processing of other images.

[0171] S22: Obtain the image reflectivity of the target area image and basis vector data of the light source scene where the target image is located.

[0172] The image reflectivity is the reflectivity of a certain color point in the target image, and the reflectivity is obtained through multiple tests. In this embodiment, the image reflectivity corresponds to the skin color in the skin color area as the skin color reflectivity.

[0173] The light source scene in which the target image is located, that is, the lighting conditions under which the target image is acquired, including: daylight source (outdoor shooting) and artificial light source (studio shooting), etc.

[0174] The basis vector data is obtained by collecting a large number of SPD (Spectral Power Distribution) of sunlight and artificial light sources, and using singular value decomposition (SVD). Singular decomposition is a mathematical calculation method and an important matrix decomposition in linear algebra. Singular value decomposition is the generalization of eigendecomposition on any matrix.

[0175] S23: Obtain the sensor sensitivity function of the terminal.

[0176] The sensor sensitivity function is a function used to describe the sensitivity of a sensor to light of different wavelengths. The sensor sensitivity functions of sensors carried by different terminals are generally different.

[0177] S24: Predicting the spectral intensity distribution of the light source scene according to the image reflectivity, basis vector data and sensor sensitivity function.

[0178] For example, it is assumed that the sensor sensitivity function (S r (λ),S g (λ),S b (λ)) and the target image contains skin color areas (through image recognition and segmentation). On this basis, the following three basic formulas can be listed:

[0179]

[0180] In the above formula, r(λ) is the skin color reflectance, and E(λ) is the spectral intensity distribution (SPD) of the ambient light. R, G, and B are the raw data obtained by demosaicing interpolation in the skin color area. The coefficient κ is related to the light intensity, which is independent of the color prediction and white balance parameters, so k can be set as a preset parameter.

[0181] In the above formula, all except E(λ) are known. In order to obtain E(λ), E(λ) can be split into a combination of 3 basis vectors (the number of basis vectors can be any positive integer greater than or equal to 2, that is, 2, 3, 4, 5, 6... are all possible), that is:

[0182] E(λ)=α1E1(λ)+α2E2(λ)+α3E3(λ)…………(4)

[0183] Then, define λ i is a uniformly distributed wavelength, and Δλ is the distance between the wavelengths.

[0184]

[0185] Therefore, the formula will be transformed into the form of (5), and then we will take out α1, α2, α3, and calculate other known quantities, multiply them into a large known quantity, and then the formula can be rewritten into the form of (6). j,p The matrix M is obtained by merging, and the discrete algebraic equation system obtained by combining equations (1)-(4) is given by the following equation:

[0186]

[0187] It is worth noting that the element m in the above matrix M j,p Depends on the basis vector E p (λ i ). The basis vector E p (λ i ) is obtained by accumulating the SPD of a large number of daylight sources and artificial light sources using singular decomposition. Element m j,p It also depends on the reflectivity of the pixel. By accumulating a large number of skin reflectivities, the average reflectivity of the skin reflectivity can be obtained. Therefore, by substituting all known quantities into equation (5), the matrix M in equation (6) can be calculated.

[0188] Similarly, the skin area in the scene is known and we can get the average camera signal of the skin area Therefore, we can use and replace R, G and B in equation (6).

[0189] Therefore, the combination coefficients α1, α2, α3 of the ambient light SPD can be obtained from equation (6), and then the spectral intensity distribution SPD of the ambient light can be obtained using equation (4).

[0190] In the above manner, by introducing the basis vectors of the spectral intensity distribution, the problem with multiple solutions is transformed into a problem with a solution, and then the spectral intensity distribution of the light source used to capture the target image is solved through multiple basic formulas.

[0191] Optionally, the step of obtaining the image reflectivity of the target area image and the basis vector data of the ambient light of the scene in which the target image is located includes: identifying the skin color of the target area image, and obtaining the corresponding image reflectivity based on the skin color, and the image reflectivity includes skin color reflectivity; obtaining the light source scene in which the target image is located, and obtaining the basis vector data of the corresponding light source based on the light source scene, and the light source scene includes a daylight light source scene and an artificial light source scene.

[0192] Skin colors are generally: white, yellow, brown and black, in addition to red / brown and bronze; in some cases, each color can be subdivided according to skin tone, such as cool tones and warm tones.

[0193] Skin color reflectivity, by collecting the skin color reflectivity of the same race and sorting them from small to large, and by studying the accuracy of color temperature, the relationship between color temperature and skin color reflectivity is confirmed, and the accuracy of light source prediction, CCT prediction and AWB Gain is further improved. Skin color reflectivity is an adjustable parameter to achieve the best effect for different people and different scenes.

[0194] Skin color area segmentation: Segment the skin color area from the target image. Generally, the entire face area is used. In special cases, partial skin color area can be used as needed, or the block combination skin color can be adjusted as needed.

[0195] Skin color values ​​can be calculated using the CIELAB system (Lab color space). This system uses three parameters L, a, and b to represent skin color, where L represents lightness, a represents the difference between red and green, and b represents the difference between yellow and blue. The L value represents lightness, ranging from 0 (black) to 100 (white). a and b represent the color saturation on the red-green axis and the blue-yellow axis, respectively. By measuring the values ​​of these parameters, a skin color value can be calculated to divide skin color. The ITA° value (individual type angle) is also an important indicator, which is used to measure the depth of skin color. The larger the ITA° value, the lighter the skin color, and vice versa.

[0196] Median of skin color area: The skin color of the same user is not completely consistent. It has been verified that the effect and stability of the median of skin color are better than the mean of skin color or the mean of skin color after removing a certain proportion of extreme values. The median of skin color is selected when the calculation amount is minimal. After confirming the skin color of the skin color area, the skin color reflectance corresponding to the skin color is obtained. The skin color reflectance is the average value of the reflectance of the corresponding skin color obtained through a large amount of data analysis and calculation.

[0197] Specifically, in order to pursue high accuracy of the predicted spectral intensity distribution, different light source scenes have certain differences in basis vectors. Therefore, the daylight and artificial light scenes can be separated to use different basis vectors respectively, avoiding partial errors (CCT prediction error of about 200K) caused by mixed use of basis vectors.

[0198] Optionally, after the step of obtaining the light source scene in which the target image is located and obtaining basis vector data of the corresponding light source according to the light source scene, it includes: when it is detected that the scene in which the target image is located is a daylight light source scene, obtaining the light source position in the daylight light source scene, and the light source position includes backlight, front light and side light.

[0199] Specifically, the spectrum difference between backlighting and frontlighting is large, and the backlighting situation needs to be separated for separate processing. Therefore, it is necessary to determine the position of the light source in the target image to determine how the target image was taken, such as front light, side light (top light) or backlighting.

[0200] Optionally, the step of predicting the spectral intensity distribution of the light source scene according to the image reflectivity, basis vector data and sensor sensitivity function includes: when the light source position is backlit, correcting the spectral intensity distribution by a backlight processing algorithm.

[0201] Specifically, in the case of backlighting of a daylight source, the final SPD prediction is corrected by using a backlighting ratio function through a backlighting processing algorithm, thereby obtaining a CCT, a Gain color temperature, and a gain that match the backlighting situation.

[0202] In other implementations, the backlight processing algorithm may also be used in the case of artificial light sources, but there are certain differences in the backlight ratio functions of artificial light sources and daylight light sources.

[0203] Optionally, the step of acquiring the sensor sensitivity function of the terminal includes: acquiring identification information of the terminal; confirming a terminal model of the terminal according to the identification information; and acquiring the sensor sensitivity function according to the terminal model.

[0204] Sensor sensitivity function: It is related to the terminal model. You need to use a sensitivity function that matches the terminal model. In order to ensure that the algorithm is compatible with various terminal models, the sensor sensitivity function needs to be replaced as a whole according to the terminal model when predicting the spectral intensity distribution.

[0205] Optionally, before the step of acquiring the target area image in the target image, the step includes: performing a first processing on the target image to acquire an RGB image of the target image, wherein the first processing includes black compensation and demosaic interpolation.

[0206] Specifically, after acquiring the target image, black compensation and early demosaic interpolation are performed on the target image. The early demosaic interpolation is to obtain three-channel data of the target image to separate the three sensors and obtain the R, G and B values ​​of the image.

[0207] Optionally, the step of obtaining the image reflectance of the target area image and the basis vector data of the light source scene where the target image is located includes: splitting the spectral intensity distribution corresponding to the light source scene into a combination of multiple combination coefficients and basis vector data.

[0208] Specifically, the R, G, and B values ​​can be calculated using the following three basic formulas:

[0209]

[0210] In the above formula, r(λ) is the skin color reflectance; E(λ) is the spectral intensity distribution (SPD) of the ambient light; R, G, B are the raw data obtained by demosaicing interpolation of the skin color area; the coefficient κ is related to the light intensity, but has nothing to do with color prediction and white balance parameters; (S r (λ),S g (λ),S b (λ)) is the sensor sensitivity function. The skin color reflectance and sensor sensitivity function are both saved in the database in advance and can be called according to actual needs.

[0211] The basis vector data of different light sources are also pre-stored in the database. In order to obtain the spectral intensity distribution, the spectral intensity distribution can be regarded as a combination of multiple basis vectors and combination coefficients. The number of basis vectors can be any positive integer greater than or equal to 2, that is, 2, 3, 4, 5, 6... In this embodiment, the spectral intensity distribution is split into a combination of 3 basis vectors and combination coefficients.

[0212] Optionally, the step of predicting the spectral intensity distribution of the light source scene based on the image reflectance, basis vector data and sensor sensitivity function includes: obtaining the multiple combination coefficients based on the image reflectance, basis vector data and sensor sensitivity function; and predicting the spectral intensity distribution based on the multiple combination coefficients and the basis vector data.

[0213] Specifically, the sensor sensitivity function obtained through the terminal signal, the skin color reflectance obtained by identifying the skin color of the person in the target image, and the basis vector data of the corresponding light source obtained by light source scene recognition are substituted into the above formula 5 to calculate the matrix M in formula 6. Then, the R, G and B three-channel data of the target image obtained by the terminal preprocessing the target image are used to obtain the average camera signal of the skin color area Reuse Replace R, G and B in Formula 6 to obtain the combination coefficients α1, α2 and α3 of the spectral intensity distribution of the corresponding light source; finally, substitute the combination coefficients α1, α2 and α3 into Formula 4 to finally calculate the spectral intensity distribution of the corresponding light source.

[0214] The image processing method of this embodiment introduces the basis vector data of the light source to split the spectral intensity distribution of the scene where the target image is located, thereby predicting the spectral intensity distribution; the basis vector data of the light source scene where the target image is located can be accurately obtained through light source scene judgment, thereby improving the prediction accuracy of the spectral intensity distribution. By identifying the skin color in the target image, the skin color reflectance corresponding to different skin colors is obtained, thereby further improving the prediction accuracy of the spectral intensity distribution.

[0215] See also Figure 6 and Figure 7 The image processing method in this embodiment implements the process as follows: Figure 7 As shown:

[0216] By performing black compensation and demosaic interpolation on the image, the image collected or shot by the camera or other camera terminal can be converted from the original tiff format to a three-channel RGB image to obtain three-channel data to separate the three sensors, and realize adaptive interpolation according to the Bayer array type.

[0217] Skin color area segmentation: Segment the skin color area from the image. Generally, the entire face area is used, and partial skin color area can be adjusted to use as needed.

[0218] Median in skin color area: The effect and stability of the median are better than the mean or the mean after removing a certain proportion of extreme values, so the median is selected with the least calculation amount.

[0219] Scene judgment: If high precision is required, it is recommended to separate the daylight and artificial light scenes and use different basis vectors. Mixing basis vectors will cause some errors (CCT prediction error of about 200K).

[0220] Determining the position of sunlight: The spectrum difference between backlighting and frontlighting is large, so the backlighting situation needs to be separated and processed separately.

[0221] Backlight processing algorithm: In the case of backlight, the backlight ratio function will be used to correct the final SPD prediction to obtain the CCT and Gain that match the backlight situation.

[0222] White point calculation: The ideal white point can be accurately calculated based on the spectral intensity distribution, so as to obtain AWB Gain (white balance compensation) more accurately, bringing better color balance and true color of the image.

[0223] The present application also provides an intelligent terminal, which includes a memory and a processor. The memory stores an image processing program, and when the image processing program is executed by the processor, the steps of the image processing method in any of the above embodiments are implemented.

[0224] The present application also provides a computer-readable storage medium, on which an image processing program is stored. When the image processing program is executed by a processor, the steps of the image processing method in any of the above embodiments are implemented.

[0225] In the embodiments of the smart terminal and computer-readable storage medium provided in the present application, all technical features of any of the above-mentioned image processing method embodiments may be included, and the expanded and explained contents of the specification are basically the same as those of the embodiments of the above-mentioned methods, and will not be repeated here.

[0226] The embodiment of the present application further provides a computer program product, which includes a computer program code. When the computer program code runs on a computer, the computer executes the methods in the above various possible implementation modes.

[0227] An embodiment of the present application also provides a chip, including a memory and a processor, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device equipped with the chip executes the methods in various possible implementation modes as described above.

[0228] It is understood that the above scenarios are only examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of the present application. The technical solutions of the present application can also be applied to other scenarios. For example, it is known to those skilled in the art that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0229] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0230] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0231] The units in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0232] In the present application, the same or similar terminology concepts, technical solutions and / or application scenario descriptions are generally described in detail only the first time they appear. When they appear again later, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of the present application, for the same or similar terminology concepts, technical solutions and / or application scenario descriptions that are not described in detail later, reference can be made to the previous related detailed descriptions.

[0233] In the present application, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0234] The various technical features of the technical solution of the present application can be arbitrarily combined. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present application.

[0235] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, a controlled terminal, or a network device, etc.) to execute the method of each embodiment of the present application.

[0236] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integrated. The available medium can be a magnetic medium, (e.g., a floppy disk, a storage disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state storage disk Solid State Disk (SSD)), etc.

[0237] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises the following steps: S1, in response to a preset operation for a target image, a white balance gain value of the target image is calculated according to the RBG value of a target color point under a light source scene of the target image, and a white balance compensation process is performed on the target image.

2. The image processing method according to claim 1, characterized in that: Also includes the steps: S2, obtaining the spectral intensity distribution of the light source scene of the target image, and calculating the RGB value of the target color point according to the spectral intensity distribution.

3. The image processing method according to claim 2, characterized in that: The step of calculating the RGB value of a target color point according to the spectral intensity distribution comprises: Based on the spectral intensity distribution, reflectivity information of the target color point and the camera sensor sensitivity function, an RGB value calculation formula is used to obtain the RGB value of the target color point.

4. The image processing method according to claim 2, wherein: The step of acquiring the spectral intensity distribution of the light source scene of the target image comprises at least one of the following: Detecting the spectral intensity distribution of the light source scene of the target image by using a plurality of spectrometers; A light source scene in the target image is identified, and a spectral intensity distribution of the light source scene is predicted.

5. The image processing method according to claim 4, characterized in that: The identifying the light source scene in the target image and predicting the spectral intensity distribution of the light source scene includes: Identify the light source scene in the target image and obtain at least two vector groups of the light source scene through singular decomposition; identify the target area in the target image and obtain the image reflectivity of the target area; and predict the spectral intensity distribution of the light source scene based on the image reflectivity and the vector group.

6. The image processing method according to any one of claims 1 to 5, characterized in that: The calculating the white balance gain value of the target image includes: According to the target color point RBG value, corresponding R, G and B three-channel data values ​​are obtained respectively; The R channel gain and the B channel gain are obtained based on the G channel data value.

7. The image processing method according to claim 6, characterized in that: Before the S1, it also includes: Perform black compensation processing and / or demosaic interpolation on the target image.

8. The image processing method according to claim 6, characterized in that: The target color points include white points and gray points.

9. An intelligent terminal, characterized in that: The intelligent terminal comprises: a memory and a processor, wherein an image processing program is stored in the memory, and when the image processing program is executed by the processor, the steps of the image processing method according to any one of claims 1 to 8 are implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 8 are implemented.

Images