Image processing method, intelligent terminal and storage medium

Abstract

The present application provides an image processing method, an intelligent terminal and a storage medium, the method comprising the following steps: obtaining or determining a first color-biased image to be processed; processing the first color-biased image through an image processing model to obtain a non-color-biased image. The technical solution of the present application can improve the correction effect of the color-biased image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, an intelligent terminal and a storage medium. Background Art

[0002] In some implementations, when shooting an image, the image may have a color cast due to factors such as shooting scene, light, improper exposure, etc. The color cast image may be color corrected to restore the true color of the color cast image.

[0003] In the process of conceiving and realizing the present application, the inventors found that there are at least the following problems: in the process of correcting a color-shifted image, the entire area of ​​the color-shifted image is usually corrected to obtain an image without color shift. However, in the above process, the degree of color shift in different areas of the color-shifted image may be different, which makes it difficult to obtain the best correction effect for different areas of the color-shifted image, thereby resulting in a poor correction effect on the color-shifted image.

[0004] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the invention

[0005] In response to the above technical problems, the present application provides an image processing method, an intelligent terminal and a storage medium, which improve the correction effect of color-biased images.

[0006] In order to solve the above technical problems, the present application provides an image processing method which can be applied to a smart terminal, comprising the following steps:

[0007] S2. Process the first color-biased image using an image processing model to obtain an image without color bias.

[0008] Optionally, before step S2, the method further includes the following steps:

[0009] S1. Acquire or determine a first color-shifted image to be processed.

[0010] Optionally, the image processing model includes a frequency separation module.

[0011] Optionally, the S2 step includes the following steps:

[0012] S21, performing frequency separation processing on the first color-shifted image by the frequency separation module to obtain a first low-frequency image and a high-frequency image corresponding to the first color-shifted image;

[0013] S22: Process the first low-frequency image and the high-frequency image to obtain the image without color deviation.

[0014] Optionally, the step S21 includes the following steps:

[0015] S211, determining at least one image block of the first color-shifted image by the frequency separation module;

[0016] S212: Determine an orthogonal transformation image corresponding to the image block to determine the first low-frequency image and the high-frequency image.

[0017] Optionally, the orthogonally transformed image comprises a discrete cosine transformed image.

[0018] Optionally, the image processing model includes a low-frequency correction module and a high-frequency correction module.

[0019] Optionally, the low-frequency correction module includes an encoder module, an attention mechanism module and a decoder module.

[0020] Optionally, the step S22 includes the following steps:

[0021] S221, performing at least one feature extraction process on the first low-frequency image by the encoder module to obtain a preliminary feature image, a first low-frequency feature image, and a second low-frequency feature image;

[0022] S222, processing the second low-frequency feature image by the attention mechanism module to obtain a weighted feature image;

[0023] S223, processing the first low-frequency feature image, the weighted feature image and the preliminary feature image by the decoder module to obtain the second low-frequency image;

[0024] S224: Perform high-frequency correction processing on the high-frequency image and the second low-frequency image by the high-frequency correction module to obtain an image without color cast.

[0025] Optionally, the image size of the second low-frequency feature image is smaller than that of the first low-frequency image.

[0026] Optionally, the decoder module includes a pooling module, an upsampling module and a dual convolution module.

[0027] Optionally, the step S223 includes:

[0028] Performing feature extraction processing on the weighted feature image through the pooling module to obtain an initial output image;

[0029] Processing the initial output image and the M first low-frequency feature images through an upsampling module to obtain at least one output image;

[0030] The at least one output image and the preliminary feature image are processed by the dual convolution module to obtain the second low-frequency image.

[0031] Optionally, the method further comprises at least one of the following:

[0032] The pooling module includes a convolution layer, a linear rectification function layer and a deconvolution layer;

[0033] The upsampling module includes two convolutional layers, two linear rectification function layers and a deconvolution layer;

[0034] The dual convolution module includes two convolution layers and two linear rectification function layers.

[0035] Optionally, the step S224 includes:

[0036] Performing an addition process on the high-frequency image and the second low-frequency image by the high-frequency correction module to obtain a combined image;

[0037] The combined image is corrected by the high-frequency correction module to obtain the image without color cast.

[0038] Optionally, obtaining or determining a first color-shifted image to be processed includes:

[0039] receiving an image processing request sent by an electronic device, wherein the image processing request includes the first color-biased image;

[0040] The first color-shifted image is determined according to the image processing request.

[0041] Optionally, the step S1 includes:

[0042] Taking pictures to obtain an initial image;

[0043] When it is determined that the initial image is a color-shifted image, the initial image is determined as the first color-shifted image.

[0044] Optionally, after the first color-biased image is processed by an image processing model to obtain an image without color bias, the method further includes:

[0045] Display the image without color cast; or,

[0046] The image without color cast is sent to an electronic device.

[0047] Optionally, processing the first color-biased image by using an image processing model to obtain an image without color bias includes:

[0048] inputting the first color-biased image into the image processing model;

[0049] The image output by the image processing model is determined as the image without color deviation.

[0050] The present application also provides an image processing device, comprising a determination module and a processing module, wherein:

[0051] The determination module is used to obtain or determine a first color-shifted image to be processed;

[0052] The processing module is used to process the first color-biased image through an image processing model to obtain an image without color bias.

[0053] Optionally, the image processing model includes a frequency separation module; the processing module is specifically used to:

[0054] Performing frequency separation processing on the first color-shifted image by the frequency separation module to obtain a first low-frequency image and a high-frequency image corresponding to the first color-shifted image;

[0055] The first low-frequency image and the high-frequency image are processed to obtain the image without color deviation.

[0056] Optionally, the image processing model includes a low-frequency correction module and a high-frequency correction module, the low-frequency correction module includes an encoder module, an attention mechanism module and a decoder module; the processing module is specifically used for:

[0057] Performing at least one feature extraction process on the first low-frequency image by the encoder module to obtain a preliminary feature image, a first low-frequency feature image, and a second low-frequency feature image, wherein the image size of the second low-frequency feature image is smaller than that of the first low-frequency image;

[0058] Processing the second low-frequency feature image by the attention mechanism module to obtain a weighted feature image;

[0059] Processing the first low-frequency feature image, the weighted feature image and the preliminary feature image by the decoder module to obtain the second low-frequency image;

[0060] The high-frequency correction module performs high-frequency correction processing on the high-frequency image and the second low-frequency image to obtain an image without color cast.

[0061] Optionally, the decoder module includes a pooling module, M upsampling modules and a dual convolution module;

[0062] Optionally, the processing module is specifically used for:

[0063] Performing feature extraction processing on the weighted feature image through the pooling module to obtain an initial output image;

[0064] Processing the initial output image and the M first low-frequency feature images through an upsampling module to obtain at least one output image;

[0065] The at least one output image and the preliminary feature image are processed by the dual convolution module to obtain the second low-frequency image.

[0066] Optionally, at least one of the following is also included:

[0067] The pooling module includes a convolution layer, a linear rectification function layer and a deconvolution layer;

[0068] The upsampling module includes two convolutional layers, two linear rectification function layers and a deconvolution layer;

[0069] The dual convolution module includes two convolution layers and two linear rectification function layers.

[0070] Optionally, the processing module is specifically used for:

[0071] Performing an addition process on the high-frequency image and the second low-frequency image by the high-frequency correction module to obtain a combined image;

[0072] The combined image is corrected by the high-frequency correction module to obtain the image without color cast.

[0073] Optionally, the determining module is specifically used to:

[0074] receiving an image processing request sent by an electronic device, wherein the image processing request includes the first color-biased image;

[0075] The first color-shifted image is determined according to the image processing request.

[0076] Optionally, the determining module is specifically used to:

[0077] Taking pictures to obtain an initial image;

[0078] When it is determined that the initial image is a color-shifted image, the initial image is determined as the first color-shifted image.

[0079] Optionally, the processing module is specifically used for:

[0080] inputting the first color-biased image into the image processing model;

[0081] The image output by the image processing model is determined as the image without color deviation.

[0082] Optionally, the image processing device further includes a display module or a sending module, wherein:

[0083] The display module is used to display the image without color deviation; or

[0084] The sending module is used to send the image without color cast to an electronic device.

[0085] The present application also provides an intelligent terminal, comprising: a memory and a processor, wherein the memory stores an image processing program, and when the image processing program is executed by the processor, the steps of any of the above-mentioned image processing methods are implemented.

[0086] The present application also provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned image processing methods are implemented.

[0087] The present application also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the image processing method as described above is implemented.

[0088] As described above, the image processing method of the present application can be applied to smart terminals, including step S1: obtaining or determining the first color-biased image to be processed, and step S2: processing the first color-biased image through an image processing model to obtain an image without color bias. The attention mechanism in the image processing model can perform weighted processing on the first low-frequency image corresponding to the color-biased image, and then accurately correct the first low-frequency image through the low-frequency correction module, and finally correct the weighted first low-frequency image and the high-frequency image through the high-frequency correction module to obtain an image without color bias. In the above manner, the correction effect on the color-biased image is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0090] Figure 1 A schematic diagram of the hardware structure of a smart terminal for implementing various embodiments of the present application;

[0091] Figure 2 A communication network system architecture diagram provided for an embodiment of the present application;

[0092] Figure 3 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0093] Figure 4 A flowchart of an image processing method provided in an embodiment of the present application;

[0094] Figure 5A A schematic diagram of a DCT coefficient graph provided in an embodiment of the present application;

[0095] Figure 5B A schematic diagram of a DCT image provided in an embodiment of the present application;

[0096] Figure 6 A schematic diagram of a first low-frequency image and a high-frequency image provided in an embodiment of the present application;

[0097] Figure 7 A schematic diagram of an image without color cast provided in an embodiment of the present application;

[0098] Figure 8 A flowchart of another image processing method provided in an embodiment of the present application;

[0099] Fig. 9 A schematic diagram of an image processing model provided in an embodiment of the present application;

[0100] Fig.10 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0101] Fig.11 A schematic diagram of the structure of another image processing device provided in an embodiment of the present application;

[0102] Fig.12 A schematic diagram of the structure of another image processing device provided in an embodiment of the present application.

[0103] The realization of the purpose, functional features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. The above-mentioned drawings have shown clear embodiments of this application, which will be described in more detail later. These drawings and textual descriptions are not intended to limit the scope of the concept of this application in any way, but to illustrate the concept of this application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0104] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0105] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0106] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in this article, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. Optionally, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "at the time of..." or "when..." or "in response to determination". Furthermore, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising", "including" indicate that there are described features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or at least one other feature, step, operation, element, component, project, kind, and / or group. The terms "or", "and / or", "including at least one of the following" etc. used in this application can be interpreted as inclusive, or mean any one or any combination. Alternatively, “comprising at least one of the following: A, B, C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”, for example, “A, B or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”. An exception to this definition will only occur when a combination of elements, functions, steps or operations are inherently mutually exclusive in some manner.

[0107] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are displayed in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and it can be performed in other orders. Moreover, at least a portion of the steps in the figure may include at least one sub-step or at least one stage, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0108] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0109] It should be noted that in this article, step codes such as S1 and S2 are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the order. When implementing the step, those skilled in the art may execute S2 first and then S1, etc., but these should all be within the scope of protection of this application.

[0110] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0111] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0112] The smart terminal may be implemented in various forms. Optionally, the smart terminal described in this application may include smart terminals such as mobile phones, tablet computers, laptop computers, PDAs, portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, and fixed terminals such as digital TVs and desktop computers.

[0113] The subsequent description will be illustrated by taking a smart terminal as an example, and those skilled in the art will understand that, in addition to components specifically used for mobile purposes, the construction according to the embodiments of the present application can also be applied to fixed-type terminals.

[0114] See also Figure 1 , which is a schematic diagram of the hardware structure of a smart terminal for implementing various embodiments of the present application, the smart terminal 100 may include: RF (Radio Frequency) unit 101, WiFi module 102, audio output unit 103, A / V (audio / video) input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, processor 110, and power supply 111 and other components. Those skilled in the art can understand that Figure 1 The structure of the smart terminal shown in the figure does not constitute a limitation on the smart terminal. The smart terminal may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0115] Combine the following Figure 1 A detailed introduction to each component of the smart terminal:

[0116] The radio frequency unit 101 can be used for receiving and sending signals during information transmission or communication. Specifically, after receiving the downlink information of the base station, it is sent to the processor 110 for processing; in addition, the uplink data is sent to the base station. Generally, the radio frequency unit 101 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 101 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution) and 5G, etc.

[0117] WiFi is a short-range wireless transmission technology. The smart terminal can help users send and receive emails, browse web pages, and access streaming media through the WiFi module 102. It provides users with wireless broadband Internet access. Figure 1 The WiFi module 102 is shown, but it is understandable that it is not a necessary component of the smart terminal and can be omitted as needed without changing the essence of the invention.

[0118] The audio output unit 103 can convert the audio data received by the RF unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output it as sound when the smart terminal 100 is in a call signal receiving mode, a call mode, a recording mode, a voice recognition mode, a broadcast receiving mode, etc. Moreover, the audio output unit 103 can also provide audio output related to a specific function performed by the smart terminal 100 (optionally, a call signal receiving sound, a message receiving sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0119] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processor (GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 106. The image frame processed by the graphics processor 1041 can be stored in the memory 109 (or other storage medium) or sent via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in a telephone call mode, a recording mode, a voice recognition mode, and other operating modes, and can process such sound into audio data. The processed audio (voice) data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 101 in the case of a telephone call mode. The microphone 1042 can implement various types of noise elimination (or suppression) algorithms to eliminate (or suppress) noise or interference generated in the process of receiving and sending audio signals.

[0120] The smart terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 1061 and / or the backlight when the smart terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can also be configured on the mobile phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0121] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0122] The user input unit 107 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the intelligent terminal. Optionally, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 1071 or near the touch panel 1071 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 110, and can receive and execute commands sent by the processor 110. In addition, the touch panel 1071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 1071, the user input unit 107 may further include other input devices 1072. Optionally, the other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, a function key (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, etc., which are not specifically limited here.

[0123] Optionally, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. Then, the processor 110 provides corresponding visual output on the display panel 1061 according to the type of touch event. Figure 1 In the figure, the touch panel 1071 and the display panel 1061 are used as two independent components to realize the input and output functions of the smart terminal. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to realize the input and output functions of the smart terminal, which is not limited here.

[0124] The interface unit 108 is used as an interface through which at least one external device can be connected to the smart terminal 100. Optionally, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 108 may be used to receive input (optionally, data information, power, etc.) from an external device and transmit the received input to one or at least one element in the smart terminal 100 or may be used to transmit data between the smart terminal 100 and an external device.

[0125] The memory 109 can be used to store software programs and various data. The memory 109 can mainly include a program storage area and a data storage area. Optionally, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 109 can include a high-speed random access memory, and can also include a non-volatile memory, optionally at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0126] The processor 110 is the control center of the smart terminal. It uses various interfaces and lines to connect various parts of the entire smart terminal. It executes various functions of the smart terminal and processes data by running or executing software programs and / or modules stored in the memory 109, and calling data stored in the memory 109, so as to monitor the smart terminal as a whole. The processor 110 may include one or at least one processing unit; preferably, the processor 110 may integrate an application processor and a modem processor. Optionally, the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 110.

[0127] The intelligent terminal 100 may also include a power supply 111 (such as a battery) for supplying power to various components. Preferably, the power supply 111 may be logically connected to the processor 110 through a power management system, thereby managing functions such as charging, discharging, and power consumption through the power management system.

[0128] although Figure 1 Not shown, the smart terminal 100 may also include a Bluetooth module, etc., which will not be described in detail here.

[0129] To facilitate understanding of the embodiments of the present application, the communication network system on which the smart terminal of the present application is based is described below.

[0130] See also Figure 2 , Figure 2 A communication network system architecture diagram is provided for an embodiment of the present application, wherein the communication network system is an LTE system of universal mobile communication technology, and the LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203 and an operator's IP service 204, which are sequentially connected for communication.

[0131] Optionally, UE201 may be the above-mentioned terminal 100, which will not be described in detail here.

[0132] E-UTRAN 202 includes eNodeB 2021 and other eNodeBs 2022 , etc. Optionally, eNodeB 2021 may be connected to other eNodeBs 2022 via a backhaul (optionally an X2 interface), and eNodeB 2021 is connected to EPC 203 , and eNodeB 2021 may provide UE 201 with access to EPC 203 .

[0133] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gate Way) 2034, PGW (PDN Gate Way) 2035 and PCRF (Policy and Charging Rules Function) 2036, etc. Optionally, MME 2031 is a control node that processes signaling between UE 201 and EPC 203, and provides bearer and connection management. HSS 2032 is used to provide some registers to manage functions such as home location register (not shown in the figure), and store some user-specific information such as service features and data rates. All user data can be sent through SGW2034, PGW2035 can provide IP address allocation and other functions for UE 201, PCRF2036 is the policy and charging control policy decision point for service data flow and IP bearer resources, which selects and provides available policy and charging control decisions for the policy and charging execution function unit (not shown in the figure).

[0134] The IP service 204 may include the Internet, an intranet, an IMS (IP Multimedia Subsystem) or other IP services.

[0135] Although the above introduction takes the LTE system as an example, those skilled in the art should know that the present application is not only applicable to the LTE system, but also to other wireless communication systems, optionally GSM, CDMA2000, WCDMA, TD-SCDMA and future new network systems (such as 5G), etc., which are not limited here.

[0136] Based on the above-mentioned electronic device hardware structure and communication network system, various embodiments of the present application are proposed.

[0137] In some implementations, during the process of correcting a color-shifted image, the entire region of the color-shifted image is usually corrected to obtain an image without color shift. However, during the above process, the degree of color shift in different regions of the color-shifted image may be different, which makes it difficult to obtain the best correction effect for different regions of the color-shifted image, thereby resulting in a poor correction effect on the color-shifted image.

[0138] In order to solve the above technical problems, the embodiments of the present application provide an image processing method, an intelligent terminal and a storage medium, wherein the image processing model can perform frequency separation processing on the first color-biased image to obtain a first low-frequency image and a high-frequency image corresponding to the first color-biased image, and can perform weighted processing on the first low-frequency image through an attention mechanism to increase the weight of the low-frequency information. The image processing model can perform color correction processing on the weighted first low-frequency image and the high-frequency image to obtain a color-free image. In this way, since the image processing model can obtain the first low-frequency image through frequency separation processing, and weight the first low-frequency image through an attention mechanism, the first low-frequency image receives more attention, and thus the image processing model can accurately correct the first low-frequency image corresponding to the color-biased image, thereby improving the correction effect on the color-biased image.

[0139] Next, combine Figure 3 , the application scenarios of the embodiments of the present application are introduced.

[0140] Figure 3 A schematic diagram of an application scenario provided by an embodiment of the present application. Figure 3 The first color-biased image can be input into the image processing model, and the first color-biased image can be processed to restore the true color information of the first color-biased image to obtain an image without color-bias.

[0141] In the process of processing the first color-shifted image, the image processing model can perform frequency separation processing on the first color-shifted image to obtain a first low-frequency image and a high-frequency image corresponding to the first color-shifted image, and can process the first low-frequency image through a low-frequency correction module to obtain a second low-frequency image, and then process the second low-frequency image and the high-frequency image through a high-frequency correction module to obtain a non-color-shifted image. In this way, the image processing model can accurately correct the low-frequency area in the color-shifted image, thereby improving the correction effect of the color-shifted image.

[0142] The technical solutions shown in the present application are described in detail through specific embodiments. Optionally, the following embodiments may exist alone or in combination with each other, and the same or similar contents will not be described repeatedly in different embodiments.

[0143] Figure 4This is a flowchart of an image processing method provided in an embodiment of the present application. Figure 4 , the method may include:

[0144] S1. Acquire or determine a first color-shifted image to be processed.

[0145] The execution subject of the embodiment of the present application may be a smart terminal, or an image processing device set in the smart terminal. Optionally, the image processing device may be implemented by software, or by a combination of software and hardware.

[0146] Optionally, acquiring or determining the first color-shifted image may include the following two situations:

[0147] Case 1: An image processing request sent by an electronic device is received, where the image processing request includes a first color-shifted image.

[0148] In this case, the intelligent terminal may acquire or determine the first color-shifted image according to the image processing request.

[0149] Optionally, the smart terminal may receive an image processing request sent by the electronic device through a wireless network or a wired network, and the image processing request includes the first color-shifted image. The smart terminal may determine the first color-shifted image in the image request.

[0150] Case 2: The initial image is captured.

[0151] In this case, the smart terminal may determine whether the initial image is a color-shifted image. If so, the initial image may be determined as the first color-shifted image.

[0152] Optionally, the smart terminal may be a terminal device equipped with a camera. If the initial image captured by the smart terminal is a color-shifted image, the initial image may be determined as the first color-shifted image.

[0153] Optionally, the size of the first color-shifted image may be expressed as C×H×W, where C represents the number of channels of the image, H represents the height of the image, which may be represented by the number of pixels of the image in the vertical dimension, and W represents the width of the image, which may be represented by the number of pixels of the image in the horizontal dimension.

[0154] Optionally, the size of the first color-shifted image may be 3×128×128, indicating that the first color-shifted image has three channels of red (Red, R), green (Green, G), and blue (Blue, B), a height of 128 pixels (pixel, px), and a width of 128 px.

[0155] It should be noted that the degree of color shift in different areas of the first color shifted image may be different.

[0156] S2. Process the first color-biased image using an image processing model to obtain an image without color-bias.

[0157] The first color-shifted image may include a low-frequency region and a high-frequency region, wherein the low-frequency region may be a region where the color changes slowly, usually a flat region, and the high-frequency region may be a region where the color changes dramatically, usually an edge, a texture, or the like.

[0158] Optionally, an image processing model may be installed in the smart terminal, and the smart terminal may input the first color-biased image into the image processing model. The image processing model may process the first color-biased image and output the image. The smart terminal may determine the image output by the image processing model as a non-color-biased image.

[0159] Optionally, the first color-shifted image may be processed by the following feasible implementation method: performing frequency separation processing on the first color-shifted image through a frequency separation module to obtain a first low-frequency image and a high-frequency image corresponding to the first color-shifted image; and processing the first low-frequency image and the high-frequency image to obtain an image without color shift.

[0160] The frequency separation module can separate the low-frequency information and the high-frequency information in the first color-shifted image based on Discrete Cosine Transform (DCT). Optionally, the image processing model can include a frequency separation module.

[0161] The first low-frequency image may be an image composed of low-frequency information separated from the first color-shifted image. Alternatively, the first low-frequency image may be a partial image of the first color-shifted image whose color changes slowly.

[0162] The high-frequency image may be an image composed of high-frequency information separated from the first color-shifted image. Alternatively, the high-frequency image may be a partial image of the first color-shifted image with a drastic color change.

[0163] Optionally, the first color-shifted image may be subjected to frequency separation processing by the following feasible implementation method: determining at least one image block of the first color-shifted image by a frequency separation module; determining a DCT image corresponding to each image block by a frequency separation module to obtain at least one DCT image, wherein the DCT image includes at least one DCT coefficient; and determining a first low-frequency image and a high-frequency image based on the at least one DCT image.

[0164] Optionally, the frequency separation module may determine at least one image block of the first color-shifted image, and may obtain an image block set based on the at least one image block. The image block set may be Represents, where N represents the total number of image blocks in the image block set, and n=1 represents the first image block in the image block set.

[0165] Optionally, the frequency separation module may divide the first color-shifted image into 256 image blocks to form an image block set: Then the image block set may include 256 image blocks, N=256. The size of each image block may be 3×8×8.

[0166] For any image block, the frequency separation module can perform discrete cosine transform on the image block through formula (1) to obtain at least one DCT coefficient corresponding to each image block. The DCT coefficient can be expressed as express:

[0167]

[0168] In formula (1), t 1 represents the horizontal coordinate index of the pixel in the first color-shifted image block; 2 represents the ordinate index of the pixel in the first color-shifted image block; i represents the abscissa index of the product function; j represents the ordinate index of the product function; I represents the first color-shifted image, n represents the image block in the first color-shifted image; C represents the channel corresponding to the image block, and the channel corresponding to the image block may include three channels: R, G, and B.

[0169] Optionally, the frequency separation module can perform discrete cosine transform on the image block-1 according to formula (1), and obtain 64 DCT coefficients corresponding to the image-1.

[0170] Optionally, at least one DCT coefficient corresponding to each image block can be sorted in order of frequency from low to high according to a Z-shaped reading method to obtain a corresponding sorting result; the DCT coefficient can be visualized to obtain a corresponding DCT image.

[0171] Next, combine Figure 5A-Figure 5B , the DCT coefficient graph and the DCT image are explained.

[0172] Figure 5A This is a schematic diagram of a DCT coefficient diagram provided in an embodiment of the present application. Figure 5A Optionally, if the DCT coefficients corresponding to the image block -1 are 64, the 64 DCT coefficients can be sorted in the order of low to high frequency according to the Z-shaped reading method, and the sorting result can be obtained as follows: Figure 5A As shown in the figure. Among the 64 DCT coefficients, the DCT coefficient in the upper left corner is the largest, indicating that the upper left corner concentrates the low-frequency information in the image block; the DCT coefficient in the lower right corner is the smallest, indicating that the lower right corner concentrates the high-frequency information in the image block. The smaller the value of the DCT coefficient, the higher the corresponding image frequency.

[0173] Figure 5B This is a schematic diagram of a DCT image provided in an embodiment of the present application. Figure 5B , Optionally, if the DCT coefficients corresponding to image block -1 are 64, and the sorting result is as follows Figure 5A As shown in , the 64 DCT coefficients can be visualized to obtain 64 DCT images, such as Figure 5B As shown. Optionally, each DCT image includes at least one DCT coefficient. Among the 64 DCT images, the DCT coefficient corresponding to the upper left corner DCT image is the largest, indicating low-frequency information in the image block; the DCT coefficient corresponding to the lower right corner DCT image is the smallest, indicating high-frequency information in the image block.

[0174] Optionally, after determining the sorting result of the DCT coefficients corresponding to each image block, the first K DCT coefficients corresponding to each image block may be determined. The first K DCT coefficients may be determined as low-frequency DCT coefficients.

[0175] For any DCT coefficient, we can perform inverse discrete cosine transform using formula (2) to obtain the low-frequency information corresponding to the DCT coefficient. The low-frequency information corresponding to the DCT coefficient can be expressed as express:

[0176]

[0177] In formula (2), t 1 , t 2 =zig -1 (t). t 1 Indicates the horizontal coordinate index of the DCT coefficient in the DCT image; t 2 It represents the ordinate index of the DCT coefficient in the DCT image; t represents the order of the DCT coefficient in the DCT image, and the value range of t is 0 to K; i represents the abscissa index of the product function; j represents the ordinate index of the product function; C represents the channel corresponding to the image block, and the channel corresponding to the image block can include three channels: R, G, and B.

[0178] Optionally, if K is set to 10, the first 10 DCT coefficients corresponding to each image block can be determined. For any image block, each of the first 10 DCT coefficients corresponding to the image block can be subjected to an inverse discrete cosine transform to obtain frequency information corresponding to the 10 DCT coefficients, thereby obtaining low-frequency information corresponding to the image block.

[0179] Optionally, since the first color-biased image may include multiple image blocks, discrete cosine transform can be performed on the multiple image blocks respectively according to formula (1) to obtain corresponding multiple DCT coefficients, and discrete cosine inverse transform can be performed on the multiple DCT coefficients respectively according to formula (2), so that low-frequency information in the multiple image blocks can be determined respectively.

[0180] Optionally, the low-frequency information in the plurality of image blocks is reorganized to obtain the low-frequency information I of the first color-shifted image. low Then the image composed of low-frequency information can be determined as the first low-frequency image, and the first low-frequency image can be represented by I low express.

[0181] Optionally, the low-frequency information can be removed from the first color-shifted image to obtain the high-frequency information. That is, the high-frequency information I in the first color-shifted image can be calculated by formula (3): high :

[0182] I high =II low Formula (3)

[0183] Wherein, I represents the overall image information of the first color-shifted image.

[0184] Determine high frequency information I high Afterwards, the image composed of high-frequency information can be determined as a high-frequency image, and the high-frequency image can be used high express.

[0185] Next, combine Figure 6 The first low-frequency image and the high-frequency image in the first color-shifted image are described.

[0186] Figure 6 Schematic diagram of a first low-frequency image and a high-frequency image provided in an embodiment of the present application. Figure 6 , including image 601, image 602 and image 603.

[0187] Optionally, if the first color-biased image is as shown in image 601, the frequency separation module can determine multiple image blocks in the first color-biased image through discrete cosine transform and inverse discrete cosine transform, and determine low-frequency information corresponding to the multiple image blocks, thereby determining a first low-frequency image. As shown in image 602, the first low-frequency image may be an image corresponding to a relatively flat area in the first color-biased image.

[0188] Furthermore, the image information of the first low-frequency image may be removed from the first color-shifted image to determine a high-frequency image. As shown in image 603, the high-frequency image may be an image corresponding to regions such as image boundaries and textures in the first color-shifted image.

[0189] Optionally, the image processing model may include a low-frequency correction module and a high-frequency correction module. Optionally, the low-frequency correction module may perform correction processing on the first low-frequency image; and the high-frequency correction module may perform correction processing on the high-frequency image.

[0190] Optionally, the first low-frequency image and the high-frequency image can be processed to obtain an image without color deviation by the following feasible implementation method: the first low-frequency image can be processed by a low-frequency correction module to obtain a second low-frequency image; the high-frequency image and the second low-frequency image can be added together by a high-frequency correction module to obtain a merged image, and the merged image can be corrected to obtain an image without color deviation.

[0191] Optionally, the low-frequency correction module can perform correction processing on the first low-frequency image to obtain a second low-frequency image. The second low-frequency image can be represented by I′ low express.

[0192] Optionally, the high-frequency image and the second low-frequency image can be added together by the high-frequency correction module to obtain a merged image, that is, the merged image can be calculated by formula (4), and the merged image can be expressed as com express:

[0193] I com =I high +I′ low Formula (4)

[0194] In formula (4), I′ low Represents the second low-frequency image, I high Represents a high-frequency image.

[0195] Get the merged image I com Afterwards, the combined image may be corrected by a high frequency correction module to obtain an image without color deviation, which may be represented by I′.

[0196] Next, combine Figure 7 An image without color cast will be described.

[0197] Figure 7 This is a schematic diagram of a color-shift-free image provided in an embodiment of the present application. Figure 7 , including image 701, image 702, image 703 and image 704.

[0198] See also Figure 7If the first low-frequency image is as shown in image 701, the first low-frequency image can be subjected to low-frequency correction processing to obtain a second low-frequency image, as shown in image 702. If the high-frequency image is as shown in image 703, the second low-frequency image and the high-frequency image can be subjected to high-frequency correction processing, and high-frequency correction processing can be performed to obtain a color-free image, as shown in image 704.

[0199] Optionally, after obtaining the image without color cast, the intelligent terminal may display the image without color cast; or send the image without color cast to an electronic device.

[0200] Optionally, if the smart terminal is a computer, the smart terminal may directly display the image without color cast; if the smart terminal is a server, the image without color cast may be sent to the electronic device.

[0201] The embodiment of the present application provides an image method, and the intelligent terminal can obtain or determine a first color-biased image, and perform frequency separation processing on the first color-biased image to obtain a first low-frequency image and a high-frequency image. The image processing model may include a low-frequency correction module and a high-frequency correction module. The first low-frequency image can be processed by the low-frequency correction module to obtain a second low-frequency image, and then the second low-frequency image and the high-frequency image are added together by the high-frequency correction module to obtain a merged image. The high-frequency correction module can process the merged image to obtain an image without color deviation. Since the low-frequency information in the color-biased image can be accurately corrected by the low-frequency correction module in the image processing model, the correction effect of the color-biased image is improved.

[0202] exist Figure 4 Based on the embodiment shown below, combined with Figure 8 , the above image processing method is further explained.

[0203] Figure 8 This is a flowchart of another image processing method provided in an embodiment of the present application. Figure 8 , the method may include:

[0204] S801: Acquire or determine a first color-shifted image to be processed.

[0205] It should be noted that the execution process of step S801 can refer to step S1, which will not be repeated here.

[0206] S802: Determine a first low-frequency image and a high-frequency image corresponding to the first color-shifted image.

[0207] Optionally, at least one image block in the first color-shifted image can be determined by a frequency separation module, and each image block can be subjected to a discrete cosine transform to obtain at least one DCT coefficient corresponding to each image block; at least one DCT coefficient corresponding to each image block can be sorted in order of frequency from low to high according to a Z-shaped reading method to obtain a corresponding sorting result, and then the first K DCT coefficients corresponding to each image block can be determined. The first K DCT coefficients corresponding to each image block can be subjected to an inverse discrete cosine transform to obtain low-frequency information corresponding to each image block, so that the low-frequency information in the first color-shifted image can be extracted to determine the first low-frequency image.

[0208] Optionally, low-frequency information may be removed from the first color-shifted image to obtain high-frequency information to form a high-frequency image.

[0209] Optionally, if the size of the first color-shifted image is 3×128×128, after frequency separation, the corresponding first low-frequency image I can be obtained. low and high frequency image I high Optionally, the first low-frequency image I low and high frequency image I high The size can be 3×128×128.

[0210] S803 . Perform at least one feature extraction process on the first low-frequency image through an encoder module to obtain a preliminary feature image, M first low-frequency feature images, and a second low-frequency feature image.

[0211] Optionally, the low-frequency correction module may include an encoder module, an attention mechanism module and a decoder module.

[0212] The encoder module may include a dual convolution module, a first downsampling module, a second downsampling module, a third downsampling module and a connection module. The encoder module may perform at least one feature extraction process on the first low-frequency image to obtain a plurality of feature maps.

[0213] Optionally, the first low-frequency image I low Input into the encoder module, the encoder module can use a double convolution module to transform the first low-frequency image I low Perform feature extraction to obtain a preliminary feature image, which can be recorded as F ori .

[0214] Optionally, if the first low-frequency image I low The size of is 3×128×128, then after processing by the double convolution module, the preliminary feature image F can be extracted ori , F ori The size of can be 24×128×128, which means the preliminary feature image F oriThe number of channels is 24, the height is 128px, and the width is 128px.

[0215] The first downsampling module, the second downsampling module, and the third downsampling module can compress the feature map to obtain corresponding thumbnails.

[0216] Optionally, the preliminary feature image F ori Input it into the first downsampling module for feature extraction processing to obtain the first low-frequency feature image, which can be recorded as F 1 ; The first low-frequency feature image F 1 Input it into the second downsampling module for feature extraction processing to obtain the second first low-frequency feature image, which can be recorded as F 2 ; The second first low-frequency feature image F 2 Input it into the third downsampling module for feature extraction processing to obtain the third first low-frequency feature image, which can be recorded as F 3 . Then there are 3 first low-frequency feature images in total, and M is 3.

[0217] Optionally, if the preliminary feature image F ori The size of is 24×128×128, then after the first downsampling module performs feature extraction processing, the first low-frequency feature image F can be obtained. 1 , F 1 The size can be 48×64×64, which means F 1 The number of channels is 48, the height is 64px, and the width is 64px. 1 The size is smaller than F ori Then, the second down-sampling module can downsample the first low-frequency feature image F 1 Perform feature extraction processing to obtain the second first low-frequency feature image F 2 , F 2 The size can be 96×32×32, which means F 2 The number of channels is 96, the height is 32px, and the width is 32px. 2 The size is smaller than F 1 Further, the third down-sampling module can be used to downsample the second first low-frequency feature image F 2 Perform feature extraction processing to obtain the third first low-frequency feature image F 3 , F 3 The size can be 192×16×16, which means F 3 The number of channels is 192, the height is 16px, the width is 16px, and the F 3 The size is smaller than F 2 size.

[0218] Optionally, the third first low-frequency feature image F 3 Input it into the connection module for feature extraction processing to obtain the second low-frequency feature image, which can be recorded as F 4 The second low-frequency feature image F 4 The image size may be smaller than the size of the first low-frequency image.

[0219] Optionally, if the third first low-frequency feature image F 3 The size of is 192×16×16, then after feature extraction processing by the connection module, the second low-frequency feature image F can be obtained. 4 , F 4 The size can be 384×8×8, which means F 4 The number of channels is 384, the height is 8px, the width is 8px, and F 4 The size is smaller than F 3 size.

[0220] It should be noted that the preliminary feature image F ori , the first low-frequency feature image F 1 , the second first low-frequency feature image F 2 , the third first low-frequency feature image F 3 and the second low-frequency feature image F 4 The features in are features of different dimensions from low to high. Optionally, the preliminary feature image F ori The dimension is the lowest, the second lowest frequency feature image F 4 The highest dimension.

[0221] S804: Process the second low-frequency feature image through an attention mechanism module to obtain a weighted feature image.

[0222] The attention mechanism module can perform weighted processing on low-frequency information to improve the expression ability of low-frequency information.

[0223] Optionally, a weighted feature image can be obtained through the following feasible implementation method: feature extraction processing is performed on the second low-frequency feature image through the attention mechanism module to obtain an intermediate feature map; the DCT coefficient feature vector, maximum pooling feature vector and average pooling feature vector corresponding to the intermediate feature map are determined through the attention mechanism module; the DCT coefficient feature vector, maximum pooling feature vector and average pooling feature vector are processed through the attention mechanism module to obtain a weighted feature vector; the weighted feature vector and the second low-frequency feature image are weightedly processed through the attention mechanism module to obtain a weighted feature image.

[0224] Optionally, the attention mechanism module can perform a convolution operation on the second low-frequency feature image F 4Perform feature extraction to obtain the intermediate feature map F′ 4 .

[0225] F′ 4 =F 0 (F 4 ) Formula (5)

[0226] Among them, F0(·) represents the strided convolution.

[0227] Optionally, if the second low-frequency feature image F 4 The size of is 384×8×8, then the attention mechanism module can use formula (5) to calculate the second low-frequency feature image F 4 Perform convolution operation to obtain the intermediate feature map F′ 4 , the intermediate feature map F′ 4 The size can still be 384×8×8, indicating that F′ 4 The number of channels is 384, the height is 8px and the width is 8px.

[0228] Optionally, the DCT coefficient feature vector can be obtained by the following feasible implementation method: the intermediate feature map F′ can be determined 4 In the feature map of each channel; the coordinates to be extracted can be determined in the feature map of each channel to calculate the DCT coefficients to be extracted, and then determine the intermediate feature map F′ 4 The corresponding DCT coefficient feature vector.

[0229] Optionally, the DCT coefficients to be extracted are related to the intermediate feature map F′ 4 The characteristics of each channel Figure 1 One to one correspondence.

[0230] Optionally, if the intermediate feature map F′ 4 The size of the intermediate feature map F′ is 384×8×8. 4 There are 384 channels, each channel has a corresponding feature map, the size of the feature map of each channel is 8×8, which can correspond to a DCT coefficient to be extracted, then the intermediate feature map F′ 4 This may correspond to 384 DCT coefficients to be extracted.

[0231] Optionally, for the feature map of any channel, if the coordinates of the DCT coefficient to be extracted are (u, v), the DCT coefficient F(u, v) corresponding to (u, v) can be calculated by formula (6), then:

[0232]

[0233]

[0234] In formula (6), u represents the horizontal coordinate of the DCT coefficient to be extracted in the feature map of the channel; v represents the vertical coordinate of the DCT coefficient to be extracted in the feature map of the channel; C represents different channels; N represents the intermediate feature map F′ 4 The size of N = 8; x represents the coefficient in the intermediate feature map F′ 4 The horizontal coordinate index of the pixel in the middle, y represents the coefficient in the middle feature map F′ 4 The pixel ordinate index.

[0235] Optionally, the obtained DCT coefficients can be obtained by using the DCT coefficient feature vector F DCT Expression, DCT coefficient feature vector F DCT It can be a C×1 dimensional vector. Where C represents the intermediate feature map F′ 4 The number of channels.

[0236] Optionally, if the intermediate feature map F′ 4 There are 384 channels, corresponding to 384 DCT coefficients to be extracted. If the coordinates of the DCT coefficients to be extracted in the feature map of each channel are (3, 2), the corresponding 384 DCT coefficients can be calculated by formula (6), and the 384 DCT coefficients can form a DCT coefficient feature vector. Then the DCT coefficient feature vector F DCT It can be a 384×1 dimensional vector.

[0237] Optionally, the DCT coefficient feature vector F DCT Can be:

[0238]

[0239] Optionally, the intermediate feature map F′ can be calculated in the spatial direction by formula (7) 4 Perform global maximum pooling to obtain the corresponding maximum pooling feature vector.

[0240] F max =Maxpool(F′ 4 ) Formula (7)

[0241] The maximum pooling feature vector can be a C×1 dimensional vector. Where C represents the intermediate feature map F′ 4 The number of channels.

[0242] Optionally, if the intermediate feature map F′ 4 The size of is 384×8×8, then the corresponding maximum pooling feature vector can be calculated by formula (7), and the corresponding maximum pooling feature vector can be a 384×1-dimensional vector.

[0243] Optionally, the maximum pooling feature vector F maxCan be:

[0244]

[0245] Optionally, the intermediate feature map F′ can be calculated in the spatial direction by formula (8) 4 Perform global average pooling to obtain the corresponding average pooling feature vector.

[0246] F avg =Avgpool(F′ 4 ) Formula (8)

[0247] The average pooled feature vector can be a C×1 dimensional vector. Where C represents the intermediate feature map F′ 4 The number of channels.

[0248] Optionally, if the intermediate feature map F′ 4 The size of is 384×8×8, then the corresponding average pooling feature vector can be calculated by formula (8), and the corresponding average pooling feature vector can be a 384×1-dimensional vector.

[0249] Optionally, the average pooled feature vector F avg Can be:

[0250]

[0251] Optionally, the weight feature vector can be obtained through the following feasible implementation method: the DCT coefficient feature vector, the maximum pooling feature vector and the average pooling feature vector are combined and processed through the attention mechanism module to obtain a first feature matrix; the first feature matrix is ​​convolved through the attention mechanism module to obtain a second feature matrix; the elements in the second feature matrix are added row by row through the attention mechanism module to obtain a first feature vector; the first feature vector is normalized through the attention mechanism module to obtain a weight feature vector.

[0252] Since the DCT coefficient feature vector, the maximum pooling feature vector and the average pooling feature vector are all C×1-dimensional vectors, the three C×1-dimensional vectors can be combined according to the following formula (9) to obtain the first feature matrix Q, which can be a C×3-dimensional matrix.

[0253]

[0254] Among them, F DCT represents the DCT coefficient feature vector, F max represents the maximum pooling feature vector, F avg represents the average pooling feature vector, Q 0，0 , Q 0，1 ,......,QC，2 Represents the elements in the first characteristic matrix.

[0255] Optionally, the DCT coefficient feature vector, the maximum pooling feature vector and the average pooling feature vector exemplified above may be combined and processed, and the obtained first feature matrix Q may be:

[0256]

[0257] Optionally, the first feature matrix Q may be input into a dual convolution module for convolution processing to obtain a second feature matrix Q′, and the second feature matrix Q′ may be a C×3-dimensional matrix.

[0258] Optionally, if the first characteristic matrix Q in the above example is subjected to convolution processing, the second characteristic matrix Q′ can be obtained as follows:

[0259]

[0260] Optionally, the elements in the second characteristic matrix Q′ may be processed by row addition according to formula (10) to obtain a first characteristic vector VectorN, which may be a C×1-dimensional vector.

[0261]

[0262] In formula (10), x represents a row in the second characteristic matrix, and j represents a column in the second characteristic matrix.

[0263] Optionally, if the second characteristic matrix is ​​as shown in the above example, the elements in the second characteristic matrix are added row by row by formula (10), and the obtained first characteristic vector VectorN can be:

[0264]

[0265] Optionally, the first eigenvector VectorN may be normalized using formula (11) to obtain a weighted eigenvector VectorAtt.

[0266]

[0267] In formula (11), x represents the eigenvalue in the first eigenvector; S(·) represents a normalization function; and e is the natural logarithm.

[0268] The weight feature vector VectorAtt can be a C×1-dimensional vector, which can be expressed as:

[0269]

[0270] Among them, X 1 , X2 ,......X C are the eigenvalues ​​in the weight feature vector VectorAtt respectively.

[0271] Optionally, if the first eigenvector is as shown in the above example, the elements in the first eigenvector are normalized by formula (11), and the obtained weighted eigenvector VectorAtt can be:

[0272]

[0273] Optionally, after determining the weight feature vector, the attention mechanism module can perform weighted processing on the weight feature vector and the second low-frequency feature image by formula (12) to obtain a weighted feature image corresponding to each channel. The weighted feature image can be expressed as att-i express:

[0274] F att-i =X i ×I i Formula (12)

[0275] In formula (12), X i is the i-th eigenvalue in the attention weight vector VectorAtt, I i is the second low-frequency feature image F 4 The H×W-dimensional feature image on the i-th channel in F. att-i is the new H×W-dimensional weighted feature image on the i-th channel.

[0276] Since the weight vector VectorAtt has a total of C eigenvalues, the second low-frequency feature image F 4 There are C channels, so there are a total of C new H×W dimensional weighted feature images F att-i , then according to the H×W-dimensional weighted feature image F on each channel att-i , we can get a weighted feature image of C×H×W dimensions, which can be recorded as F att .

[0277] Optionally, if the second low-frequency feature image F 4 The size of the second low-frequency feature image F is 384×8×8. 4 There are 384 channels, and the corresponding weight vector VectorAtt has a total of 384 eigenvalues. Then the eigenvalues ​​in the weight vector VectorAtt and the second low-frequency feature image F can be expressed as follows: 4 Corresponding channel features Figure 1 Multiply them by one to perform weighted processing and obtain the weighted feature image F att , then the weighted feature image F attThe dimensions are 384×8×8.

[0278] S805 . Process the M first low-frequency feature images, the weighted feature image, and the preliminary feature image through a decoder module to obtain a second low-frequency image.

[0279] Optionally, the decoder module may include a pooling module, M upsampling modules and a dual convolution module. Optionally, the pooling module may include a convolutional layer (conv layer), a linear rectified function layer (RectifiedLinear Units layer'Relu layer) and a deconvolution layer (ContTranspose2d layer); the upsampling module may include two convolutional layers, two linear rectified function layers and a deconvolution layer; the dual convolution module may include two convolutional layers and two linear rectified function layers.

[0280] Optionally, the second low-frequency image can be obtained in the following feasible manner: performing feature extraction processing on the weighted feature image through a pooling module to obtain an initial output image; processing the initial output image and M first low-frequency feature images through an upsampling module to obtain M output images; processing the Mth output image and the preliminary feature image through a double convolution module to obtain a second low-frequency image.

[0281] Optionally, the weighted feature image F att Input to the decoder module, the decoder module can use the pooling module to weight the feature image F att Perform feature extraction to obtain the initial output image F′ att .

[0282] Optionally, if the weighted feature image F att The size of is 384×8×8, then the weighted feature image F is pooled by the pooling module. att After correction, the initial output image can be obtained, which can be recorded as F′ att . F′ att The size can be 192×16×16, which means F′ att The number of channels is 192, the height is 16px, and the width is 16px.

[0283] If M is 3, the decoder module includes three upsampling modules, which can be respectively recorded as a first upsampling module, a second upsampling module and a third upsampling module. The three upsampling modules can enlarge the feature map to obtain a corresponding enlarged map.

[0284] Optionally, the initial output image F′ can be att and the third first low-frequency feature image F 3Input to the first upsampling module for correction processing to obtain the corresponding first output image, which can be recorded as F′ 3 ; Then the first output image F′ can be 3 and the second first low-frequency feature image F 2 Input to the second upsampling module for correction processing to obtain the corresponding second output image, which can be recorded as F′ 2 ; Then the second output image F′ can be 2 and the first low-frequency feature image F 1 Input to the third upsampling module for correction processing to obtain the corresponding third output image, which can be recorded as F′ 1 .

[0285] Optionally, if the initial output image F′ att The size of the image is 192×16×16, and the third first low-frequency feature image F 3 The size of is 192×16×16, and after the first upsampling module performs correction processing, the first output image F′ can be obtained. 3 , F′ 3 The size can be 96×32×32, which means F′ 3 The number of channels is 96, the height is 32px, the width is 32px, and F′ 3 The size is larger than F′ att size.

[0286] Next, if the second first low-frequency feature image F 2 The size of the image is 96×32×32, then the second first low-frequency feature image F 2 and the first output image F′ 3 Input to the second upsampling module for correction processing to obtain the second output image F′ 2 , F′ 2 The size can be 48×64×64, indicating that F′ 2 The number of channels is 48, the height is 64px, the width is 64px, and F′ 2 The size is larger than F′ 3 size.

[0287] Furthermore, if the first low-frequency feature image F 1 The size of F is 48×64×64, then the first low-frequency feature image F 1 and the second output image F′ 2 Input to the third upsampling module for correction processing to obtain the third output image F′ 1 , F′ 1 The size of can be 24×128×128, which means F′ 1The number of channels is 24, the height is 128px, the width is 128px, and F′ 1 The size is larger than F′ 2 size.

[0288] Optionally, the third output image F′ can be 1 and the preliminary feature image F ori Input into the double convolution module for correction processing to obtain the corresponding second low-frequency image I′ low .

[0289] Optionally, if the preliminary feature image F ori The size of the third output image F′ is 24×128×128. 1 The size of can be 24×128×128, then the third output image F′ can be 1 and the preliminary feature image F ori Input into the double convolution module for correction processing to obtain the corresponding second low-frequency image I′ low , I′ low The size can be 3×128×128, which means I′ low The number of channels is 3, the height is 128px, and the width is 128px.

[0290] S806 , performing high-frequency correction processing on the high-frequency image and the second low-frequency image by a high-frequency correction module to obtain an image without color cast.

[0291] Optionally, obtain a second low-frequency image I′ low Afterwards, the high-frequency image I can be corrected by the high-frequency correction module high and the second low-frequency image I′ low Perform addition processing to obtain the merged image I com , and then the merged image can be corrected by the high-frequency correction module to obtain an image without color cast.

[0292] Optionally, if the second low-frequency image I′ low The size of the high-frequency image I is 3×128×128. high The size of the image is 3×128×128, and the high-frequency image I can be corrected by the high-frequency correction module. high and the second low-frequency image I′ low Perform addition processing to obtain the merged image I com , merge image I com The size of I can be 3×128×128, and the high-frequency correction module is used to correct the merged image I com After the correction process, a color-shift-free image I′ is obtained. The size of the color-shift-free image I′ may be 3×128×128, which is the same as the size of the first color-shifted image.

[0293] The embodiment of the present application provides an image method, and the intelligent terminal can obtain or determine a first color-biased image, and perform frequency separation processing on the first color-biased image to obtain a first low-frequency image and a high-frequency image. The first low-frequency image can be feature extracted by the encoder module in the low-frequency correction module to obtain multiple feature images corresponding to the first low-frequency image; and the image information in the first low-frequency image can be weighted by the attention mechanism module in the low-frequency correction module to obtain the corresponding weighted feature image; and then the weighted feature image and the multiple feature maps corresponding to the first low-frequency image can be corrected by the decoder module in the low-frequency correction module to obtain a second low-frequency image. The second low-frequency image and the high-frequency image can be added by the high-frequency correction module to obtain a merged image, and then the merged image is corrected to obtain a color-free image. Since the low-frequency information in the color-biased image can be weighted by the attention mechanism module to increase the weight of the low-frequency information, and then the low-frequency information in the first color-biased image can be accurately corrected by the low-frequency correction module, the correction effect of the color-biased image is improved.

[0294] Next, combine Fig. 9 , the image processing model is explained.

[0295] Fig. 9 This is a schematic diagram of an image processing model provided in an embodiment of the present application. Fig. 9 ,The image processing model can include a frequency separation module, a low frequency correction module and a high frequency correction module.

[0296] The first color-shifted image I can be input into the image processing model, and the frequency separation module can perform frequency separation processing on the first color-shifted image I to obtain low-frequency information I in the first color-shifted image I. low and high frequency information I high . Low frequency information I low The first low-frequency image I can be composed low , high frequency information I high Can form high-frequency image I high .

[0297] The low-frequency correction module may include an encoder module, an attention mechanism module, and a decoder module. Fig. 9 , the encoder module can include a dual convolution module, 3 downsampling modules and a connection module.

[0298] The dual convolution module may include two combination layers, and one combination layer may be composed of a 3×3 convolution layer conv and a linear rectifier function layer relu. Optionally, the step size of the convolution layer may be set to 1, the size of the convolution kernel may be 3×3, and the output channels of the two combination layers may both be 24.

[0299] The first downsampling module may include a combination layer consisting of a maximum pooling layer (Maxpooling), two 3×3 convolutional layers conv and a linear rectifier function layer relu. Optionally, the step size of the maximum pooling layer can be set to 2, and the output channels can be 24; the step size of the convolution layer can be set to 1, and the size of the convolution kernel can be 3×3; the output channels of the two combination layers can both be 48.

[0300] The second downsampling module may include a combination layer consisting of a maximum pooling layer, two 3×3 convolutional layers conV and a linear rectifier layer relu. Optionally, the step size of the maximum pooling layer can be set to 2, and the output channels can be 48; the step size of the convolutional layer can be set to 1, and the size of the convolution kernel can be 3×3; the output channels of the two combination layers can both be 96.

[0301] The third downsampling module may include a combination layer consisting of a maximum pooling layer, two 3×3 convolutional layers conv (with a step size of 1) and a linear rectifier function layer relu. Optionally, the step size of the maximum pooling layer can be set to 2, and the output channels can be 96; the step size of the convolution layer can be set to 1, and the size of the convolution kernel can be 3×3; the output channels of the two combination layers can both be 192.

[0302] The connection module may include a combination layer consisting of a maximum pooling layer, a 3×3 convolution layer conv and a linear rectifier function layer relu. Optionally, the stride of the maximum pooling layer can be set to 2, and the output channels can be 192; the stride of the convolution layer can be set to 1, and the size of the convolution kernel can be 3×3; the output channels of the combination layer can be 384.

[0303] Alternatively, if Fig. 9 , the first low-frequency image I low Input to the encoder, the first low-frequency image I can be processed by a double convolution module low Perform feature extraction to obtain the preliminary feature image F ori ; Then the preliminary feature image F ori Input to the first downsampling module to obtain the first low-frequency feature image F 1 ; Then the first low-frequency feature image F 1 Input to the second downsampling module to obtain the second first low-frequency feature image F 2 ; Then the second first low-frequency feature image F 2 Input to the third down-sampling module to obtain the third first low-frequency feature image F 3 ; Finally, the third first low-frequency feature image F 3 Input to the connection module to obtain the second low-frequency feature image F 4 .

[0304] The attention mechanism module can focus on the second low-frequency feature image F 4 Perform convolution processing to obtain the intermediate feature map F′ 4 , and the intermediate feature map F′ can be determined 4 The corresponding DCT coefficient feature vector, maximum pooling feature vector and average pooling feature vector can then be combined, convolved, added row by row, etc. to obtain a weighted feature vector. The attention mechanism module can perform weighted processing on the weighted feature vector and the second low-frequency feature image to obtain a weighted feature image F att .

[0305] The decoder module may include a pooling module, three upsampling modules, and a dual convolution module.

[0306] The pooling module may include a combination layer consisting of a deconvolution layer ContTranspose2d, a 3×3 convolution layer conv, and a linear rectifier function layer relu. Optionally, the step size of the deconvolution layer can be set to 2, and the output channel can be 192; the step size of the convolution layer can be set to 1, and the size of the convolution kernel can be 3×3; the output channel of the combination layer can be 192.

[0307] The first upsampling module may include a combination layer consisting of two 3×3 convolutional layers conv and a linear rectification function layer relu, and a deconvolution layer ContTranspose2d. Optionally, the stride of the convolution layer may be set to 1, and the size of the convolution kernel may be 3×3; the output channels of one combination layer may be 192, and the output channels of the other combination layer may be 96; the stride of the deconvolution layer may be set to 2, the size of the convolution kernel may be 2×2, and the output channels may be 96.

[0308] The second upsampling module may include a combination layer consisting of two 3×3 convolutional layers conv and a linear rectification function layer relu, and a deconvolution layer ContTranspose2d. Optionally, the step size of the convolution layer can be set to 1, and the size of the convolution kernel can be 3×3; the output channels of one combination layer can be 96, and the output channels of the other combination layer can be 48; the step size of the deconvolution layer can be set to 2, the size of the convolution kernel can be 2×2, and the output channels can be 48.

[0309] The third upsampling module may include a combination layer consisting of two 3×3 convolutional layers conv and a linear rectification function layer relu, and a deconvolution layer ContTranspose2d. Optionally, the stride of the convolution layer can be set to 1, and the size of the convolution kernel can be 3×3; the output channels of one combination layer can be 48, and the output channels of the other combination layer can be 24; the stride of the deconvolution layer can be set to 2, the size of the convolution kernel can be 2×2, and the output channels can be 24.

[0310] The dual convolution module may include a combination layer consisting of two 3×3 convolution layers conv and a linear rectifier function layer relu, and a 1×1 convolution layer conv. Optionally, the stride of the convolution layer in the combination layer can be set to 1, and the size of the convolution kernel can be 3×3; the output channels of one combination layer can be 48, and the output channels of the other combination layer can be 3; the stride of the 1×1 convolution layer can be set to 1, the size of the convolution kernel can be 1×1, and the output channels can be 3.

[0311] Alternatively, if Fig. 9 , the weighted feature map F att Input to the decoder, the weighted feature map F can be processed through the pooling module att Correction processing is performed to obtain the initial output image F′ att ; Then the initial output image F′ can be att and the third first low-frequency feature image F 3 Input to the first upsampling module for correction processing to obtain the first output image F′ 3 ; Then the first output image F′ can be 3 and the second first low-frequency feature image F 2 Input to the second upsampling module for correction processing to obtain the second output image F′ 2 ; Then the second output image F′ can be 2 and the first low-frequency feature image F 1 Input to the third upsampling module for correction processing to obtain the third output image F′ 1 ; Finally, the third output image F′ can be 1 and the preliminary feature image F ori Input to the double convolution module for correction processing to obtain the second low-frequency image I′ low .

[0312] The high frequency correction module may include a combination layer consisting of two 3×3 convolution layers conv and a linear rectification function layer relu, and a 1×1 convolution layer conv. Optionally, the step size of the convolution layer in the combination layer may be set to 1, and the size of the convolution kernel may be 3×3; the output channels of one combination layer may be 6, and the output channels of the other combination layer may be 3; the step size of the 1×1 convolution layer may be set to 1, the size of the convolution kernel may be 1×1, and the output channels may be 3.

[0313] Alternatively, if Fig. 9 , the second low-frequency image I′ low and high frequency image I high The high frequency correction module can correct the second low frequency image I' low and high frequency image I high Perform merging processing to obtain the merged image I com , and then the merged image I com Correction processing is performed to obtain an image I′ without color cast.

[0314] It should be noted that, before using the image model, the image model needs to be trained. When performing module training, the training samples may include a sample color-biased image and a sample non-color-biased image corresponding to the sample color-biased image.

[0315] Optionally, training samples can be obtained in the following feasible manner: a sample image can be obtained; a sample image can be white balanced to obtain a sample image without color deviation; the sample image without color deviation is subjected to color temperature superposition processing to obtain a sample image with color deviation; and a training sample can be determined based on the sample image without color deviation and the sample image with color deviation.

[0316] Optionally, if the sample image is an image in a raw (RAW) format, the RGB values ​​corresponding to the white points of the RAW image can be used as white balance gains R_gain, G_gain, and B_gain, and the sample image can be converted into a sample color-free image in a RAW format using the data, and then the sample color-free image in the RAW format is converted into a sample color-free image in a standard red green blue (sRGB) format as the sample color-free image in the training sample;

[0317] The sample color-biased images in RAW format can be linearly superimposed, and different color temperatures such as 2850 Kelvin (K), 3800K, 5500K, 6500K and 7500K can be superimposed to obtain sample color-biased images in different RAW formats, which are then converted into sRGB images as sample color-biased images of training samples.

[0318] Optionally, the sample non-color-biased images may be cropped and / or data-enhanced to obtain a large number of sample non-color-biased images, and the sample color-biased images may be cropped and / or data-enhanced to obtain a large number of sample color-biased images.

[0319] Optionally, the sample non-color-biased images and the sample color-biased images may be randomly cropped into 3×128×128 image blocks, and data enhancement processing may be performed, thereby obtaining a large number of sample non-color-biased images and sample color-biased images to enrich the training samples.

[0320] Optionally, the corrected image output by the image model and the sample color-bias-free image can be input into the network training supervision function to calculate the loss value, and then the model parameters of the image processing model are updated according to the loss value until the loss value is stable, and a trained image processing model can be obtained.

[0321] Optionally, the Loss value may be a mean square error, an average error, or the like.

[0322] The embodiment of the present application provides an image method, and the intelligent terminal can obtain or determine a first color-biased image, and perform frequency separation processing on the first color-biased image to obtain a first low-frequency image and a high-frequency image. The first low-frequency image can be feature extracted by the encoder module in the low-frequency correction module to obtain multiple feature images corresponding to the first low-frequency image; and the image information in the first low-frequency image can be weighted by the attention mechanism module in the low-frequency correction module to obtain the corresponding weighted feature image; and then the weighted feature image and the multiple feature maps corresponding to the first low-frequency image can be corrected by the decoder module in the low-frequency correction module to obtain a second low-frequency image. The second low-frequency image and the high-frequency image can be added by the high-frequency correction module to obtain a merged image, and then the merged image is corrected to obtain a color-free image. Since the low-frequency information in the color-biased image can be weighted by the attention mechanism module to increase the weight of the low-frequency information, and then the low-frequency information in the first color-biased image can be accurately corrected by the low-frequency correction module, the correction effect of the color-biased image is improved.

[0323] Fig.10 This is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application. Fig.10 The image processing device 10 may be arranged in a smart terminal or may be a smart terminal. The image processing device 10 includes a determination module 11 and a processing module 12, wherein:

[0324] The determination module 11 is used to obtain or determine a first color-shifted image to be processed;

[0325] The processing module 12 is used to process the first color-biased image through an image processing model to obtain an image without color bias.

[0326] Optionally, the image processing model includes a frequency separation module; the processing module 12 is specifically used for:

[0327] Performing frequency separation processing on the first color-shifted image by the frequency separation module to obtain a first low-frequency image and a high-frequency image corresponding to the first color-shifted image;

[0328] The first low-frequency image and the high-frequency image are processed to obtain the image without color deviation.

[0329] Optionally, the processing module 12 is specifically used for:

[0330] Determine at least one image block of the first color-shifted image by the frequency separation module;

[0331] Determine the discrete cosine transform DCT image corresponding to each of the image blocks by the frequency separation module to obtain at least one DCT image, wherein the DCT image includes at least one DCT coefficient;

[0332] The first low-frequency image and the high-frequency image are determined according to the at least one DCT image.

[0333] Optionally, the image processing model includes a low-frequency correction module and a high-frequency correction module, the low-frequency correction module includes an encoder module, an attention mechanism module and a decoder module; the processing module 12 is specifically used for:

[0334] Performing feature extraction processing on the first low-frequency image at least once by the encoder module to obtain a preliminary feature image, M first low-frequency feature images, and a second low-frequency feature image, wherein the image size of the second low-frequency feature image is smaller than the size of the first low-frequency image, and M is an integer greater than or equal to 1;

[0335] Processing the second low-frequency feature image by the attention mechanism module to obtain a weighted feature image;

[0336] Processing the M first low-frequency feature images, the weighted feature image and the preliminary feature image by the decoder module to obtain the second low-frequency image;

[0337] The high-frequency correction module performs high-frequency correction processing on the high-frequency image and the second low-frequency image to obtain an image without color cast.

[0338] Optionally, the processing module 12 is specifically used for:

[0339] Performing feature extraction processing on the second low-frequency feature image by the attention mechanism module to obtain an intermediate feature map;

[0340] Determine the DCT coefficient feature vector, the maximum pooling feature vector and the average pooling feature vector corresponding to the intermediate feature map through the attention mechanism module;

[0341] Processing the maximum pooling feature vector, the average pooling feature vector and the DCT coefficient feature vector through the attention mechanism module to obtain a weight feature vector;

[0342] The weighted feature vector and the second low-frequency feature image are weighted by the attention mechanism module to obtain the weighted feature image.

[0343] Optionally, the processing module 12 is specifically used for:

[0344] The maximum pooling feature vector, the average pooling feature vector and the DCT coefficient feature vector are combined and processed by the attention mechanism module to obtain a first feature matrix;

[0345] Performing convolution processing on the first feature matrix through the attention mechanism module to obtain a second feature matrix;

[0346] The attention mechanism module performs row-wise addition processing on the elements in the second feature matrix to obtain a first feature vector;

[0347] The first feature vector is normalized by the attention mechanism module to obtain the weight feature vector.

[0348] Optionally, the decoder module includes a pooling module, M upsampling modules and a dual convolution module; the processing module 12 is specifically used for:

[0349] Performing feature extraction processing on the weighted feature image through the pooling module to obtain an initial output image;

[0350] Processing the initial output image and the M first low-frequency feature images through an upsampling module to obtain at least one output image;

[0351] The at least one output image and the preliminary feature image are processed by the dual convolution module to obtain the second low-frequency image.

[0352] Optionally, at least one of the following is included:

[0353] The pooling module includes a convolution layer, a linear rectification function layer and a deconvolution layer;

[0354] The upsampling module includes two convolutional layers, two linear rectification function layers and a deconvolution layer;

[0355] The dual convolution module includes two convolution layers and two linear rectification function layers.

[0356] Optionally, the processing module 12 is specifically used for:

[0357] Performing an addition process on the high-frequency image and the second low-frequency image by the high-frequency correction module to obtain a combined image;

[0358] The combined image is corrected by the high-frequency correction module to obtain the image without color cast.

[0359] Optionally, the determining module 11 is specifically configured to:

[0360] receiving an image processing request sent by an electronic device, wherein the image processing request includes the first color-biased image;

[0361] The first color-shifted image is determined according to the image processing request.

[0362] Optionally, the determining module 11 is specifically configured to:

[0363] Taking pictures to obtain an initial image;

[0364] When it is determined that the initial image is a color-shifted image, the initial image is determined as the first color-shifted image.

[0365] Optionally, the processing module 11 is specifically used for:

[0366] inputting the first color-biased image into the image processing model;

[0367] The image output by the image processing model is determined as the image without color deviation.

[0368] The image processing device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0369] Fig.11 This is a schematic diagram of the structure of another image processing device provided in an embodiment of the present application. Fig.11 ,exist Fig.10 Based on the embodiment shown, the image processing device further includes a display module 13, wherein:

[0370] The display module 13 is used to display the image without color cast.

[0371] The image processing device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0372] Fig.12 This is a structural diagram of another image processing device provided in an embodiment of the present application. Fig.12 ,exist Fig.10 Based on the embodiment shown, the image processing device further includes a sending module 14, wherein:

[0373] The sending module 14 is used to send the image without color cast to an electronic device.

[0374] The image processing device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0375] An embodiment of the present application further provides an intelligent terminal, which includes a memory and a processor. An image processing program is stored in the memory. When the image processing program is executed by the processor, the steps of the image processing method in any of the above embodiments are implemented.

[0376] An embodiment of the present application further provides a computer-readable storage medium, on which an image processing program is stored. When the image processing program is executed by a processor, the steps of the image processing method in any of the above embodiments are implemented.

[0377] In the embodiments of the smart terminal and computer-readable storage medium provided in the present application, all technical features of the above-mentioned image processing method embodiments are included, and the expanded and explained contents of the specification are basically the same as those of the above-mentioned method embodiments, and will not be repeated here.

[0378] The embodiment of the present application further provides a computer program product, which includes a computer program code. When the computer program code runs on a computer, the computer executes the methods in the above various possible implementation modes.

[0379] An embodiment of the present application also provides a chip, including a memory and a processor, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device equipped with the chip executes the methods in various possible implementation modes as described above.

[0380] It is understood that the above scenarios are only examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of the present application. The technical solutions of the present application can also be applied to other scenarios. Alternatively, it is known to those skilled in the art that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0381] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0382] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0383] The units in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0384] In the present application, the same or similar terminology concepts, technical solutions and / or application scenario descriptions are generally described in detail only the first time they appear. When they appear again later, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of the present application, for the same or similar terminology concepts, technical solutions and / or application scenario descriptions that are not described in detail later, reference can be made to the previous related detailed descriptions.

[0385] In the present application, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0386] The various technical features of the technical solution of the present application can be arbitrarily combined. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present application.

[0387] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, a controlled terminal, or a network device, etc.) to execute the method of each embodiment of the present application.

[0388] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or at least one computer instruction. When loading and executing the computer program instruction on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instruction can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, and optionally, the computer instruction can be transmitted from a website site, computer, server or data center by wired (optionally coaxial cable, optical fiber, digital subscriber line) or wireless (optionally infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, data center, etc. that contains one or at least one available medium integrated. The available medium may be a magnetic medium (optionally a floppy disk, a storage disk, a magnetic tape), an optical medium (optionally a DVD), or a semiconductor medium (optionally a solid state disk (SSD)).

[0389] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image processing method, characterized in that: The following steps are involved: Processing the first color-biased image by using an image processing model to obtain an image without color-bias; The image processing model includes a frequency separation module; The step of processing the first color-biased image by using the image processing model to obtain an image without color-bias includes: Performing frequency separation processing on the first color-shifted image by the frequency separation module to obtain a first low-frequency image and a high-frequency image corresponding to the first color-shifted image; wherein the first low-frequency image is an image composed of low-frequency information separated from the first color-shifted image; and the high-frequency image is an image composed of high-frequency information separated from the first color-shifted image; Performing feature extraction processing on the first low-frequency image at least once to obtain a preliminary feature image, M first low-frequency feature images and a second low-frequency feature image, where M is a positive integer; Performing feature extraction processing on the second low-frequency feature image to obtain an intermediate feature map; Determine a DCT coefficient feature vector, a maximum pooling feature vector, and an average pooling feature vector corresponding to the intermediate feature map; Processing the DCT coefficient feature vector, the maximum pooling feature vector and the average pooling feature vector to obtain a weight feature vector; Performing weighted processing on the weighted feature vector and the second low-frequency feature image to obtain a weighted feature image; Processing the first low-frequency feature image, the weighted feature image and the preliminary feature image to obtain a second low-frequency image; performing addition processing on the high-frequency image and the second low-frequency image to obtain a combined image; Correction processing is performed on the merged image to obtain the image without color cast.

2. The method according to claim 1, characterized in that The performing frequency separation processing on the first color-shifted image to obtain a first low-frequency image and a high-frequency image corresponding to the first color-shifted image includes: At least one image block of the first color-shifted image is determined, and an orthogonal transformation image corresponding to the image block is determined to determine the first low-frequency image and the high-frequency image.

3. The method according to claim 2, characterized in that The step of processing the first low-frequency feature image, the weighted feature image and the preliminary feature image to obtain a second low-frequency image comprises: Performing feature extraction processing on the weighted feature image through a pooling module to obtain an initial output image; Processing the initial output image and the first low-frequency feature image by an upsampling module to obtain at least one output image; The at least one output image and the preliminary feature image are processed by a dual convolution module to obtain the second low-frequency image.

4. The method according to claim 3, characterized in that Include at least one of the following: The pooling module includes a convolution layer, a linear rectification function layer and a deconvolution layer; The upsampling module includes two convolutional layers, two linear rectification function layers and a deconvolution layer; The dual convolution module includes two convolution layers and two linear rectification function layers.

5. The method according to any one of claims 1 to 4, characterized in that Before the first color-shifted image is processed by the image processing model, the method further includes: A first color-shifted image to be processed is acquired or determined.

6. An intelligent terminal, characterized in that: The intelligent terminal comprises: a memory and a processor, wherein an image processing program is stored in the memory, and when the image processing program is executed by the processor, the steps of the image processing method according to any one of claims 1 to 5 are implemented.

7. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 5 are implemented.

8. A computer program product, characterized in that The computer program product comprises: a computer program code, and when the computer program code is executed on a computer, the computer is enabled to implement the steps of the image processing method according to any one of claims 1 to 5.

Images