Image Processing Method, Apparatus, Electronic Device, and Storage Medium

Abstract

The present disclosure relates to an image processing method, apparatus, electronic device, and storage medium. The method includes: constructing a plurality of color lookup tables; fusing the color lookup tables according to the fusion weights of the color lookup tables for the image to be processed to obtain a target color lookup table corresponding to the image to be processed; for each pixel point in the image to be processed, looking up the color value corresponding to each pixel point after sharpness adjustment in the target color lookup table; processing the image to be processed according to the color values of each pixel point after sharpness adjustment to construct a target image. According to the present disclosure, the sharpness of the image can be adjusted in real time, and while reducing the computational amount, the image processing accuracy can be improved, as well as the image processing speed and efficiency.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technologies, and in particular, to an image processing method, apparatus, electronic device, and storage medium. Background Art

[0002] With the progress of science and technology and the popularization of intelligent terminal devices, people increasingly use intelligent terminal devices to watch videos. In the fields of short videos and live broadcasts, etc., methods such as reducing the resolution or compressing the bit rate are often adopted to improve the real-time and smooth user experience when users watch videos. However, this operation will cause a significant reduction in the clarity of the video. Especially after beauty effects processing such as skin smoothing is superimposed on the video image, users often feel that the picture is blurred and the clarity is poor.

[0003] In related technologies, CNN (Convolutional Neural Networks) networks or sharpening algorithms are often used to improve the clarity of the picture.

[0004] However, the calculation amount of the CNN network is large, and it is difficult to be widely used on terminal devices such as mobile phones and televisions; although the sharpening algorithm can reduce the calculation amount, this method also amplifies the noise while enhancing details, and the effect of improving clarity in low-light, backlight and other scenarios is poor. Summary of the Invention

[0005] The present disclosure provides an image processing method, apparatus, electronic device, and storage medium to at least solve the problems of poor image clarity improvement effect and low applicability in related technologies. The technical solutions of the present disclosure are as follows:

[0006] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, including:

[0007] Construct a plurality of color lookup tables;

[0008] Fuse each of the color lookup tables according to the fusion weights of the color lookup tables for the image to be processed, to obtain a target color lookup table corresponding to the image to be processed;

[0009] For each pixel point in the image to be processed, look up the color value corresponding to each pixel point after clarity adjustment in the target color lookup table;

[0010] Process the image to be processed according to the color values of each pixel point after clarity adjustment, to construct a target image.

[0011] In one of the embodiments, the fusing each of the color lookup tables according to the fusion weights of the color lookup tables for the image to be processed, to obtain a target color lookup table corresponding to the image to be processed, includes:

[0012] The to-be-processed image is subjected to prediction processing by a weight prediction network to obtain the fusion weights corresponding to the respective color lookup tables, where the weight prediction network is a neural network for performing fusion weight prediction;

[0013] The respective color lookup tables are subjected to fusion processing according to the fusion weights corresponding to the respective color lookup tables to obtain a target color lookup table corresponding to the to-be-processed image.

[0014] In one embodiment, the step of subjecting the to-be-processed image to prediction processing by a weight prediction network to obtain the fusion weights corresponding to the respective color lookup tables includes:

[0015] The to-be-processed image is subjected to downsampling processing to obtain a downsampled image;

[0016] The downsampled image is subjected to prediction processing by a weight prediction network to obtain the fusion weights corresponding to the respective color lookup tables.

[0017] In one embodiment, the step of subjecting the respective color lookup tables to fusion processing according to the fusion weights corresponding to the respective color lookup tables to obtain a target color lookup table corresponding to the to-be-processed image includes:

[0018] For any one color lookup table, weighted processing is performed according to the fusion weight corresponding to the color lookup table and the color lookup table to obtain a weighted color lookup table;

[0019] The weighted color lookup tables corresponding to the respective color lookup tables are added together to obtain a target color lookup table corresponding to the to-be-processed image.

[0020] In one embodiment, the step of, for each pixel point in the to-be-processed image, looking up the color value corresponding to the clarity adjustment of each pixel point in the target color lookup table includes:

[0021] For any one pixel point in the to-be-processed image, the mapping position of the pixel point in the target color lookup table is determined according to the pixel value of the pixel point;

[0022] The pixel value corresponding to the mapping position of each pixel point in the to-be-processed image in the target color lookup table is determined as the color value corresponding to the clarity adjustment of each pixel point in the target image.

[0023] In one embodiment, the method further includes:

[0024] Obtaining a plurality of sample groups, where each sample group includes a sample image and an annotated image corresponding to the sample image, and the clarity of the annotated image is higher than that of the sample image;

[0025] The sample image is subjected to prediction processing by an initial weight prediction network to obtain prediction fusion weights corresponding to each initial color lookup table;

[0026] Each of the initial color lookup tables is subjected to fusion processing according to the prediction fusion weights corresponding to each of the initial color lookup tables to obtain an initial target color lookup table corresponding to the sample image;

[0027] The color values corresponding to the clarity adjustment of each pixel point in the sample image are found in the initial target color lookup table;

[0028] According to the color values corresponding to the clarity adjustment of each pixel point in the sample image, a predicted target image corresponding to the sample image is constructed;

[0029] According to the difference between the predicted target image and the labeled image corresponding to the sample image, a training loss value is determined;

[0030] According to the training loss value, the weights of the initial weight prediction network are adjusted, and the color values of each of the initial color lookup tables are adjusted to obtain the weight prediction network and each of the color lookup tables.

[0031] In one embodiment, before the fusion of each color lookup table according to the fusion weights of each color lookup table for the image to be processed to obtain a target color lookup table corresponding to the image to be processed, the method further includes:

[0032] An initial image is obtained;

[0033] When the clarity of the initial image meets the adjustment condition, the initial image is used as the image to be processed.

[0034] According to a second aspect of the embodiments of the present disclosure, an image processing apparatus is provided, including:

[0035] A construction unit configured to construct multiple color lookup tables;

[0036] A first fusion unit configured to perform fusion of each color lookup table according to the fusion weights of each color lookup table for the image to be processed to obtain a target color lookup table corresponding to the image to be processed;

[0037] A first lookup unit configured to perform, for each pixel point in the image to be processed, looking up the color values corresponding to the clarity adjustment of each pixel point in the target color lookup table;

[0038] The first processing unit is configured to process the image to be processed according to the color values after sharpness adjustment for each of the pixel points, and construct a target image.

[0039] In one embodiment, the first fusion unit is further configured to perform:

[0040] Perform prediction processing on the image to be processed through a weight prediction network to obtain the fusion weights corresponding to each of the color lookup tables, where the weight prediction network is a neural network for performing fusion weight prediction;

[0041] Perform fusion processing on each of the color lookup tables according to the fusion weights corresponding to each of the color lookup tables to obtain the target color lookup table corresponding to the image to be processed.

[0042] In one embodiment, the first fusion unit is further configured to perform:

[0043] Perform downsampling processing on the image to be processed to obtain a downsampled image;

[0044] Perform prediction processing on the downsampled image through a weight prediction network to obtain the fusion weights corresponding to each of the color lookup tables.

[0045] In one embodiment, the first fusion unit is further configured to perform:

[0046] For any one of the color lookup tables, perform weighted processing according to the fusion weight corresponding to the color lookup table and the color lookup table to obtain a weighted color lookup table;

[0047] Add up the weighted color lookup tables corresponding to each of the color lookup tables to obtain the target color lookup table corresponding to the image to be processed.

[0048] In one embodiment, the first lookup unit is further configured to perform:

[0049] For any pixel point in the image to be processed, determine the mapping position of the pixel point in the target color lookup table according to the pixel value of the pixel point;

[0050] Determine the pixel value corresponding to the mapping position of each of the pixel points in the image to be processed in the target color lookup table as the color value corresponding to each of the pixel points in the target image after sharpness adjustment.

[0051] In one embodiment, the device further includes:

[0052] A first acquisition unit, configured to acquire a plurality of sample groups, where the sample groups include sample images and corresponding annotation images, and the clarity of the annotation images is higher than that of the sample images;

[0053] A prediction unit, configured to perform prediction processing on the downsampled sample images through an initial weight prediction network to obtain prediction fusion weights corresponding to respective initial color lookup tables;

[0054] A second fusion unit, configured to perform fusion processing on the respective initial color lookup tables according to the prediction fusion weights corresponding to the respective initial color lookup tables to obtain an initial target color lookup table corresponding to the sample images;

[0055] A second lookup unit, configured to perform a lookup in the initial target color lookup table for color values corresponding to each pixel point in the sample images after clarity adjustment;

[0056] A second processing unit, configured to construct a predicted target image corresponding to the sample images according to the color values corresponding to each pixel point in the sample images after clarity adjustment;

[0057] A determination unit, configured to determine a training loss value according to the difference between the predicted target image and the annotation image corresponding to the sample images;

[0058] A training unit, configured to adjust the weights of the initial weight prediction network according to the training loss value and adjust the color values of the respective initial color lookup tables to obtain the weight prediction network and the respective color lookup tables.

[0059] In one embodiment, the apparatus further includes:

[0060] A second acquisition unit, configured to acquire an initial image;

[0061] A third processing unit, configured to use the initial image as the image to be processed when the clarity of the initial image meets the adjustment condition.

[0062] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

[0063] A processor;

[0064] A memory for storing instructions executable by the processor;

[0065] Wherein, the processor is configured to execute the instructions to implement the image processing method described in any one of the foregoing items.

[0066] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image processing method as described in any one of the foregoing items.

[0067] According to a fifth aspect of the embodiments of the present disclosure, a computer program product is provided. The computer program product includes instructions that, when executed by a processor of an electronic device, enable the electronic device to execute the image processing method as described in any one of the foregoing items.

[0068] The technical solutions provided by the embodiments of the present disclosure at least bring the following beneficial effects:

[0069] The image processing method, apparatus, electronic device, and storage medium provided by this solution can construct multiple color lookup tables, and fuse the color lookup tables according to the fusion weights of the color lookup tables for the image to be processed, to obtain a target color lookup table corresponding to the image to be processed. Then, for each pixel point in the image to be processed, look up the color value corresponding to the clarity adjustment of each pixel point in the target color lookup table, and process the image to be processed according to the color values of each pixel point after clarity adjustment, to construct a target image. That is to say, the image processing method, apparatus, electronic device, and storage medium provided by the present disclosure can adaptively fuse a corresponding target color lookup table for the image to be processed based on multiple color lookup tables, and then improve the clarity of the image to be processed according to the target color lookup table to obtain a target image. Since the lookup table is a simple indexing operation, the present disclosure can reduce the amount of calculation, improve the image processing speed and efficiency, and adjust the image clarity in real time. And because the target color lookup table is an adaptive lookup table obtained by fusing multiple color lookup tables for the image to be processed, the embodiments of the present disclosure can improve the image processing accuracy while reducing the amount of calculation, as well as improve the image processing speed and efficiency.

[0070] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure, and do not constitute an improper limitation to the present disclosure.

[0072] Figure 1 is an application environment diagram of an image processing method shown according to an exemplary embodiment.

[0073] Figure 2 is a flowchart of an image processing method shown according to an exemplary embodiment.

[0074] Figure 3a It is a schematic diagram of a lookup table shown according to an exemplary embodiment.

[0075] Figure 3b It is a schematic diagram of a lookup table shown according to an exemplary embodiment.

[0076] Figure 4 It is a flowchart of step 206 in an image processing method shown according to an exemplary embodiment.

[0077] Figure 5 It is a flowchart of step 204 in an image processing method shown according to an exemplary embodiment.

[0078] Figure 6 It is a flowchart of an image processing method shown according to an exemplary embodiment.

[0079] Figure 7 It is a schematic diagram of a weight prediction network shown according to an exemplary embodiment.

[0080] Figure 8 It is a flowchart of an image processing method shown according to an exemplary embodiment.

[0081] Figure 9 It is a schematic diagram of an image processing method shown according to an exemplary embodiment.

[0082] Figure 10 It is a block diagram of an image processing apparatus shown according to an exemplary embodiment.

[0083] Figure 11 It is a block diagram of an electronic device shown according to an exemplary embodiment. Detailed implementation manners

[0084] To enable those of ordinary skill in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0085] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0086] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0087] The image processing method provided by this disclosure can be applied to an application environment as Figure 1 shown. Among them, the terminal 110 interacts with the server 120 through the network. The server 120 can compress the bit rate or reduce the resolution of the multimedia data (including video data and image data), and then send it to the terminal 110. After receiving the multimedia data, the terminal 110 can adjust the clarity of each image in the multimedia data as the image to be processed, improve the clarity of the image to be processed to obtain the corresponding target image, and display the target image. Among them, the terminal 110 can be but not limited to various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices, and the server 120 can be an independent server or a server cluster composed of multiple servers, etc.

[0088] Figure 2 is a flowchart of an image processing method shown according to an exemplary embodiment. As Figure 2 shown, the image processing method is used for Figure 1 the terminal 110 shown, and includes the following steps.

[0089] In step 202, multiple color lookup tables are constructed.

[0090] In step 204, according to the fusion weights of the image to be processed for each of the color lookup tables, each color lookup table is fused to obtain the target color lookup table corresponding to the image to be processed.

[0091] In the embodiment of this disclosure, the image to be processed can be an image to be adjusted in clarity. Among them, the image to be processed can be image information or an image frame in video data. In one example, the image to be processed can be a portrait image. In the embodiment of this disclosure, no specific limitation is imposed on the image to be processed.

[0092] N color lookup tables can be constructed or trained in advance. Among them, the color lookup table can be a 3D lookup table, that is, a three-dimensional color lookup table for color space mapping. The original color can be mapped to a new color through this color lookup table. For example, when the original R (red) value is 0, the R value can be mapped to 5 through the color lookup table; when the original R value is 1, the R value can be mapped to 6 through the color lookup table... That is, the pre-defined mapping relationship between colors can be stored in a graph, and this graph is called a color lookup table.

[0093] Exemplarily, taking the image to be processed as an RGB image as an example, under normal circumstances, the number of colors that can be represented by the RGB color mode is 256×256×256. If we want to fully record this mapping relationship, a large amount of memory is required, and the computational complexity during lookup is huge. To simplify the computational complexity and reduce memory occupancy, the color lookup table uses a certain sampling interval to record and store n adjacent colors in one mapping record. (Exemplarily, n is the sampling step size, which can be set according to requirements. For example, it can be set to 4). In this way, the color lookup table only needs to store 64×64×64 mapping relationships, reducing the memory space used to store the mapping relationship and making full use of the computing power of the GPU (graphics processing unit).

[0094] Taking the sampling step size of 4 as an example, the color lookup table can be divided into 8×8, a total of 64 squares in the horizontal and vertical directions. The B (blue) component in each square is a fixed value. The 64 squares altogether represent 64 mapping values of the B component. Refer to Figure 3a shown in Figure 3a where the numbers in each square represent the label of the square, and each square can correspond to a different B component value, Figure 3a not shown in Figure 3b Each square is further divided into 64×64 small squares in the horizontal and vertical directions. The abscissa represents 64 mapping values of the R (red) component, and the ordinate represents 64 mapping values of the G (green) component. Taking one square as an example, the value distributions of the R component and the G component can be as Figure 3b shown. In this way, the 64 value ranges of the RBG three components are mapped through this color lookup table.

[0095] Since the illumination and other conditions of different images are different, different mapping relationships can be corresponding, that is, different color lookup tables. The embodiments of the present disclosure can determine the fusion weights for each color lookup table through the image content and / or global information of the image to be processed, and then can perform weighted summation on the fusion of multiple color lookup tables according to the fusion weights corresponding to each color lookup table to obtain a target color lookup table adapted to the image to be processed. That is, different images to be processed can all obtain a target color lookup table adapted to different images to be processed through the fusion of multiple color lookup tables.

[0096] In the embodiments of the present disclosure, the number of color lookup tables is not specifically limited. The number of color lookup tables can be determined based on the requirements for clarity and the computing power of the mobile terminal. Among them, the number of color lookup tables is proportional to the requirements for clarity and / or the computing power of the mobile terminal. That is, the higher the requirements for clarity, the more color lookup tables can be set, and the stronger the computing power of the mobile terminal, the more color lookup tables can be set.

[0097] In step 206, for each pixel point in the image to be processed, look up the corresponding color value after sharpness adjustment for each pixel point in the target color lookup table;

[0098] In step 208, process the image to be processed according to the color values of each pixel point after sharpness adjustment, and construct a target image.

[0099] In the embodiments of the present disclosure, after obtaining the target color lookup table, the pixel values mapped by each pixel point in the image to be processed can be looked up through the target color lookup table, and the mapped pixel value is the corresponding color value after the sharpness adjustment of the pixel point. According to the looked-up pixel values of each pixel point, the image to be processed is processed, that is, the pixel value corresponding to the pixel point in the image to be processed is replaced with the pixel value corresponding to the pixel point, and a target image can be constructed. The target image is the image to be processed with improved sharpness, that is, the target image and the image to be processed correspond to the same image content, but the sharpness of the target image is higher than that of the image to be processed.

[0100] The image processing method provided by this solution can construct multiple color lookup tables, and fuse each color lookup table according to the fusion weights of the image to be processed for each color lookup table to obtain the target color lookup table corresponding to the image to be processed. Then, for each pixel point in the image to be processed, look up the corresponding color value after sharpness adjustment for each pixel point in the target color lookup table, and process the image to be processed according to the color values of each pixel point after sharpness adjustment to construct a target image. That is, the image processing method provided by the present disclosure can adaptively fuse the corresponding target color lookup table for the image to be processed based on multiple color lookup tables, and then improve the sharpness of the image to be processed according to the target color lookup table to obtain a target image. Since the lookup table is a simple indexing operation, the present disclosure can reduce the amount of calculation, improve the image processing speed and efficiency, and adjust the image sharpness in real time. And since the target color lookup table is an adaptive lookup table obtained by fusing multiple color lookup tables for the image to be processed, the embodiments of the present disclosure can improve the image processing accuracy while reducing the amount of calculation, as well as improve the image processing speed and efficiency.

[0101] In an exemplary embodiment, as Figure 4 shown, in step 206, for each pixel point in the image to be processed, looking up the corresponding color value after sharpness adjustment for each pixel point in the target color lookup table can be implemented through the following steps:

[0102] In step 402, for any pixel point in the image to be processed, determine the mapping position of the pixel point in the target color lookup table according to the pixel value of the pixel point;

[0103] In step 404, determine the pixel values corresponding to the mapping positions of each pixel point in the to-be-processed image in the target color lookup table, which are the color values corresponding to each pixel point in the target image after sharpness adjustment.

[0104] In the embodiments of the present disclosure, after obtaining the target color lookup table corresponding to the to-be-processed image, the pixel values corresponding to each pixel point in the to-be-processed image can be mapped and searched according to the target color lookup table. Furthermore, the pixel values mapped by each pixel point in the target color lookup table are used as the color values corresponding to each pixel point in the target image after sharpness adjustment. Then, according to the color values corresponding to each pixel point after sharpness adjustment, the mapped target image can be constructed.

[0105] Exemplarily, still taking the to-be-processed image as an RGB image as an example. For any pixel point A in the to-be-processed image, the mapping position of this pixel point in the target color lookup table can be determined according to the pixel value of this pixel point and the sampling step of the target color lookup table (i.e., the sampling step of the color lookup table). For example: Referring to Figure 3a and Figure 3b shown, for the pixel value of pixel point A (R1(32), G1(64), B1(64)), the square corresponding to pixel point A in the target color lookup table can be determined according to B1 / 4 as the 16th square. Further, the row (G1 / 4) and column (R1 / 4) of this pixel point A in the 16th square can be determined, that is, the mapping position of this pixel point A in the target color lookup table is the 16th row and the 8th column in the 16th square. The pixel value corresponding to this position is (R2, G2, B2), that is, the pixel value corresponding to the mapping position of pixel point A in the target color lookup table is (R2, G2, B2). By analogy, the pixel values mapped by each pixel point in the to-be-processed image in the target color lookup table can be obtained. According to the pixel values mapped by each pixel point in the to-be-processed image in the target color lookup table, the target image can be formed.

[0106] Based on the image processing method provided by the present disclosure, the target color lookup table adaptively fused for the to-be-processed image can be based on multiple color lookup tables, which can improve the sharpness of the to-be-processed image to obtain the target image. Since the lookup table is a simple indexing operation, the present disclosure can reduce the amount of calculation and improve the image processing speed and efficiency.

[0107] In an exemplary embodiment, as Figure 5 shown, in step 204, according to the fusion weights of the to-be-processed image for each color lookup table, each color lookup table is fused to obtain the target color lookup table corresponding to the to-be-processed image, which can be realized through the following steps:

[0108] In step 502, the image to be processed is predicted by a weight prediction network to obtain the fusion weights corresponding to each color lookup table, and the weight prediction network is a neural network for performing fusion weight prediction.

[0109] In step 504, each color lookup table is fused according to the fusion weights corresponding to each color lookup table to obtain the target color lookup table corresponding to the image to be processed.

[0110] In the embodiments of the present disclosure, each color lookup table can be fused based on the corresponding fusion weights to obtain the corresponding target color lookup table. The weight prediction network can be pre-trained, and the weight prediction network can be used to predict the fusion weights of each color lookup table corresponding to the image to be processed. Exemplarily, the weight prediction network can be a lightweight convolutional neural network.

[0111] The image to be processed can be input into the weight prediction network as the input information of the weight prediction network. The weight prediction network can predict the fusion weights of each color lookup table through the image content information and global information of the image to be processed. Then, each color lookup table is fused according to the fusion weights of each color lookup table to obtain the target color lookup table corresponding to the image to be processed.

[0112] In an exemplary embodiment, in step 504, fusing each color lookup table according to the fusion weights corresponding to each color lookup table to obtain the target color lookup table corresponding to the image to be processed includes:

[0113] For any color lookup table, perform weighted processing on the color lookup table according to the fusion weight corresponding to the color lookup table to obtain a weighted color lookup table;

[0114] Add the weighted color lookup tables corresponding to each color lookup table to obtain the target color lookup table corresponding to the image to be processed.

[0115] In the embodiments of the present disclosure, each color lookup table can be multiplied by its corresponding fusion weight respectively to implement weighted processing, and the weighted color lookup tables corresponding to each color lookup table are obtained respectively, and the weighted color lookup tables corresponding to each color lookup table are added to obtain the target color lookup table corresponding to the image to be processed.

[0116] Exemplarily, assuming that the color lookup tables include lookup table 1, lookup table 2, and lookup table 3, and the fusion weight 1 corresponding to lookup table 1, fusion weight 2 corresponding to lookup 2, and fusion weight 3 corresponding to lookup table 3 are predicted by the weight prediction network, then the target color lookup table = lookup table 1 * fusion weight 1 + lookup 2 * fusion weight 2 + lookup table 3 * fusion weight 3. Further, after the target color lookup table is fused, the pixel values mapped by each pixel point in the image to be processed can be found according to the target color lookup table, and then the target image can be obtained.

[0117] Based on the image processing method provided by the embodiments of the present disclosure, the fusion weights of each color lookup table corresponding to the image to be processed can be determined through a weight prediction network, and then the target color lookup table adapted to the image to be processed can be obtained by fusing each color lookup table and the fusion weights of each color lookup table, which can improve the accuracy of sharpness improvement.

[0118] In an exemplary embodiment, in step 502, the image to be processed is predicted by a weight prediction network to obtain the fusion weights corresponding to each color lookup table, which can be implemented through the following steps:

[0119] Perform downsampling processing on the image to be processed to obtain a downsampled image;

[0120] Predict the image to be processed through a weight prediction network to obtain the fusion weights corresponding to each color lookup table.

[0121] In the embodiments of the present disclosure, after the image to be processed is downsampled to obtain a downsampled image, the downsampled image can be used as the input information of the weight prediction network and input into the weight prediction network. The weight prediction network can predict the fusion weights of each color lookup table through the image content information and global information of the downsampled image. Then, according to the fusion weights of each color lookup table, each color lookup table is fused to obtain the target color lookup table corresponding to the image to be processed.

[0122] In this way, since the weight prediction network processes the downsampled image, the computational complexity can be reduced. A lightweight network can be used for the weight prediction network, which can reduce the occupation of computing resources of the mobile terminal and improve the image processing efficiency and speed.

[0123] In an exemplary embodiment, as Figure 6 shown, the method further includes:

[0124] In step 602, obtain multiple sample groups, where the sample group includes a sample image and an annotated image corresponding to the sample image, and the sharpness of the annotated image is higher than that of the sample image;

[0125] In step 604, predict the sample image through an initial weight prediction network to obtain the predicted fusion weights corresponding to each initial color lookup table;

[0126] In step 606, fuse each initial color lookup table according to the predicted fusion weights corresponding to each initial color lookup table to obtain the initial target color lookup table corresponding to the sample image;

[0127] In step 608, find the color values corresponding to each pixel point in the sample image after sharpness adjustment in the initial target color lookup table;

[0128] In step 610, a predicted target image corresponding to the sample image is constructed according to the color values corresponding to each pixel point in the sample image after sharpness adjustment.

[0129] In step 612, the training loss value is determined according to the difference between the predicted target image and the labeled image corresponding to the sample image.

[0130] In step 614, according to the training loss value, the weights of the initial weight prediction network are adjusted, and the color values of each initial color lookup table are adjusted to obtain the weight prediction network and each color lookup table.

[0131] In the embodiments of the present disclosure, the weight prediction network can be pre-trained and multiple color lookup tables can be constructed, where the number of color lookup tables can be determined based on the sharpness improvement requirements.

[0132] Exemplarily, a training set can be pre-constructed. For example: Images of multiple scenarios can be collected as sample images, such as images in low-light, backlight, indoor, outdoor and other scenarios, and image processing is performed on each sample image to improve sharpness (manual sharpness improvement can be used, or convolutional neural network can be used for sharpness improvement, and the present disclosure does not specifically limit the acquisition method of the labeled image). The labeled image of the sample image is constructed, and then a sample group is constructed according to the sample image and the labeled image corresponding to the sample image. Or, a training set can be constructed based on the application scenario. For example: When applied to the scenario of portrait image sharpness adjustment, the sample image can be portrait images in various lighting environments.

[0133] The sample image can be input into the initial weight prediction network for layer-by-layer calculation. The output of the initial weight prediction network is a vector of N×1, that is, the predicted fusion weights of N initial color lookup tables. After merging the N initial color lookup tables into an initial target color lookup table using the N predicted fusion weights, a sharpness improvement result map of the same size as the sample image can be obtained by looking up the initial target color lookup table according to the sample image, that is, the target image corresponding to the sample image is obtained.

[0134] Among them, the initial color lookup table can be a pre-constructed 3D lookup table. For example: a 3D lookup table constructed with a step size of 4. The sizes of the N initial color lookup tables are the same, but the contents can be the same or different. The present disclosure does not specifically limit the initial color lookup table.

[0135] The training loss value can be determined according to the difference between the target image corresponding to the sample image and the annotated image corresponding to the sample image. For example, loss functions such as L1 or L2 can be used to determine the difference between the target image corresponding to the sample image and the annotated image corresponding to the sample image, so as to obtain the training loss value. In the embodiments of the present disclosure, the specific manner of the training loss value is not limited. Further, the network parameters of the N initial color lookup tables (adjusting the pixel values corresponding to each pixel point in the initial color lookup table, that is, adjusting the mapping relationship corresponding to the initial color lookup table) and the initial weight prediction network can be adjusted according to the training loss value until the training loss value meets the training requirements, so as to obtain the weight prediction network and N color lookup tables.

[0136] Actually, before inputting the sample image into the initial weight prediction network, the sample image can be downsampled to obtain a downsampled sample image. For example, the sample image can be downsampled to a downsampled sample image of 64×64 size. Then, the downsampled sample image is input into the initial weight prediction network for layer-by-layer calculation to predict the predicted fusion weights of the N initial color lookup tables.

[0137] It should be noted that the weight prediction network can adopt the LeNet-5 network structure as shown in FIG. 7, or other lightweight regression networks such as Alexnet. In the embodiments of the present disclosure, the network structure of the weight prediction network is not specifically limited.

[0138] Based on the image processing method provided by the embodiments of the present disclosure, the weight prediction network and N color lookup tables can be pre-trained. A lightweight weight prediction network can be used to perform image content perception, determine the fusion weights of each color lookup table, and fuse through the fusion weights of each color lookup table to obtain a target color lookup table adapted to the image to be processed, so that the present solution can intelligently process various lighting scenarios; in the embodiments of the present disclosure, the weight prediction network and the 3D lookup table are perfectly combined. The weight prediction network is a lightweight network, and the 3D lookup table is a simple indexing operation, both of which have the characteristics of fast speed and high efficiency. Therefore, the intelligence, real-time and high efficiency of the present solution are ensured. And because the embodiments of the present disclosure have the advantages of fast speed and extremely high efficiency, combined with various image processing technologies such as beauty, filter, and blurring, it is suitable for various scenarios, such as various service scenarios such as live broadcast and short video on terminal devices.

[0139] In an exemplary embodiment, as Figure 8 shown, before step 204 of the above method, it may further include:

[0140] In step 802, an initial image is obtained;

[0141] In step 804, when the clarity of the initial image meets the adjustment condition, the initial image is used as the image to be processed.

[0142] In the embodiments of the present disclosure, the initial image may be image information collected by a mobile terminal. After the mobile terminal collects the image to be displayed (which may be an image frame in the collected video data), it determines whether the clarity of the initial image meets the adjustment condition. After the clarity of the initial image meets the condition, the initial image is used as the image to be processed for clarity adjustment, and the target image obtained after the adjustment is displayed.

[0143] When the initial image itself is clear enough (for example: when the live streamer does not add beauty effects, filters or other special effects, or the current lighting environment is relatively suitable, etc.), in fact, the initial image does not need to be sharpened and can be directly displayed, which can reduce the occupation of the computing resources of the mobile terminal.

[0144] Therefore, after collecting the initial image, the mobile terminal can first determine whether the clarity of the initial image meets the adjustment condition. The adjustment condition can be used to determine whether the initial image needs to be sharpened. For example, the adjustment condition may include at least one of the following conditions: the clarity of the initial image is lower than the clarity threshold, no special effects are added to the initial image, the brightness and / or saturation of the initial image meet the requirements, and an adjustment instruction for the clarity of the initial image is received.

[0145] When it is determined that the clarity of the initial image meets the adjustment condition, it can be determined that the clarity of the initial image is poor. Therefore, the initial image can be used as the image to be processed, and after the clarity is adjusted by using the image processing method of the foregoing embodiment, the target image obtained after the adjustment is displayed. Alternatively, when it is determined that the clarity of the initial image meets the adjustment condition, it can be determined that the clarity of the initial image is good and no clarity adjustment is required, and the initial image can be directly displayed.

[0146] In the embodiments of the present disclosure, the method for detecting the clarity of the initial image is not specifically limited. For example, the edge detection method, convolutional neural network, etc. can be used to detect the clarity of the initial image.

[0147] Based on the image processing method provided by the embodiments of the present disclosure, only the initial images whose clarity meets the adjustment condition can be sharpened, which can reduce the occupation of the computing resources of the mobile terminal and improve the image processing efficiency and speed.

[0148] In the embodiments of the present disclosure, the initial image may also be the image information in the multimedia data sent by the server to the mobile terminal. In an exemplary embodiment, in step 804, when the clarity of the initial image meets the adjustment condition, using the initial image as the image to be processed includes:

[0149] When the initial image has an adjustment identifier and it is determined that the clarity of the initial image meets the adjustment condition, the initial image is used as the image to be processed; wherein, the adjustment identifier is the identifier information added to the initial image by the server after detecting that the clarity of the initial image meets the adjustment condition.

[0150] In the embodiments of the present disclosure, when the server sends data to the mobile terminal, in order to reduce the occupancy of the bandwidth, ensure the real-time performance of the video, and improve the fluency of the video, it is necessary to compress the multimedia data, which will affect the clarity of the image sent to the mobile terminal side to a certain extent. Therefore, before sending the multimedia data to the mobile terminal, the server can determine whether the clarity of each frame of the initial image in the multimedia data meets the adjustment condition based on the degree of compression of the multimedia data to be compressed.

[0151] The clarity threshold corresponding to the initial image can be determined in advance for each compression degree. For any compression degree, when the clarity of the initial image is lower than the clarity threshold corresponding to this compression degree, the clarity of the initial image after compression will be affected to a greater extent. Therefore, the mobile terminal needs to further adjust the clarity of the initial image before displaying it; or, when the clarity of the initial image is higher than or equal to the clarity preset corresponding to this compression degree, the clarity of the initial image after compression is affected less. Therefore, the mobile terminal can directly display the initial image. That is to say, the adjustment condition can include that the clarity of the initial image is less than the clarity threshold corresponding to the degree of compression to be performed.

[0152] When the server sends multimedia data to the mobile terminal, it can detect the clarity of the initial image in the multimedia data frame by frame (the clarity of the initial image can be detected by methods such as edge detection and convolutional neural network), and when it is determined that the clarity of the initial image meets the adjustment condition, an adjustment identifier is added to the initial image. After receiving the initial image, the mobile terminal can determine whether the initial image carries the adjustment identifier. When the initial image carries the adjustment identifier, it can be determined that the clarity of the initial image meets the adjustment condition, and then the clarity adjustment process of the initial image can be performed. The specific adjustment process can refer to the relevant description of the foregoing embodiments, and the embodiments of the present disclosure will not elaborate on this; when the initial image does not carry the adjustment identifier, the initial image is directly displayed.

[0153] Based on the image processing method provided by the embodiments of the present disclosure, only the initial images whose clarity meets the adjustment condition can be adjusted in clarity, which can reduce the occupancy of the computing resources of the mobile terminal, and the detection process of the clarity of the initial image is implemented on the server side, which can further reduce the occupancy of the computing resources of the mobile terminal.

[0154] To enable those skilled in the art to better understand the embodiments of the present disclosure, the embodiments of the present disclosure will be described below through specific examples.

[0155] Referring to Figure 9 as shown, the image to be processed is a portrait image. First, the portrait image is downsampled, for example, downsampled to a size of 64×64 or 128×128 to obtain a downsampled image. Then, the downsampled image is input into a weight prediction network for prediction processing to obtain the fusion weights of 5 color lookup tables (Color Lookup Table 1: w1, Color Lookup Table 2: w2, Color Lookup Table 3: w3, Color Lookup Table 4: w4, Color Lookup Table 5: w5). According to the fusion weights of each color lookup table, multiple color lookup tables are fused to obtain the target color lookup table of the image to be processed. The mapping of the pixel points corresponding to the portrait area in the target color lookup table is a large change mapping, and the mapping of the pixel points corresponding to the background area is a small change mapping (approximate constant change). According to the target color lookup table, using the image to be processed as the original image, look up in the target color lookup table to obtain the mapping results of each pixel point in the image to be processed, and then obtain the target image, which is the result of improving the clarity of the image to be processed.

[0156] It should be understood that although Figures 1 - 9 the steps in the flowchart of Figures 1 - 9 are shown in sequence according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover,

[0157] at least a part of the steps in

[0158] Figure 10 may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of the steps or stages in other steps or other steps.

[0157] It can be understood that the same / similar parts among the various embodiments of the above methods in this specification can be referred to each other. Each embodiment focuses on the differences from other embodiments. For the relevant parts, refer to the descriptions of other method embodiments.

[0158] Figure 10 is a block diagram of an image processing device shown according to an exemplary embodiment. Referring to Figure 10 as shown, the device includes a construction unit 1002, a first fusion unit 1004, a first lookup unit 1006, and a first processing unit 1008.

[0159] The construction unit 1002 is configured to execute the construction of multiple color lookup tables;

[0160] The first fusion unit 1004 is configured to perform fusing the color look-up tables according to the fusion weights for each of the color look-up tables based on the image to be processed, so as to obtain a target color look-up table corresponding to the image to be processed;

[0161] The first look-up unit 1006 is configured to perform looking up, for each pixel point in the image to be processed, the color value corresponding to the clarity adjustment of each of the pixel points in the target color look-up table;

[0162] The first processing unit 1008 is configured to perform processing on the image to be processed according to the color values after the clarity adjustment of each of the pixel points, so as to construct a target image.

[0163] The image processing apparatus provided by this solution can adaptively fuse the corresponding target color look-up table for the image to be processed based on multiple color look-up tables, and further improve the clarity of the image to be processed according to the target color look-up table to obtain a target image. Since the look-up table is a simple indexing operation, the present disclosure can reduce the computational amount, improve the image processing speed and efficiency, and adjust the image clarity in real time. Moreover, since the target color look-up table is an adaptive look-up table obtained by fusing multiple color look-up tables for the image to be processed, the embodiments of the present disclosure can, while reducing the computational amount, improve the image processing accuracy, as well as improve the image processing speed and efficiency.

[0164] In an exemplary embodiment, the first fusion unit 1004 is further configured to perform:

[0165] Performing prediction processing on the image to be processed through a weight prediction network to obtain the fusion weights corresponding to each of the color look-up tables, where the weight prediction network is a neural network for performing fusion weight prediction;

[0166] Performing fusion processing on each of the color look-up tables according to the fusion weights corresponding to each of the color look-up tables to obtain a target color look-up table corresponding to the image to be processed.

[0167] In an exemplary embodiment, the first fusion unit 1004 is further configured to perform:

[0168] Performing downsampling processing on the image to be processed to obtain a downsampled image;

[0169] Performing prediction processing on the downsampled image through a weight prediction network to obtain the fusion weights corresponding to each of the color look-up tables.

[0170] In an exemplary embodiment, the first fusion unit 1004 is further configured to perform:

[0171] For any color lookup table, perform weighted processing on the color lookup table according to the fusion weight corresponding to the color lookup table to obtain a weighted color lookup table;

[0172] Add the weighted color lookup tables corresponding to each of the color lookup tables to obtain the target color lookup table corresponding to the image to be processed.

[0173] In an exemplary embodiment, the first lookup unit 1006 is further configured to perform:

[0174] For any pixel point in the image to be processed, determine the mapping position of the pixel point in the target color lookup table according to the pixel value of the pixel point;

[0175] Determine the pixel values corresponding to the mapping positions of each pixel point in the image to be processed in the target color lookup table, and use them as the color values corresponding to each pixel point in the target image after sharpness adjustment.

[0176] In an exemplary embodiment, the device further includes:

[0177] A first acquisition unit, configured to acquire a plurality of sample groups, where each sample group includes a sample image and an annotation image corresponding to the sample image, and the sharpness of the annotation image is higher than that of the sample image;

[0178] A prediction unit, configured to perform prediction processing on the downsampled sample image through an initial weight prediction network to obtain the predicted fusion weights corresponding to each initial color lookup table;

[0179] A second fusion unit, configured to perform fusion processing on each of the initial color lookup tables according to the predicted fusion weights corresponding to each of the initial color lookup tables to obtain the initial target color lookup table corresponding to the sample image;

[0180] A second lookup unit, configured to perform lookup in the initial target color lookup table to obtain the color values corresponding to the sharpness adjustment of each pixel point in the sample image;

[0181] A second processing unit, configured to construct a predicted target image corresponding to the sample image according to the color values corresponding to the sharpness adjustment of each pixel point in the sample image;

[0182] A determination unit, configured to determine a training loss value according to the difference between the predicted target image and the annotation image corresponding to the sample image;

[0183] A training unit, configured to execute adjusting weights of the initial weight prediction network according to the training loss value, and adjusting color values of each of the initial color lookup tables to obtain the weight prediction network and each of the color lookup tables.

[0184] In an exemplary embodiment, the apparatus further includes:

[0185] A second obtaining unit, configured to execute obtaining an initial image;

[0186] A third processing unit, configured to execute using the initial image as the image to be processed when sharpness of the initial image meets an adjustment condition.

[0187] Regarding the apparatus in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0188] Figure 11 It is a block diagram of an electronic device 1100 for an image processing method shown according to an exemplary embodiment. For example, the electronic device 1100 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0189] Referring to Figure 11 , the electronic device 1100 may include one or more of the following components: a processing component 1102, a memory 1104, a power supply component 1106, a multimedia component 1108, an audio component 1110, an input / output (I / O) interface 1112, a sensor component 1114, and a communication component 1116.

[0190] The processing component 1102 generally controls the overall operation of the electronic device 1100, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 1102 may include one or more processors 1120 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 1102 may include one or more modules to facilitate interaction between the processing component 1102 and other components. For example, the processing component 1002 may include a multimedia module to facilitate interaction between the multimedia component 1108 and the processing component 1102.

[0191] The memory 1104 is configured to store various types of data to support the operation of the electronic device 1100. Examples of such data include instructions for any application or method operating on the electronic device 1100, contact data, phone book data, messages, pictures, videos, and the like. The memory 1104 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disks, optical disks, or graphene memory.

[0192] The power supply component 1106 provides power to various components of the electronic device 1100. The power supply component 1106 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 1100.

[0193] The multimedia component 1108 includes a screen that provides an output interface between the electronic device 1100 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 1108 includes a front camera and / or a rear camera. When the electronic device 1100 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0194] The audio component 1110 is configured to output and / or input audio signals. For example, the audio component 1110 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 1100 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 1104 or transmitted via the communication component 1116. In some embodiments, the audio component 1110 further includes a speaker for outputting audio signals.

[0195] The I / O interface 1112 provides an interface between the processing component 1102 and a peripheral interface module, which can be a keyboard, a click wheel, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a power-on button, and a lock button.

[0196] The sensor assembly 1114 includes one or more sensors for providing a status assessment of various aspects of the electronic device 1100. For example, the sensor assembly 1114 can detect the on / off state of the electronic device 1100, the relative positioning of components, such as the display and keypad of the electronic device 1100. The sensor assembly 1114 can also detect a change in the position of the electronic device 1100 or an electronic device 1100 component, the presence or absence of user contact with the electronic device 1100, the orientation or acceleration / deceleration of the device 1100, and a change in the temperature of the electronic device 1100. The sensor assembly 1114 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 1114 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1114 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0197] The communication component 1116 is configured to facilitate communication between the electronic device 1100 and other devices in a wired or wireless manner. The electronic device 1100 can access a wireless network based on communication standards, such as WiFi, a carrier network (such as 2G, 3G, 4G, or 5G), or a combination thereof. In an exemplary embodiment, the communication component 1116 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1116 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0198] In an exemplary embodiment, the electronic device 1100 can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above-described methods.

[0199] In an exemplary embodiment, a computer-readable storage medium including instructions, such as a memory 1104 including instructions, is also provided. The above instructions can be executed by the processor 1120 of the electronic device 1100 to complete the above-described methods. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0200] In an exemplary embodiment, a computer program product is further provided. The computer program product includes instructions that can be executed by the processor 1120 of the electronic device 1100 to implement the above method.

[0201] It should be noted that the above-mentioned devices, electronic devices, computer-readable storage media, computer program products, etc. may also include other implementation manners according to the description of the method embodiments. The specific implementation manners can be referred to the description of the relevant method embodiments and will not be elaborated here one by one.

[0202] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The specification and embodiments are only to be considered as exemplary, and the true scope and spirit of the present disclosure are pointed out by the claims.

[0203] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. An image processing method, characterized in that, Applied to a mobile terminal, including: Construct multiple color lookup tables, where the number of the color lookup tables is positively correlated with the clarity requirement and / or the computing power of the mobile terminal; Obtain an initial image, and when the initial image has an adjustment identifier, determine that the clarity of the initial image meets the adjustment condition, and use the initial image as the image to be processed; where the adjustment identifier is identification information added by the server to the initial image after detecting that the clarity of the initial image meets the adjustment condition; Fuse each of the color lookup tables according to the fusion weights of the image to be processed for each of the color lookup tables to obtain a target color lookup table corresponding to the image to be processed; For each pixel point in the image to be processed, look up the color value corresponding to each pixel point after clarity adjustment in the target color lookup table; Process the image to be processed according to the color values of each pixel point after clarity adjustment to construct a target image.

2. The method according to claim 1, wherein The step of fusing each of the color lookup tables according to the fusion weights of the image to be processed for each of the color lookup tables to obtain a target color lookup table corresponding to the image to be processed includes: Perform prediction processing on the image to be processed through a weight prediction network to obtain the fusion weights corresponding to each of the color lookup tables, where the weight prediction network is a neural network for performing fusion weight prediction; Fuse each of the color lookup tables according to the fusion weights corresponding to each of the color lookup tables to obtain a target color lookup table corresponding to the image to be processed.

3. The method according to claim 2, wherein The step of performing prediction processing on the image to be processed through a weight prediction network to obtain the fusion weights corresponding to each of the color lookup tables includes: Perform downsampling processing on the image to be processed to obtain a downsampled image; Perform prediction processing on the downsampled image through a weight prediction network to obtain the fusion weights corresponding to each of the color lookup tables.

4. The method according to claim 2 or 3, characterized in that, The step of fusing each of the color lookup tables according to the fusion weights corresponding to each of the color lookup tables to obtain a target color lookup table corresponding to the image to be processed includes: For any one of the color lookup tables, perform weighted processing according to the fusion weight corresponding to the color lookup table and the color lookup table to obtain a weighted color lookup table; Add the weighted color lookup tables corresponding to each of the color lookup tables to obtain a target color lookup table corresponding to the image to be processed.

5. The method according to claim 1 or 2, characterized in that, The step of, for each pixel point in the image to be processed, looking up the color value corresponding to each pixel point after clarity adjustment in the target color lookup table includes: For any one pixel point in the image to be processed, determine the mapping position of the pixel point in the target color lookup table according to the pixel value of the pixel point; Determine that the pixel value corresponding to the mapping position of each pixel point in the image to be processed in the target color lookup table is the color value corresponding to each pixel point in the target image after clarity adjustment.

6. The method according to claim 2, characterized in that, The method further includes: Obtain multiple sample groups, where each sample group includes a sample image and a labeled image corresponding to the sample image, and the clarity of the labeled image is higher than that of the sample image; The sample image is predicted by the initial weight prediction network to obtain the predicted fusion weights corresponding to each initial color lookup table; Each of the initial color lookup tables is fused according to the predicted fusion weights corresponding to each of the initial color lookup tables to obtain the initial target color lookup table corresponding to the sample image; The color values corresponding to the clarity adjustment of each pixel point in the sample image are found in the initial target color lookup table; According to the color values corresponding to the clarity adjustment of each pixel point in the sample image, a predicted target image corresponding to the sample image is constructed; According to the difference between the predicted target image and the labeled image corresponding to the sample image, a training loss value is determined; According to the training loss value, the weights of the initial weight prediction network are adjusted, and the color values of each of the initial color lookup tables are adjusted to obtain the weight prediction network and each of the color lookup tables.

7. An image processing apparatus, characterized in that, Applied to a mobile terminal, it includes: A construction unit configured to construct multiple color lookup tables, where the number of the color lookup tables is positively correlated with the clarity requirement and / or the computing power of the mobile terminal; A determination unit configured to obtain an initial image and, when the initial image has an adjustment identifier, determine that the clarity of the initial image meets the adjustment condition and use the initial image as an image to be processed; where the adjustment identifier is identification information added to the initial image by the server after detecting that the clarity of the initial image meets the adjustment condition; A first fusion unit configured to fuse each of the color lookup tables according to the fusion weights of each of the color lookup tables for the image to be processed to obtain the target color lookup table corresponding to the image to be processed; A first lookup unit configured to look up, for each pixel point in the image to be processed, the color value corresponding to the clarity adjustment of each pixel point in the target color lookup table; A first processing unit configured to process the image to be processed according to the color values after the clarity adjustment of each pixel point to construct a target image.

8. The device according to claim 7, wherein The first fusion unit is further configured to perform: The image to be processed is predicted by a weight prediction network to obtain the fusion weights corresponding to each of the color lookup tables, where the weight prediction network is a neural network for predicting fusion weights; Each of the color lookup tables is fused according to the fusion weights corresponding to each of the color lookup tables to obtain the target color lookup table corresponding to the image to be processed.

9. The device according to claim 8, characterized in that, The first fusion unit is further configured to perform: The image to be processed is downsampled to obtain a downsampled image; The downsampled image is predicted by a weight prediction network to obtain the fusion weights corresponding to each of the color lookup tables.

10. The device according to claim 8 or 9, characterized in that, The first fusion unit is further configured to perform: For any one of the color lookup tables, weighted processing is performed according to the fusion weight corresponding to the color lookup table and the color lookup table to obtain a weighted color lookup table; Add the weighted color lookup tables corresponding to the respective color lookup tables to obtain the target color lookup table corresponding to the image to be processed.

11. The device according to claim 7 or 8, characterized in that, The first lookup unit is further configured to perform: For any pixel point in the image to be processed, determine the mapping position of the pixel point in the target color lookup table according to the pixel value of the pixel point; Determine the pixel values corresponding to the mapping positions of the respective pixel points in the target color lookup table in the image to be processed, and use them as the color values corresponding to the respective pixel points in the target image after sharpness adjustment.

12. The device according to claim 8, characterized in that, The apparatus further includes: A first acquisition unit configured to acquire a plurality of sample groups, where each sample group includes a sample image and an annotation image corresponding to the sample image, and the sharpness of the annotation image is higher than that of the sample image; A prediction unit configured to perform prediction processing on the sample image through an initial weight prediction network to obtain prediction fusion weights corresponding to the respective initial color lookup tables; A second fusion unit configured to perform fusion processing on the respective initial color lookup tables according to the prediction fusion weights corresponding to the respective initial color lookup tables to obtain an initial target color lookup table corresponding to the sample image; A second lookup unit configured to perform lookup in the initial target color lookup table to obtain the color values corresponding to the respective pixel points in the sample image after sharpness adjustment; A second processing unit configured to perform construction of a predicted target image corresponding to the sample image according to the color values corresponding to the respective pixel points in the sample image after sharpness adjustment; A determination unit configured to perform determination of a training loss value according to the difference between the predicted target image and the annotation image corresponding to the sample image; A training unit configured to perform adjustment of the weights of the initial weight prediction network according to the training loss value, and perform adjustment of the color values of the respective initial color lookup tables to obtain the weight prediction network and the respective color lookup tables.

13. An electronic device, characterized in that, Includes: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to execute the instructions to implement the image processing method according to any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to execute the image processing method according to any one of claims 1 to 6.

15. A computer program product, the computer program product including instructions, characterized in that, When the instructions are executed by the processor of the electronic device, the electronic device is enabled to execute the image processing method according to any one of claims 1 to 6.

Images