Image color enhancement method and device, equipment and storage medium

Abstract

The invention discloses an image color enhancement method and device, equipment and a storage medium, and relates to the technical field of computer vision, and the image color enhancement method comprises the steps: receiving a to-be-enhanced image, and extracting the image features of the to-be-enhanced image; performing fine processing on a preset three-dimensional lookup table feature through the image feature to obtain a content perception toning feature; performing color perception enhancement processing on the image features through the preset three-dimensional lookup table features to obtain color perception semantic features; performing fine-grained enhancement processing on the content perception toning features to obtain target perception features; and performing color enhancement on the to-be-enhanced image according to the target perception feature and the color perception semantic feature to obtain a target image. According to the invention, refined color adjustment of image content perception can be realized, and the visual quality of image color enhancement is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to an image color enhancement method and device, equipment and storage medium. BACKGROUND

[0002] Image color enhancement is a key technology in the field of computer vision and image processing. By enhancing the color of an image, the visual quality of the image can be improved. However, current image color enhancement methods lack the use of image semantic information and color transformation features, resulting in low image color enhancement quality. Therefore, how to improve the visual quality of image color enhancement is still a problem to be solved. SUMMARY

[0003] The main purpose of the present application is to provide an image color enhancement method, device, equipment and storage medium, which aims to solve the technical problem of how to improve the visual quality of image color enhancement.

[0004] To achieve the above purpose, the present application provides an image color enhancement method, which comprises:

[0005] receiving an image to be enhanced, and extracting image features of the image to be enhanced;

[0006] performing fine processing on a preset three-dimensional lookup table feature through the image features to obtain content-aware color adjustment features;

[0007] performing color perception enhancement processing on the image features through the preset three-dimensional lookup table features to obtain color perception semantic features;

[0008] performing fine-grained enhancement processing on the content-aware color adjustment features to obtain target perception features;

[0009] performing color enhancement on the image to be enhanced according to the target perception features and the color perception semantic features to obtain a target image.

[0010] In an embodiment, the step of performing fine processing on a preset three-dimensional lookup table feature through the image features to obtain content-aware color adjustment features comprises:

[0011] splitting the preset three-dimensional lookup table feature into multiple three-dimensional sub-features, and obtaining a query projection matrix according to the three-dimensional sub-features;

[0012] removing the key projection matrix and the value projection matrix of the image features and splitting them to obtain multiple image sub-features;

[0013] obtaining content-aware color adjustment features according to the three-dimensional sub-features, the query projection matrix and the image sub-features.

[0014] In an embodiment, the step of performing color perception enhancement processing on the image feature by using the preset three-dimensional lookup table feature to obtain a color perception semantic feature comprises:

[0015] The preset three-dimensional lookup table feature is split into a plurality of three-dimensional sub-features, and a key projection matrix and a value projection matrix are obtained according to the three-dimensional sub-features;

[0016] The query projection matrix of the image feature is removed and split to obtain a plurality of image sub-features;

[0017] A color perception semantic feature is obtained according to the three-dimensional sub-features, the key projection matrix, the value projection matrix, and the image sub-features.

[0018] In an embodiment, the step of performing fine-grained enhancement processing on the content-aware color adjustment feature to obtain a target perception feature comprises:

[0019] The content-aware color adjustment feature is channel enhanced to obtain a channel enhanced feature;

[0020] The content-aware color adjustment feature is spatially enhanced to obtain a spatially enhanced feature;

[0021] The channel enhanced feature and the spatially enhanced feature are fused to obtain a target perception feature.

[0022] In an embodiment, the step of performing channel enhancement on the content-aware color adjustment feature to obtain a channel enhanced feature comprises:

[0023] The content-aware color adjustment feature is globally average-pooled to generate a channel statistic;

[0024] A channel attention weight vector is generated according to the channel statistic;

[0025] A channel enhanced feature is obtained according to the channel attention weight vector and the content-aware color adjustment feature.

[0026] In an embodiment, the step of performing spatial enhancement on the content-aware color adjustment feature to obtain a spatially enhanced feature comprises:

[0027] An average pooling map and a maximum pooling map are generated according to the content-aware color adjustment feature;

[0028] A fusion feature map is obtained according to the average pooling map and the maximum pooling map;

[0029] A spatial attention weight map is generated according to the fusion feature map;

[0030] A spatially enhanced feature is obtained according to the spatial attention weight map and the content-aware color adjustment feature.

[0031] In an embodiment, the step of performing color enhancement on the image to be enhanced according to the target perceptual feature and the color perceptual semantic feature to obtain a target image comprises:

[0032] obtaining an image content adaptive weight vector according to the target perceptual feature and the color perceptual semantic feature;

[0033] obtaining a target parameter according to the image content adaptive weight vector and a preset basic three-dimensional lookup table;

[0034] obtaining a target image according to the target parameter and the image to be enhanced.

[0035] In addition, to achieve the above object, the present application further provides an image color enhancement device, which comprises:

[0036] an extraction module, configured to receive an image to be enhanced and extract image features of the image to be enhanced;

[0037] a processing module, configured to finely process preset three-dimensional lookup table features through the image features to obtain content perceptual color adjustment features;

[0038] a color module, configured to perform color perceptual enhancement processing on the image features through the preset three-dimensional lookup table features to obtain color perceptual semantic features;

[0039] a perception module, configured to perform fine-grained enhancement processing on the content perceptual color adjustment features to obtain target perceptual features;

[0040] a generation module, configured to perform color enhancement on the image to be enhanced according to the target perceptual features and the color perceptual semantic features to obtain a target image.

[0041] In addition, to achieve the above object, the present application further provides an image color enhancement device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the image color enhancement method as described above.

[0042] In addition, to achieve the above object, the present application further provides a storage medium, which is a computer readable storage medium, and a computer program is stored on the storage medium, and the computer program is executed by a processor to implement the steps of the image color enhancement method as described above.

[0043] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the image color enhancement method described above.

[0044] This application provides an image color enhancement method. The method involves receiving an image to be enhanced, extracting image features from the image to be enhanced, refining the image features using a preset three-dimensional lookup table to obtain content-aware color adjustment features, performing color-aware enhancement processing on the image features using the preset three-dimensional lookup table features to obtain color-aware semantic features, performing fine-grained enhancement processing on the content-aware color adjustment features to obtain target-aware features, and enhancing the image to be enhanced based on the target-aware features and the color-aware semantic features to obtain a target image.

[0045] In summary, this application establishes a two-way interaction mechanism between image semantic features and three-dimensional lookup table features, deeply integrates the semantic information of the image with the transformation features of the color lookup table, achieves refined image enhancement with content awareness, and improves the visual quality of image color enhancement. Attached Figure Description

[0046] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating an embodiment of the image color enhancement method of this application.

[0049] Figure 2 This is a schematic diagram of the structural framework provided for Embodiment 1 of the image color enhancement method of this application;

[0050] Figure 3 This is a flowchart illustrating Embodiment 2 of the image color enhancement method of this application;

[0051] Figure 4 This is a schematic diagram of the dual-branch parallel structure provided in Embodiment 2 of the image color enhancement method of this application;

[0052] Figure 5 A simplified flowchart illustrating the image color enhancement method provided in Embodiment 1 of this application;

[0053] Figure 6 This is a schematic diagram of the module structure of the image color enhancement device according to an embodiment of this application;

[0054] Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the image color enhancement method in the embodiments of this application.

[0055] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0056] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0057] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0058] The main solution of this application is to receive an image to be enhanced, extract image features from the image to be enhanced, refine the features of a preset three-dimensional lookup table using the image features to obtain content-aware color-grading features, perform color-aware enhancement processing on the image features using the preset three-dimensional lookup table features to obtain color-aware semantic features, perform fine-grained enhancement processing on the content-aware color-grading features to obtain target-aware features, and perform color enhancement on the image to be enhanced based on the target-aware features and the color-aware semantic features to obtain the target image.

[0059] Currently, image color enhancement lacks the utilization of semantic information and color transformation features, resulting in low quality. Therefore, improving the visual quality of image color enhancement remains a problem that needs to be solved.

[0060] This application establishes a two-way interaction mechanism between image semantic features and three-dimensional lookup table features, deeply integrates the semantic information of the image with the transformation features of the color lookup table, realizes refined image enhancement with content awareness, and improves the visual quality of image color enhancement.

[0061] Based on this, embodiments of this application provide an image color enhancement method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the image color enhancement method of this application.

[0062] In this embodiment, the image color enhancement method includes steps S10 to S50:

[0063] Step S10: Receive the image to be enhanced and extract the image features of the image to be enhanced;

[0064] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or image color enhancement device capable of performing the above functions. The following description uses an image color enhancement device as an example to illustrate this embodiment and the subsequent embodiments.

[0065] It should be noted that the structural framework diagram of this application can be referred to. Figure 2 , Figure 2 The input image X is processed by the cross-attention module, the feedforward network, and the lookup table perception module to obtain weights. The weights are then used to obtain a 3D lookup table, and linear interpolation is performed to obtain a color-enhanced output image.

[0066] It should be noted that the image to be enhanced can be a low-light or color-distorted input image X∈R. H×W×3 Where H and W are the image height and width, respectively, and 3 represents the RGB channels. A lightweight convolutional neural network can be used as the backbone network to process the input image X and extract image features. Image features are multi-scale features, including low-level visual features such as color, edges, and contours, and high-level semantic features.

[0067] Step S20: Refine the preset three-dimensional lookup table features using the image features to obtain content-aware color adjustment features;

[0068] It should be noted that a Three-Dimensional Look-Up Table (3D LUT) is a technique used for color space conversion and image processing, achieving rapid mapping through pre-stored input-output correspondences. The preset 3D look-up table features are the features Z∈R of the initialized or learnable 3D LUT. M×d Where M represents the number of LUT feature elements and d represents the feature dimension. This embodiment uses a cross-attention fusion module to receive image features from the image branch and feature Z from the LUT branch in parallel, and performs bidirectional feature interaction and fusion at different scales. The cross-attention fusion module can solve the problem of the separation between image semantic information and LUT color transformation features, achieving deep interaction and fusion of the two within a parallel framework.

[0069] In one feasible approach, the step of refining the preset three-dimensional lookup table features using the image features to obtain content-aware color tone features includes: splitting the preset three-dimensional lookup table features into multiple three-dimensional sub-features, and obtaining a query projection matrix based on the three-dimensional sub-features; removing the key projection matrix and value projection matrix of the image features and splitting them to obtain multiple image sub-features; and obtaining content-aware color tone features based on the three-dimensional sub-features, the query projection matrix, and the image sub-features.

[0070] It should be noted that the image feature is denoted as X. feat The specific dimensions depend on the backbone network and the selected layers. The specific processing procedure first splits the LUT features Z into {l1, ..., ln} according to the number of heads h. h}, where l i ∈R M×hd The second step is to extract the image features X. feat Flatten or process the sequence into a suitable form, and split it into {x1, ..., x} according to the number of heads h. h The third step is to apply the query projection matrix W only to the LUT branch. i Q (for the i-th head); the fourth step is to remove the key and value projection matrices from the image branch; the fifth step is to calculate the attention output for the i-th head:

[0071] Attn i =softmax(d k (l i W i Q)(x i ) T )x i

[0072] Where softmax is the normalization exponential function, d k is the dimension of the key vector.

[0073] The sixth step is to concatenate the outputs of all the heads: [Attn1, ..., Attn] h The seventh step involves using the output projection matrix W. O To merge:

[0074] AX→L=[Attn1,...,Attn h W O

[0075] The final refined LUT features, guided by image content, are known as content-aware color grading features.

[0076] Step S30: Perform color perception enhancement processing on the image features using the preset three-dimensional lookup table features to obtain color perception semantic features;

[0077] It should be noted that by using pre-defined three-dimensional lookup table features, i.e., LUT features (representing a specific color style), to feed back into and influence the understanding of image features, the perception of color changes by image features can be enhanced.

[0078] In one feasible approach, the step of performing color perception enhancement processing on the image features using the preset three-dimensional lookup table features to obtain color perception semantic features includes: splitting the preset three-dimensional lookup table features into multiple three-dimensional sub-features, and obtaining a key projection matrix and a value projection matrix based on the three-dimensional sub-features; removing the query projection matrix of the image features and splitting it to obtain multiple image sub-features; and obtaining color perception semantic features based on the three-dimensional sub-features, the key projection matrix, the value projection matrix, and the image sub-features.

[0079] It should be noted that the specific processing procedure is as follows: first, the LUT features Z are split into {l1, ..., l...} h} and image features X feat Let {x1, ..., x} h The second step is to apply the key projection matrix W to the LUT branch. i K and the projection matrix W of the sum of values. i V (for the i-th head); the third step is to remove the query projection matrix on the image branch; the fourth step is to calculate the attention output for the i-th head:

[0080] Attn i =softmax(d k x i (l i W i K) T (l) i W i V)

[0081] Where softmax is the normalization exponential function, and dk is the dimension of the key vector.

[0082] Step 5: Concatenate the outputs of all heads:

[0083] AL→X=[Attn1,...,Attn h ]

[0084] The final result is an image feature representation that incorporates LUT style information, namely color-perceptual semantic features. Through bidirectional cross-attention, a strong correlation is established between the image semantic content and the LUT color transformation style, enabling subsequent LUT adjustments to be based on a deep understanding of the image content.

[0085] Step S40: Perform fine-grained enhancement processing on the content-aware color-grading features to obtain target-aware features;

[0086] It should be noted that after receiving the content-aware color-grading features that have undergone preliminary fusion or refinement through cross-attention, the specific fine-grained enhancement processing method can be to work in parallel through channel attention branches and spatial attention branches, explicitly model the channel dependencies and spatial context relationships within the LUT features, and output fine-grained enhanced target-aware features.

[0087] Step S50: Perform color enhancement on the image to be enhanced based on the target perception features and the color perception semantic features to obtain the target image.

[0088] It should be noted that the color enhancement process can predict a set of image content adaptive weights for adjusting or combining the base LUT based on the target perception features and the color perception semantic features; then, the predicted weights are used to adjust or combine a set of predefined or learnable base 3D LUTs. Next, the RGB values ​​(r, g, b) of each pixel in the image to be enhanced X are interpolated and searched in the adjusted 3D LUT grid to obtain the enhanced RGB output value, and finally the target image Y^∈R is obtained. H×W×3 .

[0089] In one feasible approach, the step of color-enhancing the image to be enhanced based on the target perception features and the color perception semantic features to obtain a target image includes: obtaining an image content adaptive weight vector based on the target perception features and the color perception semantic features; obtaining target parameters based on the image content adaptive weight vector and a preset basic three-dimensional lookup table; and obtaining the target image based on the target parameters and the image to be enhanced.

[0090] It should be noted that, assuming the refined features based on LAM output, i.e., the target-aware features, are Flutout, and the image features fused with LUT information, i.e., the color-aware semantic features, are AL→X, a lightweight prediction network can be used to process the target-aware features and the color-aware semantic features to obtain the final image content adaptive weight vector w∈RK, where K is the number of basic 3D LUTs. Then, the LUTs are adjusted and combined. Using multiple basic 3D LUTs, a weighted combination is performed using the weight w:

[0091]

[0092] Furthermore, if a single learnable LUT is used, the weights w can be directly used to modulate the parameters of that LUT or as its input. For each pixel (i, j) in the image X to be enhanced, its RGB value is (r ij g ij b ij The final 3D LUT after adjustment or combination, i.e., LUT final In the grid, find the element containing (r) ij g ij b ij The 8 vertices of a cube, i.e., the corner points of the cube lattice, are denoted as:

[0093] {P(i d ,j d ,k d )} d∈(0…7)

[0094] Calculate (r) ij g ij b ij The relative position offset of the pixel is determined by its position relative to these 8 vertices. The enhanced RGB output value Y^(i,j) of this pixel is calculated using the trilinear interpolation formula:

[0095]

[0096] Among them, wr d wg d wb d These represent the linear interpolation weights of the point relative to vertex d in the three dimensions R, G, and B. Finally, after interpolating all pixels, the enhanced image Y^ is obtained, which is the target image.

[0097] This embodiment receives an image to be enhanced and extracts its image features. It then refines the image features using a preset 3D lookup table to obtain content-aware color-grading features. Next, it enhances the image features using the preset 3D lookup table to obtain color-aware semantic features. Finally, it performs fine-grained enhancement on the content-aware color-grading features to obtain target-aware features. Finally, it enhances the image to be enhanced using the target-aware features and the color-aware semantic features to obtain the target image.

[0098] In summary, this embodiment establishes a two-way interaction mechanism between image semantic features and three-dimensional lookup table features, deeply integrates the semantic information of the image with the transformation features of the color lookup table, realizes refined image enhancement with content awareness, and improves the visual quality of image color enhancement.

[0099] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 Step S40 also includes steps S401 to S403:

[0100] Step S401: Perform channel enhancement on the content-aware color tone features to obtain channel enhancement features;

[0101] It should be noted that this embodiment enhances the content-aware color correction features through dual-branch parallel processing. A schematic diagram of the dual-branch parallel structure can be found here. Figure 4 , Figure 4 The query table features are processed by channel attention and spatial attention respectively, and the two processing results are fused to obtain target perception features.

[0102] It should be noted that enhancing content-aware color grading features and explicitly modeling channels and spatial relationships can improve the precision of color transformations.

[0103] In one feasible approach, the step of performing channel enhancement on the content-aware color grading features to obtain channel-enhanced features includes: performing global average pooling on the content-aware color grading features to generate channel statistics; generating channel attention weight vectors based on the channel statistics; and obtaining channel-enhanced features based on the channel attention weight vectors and the content-aware color grading features.

[0104] It should be noted that the content-aware color grading feature is denoted as Flut∈R C×H′×W′ Where C represents the number of channels, and H' and W' represent the spatial dimensions. Global Average Pooling (GAP) is performed on the input feature Flut along the spatial dimensions (H', W') to obtain the channel statistics:

[0105]

[0106] The sc learns the dependencies between channels through a multi-layer perceptual mechanism, such as an MLP:

[0107] ac=σ(MLP(sc))

[0108] Where σ is the Sigmoid activation function.

[0109] Then output the channel attention weight vector a = [a1, a2, ..., a...]. C The weight vector a is then multiplied channel-wise with the original input feature Flut.

[0110] Flutch=a□Flut

[0111] Flutch is a channel enhancement feature.

[0112] Step S402: Spatially enhance the content-aware color-grading features to obtain spatially enhanced features;

[0113] In one feasible approach, the step of spatially enhancing the content-aware color grading features to obtain spatially enhanced features includes: generating an average pooling map and a max pooling map based on the content-aware color grading features; obtaining a fused feature map based on the average pooling map and the max pooling map; generating a spatial attention weight map based on the fused feature map; and obtaining spatially enhanced features based on the spatial attention weight map and the content-aware color grading features.

[0114] It should be noted that during spatial augmentation, the input features Flut are subjected to global average pooling (GAP) and / or global max pooling (GMP) along the channel dimension, resulting in two spatial feature maps:

[0115] savg∈R 1×H′×W′

[0116] smax∈R 1×H′×W′

[0117] Then concatenate savg and smax:

[0118] sspa = Concat(savg, smax)

[0119] Alternatively, add savg and smax together:

[0120] sspa = savg + smax

[0121] Use a small convolutional network, such as a 7x7 convolutional layer, to process SSPA and learn the importance of spatial location:

[0122] aspa=σ(Conv(sspa))

[0123] Where σ is the Sigmoid activation function.

[0124] The final output is the spatial attention weight graph Aspa∈R 1×H′×W′ .

[0125] The weighted map Aspa is multiplied element-wise with the original input feature Flut to obtain the spatial augmented feature:

[0126] Flutsp=Aspa⊙Flut

[0127] Step S403: Fuse the channel enhancement features and the spatial enhancement features to obtain target perception features.

[0128] It should be noted that the fusion methods include addition and concatenation. Adding (Summation) or concatenating the outputs of the two branches (FlutchFlutsp) followed by convolution (Concat+Conv) yields the target-aware features:

[0129] Flutout = Flututch + Flutsp

[0130] This embodiment enhances the content-aware color grading features through channel enhancement to obtain channel-enhanced features; it also enhances the content-aware color grading features through spatial enhancement to obtain spatial-enhanced features; finally, it fuses the channel-enhanced features and the spatial-enhanced features to obtain target-aware features. This embodiment enhances the content-aware color grading features through a dual-branch parallel structure, explicitly modeling the channel and spatial relationships of the content-aware color grading features, which can improve the precision of color transformation.

[0131] For example, to help understand the implementation flow of the image color enhancement method obtained by combining this embodiment with the above embodiment one, please refer to... Figure 5 , Figure 5 A simplified flowchart of an image color enhancement method is provided. Specifically: First, input preparation is performed, including the image to be enhanced. Then, feature extraction and parallel processing are conducted. Next, the processing results are fused through cross-attention. Then, the content-aware color adjustment features are refined using a LUT-aware module to obtain target-aware features. Finally, the target-aware features are used for weight prediction to obtain the target LUT, and color enhancement is achieved through the target LUT.

[0132] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the image color enhancement method of this application. Any simple transformations based on this technical concept are all within the protection scope of this application.

[0133] This application also provides an image color enhancement device, please refer to... Figure 6 The image color enhancement device includes:

[0134] Extraction module 10 is used to receive the image to be enhanced and extract the image features of the image to be enhanced;

[0135] Processing module 20 is used to refine the preset three-dimensional lookup table features through the image features to obtain content-aware color adjustment features;

[0136] Color module 30 is used to perform color perception enhancement processing on the image features through the preset three-dimensional lookup table features to obtain color perception semantic features;

[0137] The perception module 40 is used to perform fine-grained enhancement processing on the content perception color adjustment features to obtain target perception features;

[0138] The generation module 50 is used to perform color enhancement on the image to be enhanced based on the target perception features and the color perception semantic features to obtain the target image.

[0139] This embodiment receives an image to be enhanced and extracts its image features. It then refines the image features using a preset 3D lookup table to obtain content-aware color-grading features. Next, it enhances the image features using the preset 3D lookup table to obtain color-aware semantic features. Finally, it performs fine-grained enhancement on the content-aware color-grading features to obtain target-aware features. Finally, it enhances the image to be enhanced using the target-aware features and the color-aware semantic features to obtain the target image.

[0140] In summary, this embodiment establishes a two-way interaction mechanism between image semantic features and three-dimensional lookup table features, deeply integrates the semantic information of the image with the transformation features of the color lookup table, realizes refined image enhancement with content awareness, and improves the visual quality of image color enhancement.

[0141] In one embodiment, the processing module 20 is further configured to split the preset three-dimensional lookup table features into multiple three-dimensional sub-features, and obtain a query projection matrix based on the three-dimensional sub-features; remove the key projection matrix and value projection matrix of the image features and split them to obtain multiple image sub-features; and obtain content-aware color tone features based on the three-dimensional sub-features, the query projection matrix, and the image sub-features.

[0142] In one embodiment, the color module 30 is further configured to split the preset three-dimensional lookup table features into multiple three-dimensional sub-features, and obtain a key projection matrix and a value projection matrix based on the three-dimensional sub-features; remove the query projection matrix of the image features and split it to obtain multiple image sub-features; and obtain color-perceived semantic features based on the three-dimensional sub-features, the key projection matrix, the value projection matrix, and the image sub-features.

[0143] In one embodiment, the perception module 40 is further configured to perform channel enhancement on the content-aware color tone features to obtain channel-enhanced features; perform spatial enhancement on the content-aware color tone features to obtain spatial-enhanced features; and fuse the channel-enhanced features and the spatial-enhanced features to obtain target-aware features.

[0144] In one embodiment, the perception module 40 is further configured to perform global average pooling on the content-aware color grading features to generate channel statistics; generate channel attention weight vectors based on the channel statistics; and obtain channel enhancement features based on the channel attention weight vectors and the content-aware color grading features.

[0145] In one embodiment, the perception module 40 is further configured to generate an average pooling map and a max pooling map based on the content-aware color grading features; obtain a fused feature map based on the average pooling map and the max pooling map; generate a spatial attention weight map based on the fused feature map; and obtain spatial enhancement features based on the spatial attention weight map and the content-aware color grading features.

[0146] In one embodiment, the generation module 50 is further configured to obtain an image content adaptive weight vector based on the target perception features and the color perception semantic features; obtain target parameters based on the image content adaptive weight vector and a preset basic three-dimensional lookup table; and obtain a target image based on the target parameters and the image to be enhanced.

[0147] The image color enhancement device provided in this application, employing the image color enhancement method described in the above embodiments, can solve the technical problem of how to improve the visual quality of image color enhancement. Compared with the prior art, the beneficial effects of the image color enhancement device provided in this application are the same as those of the image color enhancement method described in the above embodiments, and other technical features in the image color enhancement device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0148] This application provides an image color enhancement device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the image color enhancement method in the first embodiment described above.

[0149] The following is for reference. Figure 7 This document illustrates a structural schematic diagram of an image color enhancement device suitable for implementing embodiments of this application. The image color enhancement device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7The image color enhancement device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0150] like Figure 7 As shown, the image color enhancement device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the image color enhancement device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touch screens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. The communication device 1009 allows the image color enhancement device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows image color enhancement devices with various systems, it should be understood that implementing or having all of the systems shown is not required. More or fewer systems may be implemented alternatively.

[0151] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0152] The image color enhancement device provided in this application, employing the image color enhancement method described in the above embodiments, can solve the technical problem of how to improve the visual quality of image color enhancement. Compared with the prior art, the beneficial effects of the image color enhancement device provided in this application are the same as those of the image color enhancement method described in the above embodiments, and other technical features of this image color enhancement device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0153] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0154] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0155] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to perform the image color enhancement method described in the above embodiments.

[0156] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0157] The aforementioned computer-readable storage medium may be included in the image color enhancement device; or it may exist independently and not assembled into the image color enhancement device.

[0158] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by the image color enhancement device, the image color enhancement device causes the following: receiving an image to be enhanced; extracting image features from the image to be enhanced; refining the preset three-dimensional lookup table features using the image features to obtain content-aware color adjustment features; performing color-aware enhancement processing on the image features using the preset three-dimensional lookup table features to obtain color-aware semantic features; performing fine-grained enhancement processing on the content-aware color adjustment features to obtain target-aware features; and performing color enhancement on the image to be enhanced based on the target-aware features and the color-aware semantic features to obtain a target image.

[0159] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0160] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0161] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0162] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described image color enhancement method, and is able to solve the technical problem of how to improve the visual quality of image color enhancement. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the image color enhancement method provided in the above embodiments, and will not be repeated here.

[0163] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the image color enhancement method described above.

[0164] The computer program product provided in this application can solve the technical problem of how to improve the visual quality of image color enhancement. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the image color enhancement method provided in the above embodiments, and will not be repeated here.

[0165] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. An image color enhancement method, characterized in that, The method includes: Receive the image to be enhanced and extract its image features; The image features are used to refine the features of the preset three-dimensional lookup table to obtain content-aware color adjustment features; The image features are subjected to color perception enhancement processing using the preset three-dimensional lookup table features to obtain color perception semantic features; The content-aware color-grading features are subjected to fine-grained enhancement processing to obtain target-aware features; The target image is obtained by color enhancement of the image to be enhanced based on the target perception features and the color perception semantic features.

2. The method as described in claim 1, characterized in that, The step of refining the preset three-dimensional lookup table features using the image features to obtain content-aware color tone features includes: The preset three-dimensional lookup table features are split into multiple three-dimensional sub-features, and the query projection matrix is ​​obtained based on the three-dimensional sub-features; Remove the key projection matrix and value projection matrix of the image features and split them to obtain multiple image sub-features; Content-aware color tone features are obtained based on the three-dimensional sub-features, the query projection matrix, and the image sub-features.

3. The method as described in claim 1, characterized in that, The step of performing color perception enhancement processing on the image features using the preset three-dimensional lookup table features to obtain color perception semantic features includes: The preset three-dimensional lookup table features are split into multiple three-dimensional sub-features, and the key projection matrix and value projection matrix are obtained based on the three-dimensional sub-features; Remove the query projection matrix of the image features and split it to obtain multiple image sub-features; Color perception semantic features are obtained based on the three-dimensional sub-features, the key projection matrix, the value projection matrix, and the image sub-features.

4. The method as described in claim 1, characterized in that, The step of performing fine-grained enhancement processing on the content-aware color-grading features to obtain target-aware features includes: Channel enhancement is performed on the content-aware color tone features to obtain channel enhancement features; Spatial enhancement is applied to the content-aware color-grading features to obtain spatially enhanced features; By fusing the channel enhancement features and the spatial enhancement features, target perception features are obtained.

5. The method as described in claim 4, characterized in that, The step of performing channel enhancement on the content-aware color tone features to obtain channel enhancement features includes: The content-aware color grading features are subjected to global average pooling to generate channel statistics; Generate a channel attention weight vector based on the channel statistics; Channel enhancement features are obtained based on the channel attention weight vector and the content-aware color grading features.

6. The method as described in claim 4, characterized in that, The step of spatially enhancing the content-aware color-grading features to obtain spatially enhanced features includes: Based on the content-aware color grading features, generate an average pooling graph and a max pooling graph; A fused feature map is obtained based on the average pooling map and the max pooling map; A spatial attention weight map is generated based on the fused feature map; Spatial enhancement features are obtained based on the spatial attention weight map and the content-aware color tone features.

7. The method as described in claim 1, characterized in that, The step of performing color enhancement on the image to be enhanced based on the target perception features and the color perception semantic features to obtain the target image includes: An image content adaptive weight vector is obtained based on the target perception features and the color perception semantic features; The target parameters are obtained by using the image content adaptive weight vector and a preset basic three-dimensional lookup table; The target image is obtained based on the target parameters and the image to be enhanced.

8. An image color enhancement device, characterized in that, The device includes: The extraction module is used to receive the image to be enhanced and extract the image features of the image to be enhanced; The processing module is used to refine the preset three-dimensional lookup table features through the image features to obtain content-aware color adjustment features; The color module is used to perform color perception enhancement processing on the image features through the preset three-dimensional lookup table features to obtain color perception semantic features; The perception module is used to perform fine-grained enhancement processing on the content perception color adjustment features to obtain the target perception features; The generation module is used to perform color enhancement on the image to be enhanced based on the target perception features and the color perception semantic features to obtain the target image.

9. An image color enhancement device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the image color enhancement method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the image color enhancement method as described in any one of claims 1 to 7.

Images