Color enhancement network training method and device, color enhancement method and device

Abstract

The present disclosure relates to a color enhancement network training method and device, and a color enhancement method and device. The color enhancement network training method comprises: obtaining a training image set comprising training images and corresponding target images; generating three-dimensional image color mapping information corresponding to the training images by a color enhancement network; performing color mapping processing on the training images based on the three-dimensional image color mapping information to generate predicted color enhanced images; determining texture differences between the predicted color enhanced images and the target images, and determining texture loss based on the texture differences; determining loss of a preset attribute corresponding to the three-dimensional image color mapping information; and adjusting parameters of the color enhancement network based on the texture loss and the loss of the preset attribute corresponding to the three-dimensional image color mapping information, to train the color enhancement network. The color enhancement network training method and device according to the present disclosure can improve the color enhancement effect of the trained color enhancement network.

Description

Technical Field

[0001] This disclosure relates to the field of video technology. More specifically, this disclosure relates to a training method and apparatus for a color enhancement network, and a color enhancement method and apparatus. Background Technology

[0002] With the significant breakthroughs achieved by deep learning across various fields, image enhancement has also yielded excellent results. Deep learning-based image enhancement can be broadly categorized into two types. The first type involves learning pixel-to-pixel mappings, such as constructing a generative adversarial network (GAN) for end-to-end supervised training. However, GANs amplify noise when restoring high-frequency information and perform dense calculations for each pixel, resulting in excessive memory consumption and impacting real-time performance. The second type involves training the network to obtain mapping transformation parameters. These parameters are used to derive the transformation results. A high-resolution image is downsampled to a low resolution, and the low-resolution image is used as input to the network. The model learns the affine transformation parameters of a bilateral grid, which are then applied to a guide map to obtain transformation coefficients. Finally, the enhanced image is obtained using the original image and the guide map transformation coefficients. The second type is faster than the first because the network only needs to predict the transformation coefficients, rather than predicting the result for each pixel. However, related techniques suffer from the following problems: noticeable color cast and low sharpness, excessive contrast, unrealistic colors, and video flickering due to differences in brightness between enhanced frames. Summary of the Invention

[0003] Exemplary embodiments of this disclosure provide a method and apparatus for training a color enhancement network, and a color enhancement method and apparatus, to at least solve the problems in the related art, or may not solve any of the aforementioned problems.

[0004] According to an exemplary embodiment of this disclosure, a method for training a color enhancement network is provided, comprising: acquiring a training image set, wherein the training image set includes training images and target images corresponding to the training images; generating three-dimensional image color mapping information corresponding to the training images through a color enhancement network; performing color mapping processing on the training images based on the three-dimensional image color mapping information to generate predicted color enhancement images; determining a texture difference between the predicted color enhancement images and the target images, and determining a texture loss based on the texture difference; determining a loss corresponding to a preset attribute of the three-dimensional image color mapping information; and adjusting the parameters of the color enhancement network based on the texture loss and the loss corresponding to the preset attribute of the three-dimensional image color mapping information to train the color enhancement network.

[0005] Optionally, the loss of the three-dimensional image color mapping information corresponding to the preset attributes includes the loss of the smoothness attribute of the three-dimensional image color mapping information and the loss of the monotonicity attribute of the three-dimensional image color mapping information.

[0006] Optionally, determining the texture difference between the predicted color-enhanced image and the target image may include: extracting texture features of the predicted color-enhanced image and texture features of the target image respectively; and determining the texture difference based on the texture features of the predicted color-enhanced image and the texture features of the target image.

[0007] Optionally, determining the texture loss based on the texture difference may include: determining the mean square error based on the texture difference, and using the mean square error as the texture loss.

[0008] Optionally, before generating the three-dimensional image color mapping information corresponding to the training image through the color enhancement network, the method may further include: downsampling the training image to obtain a downsampled image; performing data enhancement on the downsampled image to obtain a data-enhanced training image; performing data normalization on the training image set based on the data-enhanced training image, and using the normalized training image set as the training image set.

[0009] Optionally, the step of normalizing the training image set based on the data-augmented training images may include: determining the mean and variance of the data-augmented training images in the training image set; and normalizing the distribution of the data-augmented training images in the training image set to a Gaussian distribution based on the mean and the variance.

[0010] Optionally, obtaining the training image set may include: obtaining the training images; performing preset adjustments on the training images to obtain target images corresponding to the training images; and obtaining the training image set based on the training images and the target images corresponding to the training images.

[0011] Optionally, the preset adjustment of the training image may include: when the training image is a portrait image, adjusting at least one of the exposure, saturation, and contrast of the training image; when the training image is a landscape image, adjusting the preset details of the training image; when the training image is a low-light image, adjusting the shadow curve of the training image to brighten the shadows of the training image; and when the training image is an overexposed image, adjusting the highlight curve of the training image to darken the highlights of the training image.

[0012] Optionally, the color enhancement network may include a feature extraction network and multiple three-dimensional image color mapping networks, wherein generating the three-dimensional image color mapping information corresponding to the training image through the color enhancement network may include: inputting the training image into the feature extraction network, determining the weight of each of the multiple three-dimensional image color mapping networks through the feature extraction network, and performing a weighted sum based on the multiple three-dimensional image color mapping networks and the corresponding weights of the three-dimensional image color mapping networks to obtain the three-dimensional image color mapping information corresponding to the training image.

[0013] Optionally, the color enhancement network includes three three-dimensional image color mapping networks, which can be initialized as one standard three-dimensional image color mapping network and two all-zero three-dimensional image color mapping networks.

[0014] According to an exemplary embodiment of the present disclosure, a color enhancement method is provided, comprising: acquiring a video to be processed; inputting a current video frame to be processed in the video to be processed into a color enhancement network to generate three-dimensional image color mapping information corresponding to the current video frame to be processed; performing color enhancement processing on the current video frame to be processed based on the three-dimensional image color mapping information corresponding to the current video frame to be processed to obtain a color-enhanced video frame corresponding to the current video frame to be processed, wherein the color enhancement network is trained according to the color enhancement network training method of the present disclosure.

[0015] Optionally, before inputting the current video frame to be processed in the video to be processed into the color enhancement network to generate the three-dimensional image color mapping information corresponding to the current video frame, the method may further include: determining the grayscale histogram of the current video frame to be processed; determining the Bach distance between the grayscale histogram of the current video frame to be processed and the grayscale histogram of the previous frame of the current video frame to be processed, wherein inputting the current video frame to be processed in the video to be processed into the color enhancement network to generate the three-dimensional image color mapping information corresponding to the current video frame to be processed may include: when the Bach distance is greater than a threshold, performing the step of inputting the current video frame to be processed in the video to be processed into the color enhancement network to generate the three-dimensional image color mapping information corresponding to the current video frame to be processed.

[0016] Optionally, inputting the current video frame to be processed from the video to be processed into a color enhancement network to generate 3D image color mapping information corresponding to the current video frame to be processed may include: when the Bach distance is less than a threshold, updating the weight of the current video frame to be processed according to the Bach distance, the weight of the current video frame to be processed, and the weight of the previous frame of the current video frame to be processed; generating 3D image color mapping information of the current video frame to be processed based on the 3D image color mapping information of the previous frame of the current video frame to be processed and the weight of the current video frame to be processed.

[0017] According to an exemplary embodiment of this disclosure, a training apparatus for a color enhancement network is provided, comprising: an image set acquisition unit configured to acquire a training image set, wherein the training image set includes training images and target images corresponding to the training images; a mapping information generation unit configured to generate three-dimensional image color mapping information corresponding to the training images through the color enhancement network; a color enhancement unit configured to perform color mapping processing on the training images based on the three-dimensional image color mapping information to generate a predicted color enhancement image; a first loss calculation unit configured to determine a texture difference between the predicted color enhancement image and the target image, and determine a texture loss based on the texture difference; a second loss calculation unit configured to determine a loss corresponding to a preset attribute of the three-dimensional image color mapping information; and a parameter adjustment unit configured to adjust the parameters of the color enhancement network based on the texture loss and the loss corresponding to the preset attribute of the three-dimensional image color mapping information, so as to train the color enhancement network.

[0018] Optionally, the loss of the three-dimensional image color mapping information corresponding to the preset attribute may include the loss representing the smoothness attribute of the three-dimensional image color mapping information and the loss representing the monotonicity attribute of the three-dimensional image color mapping information.

[0019] Optionally, the first loss calculation unit may be configured to: extract the texture features of the predicted color-enhanced image and the texture features of the target image, respectively; and determine the texture difference based on the texture features of the predicted color-enhanced image and the texture features of the target image.

[0020] Optionally, the first loss calculation unit may be configured to: determine the mean square error based on the texture difference, and use the mean square error as the texture loss.

[0021] Optionally, the training apparatus for the color enhancement network may further include an image set processing unit configured to: downsample the training images to obtain downsampled images; perform data augmentation on the downsampled images to obtain data-augmented training images; perform data normalization on the training image set based on the data-augmented training images, and use the normalized training image set as the training image set.

[0022] Optionally, the image set processing unit may be configured to: determine the mean and variance of the data-enhanced training images in the training image set; and normalize the distribution of the data-enhanced training images in the training image set to a Gaussian distribution based on the mean and the variance.

[0023] Optionally, the image set acquisition unit may be configured to: acquire the training images; perform preset adjustments on the training images to obtain target images corresponding to the training images; and obtain the training image set based on the training images and the target images corresponding to the training images.

[0024] Optionally, the image set acquisition unit may be configured to: when the training image is a portrait image, adjust at least one of the exposure, saturation, and contrast of the training image; when the training image is a landscape image, adjust the preset details of the training image; when the training image is a low-light image, adjust the dark curve of the training image to brighten the dark areas of the training image; and when the training image is an overexposed image, adjust the bright curve of the training image to darken the bright areas of the training image.

[0025] Optionally, the color enhancement network may include a feature extraction network and multiple three-dimensional image color mapping networks, wherein the mapping information generation unit may be configured to: input the training image into the feature extraction network, determine the weights corresponding to each of the multiple three-dimensional image color mapping networks through the feature extraction network; and perform a weighted sum based on the multiple three-dimensional image color mapping networks and the corresponding weights of the three-dimensional image color mapping networks to obtain the three-dimensional image color mapping information corresponding to the training image.

[0026] Optionally, the color enhancement network includes three three-dimensional image color mapping networks, which can be initialized as one standard three-dimensional image color mapping network and two all-zero three-dimensional image color mapping networks.

[0027] According to an exemplary embodiment of the present disclosure, a color enhancement apparatus is provided, comprising: a video acquisition unit configured to acquire a video to be processed; a mapping information generation unit configured to input a current video frame to be processed from the video to be processed into a color enhancement network to generate three-dimensional image color mapping information corresponding to the current video frame to be processed; and a color enhancement processing unit configured to perform color enhancement processing on the current video frame to be processed based on the three-dimensional image color mapping information corresponding to the current video frame to be processed to obtain a color-enhanced video frame corresponding to the current video frame to be processed, wherein the color enhancement network is trained according to the training method of the color enhancement network in the present disclosure.

[0028] Optionally, the apparatus may further include a distance determination unit configured to: determine the grayscale histogram of the current video frame to be processed; and determine the Barcol distance between the grayscale histogram of the current video frame to be processed and the grayscale histogram of the previous frame of the current video frame to be processed.

[0029] Optionally, the mapping information generation unit can be configured to: when the Bach distance is greater than a threshold, perform the step of inputting the current video frame to be processed in the video to be processed into the color enhancement network to generate the three-dimensional image color mapping information corresponding to the current video frame to be processed.

[0030] Optionally, the mapping information generation unit can be configured to: when the Bach distance is less than a threshold, update the weight of the current video frame to be processed according to the Bach distance, the weight of the current video frame to be processed, and the weight of the previous frame of the current video frame to be processed; and generate the three-dimensional image color mapping information of the current video frame to be processed based on the three-dimensional image color mapping information of the previous frame of the current video frame to be processed and the weight of the current video frame to be processed.

[0031] According to an exemplary embodiment of the present disclosure, an electronic device is provided, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement a training method for a color enhancement network and a color enhancement method according to an exemplary embodiment of the present disclosure.

[0032] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor of an electronic device, causes the electronic device to perform a training method for a color enhancement network and a color enhancement method according to an exemplary embodiment of the present disclosure.

[0033] According to an exemplary embodiment of the present disclosure, a computer program product is provided, including a computer program / instructions, which, when executed by a processor, implement a training method for a color enhancement network and a color enhancement method according to an exemplary embodiment of the present disclosure.

[0034] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0035] By incorporating texture differences into the loss function, the training performance of the color enhancement network is improved, thereby increasing the clarity of videos enhanced using the trained color enhancement network.

[0036] Real-time video stream enhancement has been achieved.

[0037] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0039] Figure 1 A flowchart illustrating a training method for a color enhancement network according to an exemplary embodiment of the present disclosure is shown.

[0040] Figure 2 A flowchart illustrating a training method for a color enhancement network according to an exemplary embodiment of the present disclosure is shown.

[0041] Figure 3 A flowchart illustrating a color enhancement method according to an exemplary embodiment of the present disclosure is shown.

[0042] Figure 4 A block diagram of a training apparatus for a color enhancement network according to an exemplary embodiment of the present disclosure is shown.

[0043] Figure 5 A block diagram of a color enhancement device according to an exemplary embodiment of the present disclosure is shown.

[0044] Figure 6 This is a block diagram of an electronic device 600 according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0045] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0046] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following examples do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0047] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. As another example, "performing at least one of step one and step two" indicates the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0048] With the significant breakthroughs achieved by deep learning across various fields, image enhancement has also yielded excellent results. Deep learning-based image enhancement can be broadly categorized into two types. The first type involves learning pixel-to-pixel mappings, such as through end-to-end supervised training using a Generative Adversarial Network (GAN). However, GANs amplify noise and perform dense calculations for each pixel when restoring high-frequency information, resulting in excessive memory consumption and impacting real-time performance. The second type involves training the network to obtain mapping transformation parameters. These parameters are used to derive the transformation results. High-resolution images are downsampled to low resolution, and the low-resolution image is used as input to the network. The model learns affine transformation parameters for a bilateral mesh, which are then applied to a guide map to obtain transformation coefficients. Finally, the enhanced image is obtained using the original image and the guide map transformation coefficients. However, these techniques suffer from issues such as low clarity, color cast, excessive contrast, and unrealistic colors. Furthermore, differences in brightness between enhanced frames cause video flickering.

[0049] With the continuous development of short videos, video editing functions have become increasingly convenient. This disclosure saves users time in adjusting videos through automatic enhancement algorithms, promoting the creation of short videos. Below, we will refer to... Figures 1 to 6 The present disclosure provides a detailed description of a method and apparatus for training a color enhancement network, and a color enhancement method and apparatus according to exemplary embodiments thereof.

[0050] Figure 1 A flowchart illustrating a training method for a color enhancement network according to an exemplary embodiment of the present disclosure is shown.

[0051] Reference Figure 1 In step S101, a training image set is obtained. Here, the training image set includes training images and target images corresponding to the training images.

[0052] Existing open-source datasets suffer from issues such as a high proportion of low-light images, excessively high contrast in ground truth images, and unrealistic colors. They also tend to have a yellowish tint in facial images, which does not align with the aesthetic preferences of Chinese users regarding skin tones. Therefore, to improve the training performance of the color enhancement network, this publication does not directly use relevant image sets or datasets. Instead, it first obtains a training image set, i.e., it creates a dataset as the training image set.

[0053] In an exemplary embodiment of this disclosure, when acquiring a training image set, training images can be acquired first, and the training images can be pre-adjusted to obtain target images corresponding to the training images. Then, based on the training images and the target images corresponding to the training images, a training image set is obtained, thereby obtaining training images and target images corresponding to the training images for training.

[0054] In an exemplary embodiment of this disclosure, when making preset adjustments to the training image, if the training image is a portrait image, at least one of the exposure, saturation, and contrast of the training image may be adjusted; if the training image is a landscape image, preset details of the training image may be adjusted; if the training image is a low-light image, the dark curve of the training image may be adjusted to brighten the dark areas of the training image; if the training image is an overexposed image, the bright curve of the training image may be adjusted to darken the bright areas of the training image, thereby determining the target image corresponding to the training image.

[0055] In this disclosure, data pairs are collected from scenes including portraits, landscapes, low-light scenes, and overexposed scenes. The original images for portrait scenes are ordinary selfie videos selected from video platforms. For example, three frames are extracted from each video, resulting in, for example, 3000 pairs of original portrait images. Image processing applications are used to adjust the exposure, saturation, and contrast of each portrait image. To prevent skin tones from appearing yellowish, the saturation of yellow tones is reduced during adjustment. The resulting images, with higher clarity and image quality and without yellowish skin tones, are used as labeled images. The original images for landscape scenes are, for example, 2000 landscape images selected from an open-source dataset. Based on this dataset, image processing applications are used to perform batch fine-tuning on the landscape images to obtain labeled landscape images. The original images for low-light and overexposed scenes are selected by setting thresholds for the image histograms on the open-source dataset, identifying overexposed and low-light scenes. When adjusting an overexposed scene, adjust the highlight curve to darken the highlights, resulting in a labeled image of the overexposed scene; when adjusting a low-light scene, adjust the shadow curve to brighten the shadows, resulting in a labeled image of the low-light scene.

[0056] In step S102, the color enhancement network generates the three-dimensional image color mapping information corresponding to the training image.

[0057] In an exemplary embodiment of this disclosure, before generating the three-dimensional image color mapping information corresponding to the training image through the color enhancement network, the training image may first be downsampled to obtain a downsampled image, and the downsampled image may be data-enhanced to obtain a data-enhanced training image. Then, the training image set is normalized based on the data-enhanced training image, and the normalized training image set is used as the training image set, thereby improving the effect of the training image set by preprocessing the training image.

[0058] In an exemplary embodiment of this disclosure, when normalizing the training image set based on the data-enhanced training images, the mean and variance of the data-enhanced training images in the training image set can be determined first. Then, based on the mean and variance, the distribution of the data-enhanced training images in the training image set can be normalized to a Gaussian distribution, thereby improving the effect of preprocessing by using a Gaussian distribution.

[0059] For example, the training images are first downsampled to 320×320 pixels. Overfitting is prevented by adjusting the brightness, saturation, and randomly rotating the training images before data augmentation. To address color bias during the color enhancement network training process, the mean and variance of the dataset need to be calculated. The entire dataset is treated as a Gaussian distribution, and the images are normalized to, for example, a Gaussian distribution with a mean of 0 and a variance of 1, to prevent the color enhancement network from learning the mean of the dataset during training.

[0060] In exemplary embodiments of this disclosure, the color enhancement network may include a feature extraction network and multiple three-dimensional image color mapping networks. Here, the three-dimensional image color mapping network may be a three-dimensional lookup table (3D LUT).

[0061] In an exemplary embodiment of this disclosure, when obtaining a predicted color-enhanced image of a training image through a color enhancement network, the training image can first be input into a feature extraction network, and the weights corresponding to each of the plurality of three-dimensional image color mapping networks can be determined through the feature extraction network. Based on the plurality of three-dimensional image color mapping networks and the weights corresponding to the three-dimensional image color mapping networks, a weighted sum is performed to obtain the three-dimensional image color mapping information corresponding to the training image.

[0062] In an exemplary embodiment of this disclosure, the color enhancement network may include three three-dimensional image color mapping networks, which may be initialized as a standard three-dimensional image color mapping network and two all-zero three-dimensional image color mapping networks. The last layer of the feature extraction network may include three fully connected layers, the output of each fully connected layer being a numerical value, which corresponds to a weight.

[0063] For example, a color enhancement network can mainly consist of two parts: a feature extraction network (or feature extraction model) and multiple three-dimensional lookup tables. The feature extraction network (or feature extraction model) is used to learn the weights of each three-dimensional lookup table. The multiple three-dimensional lookup tables are used for image color mapping. The feature extraction network (or feature extraction model) can be a residual network or a lightweight network, or a self-built neural network with several layers, where the last layer of a convolutional neural network is transformed into a fully connected layer with an output of N, where N represents the number of three-dimensional lookup tables. Each three-dimensional lookup table can be represented by a single-layer network model, which only contains the parameters of the three-dimensional lookup table. As an example, the size of the three-dimensional lookup table can be 16×16×16, in which case the parameter layer of each three-dimensional lookup table is 3×16×16×16. In this disclosure, three three-dimensional lookup tables can be used. Because color adjustments such as contrast and brightness are required, and it is necessary to brighten overly dark areas and darken overexposed areas, the three three-dimensional lookup tables are initialized as one standard three-dimensional lookup table (i.e., a three-dimensional lookup table where the input equals the output) and two three-dimensional lookup tables with all zeros. A standard 3D lookup table is used for color adjustments such as contrast and brightness. Two 3D lookup tables with all zeros are used to adjust dark and bright areas, respectively. Finally, the weights learned by the feature extraction network (or feature extraction model) are weighted together with each 3D lookup table to obtain the final 3D lookup table. The predicted color-enhanced image can then be obtained by looking up the table.

[0064] In step S103, the training image is processed by color mapping based on the three-dimensional image color mapping information to generate a predicted color-enhanced image.

[0065] In step S104, the texture difference between the predicted color-enhanced image and the target image is determined, and the texture loss is determined based on the texture difference.

[0066] In an exemplary embodiment of this disclosure, when determining the texture difference between the predicted color-enhanced image and the target image, the texture features of the predicted color-enhanced image and the texture features of the target image can be extracted first, and then the texture difference can be determined based on the texture features of the predicted color-enhanced image and the texture features of the target image.

[0067] In an exemplary embodiment of this disclosure, when determining texture loss based on texture differences, mean square error can be determined based on texture differences and used as texture loss.

[0068] For example, the output of the color enhancement network (i.e., the predicted color-enhanced image) and the labeled image are both passed through a pre-trained convolutional neural network for extracting texture features, and the texture difference between the predicted color-enhanced image and the target image of the training image is calculated.

[0069] To prevent excessive color enhancement and significant deviation from the actual image, for example, the training image and the predicted color-enhanced image can be converted to the luminance-blue-chrominance-red-chrominance (YUV) color space. The difference between the luminance channel values ​​of the training image and the predicted color-enhanced image can be calculated and added to the prediction loss to prevent the color enhancement network from overexposing the image.

[0070] In step S105, the loss of the preset attribute corresponding to the color mapping information of the three-dimensional image is determined.

[0071] In an exemplary embodiment of this disclosure, the loss of the three-dimensional image color mapping information corresponding to the preset attribute includes the loss characterizing the smoothness attribute of the three-dimensional image color mapping information and the loss characterizing the monotonicity attribute of the three-dimensional image color mapping information.

[0072] In step S106, the parameters of the color enhancement network are adjusted based on the texture loss and the loss corresponding to the preset attributes of the 3D image color mapping information, so as to train the color enhancement network.

[0073] Figure 2 A flowchart illustrating a training method for a color enhancement network according to an exemplary embodiment of the present disclosure is shown.

[0074] like Figure 2As shown, in step 1, the original image (hereinafter referred to as the original image) and the target image (annotated image) of the original image are obtained for training.

[0075] In step 2, the original image obtained in step 1 is downsampled.

[0076] In step 3, data augmentation is performed on the downsampled image obtained in step 2.

[0077] In step 4, the data-enhanced image obtained in step 3 is subjected to data normalization processing.

[0078] In step 5, the image obtained from the data normalization process in step 4 is input into the color enhancement network (i.e., Figure 2 In the enhanced network (of the network).

[0079] In step 6, the color enhancement network (enhancement network) generates a 3D lookup table (i.e., 3D image color mapping information).

[0080] In step 7, based on the 3D lookup table generated in step 6 (i.e., 3D image color mapping information), the predicted color-enhanced image of the original image is obtained by looking up the table (i.e., ...). Figure 2 (Network prediction image in the image).

[0081] In step 8, the prediction loss is calculated using a loss function based on the original image, the target image of the original image, and the predicted color-enhanced image.

[0082] In step 9, the parameters of the color enhancement network are adjusted based on the calculated prediction loss.

[0083] Figure 3 A flowchart illustrating a color enhancement method according to an exemplary embodiment of the present disclosure is shown.

[0084] Reference Figure 3 In step S301, the video to be processed is acquired.

[0085] In step S302, the current video frame to be processed in the video to be processed is input into the color enhancement network to generate three-dimensional image color mapping information corresponding to the current video frame to be processed. Here, the color enhancement network is trained according to the training method of the color enhancement network in the exemplary embodiment of this disclosure.

[0086] In an exemplary embodiment of this disclosure, before inputting the current video frame to be processed into the color enhancement network to generate the three-dimensional image color mapping information corresponding to the current video frame to be processed, the grayscale histogram of the current video frame to be processed may be determined first, and then the Bach distance between the grayscale histogram of the current video frame to be processed and the grayscale histogram of the previous frame of the current video frame to be processed may be determined.

[0087] In an exemplary embodiment of this disclosure, when inputting the current video frame to be processed into the color enhancement network to generate the three-dimensional image color mapping information corresponding to the current video frame to be processed, the step of inputting the current video frame to be processed into the color enhancement network to generate the three-dimensional image color mapping information corresponding to the current video frame to be processed can be performed when the Bach distance is greater than a threshold.

[0088] In an exemplary embodiment of this disclosure, when the current video frame to be processed is input into the color enhancement network to generate the three-dimensional image color mapping information corresponding to the current video frame to be processed, if the Bach distance is less than a threshold, the weight of the current video frame to be processed can be updated according to the Bach distance, the weight of the current video frame to be processed, and the weight of the previous frame of the current video frame to be processed. Furthermore, the three-dimensional image color mapping information of the current video frame to be processed can be generated based on the three-dimensional image color mapping information of the previous frame of the current video frame to be processed and the weight of the current video frame to be processed.

[0089] In the exemplary embodiments of this disclosure, the color enhancement network has a fast computation speed, enabling real-time calculation and output of results for each frame of the image. However, calculating a three-dimensional lookup table for each frame can cause video brightness flicker. To address this flicker issue, this disclosure proposes a strategy for slow brightness changes in the video stream. First, the grayscale histogram of the current frame is calculated and compared with the histogram of the previous frame, primarily comparing the Bach distance between the two histograms, for example:

[0090]

[0091] Here, H1 represents the grayscale histogram of the previous frame, H2 represents the grayscale histogram of the current frame, and N represents the histogram size. In this disclosure, the histogram size is 255, i.e., N = 255. The smaller the similarity of the histograms, the larger the Bach distance. If the Bach distance is greater than the threshold, it is considered that the two consecutive frames change drastically, possibly due to scene transitions or sudden brightness changes. In this case, the color enhancement network in this disclosure is directly used to calculate the three-dimensional lookup table, and the current frame obtains a completely new three-dimensional lookup table. If the Bach distance is within the threshold range, it is considered that the two consecutive frames use a slow change method to perform a slow transition of the three-dimensional lookup table. During the slow transition, the weight of the current video frame to be processed (hereinafter referred to as the current frame) is calculated according to W2' = W1 × d(H1, H2) + W2 × d(H1, H2) (i.e., the weight of each of the multiple three-dimensional lookup tables used to calculate the final three-dimensional lookup table). Here, W1 represents the weight of the previous frame (hereinafter referred to as the previous frame), and W2 represents the weight of the current frame. W2' represents the updated weight of the current frame. Here, W2' includes the weight of each of the multiple 3D lookup tables used to calculate the final 3D lookup table. After calculating W2', W2' (i.e., the weight of each of the multiple 3D lookup tables used to calculate the final 3D lookup table) is weighted and summed with the multiple 3D lookup tables used to calculate the final 3D lookup table of the previous frame to obtain the final 3D lookup table of the current frame. That is, after obtaining the new final 3D lookup table, the output image frame is generated according to the lookup method. The method of slow transition can effectively solve the video flickering phenomenon caused by sudden changes in the scene or too fast movement.

[0092] In step S303, color enhancement processing is performed on the current video frame to be processed based on the three-dimensional image color mapping information corresponding to the current video frame to be processed, so as to obtain the color-enhanced video frame corresponding to the current video frame to be processed.

[0093] The above has been combined Figures 1 to 3 A method for training a color enhancement network and a color enhancement method according to exemplary embodiments of the present disclosure have been described. Hereinafter, reference will be made to... Figure 4 and Figure 5 A training apparatus and its units for a color enhancement network, and a color enhancement apparatus and its units, according to exemplary embodiments of the present disclosure, will be described.

[0094] Figure 4 A block diagram of a training apparatus for a color enhancement network according to an exemplary embodiment of the present disclosure is shown.

[0095] Reference Figure 4 The training device for the color enhancement network includes an image set acquisition unit 41, a mapping information generation unit 42, a color enhancement unit 43, a first loss calculation unit 44, a second loss calculation unit 45, and a parameter adjustment unit 46.

[0096] The image set acquisition unit 41 is configured to acquire a training image set. Here, the training image set includes training images and target images corresponding to the training images.

[0097] In an exemplary embodiment of this disclosure, the training apparatus for the color enhancement network may further include an image set processing unit (not shown), configured to: downsample the training image to obtain a downsampled image; perform data enhancement on the downsampled image to obtain a data-enhanced training image; perform data normalization on the training image set based on the data-enhanced training image, and use the normalized training image set as the training image set.

[0098] In an exemplary embodiment of this disclosure, the image set processing unit is configured to: determine the mean and variance of the data-enhanced training images in the training image set; and normalize the distribution of the data-enhanced training images in the training image set to a Gaussian distribution based on the mean and variance.

[0099] In an exemplary embodiment of this disclosure, the image set acquisition unit 41 may be configured to: acquire training images; perform preset adjustments on the training images to obtain target images corresponding to the training images; and obtain a training image set based on the training images and the target images corresponding to the training images.

[0100] In an exemplary embodiment of this disclosure, the image set acquisition unit 41 may be configured to: adjust at least one of the exposure, saturation, and contrast of the training image when the training image is a portrait image; adjust the preset details of the training image when the training image is a landscape image; adjust the dark curve of the training image when the training image is a low-light image to brighten the dark areas of the training image; and adjust the bright curve of the training image when the training image is an overexposed image to darken the bright areas of the training image.

[0101] The mapping information generation unit 42 is configured to generate three-dimensional image color mapping information corresponding to the training image through a color enhancement network.

[0102] In exemplary embodiments of this disclosure, the color enhancement network may include a feature extraction network and multiple three-dimensional image color mapping networks.

[0103] In an exemplary embodiment of this disclosure, the mapping information generation unit 42 may be configured to: input a training image into a feature extraction network, determine the weights of each of the multiple three-dimensional image color mapping networks through the feature extraction network, and perform a weighted summation based on the multiple three-dimensional image color mapping networks and the weights of the corresponding three-dimensional image color mapping networks to obtain the three-dimensional image color mapping information corresponding to the training image.

[0104] In an exemplary embodiment of this disclosure, the color enhancement network may include three three-dimensional image color mapping networks, which may be initialized as one standard three-dimensional image color mapping network and two all-zero three-dimensional image color mapping networks.

[0105] The color enhancement unit 43 is configured to perform color mapping processing on the training image based on the color mapping information of the three-dimensional image to generate a predicted color-enhanced image.

[0106] The first loss calculation unit 44 is configured to determine the texture difference between the predicted color-enhanced image and the target image, and to determine the texture loss based on the texture difference.

[0107] In an exemplary embodiment of this disclosure, the first loss calculation unit 44 may be configured to: extract texture features of the predicted color-enhanced image and texture features of the target image respectively; and determine texture differences based on the texture features of the predicted color-enhanced image and texture features of the target image.

[0108] In an exemplary embodiment of this disclosure, the first loss calculation unit 44 may be configured to: determine the mean square error based on texture differences, and use the mean square error as texture loss.

[0109] The second loss calculation unit 45 is configured to determine the loss of the preset attribute corresponding to the color mapping information of the three-dimensional image.

[0110] In an exemplary embodiment of this disclosure, the loss of the three-dimensional image color mapping information corresponding to a preset attribute may include the loss characterizing the smoothness attribute of the three-dimensional image color mapping information and the loss characterizing the monotonicity attribute of the three-dimensional image color mapping information.

[0111] The parameter adjustment unit 46 is configured to adjust the parameters of the color enhancement network based on the texture loss and the loss corresponding to the preset attributes of the 3D image color mapping information in order to train the color enhancement network.

[0112] Figure 5 A block diagram of a color enhancement device according to an exemplary embodiment of the present disclosure is shown.

[0113] Reference Figure 5 The color enhancement device includes a video acquisition unit 51, a mapping information generation unit 52, and a color enhancement processing unit 53.

[0114] The video acquisition unit 51 is configured to acquire the video to be processed.

[0115] The mapping information generation unit 52 is configured to input the current video frame to be processed from the video to be processed into the color enhancement network to generate three-dimensional image color mapping information corresponding to the current video frame to be processed. Here, the color enhancement network is trained according to the training method of the color enhancement network of the exemplary embodiment of this disclosure.

[0116] In an exemplary embodiment of this disclosure, the color enhancement apparatus may further include a distance determination unit (not shown), configured to: determine the grayscale histogram of the current video frame to be processed; and determine the Barcol distance between the grayscale histogram of the current video frame to be processed and the grayscale histogram of the previous frame of the current video frame to be processed.

[0117] In an exemplary embodiment of this disclosure, the mapping information generation unit 52 may be configured to: when the Bach distance is greater than a threshold, perform the step of inputting the current video frame to be processed in the video to be processed into the color enhancement network to generate the three-dimensional image color mapping information corresponding to the current video frame to be processed.

[0118] In an exemplary embodiment of this disclosure, the mapping information generation unit 52 may be configured to: when the Bach distance is less than a threshold, update the weight of the current video frame to be processed according to the Bach distance, the weight of the current video frame to be processed, and the weight of the previous frame of the current video frame to be processed; and generate the three-dimensional image color mapping information of the current video frame to be processed based on the three-dimensional image color mapping information of the previous frame of the current video frame to be processed and the weight of the current video frame to be processed.

[0119] The color enhancement processing unit 53 is configured to perform color enhancement processing on the current video frame to be processed based on the three-dimensional image color mapping information corresponding to the current video frame to be processed, so as to obtain the color-enhanced video frame corresponding to the current video frame to be processed.

[0120] Regarding the apparatus in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0121] The above has been combined Figure 4 and Figure 5 The training apparatus and color enhancement apparatus of the color enhancement network according to exemplary embodiments of the present disclosure have been described. Next, in conjunction with... Figure 6 An electronic device according to exemplary embodiments of the present disclosure will be described.

[0122] Figure 6 This is a block diagram of an electronic device 600 according to an exemplary embodiment of the present disclosure.

[0123] Reference Figure 6The electronic device 600 includes at least one memory 601 and at least one processor 602. The at least one memory 601 stores a set of computer-executable instructions. When the set of computer-executable instructions is executed by the at least one processor 602, a training method and a color enhancement method for a color enhancement network according to exemplary embodiments of the present disclosure are executed.

[0124] In exemplary embodiments of this disclosure, electronic device 600 may be a PC, tablet device, personal digital assistant, smartphone, or other device capable of executing the aforementioned set of instructions. Here, electronic device 600 is not necessarily a single electronic device, but may be a collection of any devices or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. Electronic device 600 may also be part of an integrated control system or system manager, or may be configured to interconnect with a portable electronic device locally or remotely (e.g., via wireless transmission) through an interface.

[0125] In electronic device 600, processor 602 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, processor may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, etc.

[0126] The processor 602 can execute instructions or code stored in the memory 601, which can also store data. Instructions and data can also be sent and received via a network through a network interface device, which can employ any known transmission protocol.

[0127] The memory 601 may be integrated with the processor 602, for example, by placing RAM or flash memory within an integrated circuit microprocessor. Alternatively, the memory 601 may include a separate device, such as an external disk drive, a storage array, or other storage device that can be used by any database system. The memory 601 and the processor 902 may be operatively coupled, or may communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor 602 to read files stored in the memory.

[0128] In addition, the electronic device 600 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the electronic device 600 can be interconnected via a bus and / or network.

[0129] According to exemplary embodiments of this disclosure, a computer-readable storage medium including instructions is also provided, such as a memory 601 including instructions, which can be executed by a processor 602 of the device 600 to perform the above-described method. Optionally, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0130] According to exemplary embodiments of the present disclosure, a computer program product may also be provided, the computer program product including a computer program / instruction, which, when executed by a processor, implements a training method for a color enhancement network and a color enhancement method according to exemplary embodiments of the present disclosure.

[0131] The above has been referred to Figures 1 to 6 A method and apparatus for training a color enhancement network, and a color enhancement method and apparatus according to exemplary embodiments of the present disclosure are described. However, it should be understood that: Figure 4 and Figure 5 The training device and its units of the color enhancement network shown can be configured as software, hardware, firmware, or any combination thereof to perform specific functions. Figure 6 The electronic device shown is not limited to the components shown above, but some components may be added or removed as needed, and the above components may also be combined.

[0132] According to the training method and apparatus for the color enhancement network disclosed herein, a training image set is acquired, wherein the training image set includes training images and target images corresponding to the training images; three-dimensional image color mapping information corresponding to the training images is generated through the color enhancement network; color mapping processing is performed on the training images based on the three-dimensional image color mapping information to generate predicted color-enhanced images; texture differences between the predicted color-enhanced images and the target images are determined, and texture loss is determined based on the texture differences; loss of a preset attribute corresponding to the three-dimensional image color mapping information is determined; and the parameters of the color enhancement network are adjusted based on the texture loss and the loss of the preset attribute corresponding to the three-dimensional image color mapping information to train the color enhancement network, thereby improving the color enhancement effect of the color enhancement network.

[0133] Furthermore, according to the color enhancement method and apparatus of this disclosure, firstly, a color enhancement network trained by the color enhancement network training method according to the exemplary embodiments of this disclosure generates three-dimensional image color mapping information of the current video frame to be processed in the video to be processed, and then an enhanced current video frame to be processed is generated based on the three-dimensional image color mapping information of the current video frame to be processed in the video to be processed, thereby improving the color enhancement effect.

[0134] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0135] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for training a color enhancement network, the method comprising: The method comprises the following steps: obtaining a training image set, wherein the training image set comprises a training image and a target image corresponding to the training image; generating three-dimensional image color mapping information corresponding to the training image through a color enhancement network, wherein the color enhancement network comprises a feature extraction network and a plurality of three-dimensional image color mapping networks, and the feature extraction network is used to determine the weight of each three-dimensional image color mapping network in the plurality of three-dimensional image color mapping networks; performing color mapping processing on the training image based on the three-dimensional image color mapping information to generate a predicted color enhanced image; determining the texture difference between the predicted color enhanced image and the target image, and determining the texture loss based on the texture difference; determining the loss of the three-dimensional image color mapping information corresponding to the preset attribute; obtaining the luminance channel value of the training image in the luminance blue chroma red chroma color space as the luminance channel value of the training image, and obtaining the luminance channel value of the predicted color enhanced image in the luminance blue chroma red chroma color space as the luminance channel value of the predicted color enhanced image; calculating the luminance difference between the luminance channel value of the training image and the luminance channel value of the predicted color enhanced image; adjusting the parameters of the color enhancement network based on the texture loss, the luminance difference and the loss of the three-dimensional image color mapping information corresponding to the preset attribute to train the color enhancement network, wherein the loss of the three-dimensional image color mapping information corresponding to the preset attribute comprises a loss representing the smoothness of the three-dimensional image color mapping information and a loss representing the monotonicity of the three-dimensional image color mapping information.

2. The method of Claim 1, wherein, The determination of the texture difference between the predicted color enhanced image and the target image comprises: extracting the texture features of the predicted color enhanced image and the texture features of the target image respectively; determining the texture difference based on the texture features of the predicted color enhanced image and the texture features of the target image.

3. The method of Claim 1, wherein, The determination of the texture loss based on the texture difference comprises: determining a mean square error based on the texture difference, and taking the mean square error as the texture loss.

4. The method of Claim 1, wherein, Before the generation of the three-dimensional image color mapping information corresponding to the training image through the color enhancement network, the method further comprises the following steps: down-sampling the training image to obtain a down-sampled image; performing data enhancement on the down-sampled image to obtain a data-enhanced training image; performing data normalization on the training image set based on the data-enhanced training image, and taking the normalized training image set as the training image set.

5. The method of Claim 4, wherein, The data normalization on the training image set based on the data-enhanced training image comprises: determining the mean and variance of the data-enhanced training image in the training image set; normalizing the distribution of the data-enhanced training image in the training image set to a Gaussian distribution based on the mean and the variance.

6. The method of Claim 4, wherein, The obtaining of the training image set comprises: obtaining the training image; performing a preset adjustment on the training image to obtain a target image corresponding to the training image; Based on the training image and the target image corresponding to the training image, the training image set is obtained.

7. The method of Claim 6, wherein, The preset adjustment on the training image comprises: When the training image is a portrait image, at least one of exposure, saturation and contrast of the training image is adjusted; When the training image is a landscape image, preset details of the training image are adjusted; When the training image is a dark light image, a dark part curve of the training image is adjusted so that the dark part of the training image is brightened. When the training image is an overexposed image, a bright part curve of the training image is adjusted so that the bright part of the training image is darkened.

8. The method of Claim 1, wherein the color enhancement network is trained by: It is characterized in that, The color enhancement network comprises three three-dimensional image color mapping networks, and the three three-dimensional image color mapping networks are initialized as a standard three-dimensional image color mapping network and two all-zero three-dimensional image color mapping networks. The color enhancement network comprises three three-dimensional image color mapping networks, and the three three-dimensional image color mapping networks are initialized as a standard three-dimensional image color mapping network and two all-zero three-dimensional image color mapping networks. It comprises:

9. The method of Claim 8, wherein, Obtaining a to-be-processed video; 10. A color enhancement method characterized by, inputting a current to-be-processed video frame in the to-be-processed video into a color enhancement network to generate three-dimensional image color mapping information corresponding to the current to-be-processed video frame; based on the three-dimensional image color mapping information corresponding to the current to-be-processed video frame, performing color enhancement processing on the current to-be-processed video frame to obtain a color-enhanced video frame corresponding to the current to-be-processed video frame, The color enhancement network is trained according to the color enhancement network training method of any one of claims 1-9. Before inputting the current to-be-processed video frame in the to-be-processed video into the color enhancement network to generate the three-dimensional image color mapping information corresponding to the current to-be-processed video frame, the method further comprises: determining the gray histogram of the current to-be-processed video frame; 11. The color enhancement method of claim 10, wherein, determining the Bhattacharyya distance between the gray histogram of the current to-be-processed video frame and the gray histogram of the previous frame of the current to-be-processed video frame. inputting a current to-be-processed video frame in the to-be-processed video into a color enhancement network to generate three-dimensional image color mapping information corresponding to the current to-be-processed video frame, comprising: When the Bhattacharyya distance is greater than a threshold, the step of inputting the current to-be-processed video frame in the to-be-processed video into the color enhancement network to generate the three-dimensional image color mapping information corresponding to the current to-be-processed video frame is executed.

12. The color enhancement method of claim 11, wherein, inputting a current to-be-processed video frame in the to-be-processed video into a color enhancement network to generate three-dimensional image color mapping information corresponding to the current to-be-processed video frame, comprising: When the Bhattacharyya distance is greater than a threshold, the step of inputting the current to-be-processed video frame in the to-be-processed video into the color enhancement network to generate the three-dimensional image color mapping information corresponding to the current to-be-processed video frame is executed.

13. The color enhancement method of claim 12, wherein, ​ When the Bhattacharyya distance is less than the threshold value, a weight of the current to-be-processed video frame is updated according to the Bhattacharyya distance, the weight of the current to-be-processed video frame, and a weight of a previous frame of the current to-be-processed video frame; The three-dimensional image color mapping information of the current to-be-processed video frame is generated based on the three-dimensional image color mapping information of the previous frame of the current to-be-processed video frame and the weight of the current to-be-processed video frame.

14. A training apparatus of a color enhancement network, characterized by, Comprise: An image set acquisition unit configured to acquire a training image set, wherein the training image set comprises a training image and a target image corresponding to the training image; A mapping information generation unit configured to generate three-dimensional image color mapping information corresponding to the training image through a color enhancement network, wherein the color enhancement network comprises a feature extraction network and a plurality of three-dimensional image color mapping networks, and weights of each three-dimensional image color mapping network in the plurality of three-dimensional image color mapping networks are determined through the feature extraction network; A color enhancement unit configured to perform color mapping processing on the training image based on the three-dimensional image color mapping information to generate a predicted color enhanced image; A first loss calculation unit configured to determine a texture difference between the predicted color enhanced image and the target image, and determine a texture loss based on the texture difference; A second loss calculation unit configured to determine a loss of a preset attribute corresponding to the three-dimensional image color mapping information, acquire a luminance channel value of the training image in a luminance blue chroma red chroma color space as a luminance channel value of the training image, and acquire a luminance channel value of the predicted color enhanced image in the luminance blue chroma red chroma color space as a luminance channel value of the predicted color enhanced image, and calculate a luminance difference between the luminance channel value of the training image and the luminance channel value of the predicted color enhanced image; and A parameter adjustment unit configured to adjust parameters of the color enhancement network based on the texture loss, the luminance difference, and the loss of the preset attribute corresponding to the three-dimensional image color mapping information, to train the color enhancement network, wherein the loss of the preset attribute corresponding to the three-dimensional image color mapping information comprises a loss representing a smooth attribute of the three-dimensional image color mapping information and a loss representing a monotonic attribute of the three-dimensional image color mapping information.

15. The device for training a color enhancement network according to claim 14, wherein, The first loss calculation unit is configured to: extract texture features of the predicted color enhanced image and texture features of the target image, respectively; determine the texture difference based on the texture features of the predicted color enhanced image and the texture features of the target image.

16. The device for training a color enhancement network according to claim 14, wherein, The first loss calculation unit is configured to: determine a mean square error based on the texture difference, and take the mean square error as the texture loss.

17. The device for training a color enhancement network according to claim 14, wherein, The device further comprises an image set processing unit configured to: down-sample the training image to obtain a down-sampled image; perform data enhancement on the down-sampled image to obtain a data-enhanced training image; perform data normalization on the training image set based on the data-enhanced training image, and take the normalized training image set as the training image set.

18. The training apparatus of a color enhancement network according to claim 17, wherein, The image set processing unit is configured to: determine the mean and variance of the data-augmented training images in the training image set; normalize the distribution of the data-augmented training images in the training image set to a Gaussian distribution based on the mean and the variance.

19. The device for training a color enhancement network according to claim 17, wherein, The image set obtaining unit is configured to: obtain the training images; perform a preset adjustment on the training images to obtain target images corresponding to the training images; obtain the training image set based on the training images and the target images corresponding to the training images.

20. The device for training a color enhancement network according to claim 19, wherein, The image set obtaining unit is configured to: when the training image is a portrait image, adjust at least one of exposure, saturation, and contrast of the training image; when the training image is a landscape image, adjust a preset detail of the training image; when the training image is a dark-light image, adjust a dark part curve of the training image so that the dark part of the training image becomes brighter; when the training image is an overexposed image, adjust a bright part curve of the training image so that the bright part of the training image becomes darker.

21. The device of claim 14, wherein, The color enhancement network includes a feature extraction network and a plurality of three-dimensional image color mapping networks, wherein the mapping information generation unit is configured to: input the training image into the feature extraction network to determine a weight corresponding to each three-dimensional image color mapping network in the plurality of three-dimensional image color mapping networks through the feature extraction network; perform weighted summation based on the plurality of three-dimensional image color mapping networks and the weights corresponding to the three-dimensional image color mapping networks to obtain three-dimensional image color mapping information corresponding to the training image.

22. The device for training a color enhancement network according to claim 21, wherein, The color enhancement network includes three three-dimensional image color mapping networks, which are initialized as a standard three-dimensional image color mapping network and two all-zero three-dimensional image color mapping networks.

23. A color enhancement apparatus, characterized by, It includes: a video obtaining unit configured to obtain a to-be-processed video; a mapping information generation unit configured to input a current to-be-processed video frame in the to-be-processed video into a color enhancement network to generate three-dimensional image color mapping information corresponding to the current to-be-processed video frame; a color enhancement processing unit configured to perform color enhancement processing on the current to-be-processed video frame based on the three-dimensional image color mapping information corresponding to the current to-be-processed video frame to obtain a color-enhanced video frame corresponding to the current to-be-processed video frame, wherein the color enhancement network is trained according to the color enhancement network training method in any one of claims 1-9.

24. The color enhancement apparatus of claim 23, wherein The device further includes a distance determination unit configured to: determine a gray histogram of the current to-be-processed video frame; determine the Bhattacharyya distance between the gray histogram of the current to-be-processed video frame and a gray histogram of a previous frame of the current to-be-processed video frame.

25. The color enhancement apparatus of claim 24, wherein The mapping information generation unit is configured to: when the Bhattacharyya distance is greater than a threshold, perform the step of inputting the current to-be-processed video frame in the to-be-processed video into the color enhancement network to generate three-dimensional image color mapping information corresponding to the current to-be-processed video frame.

26. The color enhancement apparatus of claim 25, wherein The mapping information generation unit is configured to: When the Bhattacharyya distance is less than the threshold value, updating the weight of the current video frame according to the Bhattacharyya distance, the weight of the current video frame and the weight of a previous frame of the current video frame; generating the three-dimensional image color mapping information of the current video frame based on the three-dimensional image color mapping information of the previous frame of the current video frame and the weight of the current video frame.

27. An electronic device, comprising: comprise: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the method of any one of claims 1 to 13.

28. A computer readable storage medium storing a computer program, wherein the computer program comprises instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1-27. When the computer program is executed by the processor of the electronic device, the electronic device is caused to perform the method of any one of claims 1 to 13.

29. A computer program product comprising a computer program, characterised in that, When the computer program is executed by the processor, the method of any one of claims 1 to 13 is implemented.

Images