Method for image color correction, training method and device of neural network model

Abstract

The present disclosure provides a method for image color correction, a training method and device of a neural network model. The method comprises: obtaining an original image and an enhanced image corresponding to the original image, the enhanced image being obtained by image enhancement on the original image; and generating a color offset tensor based on the original image and the enhanced image, the color offset tensor being capable of being used for correcting the color of the enhanced image; generating a multi-scale color consistency loss function based on the original image, the enhanced image and the color offset tensor; and updating the neural network model based on the multi-scale color consistency loss function to obtain a trained neural network model. Thus, by training an end-to-end neural network for correcting color distortion of an image generated in an image enhancement process, the color distortion problem of an output image of a current deep learning image enhancement model is overcome.

Description

Technical Field

[0001] This disclosure relates to the field of training neural network models for image color correction, and more particularly to the fields of image enhancement and image color correction. In particular, it relates to a method for image color correction, a method for training neural network models, an apparatus, electronic circuits, electronic devices, computer-readable storage media, and computer program products. Background Technology

[0002] In deep learning-based image enhancement tasks, such as video super-resolution, image denoising, and artifact removal, the network output image often exhibits color distortion. This color cast not only affects visual quality but also reduces the accuracy and robustness of downstream visual tasks (such as object detection and semantic segmentation). Most existing image enhancement methods do not introduce explicit color constraint mechanisms, and even when color-related loss functions are introduced, it is difficult to effectively model nonlinear color casts in complex scenes. Therefore, a color cast correction scheme that can adapt to various enhancement models and is trainable end-to-end is urgently needed. Summary of the Invention

[0003] This disclosure provides a method, apparatus, electronic circuit, electronic device, computer-readable storage medium, and computer program product for training a neural network model for image color correction, as well as a method for image color correction.

[0004] According to one aspect of this disclosure, a method for training a neural network model for image color correction is provided. The method includes: obtaining an original image and an enhanced image corresponding to the original image, wherein the enhanced image is obtained by image enhancement of the original image; generating a color offset tensor based on the original image and the enhanced image, wherein the color offset tensor can be used to correct the color of the enhanced image; generating a multi-scale color consistency loss function based on the original image, the enhanced image, and the color offset tensor; and updating the neural network model based on the multi-scale color consistency loss function to obtain a trained neural network model.

[0005] In some embodiments, the method further includes: generating a color difference map based on the original image and the enhanced image; and inputting the color difference map into a neural network model for training together with the original image and the enhanced image to generate a color shift tensor.

[0006] In some embodiments, generating a multi-scale color consistency loss function based on the original image, the enhanced image, and the color shift tensor includes: performing multi-scale decomposition on the original image to obtain multiple original image layers; correcting the enhanced image using the color shift tensor to obtain a corrected enhanced image; performing multi-scale decomposition on the corrected enhanced image to obtain multiple corrected enhanced image layers; and generating a multi-scale color consistency loss function based on the multiple original image layers and the multiple corrected enhanced image layers.

[0007] In some embodiments, correcting the enhanced image using a color offset tensor to obtain a corrected enhanced image includes: applying an edge detection algorithm to the enhanced image to generate a structure map of the enhanced image; generating a structure-sensitive gated mask based on the structure map; applying the structure-sensitive gated mask to the color offset tensor to obtain an updated color offset tensor; and correcting the enhanced image using the updated color offset tensor to obtain the corrected enhanced image.

[0008] In some embodiments, generating a structure-sensitive gated mask based on a structure graph includes applying a sigmoid function to the structure graph to generate the structure-sensitive gated mask.

[0009] In some embodiments, the multi-scale color consistency loss function is used to indicate the difference between multiple original image layers and multiple corrected enhanced image layers in one or more of the following: mean, variance, color histogram, and color space center.

[0010] In some embodiments, updating the neural network model based on the multi-scale color consistency loss function to obtain the trained neural network model includes: generating a comprehensive loss function based on the multi-scale color consistency loss function, the visual perception loss function, and the structure preservation loss function; and updating the neural network model based on the comprehensive loss function to obtain the trained neural network model.

[0011] In some embodiments, the neural network model is deployed after the image augmentation network.

[0012] In some embodiments, the neural network model can be trained alone or jointly trained with an image enhancement network.

[0013] According to one aspect of this disclosure, a method for image color correction is provided, comprising: obtaining an original image and an enhanced image corresponding to the original image, wherein the enhanced image is obtained by image enhancement of the original image; and inputting the original image and the enhanced image into a trained neural network model obtained using the method provided above in this disclosure to obtain a color-corrected enhanced image.

[0014] According to another aspect of this disclosure, a training apparatus for a neural network model for image color correction is provided. The apparatus includes: a first module configured to obtain an original image and an enhanced image corresponding to the original image, the enhanced image being obtained by image enhancement of the original image; a second module configured to generate a color offset tensor based on the original image and the enhanced image, the color offset tensor being capable of correcting the color of the enhanced image; a third module configured to generate a multi-scale color consistency loss function based on the original image, the enhanced image, and the color offset tensor; and a fourth module configured to update the neural network model based on the multi-scale color consistency loss function to obtain a trained neural network model.

[0015] According to another aspect of this disclosure, an electronic circuit is provided, including circuitry configured to perform the steps of the methods provided above.

[0016] According to another aspect of this disclosure, an electronic device is provided, including a processor; and a memory storing a program, the program including instructions that, when executed by the processor, cause the processor to perform the steps of the method provided above in this disclosure.

[0017] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, causes the processor to perform the methods provided above in this disclosure.

[0018] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, causes the processor to perform the methods provided above in this disclosure.

[0019] According to one or more embodiments of this disclosure, the color distortion problem of the output image of current deep learning image enhancement models is overcome by training an end-to-end neural network to correct color distortion of the image generated during the image enhancement process.

[0020] These and other aspects of this disclosure will be apparent from the embodiments described below, and will be elucidated with reference to the embodiments described below. Attached Figure Description

[0021] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of this disclosure. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0022] Figure 1This is a flowchart illustrating a training method 100 for a neural network model for image color correction according to an exemplary embodiment.

[0023] Figure 2 This is a flowchart illustrating step 160 of a training method 100 for a neural network model for image color correction according to an exemplary embodiment.

[0024] Figure 3 This is a flowchart illustrating step 260 of a training method 100 for a neural network model for image color correction according to an exemplary embodiment.

[0025] Figure 4 This is a flowchart illustrating step 180 of a training method 100 for a neural network model for image color correction according to an exemplary embodiment.

[0026] Figure 5 This is a block diagram illustrating a training apparatus for a neural network model for image color correction according to an exemplary embodiment.

[0027] Figure 6 This is a block diagram illustrating an exemplary electronic device that can be applied to exemplary embodiments. Detailed Implementation

[0028] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0029] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0030] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. As used herein, the term "multiple" means two or more, and the term "based on" should be interpreted as "at least partially based on". Furthermore, the terms "and / or" and "at least one of..." cover any one of the listed items and all possible combinations thereof.

[0031] The embodiments of this disclosure provide a training method for a neural network model for image color correction. By training a neural network to correct color distortion in images generated during image enhancement, the method overcomes the color distortion problem in the output images of current deep learning image enhancement models.

[0032] Figure 1 This is a flowchart illustrating a training method 100 for a neural network model for image color correction according to an exemplary embodiment.

[0033] like Figure 1 As shown in the embodiments of this disclosure, a training method 100 for a neural network model for image color correction is proposed, which may include the following steps: 120, obtaining an original image and an enhanced image corresponding to the original image, wherein the enhanced image is obtained by image enhancement of the original image; 140, generating a color offset tensor based on the original image and the enhanced image, wherein the color offset tensor can be used to correct the color of the enhanced image; 160, generating a multi-scale color consistency loss function based on the original image, the enhanced image, and the color offset tensor; and 180, updating the neural network model based on the multi-scale color consistency loss function to obtain a trained neural network model.

[0034] In step 120, the original image and the enhanced image corresponding to the original image are obtained. The enhanced image is obtained by image enhancement of the original image.

[0035] In the examples, the original image can be a manually captured image where the colors of the scene and objects are distributed naturally and realistically, conforming to visual naturalness. In some examples, the original image can serve as a reference for the true spectrum, and can be represented as I. GT .

[0036] In the example, the enhanced image exhibits color distortion caused by image enhancement.

[0037] In the example, while the enhanced image can enrich the details and textures of the original image compared to the original image, it is also prone to introducing color distortion. Color distortion can manifest in several ways: the image as a reddish, greenish, or yellowish tint; the colors of the scene and objects deviating from their true distribution, affecting visual naturalness; impacting the accuracy of downstream tasks such as object detection, semantic segmentation, and 3D reconstruction; and reducing the usability and credibility of the enhanced image.

[0038] In some embodiments, the neural network model for image color correction can be deployed after the image enhancement network. This neural network model can directly obtain the original image I_input input to the image enhancement network and the enhanced image I_output output by the image enhancement network corresponding to the original image I_input.

[0039] In some embodiments, the neural network model for image color correction can be trained alone or jointly trained with an image enhancement network.

[0040] In the example, the neural network model for image color correction can be trained independently and then deployed as a plug-in module after the image augmentation network, or it can be jointly trained end-to-end with the image augmentation network. For example, after completing end-to-end joint training, the original image can be input into the image augmentation network, and the color-corrected enhanced image can be output from the neural network model for image color correction.

[0041] Therefore, color correction can be performed plug-and-play without relying on a specific backbone network structure or modifying the original model (such as an image enhancement network), solving the current dilemma of poor generalization, difficulty in integration and deployment of customized color correction modules.

[0042] In some embodiments, method 100 may further include: generating a color difference map based on the original image and the enhanced image; and inputting the color difference map into a neural network model for training together with the original image and the enhanced image to generate a color offset tensor.

[0043] In the example, the color difference map D can be obtained by subtracting the original image from the enhanced image, as shown in the following formula:

[0044] D = I_output – I_input formula 1

[0045] Generally, input neural network models can adaptively extract color difference-related features during training, but training costs can also be reduced by directly feeding the color difference map D into the neural network model.

[0046] In the example, the original image and the enhanced image can be encoded via channel concatenation or dual-branch fusion and then input into the neural network model. For example, the original image I_input∈R with three channels can be used. H×W×3 and the enhanced image I_output∈R with three channels H×W×3 The formula for channel splicing is as follows:

[0047] I_concat = concat(I_output, I_input)∈R H×W×6 Formula 2

[0048] The function `concat()` is a channel concatenation function, and the resulting image `I_concat` has 6 channels.

[0049] Similarly, the original image I_input∈R with three channels can also be used. H×W×3 An enhanced image with three channels, I_output∈R H×W×3 and a color difference map D∈R with three channels H×W×3 The formula for channel splicing is as follows:

[0050] I _concat = concat(I_output, I_input, D)∈R H×W×9 Formula 3

[0051] The resulting concatenated image I_concat has 9 channels. This input format explicitly encodes the color cast pattern of the enhanced image and the color gamut difference between the original image and its true color gamut, thus reducing training costs.

[0052] In step 140, a color offset tensor is generated based on the original image and the enhanced image. The color offset tensor can be used to correct the color of the enhanced image.

[0053] In the example, a lightweight convolutional network F (e.g., three 3×3 convolutional layers + channel attention modules) can be constructed to generate the color offset matrix ΔC. The input of F is I_concat, and the output is the color offset matrix ΔC∈R. H×W×3 The formula is as follows:

[0054] ΔC=F(I_concat) Formula 4

[0055] In the example, the network F() can learn to map color casts back to the true color gamut, and its per-pixel offset output can compensate for local inconsistencies in color casts (such as yellowing of streetlights and bluishness of the sky). Therefore, the color offset tensor ΔC can be used to correct and enhance the colors of the image.

[0056] Using lightweight convolutional neural networks enables dynamic learning of color compensation per pixel / region, rather than relying on fixed formulas or lookups to obtain the color shift tensor. Furthermore, due to the small number of network parameters and low computational cost, it can be deployed on mobile devices, cameras, or real-time video enhancement systems, thereby improving color quality while maintaining real-time performance, making it suitable for resource-constrained applications.

[0057] In step 160, a multi-scale color consistency loss function is generated based on the original image, the enhanced image, and the color shift tensor.

[0058] In the example, the multi-scale color consistency loss function can be derived from the original image and the enhanced image after color correction via the color shift tensor, and is used to indicate the color difference between the original image and the enhanced image after color correction via the color shift tensor.

[0059] Figure 2 This is a flowchart illustrating a portion of the process 160 of a training method 100 for a neural network model for image color correction according to an exemplary embodiment.

[0060] refer to Figure 2 In some embodiments, step 160 may include the following steps: 220, performing multi-scale decomposition on the original image to obtain multiple original image layers; 240, correcting the enhanced image using a color offset tensor to obtain a corrected enhanced image; 260, performing multi-scale decomposition on the corrected enhanced image to obtain multiple corrected enhanced image layers; and 280, generating a multi-scale color consistency loss function based on the multiple original image layers and the multiple corrected enhanced image layers.

[0061] In step 220, the original image can be decomposed into multiple scales to obtain multiple original image layers.

[0062] In the example, multiple original image layers can be represented as , where s is the number of layers or scale corresponding to the image.

[0063] In the example, multi-scale decomposition can be Laplacian pyramid decomposition, or it can be filtering or transformation after downsampling (e.g., Gaussian filtering, wavelet transform, etc.) to obtain multiple original image layers with different resolutions (i.e., different scales).

[0064] In step 240, the enhanced image is corrected using the color offset tensor to obtain the corrected enhanced image.

[0065] In the example, the corrected enhanced image is obtained by adding the color offset tensor pixel by pixel to the enhanced image, as shown in the following formula:

[0066] I_corrected (x,y) = I_output (x,y) +ΔC(x,y)Formula 5

[0067] Where I_corrected is the corrected enhanced image, and (x,y) are the coordinates of the pixel in the image.

[0068] In step 260, the corrected enhanced image is decomposed into multiple scales to obtain multiple corrected enhanced image layers.

[0069] Figure 3 The figure illustrates a flowchart of a portion of the process 260 of a training method 100 for a neural network model for image color correction according to an exemplary embodiment.

[0070] refer to Figure 3 In some embodiments, step 260 may include the following steps: 320, using an edge detection algorithm on the enhanced image to generate a structure map of the enhanced image; 340, generating a structure-sensitive gating mask based on the structure map; 360, applying the structure-sensitive gating mask to the color offset tensor to obtain an updated color offset tensor; and 380, correcting the enhanced image using the updated color offset tensor to obtain a corrected enhanced image.

[0071] In step 320, an edge detection algorithm is used on the enhanced image to generate a structural map of the enhanced image.

[0072] In the example, the edge detection algorithm can be Sobel gradient algorithm, Prewitt gradient algorithm, Roberts cross gradient algorithm, feature attention or Canny edge detection, etc., to identify the structural regions of the enhanced image, and the generated structural map is represented as S.

[0073] In step 340, a structure-sensitive gating mask is generated based on the structure graph.

[0074] In some embodiments, the sigmoid function can be applied to the structure graph to generate a structure-sensitive gate mask, as shown in the following example formula:

[0075] Mask(x,y) = 1-sigmoid(S(x,y)) Formula 6

[0076] Where Mask is a structure-sensitive gated mask, (x,y) are the coordinates of the pixel in the image, and S is the structure graph.

[0077] In step 360, a structure-sensitive gating mask is applied to the color offset tensor to obtain an updated color offset tensor.

[0078] In step 380, the enhanced image is corrected using the updated color offset tensor to obtain the corrected enhanced image.

[0079] In the example, the color correction formula can be as follows:

[0080] I_corrected (x,y) = I_output (x,y) + Mask(x,y) Formula 7 for ΔC(x,y)

[0081] Where I_corrected is the corrected enhanced image, I_output is the enhanced image, (x,y) are the pixel coordinates in the image, and Mask(x,y) is the image output. ΔC(x,y) is the updated color offset tensor.

[0082] Therefore, since structure-sensitive gating masks can suppress color perturbations in structural regions, they can ensure that the structure of the enhanced image (e.g., texture or edges) is not destroyed, allowing for model training that only corrects color. Furthermore, by predicting the color shift matrix ΔC through the network and controlling it with structure-sensitive gating masks, the network can adaptively adjust the hue based on specific pixel locations and color contexts, rather than performing a fixed mapping, significantly improving the accuracy of color reversion.

[0083] In the example, multiple corrected and enhanced image layers can be represented as follows: .

[0084] In step 280, a multi-scale color consistency loss function is generated based on multiple original image layers and multiple corrected enhanced image layers.

[0085] In some embodiments, a multi-scale color consistency loss function can be used to indicate the differences between multiple original image layers and multiple corrected enhanced image layers in one or more of the following: mean, variance, color histogram, and color space center. An example of a multi-scale color consistency loss function is as follows:

[0086] Formula 8

[0087] in, For multi-scale color consistency loss function, () is the mean function. () represents the variance function, and 3 represents the number of scales for the example. For multiple original image layers, For multiple corrected and enhanced image layers, s represents the number of layers or scale of the image. In Equation 8, It can indicate the differences in mean and variance between multiple original image layers and multiple corrected enhanced image layers.

[0088] It should be understood that the above formula is for illustrative purposes only and not for limiting purposes. The multi-scale color consistency loss function can also indicate the differences between multiple original image layers and multiple corrected enhanced image layers in terms of color histograms and color space centers, etc., without limitation.

[0089] In step 180, the neural network model is updated based on the multi-scale color consistency loss function to obtain the trained neural network model.

[0090] In some embodiments, step 180 may include: 420, generating a comprehensive loss function based on a multi-scale color consistency loss function, a visual perception loss function, and a structure preservation loss function; and 440, updating the neural network model based on the comprehensive loss function to obtain a trained neural network model.

[0091] In step 420, the example's comprehensive loss function can be obtained by weighted summation of the multi-scale color consistency loss function, the visual perception loss function, and the structure preservation loss function, as shown in the following formula:

[0092] L = L_color + λ1 L_percept + λ2 L_struct Formula 9

[0093] Where L is the overall loss function, L_color is the multi-scale color consistency loss function, L_percept is the visual perception loss function, L_struct is the structure preservation loss function, and λ1 and λ2 can be hyperparameters. It should be understood that the overall loss function can also be calculated from the multi-scale color consistency loss function, the visual perception loss function, and the structure preservation loss function through other methods; this is not restricted here.

[0094] In some examples, the visual perception loss function can be VGG or LPIPS, and the structure preservation loss function can be SSIM + L1 or MS-SSIM + L1, etc., without restriction.

[0095] In step 440, the neural network model is updated based on the comprehensive loss function to obtain the trained neural network model.

[0096] Therefore, by constructing multiple joint loss functions, the global tone balance capability is improved while taking into account the color accuracy of local regions. This overcomes the limitations of traditional image enhancement methods and some deep learning models that rely solely on simple color loss functions (such as L1 / L2 loss) for constraint (these methods are difficult to model complex, nonlinear global and local color biases and cannot achieve accurate adaptive color restoration).

[0097] It should be noted that this disclosure does not limit the order of steps 120-180, nor does it limit the order of acquiring the various considerations. For example, in step 120, the acquisition of the original image and the enhanced image can be in any order, and this is not limited here.

[0098] Therefore, by training a neural network to correct color distortion generated during image enhancement, this method addresses the common and unpredictable color shifts, saturation anomalies, or tonal distortions in output images caused by the nonlinear transformation of the network structure and deviations from the training data in existing deep learning-based image enhancement, restoration, and generation models (such as super-resolution, denoising, and decompression artifact models). Furthermore, it alleviates the problem of decreased accuracy and robustness in downstream computer vision tasks (such as object detection, image segmentation, and classification) due to color distortion in enhanced images, thus improving the reliability of the entire vision processing pipeline.

[0099] According to one aspect of this disclosure, an image color correction method is also provided, comprising: obtaining an original image and an enhanced image corresponding to the original image, wherein the enhanced image is obtained by image enhancement of the original image; inputting the original image and the enhanced image into a trained neural network model obtained using any of the above methods to obtain a color-corrected enhanced image.

[0100] Embodiments of this disclosure also provide a training apparatus for a neural network model for image color correction.

[0101] Figure 5 This is a schematic block diagram illustrating a training apparatus 500 for a neural network model for image color correction according to an exemplary embodiment.

[0102] like Figure 5 As shown, in some embodiments, the apparatus 500 includes: a first module 510 configured to obtain an original image and an enhanced image corresponding to the original image, wherein the enhanced image is obtained by image enhancement of the original image; a second module 520 configured to generate a color offset tensor based on the original image and the enhanced image, wherein the color offset tensor can be used to correct the color of the enhanced image; a third module 530 configured to generate a multi-scale color consistency loss function based on the original image, the enhanced image, and the color offset tensor; and a fourth module 540 configured to update a neural network model based on the multi-scale color consistency loss function to obtain a trained neural network model.

[0103] The operations of the first module 510, the second module 520, the third module 530, and the fourth module 540 described above can be combined with... Figure 1 The operations of steps 120, 140, 160 and 180 are the same, so the details of each aspect will not be repeated here.

[0104] While specific functions have been discussed above with reference to specific modules, it should be noted that the functions of the modules discussed herein can be divided into multiple modules, and / or at least some functions of multiple modules can be combined into a single module. The specific actions performed by the modules discussed herein include the specific module itself performing the action, or alternatively, the specific module calling or otherwise accessing another component or module that performs the action (or performs the action in conjunction with the specific module). Therefore, a specific module performing an action can include the specific module performing the action itself and / or another module that performs the action, called or otherwise accessed by the specific module.

[0105] It should also be understood that the above regarding Figure 5 The described modules can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these modules can be implemented as computer program code / instructions configured to execute in one or more processors and stored in a computer-readable storage medium. Alternatively, these modules can be implemented as hardware logic / circuit. For example, these modules can be implemented together in a System on Chip (SoC). An SoC may include an integrated circuit chip (which includes one or more components in a processor (e.g., a Central Processing Unit (CPU), microcontroller, microprocessor, digital signal processor (DSP), etc.), memory, one or more communication interfaces, and / or other circuitry) and may optionally execute received program code and / or include embedded firmware to perform functions.

[0106] According to one aspect of this disclosure, an electronic circuit is also provided, which includes circuitry configured to perform the steps of any of the method embodiments described above.

[0107] According to one aspect of this disclosure, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory. The processor is configured to execute the computer program to implement the steps of any of the method embodiments described above.

[0108] According to one aspect of this disclosure, a non-transitory computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the method embodiments described above.

[0109] According to one aspect of this disclosure, a computer program product is also provided, which includes a computer program that, when executed by a processor, implements the steps of any of the method embodiments described above.

[0110] In the following text, combined with Figure 6 Illustrative examples describing such electronic devices, non-transitory computer-readable storage media, and computer program products.

[0111] Figure 6 An example configuration of an electronic device 600 that can be used to implement the methods described herein is shown.

[0112] Electronic device 600 can be a variety of different types of devices. Examples of electronic device 600 include, but are not limited to: desktop computers, server computers, laptop or netbook computers, mobile devices (e.g., tablet computers, cellular or other wireless phones (e.g., smartphones), notebook computers, mobile stations), wearable devices (e.g., glasses, watches), entertainment devices, etc.

[0113] Electronic device 600 may include at least one processor 602, memory 604, multiple communication interfaces 606, display device 608, other input / output (I / O) devices 610, and one or more mass storage devices 612 capable of communicating with each other, such as via system bus 614 or other suitable connections.

[0114] Processor 602 may be a single processing unit or multiple processing units, and all processing units may include single or multiple computing units or multiple cores. Processor 602 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that manipulates signals based on operating instructions. Among other capabilities, processor 602 may be configured to acquire and execute computer-readable instructions stored in memory 604, mass storage device 612, or other computer-readable media, such as program code of operating system 616, program code of application program 618, program code of other program 620, etc.

[0115] Memory 604 and mass storage device 612 are examples of computer-readable storage media for storing instructions that are executed by processor 602 to perform the various functions described above. For example, memory 604 may generally include both volatile and non-volatile memory (e.g., RAM, ROM, etc.). Furthermore, mass storage device 612 may generally include hard disk drives, solid-state drives, removable media, including external and removable drives, memory cards, flash memory, floppy disks, optical disks (e.g., CDs, DVDs), storage arrays, network-attached storage, storage area networks, etc. Both memory 604 and mass storage device 612 may be collectively referred to herein as memory or computer-readable storage media, and may be non-transitory media capable of storing computer-readable, processor-executable program instructions as computer program code, which may be executed by processor 602 as a specific machine configured to perform the operations and functions described in the examples herein.

[0116] Multiple programs may be stored on mass storage device 612. These programs include operating system 616, one or more application programs 618, other programs 620, and program data 622, and they may be loaded into memory 604 for execution. Examples of such application programs or program modules may include computer program logic (e.g., computer program code or instructions) of the methods described herein and / or other embodiments described herein.

[0117] Although Figure 6 The modules 616, 618, 620, and 622, or portions thereof, are illustrated as being stored in memory 604 of electronic device 600. However, modules 616, 618, 620, and 622 may be implemented using any form of computer-readable medium accessible by electronic device 600. As used herein, “computer-readable medium” includes at least two types of computer-readable media: computer-readable storage media and communication media.

[0118] Computer-readable storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, DVD, or other optical storage devices, magnetic cassettes, magnetic tapes, disk storage devices or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by electronic devices. In contrast, communication media can embody computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms. Computer-readable storage media as defined herein do not include communication media.

[0119] One or more communication interfaces 606 are used for exchanging data with other devices, such as via a network, direct connection, etc. Such communication interfaces can be one or more of the following: any type of network interface (e.g., a network interface card (NIC)), wired or wireless (such as IEEE 802.11 Wireless LAN (WLAN)) wireless interface, Wi-MAX interface, Ethernet interface, Universal Serial Bus (USB) interface, cellular network interface, Bluetooth. TM Interfaces, near field communication (NFC) interfaces, etc. Communication interface 606 can facilitate communication across various network and protocol types, including wired networks (e.g., LAN, cable, etc.) and wireless networks (e.g., WLAN, cellular, satellite, etc.), the Internet, etc. Communication interface 606 can also provide communication with external storage devices (not shown) such as storage arrays, network-attached storage, storage area networks, etc.

[0120] In some examples, a display device 608, such as a monitor, may be included for displaying information and images to the user. Other I / O devices 610 may be devices that receive various inputs from the user and provide various outputs to the user, and may include touch input devices, gesture input devices, cameras, keyboards, remote controls, mice, printers, audio input / output devices, and so on.

[0121] The technologies described herein can be supported by these various configurations of electronic device 600, and are not limited to specific examples of the technologies described herein. For example, the functionality can also be implemented wholly or partially on a “cloud” using a distributed system. A cloud includes and / or represents a platform for resources. The platform abstracts the underlying functionality of the cloud’s hardware (e.g., servers) and software resources. Resources may include applications and / or data that can be used when performing computational processing on a server remote from electronic device 600. Resources may also include services provided via the Internet and / or via subscriber networks such as cellular or Wi-Fi networks. The platform can abstract resources and functionality to connect electronic device 600 to other electronic devices. Therefore, the implementation of the functionality described herein can be distributed throughout the cloud. For example, the functionality may be implemented partly on electronic device 600 and partly through a platform that abstracts the functionality of the cloud.

Claims

1. A training method for a neural network model for image color correction, characterized in that, The method includes: The original image and the enhanced image corresponding to the original image are input into the neural network model through channel concatenation or dual-branch fusion encoding, wherein the enhanced image is obtained by image enhancement of the original image, and the output of the neural network model is the color-corrected enhanced image; Based on the original image and the enhanced image, a color offset tensor is generated, which can be used to correct the color of the enhanced image; Based on the original image, the enhanced image, and the color shift tensor, a multi-scale color consistency loss function is generated; and The neural network model is updated based on the multi-scale color consistency loss function to obtain the trained neural network model. The multi-scale color consistency loss function is generated based on the original image, the enhanced image, and the color shift tensor, including: The original image is decomposed into multiple scales to obtain multiple original image layers; The enhanced image is corrected using the color offset tensor to obtain the corrected enhanced image; The corrected enhanced image is decomposed into multiple scales to obtain multiple corrected enhanced image layers; and Based on the multiple original image layers and the multiple corrected enhanced image layers, a multi-scale color consistency loss function is generated.

2. The method according to claim 1, characterized in that, The method further includes: Based on the original image and the enhanced image, a color difference map is generated; and The color difference map is input into the neural network model and trained together with the original image and the enhanced image to generate the color offset tensor.

3. The method according to claim 1, characterized in that, The enhanced image is corrected using the color offset tensor to obtain the corrected enhanced image, which includes: An edge detection algorithm is used on the enhanced image to generate a structural map of the enhanced image; The sigmoid function is applied to the structure graph to generate a structure-sensitive gating mask; Applying the structure-sensitive gate mask to the color offset tensor yields an updated color offset tensor; The enhanced image is corrected using the updated color offset tensor to obtain the corrected enhanced image.

4. The method according to claim 1, characterized in that, The multi-scale color consistency loss function is used to indicate the difference between the plurality of original image layers and the plurality of corrected enhanced image layers in one or more of the following: mean, variance, color histogram, and color space center.

5. The method according to claim 1, characterized in that, The neural network model is updated based on the multi-scale color consistency loss function to obtain the trained neural network model, which includes: Based on the aforementioned multi-scale color consistency loss function, visual perception loss function, and structure preservation loss function, a comprehensive loss function is generated; and The neural network model is updated based on the comprehensive loss function to obtain the trained neural network model.

6. The method according to any one of claims 1 to 5, characterized in that, The neural network model is deployed after the image enhancement network.

7. The method according to claim 6, characterized in that, The neural network model can be trained independently or jointly with the image enhancement network.

8. A method for image color correction, characterized in that, include: Obtain an original image and an enhanced image corresponding to the original image, wherein the enhanced image is obtained by image enhancement of the original image; The original image and the enhanced image are input into the trained neural network model obtained by the method according to any one of claims 1 to 7 through channel splicing or dual-branch fusion encoding to obtain the color-corrected enhanced image.

9. A training device for a neural network model for image color correction, characterized in that, The device includes: The first module is configured to input the original image and the enhanced image corresponding to the original image into the neural network model through channel concatenation or dual-branch fusion encoding, wherein the enhanced image is obtained by image enhancement of the original image, and the output of the neural network model is the color-corrected enhanced image; The second module is configured to generate a color offset tensor based on the original image and the enhanced image, the color offset tensor being used to correct the color of the enhanced image; The third module is configured to generate a multi-scale color consistency loss function based on the original image, the enhanced image, and the color shift tensor; and The fourth module is configured to update the neural network model based on a multi-scale color consistency loss function to obtain the trained neural network model. The third module further includes: The first submodule is configured to perform multi-scale decomposition on the original image to obtain multiple original image layers; The second submodule is configured to correct the enhanced image using the color offset tensor to obtain a corrected enhanced image; The third submodule is configured to perform multi-scale decomposition on the corrected enhanced image to obtain multiple corrected enhanced image layers; and The fourth submodule is configured to generate a multi-scale color consistency loss function based on the plurality of original image layers and the plurality of corrected enhanced image layers.

10. An electronic circuit, characterized in that, The electronic circuit includes: A circuit configured to perform the steps of the method according to any one of claims 1 to 8.

11. An electronic device, characterized in that, The electronic device includes: Processor; and A memory storing a program, the program comprising instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the method according to any one of claims 1 to 8.

13. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, causes the processor to perform the method according to any one of claims 1 to 8.

Images