Training method of image color conversion model, color conversion method and related device

Abstract

This application provides a training method for an image color conversion model, a color conversion method, and related apparatus. The method includes: receiving an original image and a target color style vector; using an initial image color conversion model, performing color style conversion processing on the original image under the constraint of the target color style vector to obtain a target image; reconstructing the original image based on its brightness and chromaticity features to obtain a reconstructed image; updating the parameters of the initial image color conversion model according to a first loss representing the degree of fit between the target image's color style and the target color style, and a second loss representing the degree of similarity between the reconstructed image and the original image, until the first and second losses satisfy predetermined conditions, thus obtaining a trained image color conversion model. This improves the visual effect of the target image obtained after color conversion of the original image.

Description

Technical Field

[0001] The embodiments described in this application relate to the field of deep learning technology, specifically to a training method for an image color conversion model, a color conversion method, and related apparatus. Background Technology

[0002] With the development of computer vision and image processing technologies, image color conversion technology has been widely applied in image editing, post-production photography, film and television production, and scanning imaging. By converting the color style of an image, it is possible to transfer between different color styles while maintaining the image's content structure, thereby meeting diverse visual presentation needs.

[0003] In existing technologies, image color conversion methods are typically based on deep learning models. However, when the image content is complex, the visual quality of images generated by existing deep learning models is subpar. For example, the plausibility or naturalness of images generated by existing deep learning models may not meet expectations.

[0004] Therefore, when using deep learning models to achieve image color conversion, improving the visual effect of the images generated by the deep learning models has become an urgent technical problem to be solved. Summary of the Invention

[0005] In view of this, this application provides a training method for an image color conversion model, a color conversion method, and a related apparatus, which can improve the visual effect of the target image obtained after color conversion of the original image to a certain extent.

[0006] In a first aspect, one embodiment of this application provides a training method for an image color conversion model. The method includes: receiving an original image and a target color style vector, wherein the target color style vector is used to characterize a target color style; performing color style conversion processing on the original image under the constraint of the target color style vector using an initial image color conversion model to obtain a target image; reconstructing the original image based on the brightness features and chromaticity features of the original image to obtain a reconstructed image; updating the parameters of the initial image color conversion model according to a first loss characterizing the degree of fit between the color style of the target image and the target color style, and a second loss characterizing the degree of similarity between the reconstructed image and the original image, until the first loss and the second loss satisfy a set condition, thereby obtaining the trained image color conversion model.

[0007] Optionally, the initial image color conversion model includes a first encoder and a first decoder; using the initial image color conversion model, under the constraint of the target color style vector, the original image is subjected to color style conversion processing to obtain a target image, including: inputting the original image into the first encoder to obtain an original image feature vector; using the first decoder, the original image feature vector, and the target color style vector to generate the target image; correspondingly, based on the brightness features and chromaticity features of the original image, the original image is reconstructed to obtain a reconstructed image, including: inputting the brightness features and chromaticity features of the original image into the first encoder to obtain an auxiliary image feature vector; using a second decoder and the auxiliary image feature vector to generate the reconstructed image; wherein, the second decoder is different from the first decoder.

[0008] Optionally, the parameters of the initial image color conversion model are updated based on a first loss characterizing the degree of fit between the color style of the target image and the target color style, and a second loss characterizing the degree of similarity between the reconstructed image and the original image. This includes updating the parameters of the first encoder, the first decoder, and the second decoder based on the first loss and the second loss.

[0009] Optionally, generating the target image using the first decoder, the original image feature vector, and the target color style vector includes: during the upsampling of the original image feature vector using the first decoder, performing feature modulation on at least one feature map formed during the upsampling process using the target color style vector to generate the target image.

[0010] Optionally, the first loss includes color style loss and / or adversarial loss; wherein the color style loss is used to characterize the degree of deviation between the color style of the target image and the target color style; and the adversarial loss is used to constrain the color style of the target image to be natural.

[0011] Optionally, the training method for the image color conversion model further includes: extracting the Y component, U component, and V component of the original image in the YUV color space; wherein the Y component of the original image forms the luminance feature of the original image; and the U component and V component of the original image form the chromaticity feature of the original image.

[0012] Secondly, one embodiment of this application also provides an image color conversion method, comprising: receiving an original image and a target color style vector; using an image color conversion model trained by the training method of the image color conversion model described in any of the above embodiments to convert the color style of the original image into the target color style represented by the target color style vector, thereby obtaining a target image.

[0013] Thirdly, one embodiment of this application also provides a training apparatus for an image color conversion model, comprising: a first receiving module for receiving an original image and a target color style vector, wherein the target color style vector is used to characterize a target color style; a first color style conversion module for performing color style conversion processing on the original image under the constraint of the target color style vector using an initial image color conversion model to obtain a target image; an image reconstruction module for reconstructing the original image based on the brightness features and chromaticity features of the original image to obtain a reconstructed image; and a parameter update module for updating the parameters of the initial image color conversion model according to a first loss characterizing the degree of fit between the color style of the target image and the target color style, and a second loss characterizing the degree of similarity between the reconstructed image and the original image, until the first loss and the second loss satisfy a set condition, thereby obtaining the trained image color conversion model.

[0014] Fourthly, one embodiment of this application also provides an image color conversion apparatus, comprising: a first receiving module for receiving an original image and a target color style vector; and a first color style conversion module for converting the color style of the original image into the target color style represented by the target color style vector using an image color conversion model trained by the training method of the image color conversion model described in any of the above embodiments, thereby obtaining a target image.

[0015] Fifthly, one embodiment of this application also provides a computer device, the computer device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method described in any of the above embodiments.

[0016] Sixthly, one embodiment of this application also provides a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the methods described in any of the above embodiments.

[0017] In several embodiments provided in this application, during the training of the initial image color conversion model, the initial image color conversion model can be used to perform color style conversion processing on the original image under the constraint of the target color style vector to obtain the target image. Further, based on the brightness features of the target image and the original image, and the chromaticity features of the original image, the original image can be reconstructed to obtain the reconstructed image. Then, according to a first loss representing the degree of fit between the color style of the target image and the target color style, and a second loss representing the degree of similarity between the reconstructed image and the original image, the parameters of the initial image color conversion model are updated until the first and second losses meet the set conditions, resulting in a trained image color conversion model. By synergistically adjusting the parameters of the initial image color conversion model using the first and second losses, the visual effect of the target image obtained after color conversion of the original image using the trained image color conversion model can be improved. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a training method for an image color conversion model provided in one embodiment of this application.

[0019] Figure 2 A schematic diagram of an image color conversion model provided for one embodiment of this application.

[0020] Figure 3 This is a schematic diagram illustrating the training process of an image color conversion model provided in one embodiment of this application.

[0021] Figure 4 This is a flowchart illustrating an image color conversion method provided in one embodiment of this application.

[0022] Figure 5 A schematic diagram of a training apparatus for an image color conversion model provided in one embodiment of this application.

[0023] Figure 6 This is a schematic diagram of an image color conversion device provided in one embodiment of this application.

[0024] Figure 7 A schematic diagram of a computer device provided for one embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments.

[0026] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0027] In related technologies, image color conversion methods can be achieved by pre-setting mapping relationships between colors. For example, the original image can be regularized based on look-up tables (LUTs), curve mapping, or color space transformation to obtain the desired color style.

[0028] However, the fixed mapping method in related technologies has limited adaptability to differences in image content. When the image content is complex or there are significant changes in lighting / material, problems such as unnatural colors, local color shifts, or detail distortion may easily occur.

[0029] Therefore, this application provides a training method for an image color conversion model, a color conversion method, and a related apparatus, which can improve the visual effect and naturalness of the target image to a certain extent.

[0030] This application provides a method for training an image color conversion model. This method for training the image color conversion model can be applied to computer devices.

[0031] Image color conversion models can be generative models based on deep learning technology. During the inference stage, image color conversion models can transfer the color style of the original image to the target color style while maintaining the basic stability of the image content structure. Therefore, they are applicable to one or more scenarios such as image editing, post-processing of photography, film and television production, and scanning imaging.

[0032] Please see Figure 1 The training method for an image color conversion model may include the following steps.

[0033] Step S110: Receive the original image and the target color style vector, wherein the target color style vector is used to represent the target color style.

[0034] The original image is used to represent the image that needs color style conversion processing. The original image can be a color image, such as an RGB color space image. Of course, the original image can also be an image with other color spaces, and color space unification can be performed before entering the training process. The implementation method of this application is not specifically limited herein.

[0035] In the embodiments of this application, the original image may be derived from photographic images, video frame images, scanned images, etc. The embodiments of this application are not specifically limited herein.

[0036] The target color style vector is used to represent the target color style. The target color style is the desired image style that the original image will possess after color conversion. Specifically, the target color style vector can be conditional information in vector form. For example, the target color style vector can be a vector output from a preset style encoding table, a vector extracted from a color reference image through an embedding network, or a vector obtained by mapping a user-selected filter identifier through an embedding layer. In some embodiments, the target color style vector may also include a combination encoding of style parameters such as hue, saturation, contrast, and color temperature. The embodiments described in this application are not specifically limited herein.

[0037] In this embodiment, the computer device can read and receive the original image and target color style vector from memory, or it can receive the original image and target color style vector sent by other terminals. Of course, the computer device can also receive the original image and target color style vector from different information sources. This embodiment is not specifically limited herein.

[0038] Step S120: Using the initial image color conversion model, perform color style conversion processing on the original image under the constraint of the target color style vector to obtain the target image.

[0039] The initial image color conversion model is an untrained image color conversion model. It uses a target color style vector to perform color style conversion on the original image, so that the original image possesses or tends towards the target color style indicated by the target color style vector.

[0040] The constraint of the target color style vector can be specifically manifested in the process of generating the target image using the initial image color conversion model, where the target color style vector is incorporated into the target image in a conditional injection manner.

[0041] For example, please see Figure 2 The initial image color conversion model can include a first encoder and a first decoder. The first encoder can downsample the original image to obtain an original image feature vector. The original image feature vector can be combined with the target color style vector and upsampled using the first decoder to obtain the target image. The combination of the original image feature vector and the target color style vector can include conditional normalization or attention mechanisms. Therefore, during the upsampling of the original image feature vector by the first decoder, the target color style vector is incorporated into the feature map generated during the upsampling process, thereby improving the fit between the final generated target image and the target color style.

[0042] Step S130: Based on the brightness features and chromaticity features of the original image, reconstruct the original image to obtain the reconstructed image.

[0043] Luminance features are used to characterize the luminance information of the original image. Chromaticity features are used to characterize the color information of the original image. Specifically, luminance and chromaticity features can be extracted based on the components of the original image in the YUV color space. For example, the Y component obtained after converting the original image to the YUV color space can be used as a luminance feature. And the U and V components obtained after converting the original image to the YUV color space can be used as chromaticity features.

[0044] In some implementations, to facilitate the input of luminance and color information into the corresponding encoders, luminance features can be implemented using the luminance feature map of the original image, and chrominance features can be implemented using the chrominance feature map of the original image. Specifically, the computer device can convert the original image from the RGB color space to the YUV color space, extract the Y component to form the luminance feature map of the original image, extract the U and V components, and concatenate them along the channel dimension to form the chrominance feature map of the original image. To adapt to the input scale of the first encoder, the computer device can perform interpolation scaling, cropping, or padding processing on the luminance feature map and the chrominance feature map of the original image, and perform normalization processing on the values. In some implementations, the computer device can also add perturbations to the luminance feature map and the chrominance feature map of the original image, thereby improving training stability. The implementation methods in this application are not specifically limited herein.

[0045] In some embodiments, luminance and chromaticity features can also be represented by other equivalent methods. For example, the L component of the Lab color space can be used as the luminance feature, and the a and b components as the chromaticity features. Alternatively, the V component of the HSV color space can be used as the luminance feature, and the H and S components as the chromaticity features. The embodiments described in this application are not specifically limited herein.

[0046] The reconstructed image is used to represent the reconstruction result of the original image and to provide auxiliary constraints on the training results. Specifically, the reconstruction process can be implemented using a reconstruction network. The reconstruction network can include a second encoder and a second decoder. The computer device can input the luminance features and chrominance features of the original image into the second encoder, which can encode the input luminance and chrominance features to obtain an auxiliary image feature vector. The second decoder can upsample based on the auxiliary image feature vector to generate the reconstructed image. The embodiments described in this application are not specifically limited herein.

[0047] In some implementations, the first encoder and the second encoder can be the same. That is, the first encoder can be used to encode the original image, or it can be used to encode the luminance features and chrominance features of the original image. Alternatively, the first encoder and the second encoder can share parameters in real time during training. Thus, the reconstruction network can be considered to include a first encoder and a second decoder.

[0048] Step S140: Update the parameters of the initial image color conversion model according to the first loss representing the degree of fit between the color style of the target image and the target color style, and the second loss representing the degree of similarity between the reconstructed image and the original image, until the first loss and the second loss meet the set conditions, and obtain the trained image color conversion model.

[0049] The first loss characterizes the degree of fit between the color style of the target image and the target color style. Specifically, the first loss may include a color style loss, used to measure the degree of deviation between the target image and the target color style represented by the target color style vector. The color style loss may be implemented based on one or more of the following: color histogram, color statistics, perceptual features, or style feature distance. In some implementations, the first loss may also include an adversarial loss, used to constrain the target image to have naturalness and visual realism, thereby mitigating incongruous colors and unnatural textures in complex content scenes.

[0050] The second loss characterizes the similarity between the reconstructed image and the original image. Specifically, the second loss can be a reconstruction loss, used to constrain the reconstructed image to be close to the original image at the pixel level. In some implementations, the second loss can also be superimposed with a structural consistency-related loss or a perceptual consistency-related loss to improve the ability to preserve texture and structure.

[0051] The parameter update of the initial image color conversion model can be achieved based on the joint optimization of the first loss and the second loss. Specifically, the parameter update of the initial image color conversion model can be achieved by backpropagation and updating the model parameters of the initial image color conversion model using a specified optimization algorithm, and gradually reducing the first loss and the second loss during training iterations. The specified optimization algorithm includes, but is not limited to, gradient descent, stochastic gradient descent, or the Adam optimizer. The implementation method described in this application is not specifically limited herein.

[0052] The set conditions can represent the convergence conditions of the image color conversion model. For example, the set conditions may include one or more of the following: both the first loss and the second loss are less than a preset threshold; the decrease in the first loss and the second loss is less than a preset threshold; the weighted sum of the first loss and the second loss is less than a preset threshold; or the number of training iterations reaches a preset number. When the first loss and the second loss meet the set conditions, the trained image color conversion model can be obtained.

[0053] In this embodiment, by introducing auxiliary constraints for reconstructing the original image during training, the trained image color conversion model can more fully consider the consistency of image content structure and semantics when performing color style conversion processing, and generate more natural and harmonious target images in complex scenes. Specifically, the reconstruction constraints of the reconstructed image can enable the initial image color conversion model to enhance its ability to extract and express image content and structural information while learning the target color style, thereby reducing the risk of unnatural color diffusion or local distortion caused by relying solely on color mapping, and thus improving the visual effect and stability of the target image.

[0054] In some implementations, please refer to Figure 3 The initial image color conversion model includes a first encoder and a first decoder. Using the initial image color conversion model, the original image is processed for color style conversion under the constraint of the target color style vector to obtain the target image. This can include: inputting the original image into the first encoder to obtain the original image feature vector; using the first decoder, the original image feature vector, and the target color style vector to generate the target image; correspondingly, based on the brightness and chromaticity features of the original image, the original image is reconstructed to obtain the reconstructed image. This can include: inputting the brightness and chromaticity features of the original image into the first encoder to obtain the auxiliary image feature vector; using the second decoder and the auxiliary image feature vector to generate the reconstructed image; wherein the second decoder is different from the first decoder.

[0055] The first encoder is used to extract features from the input image to output an image feature vector for decoding. Specifically, the first encoder may include convolutional layers, normalization layers, and nonlinear activation layers, and downsampling is achieved through strided convolution or pooling to extract structural and semantic information from the input image. For example, the first encoder may employ a multi-layer convolutional network to extract low-level texture features and high-level semantic features from the input image, and output the image feature vector as input to the subsequent decoder. In some embodiments, the first encoder may also include residual structures or attention structures to improve the expressive power of the image feature vector. The embodiments described in this application are not specifically limited herein.

[0056] In some embodiments, the input image may undergo one or more preprocessing steps, such as image resizing or normalization, before being input to the first encoder. The embodiments described in this application are not specifically limited herein.

[0057] The original image can be used as the input image for the first encoder. The first encoder can encode the original image based on its parameters to generate an original image feature vector. The original image feature vector is the feature representation extracted from the original image by the first encoder. It carries the structural and semantic information of the original image, thus providing the content basis for the first decoder to generate the target image.

[0058] The first decoder generates a target image based on the feature vector of the original image under the constraint of the target color style vector. Specifically, the first decoder may include an upsampling layer and a convolutional layer, and generates the target image by progressively restoring resolution. Furthermore, the first decoder may introduce the target color style vector as conditional information during the upsampling process to control the color style of the target image to tend towards the target color style represented by the target color style vector. For example, the first decoder may inject the target color style vector into at least one feature map generated during the upsampling process using feature modulation methods such as conditional normalization or attention fusion, thereby improving the fit between the target image and the target color style. The embodiments described in this application are not specifically limited herein.

[0059] The luminance and chrominance features of the original image can also be used as input to the first encoder. The first encoder can encode the luminance and chrominance features of the original image based on its parameters to generate an auxiliary image feature vector. The auxiliary image feature vector is used to characterize the feature representations extracted by the first encoder from the luminance and chrominance feature maps of the original image, thus providing an input basis for the second decoder to generate the reconstructed image.

[0060] The second decoder is used to generate a reconstructed image based on the feature vector of the auxiliary image, thereby reconstructing the original image. Specifically, the second decoder may include upsampling layers and convolutional layers, and recovers the resolution step by step based on the feature vector of the auxiliary image to output the reconstructed image. The embodiments described in this application are not specifically limited herein.

[0061] During the training of the initial image color conversion model, the color conversion task of generating the target image based on the original image and the reconstruction task of reconstructing the original image based on the brightness and chromaticity features of the original image can be jointly trained. Specifically, the first encoder reuses the same set or the same set of first encoder parameters when encoding the original image and when encoding the brightness and chromaticity features of the original image.

[0062] By jointly training the parameters of the first encoder during the color conversion and reconstruction tasks, the first encoder can better express the structural and semantic information of the image content when extracting image features, and reduce its dependence on simple color mapping relationships. The reconstruction task, by constraining the similarity between the reconstructed image and the original image, applies the gradient signal obtained from backpropagation to the first encoder, enabling it to more fully preserve structural features consistent with the original image content when encoding the brightness and chromaticity features. Therefore, the trained image color conversion model can more effectively balance the stability of the image content structure and color style consistency during the inference phase, thereby improving the visual effect and stability of the target image.

[0063] In some implementations, the parameters of the initial image color conversion model are updated based on a first loss representing the degree of fit between the color style of the target image and the target color style, and a second loss representing the degree of similarity between the reconstructed image and the original image, including: updating the parameters of the first encoder, the first decoder and the second decoder based on the first loss and the second loss.

[0064] The parameter updates of the first encoder, first decoder, and second decoder can be achieved using joint backpropagation. Specifically, the total gradient can be calculated based on the first loss and the second loss, and then backpropagated to the first encoder, first decoder, and second decoder respectively, thereby updating the parameters of the three synchronously. In some implementations, weight coefficients can be set for the first loss and the second loss to balance the training intensity of color style matching and reconstruction similarity, and to a certain extent avoid the dominance of one loss in the early stages of training, which could lead to training instability.

[0065] In some implementations, please refer to Figure 3 The parameters of the first encoder, the first decoder, and the second decoder are updated based on the first loss and the second loss, including: calculating the weighted sum of the first loss and the second loss to obtain the joint loss; and updating the parameters of the first encoder, the first decoder, and the second decoder based on the joint loss.

[0066] By updating the parameters of the first encoder, first decoder, and second decoder using the first and second losses, a closer collaborative constraint can be formed between the color conversion and reconstruction tasks during the training phase. On the one hand, the first decoder, constrained by the target color style vector, is more likely to generate a target image with a matching color style. On the other hand, the second decoder, based on the auxiliary image feature vector obtained by encoding the brightness and chromaticity features of the original image through the first encoder, is more likely to generate a reconstructed image similar to the original image, and forms a stable inverse constraint on the content structure expression by sharing the first encoder. Therefore, the trained image color conversion model can more fully consider the stability of the image content structure and the color style fit during the inference phase, thereby improving the visual effect and stability of the target image.

[0067] In some implementations, generating a target image using a first decoder, an original image feature vector, and a target color style vector includes: during the upsampling of the original image feature vector using the first decoder, performing feature modulation on at least one feature map formed during the upsampling process using the target color style vector to generate the target image.

[0068] The feature map is used to characterize the intermediate features formed by the first decoder during the upsampling process.

[0069] Feature modulation is used to adjust at least one feature map formed during the upsampling process using a target color style vector, so as to inject the target color style represented by the target color style vector into the target image generation process. Specifically, feature modulation may include adjusting the amplitude, bias, or weight of the feature map, thereby guiding the first decoder to output a target image that better matches the target color style.

[0070] In some implementations, feature modulation can be achieved using methods such as conditional normalization or attention fusion.

[0071] Conditional normalization is used to perform an affine transformation on a normalized feature map by introducing scaling and bias parameters generated from the target color style vector, thereby achieving conditional adjustment of the feature map. Specifically, the target color style vector can generate scaling and bias parameters via a mapping network, which may include a multilayer perceptron or a convolutional mapping structure, etc. The implementation method described in this application is not specifically limited herein.

[0072] Attention fusion is used to generate attention weights for feature maps based on the target color style vector, and to weight or fuse the feature maps based on these attention weights, thereby highlighting feature responses related to the target color style and suppressing feature responses unrelated to the target color style. Specifically, attention fusion can be channel attention, spatial attention, or a fusion method based on cross-attention. The embodiments described in this application are not specifically limited herein.

[0073] In some implementations, the computer device may perform feature modulation on only one level of feature map generated during the upsampling process to reduce computational overhead. Alternatively, the computer device may perform feature modulation on multiple levels of feature maps generated during the upsampling process to enhance coordinated control over global tone and local details. The embodiments described in this application are not specifically limited herein.

[0074] By using the target color style vector to modulate at least one feature map formed during the upsampling process of the original image feature vector in the first decoder, the constraint effect of the target color style vector can participate more directly in the step-by-step generation process of the target image, thereby improving the fit between the target image and the target color style, and improving the visual effect and stability in complex scenes.

[0075] In some implementations, the first loss includes color style loss and / or adversarial loss; wherein the color style loss is used to characterize the degree of deviation between the color style of the target image and the target color style; and the adversarial loss is used to constrain the color style of the target image to be natural.

[0076] Color style loss is used to characterize the degree of deviation between the color style of the target image and the intended color style. Color style can include the image's performance in one or more dimensions such as overall hue, saturation, contrast, color temperature, and color distribution.

[0077] Specifically, color style loss can be used to constrain the target image to approximate the target color style in one or more of the aforementioned dimensions. In some embodiments, color style loss can be calculated based on one or more methods such as color histogram distance, color moment statistic difference, or color distribution divergence. Alternatively, color style loss can also be calculated based on perceptual features or style feature distance. The embodiments described in this application are not specifically limited herein.

[0078] Adversarial loss is used to constrain the color style of the target image to be natural. Naturalness means that the target image is closer to the overall visual distribution of a real image, avoiding inconsistent color distributions. Specifically, adversarial loss can be implemented through an adversarial training framework. This framework can include a discriminative model and a generative model. The generative model can be an initial image color conversion model or a trained image color conversion model, while the discriminative model is used to determine whether the input image is a real image. Thus, the discriminative model receives the target image and outputs a discrimination result, while the adversarial loss drives the generative model to generate target images that are more difficult for the discriminative model to distinguish, thereby improving the visual realism and naturalness of the target image.

[0079] By introducing a combination of color style loss and adversarial loss into the first loss, the accurate transfer of the target color style and the naturalness of the target image can be more fully considered in complex content scenarios, thereby improving the visual effect and stability of the target image output by the trained image color conversion model.

[0080] In some implementations, the training method for the image color conversion model further includes: extracting the Y component, U component, and V component of the original image in the YUV color space; wherein the Y component of the original image forms the luminance feature of the original image; and the U component and V component of the original image form the chromaticity feature of the original image.

[0081] The YUV color space is used to represent images through a combination of luminance and chrominance components. Specifically, the Y component characterizes the luminance information of the image, which describes the distribution of light and dark areas. The U and V components characterize the chrominance information of the image, which describes color shifts and color distribution. Compared to the RGB color space, the component format of the YUV color space makes it easier to apply luminance and chrominance constraints separately during training, thereby reducing the instability risk caused by the coupling of luminance and chrominance during different color style conversions.

[0082] The Y component of the original image can be used to form the brightness feature of the original image. Specifically, the brightness feature of the original image can be directly derived from the Y component. Alternatively, the brightness feature can be formed after normalizing, downsampling, or adding perturbation to the Y component to improve the stability of model training. In some implementations, the brightness feature can be a brightness feature map, which can be obtained by interpolating and scaling the Y component. Each pixel in the brightness feature map can be used to represent the brightness of the corresponding pixel in the original image.

[0083] The U and V components of the original image can form the chromaticity features of the original image. Specifically, the chromaticity features of the original image can be directly derived from the U and V components. Alternatively, the chromaticity features can be formed by concatenating, normalizing, or scaling the U and V components. For example, the chromaticity features can be a chromaticity feature map containing two channels, with the two channels corresponding to the U and V components respectively, so that constraints can be applied to color direction and color intensity during subsequent reconstruction.

[0084] Brightness and chromaticity features can be used as input conditions for reconstructing the original image. Specifically, the computer device can input the brightness and chromaticity features of the original image into a first encoder to obtain an auxiliary image feature vector, and then decode the auxiliary image feature vector based on a second decoder to generate the reconstructed image. Since the brightness and chromaticity features are extracted from the original image, the above reconstruction process can improve the similarity between the reconstructed image and the original image, and further enhance the constraint effect on the consistency of image content structure and semantics.

[0085] Please see Figure 4 This application provides an image color conversion method. The image color conversion method can be applied to computer devices. The image color conversion method includes the following steps.

[0086] Step S210: Receive the original image and the target color style vector.

[0087] Step S220: The image color conversion model trained using the image color conversion model training method converts the color style of the original image into the target color style represented by the target color style vector, thus obtaining the target image.

[0088] An image color conversion model can include a first encoder and a first decoder that have undergone parameter training. Through the first encoder and the first decoder, the color style of the original image can be transferred more accurately to the target color style represented by the target color style vector. In the conversion process, the consistency of image content structure and semantics is taken into full account, thereby reducing the risk of unnatural color diffusion, local texture anomalies, or color inconsistencies in complex scenes, and thus improving the visual effect and stability of the output target image.

[0089] For a description of the image color conversion method, please refer to other embodiments of this application. This embodiment will not be described in detail here.

[0090] Please see Figure 5 This application provides a training apparatus for an image color conversion model. The training apparatus for the image color conversion model includes: a first receiving module, a first color style conversion module, an image reconstruction module, and a parameter update module.

[0091] The first receiving module is used to receive the original image and the target color style vector, wherein the target color style vector is used to represent the target color style.

[0092] The first color style conversion module is used to perform color style conversion processing on the original image under the constraint of the target color style vector using the initial image color conversion model to obtain the target image.

[0093] The image reconstruction module is used to reconstruct the original image based on its brightness and chromaticity features, thus obtaining the reconstructed image.

[0094] The parameter update module is used to update the parameters of the initial image color conversion model based on the first loss, which represents the degree of fit between the color style of the target image and the target color style, and the second loss, which represents the degree of similarity between the reconstructed image and the original image, until the first loss and the second loss meet the set conditions, and the trained image color conversion model is obtained.

[0095] For a description of the training device for the image color conversion model, please refer to other embodiments of this application. This embodiment will not be described in detail here.

[0096] Please see Figure 6 This application provides an image color conversion device. The image color conversion device includes: a second receiving module and a second color style conversion module.

[0097] The second receiving module is used to receive the original image and the target color style vector.

[0098] The second color style conversion module is used to convert the color style of the original image into the target color style represented by the target color style vector using the image color conversion model trained by the training method of the image color conversion model mentioned in any of the above embodiments, so as to obtain the target image.

[0099] For a description of the image color conversion device, please refer to other embodiments of this application. This embodiment will not be described in detail here.

[0100] Please see Figure 7 This application provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method mentioned in any of the above embodiments.

[0101] For a description of the computer equipment, please refer to other embodiments of this application. This embodiment will not be described in detail here.

[0102] This application provides a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the methods mentioned in any of the above embodiments.

[0103] For a description of the image color conversion device, please refer to other embodiments of this application. This embodiment will not be described in detail here.

[0104] It is understood that the specific examples in this document are only intended to help those skilled in the art better understand the embodiments of this application, and are not intended to limit the scope of the invention.

[0105] It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0106] It is understood that the various implementation methods described in this application can be implemented individually or in combination, and the implementation methods in this application are not limited in this respect.

[0107] Unless otherwise stated, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this application includes any and all combinations of one or more of the associated listed items. The singular forms "a," "the," and "the" as used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0108] It is understood that the processor in the embodiments of this application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiments can be completed by the integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0109] It is understood that the memory in the embodiments of this application may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Specifically, non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM). It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0110] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0111] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the aforementioned method implementations, and will not be repeated here.

[0112] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0113] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0114] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0115] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0116] The above description is merely a specific embodiment of this application, but the scope of protection of this invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this invention should be determined by the scope of the claims.

Claims

1. A training method for an image color conversion model, characterized in that, The method includes: Receive an original image and a target color style vector, wherein the target color style vector is used to characterize the target color style; Using the initial image color conversion model, the original image is subjected to color style conversion processing under the constraint of the target color style vector to obtain the target image; Based on the brightness and chromaticity features of the original image, the original image is reconstructed to obtain a reconstructed image; The parameters of the initial image color conversion model are updated based on a first loss representing the degree of fit between the color style of the target image and the target color style, and a second loss representing the degree of similarity between the reconstructed image and the original image, until the first loss and the second loss meet the set conditions, and the trained image color conversion model is obtained. The initial image color conversion model includes a first encoder and a first decoder; Using an initial image color conversion model, the original image is subjected to color style conversion processing under the constraint of the target color style vector to obtain a target image, including: inputting the original image into the first encoder to obtain an original image feature vector; and using the first decoder, the original image feature vector, and the target color style vector to generate the target image. Accordingly, based on the luminance features and chrominance features of the original image, the original image is reconstructed to obtain a reconstructed image, including: inputting the luminance features and chrominance features of the original image into the first encoder to obtain an auxiliary image feature vector; and using a second decoder and the auxiliary image feature vector to generate the reconstructed image; wherein the second decoder is different from the first decoder; The first encoder reuses the same set or the same first encoder parameters when encoding the original image and when encoding the brightness and chromaticity features of the original image.

2. The method according to claim 1, characterized in that, The parameters of the initial image color conversion model are updated based on a first loss characterizing the degree of fit between the color style of the target image and the target color style, and a second loss characterizing the degree of similarity between the reconstructed image and the original image, including: The parameters of the first encoder, the first decoder, and the second decoder are updated based on the first loss and the second loss.

3. The method according to claim 1, characterized in that, Generating the target image using the first decoder, the original image feature vector, and the target color style vector includes: During the upsampling of the original image feature vector using the first decoder, the target color style vector is used to modulate at least one feature map formed during the upsampling process to generate the target image.

4. The method according to claim 1, characterized in that, The first loss includes color style loss and / or adversarial loss; wherein the color style loss is used to characterize the degree of deviation between the color style of the target image and the target color style; and the adversarial loss is used to constrain the color style of the target image to be natural.

5. The method according to claim 1, characterized in that, Also includes: Extract the Y, U, and V components of the original image in the YUV color space; wherein, the Y component of the original image forms the luminance feature of the original image; and the U and V components of the original image form the chromaticity feature of the original image.

6. An image color conversion method, characterized in that, include: Receive the original image and the target color style vector; The image color conversion model, trained using the training method of the image color conversion model as described in any one of claims 1 to 5, converts the color style of the original image into the target color style represented by the target color style vector, thereby obtaining the target image.

7. A training device for an image color conversion model, characterized in that, include: The first receiving module is used to receive the original image and the target color style vector, wherein the target color style vector is used to characterize the target color style; The first color style conversion module is used to perform color style conversion processing on the original image under the constraint of the target color style vector using the initial image color conversion model to obtain the target image; An image reconstruction module is used to reconstruct the original image based on the brightness features and chromaticity features of the original image to obtain a reconstructed image; The parameter update module is used to update the parameters of the initial image color conversion model according to a first loss representing the degree of fit between the color style of the target image and the target color style, and a second loss representing the degree of similarity between the reconstructed image and the original image, until the first loss and the second loss meet the set conditions, and the trained image color conversion model is obtained. The initial image color conversion model includes a first encoder and a first decoder; Using an initial image color conversion model, the original image is subjected to color style conversion processing under the constraint of the target color style vector to obtain a target image, including: inputting the original image into the first encoder to obtain an original image feature vector; and using the first decoder, the original image feature vector, and the target color style vector to generate the target image. Accordingly, based on the luminance features and chrominance features of the original image, the original image is reconstructed to obtain a reconstructed image, including: inputting the luminance features and chrominance features of the original image into the first encoder to obtain an auxiliary image feature vector; and using a second decoder and the auxiliary image feature vector to generate the reconstructed image; wherein the second decoder is different from the first decoder; The first encoder reuses the same set or the same first encoder parameters when encoding the original image and when encoding the brightness and chromaticity features of the original image.

8. An image color conversion device, characterized in that, include: The second receiving module is used to receive the original image and the target color style vector; The second color style conversion module is used to convert the color style of the original image into the target color style represented by the target color style vector using the image color conversion model trained by the training method of the image color conversion model as described in any one of claims 1 to 5, thereby obtaining the target image.

9. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method of any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that, The readable storage medium stores computer program instructions that, when executed by a processor, implement the method described in any one of claims 1 to 5.

Images