Image coloring method, electronic device, storage medium and computer program product

Abstract

The present disclosure relates to an image coloring method, an electronic device, a storage medium, and a computer program product. The image coloring method includes: obtaining a first image to be colored, where the first image to be colored is a single-channel grayscale image; respectively obtaining corresponding LUTs for each color channel in the RGB color channels, and respectively performing coloring processing on the first image to be colored through each LUT to obtain three colored images; and performing channel splicing on the three colored images to obtain a first color image of the first image to be colored. Through the present disclosure, coloring of a grayscale image can be achieved, and a color image with better color vividness can be obtained after coloring.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technologies, and in particular, to an image coloring method, an electronic device, a storage medium, and a computer program product. Background Art

[0002] Image coloring is one of the means of image processing, aiming to convert a single-channel grayscale image into a three-channel color image, and at the same time, it is expected that the obtained color image can conform to the real scene of the physical world as much as possible.

[0003] In related technologies, the color images obtained by converting grayscale images usually have the problem of poor color vividness. Summary of the Invention

[0004] To overcome the problems existing in related technologies, the present disclosure provides an image coloring method, an electronic device, a storage medium, and a computer program product.

[0005] According to the first aspect of the embodiments of the present disclosure, an image coloring method is provided. The image coloring method includes:

[0006] Obtain a first image to be colored, where the first image to be colored is a single-channel grayscale image; respectively obtain display look-up tables (LUTs) corresponding to the R, G, and B color channels, and respectively perform coloring processing on the first image to be colored input to the R, G, and B color channels through the LUTs to obtain first colored images of the R, G, and B color channels; splice the first colored images of the R, G, and B color channels to obtain a first color image after coloring the first image to be colored.

[0007] In one implementation, the LUTs corresponding to the R, G, and B color channels respectively include multiple first LUTs corresponding to the R, G, and B color channels; the following method is used to perform coloring processing on the first image to be colored input to a specified color channel through the LUT to obtain a first colored image of the specified color channel, where the specified color channel includes the R color channel, the G color channel, or the B color channel: based on the multiple first LUTs corresponding to the specified color channel, respectively perform coloring processing on the first image to be colored input to the specified color channel; weight the multiple images obtained after coloring processing according to the target weight corresponding to the specified color channel to obtain a first colored image of the specified color channel.

[0008] In one implementation, the LUTs corresponding to the three color channels R, G, and B respectively include second LUTs corresponding to the three color channels R, G, and B respectively; wherein, the second LUT corresponding to a specified color channel is obtained by weighting a plurality of first LUTs corresponding to the specified color channel according to the target weight corresponding to the specified color channel, and the specified color channel includes the R color channel, the G color channel, or the B color channel; the following method is used to perform coloring processing on the first to-be-colored image input to the specified color channel through the LUT to obtain the first colored image of the specified color channel: Based on the second LUT corresponding to the specified color channel, perform coloring processing on the first to-be-colored image input to the specified color channel to obtain the first colored image of the specified color channel.

[0009] In one implementation, the following method is used to obtain the target weights corresponding to the three color channels R, G, and B respectively: Convert the second color image into a second to-be-colored image, where the second color image is different from the first color image, and the second to-be-colored image is a single-channel grayscale image; Obtain a plurality of different first LUTs respectively configured for the three color channels R, G, and B, and obtain the initial weights respectively configured for the three color channels R, G, and B; Based on the plurality of different first LUTs, perform coloring processing on the second to-be-colored image respectively, and weight the multiple images obtained after coloring processing according to the initial weights respectively configured for the three color channels R, G, and B to obtain the second colored images of the three color channels R, G, and B; Perform channel stitching on the second colored images of the three color channels R, G, and B, and adjust the initial weights respectively corresponding to the three color channels R, G, and B according to the difference degree between the image after channel stitching and the second color image to obtain the target weights respectively corresponding to the three color channels R, G, and B.

[0010] In one implementation, the initial weights respectively corresponding to the three color channels R, G, and B are obtained by inputting the second to-be-colored image into a specified neural network model and through the output of the specified neural network model; The step of adjusting the initial weights respectively corresponding to the three color channels R, G, and B according to the difference degree between the image after channel stitching and the second color image to obtain the target weights respectively corresponding to the three color channels R, G, and B includes: Based on the difference degree between the image after channel stitching and the second color image, determine the target loss, and train the specified neural network model with the target loss until the specified neural network model converges; Input the second to-be-colored image into the converged specified neural network model, and obtain the target weights respectively corresponding to the three color channels R, G, and B through the output result of the converged specified neural network model.

[0011] In one implementation, the multiple first LUTs corresponding to the specified color channel include a cool color LUT, a warm color LUT, and a medium color LUT.

[0012] In one implementation, the multiple first LUTs corresponding to the specified color channel include: multiple different first LUTs corresponding to multiple different color depths, and there is a one-to-one correspondence between the multiple different color depths and the multiple different first LUTs.

[0013] According to a second aspect of the embodiments of the present disclosure, there is provided an image coloring device, the device includes:

[0014] An acquisition unit, configured to acquire a first image to be colored, where the first image to be colored is a single-channel grayscale image; and configured to respectively acquire display lookup tables (LUTs) corresponding to the three color channels of R, G, and B; A processing unit, configured to perform coloring processing on the first image to be colored input to the three color channels of R, G, and B through the LUTs respectively, to obtain first colored images of the three color channels of R, G, and B; and configured to splice the first colored images of the three color channels of R, G, and B to obtain a first color image after coloring the first image to be colored.

[0015] In one implementation, the LUTs corresponding to the three color channels of R, G, and B respectively include multiple first LUTs corresponding to the three color channels of R, G, and B respectively; the processing unit performs coloring processing on the first image to be colored input to the specified color channel through the LUTs in the following manner to obtain the first colored image of the specified color channel, where the specified color channel includes the R color channel, the G color channel, or the B color channel: Based on the multiple first LUTs corresponding to the specified color channel, perform coloring processing on the first image to be colored input to the specified color channel respectively; weight the multiple images obtained after coloring processing according to the target weight corresponding to the specified color channel to obtain the first colored image of the specified color channel.

[0016] In one implementation, the LUTs corresponding to the three color channels R, G, and B respectively include second LUTs corresponding to the three color channels R, G, and B respectively; wherein, the second LUT corresponding to a specified color channel is obtained by weighting a plurality of first LUTs corresponding to the specified color channel according to the target weight corresponding to the specified color channel, and the specified color channel includes the R color channel, the G color channel, or the B color channel; the processing unit performs color processing on the first image to be colored input to the specified color channel through the LUT in the following manner to obtain the first colored image of the specified color channel: based on the second LUT corresponding to the specified color channel, perform color processing on the first image to be colored input to the specified color channel to obtain the first colored image of the specified color channel.

[0017] In one implementation, the processing unit obtains the target weights corresponding to the three color channels R, G, and B respectively in the following manner: convert the second color image into a second image to be colored, where the second color image is different from the first color image, and the second image to be colored is a single-channel grayscale image; obtain a plurality of different first LUTs respectively configured for the three color channels R, G, and B, and obtain the initial weights respectively configured for the three color channels R, G, and B; based on the plurality of different first LUTs, perform color processing on the second image to be colored respectively, and weight the plurality of images obtained after color processing according to the initial weights respectively configured for the three color channels R, G, and B to obtain the second colored images of the three color channels R, G, and B; splice the second colored images of the three color channels R, G, and B, and adjust the initial weights respectively corresponding to the three color channels R, G, and B according to the difference degree between the image after channel splicing and the second color image to obtain the target weights respectively corresponding to the three color channels R, G, and B.

[0018] In one implementation, the initial weights corresponding to the R, G, and B color channels respectively are obtained by inputting the second image to be colored into a specified neural network model and through the output of the specified neural network model; the processing unit adjusts the initial weights corresponding to the R, G, and B color channels respectively according to the difference degree between the image after channel splicing and the second color image in the following manner to obtain the target weights corresponding to the R, G, and B color channels respectively: determining a target loss based on the difference degree between the image after channel splicing and the second color image, and training the specified neural network model with the target loss until the specified neural network model converges; inputting the second image to be colored into the converged specified neural network model, and obtaining the target weights corresponding to the R, G, and B color channels respectively through the output result of the converged specified neural network model.

[0019] In one implementation, the multiple first LUTs corresponding to the specified color channel include a cold color LUT, a warm color LUT, and a medium color LUT.

[0020] In one implementation, the multiple first LUTs corresponding to the specified color channel include: multiple different first LUTs corresponding to multiple different color depths respectively, and there is a one-to-one correspondence between the multiple different color depths and the multiple different first LUTs.

[0021] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

[0022] a processor; a memory for storing instructions executable by the processor;

[0023] wherein, the processor is configured to: execute the image coloring method described in the first aspect or any one of the implementations of the first aspect.

[0024] According to a fourth aspect of the embodiments of the present disclosure, there is provided a storage medium, in which instructions are stored, and when the instructions in the storage medium are executed by a processor, the processor can execute the image coloring method described in the first aspect or any one of the implementations of the first aspect.

[0025] According to a fifth aspect of the embodiments of the present disclosure, there is provided a computer program product, the computer program product includes a computer program, and when the computer program is executed by a processor, it implements the image coloring method described in the first aspect or any one of the implementations of the first aspect.

[0026] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects: LUTs corresponding to the R, G, and B color channels can be prefabricated respectively, and the first image to be colored is colored through the LUTs corresponding to the R, G, and B color channels respectively to obtain the first colored images of the three color channels. Further, the first color image after coloring the first image to be colored can be obtained by splicing the channels of the first colored images. Since the LUTs used in the above process have excellent color expressiveness, the color image obtained by coloring and superimposing through the LUTs has an excellent visual effect.

[0027] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure and used together with the specification to explain the principles of the present disclosure.

[0029] Figure 1 is a flowchart of an image coloring method shown according to an exemplary embodiment.

[0030] Figure 2 is a flowchart of a method for coloring a first image to be colored input to a specified color channel through an LUT to obtain a first colored image of the specified color channel.

[0031] Figure 3 is a schematic flowchart of an image coloring process through a first LUT shown according to an exemplary embodiment.

[0032] Figure 4 is a schematic flowchart of an image coloring process through a second LUT shown according to an exemplary embodiment. Figure 5 is a flowchart of a method for configuring target weights for multiple different initial LUTs shown according to an exemplary embodiment.

[0033] Figure 6 is a flowchart of a method for configuring target weights for multiple different initial LUTs shown according to an exemplary embodiment.

[0034] Figure 7 is a schematic diagram of determining the target weights corresponding to the R, G, and B color channels respectively shown according to an exemplary embodiment.

[0035] Figure 8 is a block diagram of an image coloring device shown according to an exemplary embodiment.

[0036] Figure 9A block diagram of an electronic device for image coloring according to an exemplary embodiment is shown. Detailed implementation manners

[0037] Here, the exemplary embodiments will be described in detail, and examples thereof are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present disclosure.

[0038] In the drawings, the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The described embodiments are some, but not all, of the embodiments of the present disclosure. The embodiments described below with reference to the drawings are exemplary and are intended to explain the present disclosure and should not be construed as a limitation of the present disclosure. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present disclosure without creative efforts fall within the scope of protection of the present disclosure. The embodiments of the present disclosure will be described in detail below with reference to the drawings.

[0039] In recent years, important progress has been made in the research of technologies such as computer vision, deep learning, machine learning, image processing, and image recognition based on artificial intelligence. Artificial Intelligence (AI) is an emerging science and technology that studies and develops theories, methods, technologies, and application systems for simulating and extending human intelligence. The discipline of artificial intelligence is a comprehensive discipline involving many technical categories such as chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, and neural networks. As an important branch of artificial intelligence, computer vision specifically enables machines to recognize the world. Computer vision technologies usually include face recognition, liveness detection, fingerprint recognition and anti-counterfeiting verification, biometric recognition, face detection, pedestrian detection, object detection, pedestrian recognition, image processing, image recognition, image semantic understanding, image retrieval, character recognition, video processing, video content recognition, behavior recognition, 3D reconstruction, virtual reality, augmented reality, Simultaneous Localization and Mapping (SLAM), computational photography, robot navigation and positioning, etc. With the research and progress of artificial intelligence technology, this technology has been applied in many fields, such as security, urban management, traffic management, building management, park management, face access, face attendance, logistics management, warehouse management, robots, intelligent marketing, computational photography, mobile imaging, cloud services, smart home, wearable devices, driverless, autonomous driving, intelligent healthcare, face payment, face unlocking, fingerprint unlocking, person-certificate verification, smart screens, smart TVs, cameras, mobile Internet, webcasting, beauty, makeup, medical beauty, intelligent temperature measurement, etc.

[0040] The image coloring method provided by the embodiments of the present disclosure can be applied to the scenario of coloring grayscale images.

[0041] Image coloring is one of the means of image processing, aiming to convert a single-channel grayscale image into a three-channel color image, and at the same time, it is expected that the obtained color image can conform to the real scene of the physical world as much as possible. In the related art, the color images obtained by converting grayscale images usually have the problem of poor color vividness.

[0042] In view of this, the present disclosure provides an image coloring method, which can prefabricate corresponding display look-up tables (LUTs) for the R, G, and B color channels respectively, and perform coloring processing on the first image to be colored through the LUTs corresponding to the R, G, and B color channels respectively, to obtain the first colored images of the three color channels. Further, the first color image after coloring the first image to be colored can be obtained by means of channel splicing of the first colored images. Since in the above process, the LUTs used have better color expressiveness, the color images obtained by coloring with LUTs and superimposing have better visual effects. For the convenience of description below in the present disclosure, the grayscale image to be colored is referred to as the first image to be colored, and the color image obtained after coloring the first image to be colored is referred to as the first color image.

[0043] Figure 1 is a flowchart of an image coloring method shown according to an exemplary embodiment, as Figure 1 shown, including the following steps.

[0044] In step S11, the first image to be colored is obtained.

[0045] Among them, the obtained first image to be colored is a single-channel grayscale image.

[0046] In step S12, the LUTs corresponding to the R, G, and B color channels are obtained respectively, and the first image to be colored input to the R, G, and B color channels is colored through the LUTs respectively, to obtain the first colored images of the R, G, and B color channels.

[0047] In the embodiments of the present disclosure, the LUTs corresponding to the R, G, and B color channels can be, for example, 1D LUTs or 3D LUTs. Among them, since compared with 1D LUTs, 3D LUTs can make the colored images have better color expressiveness, 3D LUTs corresponding to the R, G, and B color channels can be configured respectively to realize more delicate adjustment of the color values of the images to be colored.

[0048] Among them, it can be understood that the RGB color channels include the R color channel, the G color channel, and the B color channel. Exemplarily, the first image to be colored can be colored respectively through the LUT corresponding to the R color channel, the LUT corresponding to the G color channel, and the LUT corresponding to the B color channel, to obtain three colored images.

[0049] In step S13, the first colored images of the R, G, and B color channels are channel - stitched to obtain the first color image after coloring the first image to be colored.

[0050] The image coloring method provided by the embodiments of the present disclosure can color the image to be colored respectively through the LUT corresponding to the R color channel, the LUT corresponding to the G color channel, and the LUT corresponding to the B color channel, and channel - stitch the three obtained colored images, so as to obtain the first color image with better color saturation and color richness.

[0051] Exemplarily, for the convenience of adjustment, each of the R, G, and B color channels is usually configured with multiple LUTs. On this basis, the coloring effect of the image can be adjusted by adjusting the weights of the multiple LUTs. Further, in the case where the weight adjustment is completed, each color channel can still retain the multiple configured LUTs to complete the image coloring. Of course, the multiple LUTs configured for each channel can also be weighted and combined to obtain a final LUT for coloring. The following is an exemplary description of the implementation methods for coloring the image of a color channel through one or more LUTs respectively.

[0052] In one implementation manner, the coloring of the image to be colored can be completed through multiple LUTs respectively configured for each color channel. For the convenience of description below in the present disclosure, the multiple LUTs configured for a color channel are referred to as the first LUT.

[0053] Figure 2 It is a flowchart of a method for coloring the first image to be colored input to a specified color channel through an LUT to obtain the first colored image of the specified color channel according to an exemplary embodiment, as Figure 2 shown, including the following steps. Among them, the specified color channel can be understood as any one of the R, G, and B color channels.

[0054] In step S21, based on the multiple first LUTs corresponding to the specified color channel, the first image to be colored input to the specified color channel is colored respectively.

[0055] In step S22, according to the target weight corresponding to the specified color channel, the multiple images obtained after coloring are weighted to obtain the first colored image of the specified color channel.

[0056] Figure 3 It is a schematic flowchart of image coloring through a first LUT shown according to an exemplary embodiment.

[0057] Exemplarily, as Figure 3 shown, taking the R color channel as an example, a total of three first LUTs, namely R1, R2, and R3, can be configured for the R color channel, a target weight Wa can be configured for R1, a target weight Wb can be configured for R2, and a target weight Wc can be configured for R3. If the first image to be colored is input into the R, G, and B color channels respectively, and the image processed by R1 is P1, the image processed by R2 is P2, and the image processed by R3 is P3, then a target weight Wa can be configured for P1, a target weight Wb can be configured for P2, and a target weight Wc can be configured for P3, and P1, P2, and P3 can be weighted to obtain the first colored image of the R color channel. In addition, the implementation process of determining the first colored image of the G color channel (or B color channel) is similar to the above implementation process for the R color channel, and the relevant content can refer to the implementation process for the R color channel, which will not be elaborated here.

[0058] In another implementation, multiple first LUTs respectively configured for each color channel can be weighted to obtain one LUT corresponding to the color channel. Taking the R color channel as an example, a total of three first LUTs, namely R1, R2, and R3, can be configured for the R color channel, a target weight Wa can be configured for R1, a target weight Wb can be configured for R2, and a target weight Wc can be configured for R3. On this basis, R1, R2, and R3 can be weighted according to the target weights Wa, Wb, and Wc to obtain one LUT corresponding to the R color channel. Further, according to one LUT corresponding to the specified color channel, the first image to be colored input into the specified color channel can be colored to obtain the first colored image of the specified color channel. For the convenience of description hereinafter in this disclosure, one LUT obtained by weighting multiple first LUTs is referred to as a second LUT.

[0059] Figure 4 It is a schematic flowchart of image coloring through a second LUT shown according to an exemplary embodiment. Exemplarily, as Figure 4 shown, the second LUT corresponding to the R color channel is R`, the second LUT corresponding to the G color channel is G`, and the second LUT corresponding to the B color channel is B`. The first image to be colored can be input into R`, G`, and B` respectively to obtain three colored images corresponding to R`, G`, and B` respectively. Further, the three obtained colored images can be spliced to obtain the first color image after coloring the first image to be colored.

[0060] In the embodiments of the present disclosure, a first LUT can be configured for each of the R, G, and B color channels respectively, and initial weights can be configured for each of the R, G, and B color channels respectively. Further, the coloring results of multiple different first LUTs can be compared with the true results of the color image corresponding to the grayscale image, and the initial weights configured for each of the R, G, and B color channels can be updated accordingly until the target weights are obtained. For the convenience of description in the present disclosure, the color image used to update the weights is referred to as the second color image, and the grayscale image obtained by converting the second color image is referred to as the second image to be colored. It can be understood that the second color image is a color image different from the first color image.

[0061] Figure 5 is a flowchart of a method for configuring target weights for multiple different initial LUTs shown according to an exemplary embodiment, as Figure 5 shown, and includes the following steps S31 to S34.

[0062] In step S31, the second color image is converted into the second image to be colored.

[0063] Among them, the second image to be colored is a single-channel grayscale image. By way of example, converting the second color image into the second image to be colored can be to convert the second color image into an image in the color-opponent space (Lab) format, and the second image to be colored can be obtained by extracting the image data of the L channel. In addition, for the second image to be colored obtained by converting the second image to be colored, the image data of each channel is the same as the image data of the second image to be colored.

[0064] In step S32, multiple different first LUTs configured for each of the R, G, and B color channels respectively are obtained, and the initial weights configured for each of the R, G, and B color channels respectively are obtained.

[0065] In step S33, based on the multiple different first LUTs, the second image to be colored is colored respectively, and the multiple images obtained after the coloring process are weighted according to the initial weights configured for each of the R, G, and B color channels respectively, to obtain the second colored images of the R, G, and B color channels.

[0066] Exemplarily, taking the R color channel as an example, three first LUTs, namely R1, R2, and R3, can be configured for the R color channel respectively. An initial weight W1 is configured for R1, an initial weight W2 is configured for R2, and an initial weight W3 is configured for R3. If the image after coloring processing through R1 is P4, the image after coloring processing through R2 is P5, and the image after coloring processing through R3 is P6, then a weight W1 can be configured for P4, a weight W2 can be configured for P5, and a weight W3 can be configured for P6, and P4, P5, and P6 are weighted to obtain the second colored image of the R color channel. In addition, the implementation process of determining the second colored image of the G color channel (or B color channel) is similar to the above implementation process for the R color channel. For relevant content, reference can be made to the implementation process for the R color channel, which will not be elaborated here.

[0067] In step S34, the second colored images of the three color channels R, G, and B are subjected to channel splicing, and based on the difference degree between the image after channel splicing and the second color image, the initial weights corresponding to the three color channels R, G, and B are adjusted to obtain the target weights corresponding to the three color channels R, G, and B respectively.

[0068] In addition, in the above embodiments, the initial weights and target weights configured for the three color channels R, G, and B can be determined by a network model, for example. Exemplarily, a specified neural network model with the input being the image to be colored and the output being the initial weight can be pre-configured. On this basis, the initial weights corresponding to the three color channels R, G, and B respectively are obtained by inputting the second image to be colored into the specified neural network model and through the output of the specified neural network model. Further, the target weights corresponding to the three color channels R, G, and B respectively can be obtained by training the specified neural network model. The following is an exemplary description of the implementation manner of obtaining the target weights.

[0069] Figure 6 It is a flowchart of a method for configuring target weights for multiple different initial LUTs shown according to an exemplary embodiment. As Figure 6 shown, steps S41, S42, and S43 in the embodiments of the present disclosure are similar to the execution methods of S31, S32, and S33 in Figure 5 and will not be elaborated here.

[0070] In step S44, the second colored images of the three color channels R, G, and B are subjected to channel splicing, and based on the difference degree between the image after channel splicing and the second color image, a target loss is determined, and the specified neural network model is trained with the target loss until the specified neural network model converges.

[0071] In step S45, the second image to be colored is input into the specified neural network model after convergence, and the target weights corresponding to the R, G, and B color channels are obtained through the output result of the specified neural network model after convergence.

[0072] Among them, the specified color channel corresponds to the specified neural network model one by one. The specified neural network model can, for example, include a first neural network model corresponding to the R color channel, a second neural network model corresponding to the G color channel, and a third neural network model corresponding to the B color channel. Further, by using the specified neural network model corresponding to the specified color channel to determine the target weight configured for the specified color channel, it is possible to perform differential configuration of the target weights for different color channels, so that the finally output color image has a better visual effect.

[0073] Since the overall training process of the present disclosure only involves the training and update of the specified neural network model. Therefore, for the image coloring method provided in the embodiments of the present disclosure, the required sample base for training is small and pre-training is not required. Compared with the conventional method that needs to be trained through prior knowledge, the training cost can be reduced.

[0074] Exemplarily, for each specified color channel, the multiple different initial LUTs configured may include, for example, a cold color LUT, a warm color LUT, and a neutral color LUT. Among them, the cold color LUT, the warm color LUT, and the neutral color LUT can be divided according to the pre-configured hue.

[0075] Exemplarily, multiple different first LUTs can also be configured for the specified color channel according to the depth of the color. For example, a light color LUT, a medium color LUT, and a dark color LUT can be configured for the specified color channel. On this basis, determining the multiple different first LUTs configured for the specified color channel can, for example, be determining multiple different first LUTs corresponding to multiple different color depths for the specified color channel. Among them, it can be understood that there is a one-to-one correspondence between the multiple different color depths and the multiple different first LUTs. In addition, other types of multiple different initial LUTs can also be configured for the specified color channel according to other division methods, and the present disclosure does not make specific limitations on this.

[0076] Figure 7 It is a schematic diagram showing the determination of the target weights corresponding to the R, G, and B color channels respectively according to an exemplary embodiment. Exemplarily, as Figure 7As shown, the second image to be colored can be input into the specified neural network models of the three color channels of R, G, and B respectively to obtain the initial weights configured for the three color channels of R, G, and B. Taking the R color channel as an example, the multiple first LUTs configured for the R color channel can include, for example, a dark color LUT (denoted as R1 in the example), a medium color LUT (denoted as R2 in the example), and a light color LUT (denoted as R3 in the example). On this basis, the second image to be colored can be input into the first neural network model of the R color channel (denoted as N1 in the example), and according to the output result of the first neural network model, the initial weight W1 configured for R1, the initial weight W2 configured for R2, and the initial weight W3 configured for R3 can be obtained. In addition, the implementation of determining the initial weights corresponding to the G color channel (or B color channel) is similar to the above implementation of determining the initial weights corresponding to the R color channel, and will not be elaborated here.

[0077] Exemplarily, in the case of obtaining the initial weights, the multiple colored images corresponding to each color channel can be weighted, and the colored images of the three color channels of R, G, and B can be channel - stitched to obtain the superimposed image. Based on this, the target loss for training the specified neural network model can be determined according to the difference degree between the superimposed image and the second color image, and subsequently, the specified neural network model can be trained through the target loss until the specified neural network model converges.

[0078] Exemplarily, in the case where the specified neural network model converges, the result output by the neural network model is the target weight. Subsequently, through the target weight and the first LUT (or multiple second LUTs) corresponding to each of the three color channels of R, G, and B, coloring of the grayscale image can be achieved.

[0079] Based on the same concept, the embodiments of the present disclosure also provide an image coloring device.

[0080] It can be understood that, in order to implement the above functions, the image coloring device provided by the embodiments of the present disclosure includes the corresponding hardware structure and / or software module for executing each function. Combining the units and algorithm steps of the examples disclosed in the embodiments of the present disclosure, the embodiments of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the way of hardware or computer software driving hardware depends on the specific application and design constraint conditions of the technical solution. Those skilled in the art can use different methods for each specific application to implement the described function, but such implementation should not be considered to exceed the scope of the technical solution of the embodiments of the present disclosure.

[0081] Figure 8 is a block diagram of an image coloring device shown according to an exemplary embodiment. Refer to Figure 8, the device 100 includes an acquisition unit 101 and a processing unit 102.

[0082] The acquisition unit 101 is configured to acquire a first image to be colored, where the first image to be colored is a single-channel grayscale image, and to respectively acquire display look-up tables (LUTs) corresponding to the R, G, and B color channels. The processing unit 102 is configured to perform coloring processing on the first image to be colored input to the R, G, and B color channels through the LUTs to obtain first colored images of the R, G, and B color channels, and to splice the first colored images of the R, G, and B color channels to obtain a first color image after coloring the first image to be colored.

[0083] In one implementation, the LUTs corresponding to the R, G, and B color channels respectively include multiple first LUTs corresponding to the R, G, and B color channels. The processing unit 102 performs coloring processing on the first image to be colored input to a specified color channel through the LUTs in the following manner to obtain a first colored image of the specified color channel, where the specified color channel includes the R color channel, the G color channel, or the B color channel: Based on the multiple first LUTs corresponding to the specified color channel, perform coloring processing on the first image to be colored input to the specified color channel, and weight the multiple images obtained after the coloring processing according to the target weight corresponding to the specified color channel to obtain the first colored image of the specified color channel.

[0084] In one implementation, the LUTs corresponding to the R, G, and B color channels respectively include second LUTs corresponding to the R, G, and B color channels. Among them, the second LUT corresponding to the specified color channel is obtained by weighting the multiple first LUTs corresponding to the specified color channel according to the target weight corresponding to the specified color channel, where the specified color channel includes the R color channel, the G color channel, or the B color channel. The processing unit 102 performs coloring processing on the first image to be colored input to the specified color channel through the LUTs in the following manner to obtain a first colored image of the specified color channel: Based on the second LUT corresponding to the specified color channel, perform coloring processing on the first image to be colored input to the specified color channel to obtain the first colored image of the specified color channel.

[0085] In one implementation, the processing unit 102 obtains the target weights corresponding to the R, G, and B color channels in the following manner: Convert the second color image into a second image to be colored. The second color image is different from the first color image, and the second image to be colored is a single-channel grayscale image. Obtain multiple different first LUTs respectively configured for the R, G, and B color channels, and obtain the initial weights respectively configured for the R, G, and B color channels. Based on the multiple different first LUTs, perform coloring processing on the second image to be colored respectively, and weight the multiple images obtained after the coloring processing according to the initial weights respectively configured for the R, G, and B color channels to obtain the second colored images of the R, G, and B color channels. Concatenate the channels of the second colored images of the R, G, and B color channels, and adjust the initial weights respectively corresponding to the R, G, and B color channels according to the difference degree between the image after channel concatenation and the second color image to obtain the target weights respectively corresponding to the R, G, and B color channels.

[0086] In one implementation, the initial weights respectively corresponding to the R, G, and B color channels are obtained by inputting the second image to be colored into a specified neural network model and through the output of the specified neural network model. The processing unit 102 adjusts the initial weights respectively corresponding to the R, G, and B color channels according to the difference degree between the image after channel concatenation and the second color image in the following manner to obtain the target weights respectively corresponding to the R, G, and B color channels: Determine the target loss based on the difference degree between the image after channel concatenation and the second color image, and train the specified neural network model with the target loss until the specified neural network model converges. Input the second image to be colored into the converged specified neural network model, and obtain the target weights respectively corresponding to the R, G, and B color channels through the output result of the converged specified neural network model.

[0087] In one implementation, the multiple first LUTs corresponding to a specified color channel include a cool color LUT, a warm color LUT, and a neutral color LUT.

[0088] In one implementation, the multiple first LUTs corresponding to a specified color channel include: multiple different first LUTs respectively corresponding to multiple different color depths, and there is a one-to-one correspondence between the multiple different color depths and the multiple different first LUTs.

[0089] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0090] Figure 9 It is a block diagram of an electronic device 200 for image coloring shown according to an exemplary embodiment.

[0091] As Figure 9 shown, an embodiment of the present disclosure provides an electronic device 200. Among them, the electronic device 200 includes a memory 201, a processor 202, and an input / output (I / O) interface 203. Among them, the memory 201 is used to store instructions. The processor 202 is used to call the instructions stored in the memory 201 to execute the image coloring method of the embodiments of the present disclosure. Among them, the processor 202 is respectively connected to the memory 201 and the I / O interface 203, and can be connected, for example, through a bus system and / or other forms of connection mechanisms (not shown). The memory 201 can be used to store programs and data, including the programs of the image coloring method involved in the embodiments of the present disclosure. The processor 202 executes various functional applications and data processing of the electronic device 200 by running the programs stored in the memory 201.

[0092] In the embodiments of the present disclosure, the processor 202 can be implemented in at least one hardware form of a digital signal processor (DSP), a field programmable gate array (FPGA), and a programmable logic array (PLA). The processor 202 can be a central processing unit (CPU) or a combination of one or more of other forms of processing units with data processing capabilities and / or instruction execution capabilities.

[0093] The memory 201 in the embodiments of the present disclosure may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include, for example, read only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD), etc.

[0094] In the embodiments of the present disclosure, the I / O interface 203 can be used to receive input instructions (such as numerical or character information, and generate key signal inputs related to user settings and function controls of the electronic device 200, etc.), and can also output various information to the outside (such as images or sounds, etc.). In the embodiments of the present disclosure, the I / O interface 203 can include one or more of a physical keyboard, function keys (such as volume control keys, power-on keys, etc.), a mouse, a joystick, a trackball, a microphone, a speaker, and a touch panel, etc.

[0095] In some embodiments, the present disclosure provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, perform any of the methods described above.

[0096] In some embodiments, the present disclosure provides a computer program product including a computer program that, when executed by a processor, performs any of the methods described above.

[0097] Although the operations are depicted in the drawings in a particular order, it should not be construed as requiring that the operations be performed in the particular order shown or in a sequential order, or that all of the illustrated operations be performed to obtain the desired result. Multitasking and parallel processing may be advantageous in certain environments.

[0098] The methods and apparatuses of the present disclosure can be accomplished using standard programming techniques, implementing various method steps using rule-based logic or other logics. It should also be noted that the terms "apparatus" and "module" as used herein and in the claims are intended to include implementations using one or more lines of software code and / or hardware implementations and / or devices for receiving inputs.

[0099] Any of the steps, operations, or programs described herein can be executed or implemented using one or more hardware or software modules alone or in combination with other devices. In one embodiment, the software module is implemented using a computer program product including a computer-readable medium containing computer program code that can be executed by a computer processor to execute any or all of the described steps, operations, or programs.

[0100] For purposes of example and description, the foregoing description of the embodiments of the present disclosure has been given. The foregoing description is not exhaustive nor is it intended to limit the present disclosure to the exact forms disclosed, and various modifications and variations may be possible in light of the above teachings, or various modifications and variations may be obtained from the practice of the present disclosure. These embodiments were chosen and described in order to illustrate the principles of the present disclosure and its practical application so that those skilled in the art can utilize the present disclosure in various embodiments and various modifications suitable for the particular purposes contemplated.

[0101] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0102] It can be understood that "a plurality of" in the present disclosure means two or more, and other quantifiers are similar thereto. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after. The singular forms of "a", "the", and "said" are also intended to include the plural forms unless the context clearly indicates otherwise.

[0103] Furthermore, it can be understood that the terms "first", "second", etc. are used to describe various information, but this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other, and do not represent a specific order or degree of importance. In fact, the expressions such as "first" and "second" can be used interchangeably. For example, without departing from the scope of the present disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information.

[0104] Furthermore, it can be understood that unless otherwise specified, "connection" includes direct connection without other components between the two, and also includes indirect connection with other elements between the two.

[0105] Furthermore, it can be understood that although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood as requiring these operations to be performed in the specific order or serial order shown, or requiring all the operations shown to obtain the desired result. In a specific environment, multitasking and parallel processing may be advantageous.

[0106] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0107] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. An image coloring method, characterized in that, the image coloring method includes: obtaining a first image to be colored, where the first image to be colored is a single-channel grayscale image; respectively obtaining display look-up tables LUT corresponding to the R, G, and B color channels, and respectively performing coloring processing on the first image to be colored input to the R, G, and B color channels through the LUT to obtain first colored images of the R, G, and B color channels; performing channel splicing on the first colored images of the R, G, and B color channels to obtain a first color image after coloring the first image to be colored; the LUT corresponding to each of the R, G, and B color channels includes multiple first LUTs corresponding to the R, G, and B color channels respectively; the following method is used to perform coloring processing on the first image to be colored input to a specified color channel through the LUT to obtain a first colored image of the specified color channel, where the specified color channel includes the R color channel, the G color channel, or the B color channel: based on the multiple first LUTs corresponding to the specified color channel, respectively performing coloring processing on the first image to be colored input to the specified color channel; weighting the multiple images obtained after coloring processing according to the target weight corresponding to the specified color channel to obtain the first colored image of the specified color channel; alternatively, the LUT corresponding to each of the R, G, and B color channels includes a second LUT corresponding to the R, G, and B color channels respectively; wherein, the second LUT corresponding to the specified color channel is obtained by weighting the multiple first LUTs corresponding to the specified color channel according to the target weight corresponding to the specified color channel, and the specified color channel includes the R color channel, the G color channel, or the B color channel; the following method is used to perform coloring processing on the first image to be colored input to the specified color channel through the LUT to obtain a first colored image of the specified color channel: based on the second LUT corresponding to the specified color channel, performing coloring processing on the first image to be colored input to the specified color channel to obtain a first colored image of the specified color channel; the following method is used to obtain the target weights corresponding to the R, G, and B color channels respectively: Convert the second color image into a second image to be colored, where the second color image is different from the first color image, and the second image to be colored is a single-channel grayscale image; obtain a plurality of different first LUTs respectively configured for the three color channels of R, G, and B, and obtain initial weights respectively configured for the three color channels of R, G, and B; based on the plurality of different first LUTs, perform coloring processing on the second image to be colored respectively, and weight the multiple images obtained after the coloring processing according to the initial weights respectively configured for the three color channels of R, G, and B to obtain second colored images for the three color channels of R, G, and B; splice the second colored images of the three color channels of R, G, and B, and adjust the initial weights respectively corresponding to the three color channels of R, G, and B according to the difference degree between the image after channel splicing and the second color image to obtain target weights respectively corresponding to the three color channels of R, G, and B.

2. The image coloring method according to claim 1, wherein, the initial weights respectively corresponding to the three color channels of R, G, and B are obtained by inputting the second image to be colored into a specified neural network model and through the output of the specified neural network model; the adjusting the initial weights respectively corresponding to the three color channels of R, G, and B according to the difference degree between the image after channel splicing and the second color image to obtain target weights respectively corresponding to the three color channels of R, G, and B includes: determining a target loss based on the difference degree between the image after channel splicing and the second color image, and training the specified neural network model with the target loss until the specified neural network model converges; inputting the second image to be colored into the converged specified neural network model, and obtaining the target weights respectively corresponding to the three color channels of R, G, and B through the output result of the converged specified neural network model.

3. The image coloring method according to claim 1, wherein, the plurality of first LUTs corresponding to the specified color channel include a cool color LUT, a warm color LUT, and a neutral color LUT.

4. The image coloring method according to claim 1, wherein, the plurality of first LUTs corresponding to the specified color channel include: a plurality of different first LUTs respectively corresponding to a plurality of different color depths, and there is a one-to-one correspondence between the plurality of different color depths and the plurality of different first LUTs.

5. An electronic device, wherein, comprising: a processor; a memory for storing instructions executable by the processor; wherein, the processor is configured to: execute the image coloring method according to any one of claims 1 to 4.

6. A storage medium, wherein, instructions are stored in the storage medium, and when the instructions in the storage medium are executed by the processor, the processor can execute the image coloring method according to any one of claims 1 to 4.

7. A computer program product, wherein, The computer program product includes a computer program which, when executed by a processor, implements the image coloring method according to any one of claims 1 to 4.

Images