Image coloring method and device, colored image generation model generating device, and program

Abstract

To provide an image coloring method and device capable of appropriately coloring an uncolored monochrome image, a generation device for a colored image generation model, and a program.SOLUTION: An image coloring device 100 generates a colored image generation model 130 for generating a colored image 30 from a monochrome image 10 and a solid-colored image 20, by machine learning which uses the monochrome image 10, the solid-colored image 20, and the colored image 30 as learning data. The image coloring device 100 generates a corresponding colored image 30a based on the colored image generation model 130, a target monochrome image 10a and a corresponding solid-colored image 20a.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technique for coloring achromatic monochrome images such as manga images. [Background technology]

[0002] Manga are traditionally often created as achromatic monochrome images. However, in recent years, there has been a demand for not only monochrome manga but also colored manga. For this reason, there is an increasing demand for post-coloring of already completed manga consisting of monochrome images. Since such coloring processing is very costly and time-consuming when performed manually, systems have been developed that perform automatic or semi-automatic coloring of monochrome images. As an example of such coloring processing, the one described in Non-Patent Document 1 is known.

[0003] The technique described in Non-Patent Document 1 performs coloring using machine learning based on an image generation algorithm called Pix2Pix, and in particular, adds color hint information provided by the user in addition to a line drawing image as input information. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Lvmin Zhang, Chengze Li, Tien-Tsin Wong, Yi Ji, and Chunping Liu, “Two-stage sketch colorization,” ACM Transactions on Graphics, vol. 37, no. 6, pp. 261:1-261:14, 2018. [Non-Patent Document 2] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros, “Image-to-image translation with conditional adversarial networks,” CVPR, pp. 5967-5976, 2017. Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technique described in Non-Patent Document 1 has a problem in that it is not accurate enough for coloring manga. This problem will be explained below.

[0006] In the method described in Non-Patent Document 1, learning using a large amount of learning data is required in advance, so that works by multiple authors are inevitably used as learning data. However, not only do manga styles vary greatly depending on the author, but even the same author's style can vary depending on the work and the time of production. For this reason, there is a problem in that it is difficult to perform learning appropriate for a specific manga, and therefore it is difficult to improve coloring accuracy.

[0007] Moreover, manga is not composed of only line drawings, but generally includes shading and other representations that express shading, color, or texture using achromatic patterns or shades. In paper media, these shading and other representations are formed by attaching a template called a "screen tone" to the line drawing. However, in the case of manga that includes shading and other representations, the method described in Non-Patent Document 1 does not allow the position of the shading and other representations to be specified at the learning stage or coloring process stage. For this reason, it is difficult to represent the shading and other representations with shading, color, and texture appropriate for the shading and other representations as an output result, and therefore there is a problem in that it is difficult to improve coloring accuracy.

[0008] The present invention has been made in consideration of the above circumstances, and an object of the present invention is to provide an image coloring method and device capable of appropriately coloring an achromatic monochrome image, a colored image generation model generation device, and a program. [Means for solving the problem]

[0009] In order to achieve the above-mentioned object, the image coloring method of the present invention is characterized in that it includes a model generation step in which a computer generates a colored image generation model that generates the colored image from the monochrome image and the solid image by machine learning using as learning data an achromatic monochrome image, a solid image in which a predetermined colored area corresponding to the monochrome image is colored with a single color, and a colored image corresponding to the monochrome image, and a colored image generation step in which a computer generates a corresponding colored image that is a colored image and corresponds to the target monochrome image based on the generated colored image generation model, a target monochrome image that is the monochrome image to be colored, and a corresponding solid image that is the solid image and corresponds to the target monochrome image.

[0010] In addition, the image coloring device of the present invention is characterized in that it includes a model generation unit that generates a colored image generation model that generates the colored image from the monochrome image and the solid image by machine learning using an achromatic monochrome image, a solid image in which a predetermined coloring area corresponding to the monochrome image is colored with a single color, and a colored image corresponding to the monochrome image as learning data, and a coloring processing unit that generates the colored image, which is the target monochrome image, based on the colored image generation model generated by the model generation unit, a target monochrome image, which is the monochrome image to be colored, and a corresponding solid image, which is the solid image and corresponds to the target monochrome image.

[0011] In addition, the image coloring device of the present invention is characterized in that it comprises a colored image generation model that is generated by machine learning using as learning data an achromatic monochrome image, a solid image in which a predetermined coloring area corresponding to the monochrome image is colored with a single color, and a colored image corresponding to the monochrome image, and that generates the colored image from the monochrome image and the solid image, and a coloring processing unit that generates the colored image, which is the target monochrome image, based on the colored image generation model, a target monochrome image, which is the monochrome image to be colored, and a corresponding solid image, which is the solid image and corresponds to the target monochrome image.

[0012] In addition, the device for generating a colored image generation model according to the present invention is characterized in that it includes a model generation unit that generates a colored image generation model that generates a colored image from a monochrome image and a solid image by machine learning using, as learning data, an achromatic monochrome image, a solid image in which a predetermined colored area corresponding to the monochrome image is colored with a single color, and a colored image corresponding to the monochrome image. Effect of the Invention

[0013] According to the present invention, a colored image generation model is generated by machine learning using, as training data, an achromatic monochrome image, a solid image in which a predetermined colored region corresponding to the monochrome image is colored with a single color and does not include the monochrome image, and a colored image corresponding to the monochrome image. That is, since a solid image including information on the position and color of the coloring target is used for machine learning, the accuracy of the position and color of the coloring for the target monochrome image is improved. That is, according to the present invention, appropriate coloring is possible. Furthermore, since a colored image generation model can be created by simply preparing a small amount of training data (a combination of a monochrome image, a solid image, and a colored image), not only is the effort of creating training data reduced, but coloring that matches the work or the style of the artist is also possible.

[0014] As described above, the present invention requires a solid image. However, the solid image can be easily derived from a monochrome image by hand or by an image processing device without highly specialized skills or knowledge. Therefore, the present invention enables easy and appropriate coloring without highly specialized skills or knowledge. [Brief description of the drawings]

[0015] [Figure 1] Functional block diagram of an image coloring device according to a first embodiment [Diagram 2] FIG. 1 is a diagram showing an example of a monochrome image. [Diagram 3] FIG. 13 is a diagram showing an example of a solid image. [Figure 4] FIG. 1 is a diagram showing an example of a colored image. [Diagram 5] Diagram explaining the first stage of the learning process [Figure 6] Diagram explaining the second stage of the learning process [Figure 7] Flowchart for explaining the operation of the image coloring device [Figure 8] FIG. 13 is a diagram showing an example of a coloring process. [Figure 9] Functional block diagram of an image coloring device according to a second embodiment [Figure 10] An example of the color hint creation screen DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] (First embodiment) An image coloring device according to a first embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a functional block diagram of the image coloring device according to the first embodiment, Fig. 2 is a diagram showing an example of a monochrome image, Fig. 3 is a diagram showing an example of a solid-colored image, and Fig. 4 is a diagram showing an example of a colored image. Note that in this application, as an image sample, "Nekodama", a work by author "Ebi Fry" included in the Manga109-s data set, is used.

[0017] The image coloring device 100 of this embodiment generates a colored image generation model that generates the colored image 30 from the monochrome image 10 and the solid image 20 by machine learning using as learning data an achromatic monochrome image 10, a solid image 20 in which a single color is colored in a predetermined colored area corresponding to the monochrome image 10 and does not include the monochrome image 10, and a colored image 30 corresponding to the monochrome image 10.

[0018] The image coloring device 100 is a device that generates a colored image 30, which is a corresponding colored image 30a corresponding to the target monochrome image 10, based on a target monochrome image 10a, which is a monochrome image 10 to be colored, and a corresponding solid image 20a, which is a solid image 20 and corresponds to the target monochrome image 10. Each of the images 10, 10a, 20, 20a, 30, 30a is made up of digital data of any file format, resolution, and depth.

[0019] The monochrome image 10 includes a target monochrome image 10a. That is, the target monochrome image 10a is one of the monochrome images 10, and is input to the image coloring device 100 as a target for coloring processing.

[0020] The monochrome image 10 means an achromatic image. Here, the monochrome image 10 may be black and white binary digital data, or grayscale digital data. In this embodiment, the monochrome image 10 is made up of digital data obtained by scanning a manga at a predetermined resolution, or digital data created by a computer in a style equivalent to the monochrome image 10 obtained by scanning a manga.

[0021] As shown in FIG. 2, a monochrome image 10 relating to a comic includes an achromatic line drawing portion 11 and a shading or similar expression portion 12 that expresses shading, color, or texture using achromatic patterns or shading.

[0022] The line drawing portion 11 is an area drawn with a pen or brush on a paper medium, and is mainly a monochrome image with strong contrast. That is, the line drawing portion 11 is essentially mainly a black and white binary image. The line drawing portion 11 may be drawn in intermediate gradation gray. Furthermore, in the monochrome image 10 as digital data, the line drawing portion 11 may contain intermediate gradation gray pixels so that the black and white binary line drawing becomes smooth. In the example of FIG. 2, the line drawing portion 11 is the portion expressing the contour lines and ridges of each part such as the face, hands, body, clothes, and accessories.

[0023] The shading and the like expression section 12 can be formed by pasting a template called "screen tone" on paper media. Screen tone is a process that allows the expression of intermediate gradations to be expressed in a pseudo manner by continuous patterns such as fine black and white binary dots, patterns, and lines. The shading and the like expression section 12 may include intermediate gradation gray. The shading and the like expression section 12 may also be formed by handwriting so as to obtain an effect equivalent to that of a screen tone. In the example of FIG. 2, the shading and the like expression section 12 is a section that expresses the shading of the skin under the chin and under the armpits, the color and texture of the hair, and the color and texture of the clothes. The shading and the like expression section 12 may include intermediate gradation gray pixels so that the black and white binary pattern becomes smooth in the monochrome image 10 as digital data. Depending on the resolution of the image data, the shading and the like expression section 12 may be substantially a collection of intermediate gradation gray pixels in the monochrome image 10 as digital data.

[0024] The solid image 20 includes a corresponding solid image 20a. That is, the corresponding solid image 20a is one of the solid images 20, and is input to the image coloring device 100 as a pair with the target monochrome image 10a in the coloring process.

[0025] The solid image 20 is an image that corresponds to the monochrome image 10. The solid image 20 is an image that indicates the color and the position (area) for coloring the corresponding monochrome image 10. The solid image 20 is generated manually or by a computer based on the corresponding monochrome image 10. In this embodiment, the solid image 20 used is one that is generated manually.

[0026] FIG. 3 is an example of a solid-colored image corresponding to the monochrome image 10 illustrated in FIG. 2. As shown in FIG. 3, the solid-colored image 20 has a predetermined colored region 21 colored with a single arbitrary color. The solid-colored image 20 may include a plurality of colored regions 21. In this case, the plurality of colored regions 21 may be adjacent to each other or may be separated from each other. In this embodiment, the solid-colored image 20 corresponds to the monochrome image 10, but does not include the monochrome image 10. The colored region 21 includes a region corresponding to the shading and the like representation portion 12 in the corresponding monochrome image 10. In the example of FIG. 3, the colored region 21 from the face to the neck includes a region corresponding to the shading and the like representation portion 12 formed under the chin in FIG. 2.

[0027] The colored image 30 includes a corresponding colored image 30a. That is, the corresponding colored image 30a is one of the colored images 30, and is output from the image coloring device 100 by a coloring process in which the target monochrome image 10a and the corresponding solid image 20a are input. The colored image 30 used as one of the input images in the learning process of the image coloring device 100 is manually generated based on the corresponding monochrome image 10 and solid image 20. The colored image 30 used in this learning process corresponds to the "ground truth" in the learning process.

[0028] Fig. 4 is an example of a colored image corresponding to the monochrome image 10 illustrated in Fig. 2 and the solid image 20 illustrated in Fig. 3. As shown in Fig. 4, in the colored image 30, an area corresponding to the shading or the like representation portion 12 in the monochrome image 10 may be colored with a color different from the color applied to the colored area 21 including the shading or the like representation portion 12 in the solid image 20. That is, since the monochrome image 10 is achromatic, the shading or the like representation portion 12 is formed of achromatic patterns or shades in order to represent shadows, colors, or textures, but in the colored image 30, such representation is replaced with a representation using color.

[0029] Next, a detailed description will be given of the image coloring device 100. As shown in Fig. 1, the image coloring device 100 includes a learning processing unit 110, a colored image generation processing unit 120, and a colored image generation model .

[0030] The image coloring device 100 is a conventionally known computer equipped with a main processing unit, a main memory device, an auxiliary memory device, an input device, a display device, a network device, etc. Each part of the image coloring device 100 can be configured by installing a program in the computer. The implementation form of the image coloring device 100 is not important. For example, the image coloring device 100 can be distributed and implemented in multiple devices.

[0031] The learning processing unit 110 generates a colored image generation model 130 that generates the colored image 30 from the monochrome image 10 and the solid image 20 by machine learning using the monochrome image 10, the solid image 20 corresponding to the monochrome image 10, and the colored image 30 corresponding to the monochrome image 10 and the solid image 20 as learning data. Each of the images 10, 20, and 30 may be stored in advance in a predetermined storage device, may be acquired from a predetermined external storage medium, or may be acquired from another device via a network.

[0032] The colored image generation processing unit 120 generates a corresponding colored image 30a, which is a colored image 30 and corresponds to the target monochrome image 10a, based on the generated colored image generation model 130, a target monochrome image 10a, which is a monochrome image 10 to be colored, and a corresponding solid image 20a, which is a solid image 20 and corresponds to the target monochrome image 10a. The target monochrome image 10a and the corresponding solid image 20a may be stored in advance in a predetermined storage device of the device itself, may be acquired from a predetermined external storage medium, or may be acquired from another device via a network. The colored image generation processing unit 120 can output the generated corresponding colored image 30a to its own display device, to its own predetermined storage device, to a predetermined external storage medium, or to another device via a network.

[0033] The colored image generation model 130 is composed of a generative adversarial network. The substance of the colored image generation model 130 is composed of a program stored in a predetermined storage device of the image coloring device 100 and various parameters used by the program and changed by learning processing. The configuration of the colored image generation model 130 and the processing of the learning processing unit 110 will be described in detail below with reference to Figs. 5 and 6. Fig. 5 is a diagram for explaining the first stage of learning, and Fig. 6 is a diagram for explaining the second stage of learning.

[0034] The colored image generation model 130 includes two generators (generative networks). The first generator converts a colored image 30 into a monochrome image 10, as shown in FIG. 5. Meanwhile, the second generator generates a colored image 30 from a set of a monochrome image 10 and a solid image 20, as shown in FIG. 6. These two generators are trained separately. Here, the stage of converting a colored image 30 into a monochrome image 10 is called the first stage. Also, the stage of generating a colored image 30 from a set of a monochrome image 10 and a solid image 20 is called the second stage.

[0035] The training data consists of a set (x, y, z) of a colored image 30, a monochrome image 20, and a solid-color image 20. First, in the first stage, a generator G AThe process learns how to generate a monochrome image 10 from a colorized image 30. This process removes color information from the colorized image 30 and predicts the position and pattern of the corresponding monochrome image 10. As shown in FIG. 4, the colorized image 30 contains enough information to predict the monochrome image 10. This learning process is similar to the process in Pix2Pix. Please refer to Non-Patent Document 2 for more information on the process in Pix2Pix.

[0036] [Stage 1] In this embodiment, the UNet architecture is a generator G A Let the colored image 30 be x and the monochrome image 10 be y. A The discriminative loss is expressed by the following equation (1).

[0037]

number

[0038] Here, the generator G A How to create a classifier (classification network) D A On the other hand, the classifier D A In addition to the loss function of the above formula (1), in this embodiment, as shown in the following formula (2), the correct monochrome image y and the generated image G A (x) is used.

[0039]

number

[0040] Generator G A The final goal is as shown in equation (3) below.

[0041]

number

[0042] [Second Stage] After the first stage, the system moves to the second stage. In the second stage, a pair of a solid-color image 20 and a monochrome image 10 is input. B The model learns how to generate a colored image 30 from a solid image 20 and a monochrome image 10. The generative model is an extension of UNet. To obtain one output from two inputs, the model has a two-stream structure.

[0043] Let the colored image 30 be x, the monochrome image 10 be y, and the solid image 20 be z. B The loss function is expressed by the following equation (4).

[0044]

number

[0045] Here, the generator G B How to create a classifier (classification network) D B On the other hand, the classifier D A learns to classify fakes from real ones. In this embodiment, a loss based on the L1 distance is used to improve the quality (accuracy) of the output, as shown in the following formula (5).

[0046]

number

[0047] Furthermore, to maintain cycle consistency, the generator G B The colored image 30 generated by A Input the trained generator G A Calculate the L1 distance between the false monochrome image and the correct monochrome image from the generator G B The final goal is expressed as follows:

[0048]

number

[0049] As described above, in the colored image generation model 130 according to the present embodiment, in the second stage, the generator G B The colored image 30 generated by A The learning process is also performed using monochrome images generated by

[0050] The colored image generation processing unit 120 is a trained generator G B is used to generate a corresponding colored image 30a from the target monochrome image 10a and the corresponding solid image 20a.

[0051] Next, the operation of the image coloring device 100 according to the present embodiment will be described with reference to the flowchart of FIG.

[0052] First, the image coloring device 100 performs a learning process using learning data consisting of a monochrome image 10, a solid image 20, and a colored image 30 to generate a colored image generation model 130 (step S1). Next, the image coloring device 100 acquires a monochrome image 10a to be processed and a corresponding solid image 20a corresponding to the monochrome image 10a (steps S2 and S3), and generates a corresponding colored image 30a using the colored image generation model 130 (step S4).

[0053] FIG. 8 shows an example of coloring processing by the image coloring device 100 according to the present embodiment. In this example, 10 pages were randomly selected from the above-mentioned work "Nekodama," 5 pages were used for learning processing, and the other 5 pages were used as coloring targets. In the example of FIG. 8, as comparison target images, the colored image generation model 130 according to the present embodiment generates a colored image using the generator G A 13 shows a case where a color image generation model is used in which the processing for is omitted.

[0054] As shown in Fig. 8, according to the image coloring device 100 of the present embodiment, the corresponding colored image 30a, which is an output image, is extremely similar to the colored image 30, which is a correct image, and it was confirmed that the coloring accuracy is high. In particular, it was confirmed that the image coloring device 100 of the present embodiment can obtain high coloring accuracy even with a learning process using a small amount of learning data, and that the coloring of the shading and the like representation unit 12 is appropriate.

[0055] According to such an image coloring device 100, a colored image generation model is generated by machine learning using, as learning data, an achromatic monochrome image 10, a solid image 20 in which a predetermined coloring region corresponding to the monochrome image 10 is colored with a single color and does not include the monochrome image 10, and a colored image 30 corresponding to the monochrome image 10. That is, since the solid image 20 including information on the position and color of the coloring target is used for machine learning, the accuracy of the coloring position and color for the target monochrome image 10a is improved. That is, according to the present invention, appropriate coloring is possible.

[0056] As described above, the present invention requires a solid image 20. However, the solid image 20 can be easily derived from the monochrome image 10 by hand or by an image processing device without advanced specialized skills or knowledge. Therefore, the present invention enables easy and appropriate coloring without advanced specialized skills or knowledge.

[0057] (Second embodiment) An image coloring device according to a second embodiment of the present invention will be described with reference to the drawings. Fig. 9 is a functional block diagram of the image coloring device according to the second embodiment, and Fig. 10 shows an example of a color hint creation screen.

[0058] The image coloring device according to this embodiment differs from the first embodiment in the method of creating the solid image 20. That is, in the first embodiment, the solid image 20 was manually created from the corresponding monochrome image 10, but in this embodiment, the image coloring device 100' creates the solid image 20 from the monochrome image 10. Since other points are the same as in the first embodiment, only the differences will be described here.

[0059] The image coloring device 100' according to this embodiment includes a solid image generating unit 140, as shown in FIG. 9. The solid image generating unit 140 generates a solid image 20 corresponding to the monochrome image 10 from the monochrome image 10. More specifically, as shown in FIG. 10, the solid image generating unit 140 outputs the monochrome image 10 to a predetermined display device (not shown) and accepts input of one or more color hints 141 from a user. The color hint 141 indicates color information and position information within the image. The solid image generating unit 140 superimposes and displays the input color hint 141 on the monochrome image 10 in a predetermined display form. In the example of FIG. 10, the color hint 141 is displayed as a circular mark having a color. Based on the position information of the input color hint 141, the solid-colored image generating unit 140 searches for a closed area in the monochrome image 10 with the line drawing portion 11 as a boundary, and generates a solid-colored image 20 by coloring the closed area with the color of the color hint as a coloring area. Various conventionally known algorithms for searching for a closed area can be used. The solid-colored image generating unit 140 can store the generated solid-colored image 20 in a predetermined storage device or an external storage device, or transmit it to an external device.

[0060] According to such an image coloring device 100', the efficiency of the coloring process is improved since the solid image 20 can be generated semi-automatically. Other functions and effects are the same as those of the first embodiment.

[0061] Although one embodiment of the present invention has been described in detail above, the present invention is not limited to the above embodiment, and various improvements and modifications may be made without departing from the spirit and scope of the present invention.

[0062] For example, in the above embodiment, a monochrome image 10 in which a shadow representation portion 12 is formed using a screen or the like is treated as the subject of coloring, but the present invention can also be applied to a monochrome image 10 in which a shadow representation portion 12 is not formed.

[0063] In addition, in the above embodiment, the solid image 20 corresponds to the monochrome image 10 but does not include the monochrome image 10 itself. However, the solid image 20 may include a part or all of the corresponding monochrome image 10.

[0064] In the above embodiment, a generative adversarial network is used as the colored image generation model 130, but the present invention can be applied to other models. For example, the present invention can be applied to a convolutional neural network that receives a solid image and a monochrome image as input and outputs a colored image.

[0065] In the above embodiment, the learning processing unit 110 that generates the colored image generation model 130 and the colored image generation unit 120 that generates the corresponding colored image 30a using the colored image generation model 130 are implemented in the same device, but they may be distributed and implemented in different devices. In this case, the colored image generation model 130 generated by the learning processing unit 110 may be transferred and implemented from the device in which the learning processing unit 110 is implemented to the device in which the colored image generation unit 120 is implemented. This allows the generation process of the colored image generation model 130 and the coloring process by the colored image generation model 130 to be performed independently by different people, places, and times, thereby improving convenience. [Explanation of symbols]

[0066] 10. Monochrome image 10a…Target monochrome image 20...Solid color image 20a…Compatible solid image 30...Colored image 30a… Corresponding color image 100,100'...Image coloring device 110...Learning processing unit 120...Colored image generation processing unit 130...Color image generation model 140...Solid color image generation unit

Claims

1. The computer a model generation step of generating a colored image generation model that generates the colored image from the monochrome image and the solid image by machine learning using, as learning data, an achromatic monochrome image, a solid image in which a predetermined colored region corresponding to the monochrome image is colored with a single color, and a colored image corresponding to the monochrome image; and a colored image generating step of generating a corresponding colored image which is the colored image and corresponds to the target monochrome image, based on the generated colored image generation model, a target monochrome image which is the monochrome image to be colored, and a corresponding solid image which is the solid image and corresponds to the target monochrome image. A method for coloring an image, comprising:

2. the monochrome image includes an achromatic line drawing portion and a shading or other representation portion that represents shading, color, or texture by an achromatic pattern or shading, The colored area of ​​the solid image includes an area corresponding to the portion expressing shading, etc.

2. The method for coloring an image according to claim 1.

3. In the colored image, an area of ​​the monochrome image corresponding to the portion expressing shading or the like is colored with a color different from a color applied to a colored area including the portion expressing shading or the like in the solid image.

3. The method for coloring an image according to claim 2.

4. The color image generation model is a generative adversarial network.

4. The method for coloring an image according to claim 1, wherein the image is colored in a colored state.

5. The colored image generation model is A first generative adversarial network including a first generative network that generates the monochrome image based on the colored image, and a first discrimination network that performs authenticity determination based on the colored image input to the first generative network and the monochrome image generated by the first generative network; a second generative adversarial network including a second generative network that generates the colored image based on the solid image and the monochrome image, and a second discrimination network that performs authenticity determination based on the colored image generated by the second generative network and the solid image and the monochrome image input to the second generative network; The model generation step includes a training step of the first generative adversarial network and a training step of the second generative adversarial network, In the learning step of the second generative adversarial network, in addition to the colored image generated by the second generative network and the solid image and monochrome image input to the second generative network, a learning process is performed using the monochrome image generated by the first generative network that has been trained using the colored image generated by the second generative network as an input, In the colored image generating step, the corresponding colored image is generated from the target monochrome image and the corresponding solid image using a second generative network of the second generative adversarial network that has been trained.

5. The method for coloring an image according to claim 4.

6. a model generation unit that generates a colored image generation model that generates the colored image from the monochrome image and the solid image by machine learning using, as learning data, an achromatic monochrome image, a solid image in which a predetermined colored region corresponding to the monochrome image is colored with a single color, and a colored image corresponding to the monochrome image; a coloring processing unit that generates a corresponding colored image that is the colored image and corresponds to the target monochrome image based on a colored image generation model generated by the model generation unit, a target monochrome image that is the monochrome image to be colored, and a corresponding solid image that is the solid image and corresponds to the target monochrome image.

1. An image coloring device comprising:

7. a colored image generation model that is generated by machine learning using, as learning data, an achromatic monochrome image, a solid-color image in which a predetermined colored region corresponding to the monochrome image is colored with a single color, and a colored image corresponding to the monochrome image, and that generates the colored image from the monochrome image and the solid-color image; a coloring processing unit that generates the corresponding colored image, which is the colored image and corresponds to the target monochrome image, based on the colored image generation model, a target monochrome image, which is the monochrome image to be colored, and a corresponding solid image, which is the solid image and corresponds to the target monochrome image.

1. An image coloring device comprising:

8. The present invention includes a model generation unit that generates a colored image generation model that generates a colored image from a monochrome image and a solid image by machine learning using, as learning data, an achromatic monochrome image, a solid image in which a predetermined colored region corresponding to the monochrome image is colored with a single color, and a colored image corresponding to the monochrome image. A colored image generation model generating device comprising:

9. A program for causing a computer to function as the image coloring device according to claim 6 or 7.

10. A program for causing a computer to function as the generating device for the colored image generation model according to claim 8.

Images