Image coloring apparatus, image coloring method, and program
The image coloring apparatus addresses color inconsistencies by employing frame and object detection, display control, and AI-assisted coloring to ensure uniform color application across images, enhancing user-guided color consistency.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- TOPPAN HOLDINGS INC
- Filing Date
- 2022-08-22
- Publication Date
- 2026-06-30
Smart Images
Figure 0007882050000001 
Figure 0007882050000002 
Figure 0007882050000003
Abstract
Description
Technical Field
[0001] The present invention relates to an image coloring device, an image coloring method, and a program.
Background Art
[0002] In recent years, it has been common to distribute comics colored by an electronic distribution application. When distributing uncolored comics issued in the past by an electronic distribution application, it is necessary to color the comics. Also, when creating an animation, it is necessary to color many frames. A technique for semi-automatically coloring comics is described in, for example, Non-Patent Document 1 below. In Non-Patent Document 1, an input image is automatically divided into frames, and a reference image serving as a coloring sample is prepared to perform coloring on a plurality of frames.
Prior Art Documents
Non-Patent Documents
[0003]
Non-Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] However, in Non-Patent Document 1, when automatically coloring by AI (artificial intelligence), there are cases where coloring is performed on incorrect areas or different colors are applied to the same object. For example, the area of a person's skin may be colored with the color of the hair adjacent to the person's skin, different colors may be applied to the same clothes of the same character, or unevenness may occur.
[0005] This disclosure is made in view of these circumstances and aims to provide an image coloring apparatus, an image coloring method, and a program that can suppress color inconsistencies between images. [Means for solving the problem]
[0006] This disclosure was made to solve the problems described above, and one aspect of this disclosure is: A frame detection unit that detects frame images contained in the content, and an object detection unit that detects objects from the frame images, A reception unit that receives user input, a coloring unit that colors an object based on the input received by the reception unit, and a plurality of units including the object. The aforementioned frame A first area for displaying an image, and a reference area containing the object The aforementioned frame A second area for displaying an image, and a colored area containing the object. The aforementioned frame The system comprises a third area for displaying an image, and a display control unit that arranges these areas to set the display area of the display unit, and the display control unit is When the reception unit receives an operation to select a frame image, it displays the selected frame image in the third area as a frame image to be colored. The object detection unit detects an object that is the same as the object included in the third area. The display control unit displays the frame images in which the same object was detected as thumbnails in the first area as related frame images. When the unit receives an operation to select one frame image from the thumbnail-displayed related frame images, it displays the selected frame image in the second area as a reference frame image. It is an image coloring device.
[0007] Other aspects of this disclosure are: The steps include detecting frame images contained in the content, and detecting an object from the frame images, The process includes the steps of: displaying a plurality of images containing an object in a first area of the display unit; displaying a reference image from among the images displayed in the first area in a second area of the display unit adjacent to the first area; displaying an image to be colored in a third area of the display unit adjacent to the first and second areas; and coloring the image to be colored based on user operation. When an operation to select a frame image is received, the selected frame image is displayed in the third area as a frame image to be colored, an object identical to the object included in the third area is detected, and the frame images in which the same object is detected are displayed as thumbnails in the first area as related frame images, and when an operation to select one frame image from the thumbnail-displayed related frame images is received, the selected frame image is displayed in the second area as a reference frame image. This is a method for coloring images.
[0008] Other aspects of this disclosure relate to the computer of an image coloring apparatus, The steps include detecting frame images contained in the content, and detecting an object from the frame images, The process involves: displaying multiple images containing an object in a first area of the display unit; displaying a reference image from among the images displayed in the first area in a second area of the display unit adjacent to the first area; displaying an image to be colored in a third area of the display unit adjacent to the first and second areas; and performing coloring on the image to be colored based on user operation. When an operation to select a frame image is received, the selected frame image is displayed in the third area as a frame image to be colored, objects identical to those included in the third area are detected, the frame images in which the same objects are detected are displayed as thumbnails in the first area as related frame images, and when an operation to select one frame image from the thumbnail-displayed related frame images is received, the selected frame image is displayed in the second area as a reference frame image.It's a program. [Effects of the Invention]
[0009] According to one aspect of the present invention, it is possible to suppress color inconsistencies between images. [Brief explanation of the drawing]
[0010] [Figure 1] This is a block diagram showing one example configuration of an image coloring device in an embodiment. [Figure 2] This is a block diagram showing an example of the AI coloring area in the embodiment. [Figure 3] This is a block diagram showing an example of a correction unit in an embodiment. [Figure 4] This figure shows an example of a coloring screen to be displayed on the display unit. [Figure 5] This flowchart shows an example of the processing procedure of the image coloring device in the embodiment. [Figure 6] This figure shows an example of a coloring screen in which the frame image to be colored is displayed in the frame area to be colored. [Figure 7] This figure shows an example of a coloring screen with related frame images displayed in the related frame area. [Figure 8] This figure shows an example of a coloring screen where the frame image to be colored and the reference frame image are displayed side by side. [Modes for carrying out the invention]
[0011] The image coloring apparatus, image coloring method, and program to which the present invention is applied will be described below with reference to the drawings.
[0012] [Example of an image coloring device configuration] FIG. 1 is a block diagram showing an example of the configuration of an image coloring device according to an embodiment. The image coloring device 100 is an information processing device such as a personal computer operated by a user, for example. The image coloring device 100 is connected to a content storage device 200, for example, and acquires contents such as comics and animations. The image coloring device 100 performs a process of coloring an object included in an image included in the content. In the present embodiment, the content is a black-and-white comic in which a single work includes a large number of frames, and an object is drawn as a line drawing in each frame, but is not limited thereto, and may be an animation including a plurality of frames. The object is a person, an object, a background, etc. that appear in the comic. A frame is one or more included in each page of the black-and-white comic and is an image surrounded by a frame line.
[0013] The image coloring device 100 includes, for example, a user interface unit 110, a related item detection unit 120, a coloring unit 130, and a storage unit 140. The user interface unit 110, the related item detection unit 120, and the coloring unit 130 may be functional units realized by a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) executing a program stored in a program memory for performing processes described later, for example. The storage unit 140 may be realized by a storage device such as a memory or a hard disk drive, for example. Further, a part of the user interface unit 110, the related item detection unit 120, the coloring unit 130, and the storage unit 140 may be configured to be distributed to other devices.
[0014] The user interface unit 110 includes, for example, a display unit 112, a display control unit 114, and a reception unit 116. The display unit 112 is a liquid crystal display or the like, and displays an image in response to an input of a video signal. The display control unit 114 controls the image to be displayed on the display unit 112. The reception unit 116 is an input device such as a pointing device such as a mouse or a keyboard.
[0015] The related item detection unit 120 includes, for example, a frame detection unit 122 and an object detection unit 124. The frame detection unit 122 detects frames included in a black-and-white comic. The object detection unit 124 detects objects included in the frames. The relationship between the frame information indicating the frame image detected by the frame detection unit 122 and the object information indicating the object detected by the object detection unit 124 is registered in the database constructed in the storage unit 140. Thereby, the storage unit 140 functions as an object registration unit. The frame information is, for example, information that uniquely identifies a frame in a comic. The object information is, for example, information that uniquely identifies an object in a comic. The object information may include information that uniquely identifies a partial region of the object. The related item detection unit 120 detects frames and objects as related items for coloring processing, but may detect other elements such as pages and frames in an animation as related items. Further, the object detection unit 124 may detect, as objects, an image composed only of lines (line drawing), an image in which solid or tone is used in addition to lines, a rough image that is a draft of a line drawing, and the like.
[0016] The related item detection unit 120 may be implemented using a machine learning model trained with a comic learning dataset including frames, pages, characters, objects, backgrounds, etc. as learning data. The machine learning model is realized by a neural network trained by a supervised learning technique such as Instance Segmentation or Panoptic Segmentation, for example. When coloring a specific comic, it is desirable for the related item detection unit 120 to use a machine learning model that has been additionally trained using learning data created from the specific comic. Thereby, the related item detection unit 120 can improve the detection accuracy of related items for a specific comic.
[0017] The coloring unit 130 includes, for example, a manual coloring unit 132, an automatic coloring unit 134, an AI coloring unit 136, and a correction unit 138. The manual coloring unit 132 colors the object based on the results of user input. The automatic coloring unit 134 automatically replicates the color of an already colored object to an uncolored object. The AI coloring unit 136 colors the uncolored object by processing based on a machine learning-based coloring model. The correction unit 138 corrects the color applied to the object by processing based on a machine learning-based correction model. The relationship between the frame detection unit 122 and the object detection unit 124, and the resulting color information applied by the coloring unit 130, is registered in a database constructed in the storage unit 140. Thus, the storage unit 140 functions as a color information registration unit. The color information is information indicating the color of an object or a part of an object, but is not limited to this, and may include information indicating the color before correction and information indicating the color after correction.
[0018] Figure 2 is a block diagram showing an example of the AI coloring unit 136 of the embodiment. The AI coloring unit 136 takes an uncolored image as input to a coloring model 136A, which is a machine learning model, and outputs a colored image from the coloring model 136A. The coloring model 136A is a machine learning model, for example, a convolutional neural network (CNN). The AI coloring unit 136 may output image data showing the colored image, but may also output data including frame information, object information, and color information showing the colors applied to each part of the object.
[0019] The coloring model 136A is trained by the model building unit 136B. The model building unit 136B acquires a dataset as training data and inputs the dataset to the coloring model 136A during training. The dataset includes, for example, colored images, but is not limited to this, and may include both colored and uncolored images. When the dataset is input, the coloring model 136A outputs a coloring result.
[0020] The model building unit 136B recursively updates the processing parameters of the coloring model 136A so that the coloring result output from the coloring model 136A matches the training data. The processing parameters are, for example, at least one of the following in a convolutional neural network: the number of layers, the number of nodes in each layer, the node connection method between layers, the activation function, the error function, and the gradient descent algorithm, the pooling region, the kernel, the weight coefficients, and the weight matrix. As a result, the model building unit 136B performs, for example, deep learning to obtain the processing parameters. Deep learning is a machine learning method that uses a multi-layered structure, especially a neural network with three or more layers. Note that the model building unit 136B does not necessarily have to be included in the AI coloring unit 136; it is sufficient if the coloring model 136A can be introduced into the image coloring device 100 during initial setup or maintenance of the image coloring device 100. When coloring, the coloring model 136A takes an uncolored image as input and outputs a colored image as the processing result.
[0021] The model building unit 136B may realize the coloring model 136A by training a neural network using a pre-published manga training dataset, for example, by employing adversarial learning techniques for image transformation such as Imate to Image Translation with Conditional Adversarial Networks. However, when coloring a specific manga, training may be performed using only training data created from that specific manga, and the coloring accuracy of the coloring model 136A can be improved by performing additional training on the neural network trained with the publicly available manga training dataset. The model building unit 136B may input monochrome and grayscale images, such as images containing solid colors or tones that are not colored, or line drawings, into the coloring model 136A, and train the coloring model 136A to output colored images. In addition to monochrome and grayscale images, the model building unit 136B may also input auxiliary information automatically acquired by object detection or segmentation as another channel into the coloring model 136A.
[0022] Figure 3 is a block diagram showing an example of the correction unit 138 of the embodiment. The correction unit 138 inputs the pre-corrected colored image to the correction model 138A, which is a machine learning model, and outputs the corrected colored image from the correction model 138A. The correction model 138A is a machine learning model, for example, a convolutional neural network. The correction unit 138 may output image data showing the corrected colored image, but may also output data including frame information, object information, and color information showing the colors applied to each part of the object.
[0023] The correction model 138A is trained by the model building unit 138B. The model building unit 138B acquires datasets of pre-correction data and post-correction data as training data, and inputs these datasets into the correction model 138A during training. The datasets may include colored images colored based on user operations, correctly corrected data, and incorrectly corrected data. When the datasets are input, the correction model 138A outputs the coloring result.
[0024] The model building unit 138B recursively updates the processing parameters of the correction model 138A so that the coloring result output from the correction model 138A matches the training data. The processing parameters are, for example, at least one of the following in a convolutional neural network: the number of layers, the number of nodes in each layer, the node connection method between layers, the activation function, the error function, and the gradient descent algorithm, the pooling region, the kernel, the weight coefficients, and the weight matrix. As a result, the model building unit 138B performs, for example, deep learning to obtain the processing parameters. Deep learning is a machine learning method that uses a multi-layered structure, especially a neural network with three or more layers. Note that the model building unit 138B does not necessarily have to be included in the correction unit 138; it is sufficient if the correction model 138A can be introduced into the image coloring device 100 during initial setup or maintenance of the image coloring device 100. During coloring, the correction model 138A takes the image before correction as input and outputs the corrected image as the processing result.
[0025] The correction unit 138 corrects the coloring results when the color information of manually colored objects is copied to uncolored objects, the color information of colored objects is copied to uncolored objects, and the coloring results are different among colored objects or differ from the desired coloring results. The correction unit 138 can correct the colors using a manual coloring function or a semi-automatic coloring function. The semi-automatic coloring function can be realized by inputting a color image to show the corrected color into a correction model 138A using adversarial learning technology and training it. The color image is one in which only a part of the object's region is colored with strokes or dots, and it is not necessary for the entire region of the object, the entire object, or the entire frame to be colored. The model building unit 136B can improve the correction accuracy by storing the colored color when an object is colored or color corrected, the color before correction and the color after correction in a database in association with the object, and training the correction model 138A.
[0026] Figure 4 shows an example of a coloring screen displayed on the display unit 112. The display control unit 114 controls the display unit 112 to display the coloring screen 300. The coloring screen 300 includes a coloring work area which includes a frame area to be colored 310 and a reference frame area 320, and a related frame area 330.
[0027] The related frame area 330 corresponds to a first area that displays multiple images containing the object. The related frame area 330 displays images (related frame images) that correspond to frames in which the object detection unit 124 has detected the object, out of the multiple frames detected by the frame detection unit 122. The related frame images may be thumbnail images that have been reduced in size from the frame images. The related frame area 330 includes a scroll bar 330a. The related frame images displayed in the related frame area 330 are displayed by scrolling according to the operation of the scroll bar 330a.
[0028] The reference frame area 320 corresponds to a second area that displays a reference frame image containing the object. The reference frame image is, for example, an image corresponding to a frame containing a colored object that the user references and compares to in order to color the object. The reference frame area 320 displays a frame image corresponding to one of the frames displayed in the related frame area 330 based on, for example, user operations.
[0029] The coloring target frame region 310 corresponds to a third region that displays the coloring target frame image containing the target object. The coloring target frame image is, for example, a frame image that contains an uncolored target object that the user intends to color.
[0030] The colored frame area 310, the reference frame area 320, and the related frame area 330 are set side by side on the display screen. In this embodiment, the colored frame area 310, the reference frame area 320, and the related frame area 330 are set side by side horizontally, but are not limited to this, and may be set side by side vertically, or two of the three areas may be set side by side horizontally or vertically.
[0031] The display control unit 114 displays a color duplication button 322, a color duplication button 312, an AI coloring button 314, and a correction button 316 on the coloring screen 300. The color duplication button 322 is a button for duplicating the color information of an object contained in the reference frame image displayed in the reference frame area 320 to an object contained in the reference frame image among the objects contained in the color target frame image displayed in the color target frame area 310. The color duplication button 312 is a button for duplicating the color information of an object contained in the color target frame image displayed in the color target frame area 310 to an object contained in the color target frame image among the objects contained in the related frame image displayed in the related frame area 330. The AI coloring button 314 is a button for coloring an object contained in the color target frame image displayed in the color target frame area 310 using the AI coloring unit 136. The correction button 316 is a button used by the correction unit 138 to apply color correction to objects included in the frame image to be colored, which is displayed in the frame area 310 to be colored.
[0032] Figure 5 is a flowchart showing an example of the processing procedure of the image coloring device 100 in the embodiment. The image coloring device 100 acquires content from the content storage device 200 according to user operation, for example, the frame detection unit 122 detects frames included in the acquired content, and registers the detected frame information in the storage unit 140 (step S100).
[0033] Next, the user interface unit 110 displays the frame image of the detected frame on the display unit 112. The reception unit 116 accepts the operation to select a frame image and displays the selected frame image as the frame image to be colored in the frame area 310 to be colored (step S102). Figure 6 shows an example of a coloring screen 300 with the frame image to be colored displayed in the frame area 310 to be colored.
[0034] Next, the object detection unit 124 detects the same object as the object included in the coloring target frame image from the frame images of the frame information registered in the storage unit 140, and displays the detected frame images as thumbnails in the related frame area 330 as related frame images (step S106). Figure 7 is a diagram showing an example of a coloring screen 300 with related frame images displayed in the related frame area 330.
[0035] Next, the reception unit 116 accepts an operation to select one frame image from the thumbnail-displayed related frame images, and the display control unit 114 displays the selected frame image as a reference frame image in the reference frame area 320 (step S108). This allows the display unit 112 to display the frame image to be colored and the reference frame image side by side. The user can color the image while maintaining color consistency by visually comparing the frame image to be colored and the reference frame image. Note that the image coloring device 100 does not need to display a frame image in the reference frame area 320 if there are no frame images to be colored. Furthermore, it is desirable that the display control unit 114 be able to freely change the area size between the frame area to be colored 310 and the reference frame area 320. Figure 8 is a diagram showing an example of a coloring screen 300 in which the frame image to be colored and the reference frame image are displayed side by side.
[0036] Next, the coloring unit 130 performs coloring on the frame image to be colored (step S110). The manual coloring unit 132 displays a palette image for selecting a color on the display unit 112 when the reception unit 116 receives an operation to select a part of the object in the frame image to be colored. If any of the colors included in the palette image is selected, the manual coloring unit 132 registers the color information of the selected color in the storage unit 140 as color information for a part of the object. The manual coloring unit 132 may set the frame area 310 to be colored as a canvas area and color the object directly using a pen-type pointer device. The color information colored by the coloring unit 130 is registered in the database in association with the frame information and object information.
[0037] When the color duplication button 312 is selected based on an operation received by the reception unit 116, the automatic coloring unit 134 can refer to the database and duplicate the color information of the objects included in the frame image to the objects included in the related frame image. This allows the coloring unit 130 to duplicate the color information that has been colored in the frame image to be colored, either manually or using AI, to the uncolored objects in the related frame area.
[0038] When the color duplication button 322 is selected based on an operation received by the reception unit 116, the automatic coloring unit 134 can refer to the database and duplicate the color information of the objects included in the reference frame image to the objects included in the frame image to be colored. As a result, when a colored frame image included in the related frame area is selected as the reference frame image, the coloring unit 130 can duplicate the color information of the selected reference frame image to the uncolored objects in the related frame area.
[0039] The AI coloring unit 136 can input the frame image to be colored into the coloring model 136A when the AI coloring button 314 is selected based on an operation received by the reception unit 116, and display the colored image output from the coloring model 136A in the frame area 310 to be colored. The correction unit 138 can input the frame image to be colored into the correction model 138A when the correction button 316 is selected based on an operation received by the reception unit 116, and display the colored image output from the correction model 138A in the frame area 310 to be colored. The correction unit 138 may specify the color of the object (before correction) to one of the colors included in the palette (after correction), and if the correction button 316 is operated, it may correct the color of the related frame image from the color of the object before correction to the color after correction.
[0040] Next, the coloring unit 130 determines whether or not coloring has been completed for all frames (step S112). If coloring has not been completed for all frames (step S112: NO), the process from step S102 onwards is repeated. If coloring has been completed for all frames (step S112: YES), the process ends.
[0041] Furthermore, the image coloring device 100 is not limited to the process shown in Figure 5. In step S100, it may detect frame images from the content and register them in the database, as well as detect objects included in the frame images, and detect and register the color information of the colored objects in the database. If the frame images contain uncolored objects, the image coloring device 100 may refer to the database and copy the color information of the colored objects to the uncolored objects.
[0042] [Effects of the embodiment] As described above, the image coloring device 100 of this embodiment allows the display area of the display unit 112 to be set in a row, with a first area for displaying multiple images containing the object, a second area for displaying a reference image containing the object, and a third area for displaying the image to be colored containing the object. The image coloring device 100 allows the colored frame image and the reference frame image to be displayed side by side, enabling color comparison across comic panels, pages, and stories. As a result, the image coloring device 100 can suppress color inconsistencies between images.
[0043] According to the image coloring device 100 of this embodiment, when an object is detected by the object detection unit 124, the frame image containing the object can be displayed in the related frame area 330. As a result, the image coloring device 100 makes it easy to select a reference frame image for coloring the frame image to be colored, thereby suppressing inconsistencies in coloring between images.
[0044] According to the image coloring device 100 of this embodiment, the object detection unit 124 associates the detected object with the frame image and registers it in the database, the object and color information are associated and registered in the database, and the coloring unit 130 can copy the color information associated with the object registered in the database to the uncolored object. As a result, the image coloring device 100 can color the same object with the same color across frame images using the object information and color information registered in the database. Furthermore, the image coloring device 100 saves the frame, page, object, and partial area of the object detected by the related item detection unit 120 in the database, so that reference frame images depicting the same object as the object included in the frame image to be colored can be placed side by side, not only for frames or pages entered in a batch, but also for frames or pages registered in the past.
[0045] According to the image coloring device 100 of this embodiment, when the object detection unit 124 detects multiple frame images containing an object, the coloring unit 130 can copy the colored object's color from the frame images in which the object was detected to the uncolored object in the frame images in which the object was detected. As a result, the image coloring device 100 can color the same object with the same color across frame images while reducing the workload for the user in coloring. The image coloring device 100 can, for example, automatically perform coloring by batch processing or the like when the frame detection unit 122 and the object detection unit 124 detect the frame and the object.
[0046] According to the image coloring device 100 of this embodiment, when the coloring unit 130 colors an uncolored object in an image in which an object has been detected, it is possible to color the same object with the same color as the object that has been colored. As a result, the image coloring device 100 can color the same object with the same color across frames of images while reducing the workload for the user in coloring.
[0047] According to the image coloring device 100 of this embodiment, when the object detection unit 124 detects multiple images containing an object, the display control unit 114 displays the multiple images in the related frame area 330. When the reception unit 116 receives an operation to select an image containing a colored object from among the images displayed in the related frame area 330, the coloring unit 130 can copy the color applied to the object in the selected image onto the uncolored object.
[0048] According to the image coloring device 100 of this embodiment, the AI coloring unit 136 is trained using images containing colored objects as training data, and can perform coloring based on a coloring model 136A that outputs an image containing a colored object when an image containing an object is input. As a result, the image coloring device 100 can reduce the workload for the user in coloring, while also being able to color the same object with the same color across frame images.
[0049] According to the image coloring device 100 of this embodiment, the correction unit 138 uses a correction model 138A, which is machine-learned using an image containing a colored object, the color information before correction when the color of the object is corrected based on user operation, and the color information after correction as training data, to perform coloring based on the correction model 138A that outputs an image containing a colored object when an image containing an object is input. As a result, the image coloring device 100 can color the same object with the same color across frames of images while reducing the workload for the user to perform color correction.
[0050] Although various embodiments and variations have been described, these are merely examples and are not limited to these. For example, one embodiment or variation, or a part of one embodiment or variation, may be combined with one or more other embodiments or variations to realize one aspect of the present invention. [Explanation of symbols]
[0051] 100...Image coloring device 110...User Interface Department 112...Display section 114...Display Control Unit 116... Reception Department 120... Related item detection unit 122... Frame detection unit 124...Object detection unit 130...Coloring section 132...Manual coloring section 134…Automatic coloring section 136…AI coloring section 136A…Colored model 136B, 138B...Model Construction Section 138... Correction section 138A...Correction Model 140...Storage section 200... Content storage device 300…Coloring screen 310... Area of the frame to be colored 312, 322… Color duplication button 314... AI coloring button 316... Correction button 320...Reference frame area 330... Related frame area 330a...Scroll bar
Claims
1. A frame detection unit that detects frame images included in the content, An object detection unit that detects an object from the aforementioned frame image, A reception desk that accepts user input, A coloring unit that colors the object based on the operation received by the reception unit, The display unit comprises a first area for displaying a plurality of frame images containing the object, a second area for displaying a reference frame image containing the object, and a third area for displaying a frame image of a colored object containing the object, arranged side by side in the display area of the display unit. When the display control unit receives an operation from the reception unit to select a frame image, it displays the selected frame image in the third area as the frame image to be colored. The object detection unit detects an object that is the same as the object included in the third region, The display control unit displays thumbnails of frame images in which the same object is detected as related frame images in the first area, and when it receives an operation to select one frame image from the thumbnail-displayed related frame images, it displays the selected frame image as a reference frame image in the second area. Image coloring device.
2. The display control unit displays a first color duplication button corresponding to the third region and a second color duplication button corresponding to the second region, When the first color duplication button is selected based on user operation, the coloring unit duplicates the color information of the object included in the coloring target frame image to the object included in the related frame image. When the second color duplication button is selected based on user operation, the color information of the object included in the reference frame image is duplicated to the object included in the color target frame image. The image coloring apparatus according to claim 1.
3. An object registration unit that associates the object detected by the object detection unit with the frame image and registers it in a database, The system includes a color information registration unit that associates the object with the color information colored by the coloring unit and registers it in the database. The coloring unit copies the color information associated with the object registered in the database to the uncolored object. The image coloring apparatus according to claim 2.
4. When the object detection unit detects multiple frame images containing the object, the display control unit displays the multiple frame images in the first area. The image coloring apparatus according to claim 3, wherein when the coloring unit receives an operation from the receiving unit to select a frame image that includes a colored object from among the frame images displayed in the first region, the coloring unit copies the color applied to the object included in the selected frame image to an uncolored object.
5. The image coloring apparatus according to claim 3, wherein the coloring unit performs coloring based on a machine learning model that is trained using the frame image containing the colored object as training data, and outputs the frame image containing the colored object when the frame image containing the object is input.
6. The image coloring apparatus according to claim 3, wherein the coloring unit performs coloring based on a machine learning model that uses an image containing a colored object, the color information before correction and the color information after correction when the color of the object is corrected based on user operation as training data, and outputs a colored image containing the object when an image containing the object is input.
7. A step of detecting frame images included in the content, The steps include detecting an object from the aforementioned frame image, The steps include displaying multiple images containing the object in the first area of the display unit, The steps include: displaying a reference image from among the images displayed in the first region in the second region of the display unit arranged in the first region; The steps include displaying an image to be colored in a third region of the display unit, which is arranged in the first region and the second region, The step includes coloring the image to be colored based on user input, When the operation to select the aforementioned frame image is received, the selected frame image is displayed in the third area as the frame image to be colored. Detect the same object as the object included in the third region, Frame images in which the same object is detected are displayed as thumbnails in the first area as related frame images, and when an operation is received to select one frame image from the thumbnail-displayed related frame images, the selected frame image is displayed in the second area as a reference frame image. Image coloring method.
8. In the computer of the image coloring device, The steps include detecting frame images contained in the content, The steps include detecting an object from the aforementioned frame image, The steps include displaying multiple images containing the object in the first area of the display unit, The steps include: displaying a reference image from among the images displayed in the first region in the second region of the display unit arranged in the first region; The steps include displaying an image to be colored in a third region of the display unit, which is arranged in the first region and the second region, The process involves performing the steps of coloring the image to be colored based on user input, When the operation to select the aforementioned frame image is received, the selected frame image is displayed in the third area as the frame image to be colored. To detect an object that is the same as the object included in the third region, Frame images in which the same object is detected are displayed as thumbnails in the first area as related frame images, and when an operation is received to select one frame image from the thumbnail-displayed related frame images, the selected frame image is displayed in the second area as a reference frame image. program.