Information processing system, information processing method, and computer program

Abstract

An information processing system that performs processing related to animation production is provided. [Solution] The information processing system includes an acquisition unit that acquires original drawings for animation production created by a first user, and an output unit that inputs the original drawings into a machine learning model that has learned the relationship between modifications to the original drawings made by a second user different from the first user, and outputs the modifications to the original drawing data. The acquisition unit further acquires supplemental information about the original drawings. Then, the output unit inputs the supplemental information together with the original drawings into the machine learning model, and outputs the modifications to the original drawings.

Description

[Technical Field]

[0001] The technology disclosed in this specification (hereinafter referred to as "the present disclosure") relates to an information processing system, an information processing method, and a computer program for performing processing related to animation production. [Background technology]

[0002] Since animation is composed of a large number of moving images, creating the moving images requires a huge amount of effort and expense. For this reason, the introduction of digital technology using computers into the animation production process is being promoted. For example, a moving image creation method has been proposed that includes a process of creating a tracing moving image by tracing original drawings with paths, and a process of creating an in-between moving image by automatic calculation based on a pair of original moving images composed of paths (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-90830 [Non-patent literature]

[0004] [Non-Patent Document 1] Vision-Language Models for Vision Tasks: A Survey(arXiv:2304.00685v2 [cs.CV] 16 Feb 2024) [Non-patent document 2] Learning Transferable Visual Models From Natural Language Supervision(arXiv:2103.00020v1 [cs.CV] 26 Feb 2021) Summary of the Invention [Problem to be solved by the invention]

[0005] An object of the present disclosure is to provide an information processing system, an information processing method, and a computer program that perform processing related to drawing work in the animation production process. [Means for solving the problem]

[0006] The present disclosure has been made in consideration of the above problems, and a first aspect thereof is: an acquisition unit that acquires original drawings related to animation production created by a first user; an output unit that inputs the original image into a machine learning model that has learned the relationship between modifications made to the original image by a second user different from the first user, and outputs the modifications made to the original image; The information processing system is provided with the above.

[0007] However, the term "system" used here refers to a logical collection of multiple devices (or functional modules that realize specific functions), regardless of whether each device or functional module is contained within a single housing. In other words, both a single device consisting of multiple parts or functional modules and a collection of multiple devices are considered "systems."

[0008] The acquisition unit further acquires supplemental information related to the original drawings. The output unit then inputs the supplemental information along with the original drawings into the machine learning model and outputs corrections to the original drawings. The supplemental information includes at least one of plot information, script information, script information, setting information, scenario information, storyboard information, and layout information related to animation production. The output unit also outputs, as corrections to the original drawings, at least one of a corrected original drawing obtained by correcting the original drawing and a comment related to the correction of the original drawing.

[0009] The information processing system according to a first aspect further includes a collection unit that collects training data for training the machine learning model, and a learning unit that trains the machine learning model using the training data collected by the collection unit. For example, the information processing system further includes a correction input unit that allows the second user to input corrections to the original image, and the collection unit collects training data based on information input by the second user to the correction input unit.

[0010] Furthermore, a second aspect of the present disclosure is An acquisition step of acquiring original drawings related to animation production created by the first user; an output step of inputting the original image into a machine learning model that has learned the relationship between modifications to the original image made by a second user different from the first user, and outputting the modifications to the original image; The information processing method has the following features.

[0011] Furthermore, a third aspect of the present disclosure is an acquisition unit for acquiring original drawings related to animation production created by the first user; an output unit that inputs the original image into a machine learning model that has learned the relationship between modifications made to the original image by a second user different from the first user, and outputs the modifications made to the original image; It is a computer program written in a computer-readable format to cause a computer to function as a

[0012] A computer program according to a third aspect of the present disclosure defines a computer program written in a computer-readable format to perform predetermined processing on a computer. The computer program can be provided to a computer capable of executing various program codes in a computer-readable format via a storage medium or communication medium, such as an optical disk, a magnetic disk, or a semiconductor memory, or via a communication medium such as a network. By installing the computer program according to the third aspect of the present disclosure on a computer via any of these media, a cooperative effect is exerted on the computer, and the same effects as those of the information processing system according to the first aspect of the present disclosure can be obtained. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram showing the functional configuration of an information processing system 100. As shown in FIG. [Figure 2] FIG. 2 is a diagram showing the flow of collecting new training data. [Figure 3] FIG. 3 shows the conversion flow when existing data is used as training data. [Figure 4] FIG. 4 is a diagram illustrating supplementary information (script information). [Figure 5] FIG. 5 is a diagram illustrating supplementary information (script information). [Figure 6] FIG. 6 is a diagram illustrating supplementary information (storyboard information). [Figure 7] FIG. 7 is a diagram illustrating supplementary information (layout information). [Figure 8] FIG. 8 is a diagram showing an example of the annotation UI. [Figure 9] FIG. 9 is a diagram showing the configuration of the animation director model. [Figure 10] FIG. 10 is a flowchart showing the procedure by which the key artist and animation director work using the animation director model. [Figure 11]FIG. 11 is a diagram showing an example of the configuration of a UI screen that takes into consideration the application of an animation director model used in creating key animation. [Figure 12] FIG. 12 is a diagram showing an example of the hardware configuration of the information processing device 2000. [Figure 13] FIG. 13 shows an example of the structure of an animation work that is created by overlapping multiple cells. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the present disclosure will be described in the following order with reference to the drawings.

[0015] A. Overview B. System Configuration B-1. Data collection section B-1-1. New data collection B-1-2. Reusing existing data B-1-3.Annotation UI B-2. Model learning section B-3. ​​Model Utilization Department C. Variations D. Configuration of information processing device

[0016] A. Overview The animation production process is broken down into several steps, and each step requires the work of multiple people. Different people are often assigned to different scenes for the drawing tasks, and some productions involve as many as dozens of animators. For this reason, there is a role called an animation director who oversees all the animators. "Animation" here refers to the process of drawing key animations from storyboards, which then lead to the animation work that fills in the gaps between the key animations (storyboards generally include rough sketches expressing the story and directorial intent, as well as written instructions). The animation director generally corrects the finer details of the key animation, such as composition and facial expressions, and may also make corrections to the in-between animation, which is the section where movement is added.

[0017] The animation director is responsible for unifying the animation drawings and is essential for improving the quality of the work. The animation director may actually make corrections to the drawings themselves or give orders. Depending on the scale of the work, there may be multiple animation directors, and some works may have a chief animation director who oversees the animation directors. However, for the sake of simplicity, this specification will be explained assuming a one-level structure consisting of multiple animation staff and an animation director who oversees them.

[0018] When the animation director checks the original drawings created by the key animator and adds corrections and comments, this is called "animation director corrections." Generally, the animation director writes corrections and comments by hand on a correction sheet during the animation director's corrections process and returns it to the key animator. The key animator then corrects the original drawings based on the correction sheet from the animation director and has them checked again by the animation director. This flow of creating original drawings → animation director corrections → original drawing corrections is repeated until the animation director gives his or her approval (OK). Typically, one scene in an animation work is made up of multiple overlapping cels. The animation work is then completed by arranging the completed cels for each scene in chronological order. The animation director's corrections are made on the original drawings for each cel and scene.

[0019] The role of animation directors is to unify the drawings of numerous key animators and to ensure the quality of the work. However, since there are usually only a few animation directors, animation director corrections become a bottleneck in the animation production process and often require key animators to go back and do the work. Also, because animation directors and others have different approaches to animation director corrections, understanding their intentions and working on them can be a burden, not only for beginners but also for experienced artists. In key animators, storyboards are rough sketches, and it is difficult to reflect higher-level context such as detailed facial expressions and directorial intentions.

[0020] This disclosure relates to a technique for drawing animation, particularly key animation, and is primarily a technique for streamlining communication between animators and animation directors. Specifically, this disclosure relates to a technique for automatically generating correction sheets and correction comments from animation directors for drawings based on key animation during the drawing process. Similarly, this disclosure is believed to be applicable to moving images.

[0021] This disclosure uses a trained model that learns patterns of past key animation and animation director corrections, including supplementary information such as scripts. In this specification, this trained model is referred to as the "animation director model." The animation director model predicts or estimates the animation director's corrections to the key animations created by the key animator, and automatically generates correction sheets and correction comments. Furthermore, the animation director model automatically highlights parts of the key animation that are likely to be corrected by the animation director during the key animation worker's animation work and issues an alert, thereby reducing the amount of rework required by the key animation worker due to the animation director's corrections. In addition to issuing alerts, the animation director model also illustrates (generates) candidate corrections, providing the key animation worker with examples of desirable corrections. The animation director model is designed to predict the actual corrections made by the animation director. By having the key animation worker correct the key animation using the animation director model in advance, the bottleneck of the animation director's correction work can be alleviated.

[0022] Because an animation director model cannot completely replace an animation director, even if a key animator uses an animation director model, it is not possible to completely eliminate rework. Therefore, in this disclosure, the results of rework are fed back in the form of human-in-the-loop, thereby improving the quality of the animation director model and reducing the occurrence of rework. Here, human-in-the-loop refers to the inclusion of human interaction, i.e., human feedback, in the machine learning process and life cycle (loop) of the animation director model, which consists of data collection and preprocessing, training of the animation director model, operational monitoring of the trained animation director model, and continuous updates.

[0023] According to the present disclosure, it is possible to reduce rework in the key animation work, which results in more efficient drawing work and ensures the quality of the animation work.

[0024] This disclosure can solve both the challenges of developing data models and the challenges of operating the models.

[0025] Challenges when developing a data model: Currently, animation directors mainly make corrections and add comments directly to the original drawings. Therefore, when collecting training data for an animation director model, it is necessary to sort out which parts of the original drawings have been corrected, which parts the comments are about, and which information is unnecessary. This disclosure solves this problem by reusing existing data and designing a user interface (UI) that reduces the cost of creating data.

[0026] Challenges in model operation: There are two issues with model operation: even when using a model, animation director corrections are never completely eliminated, and rework itself occurs, and there is a risk of accuracy being reduced when working on new works. According to this disclosure, these issues are resolved by building a system that improves the accuracy of the model through human-in-the-loop during operation.

[0027] It should be noted that the effects described in this specification are merely examples, and the effects brought about by the present disclosure are not limited thereto. Furthermore, the present disclosure may have additional effects in addition to the effects described above. Further objects, features, and advantages of the present disclosure will become apparent from the following detailed description of the embodiments and the accompanying drawings.

[0028] B. System Configuration 1 shows the functional configuration of an information processing system 100 according to the present disclosure. The illustrated information processing system 100 includes a data collection unit 101, a model learning unit 102, and a model utilization unit 103.

[0029] The data collection unit 101 is a functional module for collecting training data for learning the animation director model. The data collection unit 101 mainly collects data in two ways: "reusing existing data," which collects training data from existing data accumulated at the animation production site (i.e., data that has been corrected by the animation director in the past), and "collecting new data," which collects training data through corrections made by the animation director to new key drawings.

[0030] As an example of reusing existing data, the data collection unit 101 uses a "past works database" accumulated at animation production sites to create training data for learning the animation director model. The past works database contains a wealth of data that can be used to learn the animation director model, such as key animation drawings before and after corrections by the animation director, as well as supplementary information about animation works. The data collection unit 101 converts this data into training data. In the example shown in Figure 1, a "part decomposition" process is performed to break down the key animation drawings before and after corrections stored in the past works database into their respective parts, a "comment detection" process is performed to detect comments from the correction sheets for each pre-correction key animation, and a "part-comment correspondence" process is performed to associate each part with each comment made by the animation director. This results in training data consisting of data sets for each part of the key animation: [pre-correction key animation, post-correction key animation, comments, supplementary information], and is then stored in the correction database.

[0031] Furthermore, as new data collection, the data collection unit 101 creates new training data for learning the animation director model. In the example shown in Figure 1, training data is collected through the animation director's confirmation of the original drawings (editing by the animation director) carried out within the model utilization unit 103. Specifically, the animation director performs the original drawing corrections using an "annotation UI" (described later) that associates the parts to be corrected with comments, and in parallel with the original drawing confirmation, the animation director collects training data consisting of data sets of [original drawing before correction, original drawing after correction, comments, supplementary information] for each part of the original drawing and stores this in the correction database.

[0032] The model learning unit 102 is a functional module that learns the animation director model based on training data (correction database) stored in the correction database. The animation director model is a machine learning model that predicts or estimates the animation director's corrections to the original drawings created by the key animator. The animation director model has, for example, an encoder-decoder format, and the model learning unit 102 performs supervised learning using the training data stored in the correction database. Alternatively, the model learning unit 102 may learn the animation director model in the form of fine-tuning to a large-scale visual language model.

[0033] The model utilization unit 103 is a functional module for using the animation director model learned by the model learning unit 102 in the drawing work of the actual animation production process.

[0034] Animators create the original drawings for an animation work. The animation director model inputs the original drawings created by the original animator and supplementary information about the animation work, predicts or estimates the animation director's corrections, and automatically generates correction sheets and correction comments. The original animator then makes corrections to the original drawings input into the animation director model based on the correction sheets and correction comments for the original drawings output from the animation director model.

[0035] It is assumed that key animators will create and correct key drawings using the "Key Drawing Work UI" (described below). This key drawing work UI has the function of displaying, on separate layers, the original key drawings initially created by the key animator and each key drawing corrected according to correction instructions output from the animation director model and corrections by the animation director, and the function of displaying comments from the animation director model and the actual animation director for each correction layer.

[0036] The key animations corrected by the key animator based on the corrections and comments made by the animation director model are then checked by the animation director, i.e., corrected by the animation director. The animation director's corrections are expected to be performed using the "annotation UI" (described below). Using this annotation UI, the parts of the key animation that the animation director has corrected are matched with the comments, and in parallel with the animation director's corrections, training data consisting of data sets for each part of the key animation (original key animation, corrected key animation, comments, supplementary information) is collected and saved in a correction database.

[0037] The key animator will make further corrections to the key animation based on the animation director's corrections. Here too, it is assumed that the key animator will make corrections using the "Key Animation Work UI" (described below). The animation director will then make further corrections to the key animation that has been corrected by the animation director. It is assumed that the "Annotation UI" (described below) will also be used for corrections from the second time onwards by the animation director, and training data for the animation director model is collected in parallel with the animation director's corrections. Once the key animation has been confirmed by the animation director, it is output to the subsequent process.

[0038] Even when an animation director model is used, rework by the animation director may occur. According to the information processing system 100, the results of rework are fed back in a human-in-the-loop format, and training data for the animation director model is added, thereby improving the quality of the animation director model.

[0039] 1 can be configured using information devices such as computer systems. Of course, each functional module, such as the data collection unit 101, the model learning unit 102, and the model utilization unit 103, may be configured using a different computer system. Furthermore, a functional module that performs processing with a high computational load, such as the model learning unit 102, may be configured using multiple computer systems to utilize parallel processing.

[0040] In addition, the original drawing worker may create and modify the original drawing using the ``original drawing work UI'' (described below) on a device physically separated from the information processing system 100, such as a tablet terminal or a personal computer (PC), and the data resulting from the work may be transferred to the information processing system 100.

[0041] In addition, the animation director's correction work using the "annotation UI" (described below) may be performed on a device physically separated from the information processing system 100, such as a tablet terminal or personal computer (PC), and the animation director's correction data may be transferred to the information processing system 100.

[0042] Each unit of the information processing system 100 will be described in detail below.

[0043] B-1. Data collection section Training data is necessary to build an animation director model. In the information processing system 100, the data collection unit 101 handles two flows: new data collection and reuse of existing data.

[0044] B-1-1. New data collection In the new data collection flow, the data collection unit 101 collects new training data for learning the animation director model. The data collection unit 101 collects training data in conjunction with the flow in which the animation director makes corrections to the original drawings, which is carried out in the model utilization unit 103. This is intended to incorporate the collection of new training data into the animation production process as the drawing work progresses.

[0045] In general, animation director corrections require additional work to be used as training data, resulting in low machine readability. Therefore, in this disclosure, in order to improve machine readability, we consider tagging each item corrected by the animation director. It is also possible to implement the tagging process for each item using the annotation UI (described below).

[0046] 2 shows an outline of a new data collection flow in which the data collection unit 101 collects new training data in conjunction with production supervision and correction. The data collection unit 101 includes a parts decomposition unit 201 and an annotation unit 202 as functional modules for collecting new data.

[0047] Parts Disassembly Section: During the animation director's revisions, the director checks whether the characters look out of place compared to other scenes and whether the background composition is appropriate. While the entire drawing may be redrawn during the director's revisions, revisions are typically made to individual parts. Therefore, it is important to check whether the animation director will make any revisions to individual parts (e.g., eyes, mouths, or hands for a character). Therefore, the part decomposition unit 201 detects each part from the original drawing before revisions and prepares to associate each part with annotations, such as revisions and comments made by the animation director. The part decomposition unit 201 decomposes the original drawing into parts based on the items frequently checked by the animation director. The part decomposition unit 201 may extract each part from the original drawing using an object recognition model, or may have the object recognition model learn items frequently checked by the animation director in the past.

[0048] Annotation part: The annotation unit 202 is a functional module that allows the animation director to add annotations such as corrections and comments to each part into which the original drawings have been decomposed by the part decomposition unit 201. The annotation unit 202 is equipped with a display device that presents the target original drawings to the animation director, and an input device for inputting annotations such as corrections and comments on the original drawings by handwriting or by other means.

[0049] The animation director adds corrections and comments to each part of the original drawing as part of the animation director's corrections in the annotation unit 202. The annotation unit 202 outputs data consisting of a set of [original drawing before correction, original drawing after correction, comment, supplementary information] for each part included in the original drawing. The output data from the annotation unit 202 is saved in the correction database 203 and used as training data for the animation director model.

[0050] Furthermore, when adding corrections or comments to each part of the original drawing, the animation director may assign metadata such as a priority. This is because, by encouraging the original artist to start making corrections from the highest priority points within the time-constrained original drawing work, it is possible to shorten the overall work time. Alternatively, the annotation unit 202 may automatically assign a priority to corrections and comments for each part of the original drawing. Metadata such as the priority of each part of the original drawing is also stored in the correction database 203.

[0051] As already explained with reference to Figure 1, the new data collection flow by the data collection unit 101 is incorporated into the animation supervision revisions carried out in the model utilization unit 103. The animation supervision revisions are expected to be made using the "annotation UI" (described later). By using the annotation UI, parts of the key animation that have been revised by the animation director are associated with comments, and in parallel with the animation supervision revisions, training data consisting of data sets for each part of the key animation, [key animation before revision, key animation after revision, comments, supplementary information], is collected and saved in a revision database.

[0052] In some cases, the animation director may use the annotation unit 202 to provide comments not for each part but for the entire key animation. Furthermore, in an animation work composed of multiple overlapping cells, the animation director may provide comments covering multiple consecutive scenes for a certain cell only for one of the key animations, or may quote comments made for another cell for a certain cell. For example, in an animation work composed of overlapping three layers of cells, as shown in FIG. 13, namely, cell A depicting an object in the foreground, cell B depicting an object in the background, and cell C depicting the background, the animation director may provide comments such as, "The comment written in the key animation for the sixth scene in cell A should be applied to the subsequent specified scene interval," or "The comment written in the eighth cell in cell A should also be applied to the same scene in cell B" (in practice, comments are generally written in concise terms that can be understood by all the workers). The annotation unit 202 or the annotation UI applied by the annotation unit 202 preferably has a function that allows the animation director to efficiently input not only part-by-part comments, but also overall comments on the master drawings, comments covering master drawings of multiple consecutive scenes, and comments spanning multiple cells, and also has a function that allows these comments to be appropriately collected as new training data.

[0053] B-1-2. Reusing existing data As mentioned above, animation director corrections are widely performed in existing animation production, and data and comments (existing data) before and after the animation director's corrections are accumulated at animation production sites. However, this existing data is not in a state that can be used as is for learning the animation director model, so it needs to be converted into training data. Therefore, in the existing data reuse flow, the data collection unit 101 performs data conversion so that the existing data can be reused as training data for the animation director model.

[0054] 3 shows an outline of the existing data reuse flow in which the data collection unit 101 reuses a "past works database" accumulated at animation production sites to create training data for learning an animation director model. The data collection unit 101 includes functional modules for reusing existing data: a parts decomposition unit 301, a comment detection unit 302, a parts / comment correspondence unit 303, and a data storage unit 304.

[0055] Data Entry: The data collection unit 101 inputs pre-edited and edited original drawings from the past works database 305 as basic training data, and also inputs any other reference information as additional information. The past works database 305 stores existing data generated in animation production sites, such as pre-edited original drawings and edited original drawings, correction sheets used during animation director edits, and other supplementary information. This existing data cannot be used as is for learning the animation director model, so it needs to be converted into training data.

[0056] Supplementary information includes the plot, script, script, setting, scenario, storyboard, layout, etc. of an animation work. The key animator chooses which supplementary information to use when drawing the key animation. With regard to supplementary information, various points can be taken into consideration, as key animation work involves going through various processes in animation production. Supplementary information used in creating the key animation can also be used to train the animation director model. In addition, correction sheets on which the animation director handwrites corrections and comments during the animation supervision corrections can be used as supplementary information to train the animation director model.

[0057] Figures 4 to 7 show examples of supplementary information such as script information, script information, storyboard information, and layout information. Script information (Figure 4) includes the lines of the characters as well as brief information about the scene and a description of the scenery. Script information (Figure 5) describes the scenery and dialogue for each cut. Storyboard information (Figure 6) describes the composition of each cut, the movements of the actors, and information about the content, sound effects, dialogue, and time of each cut. Layout information (Figure 7) shows detailed information about the screen layout. Supplementary information such as that shown in Figures 4 to 7 can be used by the supplementary information animation director to revise the key animation.

[0058] Parts Disassembly Section: The part decomposition unit 301 detects each part from the unedited original drawings input from the past works database 305 and prepares to associate each part with annotations such as edits and comments made by the animation director. The part decomposition unit 301 may be similar to the part decomposition unit 201 for collecting new data described above, and decomposes the original drawings into parts for each item that the animation director frequently checks. The part decomposition unit 301 may extract each part from the original drawings using an object recognition model, or may have the object recognition model learn items that the animation director frequently checked in the past.

[0059] Comment detector: Animation director corrections are often handwritten on correction sheets, and converting these handwritten characters into text data is necessary for learning the animation director model. Therefore, the comment detection unit 302 performs optical character recognition (OCR) on the handwritten correction sheets to convert the images into text data. During this process, multiple comments are converted into text individually.

[0060] Parts and comments: A single correction sheet may contain comments for multiple locations, or may contain comments not directly related to corrections (e.g., greetings such as "Thank you in advance"). Therefore, the part / comment matching unit 303 associates each comment detected from the correction sheet by the comment detection unit 302 with a specific part of the original drawing decomposed by the part decomposition unit 301, or whether the comment is unrelated to the animation itself. Correction comments often point out aspects that require particular attention in animation, and various technical terms, such as eye shadow, are used. The part / comment matching unit 304 associates these terms with parts of the original drawing. By associating each part with each comment made by the animation director, the part / comment matching unit 303 converts the original drawings and correction sheets, both before and after correction, stored at the animation production site into training data consisting of a set of data for each part of the original drawing: [original drawing before correction, original drawing after correction, comment, supplementary information].

[0061] Animation directors fill out correction sheets in various ways. Some correction sheets include comments on character corrections, such as the eyes, nose, hair, and mouth, while others include comments on non-character corrections, such as the background. Some correction sheets include numerous comments for a single original drawing, while others include no comments, and corrections are simply handwritten directly onto the original drawing. The part / comment matching unit 303 may associate parts with comments based on physical information, such as the distance between the part and the location where the correction comment is written. The part / comment matching unit 303 may also associate parts with comments using, for example, an image classification model that learns the association between images and text, such as CLIP (Contrastive Language-Image Pre-training) (see Non-Patent Document 2).

[0062] In addition, the animation director may enter not only correction comments for each part but also overall correction comments for the key drawings on the correction sheet. Furthermore, the animation director may enter correction comments on the correction sheet for one key drawing to indicate corrections that cover key drawings for multiple consecutive scenes, such as "apply the comment entered for the key drawing in cell A for the sixth scene to a specific scene section thereafter," or may enter correction comments on the correction sheet for one key drawing to indicate corrections that cover other cells in the same scene, such as "apply the comment entered for cell A for the eighth scene to cell B for the same scene." In these cases as well, the parts / comment correspondence unit 303 associates the comments entered on the correction sheet for one key drawing with parts and comments for all the corresponding key drawings.

[0063] Data storage: The data storage unit 304 stores data consisting of sets of data for each part of the original drawing, including [original drawing before modification, original drawing after modification, comment, supplementary information], which have been shaped based on the part-comment correspondence made by the previous part-comment correspondence unit 303. Each set of data stored in the data storage unit 305 becomes training data used to learn the animation director model. The data storage unit 305 may be shared with the modification database 203 used to collect new data.

[0064] B-1-3.Annotation UI The animation director uses the annotation UI to make corrections to the original drawings. The annotation UI has a function that associates the parts to be corrected with the comments entered by the animation director, so training data consisting of data sets for each part of the original drawing (original drawing before correction, original drawing after correction, comments, supplementary information) can be collected in parallel with the original drawing confirmation.

[0065] An example of the annotation UI is shown in Figure 8. The animation director can use the annotation UI shown in the figure to add annotations such as corrections and comments to each key drawing. When the animation director makes corrections on a device physically separated from the information processing system 100, such as a tablet terminal or PC, the annotation UI shown in Figure 8 is displayed on the screen of the tablet terminal or PC held by the animation director.

[0066] The annotation UI shown in FIG. 8 includes a correction sheet display section 801, a production supervisor correction section 802, and a confirmation section 803.

[0067] The correction sheet display area 801 occupies the left half of the annotation UI and displays the correction sheet filled in by the animation director. The correction sheet display area 801 may display a scanned image of a paper correction sheet filled in by hand by the animation director, or may allow corrections and comments to be written directly on the screen of the correction sheet display area 801 using a stylus pen.

[0068] The animation supervision correction section 802 and the confirmation section 803 are displayed superimposed on the right half of the annotation UI. When the "Correction" tab indicated by reference numeral 804 is selected, the animation supervision correction section 802 appears in the foreground. Similarly, when the "Confirmation" tab indicated by reference numeral 805 is selected, the confirmation section 803 appears in the foreground. In short, by selecting either the correction tab 804 or the confirmation tab 805, the display of the animation supervision correction section 802 and the confirmation section 803 can be switched. For ease of explanation, FIG. 8 shows the animation supervision correction section 802 and the confirmation section 803 side by side, each appearing in the foreground.

[0069] Animation Supervision and Editing: The animation director correction unit 802 has a function that allows the animation director to select parts he or she wants to correct and to enter comments for the selected parts. The animation director correction unit 802 allows the animation director to individually select parts to be annotated or to specify a target range, and then add corrections or comments to the selected parts or the specified range.

[0070] The part decomposition unit 201 / 301 decomposes the original drawings into parts for each item that the animation director frequently checks (as described above). The parts into which the original drawings are divided by the part decomposition unit 201 / 301 are listed in the animation director correction unit 802 as "selected parts" indicated by the reference numeral 806. The "selected parts" list may be in the form of a drop-down menu.

[0071] The animation director can individually select parts for which he or she wishes to write a comment from this "selected parts" list 806. Then, in the correction sheet display area 801 in the left half of the UI screen, the parts selected in the "selected parts" list 806 are displayed in the form of bounding boxes. In the example shown in Figure 8, "eyes" are selected in the "selected parts" list 806 in the animation director correction area 802 and switched to a highlighted display, and in the correction sheet display area 801, a bounding box indicated by reference numeral 807 is displayed in the area of ​​the eyes in the original drawing.

[0072] Also, a "Specify Range" button, indicated by the reference numeral 808, is provided to the right of the "Selected Parts" list 806. The animation director can simultaneously specify multiple parts within a range to which he or she wishes to make corrections or add comments by pressing this "Specify Range" button 808 and then dragging the "Selected Parts" list 806. In this case, all parts within the specified range are highlighted in the "Selected Parts" list 806, and are also displayed in the correction sheet display area 801 as a bounding box (not shown) that surrounds all parts within the specified range.

[0073] After selecting the part that the animation director wants to correct, he or she presses the "Correction Work" button, indicated by the symbol 809, located below the drop-down menu. A separate layer ("Correction Layer") (not shown) will then appear in the correction sheet display area 801, and the animation director can make corrections to the original drawing on that layer, for example, by using a stylus pen.

[0074] The animation director can also enter annotations such as corrections or comments that he or she wishes to make to an individually selected part or a specified range of parts in a comment input field indicated by the reference symbol 810, which is provided below the "Revision Work" button 809. Entry into the comment input field 810 may be made, for example, in the form of text input using a keyboard, or in the form of voice input and voice recognition. In the example shown in FIG. 8, the comment "Please refer to Mekage / Episode 3" has been entered in the comment input field 810 for the key animation. Among the entered comments, the phrase "Please refer to Episode 3" indicates that the corrections made to the key animation of another scene (Episode 3) should also be applied to the key animation currently being corrected.

[0075] confirmation: The confirmation unit 803 has the function of confirming all of the corrections entered by the animation supervision correction unit 802. The confirmation unit 803 displays a list of corrections for each part of the original drawing entered by the animation supervision correction unit 802. Parts to which no corrections have been made are displayed as "no corrections" on the list. If it is not possible to display all of the parts in a list on the screen, vertical scrolling is used. The animation director can prevent any omissions in corrections to the original drawing by looking at the list displayed by the confirmation unit 803 and checking comments on parts to which corrections have been made and parts to which no corrections have been made.

[0076] 8, the animation director can correct each part of the master drawing and add correction comments (i.e., annotations), and in parallel with the animation director's corrections, a set of data for each part of the master drawing (master drawing before correction, master drawing after correction, comments, supplementary information) can be collected and used as training data for the animation director model in the model learning unit 102. In this way, by feeding back the results of rework in a human-in-the-loop format, the quality of the animation director model can be improved and the occurrence of rework can be reduced.

[0077] B-2. Model learning section The animation director model inputs the original key drawings and supplementary information about the animation work (storyboard, layout, script, screenplay, settings, correction sheets, etc.) to predict or estimate the actual key drawings that the animation director will make corrections to, and generates corrected key drawings (including differences in partial corrections) and correction comments. While the key animator is working on the animation, the animation director model automatically highlights and alerts parts of the key drawings that are likely to be corrected by the animation director. Based on the corrected key drawings and correction comments output by the animation director model, the key animator can correct the key drawings before the actual animation director makes corrections, reducing the amount of rework from the animation director to the key animator. Key animator corrections are one of the bottlenecks in the animation production process, but using the animation director model to reduce rework can improve work efficiency.

[0078] 9 shows an example of the configuration of an animation director model. The illustrated animation director model 900 has a general encoder-decoder configuration. That is, the animation director model includes an image encoder 901, a text encoder 902, an image decoder 903, and a text decoder 904.

[0079] An image encoder 901 receives an unmodified original drawing as an input and generates an intermediate representation (image feature quantity) 911, while a text encoder 902 receives supplemental information for the unmodified original drawing and generates an intermediate representation (text feature quantity) 912. The supplemental information includes the plot, setting, scenario, storyboard, layout, and correction sheets created during the animation director's corrections of the corresponding work, and is supplied, for example, from a work database 905. An image decoder 903 and a text decoder 904 then generate modified original drawings and correction comments from the intermediate representations 911 and 912 of the unmodified original drawing and the supplemental information, respectively.

[0080] The model learning unit 102 performs supervised learning of the animation director model using training data collected by the data collection unit 101, which consists of data sets for each part of the original drawing: [original drawing before correction, original drawing after correction, comments, supplementary information].

[0081] The model learning unit 102 may learn the animation director model from scratch using only training data, or may achieve this by fine-tuning a large-scale visual language model such as a Vision Language Model (VLM). A VLM is a model that can simultaneously understand and process visual information (images and videos) and text information (language data) (see Non-Patent Document 1).

[0082] In addition, since data such as original drawings and settings for animation production is not widely available on the Internet, it is expected that the amount of data available as training data for distributed large-scale language models will be small. Therefore, it is possible to prepare a new large-scale visual language model specialized for animation using animation production data.

[0083] The degree to which the revised original drawings and revision comments output from the animation director model are reliable varies depending on the input data or output content. Therefore, it is conceivable that the animation director model could output the degree of reliability of the comments based on its internal representation in the form of a numerical value such as a reliability level. In this case, it is also conceivable to set a threshold and output only those that are reliable above a certain level.

[0084] B-3. ​​Model Utilization Department The model utilization unit 103 utilizes the animation director model learned by the model learning unit 102 in the animation work that is part of the actual animation production process. Figure 10 shows in the form of a flowchart the processing procedure in which the key artist and animation director work using the animation director model.

[0085] Original artwork: In step S1001, the original artist draws an original drawing as usual and completes the original drawing. The original artist may use the original drawing work UI (see FIG. 11) described later, or may draw an original drawing as usual.

[0086] Select additional information: As described above, key animators use supplemental information such as the plot, script, screenplay, setting, scenario, storyboard, and layout of an animation work as context information related to animation. In step S1002, a user (e.g., a key artist) selects which supplemental information to use from the past works database when generating revision comments using the animation director model. Alternatively, the supplemental information to use may be automatically selected based on related information such as cut information.

[0087] Animation Director Model: In step S1003, the original drawings created by the key animator are input into the animation director model, and corrections and comments on the original drawings are obtained as output. The key drawings created by the key animator in step S1001 and the supplemental information selected in step S1002 are input into the animation director model.

[0088] Rather than presenting the proposed revisions obtained from the animation director model directly to the original artist, it is possible to simply issue an alert for the parts of the original drawing that need to be revised, or to provide an alert and also suggest what kind of revisions are desirable.

[0089] In this case, instead of presenting all revision proposals output from the animation director model to the key artist at the same time, they may be presented in order of priority. There is no particular limitation on how the priorities are determined. For example, the animation director model may set the priorities. The animation director model may also learn the priorities assigned by real animation directors.

[0090] Furthermore, the reliability of the revised original drawings and revision comments output from the animation director model varies depending on the input data or output content. Therefore, the animation director model outputs the degree of reliability of the comments based on the internal representation in the form of a numerical value such as a reliability level. Then, in step S1003, it is possible to present only a certain number of revision proposals output from the animation director model with a reliability level that exceeds a certain threshold.

[0091] Original artist self-confirmation: In step S1004, the key artist checks the corrections and comments on the key animation output from the animation director model in step S1003, and determines whether or not it is OK to proceed directly to the animation director's confirmation.

[0092] Here, if the key artist chooses to make corrections to the original drawings based on the corrections and comments output from the animation director model (NG in step S1004), the process returns to step S1001, and the key artist corrects the original drawings based on the corrections and comments from the animation director model. The key artist uses the key artist work UI (see FIG. 11) to correct the original drawings. Each time a key artist corrects a key artist, a correction layer is added to the key artist work UI (described below). However, the key artist does not need to follow all the corrections and comments output from the animation director model, and may correct only the corrections and comments that concern him.

[0093] On the other hand, if the original artist chooses not to modify the original drawing (OK in step S1004), the process proceeds to the subsequent step S1005.

[0094] Animation director confirmation: In step S1005, the animation director receives the original drawings that have passed the self-confirmation by the original artist in step S1004, and makes corrections and comments to the original drawings. The animation director may add corrections and comments to the original drawings using the annotation UI (see Figure 8), or may fill out a correction sheet as in the past.

[0095] Here, if the animation director wants the original artist to make corrections to the original drawings that he or she has confirmed himself or herself, that is, if there are corrections to be made by the animation director (NG in step S1005), the process goes back to the original artist. That is, the process returns to step S1001, and the original drawings are returned to the original artist along with any corrections or comments, and the original artist is instructed to make corrections to the original drawings.

[0096] On the other hand, if the animation director checks the original drawings that have passed the artist's self-confirmation and approves them (OK in step S1005), the work on the original drawings is completed and the original drawings are handed over to the next process.

[0097] It is also possible to replace the work data in the model utilization unit 103 with data collection. That is, when the animation director makes corrections using the annotation UI (described above), training data consisting of a set of data for each part of the original drawing (original drawing before correction, original drawing after correction, comments, supplementary information) is collected in parallel and saved in a correction database.

[0098] In the flowchart shown in FIG. 10, steps S1001 to S1004 are performed by the original artist, and only step S1005 is performed by the animation director. Whether step S1004 involves modifying the original drawings based on corrections and comments output from the animation director model or step S1005 involves modifying the original drawings based on corrections and comments added by the animation director, the process is the same in that the original artist returns to step S1001 and performs the original drawings. However, the loop consisting of steps S1001 to S1004 is a closed process performed by the original artist, and even if it is repeated multiple times, it can be completed efficiently and in a short time. On the other hand, it takes time for the original drawings to be reviewed by the animation director in step S1005, and multiple reviews can be repeated, resulting in a long wait. Furthermore, because the animation director reviews the original drawings of multiple original artists, even if an original drawing that has been self-reviewed by a certain original artist is transferred to the animation director for review, there is a waiting period before the review can begin.

[0099] According to the present disclosure, by using the animation director model, the original artist can correct any parts that are likely to require corrections from the animation director before proceeding to the animation director's confirmation. Therefore, according to the present disclosure, even if it is not possible to completely eliminate corrections from the animation director, it is expected that the number of reworks can be reduced by the original artist's self-confirmation using the animation director model. According to the present disclosure, communication between the animation worker and the animation director can be made more efficient, thereby shortening work time.

[0100] Keyframe UI: Fig. 11 shows an example of the configuration of a key animation work UI screen used by a key animator when creating and correcting key drawings. This key animation work UI screen is configured with consideration given to the application of an animation director model. Of course, the results of corrections made by an actual animation director can also be imported into this key animation work UI screen, and it can also be used to correct key drawings in response to the animation director's corrections. This key animation work UI screen is intended to be used by a key animator when creating and correcting key drawings in step S1001 of the flowchart shown in Fig. 10.

[0101] The area on the left side of the screen, designated by reference numeral 1101, is a drawing field where the user (original artist) can perform drawing work. The drawing field 1101 may have the same functions as general drawing software, but since this is not directly related to the present disclosure, a description of the drawing functions will be omitted.

[0102] At the bottom right of the screen is a "Request confirmation from animation director model" button, designated by the reference numeral 1102. After creating an original drawing in the animation field 1101, the user can press the "Request confirmation from animation director model" button 1102, whereupon the original drawing created in the animation field 1101 is input to the animation director model, and the user can obtain corrections and comments on the original drawing from the animation director model.

[0103] When the "Request confirmation from animation director model" button 1102 is pressed, the original drawings are input into the animation director model in the animation field 1101 along with supplemental information in the background, and corrections and comments on the original drawings output from the animation director model can be obtained. A "correction layer" is then added to the animation field 1101 to display the corrected original drawings output from the animation director model. A highlight indicating the corrections made by the animation director model is superimposed on the correction layer (not shown). Additionally, the "animation director's comments" field indicated by the reference numeral 1103 in the upper right corner of the screen displays comments obtained from the animation director model as text.

[0104] If the artist selects to make corrections to the master animation based on the corrections and comments output from the animation director model in step S1004 "artist self-confirmation" in the flowchart shown in FIG. 10, the process returns to step S1001, and the artist makes corrections and receives corrections and comments from the animation director model. Each time the user repeats corrections to the master animation and receives corrections and comments from the animation director model, correction layers displaying the corrected master animation are added sequentially, such as correction layer 1, correction layer 2, and so on. The bottom of the animation field 1101 is provided with layer tabs 1111, 1112, 1113, 1113, and so on, for selecting each layer. By selecting the desired layer tab, the artist can display the layer of the master animation to be corrected at the forefront of the animation field 1101 and make the correction. Alternatively, the selected layer may be displayed transparently. On the other hand, in the "animation director's comment" field 1103, comments for each correction layer, such as correction layer 1, correction layer 2, . . . are displayed in a timeline format.

[0105] Here, the configuration and functions of the key animation work UI screen have been described with consideration given to the application of an animation director model to the creation of key animation. However, the key animation work UI screen shown in FIG. 11 can also be used when the key animation model is not applied and correction instructions and comments are received solely from the actual animation director. The key animation work UI screen may also be configured to import corrections and comments from the actual animation director. For example, after correcting the key animation using the animation director model and completing the key animation artist's self-confirmation, pressing the "Request confirmation from animation director model" button 1102 causes the key animation drawn in the animation field 1101 to be transferred to the actual animation director's tablet device or PC. The animation director then makes corrections to the key animation director using the annotation UI (as described above), and the correction instructions and comments entered using the annotation UI are imported into the key animation work UI screen. A correction layer highlighting the corrections made by the animation director is added to the key animation work UI screen, and the animation director's comments are displayed in the "animation director's comments" field 1103.

[0106] In the example shown in Figure 11, correction layer 1 and correction layer 2 are corrections made by the animation director model, and the final correction layer 3 is assumed to be imported animation director corrections made by the animation director (animation director xx) after passing the key artist self-confirmation. In the "animation director comments" field 1103, in order to distinguish between comments generated by the animation director model and comments made by imported animation director corrections, the creator of the comment is clearly indicated for each comment, such as "(model output)" or "(animation director xx)."

[0107] As mentioned above, the key animator uses supplemental information such as the plot, script, screenplay, setting, scenario, storyboard, and layout of the animation work as context information related to the animation. The key animator work UI may further include a UI component (not shown in FIG. 11) for selecting supplemental information to be used in the animation. In this case, the animation director model inputs the key drawings and supplemental information drawn in the animation field 1101, and outputs corrections and comments on the key drawings. The fact that the animation director model inputs the key drawings and supplemental information and outputs corrections and comments on the key drawings has already been explained with reference to FIG. 9.

[0108] C. Variations In this section C, modified examples and application examples of the information processing system 100 are summarized.

[0109] (1) The information processing system 100 may acquire training data for the animation director model not only from corrections and comments exchanged between the original artist and the animation director, but also from chronological corrections completed by the original artist himself. This allows for the construction of an animation director model that can be used not only from the early stages of animation work but also to assist more beginners. For example, on the original animation work UI screen shown in FIG. 11, a correction layer displaying the original drawings after corrections completed by the original artist himself may be added to the animation field 1101, and the original artist's own correction comments may be added to the comment field 1103. In the "animation director's comment" field 1103, correction comments made by the original artist themselves are clearly indicated as the creator of the comment, such as "(original artist yy)."

[0110] (2) The above describes an embodiment of the present disclosure in which an "animation director model" is constructed and operated to intervene in the workflow of "creation" of animation key drawings by a key artist, "feedback" of corrections and comments by the animation director, and "correction" of the key drawings by the key artist. The present disclosure can be applied to various production processes by replacing the model depending on the target flow. For example, the present disclosure can be used to learn and utilize a model that can also be applied to 3DCG production.

[0111] (3) In animation production, the animation director is responsible for unifying the animation drawings. From this perspective, the animation director model constructed by this disclosure is primarily intended to point out overall consistency and the like in the interactions between the key artist and the animation director. On the other hand, because there are expressions that should not be drawn in animation, the animation director model also needs to point out from a different perspective. For example, if there are marks that should not be drawn from a compliance perspective, the animation director model can receive input from such perspective, check the key artwork, and check for the presence of marks that should not be drawn by the key artist and point them out to the key animator.

[0112] (4) In the above, an example of the present disclosure has been described in which an animation director model intervenes in the presentation of corrections and comments from the animation director to the key artist. Furthermore, in later stages of work, feedback can be provided in the form of multiple people discussing the finished video (such as a rush check). By intervening in such a process and accumulating comments from multiple people and using them for model learning, it is possible to provide an animation director model that provides correction feedback from a variety of perspectives.

[0113] (5) Many animations are based on manga or novels. When the animation director model outputs corrections and comments on the original drawings of such works, it can refer to the original information as well as information generated during the original production process, such as correction comments between the original author and the editor, to train the model.

[0114] D. Configuration of information processing device 12 shows an example of the hardware configuration of an information processing device 2000 that can operate as the information processing system 100 according to the present disclosure. The illustrated information processing device 2000 includes a CPU (Central Processing Unit) 2001, a ROM (Read Only Memory) 2002, a RAM (Random Access Memory) 2003, a host bus 2004, a bridge 2005, an expansion bus 2006, an interface unit 2007, an input unit 2008, an output unit 2009, a storage unit 2010, a drive 2011, and a communication unit 2013. The information processing device 2000 is configured, for example, by a personal computer, but some of its functions may be configured by an information terminal such as a tablet or a smartphone.

[0115] The CPU 2001 controls the overall operation of the information processing device 2000 in accordance with various programs. When performing computationally intensive processing such as learning an animation director model (described above) on the information processing device 2000, it is desirable that the CPU 2001 be a multi-core CPU (such as an Apple M1 Max) or that the information processing device 2000 further include a multi-core processor (such as an NVIDIA R6000) such as a GPU (Graphics Processing Unit) or GPGPU (General-purpose computing on graphics processing units) in addition to the CPU 2001. However, for convenience, these will hereinafter be collectively referred to simply as the CPU 2001.

[0116] ROM 2002 stores in a nonvolatile manner programs (such as a basic input / output system) and calculation parameters used by CPU 2001. RAM 2003 is used to load programs used in the execution of CPU 2001 and to temporarily store parameters such as working data that change as appropriate during program execution. Programs loaded into RAM 2003 and executed by CPU 2001 include, for example, various application programs and an operating system (OS).

[0117] The CPU 2001, ROM 2002, and RAM 2003 are interconnected by a host bus 2004, which includes a CPU bus and other components. The CPU 2001 executes various application programs in an execution environment provided by the OS through cooperative operation of the ROM 2002 and RAM 2003, thereby realizing a variety of functions and services. If the information processing device 2000 is a personal computer, the OS may be, for example, Microsoft Windows (registered trademark), Unix (registered trademark), or a successor OS. For example, the following application program is executed on the information processing device 2000:

[0118] Data collection application: An application program that collects training data for learning an animation director model. In addition to the original drawings before and after corrections by the animation director that are required for learning an animation director model, it also includes code that converts supplementary information from existing data and creates new training data for creating an animation director model. Model Training Application: Contains code that performs the animation director model training process based on the training data collected from the data collection application. Model utilization application: includes code that performs the process of performing animation work using the animation director model learned by the model learning application.

[0119] The host bus 2004 is connected to an expansion bus 2006 via a bridge 2005. The expansion bus 2006 is, for example, a PCI (Peripheral Component Interconnect) bus or PCI Express, and the bridge 2005 is based on the PCI standard. However, the information processing device 2000 does not need to be configured so that the circuit components are separated by the host bus 2004, bridge 2005, and expansion bus 2006, and may be implemented so that almost all circuit components are interconnected by a single bus (not shown).

[0120] The interface unit 2007 connects peripheral devices such as an input unit 2008, an output unit 2009, a storage unit 2010, a drive 2011, and a communication unit 2013 in accordance with the standards of the expansion bus 2006. However, not all of the peripheral devices shown in Fig. 12 are necessarily required, and the information processing device 2000 may further include peripheral devices not shown. Furthermore, the peripheral devices may be built into the main body of the information processing device 2000, or some of the peripheral devices may be externally connected to the main body of the information processing device 2000.

[0121] The input unit 2008 is composed of an input control circuit that generates an input signal based on an input from a user and outputs the signal to the CPU 2001. When the information processing device 2000 is a personal computer, the input unit 2008 may include a keyboard, a mouse, a touch panel, a camera, and a microphone. The output unit 2009 includes display devices such as a liquid crystal display (LCD) device, an organic EL (Electro-Luminescence) display device, and an LED (Light Emitting Diode), as well as an audio output device such as a speaker. The input unit 2008 is used to input a moving image to be processed, and the output unit 2009 is used to display a GUI screen, etc.

[0122] The storage unit 2010 stores files such as programs (applications, OS, etc.) and various data executed by the CPU 2001. The storage unit 2010 is configured with a large-capacity storage device such as an SSD (Solid State Drive) or HDD (Hard Disk Drive), but may also include an external storage device.

[0123] The removable storage medium 2012 is a storage medium configured as a cartridge, such as a microSD card. The drive 2011 performs read and write operations on the loaded removable storage medium 113. The drive 2011 outputs data read from the removable storage medium 2012 to the RAM 2003 or the storage unit 2010, and writes data on the RAM 2003 or the storage unit 2010 to the removable storage medium 2012.

[0124] The communication unit 2013 is a device that performs wireless communication such as Wi-Fi (registered trademark), Bluetooth (registered trademark), or a cellular communication network such as 4G or 5G. The communication unit 2013 may also include terminals such as a Universal Serial Bus (USB) and a High-Definition Multimedia Interface (HDMI) (registered trademark), and may further include a function for performing HDMI (registered trademark) communication with USB devices such as scanners and printers, displays, etc. Programs executed on the information processing device 2000 are installed from the outside via the communication unit 2013, for example. [Industrial Applicability]

[0125] The present disclosure has been described in detail above with reference to specific embodiments. However, the present disclosure should not be construed as being limited to the above-described embodiments, and it is obvious that those skilled in the art can modify or substitute the embodiments without departing from the spirit of the present disclosure. Furthermore, the effects described in this specification are merely examples, and the effects brought about by the present disclosure are not limited thereto, and additional effects not described in this specification may exist.

[0126] For the sake of simplicity, this specification describes an embodiment in which the present disclosure is applied to an animation production process that is based on a single-tier structure consisting of an animation director and an animation staff member. However, the present disclosure can also be applied to an animation production process that has a multi-tier structure in which there are multiple animation directors and a general animation director who oversees the animation directors.

[0127] Furthermore, although this specification has described the present disclosure primarily in terms of an embodiment for constructing and operating an animation director model applicable to key animation work, which is one of the drawing tasks in the animation production process, the gist of the present disclosure is not limited thereto. The present disclosure can also be applied to the construction and operation of learning models that can be applied to tasks other than key animation work in the animation production process to improve efficiency. Furthermore, the scope of application of the present disclosure is not limited to animation production, but can be extended to the production of various video content such as live-action videos (dramas, movies, plays, etc.), and even non-video content such as manga, novels, and music. Furthermore, the scope of application of the present disclosure is not limited to content production; the present disclosure can also be used to construct and operate learning models that recruit, train, and instruct workers in various industries, such as education, sports, crafts, manufacturing, and medicine.

[0128] In short, the present disclosure has been described in the form of examples, and the contents of the specification should not be interpreted as limiting. To determine the gist of the present disclosure, the claims should be taken into consideration.

[0129] The series of processes described in this specification can be executed by hardware, software, or a configuration that combines hardware and software. When executing processes by software, a program recording a processing sequence related to realizing the present disclosure is installed in memory in a computer incorporated in dedicated hardware and executed. It is also possible to install the program in a general-purpose computer capable of executing various processes and execute the processes related to realizing the present disclosure.

[0130] The program can be stored in advance on a recording medium installed in the computer, such as a HDD, SSD, or ROM. Alternatively, the program can be temporarily or permanently stored on a removable recording medium, such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto optical) disk, a DVD (Digital Versatile Disc), a BD (Blu-Ray Disc (registered trademark)), a magnetic disk, or a USB (Universal Serial Bus) memory. Using such a removable recording medium, a program related to realizing the present disclosure can be provided as so-called package software.

[0131] The program may also be transferred wirelessly or via a wired connection from a download site to a computer via a network such as a wide area network (WAN) (e.g., a cellular network), a local area network (LAN), or the Internet. The computer can receive the program transferred in this manner and install it on a large-capacity storage device such as a hard disk drive (HDD) or solid state drive (SSD) within the computer.

[0132] The present disclosure may also be configured as follows.

[0133] (1) an acquisition unit for acquiring original drawings related to animation production created by a first user; an output unit that inputs the original image into a machine learning model that has learned the relationship between modifications made to the original image by a second user different from the first user, and outputs the modifications made to the original image; An information processing system comprising:

[0134] (2) the acquisition unit further acquires supplementary information regarding the original image; the output unit inputs supplementary information together with the original image into the machine learning model and outputs corrections to the original image. The information processing system according to (1) above.

[0135] (3) The supplemental information includes at least one of plot information, script information, script information, setting information, scenario information, storyboard information, and layout information related to animation production. The information processing system according to (2) above.

[0136] (4) The output unit outputs, as the correction to the original image, at least one of a corrected original image obtained by correcting the original image and a comment regarding the correction to the original image. The information processing system according to any one of (1) to (3) above.

[0137] (5) a collection unit that collects training data for learning the machine learning model; a learning unit that learns the machine learning model using the training data collected by the collection unit, The information processing system according to any one of (1) to (4) above.

[0138] (6) A correction item input unit for the second user to input correction items to the original image is further provided, The collection unit collects training data based on information input by the second user to the correction input unit. The information processing system according to (5) above.

[0139] (7) The collection unit disassembles the original drawing into parts, presents a user interface to the second user for the second user to select a part that the second user wants to modify and to enter modifications to the selected part, and collects training data including the relationship of the modifications to each part with respect to the original drawing based on an input by the second user to the user interface. The information processing system according to (6) above.

[0140] (8) The user interface includes an original image display unit for displaying the original image before or after correction, a correction unit for selecting a part of the original image to be corrected and entering correction details for the selected part, and a confirmation unit for confirming the correction details entered in the correction unit. The information processing system according to (7) above.

[0141] (9) The collection unit disassembles the original image into parts based on an unmodified original image previously created by the first user, a modified original image modified by the first user based on modifications made to the original image by the second user, and supplemental information, and detects modifications made by the second user to each part, thereby collecting training data including the relationship between the modifications made to each part and the original image. The information processing system according to any one of (6) to (8) above.

[0142] (10) The acquisition unit acquires the original image via an original image working unit where the first user works on the original image. The information processing system according to any one of (1) to (9) above.

[0143] (11) The first user performs correction work on the original image based on the correction items for the original image via the original image work unit. The information processing system according to (10) above.

[0144] (12) The original image working unit presents the modifications made by the second user and the machine learning model in layers. The information processing system according to (11) above.

[0145] (13) The original image working unit includes a user interface including an original image display unit that displays the original image to be corrected for each layer, and a correction item presentation unit that presents correction items for the original image for each layer. The information processing system according to (12) above.

[0146] (14) an acquisition step of acquiring original drawings related to animation production created by the first user; an output step of inputting the original image into a machine learning model that has learned the relationship between modifications to the original image made by a second user different from the first user, and outputting the modifications to the original image; An information processing method comprising:

[0147] (15) an acquisition unit for acquiring original drawings related to animation production created by the first user; an output unit that inputs the original image into a machine learning model that has learned the relationship between modifications made to the original image by a second user different from the first user, and outputs the modifications made to the original image; A computer program written in a computer-readable format to cause a computer to function as a [Explanation of symbols]

[0148] 100...information processing system, 101...data collection unit 102...Model Learning Department, 103...Model Utilization Department 202...parts disassembly section, 202...annotation section 203... Corrected Database 301...input unit, 302...parts disassembly unit, 303...comment detection unit 304...Parts and comments correspondence section, 305...Data storage section 306...Past works database 901...Image encoder, 902...Text encoder 903...image decoder, 904...text decoder 905...Works database 2000...information processing device, 2001...CPU, 2002...ROM 2003...RAM, 2004...Host Bus, 2005...Bridge 2006... Expansion bus, 2007... Interface section 2008...input section, 2009...output section, 2010...storage section 2011...Drive, 2012...Removable Recording Media 2013…Communications Department

Claims

1. an acquisition unit that acquires original drawings related to animation production created by the first user; an output unit that inputs the original image into a machine learning model that has learned the relationship between modifications made to the original image by a second user different from the first user, and outputs the modifications made to the original image; An information processing system comprising:

2. The acquisition unit further acquires supplemental information related to the original image, the output unit inputs supplementary information together with the original image into the machine learning model and outputs corrections to the original image. The information processing system according to claim 1 .

3. The supplemental information includes at least one of plot information, script information, screenplay information, setting information, scenario information, storyboard information, and layout information related to animation production. The information processing system according to claim 2 .

4. the output unit outputs, as the correction to the original image, at least one of a corrected original image obtained by correcting the original image and a comment regarding the correction to the original image. The information processing system according to claim 1 .

5. a collection unit that collects training data for learning the machine learning model; a learning unit that learns the machine learning model using the training data collected by the collection unit, The information processing system according to claim 1 .

6. a correction input unit for the second user to input corrections to the original image; The collection unit collects training data based on information input by the second user to the correction input unit. The information processing system according to claim 5 .

7. the collection unit disassembles the original drawing into parts, presents a user interface to the second user for the second user to select a part that the second user wishes to modify and to enter modifications to the selected part, and collects training data including a relationship between the modifications to each part and the original drawing based on an input by the second user to the user interface. The information processing system according to claim 6.

8. The user interface includes an original image display unit for displaying the original image before or after modification, a modification unit for selecting a part of the original image to be modified and entering modifications for the selected part, and a confirmation unit for confirming the modifications entered in the modification unit. The information processing system according to claim 7 .

9. the collection unit disassembles the original image into parts based on an unmodified original image previously created by the first user, a modified original image modified by the first user based on modifications made to the original image by the second user, and supplemental information, detects modifications made to each part by the second user, and collects training data including a relationship between the modifications made to each part and the original image. The information processing system according to claim 6.

10. the acquiring unit acquires the original image via an original image working unit in which the first user works on the original image; The information processing system according to claim 1 .

11. the first user performs correction work on the original drawing based on correction items for the original drawing via the original drawing work unit; The information processing system according to claim 10.

12. The original image working unit presents the modifications made by the second user and the machine learning model in layers. The information processing system according to claim 11.

13. the original image working unit is provided with a user interface including an original image display unit that displays an original image to be corrected for each layer, and a correction item presentation unit that presents correction items for the original image for each layer; The information processing system according to claim 12.

14. an acquisition step of acquiring original drawings related to animation production created by the first user; an output step of inputting the original image into a machine learning model that has learned the relationship between modifications made to the original image by a second user different from the first user, and outputting the modifications made to the original image; An information processing method comprising:

15. an acquisition unit for acquiring original drawings related to animation production created by the first user; an output unit that inputs the original image into a machine learning model that has learned the relationship between modifications made to the original image by a second user different from the first user, and outputs the modifications made to the original image; A computer program written in a computer-readable format to cause a computer to function as a

Images