System

Abstract

A system is provided.SOLUTION: This system is provided with a means for inputting the outline of the cartoon by a user, a means for generating detailed story development based on the outline for which a generation AI is inputted, a means for automatically generating the layout of respective pages and frames based on the generated story development, a means for automatically generating the pictures of the respective frames based on the generated layout and a means for constituting the final cartoon and providing it to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, advances in generative AI technology have made it easier to generate text and images. However, creating manga remains a time-consuming and labor-intensive task, placing a significant burden on manga artists. Specifically, it requires time-consuming tasks such as coming up with a storyline, creating a panel layout, and drawing the pictures for each panel. Therefore, there is a need for a method to automate these tasks, thereby reducing the burden on manga artists and creating manga more efficiently. [Means for solving the problem]

[0005] The present invention provides a system in which a user inputs a rough outline of a manga, and a generation AI automatically generates a detailed storyline, and automatically generates the layout of each page and frame based on that story. The AI ​​then automatically generates pictures for each frame based on the generated layout, and the final manga is constructed and provided to the user. This system allows the user to create a high-quality manga in a short amount of time without performing any specific work. Specifically, the system includes the following means:

[0006] 1. A way for users to input a summary of a manga

[0007] 2. A means for generative AI to generate detailed storylines based on input synopsis

[0008] 3. A method for automatically generating layouts for each page and panel based on the generated storyline

[0009] 4. A method for automatically generating pictures for each frame based on the generated layout

[0010] 5. How to compose the final manga and provide it to users

[0011] As a result, it is possible to significantly reduce the time and effort required to create manga, and to efficiently create high-quality manga.

[0012] "User" refers to a person or entity who accesses the system and inputs a comic book synopsis.

[0013] "Generative AI" refers to artificial intelligence that automatically generates detailed story development, layout, and illustrations based on an outline entered by the user.

[0014] "Manga outline" refers to input data that includes basic information about the manga, such as the overall story setting, main characters, and important events.

[0015] A "detailed storyline" refers to a sequence of specific scenes and events generated based on the outline.

[0016] "Layout" refers to the information that defines the placement, size, and relative positions of each page and panel in a manga.

[0017] "Panel art" refers to the visual elements of each panel, such as the character's facial expression, pose, and background.

[0018] The "final manga" refers to the finished manga that combines the detailed storyline, layout, and panel art created by the generative AI.

[0019] "Means of providing" refers to the process of providing the final manga to users in a downloadable format or manner. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9]1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] System Overview

[0042] This invention is a system in which a generation AI automatically generates a detailed storyline, page and frame layouts, and the pictures for each frame based on a manga outline entered by a user, and then constructs and provides the final manga to the user. The system is configured around a server, receives user input, uses the generation AI to perform various processes, and provides the final product to the user.

[0043] Program Overview

[0044] The program of this system executes a series of processes in a specific order. Below we will explain the process in brief, with specific examples.

[0045] 1. Enter the summary

[0046] A user accesses the system and inputs a rough outline of the manga. The input information is received and stored by the server.

[0047] Example: A high school student protagonist is reincarnated into another world and becomes active at a magic academy.

[0048] 2. Storyline generation

[0049] The server passes the user-entered synopsis to a generation AI, which then generates a detailed storyline based on the synopsis and builds out the specific events that occur in each scene.

[0050] Examples:

[0051] Scene 1: The scene where the protagonist is reincarnated into another world

[0052] Scene 2: First lesson at the Magic Academy

[0053] Scene 3: Confronting a Sassy Rival

[0054] 3. Generate layout

[0055] The server automatically generates the layout of each page and frame based on the generated storyline, using a generation AI to determine the frame layout, size, and placement for each scene.

[0056] Examples:

[0057] Scene 1: 4 frames

[0058] Scene 2: 6 frames

[0059] Scene 3: Consists of 3 frames

[0060] 4. Image Generation

[0061] The server automatically generates the images for each frame based on the generated layout information. The AI ​​then draws the character's facial expressions, poses, background, and other elements.

[0062] Examples:

[0063] Scene 1, Frame 1: The protagonist is thrown into another world with a surprised expression.

[0064] Scene 2, Frame 2: The vast landscape of the magic academy

[0065] Scene 3, Frame 3: Dialogue with a new friend

[0066] 5. Creation and Delivery of Deliverables

[0067] The server combines the illustrations and story of each scene to create the final manga, which it then checks for errors and makes available to users in a downloadable format.

[0068] Examples:

[0069] Deliver the final manga in PDF format or other suitable format

[0070] Users can download, view, and print

[0071] Specific examples of work processes

[0072] Below is a concrete example of generating a manga in which the protagonist, a high school student, is reincarnated into another world and becomes active at a magic academy.

[0073] 1. The user enters the synopsis: "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy."

[0074] 2. The server receives the summary and checks for errors.

[0075] 3. The server passes the outline to the generation AI, which generates the following detailed storyline:

[0076] The scene where the main character is reincarnated into another world

[0077] Meeting new friends at the magic academy

[0078] Magical training and rivalry

[0079] 4. The server analyzes each scene and determines the frame layout (scene 1 is 4 frames, scene 2 is 6 frames, scene 3 is 3 frames).

[0080] 5. The server uses AI to generate the images for each frame (the surprised main character, the magic effects, the school background).

[0081] 6. The server combines the pictures and story, generates the final manga, and performs error checking.

[0082] 7. The server provides the final manga to the user in PDF format, which the user can download and view.

[0083] In this way, the system of the present invention significantly reduces the effort required to create manga and efficiently provides high-quality manga.

[0084] The processing flow will be explained below.

[0085] Step 1:

[0086] A user accesses the system and enters a rough outline of a manga. The user enters a text summary of the story, character settings, and major events into the input fields of a web form or application, and then presses the submit button.

[0087] Step 2:

[0088] The server receives the summary sent by the user and saves the text data. It performs an initial error check on the saved text data to ensure that the correct format and content are entered.

[0089] Step 3:

[0090] The server analyzes the summary text and passes it to the generation AI, which then generates a detailed storyline based on the summary, including specific events and character actions in each scene.

[0091] Step 4:

[0092] The server automatically generates the layout of each page and frame based on the generated storyline. The AI ​​determines the frame layout, size, and placement for each scene.

[0093] Step 5:

[0094] The server automatically generates the images for each frame based on the generated layout information, using a generative AI to depict visual elements such as character expressions, poses, and backgrounds.

[0095] Step 6:

[0096] The server combines the illustrations of each panel with the storyline to create the final manga. It then performs error checks to ensure that each page is generated correctly and that there are no problems with the images or text.

[0097] Step 7:

[0098] The server converts the final comic into a downloadable format (e.g. PDF) and provides it to the user, who clicks on the download link to retrieve the generated comic.

[0099] Step 8:

[0100] The user can view the downloaded manga and, if necessary, make edits or print it.

[0101] Example 1

[0102] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0103] Traditional manga production requires a great deal of time and effort, and many manual processes result in low productivity. In particular, there is a lot of creative work involved, such as developing the storyline, setting the layout of pages and panels, and creating the illustrations, which can lead to variations in quality. There is a need for a system that can solve these problems and automatically generate high-quality manga efficiently.

[0104] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0105] In this invention, the server includes: means for a user to input a summary of a manga; means for a generative AI model to generate a detailed storyline based on the input summary; means for automatically generating a layout for each page and frame based on the generated storyline; means for automatically generating pictures for each frame based on the generated layout; means for combining the pictures for each frame and the storyline to create a final manga and perform error checking; and means for providing the final manga to the user. This makes it possible to automatically generate and quickly provide a high-quality manga efficiently based on the summary input by the user.

[0106] "User" refers to a person who uses the system to input a manga outline and receive the final manga.

[0107] A "manga summary" refers to information that indicates the general content and structure of the story.

[0108] A "generative AI model" refers to an artificial intelligence model that generates storylines and images based on input information.

[0109] A "detailed storyline" refers to a progression of the story that includes specific scenes and events generated based on the outline.

[0110] "Page and panel layout" refers to the composition information such as the panel layout, size, and placement for each page of a manga.

[0111] "Artwork in each panel" refers to the characters, background, and other visual elements that appear in each panel of a manga.

[0112] "Error checking" refers to the process of checking the consistency and quality of the generated manga and detecting errors.

[0113] "PDF format" is an abbreviation for Portable Document Format and refers to one of the formats for electronic documents.

[0114] "Means for providing to the user" refers to a method for providing the final generated manga in a form that is accessible to the user.

[0115] System Overview

[0116] This invention is a system in which a generative AI model automatically generates a detailed storyline, page and frame layouts, and the pictures for each frame based on a manga outline entered by a user, composing the final manga and providing it to the user. The system is centered around a server, which receives user input, uses the generative AI model to perform various processes, and provides the final product to the user.

[0117] Program Overview

[0118] The system executes a series of steps in a specific order to generate a manga based on a user-supplied synopsis, as detailed below.

[0119] Enter a summary

[0120] A user accesses the system through a terminal and inputs a rough outline of the manga. The terminal sends the input information to the server, which then stores it in a database.

[0121] Examples:

[0122] The user enters a summary such as "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy."

[0123] Storyline generation

[0124] The server then passes the saved summary data to a generative AI model, which uses this summary to generate a detailed storyline and build specific events for each scene. This model uses OpenAI's GPT-4 and other AI models.

[0125] Examples:

[0126] Scene 1: The scene where the protagonist is reincarnated into another world

[0127] Scene 2: First lesson at the Magic Academy

[0128] Scene 3: A confrontation with a nasty rival

[0129] Generate layout

[0130] The server automatically generates the layout of each page and frame based on the generated storyline. It uses a generative AI model to determine the frame layout, size, and placement for each scene. OpenAI's DALL-E and other tools are used here.

[0131] Examples:

[0132] Scene 1: 4 frames

[0133] Scene 2: 6 frames

[0134] Scene 3: Consists of 3 frames

[0135] Picture generation

[0136] The server automatically generates the images for each frame based on the generated layout information. It uses a generative AI model to draw the character's facial expressions, poses, background, etc. DALL-E and Stable Diffusion are used here.

[0137] Examples:

[0138] Scene 1, Frame 1: The protagonist is thrown into another world with a surprised expression.

[0139] Scene 2, Frame 2: The vast landscape of the magic academy

[0140] Scene 3, Frame 3: Dialogue with a new friend

[0141] Generate and deliver deliverables

[0142] The server then combines the illustrations and story of each scene to create the final manga, performs error checking, and delivers it to the user in a format that could be PDF or EPUB.

[0143] Examples:

[0144] The final manga will be provided to users in PDF format, which they can download, view, and print.

[0145] Example prompt

[0146] Below is a specific example of a prompt sentence to be passed to the generative AI model.

[0147] Storyline generation prompt:

[0148] "Generate a detailed storyline based on the outline of a high school student protagonist who is reincarnated into another world and becomes active at a magic academy."

[0149] Layout generation prompt:

[0150] "Generate a page and frame layout where Scene 1 is 4 frames, Scene 2 is 6 frames, and Scene 3 is 3 frames."

[0151] Picture generation prompt:

[0152] "In Scene 1, Frame 1, please draw the main character with a surprised expression as he is thrown into another world."

[0153] The system of the present invention efficiently and automatically generates manga from an outline entered by a user based on the above procedures and elements, enabling high-quality and rapid provision.

[0154] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0155] System program processing flow

[0156] Step 1: Enter a summary

[0157] The user accesses the system through a terminal and inputs a story outline, including themes and major events of the manga.

[0158] Input: Manga summary (text)

[0159] Output: Summary data (sent to server)

[0160] Specific operation: The user accesses the input form in a web browser and enters a summary such as "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy." The device detects a click on the "Submit" button and sends the input data to the server as a POST request. The server receives the POST request and saves the entered summary data in a database.

[0161] Step 2: Analyze the summary and check for errors

[0162] The server parses the received summary data and performs some initial error checking to ensure there are no typos or missing information.

[0163] Input: Summary data (fetched from database)

[0164] Output: Verified summary data

[0165] What it does: The server fetches summary data from the database and uses a text analysis module to detect grammatical errors and missing information. If errors are found, it sends feedback to the user to prompt them to make corrections.

[0166] Step 3: Generate a storyline

[0167] The server passes the error-free summary data to a generative AI model (e.g., GPT-4), which generates a detailed storyline based on the prompt.

[0168] Input: Validated summary data

[0169] Output: Detailed storyline

[0170] Specific operation: The server generates a prompt and sends an API request to the generative AI model saying, "Please generate a detailed storyline based on the outline that the high school protagonist is reincarnated in another world and becomes active at a magic academy." The generative AI model generates a detailed storyline and returns it to the server. The server temporarily stores the received storyline data.

[0171] Step 4: Generate the layout

[0172] The server automatically generates the layout of each page and frame based on the generated storyline. It uses a generative AI model (e.g., DALL-E) to determine the frame layout, size, and placement for each scene.

[0173] Input: Detailed storyline

[0174] Output: Page and frame layout

[0175] Specific operation: The server analyzes the story development data and sends an API request to the generation AI model with the prompt "Please generate a page and frame layout where Scene 1 is 4 frames, Scene 2 is 6 frames, and Scene 3 is 3 frames." The generation AI model generates the page and frame layout and returns it to the server. The server temporarily stores the generated layout information.

[0176] Step 5: Generate the image

[0177] The server automatically generates the images for each frame based on the generated layout information, using a generative AI model (such as DALL-E or Stable Diffusion) to draw the character's facial expressions, poses, background, and other details.

[0178] Input: Page and frame layout

[0179] Output: Picture of each frame

[0180] Specific operation: The server fetches layout information, generates a prompt for each frame: "In Scene 1, Frame 1, please draw the main character with a surprised expression as he is thrown into another world," and sends an API request to the generative AI model. The generative AI model generates an image for each frame and returns it to the server. The server temporarily stores the received image data.

[0181] Step 6: Generate and deliver artifacts

[0182] The server combines the illustrations of each scene with the story to create the final manga, which is then verified and checked for errors before being delivered to the user.

[0183] Input: Pictures of each frame and detailed storyline

[0184] Output: Finished manga (PDF format, etc.)

[0185] Specific operation: The server integrates the saved storyline data and illustration data, performs error checks and quality inspections (checking page layout and illustration consistency), converts the final manga into PDF or other formats, and provides a download link to the user. The user clicks the download link on their device to obtain the work.

[0186] In this way, this system uses a series of automated processes to efficiently generate and quickly deliver high-quality manga based on the outline entered by the user.

[0187] (Application example 1)

[0188] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0189] Traditional manga production requires a lot of time and effort, and there is no way to automate the entire process, from story development to layout and illustration creation. This makes it difficult for individual creators or small teams to quickly produce high-quality manga. There is also a lack of an effective platform for sharing and evaluating the manga. Furthermore, there are challenges with error checking and providing the manga in digital format.

[0190] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0191] In this invention, the server includes: a means for a user to input a summary of a manga; a means for a generation AI to generate a detailed storyline based on the input summary; a means for automatically generating a layout for each page and frame based on the generated storyline; a means for automatically generating pictures for each frame based on the generated layout; a means for composing the final manga and providing it to the user; a means for uploading the generated manga to a content platform and sharing it with other users; and a means for other users to rate and comment on the generated manga. This allows users to easily generate high-quality manga, share it with other users, and receive ratings. Error checking and provision in electronic format can also be efficiently performed.

[0192] To match the application example, we will create definitions of the important words included in the patent claims in the following format.

[0193] A "user" is an individual or entity that uses the system to input a comic outline and receive the generated comic.

[0194] "Generative AI" is an artificial intelligence technology that automatically generates detailed story development, page and panel layouts, and pictures for each panel from an input outline.

[0195] A "storyline" is a sequence of specific events and scenes that the generative AI generates based on a summary entered by the user.

[0196] "Layout" refers to the structure including the placement and size of each page and frame, and the relationships between frames.

[0197] "Pictures" refer to visual elements such as the character's facial expression, pose, and background drawn in each panel.

[0198] The "content platform" is an online system that allows users to share their manga with other users and receive ratings.

[0199] "Sharing" is the act of sharing the created manga with other users.

[0200] "Evaluation" is the act of other users providing feedback on the generated comic.

[0201] "Error checking" is the process of checking the user-entered synopsis and generated content for defects and errors.

[0202] "Electronic format" refers to the method of providing the final manga in a digital data format such as PDF.

[0203] MODE FOR CARRYING OUT THE INVENTION

[0204] This invention relates to a system in which a generation AI automatically generates a detailed storyline, page and frame layouts, and frame art based on a manga outline entered by a user, and then constructs the final manga and provides it to the user. Specific embodiments for carrying out this invention are described below.

[0205] System configuration

[0206] The system of this invention consists of a terminal used by a user, a server that runs a generative AI model, and a content platform.

[0207] Device: A mobile device such as a smartphone or tablet is used. Users input the outline of the manga through this device.

[0208] Server: A generative AI model is installed on a cloud server that analyzes the outline, generates detailed story development, determines the layout, and generates the images for each frame.

[0209] Content Platform: The manga will be uploaded to a content platform where other users can view and rate it.

[0210] Program processing

[0211] The server proceeds with the following steps:

[0212] 1. Enter summary and check for errors:

[0213] The user inputs a summary of the manga using a terminal, and the server receives this summary and performs some initial error checking.

[0214] Example: The prompt is "The high school student protagonist is reincarnated into another world and becomes active at a magic academy."

[0215] 2. Storyline generation:

[0216] The server passes the received synopsis to a generative AI model to generate a detailed storyline.

[0217] Examples:

[0218] Scene 1: The scene where the protagonist is reincarnated into another world

[0219] Scene 2: First lesson at the Magic Academy

[0220] Scene 3: Confronting a Sassy Rival

[0221] 3. Generate layout:

[0222] The layout of each page and panel is automatically generated based on the story development. Panel layout, size, and placement are determined.

[0223] Examples:

[0224] Scene 1: 4 frames

[0225] Scene 2: 6 frames

[0226] Scene 3: Consists of 3 frames

[0227] 4. Picture generation:

[0228] Based on the generated layout, the images for each frame are generated, including the character's facial expressions, poses, background, etc.

[0229] Examples:

[0230] Scene 1, Frame 1: The protagonist is thrown into another world with a surprised expression.

[0231] Scene 2, Frame 2: The vast landscape of the magic academy

[0232] Scene 3, Frame 3: Dialogue with a new friend

[0233] 5. Composition and presentation of the manga:

[0234] The illustrations for each scene are combined with the story to create the final manga, which is then checked for errors. The finished manga is then provided in electronic format.

[0235] Examples:

[0236] The final manga is provided to users in PDF format, which they can download and view.

[0237] Prototyping:

[0238] This system allows users to easily generate high-quality manga, share them with other users, and receive feedback. The server utilizes a generative AI model to efficiently analyze, generate, and organize data. Specifically, OpenAI's generative AI model and cloud server technology are used to achieve a highly accurate generation process in real time.

[0239] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0240] Program processing steps

[0241] Step 1:

[0242] A user inputs a summary of a manga using a terminal. The input summary (prompt sentence) is received by the server. The server performs an initial error check on this summary.

[0243] Input: The prompt entered by the user (e.g., "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy.")

[0244] Data processing: Check for errors to ensure the summary is entered correctly.

[0245] Output: Prompt statement that passes error checking

[0246] Step 2:

[0247] The server passes the prompt sentences that pass error checks to a generative AI model, which then generates a detailed storyline.

[0248] Input: A prompt that passes error checking

[0249] Data computation: Generative AI models are used to generate detailed storylines (determine the specific events for each scene).

[0250] Output: A detailed storyline generated

[0251] Step 3:

[0252] The server automatically generates the layout of each page and frame based on the generated story development.

[0253] Input: A detailed generated storyline

[0254] Data calculation: An algorithm is used to automatically generate the layout (panel division, size, and placement) of each page and panel.

[0255] Output: The layout of each generated page and frame

[0256] Step 4:

[0257] The server automatically generates pictures for each frame based on the generated layout information.

[0258] Input: The layout of each generated page and frame

[0259] Data calculation: Uses generative AI models to draw character expressions, poses, backgrounds, etc.

[0260] Output: Picture of each frame

[0261] Step 5:

[0262] The server combines the pictures of each scene with the story to create the final manga and performs error checks.

[0263] Input: Pictures of each frame and detailed storyline

[0264] Data processing: Combining the data and checking for errors in the final manga.

[0265] Output: Final cartoon

[0266] Step 6:

[0267] The server uploads the completed manga to a content platform where other users can view, rate, and comment on it.

[0268] Input: Final cartoon

[0269] Data Calculation: Processing the upload and applying publishing settings on the content platform.

[0270] Output: Manga published on a content platform

[0271] Step 7:

[0272] Other users view the comics created on the content platform and provide ratings and comments.

[0273] Input: Manga published on content platforms

[0274] Data calculation: Performs the process of accepting ratings and comments from other users.

[0275] Output: Manga with ratings and comments

[0276] In this way, a system has been realized in which users, servers, and content platforms work together to efficiently generate, share, and evaluate high-quality manga.

[0277] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0278] System Overview

[0279] This invention is a system in which a generation AI automatically generates a detailed storyline, page and panel layout, and illustrations for each panel based on a manga outline entered by the user, composing the final manga and providing it to the user. Furthermore, by combining it with an emotion engine, the system is characterized by recognizing the user's emotions and adjusting the content of the story and illustrations accordingly. The system is centered around a server, which receives user input, utilizes the generation AI and emotion engine to perform various processes, and provides the final product to the user.

[0280] Program Overview

[0281] The program of this system executes a series of processes in a specific order. Below we will explain the process in brief, with specific examples.

[0282] 1. Summary input and emotion recognition

[0283] A user accesses the system and inputs a rough outline of a manga. The user enters a summary of the story, character settings, and major events in text format into the input fields of a web form or application, and then presses the submit button. At the same time, the emotion engine analyzes the user's emotions as they type.

[0284] Example: When the user inputs a summary of the story in which the protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy, the emotion engine recognizes that the user is smiling happily.

[0285] 2. Receiving and analyzing the summary

[0286] The server receives the summary sent by the user and saves the text data. It performs an initial error check on the saved text data to ensure that the format and content are correct. At the same time, it saves the emotion data analyzed by the emotion engine.

[0287] 3. Storyline generation

[0288] The server analyzes the summary text and emotion data and passes it to the generation AI, which then generates a detailed storyline based on the summary and the user's emotions. This storyline includes specific events and character actions in each scene, and the content is adjusted based on the emotion data.

[0289] Examples:

[0290] Scene 1: The scene where the protagonist gets excited and is reincarnated into another world.

[0291] Scene 2: The exciting first lesson at the Magic Academy

[0292] Scene 3: Meeting a fun-loving friend

[0293] 4. Generate layout

[0294] The server automatically generates the layout of each page and frame based on the generated storyline. The AI ​​determines the frame layout, size, and placement for each scene.

[0295] Examples:

[0296] Scene 1: Consists of four frames (each expressing excitement)

[0297] Scene 2: 6 frames (emphasizes the fun of class)

[0298] Scene 3: Consists of 3 frames (depicting a friendly conversation with a friend)

[0299] 5. Image Generation

[0300] The server automatically generates the images for each frame based on the generated layout information, using a generative AI to depict visual elements such as character expressions, poses, and backgrounds based on emotional data.

[0301] Examples:

[0302] Scene 1, Frame 1: The protagonist is thrown into another world with a smile on his face.

[0303] Scene 2, Frame 2: A vast and fun view of the magic academy

[0304] Scene 3, Frame 3: The protagonist chats happily with a new friend

[0305] 6. Creation and Delivery of Deliverables

[0306] The server combines the illustrations and story of each scene to create the final manga. It then performs error checks to ensure that each page is generated correctly and that there are no problems with the images or text.

[0307] 7. Providing Results

[0308] The server converts the final comic into a downloadable format (e.g. PDF) and provides it to the user, who clicks on the download link to retrieve the generated comic.

[0309] 8. Emotional feedback during execution

[0310] Even while a user is browsing a manga, the emotion engine can analyze the user's emotions in real time, and the generation AI can fine-tune the content based on the emotions.

[0311] Specific examples of work processes

[0312] Below is a concrete example of generating a manga in which the protagonist, a high school student, is reincarnated into another world and becomes active at a magic academy.

[0313] 1. The user enters and submits a summary: "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy." At the same time, the emotion engine recognizes a happy, smiling face.

[0314] 2. The server receives the summary and checks for errors.

[0315] 3. The server passes the synopsis to the generation AI and emotion engine, which generates the following detailed storyline:

[0316] A scene where the main character gets excited and is reincarnated into another world

[0317] Meet exciting new friends at the Magic Academy

[0318] 4. The server analyzes each scene and determines the frame layout (scene 1 is 4 frames, scene 2 is 6 frames, scene 3 is 3 frames).

[0319] 5. The server uses AI to generate pictures for each frame (a smiling protagonist and scenes of fun school life).

[0320] 6. The server combines the pictures and story, generates the final manga, and performs error checking.

[0321] 7. The server provides the final manga to the user in PDF format, which the user can download and view.

[0322] 8. Users can view the manga and the emotion engine can provide real-time feedback and regenerate it as needed.

[0323] In this way, the system of the present invention, which is combined with an emotion engine, can efficiently provide high-quality manga that reflects the user's emotions.

[0324] The processing flow will be explained below.

[0325] Step 1:

[0326] A user accesses the system and inputs a rough outline of a manga. The user enters a summary of the story, character settings, and major events in text format into the input fields of a web form or application, and then presses the submit button. At the same time, the emotion engine analyzes the user's facial expressions and voice to recognize the emotion they are expressing as they type.

[0327] Step 2:

[0328] The server receives the summary sent by the user and saves the text data. At the same time, the emotion data analyzed by the emotion engine is also sent to the server and saved. An initial error check is performed on the saved text data to ensure that the correct format and content have been entered.

[0329] Step 3:

[0330] The server analyzes the summary text and emotion data and passes it to the generation AI, which then generates a detailed storyline based on the summary and the user's emotions. This storyline includes specific events and character actions in each scene, and the content is adjusted based on the emotion data.

[0331] Step 4:

[0332] The server automatically generates the layout of each page and frame based on the generated storyline. The AI ​​determines the frame layout, size, and placement for each scene.

[0333] Step 5:

[0334] The server automatically generates the images for each frame based on the generated layout information, using a generative AI to depict visual elements such as character expressions, poses, and backgrounds based on emotional data.

[0335] Step 6:

[0336] The server combines the generated pictures of each panel with the storyline to create the final manga. It then performs error checks to ensure that each page is generated correctly and that there are no problems with the images or text.

[0337] Step 7:

[0338] The server converts the final comic into a downloadable format (e.g. PDF) and provides it to the user, who clicks on the download link to retrieve the generated comic.

[0339] Step 8:

[0340] As users browse the manga, the emotion engine analyzes their emotions in real time. Depending on their emotions, the generative AI readjusts the content and updates the storyline and frame art until the user is satisfied.

[0341] Example 2

[0342] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0343] Modern manga production requires a lot of time and effort, creating an efficient automatic generation system. However, existing automatic generation systems only generate stories and layouts based on user input, and are unable to adjust content to reflect the user's emotions. Therefore, a system that can recognize the user's emotions and adjust content based on them is needed. Furthermore, they lack error checking capabilities at each stage of story generation, panel layout, image generation, and delivery of the final product, which can result in incomplete manga.

[0344] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0345] In this invention, the server includes: a means for a user to input a summary of a manga; an emotion recognition means for analyzing the user's emotions at the time of input; a generation AI for generating a detailed storyline based on the input summary and emotion data; a means for automatically generating a layout for each page and frame based on the generated storyline; a means for automatically generating pictures for each frame based on the generated layout; and a means for constructing and providing the final manga to the user. This allows for efficient automatic generation of high-quality manga that reflects the user's emotions and allows for fine-tuning of the content in real time in response to the user's emotions. Furthermore, error checking at each step allows for a highly refined final product.

[0346] "User" refers to a person who accesses the system and inputs a summary of a manga.

[0347] "Emotion recognition means" refers to technology that analyzes the user's emotions while the user is entering a summary.

[0348] "Generative AI" refers to artificial intelligence that generates detailed storylines based on a summary and emotional data entered by the user.

[0349] "Story development" refers to the detailed narrative progression generated by the generative AI, including specific events and character actions in each scene.

[0350] "Layout" refers to the design that determines the placement and size of each page and panel based on the generated story development.

[0351] "Automatic picture generation means" refers to a technology that automatically generates pictures for each frame based on the generated layout information.

[0352] "Error checking" refers to the process of checking whether generated data or final products are fraudulent or incorrect.

[0353] "Final product" refers to the finished manga that is created by integrating the generated story, layout, and illustrations.

[0354] "Delivery means" refers to the technology by which the final product is delivered to users in a downloadable format.

[0355] "Real-time feedback" refers to the process in which the emotion engine analyzes the user's emotions in real time while they are reading the manga, and the generative AI fine-tunes the content.

[0356] System Overview

[0357] This invention is a system in which a generation AI automatically generates a detailed storyline, page and panel layout, and the illustrations for each panel based on a manga outline entered by the user, and provides the final manga to the user. This system is characterized by recognizing the user's emotions by combining it with an emotion engine, and adjusting the content of the story and illustrations accordingly. The system is configured around a server, which receives user input, uses the generation AI and emotion engine to perform various processes, and provides the final product to the user.

[0358] Hardware and software used

[0359] This system uses the following hardware and software:

[0360] Server: The central location for receiving, storing, analyzing, and generating data. This is where the emotion engine and generative AI models run.

[0361] Generative AI model: Generates detailed storylines and frame illustrations based on user input and emotional data.

[0362] Emotion Engine: Recognizes user emotions in real time and provides emotional data.

[0363] User interface: A web form or application for users to enter a synopsis of their manga.

[0364] Program processing

[0365] 1. Summary input and emotion recognition

[0366] Users enter a summary of their manga through a web form or application, and the emotion engine analyzes their facial expressions and tone to capture emotional data.

[0367] Example: A user enters the summary "The protagonist is a high school student who is reincarnated in another world and becomes active at a magic academy," and the emotion engine recognizes that the user is smiling happily.

[0368] 2. Receiving and analyzing the summary

[0369] The server receives the summary sent by the user and saves the text data. It performs an initial error check on the saved text data to ensure that it is in the correct format and content. At the same time, it saves the emotion data obtained from the emotion engine.

[0370] 3. Storyline generation

[0371] The server analyzes the summary text and emotional data and passes it to the generation AI, which then generates a detailed storyline based on the summary and emotional data, including specific events and character actions in each scene, and adjusts the content based on the emotional data.

[0372] Examples:

[0373] Scene 1: The scene where the protagonist gets excited and is reincarnated into another world.

[0374] Scene 2: The exciting first lesson at the magic academy

[0375] Scene 3: Meeting a fun-loving friend

[0376] 4. Generate layout

[0377] The server automatically generates the layout of each page and frame based on the storyline. The AI ​​determines the frame layout, size, and placement for each scene.

[0378] Examples:

[0379] Scene 1: 4-frame composition (expresses excitement)

[0380] Scene 2: 6 frames (emphasizes the fun of class)

[0381] Scene 3: Consists of 3 frames (depicting a friendly conversation with a friend)

[0382] 5. Image Generation

[0383] The server automatically generates the images for each frame based on the generated layout information. The AI ​​then depicts visual elements such as character expressions, poses, and backgrounds based on emotional data.

[0384] Examples:

[0385] Scene 1, Frame 1: The protagonist is thrown into another world with a smile on his face.

[0386] Scene 2, Frame 2: A vast and fun view of the magic academy

[0387] Scene 3, Frame 3: The protagonist chats happily with a new friend

[0388] 6. Creation and Delivery of Deliverables

[0389] The server combines the pictures and story of each scene to create the final manga. It checks each page to ensure it is generated correctly and that there are no problems with the images or text. It converts the final manga into a downloadable format (e.g., PDF) and provides it to the user. The user clicks the download link to obtain the generated manga.

[0390] 7. Emotional feedback during execution

[0391] While the user is viewing the manga, the emotion engine analyzes the user's emotions in real time and provides feedback to the generation AI, allowing the content to be fine-tuned.

[0392] Specific examples of work processes

[0393] Below is a concrete example of generating a manga in which the protagonist, a high school student, is reincarnated into another world and becomes active at a magic academy.

[0394] Prompt statement:

[0395] 1. The user enters a summary such as "The protagonist is a high school student who is reincarnated in another world and becomes active at a magic academy" and submits it. The emotion engine recognizes a happy, smiling face.

[0396] 2. The server receives the summary and performs error checking.

[0397] 3. The server passes the synopsis to the generation AI and emotion engine, which generates the following detailed storyline:

[0398] Scene 1: The scene where the protagonist gets excited and is reincarnated into another world.

[0399] Scene 2: Meeting exciting new friends at the Magic Academy

[0400] 4. The server analyzes each scene and determines the frame layout (scene 1 is 4 frames, scene 2 is 6 frames, scene 3 is 3 frames).

[0401] 5. The server uses AI to generate pictures for each frame (a smiling protagonist and scenes of fun school life).

[0402] 6. The server combines the pictures and story, generates the final manga, and performs error checking.

[0403] 7. The server provides the final manga to the user in PDF format, which the user can download and view.

[0404] 8. Users can view the manga and the emotion engine can provide real-time feedback and regenerate it as needed.

[0405] As a result, the system of the present invention can efficiently provide high-quality manga that reflects the user's emotions.

[0406] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0407] Step 1:

[0408] The user enters a summary of the manga through a web form or application. The user enters a summary of the story, character settings, and key events in text format and presses the submit button. This entered text data becomes the input. At the same time, the emotion engine analyzes the user's facial expressions and tone to obtain emotion data (data based on facial expressions and tone). This becomes the output of emotion recognition data.

[0409] Specific operation: The user enters "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy" into the text field and presses the send button. The emotion engine analyzes the user's happy facial expression and tone of voice via the camera and microphone, and obtains the emotion data of a "happy smile."

[0410] Step 2:

[0411] The server receives the summary text data and emotion recognition data sent by the user and stores them in a database. An initial error check is performed on the received text data to ensure that the format and content are correct. The results of this error check are output. At the same time, the acquired emotion data is also stored in the database.

[0412] Specific operation: The server receives and saves the text data "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy." It then checks the text for consistency using an error checking tool. It also saves the emotion data "a happy smile."

[0413] Step 3:

[0414] The server analyzes the stored text data and emotion data and passes it to the generation AI. The analyzed data is the output as a result of this analysis. The generation AI generates a detailed storyline based on the analyzed data it receives. This generated storyline is the output.

[0415] Specific operation: The generation AI takes into account the emotional data of a "happy smile" and generates the following story development.

[0416] Scene 1: The scene where the protagonist gets excited and is reincarnated into another world.

[0417] Scene 2: The exciting first lesson at the magic academy

[0418] Scene 3: Meeting a fun-loving friend

[0419] Step 4:

[0420] The server receives the generated storyline data and generates the layout for each page and frame. The layout data is the output of this layout generation process. The generation AI determines the frame layout, size, and placement for each scene.

[0421] Specific operation: The server determines the following frame layout based on the story development.

[0422] Scene 1: 4-frame composition (expresses excitement)

[0423] Scene 2: 6 frames (emphasizes the fun of class)

[0424] Scene 3: Consists of 3 frames (depicting a friendly conversation with a friend)

[0425] Step 5:

[0426] The server automatically generates the images for each frame based on the generated layout data. The output of this process is image data for each frame. The generation AI depicts visual elements such as character expressions, poses, and backgrounds based on the emotional data.

[0427] Specific operation: The generation AI generates the following image.

[0428] Scene 1, Frame 1: The protagonist is thrown into another world with a smile on his face.

[0429] Scene 2, Frame 2: A vast and fun view of the magic academy

[0430] Scene 3, Frame 3: The protagonist chats happily with a new friend

[0431] Step 6:

[0432] The server integrates the illustrations of each scene with the storyline to create the final manga. The output of this integration process is the final manga data. An error check is performed on the final product. The results of the error check are also output.

[0433] What it does: Ensures that all pages are generated correctly and checks for missing images and text errors.

[0434] Step 7:

[0435] The server converts the final manga into a downloadable format such as PDF. The result of this conversion process is a downloadable file. The user clicks on the provided download link to get the final manga.

[0436] Specific operation: The server converts the generated manga into a PDF and displays a download link in an email or on a web page. The user clicks the link to download the PDF.

[0437] Step 8:

[0438] While the user is browsing the manga, the emotion engine analyzes the user's emotions in real time and provides feedback to the generation AI. As a result of this feedback process, adjustment instruction data is output, which allows the generation AI to fine-tune the content and improve user satisfaction.

[0439] Specific operation: The emotion engine analyzes the user's emotions through the camera and microphone, and instructs the generation AI to regenerate or adjust as necessary.

[0440] (Application example 2)

[0441] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0442] In the past, creating high-quality manga based on users' ideas required drawing skills and storytelling ability, which required a great deal of time and effort. It was also difficult to create manga that reflected the user's emotions, and there was no easy way to create a work that expressed one's own feelings and thoughts. For this reason, many people gave up on turning their ideas into manga.

[0443] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input a summary of a manga; a means for a generation AI to generate a detailed story development based on the input summary; a means for automatically generating a layout for each page and frame based on the generated story development; a means for automatically generating pictures for each frame based on the generated layout; a means for composing the final manga and providing it to the user; a means for analyzing the user's emotions at the time of input using an emotion recognition engine; a means for adjusting the content of the generated story and pictures based on the user's emotions; and a means for saving the generated manga in cloud storage and providing a URL from which the user can download it. This allows users to easily generate high-quality manga that reflect their own emotions and instantly download and share them.

[0444] "User" refers to an individual or organization that uses the system to create manga.

[0445] A "manga synopsis" is a brief description of the manga's story, character settings, major events, etc. that the user enters into the system.

[0446] "Generative AI" is an artificial intelligence that automatically generates detailed story development, page and panel layouts, and pictures for each panel based on an outline entered by the user.

[0447] A "detailed storyline" is a story that includes specific scenes and character actions, which the generation AI generates from the outline entered by the user.

[0448] "Layout" refers to the design of each page and panel, size, placement, etc., which is determined based on the story development generated by the generation AI.

[0449] An "emotion recognition engine" is a system that analyzes the emotions expressed by a user when they input information and acquires the user's emotional data.

[0450] "Cloud storage" refers to an internet storage service used to store generated manga data and provide an accessible URL.

[0451] "URL" refers to a link indicating the location of the manga data on cloud storage, which is used by the user to download the generated manga.

[0452] The present invention is a system that comprises a means for a user to input an outline of a manga, a means for a generation AI to generate a detailed story, a means for automatically generating the layout of each page and panel, a means for automatically generating pictures for each panel, a means for composing the final manga and providing it to the user, a means for analyzing the user's emotions using an emotion recognition engine, a means for adjusting the content of the generated story and pictures based on the emotions, and a means for saving the generated manga in cloud storage and providing a URL from which the user can download it.

[0453] System Configuration

[0454] The system is implemented using the following hardware and software.

[0455] Hardware:

[0456] Smartphone: Used by users to access the system, input synopsis and download generated manga.

[0457] Server: A cloud-based server (e.g., AWS, GCP) runs the generative AI model and emotion recognition engine.

[0458] software:

[0459] Generative AI model: Automatically generates storyline, layout, and illustrations based on a user-supplied outline.

[0460] Emotion recognition engine: Analyzes the emotions of the user when they input and provides feedback to the generative AI.

[0461] Cloud Storage: Store generated comics and manage download links.

[0462] Python: Responsible for data processing and calculations, and libraries used for API calls and image processing (Requests, PIL, etc.).

[0463] Specific examples of processing

[0464] Overview input and emotion recognition

[0465] The user inputs a summary of the manga using the system's smartphone application. For example,

[0466] "The protagonist, a high school student, is reincarnated into another world and becomes active at a magic academy."

[0467] At this time, the emotion recognition engine analyzes the user's emotion and acquires the corresponding emotion data. As a result of the analysis, emotion data such as "it looks fun" is acquired.

[0468] Storyline generation

[0469] The server receives the summary and emotion data sent by the user and passes it to the generative AI model as a prompt.

[0470] "The story is about a high school student protagonist who is reincarnated into another world and becomes active at a magic academy. The emotions recognized by the emotion engine seem fun."

[0471] This will generate a detailed storyline, for example the following scene structure:

[0472] Scene 1: The scene where the protagonist gets excited and is reincarnated into another world.

[0473] Scene 2: The exciting first lesson at the Magic Academy

[0474] Scene 3: Meeting a fun-loving friend

[0475] Layout and picture generation

[0476] The generative AI model determines the layout of each page and panel based on the generated storyline. After each scene and panel layout is determined, the illustrations for each panel are automatically generated. For example:

[0477] Scene 1, Frame 1: The protagonist is thrown into another world with a smile on his face.

[0478] Scene 2, Frame 2: A vast and fun view of the magic academy

[0479] Scene 3, Frame 3: The protagonist chats happily with a new friend

[0480] Preserving and providing manga

[0481] The server stores the generated manga in cloud storage and provides a URL for users to download. Using this download link, users can view and share the generated manga.

[0482] In this way, the present invention provides a system that allows users to easily generate high-quality, emotionally-reflective cartoons.

[0483] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0484] Step 1:

[0485] A user uses a smartphone application to input and submit a summary of a manga. For example, they might input, "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy." At this time, the summary text data is sent to the system. The user's face is also analyzed by a camera, and an emotion recognition engine obtains the user's emotional data. For example, it may recognize that the manga is "fun."

[0486] Step 2:

[0487] The server receives the summary text and emotion data sent by the user. It performs an initial error check on the received text data to ensure that it is in the correct format and content. If there are no errors, it prepares it to be passed as a prompt to the generative AI model along with the emotion data (e.g., "Looks fun").

[0488] Step 3:

[0489] The server passes the summary text and sentiment data to a generative AI model, which generates a detailed storyline using the following prompt:

[0490] "The story is about a high school student protagonist who is reincarnated into another world and becomes active at a magic academy. The emotions recognized by the emotion engine seem fun."

[0491] The generative AI model creates a detailed storyline based on these prompts, outputting story data including, for example, specific events and character actions for each scene.

[0492] Step 4:

[0493] The server receives the story data output from the generative AI model and then generates the layout of each page and frame. Based on the generated story development, the generative AI model automatically generates the frame division, size, and placement of each scene. For example, it outputs layout data such as Scene 1 consisting of four frames, Scene 2 consisting of six frames, etc.

[0494] Step 5:

[0495] The server again uses the generative AI model to generate the image for each frame based on the layout data. The image for each frame includes visual elements such as the character's facial expression, pose, and background, and is adjusted based on the emotional data. For example, in Scene 1, Frame 1, an image of the protagonist smiling and being thrown into a different world is generated. In this way, the image for each frame is output.

[0496] Step 6:

[0497] The server then combines the generated images and story data to create the final manga. After combining, it checks the images and text for errors to ensure the final manga has been generated correctly. After the error check is complete, it saves the generated manga to cloud storage.

[0498] Step 7:

[0499] The server generates a download URL for the manga stored in the cloud storage and provides it to the user. The user can use the URL to download, view, and share the generated manga. In addition, an emotion recognition engine provides real-time emotional feedback to the user as they view the manga, and the manga can be regenerated as needed.

[0500] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0501] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0502] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0503] [Second embodiment]

[0504] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0505] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0506] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0507] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0508] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0509] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0510] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0511] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0512] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0513] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0514] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0515] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0516] System Overview

[0517] This invention is a system in which a generation AI automatically generates a detailed storyline, page and frame layouts, and the pictures for each frame based on a manga outline entered by a user, and then constructs and provides the final manga to the user. The system is configured around a server, receives user input, uses the generation AI to perform various processes, and provides the final product to the user.

[0518] Program Overview

[0519] The program of this system executes a series of processes in a specific order. Below we will explain the process in brief, with specific examples.

[0520] 1. Enter the summary

[0521] A user accesses the system and inputs a rough outline of the manga. The input information is received and stored by the server.

[0522] Example: A high school student protagonist is reincarnated into another world and becomes active at a magic academy.

[0523] 2. Storyline generation

[0524] The server passes the user-entered synopsis to a generation AI, which then generates a detailed storyline based on the synopsis and builds out the specific events that occur in each scene.

[0525] Examples:

[0526] Scene 1: The scene where the protagonist is reincarnated into another world

[0527] Scene 2: First lesson at the Magic Academy

[0528] Scene 3: Confronting a Sassy Rival

[0529] 3. Generate layout

[0530] The server automatically generates the layout of each page and frame based on the generated storyline, using a generation AI to determine the frame layout, size, and placement for each scene.

[0531] Examples:

[0532] Scene 1: 4 frames

[0533] Scene 2: 6 frames

[0534] Scene 3: Consists of 3 frames

[0535] 4. Image Generation

[0536] The server automatically generates the images for each frame based on the generated layout information. The AI ​​then draws the character's facial expressions, poses, background, and other elements.

[0537] Examples:

[0538] Scene 1, Frame 1: The protagonist is thrown into another world with a surprised expression.

[0539] Scene 2, Frame 2: The vast landscape of the magic academy

[0540] Scene 3, Frame 3: Dialogue with a new friend

[0541] 5. Creation and Delivery of Deliverables

[0542] The server combines the illustrations and story of each scene to create the final manga, which it then checks for errors and makes available to users in a downloadable format.

[0543] Examples:

[0544] Deliver the final manga in PDF format or other suitable format

[0545] Users can download, view, and print

[0546] Specific examples of work processes

[0547] Below is a concrete example of generating a manga in which the protagonist, a high school student, is reincarnated into another world and becomes active at a magic academy.

[0548] 1. The user enters the synopsis: "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy."

[0549] 2. The server receives the summary and checks for errors.

[0550] 3. The server passes the outline to the generation AI, which generates the following detailed storyline:

[0551] The scene where the main character is reincarnated into another world

[0552] Meeting new friends at the magic academy

[0553] Magical training and rivalry

[0554] 4. The server analyzes each scene and determines the frame layout (scene 1 is 4 frames, scene 2 is 6 frames, scene 3 is 3 frames).

[0555] 5. The server uses AI to generate the images for each frame (the surprised main character, the magic effects, the school background).

[0556] 6. The server combines the pictures and story, generates the final manga, and performs error checking.

[0557] 7. The server provides the final manga to the user in PDF format, which the user can download and view.

[0558] In this way, the system of the present invention significantly reduces the effort required to create manga and efficiently provides high-quality manga.

[0559] The processing flow will be explained below.

[0560] Step 1:

[0561] A user accesses the system and enters a rough outline of a manga. The user enters a text summary of the story, character settings, and major events into the input fields of a web form or application, and then presses the submit button.

[0562] Step 2:

[0563] The server receives the summary sent by the user and saves the text data. It performs an initial error check on the saved text data to ensure that the correct format and content are entered.

[0564] Step 3:

[0565] The server analyzes the summary text and passes it to the generation AI, which then generates a detailed storyline based on the summary, including specific events and character actions in each scene.

[0566] Step 4:

[0567] The server automatically generates the layout of each page and frame based on the generated storyline. The AI ​​determines the frame layout, size, and placement for each scene.

[0568] Step 5:

[0569] The server automatically generates the images for each frame based on the generated layout information, using a generative AI to depict visual elements such as character expressions, poses, and backgrounds.

[0570] Step 6:

[0571] The server combines the illustrations of each panel with the storyline to create the final manga. It then performs error checks to ensure that each page is generated correctly and that there are no problems with the images or text.

[0572] Step 7:

[0573] The server converts the final comic into a downloadable format (e.g. PDF) and provides it to the user, who clicks on the download link to retrieve the generated comic.

[0574] Step 8:

[0575] The user can view the downloaded manga and, if necessary, make edits or print it.

[0576] Example 1

[0577] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0578] Traditional manga production requires a great deal of time and effort, and many manual processes result in low productivity. In particular, there is a lot of creative work involved, such as developing the storyline, setting the layout of pages and panels, and creating the illustrations, which can lead to variations in quality. There is a need for a system that can solve these problems and automatically generate high-quality manga efficiently.

[0579] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0580] In this invention, the server includes: means for a user to input a summary of a manga; means for a generative AI model to generate a detailed storyline based on the input summary; means for automatically generating a layout for each page and frame based on the generated storyline; means for automatically generating pictures for each frame based on the generated layout; means for combining the pictures for each frame and the storyline to create a final manga and perform error checking; and means for providing the final manga to the user. This makes it possible to automatically generate and quickly provide a high-quality manga efficiently based on the summary input by the user.

[0581] "User" refers to a person who uses the system to input a manga outline and receive the final manga.

[0582] A "manga summary" refers to information that indicates the general content and structure of the story.

[0583] A "generative AI model" refers to an artificial intelligence model that generates storylines and images based on input information.

[0584] A "detailed storyline" refers to a progression of the story that includes specific scenes and events generated based on the outline.

[0585] "Page and panel layout" refers to the composition information such as the panel layout, size, and placement for each page of a manga.

[0586] "Artwork in each panel" refers to the characters, background, and other visual elements that appear in each panel of a manga.

[0587] "Error checking" refers to the process of checking the consistency and quality of the generated manga and detecting errors.

[0588] "PDF format" is an abbreviation for Portable Document Format and refers to one of the formats for electronic documents.

[0589] "Means for providing to the user" refers to a method for providing the final generated manga in a form that is accessible to the user.

[0590] System Overview

[0591] This invention is a system in which a generative AI model automatically generates a detailed storyline, page and frame layouts, and the pictures for each frame based on a manga outline entered by a user, composing the final manga and providing it to the user. The system is centered around a server, which receives user input, uses the generative AI model to perform various processes, and provides the final product to the user.

[0592] Program Overview

[0593] The system executes a series of steps in a specific order to generate a manga based on a user-supplied synopsis, as detailed below.

[0594] Enter a summary

[0595] A user accesses the system through a terminal and inputs a rough outline of the manga. The terminal sends the input information to the server, which then stores it in a database.

[0596] Examples:

[0597] The user enters a summary such as "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy."

[0598] Storyline generation

[0599] The server then passes the saved summary data to a generative AI model, which uses this summary to generate a detailed storyline and build specific events for each scene. This model uses OpenAI's GPT-4 and other AI models.

[0600] Examples:

[0601] Scene 1: The scene where the protagonist is reincarnated into another world

[0602] Scene 2: First lesson at the Magic Academy

[0603] Scene 3: A confrontation with a nasty rival

[0604] Generate layout

[0605] The server automatically generates the layout of each page and frame based on the generated storyline. It uses a generative AI model to determine the frame layout, size, and placement for each scene. OpenAI's DALL-E and other tools are used here.

[0606] Examples:

[0607] Scene 1: 4 frames

[0608] Scene 2: 6 frames

[0609] Scene 3: Consists of 3 frames

[0610] Picture generation

[0611] The server automatically generates the images for each frame based on the generated layout information. It uses a generative AI model to draw the character's facial expressions, poses, background, etc. DALL-E and Stable Diffusion are used here.

[0612] Examples:

[0613] Scene 1, Frame 1: The protagonist is thrown into another world with a surprised expression.

[0614] Scene 2, Frame 2: The vast landscape of the magic academy

[0615] Scene 3, Frame 3: Dialogue with a new friend

[0616] Generate and deliver deliverables

[0617] The server then combines the illustrations and story of each scene to create the final manga, performs error checking, and delivers it to the user in a format that could be PDF or EPUB.

[0618] Examples:

[0619] The final manga will be provided to users in PDF format, which they can download, view, and print.

[0620] Example prompt

[0621] Below is a specific example of a prompt sentence to be passed to the generative AI model.

[0622] Storyline generation prompt:

[0623] "Generate a detailed storyline based on the outline of a high school student protagonist who is reincarnated into another world and becomes active at a magic academy."

[0624] Layout generation prompt:

[0625] "Generate a page and frame layout where Scene 1 is 4 frames, Scene 2 is 6 frames, and Scene 3 is 3 frames."

[0626] Picture generation prompt:

[0627] "In Scene 1, Frame 1, please draw the main character with a surprised expression as he is thrown into another world."

[0628] The system of the present invention efficiently and automatically generates manga from an outline entered by a user based on the above procedures and elements, enabling high-quality and rapid provision.

[0629] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0630] System program processing flow

[0631] Step 1: Enter a summary

[0632] The user accesses the system through a terminal and inputs a story outline, including themes and major events of the manga.

[0633] Input: Manga summary (text)

[0634] Output: Summary data (sent to server)

[0635] Specific operation: The user accesses the input form in a web browser and enters a summary such as "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy." The device detects a click on the "Submit" button and sends the input data to the server as a POST request. The server receives the POST request and saves the entered summary data in a database.

[0636] Step 2: Analyze the summary and check for errors

[0637] The server parses the received summary data and performs some initial error checking to ensure there are no typos or missing information.

[0638] Input: Summary data (fetched from database)

[0639] Output: Verified summary data

[0640] What it does: The server fetches summary data from the database and uses a text analysis module to detect grammatical errors and missing information. If errors are found, it sends feedback to the user to prompt them to make corrections.

[0641] Step 3: Generate a storyline

[0642] The server passes the error-free summary data to a generative AI model (e.g., GPT-4), which generates a detailed storyline based on the prompt.

[0643] Input: Validated summary data

[0644] Output: Detailed storyline

[0645] Specific operation: The server generates a prompt and sends an API request to the generative AI model saying, "Please generate a detailed storyline based on the outline that the high school protagonist is reincarnated in another world and becomes active at a magic academy." The generative AI model generates a detailed storyline and returns it to the server. The server temporarily stores the received storyline data.

[0646] Step 4: Generate the layout

[0647] The server automatically generates the layout of each page and frame based on the generated storyline. It uses a generative AI model (e.g., DALL-E) to determine the frame layout, size, and placement for each scene.

[0648] Input: Detailed storyline

[0649] Output: Page and frame layout

[0650] Specific operation: The server analyzes the story development data and sends an API request to the generation AI model with the prompt "Please generate a page and frame layout where Scene 1 is 4 frames, Scene 2 is 6 frames, and Scene 3 is 3 frames." The generation AI model generates the page and frame layout and returns it to the server. The server temporarily stores the generated layout information.

[0651] Step 5: Generate the image

[0652] The server automatically generates the images for each frame based on the generated layout information, using a generative AI model (such as DALL-E or Stable Diffusion) to draw the character's facial expressions, poses, background, and other details.

[0653] Input: Page and frame layout

[0654] Output: Picture of each frame

[0655] Specific operation: The server fetches layout information, generates a prompt for each frame: "In Scene 1, Frame 1, please draw the main character with a surprised expression as he is thrown into another world," and sends an API request to the generative AI model. The generative AI model generates an image for each frame and returns it to the server. The server temporarily stores the received image data.

[0656] Step 6: Generate and deliver artifacts

[0657] The server combines the illustrations of each scene with the story to create the final manga, which is then verified and checked for errors before being delivered to the user.

[0658] Input: Pictures of each frame and detailed storyline

[0659] Output: Finished manga (PDF format, etc.)

[0660] Specific operation: The server integrates the saved storyline data and illustration data, performs error checks and quality inspections (checking page layout and illustration consistency), converts the final manga into PDF or other formats, and provides a download link to the user. The user clicks the download link on their device to obtain the work.

[0661] In this way, this system uses a series of automated processes to efficiently generate and quickly deliver high-quality manga based on the outline entered by the user.

[0662] (Application example 1)

[0663] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0664] Traditional manga production requires a lot of time and effort, and there is no way to automate the entire process, from story development to layout and illustration creation. This makes it difficult for individual creators or small teams to quickly produce high-quality manga. There is also a lack of an effective platform for sharing and evaluating the manga. Furthermore, there are challenges with error checking and providing the manga in digital format.

[0665] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0666] In this invention, the server includes: a means for a user to input a summary of a manga; a means for a generation AI to generate a detailed storyline based on the input summary; a means for automatically generating a layout for each page and frame based on the generated storyline; a means for automatically generating pictures for each frame based on the generated layout; a means for composing the final manga and providing it to the user; a means for uploading the generated manga to a content platform and sharing it with other users; and a means for other users to rate and comment on the generated manga. This allows users to easily generate high-quality manga, share it with other users, and receive ratings. Error checking and provision in electronic format can also be efficiently performed.

[0667] To match the application example, we will create definitions of the important words included in the patent claims in the following format.

[0668] A "user" is an individual or entity that uses the system to input a comic outline and receive the generated comic.

[0669] "Generative AI" is an artificial intelligence technology that automatically generates detailed story development, page and panel layouts, and pictures for each panel from an input outline.

[0670] A "storyline" is a sequence of specific events and scenes that the generative AI generates based on a summary entered by the user.

[0671] "Layout" refers to the structure including the placement and size of each page and frame, and the relationships between frames.

[0672] "Pictures" refer to visual elements such as the character's facial expression, pose, and background drawn in each panel.

[0673] The "content platform" is an online system that allows users to share their manga with other users and receive ratings.

[0674] "Sharing" is the act of sharing the created manga with other users.

[0675] "Evaluation" is the act of other users providing feedback on the generated comic.

[0676] "Error checking" is the process of checking the user-entered synopsis and generated content for defects and errors.

[0677] "Electronic format" refers to the method of providing the final manga in a digital data format such as PDF.

[0678] MODE FOR CARRYING OUT THE INVENTION

[0679] This invention relates to a system in which a generation AI automatically generates a detailed storyline, page and frame layouts, and frame art based on a manga outline entered by a user, and then constructs the final manga and provides it to the user. Specific embodiments for carrying out this invention are described below.

[0680] System configuration

[0681] The system of this invention consists of a terminal used by a user, a server that runs a generative AI model, and a content platform.

[0682] Device: A mobile device such as a smartphone or tablet is used. Users input the outline of the manga through this device.

[0683] Server: A generative AI model is installed on a cloud server that analyzes the outline, generates detailed story development, determines the layout, and generates the images for each frame.

[0684] Content Platform: The manga will be uploaded to a content platform where other users can view and rate it.

[0685] Program processing

[0686] The server proceeds with the following steps:

[0687] 1. Enter summary and check for errors:

[0688] The user inputs a summary of the manga using a terminal, and the server receives this summary and performs some initial error checking.

[0689] Example: The prompt is "The high school student protagonist is reincarnated into another world and becomes active at a magic academy."

[0690] 2. Storyline generation:

[0691] The server passes the received synopsis to a generative AI model to generate a detailed storyline.

[0692] Examples:

[0693] Scene 1: The scene where the protagonist is reincarnated into another world

[0694] Scene 2: First lesson at the Magic Academy

[0695] Scene 3: Confronting a Sassy Rival

[0696] 3. Generate layout:

[0697] The layout of each page and panel is automatically generated based on the story development. Panel layout, size, and placement are determined.

[0698] Examples:

[0699] Scene 1: 4 frames

[0700] Scene 2: 6 frames

[0701] Scene 3: Consists of 3 frames

[0702] 4. Picture generation:

[0703] Based on the generated layout, the images for each frame are generated, including the character's facial expressions, poses, background, etc.

[0704] Examples:

[0705] Scene 1, Frame 1: The protagonist is thrown into another world with a surprised expression.

[0706] Scene 2, Frame 2: The vast landscape of the magic academy

[0707] Scene 3, Frame 3: Dialogue with a new friend

[0708] 5. Composition and presentation of the manga:

[0709] The illustrations for each scene are combined with the story to create the final manga, which is then checked for errors. The finished manga is then provided in electronic format.

[0710] Examples:

[0711] The final manga is provided to users in PDF format, which they can download and view.

[0712] Prototyping:

[0713] This system allows users to easily generate high-quality manga, share them with other users, and receive feedback. The server utilizes a generative AI model to efficiently analyze, generate, and organize data. Specifically, OpenAI's generative AI model and cloud server technology are used to achieve a highly accurate generation process in real time.

[0714] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0715] Program processing steps

[0716] Step 1:

[0717] A user inputs a summary of a manga using a terminal. The input summary (prompt sentence) is received by the server. The server performs an initial error check on this summary.

[0718] Input: The prompt entered by the user (e.g., "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy.")

[0719] Data processing: Check for errors to ensure the summary is entered correctly.

[0720] Output: Prompt statement that passes error checking

[0721] Step 2:

[0722] The server passes the prompt sentences that pass error checks to a generative AI model, which then generates a detailed storyline.

[0723] Input: A prompt that passes error checking

[0724] Data computation: Generative AI models are used to generate detailed storylines (determine the specific events for each scene).

[0725] Output: A detailed storyline generated

[0726] Step 3:

[0727] The server automatically generates the layout of each page and frame based on the generated story development.

[0728] Input: A detailed generated storyline

[0729] Data calculation: An algorithm is used to automatically generate the layout (panel division, size, and placement) of each page and panel.

[0730] Output: The layout of each generated page and frame

[0731] Step 4:

[0732] The server automatically generates pictures for each frame based on the generated layout information.

[0733] Input: The layout of each generated page and frame

[0734] Data calculation: Uses generative AI models to draw character expressions, poses, backgrounds, etc.

[0735] Output: Picture of each frame

[0736] Step 5:

[0737] The server combines the pictures of each scene with the story to create the final manga and performs error checks.

[0738] Input: Pictures of each frame and detailed storyline

[0739] Data processing: Combining the data and checking for errors in the final manga.

[0740] Output: Final cartoon

[0741] Step 6:

[0742] The server uploads the completed manga to a content platform where other users can view, rate, and comment on it.

[0743] Input: Final cartoon

[0744] Data Calculation: Processing the upload and applying publishing settings on the content platform.

[0745] Output: Manga published on a content platform

[0746] Step 7:

[0747] Other users view the comics created on the content platform and provide ratings and comments.

[0748] Input: Manga published on content platforms

[0749] Data calculation: Performs the process of accepting ratings and comments from other users.

[0750] Output: Manga with ratings and comments

[0751] In this way, a system has been realized in which users, servers, and content platforms work together to efficiently generate, share, and evaluate high-quality manga.

[0752] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0753] System Overview

[0754] This invention is a system in which a generation AI automatically generates a detailed storyline, page and panel layout, and illustrations for each panel based on a manga outline entered by the user, composing the final manga and providing it to the user. Furthermore, by combining it with an emotion engine, the system is characterized by recognizing the user's emotions and adjusting the content of the story and illustrations accordingly. The system is centered around a server, which receives user input, utilizes the generation AI and emotion engine to perform various processes, and provides the final product to the user.

[0755] Program Overview

[0756] The program of this system executes a series of processes in a specific order. Below we will explain the process in brief, with specific examples.

[0757] 1. Summary input and emotion recognition

[0758] A user accesses the system and inputs a rough outline of a manga. The user enters a summary of the story, character settings, and major events in text format into the input fields of a web form or application, and then presses the submit button. At the same time, the emotion engine analyzes the user's emotions as they type.

[0759] Example: When the user inputs a summary of the story in which the protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy, the emotion engine recognizes that the user is smiling happily.

[0760] 2. Receiving and analyzing the summary

[0761] The server receives the summary sent by the user and saves the text data. It performs an initial error check on the saved text data to ensure that the format and content are correct. At the same time, it saves the emotion data analyzed by the emotion engine.

[0762] 3. Storyline generation

[0763] The server analyzes the summary text and emotion data and passes it to the generation AI, which then generates a detailed storyline based on the summary and the user's emotions. This storyline includes specific events and character actions in each scene, and the content is adjusted based on the emotion data.

[0764] Examples:

[0765] Scene 1: The scene where the protagonist gets excited and is reincarnated into another world.

[0766] Scene 2: The exciting first lesson at the Magic Academy

[0767] Scene 3: Meeting a fun-loving friend

[0768] 4. Generate layout

[0769] The server automatically generates the layout of each page and frame based on the generated storyline. The AI ​​determines the frame layout, size, and placement for each scene.

[0770] Examples:

[0771] Scene 1: Consists of four frames (each expressing excitement)

[0772] Scene 2: 6 frames (emphasizes the fun of class)

[0773] Scene 3: Consists of 3 frames (depicting a friendly conversation with a friend)

[0774] 5. Image Generation

[0775] The server automatically generates the images for each frame based on the generated layout information, using a generative AI to depict visual elements such as character expressions, poses, and backgrounds based on emotional data.

[0776] Examples:

[0777] Scene 1, Frame 1: The protagonist is thrown into another world with a smile on his face.

[0778] Scene 2, Frame 2: A vast and fun view of the magic academy

[0779] Scene 3, Frame 3: The protagonist chats happily with a new friend

[0780] 6. Creation and Delivery of Deliverables

[0781] The server combines the illustrations and story of each scene to create the final manga. It then performs error checks to ensure that each page is generated correctly and that there are no problems with the images or text.

[0782] 7. Providing Results

[0783] The server converts the final comic into a downloadable format (e.g. PDF) and provides it to the user, who clicks on the download link to retrieve the generated comic.

[0784] 8. Emotional feedback during execution

[0785] Even while a user is browsing a manga, the emotion engine can analyze the user's emotions in real time, and the generation AI can fine-tune the content based on the emotions.

[0786] Specific examples of work processes

[0787] Below is a concrete example of generating a manga in which the protagonist, a high school student, is reincarnated into another world and becomes active at a magic academy.

[0788] 1. The user enters and submits a summary: "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy." At the same time, the emotion engine recognizes a happy, smiling face.

[0789] 2. The server receives the summary and checks for errors.

[0790] 3. The server passes the synopsis to the generation AI and emotion engine, which generates the following detailed storyline:

[0791] A scene where the main character gets excited and is reincarnated into another world

[0792] Meet exciting new friends at the Magic Academy

[0793] 4. The server analyzes each scene and determines the frame layout (scene 1 is 4 frames, scene 2 is 6 frames, scene 3 is 3 frames).

[0794] 5. The server uses AI to generate pictures for each frame (a smiling protagonist and scenes of fun school life).

[0795] 6. The server combines the pictures and story, generates the final manga, and performs error checking.

[0796] 7. The server provides the final manga to the user in PDF format, which the user can download and view.

[0797] 8. Users can view the manga and the emotion engine can provide real-time feedback and regenerate it as needed.

[0798] In this way, the system of the present invention, which is combined with an emotion engine, can efficiently provide high-quality manga that reflects the user's emotions.

[0799] The processing flow will be explained below.

[0800] Step 1:

[0801] A user accesses the system and inputs a rough outline of a manga. The user enters a summary of the story, character settings, and major events in text format into the input fields of a web form or application, and then presses the submit button. At the same time, the emotion engine analyzes the user's facial expressions and voice to recognize the emotion they are expressing as they type.

[0802] Step 2:

[0803] The server receives the summary sent by the user and saves the text data. At the same time, the emotion data analyzed by the emotion engine is also sent to the server and saved. An initial error check is performed on the saved text data to ensure that the correct format and content have been entered.

[0804] Step 3:

[0805] The server analyzes the summary text and emotion data and passes it to the generation AI, which then generates a detailed storyline based on the summary and the user's emotions. This storyline includes specific events and character actions in each scene, and the content is adjusted based on the emotion data.

[0806] Step 4:

[0807] The server automatically generates the layout of each page and frame based on the generated storyline. The AI ​​determines the frame layout, size, and placement for each scene.

[0808] Step 5:

[0809] The server automatically generates the images for each frame based on the generated layout information, using a generative AI to depict visual elements such as character expressions, poses, and backgrounds based on emotional data.

[0810] Step 6:

[0811] The server combines the generated pictures of each panel with the storyline to create the final manga. It then performs error checks to ensure that each page is generated correctly and that there are no problems with the images or text.

[0812] Step 7:

[0813] The server converts the final comic into a downloadable format (e.g. PDF) and provides it to the user, who clicks on the download link to retrieve the generated comic.

[0814] Step 8:

[0815] As users browse the manga, the emotion engine analyzes their emotions in real time. Depending on their emotions, the generative AI readjusts the content and updates the storyline and frame art until the user is satisfied.

[0816] Example 2

[0817] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0818] Modern manga production requires a lot of time and effort, creating an efficient automatic generation system. However, existing automatic generation systems only generate stories and layouts based on user input, and are unable to adjust content to reflect the user's emotions. Therefore, a system that can recognize the user's emotions and adjust content based on them is needed. Furthermore, they lack error checking capabilities at each stage of story generation, panel layout, image generation, and delivery of the final product, which can result in incomplete manga.

[0819] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0820] In this invention, the server includes: a means for a user to input a summary of a manga; an emotion recognition means for analyzing the user's emotions at the time of input; a generation AI for generating a detailed storyline based on the input summary and emotion data; a means for automatically generating a layout for each page and frame based on the generated storyline; a means for automatically generating pictures for each frame based on the generated layout; and a means for constructing and providing the final manga to the user. This allows for efficient automatic generation of high-quality manga that reflects the user's emotions and allows for fine-tuning of the content in real time in response to the user's emotions. Furthermore, error checking at each step allows for a highly refined final product.

[0821] "User" refers to a person who accesses the system and inputs a summary of a manga.

[0822] "Emotion recognition means" refers to technology that analyzes the user's emotions while the user is entering a summary.

[0823] "Generative AI" refers to artificial intelligence that generates detailed storylines based on a summary and emotional data entered by the user.

[0824] "Story development" refers to the detailed narrative progression generated by the generative AI, including specific events and character actions in each scene.

[0825] "Layout" refers to the design that determines the placement and size of each page and panel based on the generated story development.

[0826] "Automatic picture generation means" refers to a technology that automatically generates pictures for each frame based on the generated layout information.

[0827] "Error checking" refers to the process of checking whether generated data or final products are fraudulent or incorrect.

[0828] "Final product" refers to the finished manga that is created by integrating the generated story, layout, and illustrations.

[0829] "Delivery means" refers to the technology by which the final product is delivered to users in a downloadable format.

[0830] "Real-time feedback" refers to the process in which the emotion engine analyzes the user's emotions in real time while they are reading the manga, and the generative AI fine-tunes the content.

[0831] System Overview

[0832] This invention is a system in which a generation AI automatically generates a detailed storyline, page and panel layout, and the illustrations for each panel based on a manga outline entered by the user, and provides the final manga to the user. This system is characterized by recognizing the user's emotions by combining it with an emotion engine, and adjusting the content of the story and illustrations accordingly. The system is configured around a server, which receives user input, uses the generation AI and emotion engine to perform various processes, and provides the final product to the user.

[0833] Hardware and software used

[0834] This system uses the following hardware and software:

[0835] Server: The central location for receiving, storing, analyzing, and generating data. This is where the emotion engine and generative AI models run.

[0836] Generative AI model: Generates detailed storylines and frame illustrations based on user input and emotional data.

[0837] Emotion Engine: Recognizes user emotions in real time and provides emotional data.

[0838] User interface: A web form or application for users to enter a synopsis of their manga.

[0839] Program processing

[0840] 1. Summary input and emotion recognition

[0841] Users enter a summary of their manga through a web form or application, and the emotion engine analyzes their facial expressions and tone to capture emotional data.

[0842] Example: A user enters the summary "The protagonist is a high school student who is reincarnated in another world and becomes active at a magic academy," and the emotion engine recognizes that the user is smiling happily.

[0843] 2. Receiving and analyzing the summary

[0844] The server receives the summary sent by the user and saves the text data. It performs an initial error check on the saved text data to ensure that it is in the correct format and content. At the same time, it saves the emotion data obtained from the emotion engine.

[0845] 3. Storyline generation

[0846] The server analyzes the summary text and emotional data and passes it to the generation AI, which then generates a detailed storyline based on the summary and emotional data, including specific events and character actions in each scene, and adjusts the content based on the emotional data.

[0847] Examples:

[0848] Scene 1: The scene where the protagonist gets excited and is reincarnated into another world.

[0849] Scene 2: The exciting first lesson at the magic academy

[0850] Scene 3: Meeting a fun-loving friend

[0851] 4. Generate layout

[0852] The server automatically generates the layout of each page and frame based on the storyline. The AI ​​determines the frame layout, size, and placement for each scene.

[0853] Examples:

[0854] Scene 1: 4-frame composition (expresses excitement)

[0855] Scene 2: 6 frames (emphasizes the fun of class)

[0856] Scene 3: Consists of 3 frames (depicting a friendly conversation with a friend)

[0857] 5. Image Generation

[0858] The server automatically generates the images for each frame based on the generated layout information. The AI ​​then depicts visual elements such as character expressions, poses, and backgrounds based on emotional data.

[0859] Examples:

[0860] Scene 1, Frame 1: The protagonist is thrown into another world with a smile on his face.

[0861] Scene 2, Frame 2: A vast and fun view of the magic academy

[0862] Scene 3, Frame 3: The protagonist chats happily with a new friend

[0863] 6. Creation and Delivery of Deliverables

[0864] The server combines the pictures and story of each scene to create the final manga. It checks each page to ensure it is generated correctly and that there are no problems with the images or text. It converts the final manga into a downloadable format (e.g., PDF) and provides it to the user. The user clicks the download link to obtain the generated manga.

[0865] 7. Emotional feedback during execution

[0866] While the user is viewing the manga, the emotion engine analyzes the user's emotions in real time and provides feedback to the generation AI, allowing the content to be fine-tuned.

[0867] Specific examples of work processes

[0868] Below is a concrete example of generating a manga in which the protagonist, a high school student, is reincarnated into another world and becomes active at a magic academy.

[0869] Prompt statement:

[0870] 1. The user enters a summary such as "The protagonist is a high school student who is reincarnated in another world and becomes active at a magic academy" and submits it. The emotion engine recognizes a happy, smiling face.

[0871] 2. The server receives the summary and performs error checking.

[0872] 3. The server passes the synopsis to the generation AI and emotion engine, which generates the following detailed storyline:

[0873] Scene 1: The scene where the protagonist gets excited and is reincarnated into another world.

[0874] Scene 2: Meeting exciting new friends at the Magic Academy

[0875] 4. The server analyzes each scene and determines the frame layout (scene 1 is 4 frames, scene 2 is 6 frames, scene 3 is 3 frames).

[0876] 5. The server uses AI to generate pictures for each frame (a smiling protagonist and scenes of fun school life).

[0877] 6. The server combines the pictures and story, generates the final manga, and performs error checking.

[0878] 7. The server provides the final manga to the user in PDF format, which the user can download and view.

[0879] 8. Users can view the manga and the emotion engine can provide real-time feedback and regenerate it as needed.

[0880] As a result, the system of the present invention can efficiently provide high-quality manga that reflects the user's emotions.

[0881] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0882] Step 1:

[0883] The user enters a summary of the manga through a web form or application. The user enters a summary of the story, character settings, and key events in text format and presses the submit button. This entered text data becomes the input. At the same time, the emotion engine analyzes the user's facial expressions and tone to obtain emotion data (data based on facial expressions and tone). This becomes the output of emotion recognition data.

[0884] Specific operation: The user enters "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy" into the text field and presses the send button. The emotion engine analyzes the user's happy facial expression and tone of voice via the camera and microphone, and obtains the emotion data of a "happy smile."

[0885] Step 2:

[0886] The server receives the summary text data and emotion recognition data sent by the user and stores them in a database. An initial error check is performed on the received text data to ensure that the format and content are correct. The results of this error check are output. At the same time, the acquired emotion data is also stored in the database.

[0887] Specific operation: The server receives and saves the text data "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy." It then checks the text for consistency using an error checking tool. It also saves the emotion data "a happy smile."

[0888] Step 3:

[0889] The server analyzes the stored text data and emotion data and passes it to the generation AI. The analyzed data is the output as a result of this analysis. The generation AI generates a detailed storyline based on the analyzed data it receives. This generated storyline is the output.

[0890] Specific operation: The generation AI takes into account the emotional data of a "happy smile" and generates the following story development.

[0891] Scene 1: The scene where the protagonist gets excited and is reincarnated into another world.

[0892] Scene 2: The exciting first lesson at the magic academy

[0893] Scene 3: Meeting a fun-loving friend

[0894] Step 4:

[0895] The server receives the generated storyline data and generates the layout for each page and frame. The layout data is the output of this layout generation process. The generation AI determines the frame layout, size, and placement for each scene.

[0896] Specific operation: The server determines the following frame layout based on the story development.

[0897] Scene 1: 4-frame composition (expresses excitement)

[0898] Scene 2: 6 frames (emphasizes the fun of class)

[0899] Scene 3: Consists of 3 frames (depicting a friendly conversation with a friend)

[0900] Step 5:

[0901] The server automatically generates the images for each frame based on the generated layout data. The output of this process is image data for each frame. The generation AI depicts visual elements such as character expressions, poses, and backgrounds based on the emotional data.

[0902] Specific operation: The generation AI generates the following image.

[0903] Scene 1, Frame 1: The protagonist is thrown into another world with a smile on his face.

[0904] Scene 2, Frame 2: A vast and fun view of the magic academy

[0905] Scene 3, Frame 3: The protagonist chats happily with a new friend

[0906] Step 6:

[0907] The server integrates the illustrations of each scene with the storyline to create the final manga. The output of this integration process is the final manga data. An error check is performed on the final product. The results of the error check are also output.

[0908] What it does: Ensures that all pages are generated correctly and checks for missing images and text errors.

[0909] Step 7:

[0910] The server converts the final manga into a downloadable format such as PDF. The result of this conversion process is a downloadable file. The user clicks on the provided download link to get the final manga.

[0911] Specific operation: The server converts the generated manga into a PDF and displays a download link in an email or on a web page. The user clicks the link to download the PDF.

[0912] Step 8:

[0913] While the user is browsing the manga, the emotion engine analyzes the user's emotions in real time and provides feedback to the generation AI. As a result of this feedback process, adjustment instruction data is output, which allows the generation AI to fine-tune the content and improve user satisfaction.

[0914] Specific operation: The emotion engine analyzes the user's emotions through the camera and microphone, and instructs the generation AI to regenerate or adjust as necessary.

[0915] (Application example 2)

[0916] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0917] In the past, creating high-quality manga based on users' ideas required drawing skills and storytelling ability, which required a great deal of time and effort. It was also difficult to create manga that reflected the user's emotions, and there was no easy way to create a work that expressed one's own feelings and thoughts. For this reason, many people gave up on turning their ideas into manga.

[0918] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input a summary of a manga; a means for a generation AI to generate a detailed story development based on the input summary; a means for automatically generating a layout for each page and frame based on the generated story development; a means for automatically generating pictures for each frame based on the generated layout; a means for composing the final manga and providing it to the user; a means for analyzing the user's emotions at the time of input using an emotion recognition engine; a means for adjusting the content of the generated story and pictures based on the user's emotions; and a means for saving the generated manga in cloud storage and providing a URL from which the user can download it. This allows users to easily generate high-quality manga that reflect their own emotions and instantly download and share them.

[0919] "User" refers to an individual or organization that uses the system to create manga.

[0920] A "manga synopsis" is a brief description of the manga's story, character settings, major events, etc. that the user enters into the system.

[0921] "Generative AI" is an artificial intelligence that automatically generates detailed story development, page and panel layouts, and pictures for each panel based on an outline entered by the user.

[0922] A "detailed storyline" is a story that includes specific scenes and character actions, which the generation AI generates from the outline entered by the user.

[0923] "Layout" refers to the design of each page and panel, size, placement, etc., which is determined based on the story development generated by the generation AI.

[0924] An "emotion recognition engine" is a system that analyzes the emotions expressed by a user when they input information and acquires the user's emotional data.

[0925] "Cloud storage" refers to an internet storage service used to store generated manga data and provide an accessible URL.

[0926] "URL" refers to a link indicating the location of the manga data on cloud storage, which is used by the user to download the generated manga.

[0927] The present invention is a system that comprises a means for a user to input an outline of a manga, a means for a generation AI to generate a detailed story, a means for automatically generating the layout of each page and panel, a means for automatically generating pictures for each panel, a means for composing the final manga and providing it to the user, a means for analyzing the user's emotions using an emotion recognition engine, a means for adjusting the content of the generated story and pictures based on the emotions, and a means for saving the generated manga in cloud storage and providing a URL from which the user can download it.

[0928] System Configuration

[0929] The system is implemented using the following hardware and software.

[0930] Hardware:

[0931] Smartphone: Used by users to access the system, input synopsis and download generated manga.

[0932] Server: A cloud-based server (e.g., AWS, GCP) runs the generative AI model and emotion recognition engine.

[0933] software:

[0934] Generative AI model: Automatically generates storyline, layout, and illustrations based on a user-supplied outline.

[0935] Emotion recognition engine: Analyzes the emotions of the user when they input and provides feedback to the generative AI.

[0936] Cloud Storage: Store generated comics and manage download links.

[0937] Python: Responsible for data processing and calculations, and libraries used for API calls and image processing (Requests, PIL, etc.).

[0938] Specific examples of processing

[0939] Overview input and emotion recognition

[0940] The user inputs a summary of the manga using the system's smartphone application. For example,

[0941] "The protagonist, a high school student, is reincarnated into another world and becomes active at a magic academy."

[0942] At this time, the emotion recognition engine analyzes the user's emotion and acquires the corresponding emotion data. As a result of the analysis, emotion data such as "it looks fun" is acquired.

[0943] Storyline generation

[0944] The server receives the summary and emotion data sent by the user and passes it to the generative AI model as a prompt.

[0945] "The story is about a high school student protagonist who is reincarnated into another world and becomes active at a magic academy. The emotions recognized by the emotion engine seem fun."

[0946] This will generate a detailed storyline, for example the following scene structure:

[0947] Scene 1: The scene where the protagonist gets excited and is reincarnated into another world.

[0948] Scene 2: The exciting first lesson at the Magic Academy

[0949] Scene 3: Meeting a fun-loving friend

[0950] Layout and picture generation

[0951] The generative AI model determines the layout of each page and panel based on the generated storyline. After each scene and panel layout is determined, the illustrations for each panel are automatically generated. For example:

[0952] Scene 1, Frame 1: The protagonist is thrown into another world with a smile on his face.

[0953] Scene 2, Frame 2: A vast and fun view of the magic academy

[0954] Scene 3, Frame 3: The protagonist chats happily with a new friend

[0955] Preserving and providing manga

[0956] The server stores the generated manga in cloud storage and provides a URL for users to download. Using this download link, users can view and share the generated manga.

[0957] In this way, the present invention provides a system that allows users to easily generate high-quality, emotionally-reflective cartoons.

[0958] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0959] Step 1:

[0960] A user uses a smartphone application to input and submit a summary of a manga. For example, they might input, "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy." At this time, the summary text data is sent to the system. The user's face is also analyzed by a camera, and an emotion recognition engine obtains the user's emotional data. For example, it may recognize that the manga is "fun."

[0961] Step 2:

[0962] The server receives the summary text and emotion data sent by the user. It performs an initial error check on the received text data to ensure that it is in the correct format and content. If there are no errors, it prepares it to be passed as a prompt to the generative AI model along with the emotion data (e.g., "Looks fun").

[0963] Step 3:

[0964] The server passes the summary text and sentiment data to a generative AI model, which generates a detailed storyline using the following prompt:

[0965] "The story is about a high school student protagonist who is reincarnated into another world and becomes active at a magic academy. The emotions recognized by the emotion engine seem fun."

[0966] The generative AI model creates a detailed storyline based on these prompts, outputting story data including, for example, specific events and character actions for each scene.

[0967] Step 4:

[0968] The server receives the story data output from the generative AI model and then generates the layout of each page and frame. Based on the generated story development, the generative AI model automatically generates the frame division, size, and placement of each scene. For example, it outputs layout data such as Scene 1 consisting of four frames, Scene 2 consisting of six frames, etc.

[0969] Step 5:

[0970] The server again uses the generative AI model to generate the image for each frame based on the layout data. The image for each frame includes visual elements such as the character's facial expression, pose, and background, and is adjusted based on the emotional data. For example, in Scene 1, Frame 1, an image of the protagonist smiling and being thrown into a different world is generated. In this way, the image for each frame is output.

[0971] Step 6:

[0972] The server then combines the generated images and story data to create the final manga. After combining, it checks the images and text for errors to ensure the final manga has been generated correctly. After the error check is complete, it saves the generated manga to cloud storage.

[0973] Step 7:

[0974] The server generates a download URL for the manga stored in the cloud storage and provides it to the user. The user can use the URL to download, view, and share the generated manga. In addition, an emotion recognition engine provides real-time emotional feedback to the user as they view the manga, and the manga can be regenerated as needed.

[0975] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0976] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0977] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0978] [Third embodiment]

[0979] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0980] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0981] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0982] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0983] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0984] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0985] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0986] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0987] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0988] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0989] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0990] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0991] System Overview

[0992] This invention is a system in which a generation AI automatically generates a detailed storyline, page and frame layouts, and the pictures for each frame based on a manga outline entered by a user, and then constructs and provides the final manga to the user. The system is configured around a server, receives user input, uses the generation AI to perform various processes, and provides the final product to the user.

[0993] Program Overview

[0994] The program of this system executes a series of processes in a specific order. Below we will explain the process in brief, with specific examples.

[0995] 1. Enter the summary

[0996] A user accesses the system and inputs a rough outline of the manga. The input information is received and stored by the server.

[0997] Example: A high school student protagonist is reincarnated into another world and becomes active at a magic academy.

[0998] 2. Storyline generation

[0999] The server passes the user-entered synopsis to a generation AI, which then generates a detailed storyline based on the synopsis and builds out the specific events that occur in each scene.

[1000] Examples:

[1001] Scene 1: The scene where the protagonist is reincarnated into another world

[1002] Scene 2: First lesson at the Magic Academy

[1003] Scene 3: Confronting a Sassy Rival

[1004] 3. Generate layout

[1005] The server automatically generates the layout of each page and frame based on the generated storyline, using a generation AI to determine the frame layout, size, and placement for each scene.

[1006] Examples:

[1007] Scene 1: 4 frames

[1008] Scene 2: 6 frames

[1009] Scene 3: Consists of 3 frames

[1010] 4. Image Generation

[1011] The server automatically generates the images for each frame based on the generated layout information. The AI ​​then draws the character's facial expressions, poses, background, and other elements.

[1012] Examples:

[1013] Scene 1, Frame 1: The protagonist is thrown into another world with a surprised expression.

[1014] Scene 2, Frame 2: The vast landscape of the magic academy

[1015] Scene 3, Frame 3: Dialogue with a new friend

[1016] 5. Creation and Delivery of Deliverables

[1017] The server combines the illustrations and story of each scene to create the final manga, which it then checks for errors and makes available to users in a downloadable format.

[1018] Examples:

[1019] Deliver the final manga in PDF format or other suitable format

[1020] Users can download, view, and print

[1021] Specific examples of work processes

[1022] Below is a concrete example of generating a manga in which the protagonist, a high school student, is reincarnated into another world and becomes active at a magic academy.

[1023] 1. The user enters the synopsis: "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy."

[1024] 2. The server receives the summary and checks for errors.

[1025] 3. The server passes the outline to the generation AI, which generates the following detailed storyline:

[1026] The scene where the main character is reincarnated into another world

[1027] Meeting new friends at the magic academy

[1028] Magical training and rivalry

[1029] 4. The server analyzes each scene and determines the frame layout (scene 1 is 4 frames, scene 2 is 6 frames, scene 3 is 3 frames).

[1030] 5. The server uses AI to generate the images for each frame (the surprised main character, the magic effects, the school background).

[1031] 6. The server combines the pictures and story, generates the final manga, and performs error checking.

[1032] 7. The server provides the final manga to the user in PDF format, which the user can download and view.

[1033] In this way, the system of the present invention significantly reduces the effort required to create manga and efficiently provides high-quality manga.

[1034] The processing flow will be explained below.

[1035] Step 1:

[1036] A user accesses the system and enters a rough outline of a manga. The user enters a text summary of the story, character settings, and major events into the input fields of a web form or application, and then presses the submit button.

[1037] Step 2:

[1038] The server receives the summary sent by the user and saves the text data. It performs an initial error check on the saved text data to ensure that the correct format and content are entered.

[1039] Step 3:

[1040] The server analyzes the summary text and passes it to the generation AI, which then generates a detailed storyline based on the summary, including specific events and character actions in each scene.

[1041] Step 4:

[1042] The server automatically generates the layout of each page and frame based on the generated storyline. The AI ​​determines the frame layout, size, and placement for each scene.

[1043] Step 5:

[1044] The server automatically generates the images for each frame based on the generated layout information, using a generative AI to depict visual elements such as character expressions, poses, and backgrounds.

[1045] Step 6:

[1046] The server combines the illustrations of each panel with the storyline to create the final manga. It then performs error checks to ensure that each page is generated correctly and that there are no problems with the images or text.

[1047] Step 7:

[1048] The server converts the final comic into a downloadable format (e.g. PDF) and provides it to the user, who clicks on the download link to retrieve the generated comic.

[1049] Step 8:

[1050] The user can view the downloaded manga and, if necessary, make edits or print it.

[1051] Example 1

[1052] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1053] Traditional manga production requires a great deal of time and effort, and many manual processes result in low productivity. In particular, there is a lot of creative work involved, such as developing the storyline, setting the layout of pages and panels, and creating the illustrations, which can lead to variations in quality. There is a need for a system that can solve these problems and automatically generate high-quality manga efficiently.

[1054] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1055] In this invention, the server includes: means for a user to input a summary of a manga; means for a generative AI model to generate a detailed storyline based on the input summary; means for automatically generating a layout for each page and frame based on the generated storyline; means for automatically generating pictures for each frame based on the generated layout; means for combining the pictures for each frame and the storyline to create a final manga and perform error checking; and means for providing the final manga to the user. This makes it possible to automatically generate and quickly provide a high-quality manga efficiently based on the summary input by the user.

[1056] "User" refers to a person who uses the system to input a manga outline and receive the final manga.

[1057] A "manga summary" refers to information that indicates the general content and structure of the story.

[1058] A "generative AI model" refers to an artificial intelligence model that generates storylines and images based on input information.

[1059] A "detailed storyline" refers to a progression of the story that includes specific scenes and events generated based on the outline.

[1060] "Page and panel layout" refers to the composition information such as the panel layout, size, and placement for each page of a manga.

[1061] "Artwork in each panel" refers to the characters, background, and other visual elements that appear in each panel of a manga.

[1062] "Error checking" refers to the process of checking the consistency and quality of the generated manga and detecting errors.

[1063] "PDF format" is an abbreviation for Portable Document Format and refers to one of the formats for electronic documents.

[1064] "Means for providing to the user" refers to a method for providing the final generated manga in a form that is accessible to the user.

[1065] System Overview

[1066] This invention is a system in which a generative AI model automatically generates a detailed storyline, page and frame layouts, and the pictures for each frame based on a manga outline entered by a user, composing the final manga and providing it to the user. The system is centered around a server, which receives user input, uses the generative AI model to perform various processes, and provides the final product to the user.

[1067] Program Overview

[1068] The system executes a series of steps in a specific order to generate a manga based on a user-supplied synopsis, as detailed below.

[1069] Enter a summary

[1070] A user accesses the system through a terminal and inputs a rough outline of the manga. The terminal sends the input information to the server, which then stores it in a database.

[1071] Examples:

[1072] The user enters a summary such as "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy."

[1073] Storyline generation

[1074] The server then passes the saved summary data to a generative AI model, which uses this summary to generate a detailed storyline and build specific events for each scene. This model uses OpenAI's GPT-4 and other AI models.

[1075] Examples:

[1076] Scene 1: The scene where the protagonist is reincarnated into another world

[1077] Scene 2: First lesson at the Magic Academy

[1078] Scene 3: A confrontation with a nasty rival

[1079] Generate layout

[1080] The server automatically generates the layout of each page and frame based on the generated storyline. It uses a generative AI model to determine the frame layout, size, and placement for each scene. OpenAI's DALL-E and other tools are used here.

[1081] Examples:

[1082] Scene 1: 4 frames

[1083] Scene 2: 6 frames

[1084] Scene 3: Consists of 3 frames

[1085] Picture generation

[1086] The server automatically generates the images for each frame based on the generated layout information. It uses a generative AI model to draw the character's facial expressions, poses, background, etc. DALL-E and Stable Diffusion are used here.

[1087] Examples:

[1088] Scene 1, Frame 1: The protagonist is thrown into another world with a surprised expression.

[1089] Scene 2, Frame 2: The vast landscape of the magic academy

[1090] Scene 3, Frame 3: Dialogue with a new friend

[1091] Generate and deliver deliverables

[1092] The server then combines the illustrations and story of each scene to create the final manga, performs error checking, and delivers it to the user in a format that could be PDF or EPUB.

[1093] Examples:

[1094] The final manga will be provided to users in PDF format, which they can download, view, and print.

[1095] Example prompt

[1096] Below is a specific example of a prompt sentence to be passed to the generative AI model.

[1097] Storyline generation prompt:

[1098] "Generate a detailed storyline based on the outline of a high school student protagonist who is reincarnated into another world and becomes active at a magic academy."

[1099] Layout generation prompt:

[1100] "Generate a page and frame layout where Scene 1 is 4 frames, Scene 2 is 6 frames, and Scene 3 is 3 frames."

[1101] Picture generation prompt:

[1102] "In Scene 1, Frame 1, please draw the main character with a surprised expression as he is thrown into another world."

[1103] The system of the present invention efficiently and automatically generates manga from an outline entered by a user based on the above procedures and elements, enabling high-quality and rapid provision.

[1104] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1105] System program processing flow

[1106] Step 1: Enter a summary

[1107] The user accesses the system through a terminal and inputs a story outline, including themes and major events of the manga.

[1108] Input: Manga summary (text)

[1109] Output: Summary data (sent to server)

[1110] Specific operation: The user accesses the input form in a web browser and enters a summary such as "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy." The device detects a click on the "Submit" button and sends the input data to the server as a POST request. The server receives the POST request and saves the entered summary data in a database.

[1111] Step 2: Analyze the summary and check for errors

[1112] The server parses the received summary data and performs some initial error checking to ensure there are no typos or missing information.

[1113] Input: Summary data (fetched from database)

[1114] Output: Verified summary data

[1115] What it does: The server fetches summary data from the database and uses a text analysis module to detect grammatical errors and missing information. If errors are found, it sends feedback to the user to prompt them to make corrections.

[1116] Step 3: Generate a storyline

[1117] The server passes the error-free summary data to a generative AI model (e.g., GPT-4), which generates a detailed storyline based on the prompt.

[1118] Input: Validated summary data

[1119] Output: Detailed storyline

[1120] Specific operation: The server generates a prompt and sends an API request to the generative AI model saying, "Please generate a detailed storyline based on the outline that the high school protagonist is reincarnated in another world and becomes active at a magic academy." The generative AI model generates a detailed storyline and returns it to the server. The server temporarily stores the received storyline data.

[1121] Step 4: Generate the layout

[1122] The server automatically generates the layout of each page and frame based on the generated storyline. It uses a generative AI model (e.g., DALL-E) to determine the frame layout, size, and placement for each scene.

[1123] Input: Detailed storyline

[1124] Output: Page and frame layout

[1125] Specific operation: The server analyzes the story development data and sends an API request to the generation AI model with the prompt "Please generate a page and frame layout where Scene 1 is 4 frames, Scene 2 is 6 frames, and Scene 3 is 3 frames." The generation AI model generates the page and frame layout and returns it to the server. The server temporarily stores the generated layout information.

[1126] Step 5: Generate the image

[1127] The server automatically generates the images for each frame based on the generated layout information, using a generative AI model (such as DALL-E or Stable Diffusion) to draw the character's facial expressions, poses, background, and other details.

[1128] Input: Page and frame layout

[1129] Output: Picture of each frame

[1130] Specific operation: The server fetches layout information, generates a prompt for each frame: "In Scene 1, Frame 1, please draw the main character with a surprised expression as he is thrown into another world," and sends an API request to the generative AI model. The generative AI model generates an image for each frame and returns it to the server. The server temporarily stores the received image data.

[1131] Step 6: Generate and deliver artifacts

[1132] The server combines the illustrations of each scene with the story to create the final manga, which is then verified and checked for errors before being delivered to the user.

[1133] Input: Pictures of each frame and detailed storyline

[1134] Output: Finished manga (PDF format, etc.)

[1135] Specific operation: The server integrates the saved storyline data and illustration data, performs error checks and quality inspections (checking page layout and illustration consistency), converts the final manga into PDF or other formats, and provides a download link to the user. The user clicks the download link on their device to obtain the work.

[1136] In this way, this system uses a series of automated processes to efficiently generate and quickly deliver high-quality manga based on the outline entered by the user.

[1137] (Application example 1)

[1138] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1139] Traditional manga production requires a lot of time and effort, and there is no way to automate the entire process, from story development to layout and illustration creation. This makes it difficult for individual creators or small teams to quickly produce high-quality manga. There is also a lack of an effective platform for sharing and evaluating the manga. Furthermore, there are challenges with error checking and providing the manga in digital format.

[1140] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1141] In this invention, the server includes: a means for a user to input a summary of a manga; a means for a generation AI to generate a detailed storyline based on the input summary; a means for automatically generating a layout for each page and frame based on the generated storyline; a means for automatically generating pictures for each frame based on the generated layout; a means for composing the final manga and providing it to the user; a means for uploading the generated manga to a content platform and sharing it with other users; and a means for other users to rate and comment on the generated manga. This allows users to easily generate high-quality manga, share it with other users, and receive ratings. Error checking and provision in electronic format can also be efficiently performed.

[1142] To match the application example, we will create definitions of the important words included in the patent claims in the following format.

[1143] A "user" is an individual or entity that uses the system to input a comic outline and receive the generated comic.

[1144] "Generative AI" is an artificial intelligence technology that automatically generates detailed story development, page and panel layouts, and pictures for each panel from an input outline.

[1145] A "storyline" is a sequence of specific events and scenes that the generative AI generates based on a summary entered by the user.

[1146] "Layout" refers to the structure including the placement and size of each page and frame, and the relationships between frames.

[1147] "Pictures" refer to visual elements such as the character's facial expression, pose, and background drawn in each panel.

[1148] The "content platform" is an online system that allows users to share their manga with other users and receive ratings.

[1149] "Sharing" is the act of sharing the created manga with other users.

[1150] "Evaluation" is the act of other users providing feedback on the generated comic.

[1151] "Error checking" is the process of checking the user-entered synopsis and generated content for defects and errors.

[1152] "Electronic format" refers to the method of providing the final manga in a digital data format such as PDF.

[1153] MODE FOR CARRYING OUT THE INVENTION

[1154] This invention relates to a system in which a generation AI automatically generates a detailed storyline, page and frame layouts, and frame art based on a manga outline entered by a user, and then constructs the final manga and provides it to the user. Specific embodiments for carrying out this invention are described below.

[1155] System configuration

[1156] The system of this invention consists of a terminal used by a user, a server that runs a generative AI model, and a content platform.

[1157] Device: A mobile device such as a smartphone or tablet is used. Users input the outline of the manga through this device.

[1158] Server: A generative AI model is installed on a cloud server that analyzes the outline, generates detailed story development, determines the layout, and generates the images for each frame.

[1159] Content Platform: The manga will be uploaded to a content platform where other users can view and rate it.

[1160] Program processing

[1161] The server proceeds with the following steps:

[1162] 1. Enter summary and check for errors:

[1163] The user inputs a summary of the manga using a terminal, and the server receives this summary and performs some initial error checking.

[1164] Example: The prompt is "The high school student protagonist is reincarnated into another world and becomes active at a magic academy."

[1165] 2. Storyline generation:

[1166] The server passes the received synopsis to a generative AI model to generate a detailed storyline.

[1167] Examples:

[1168] Scene 1: The scene where the protagonist is reincarnated into another world

[1169] Scene 2: First lesson at the Magic Academy

[1170] Scene 3: Confronting a Sassy Rival

[1171] 3. Generate layout:

[1172] The layout of each page and panel is automatically generated based on the story development. Panel layout, size, and placement are determined.

[1173] Examples:

[1174] Scene 1: 4 frames

[1175] Scene 2: 6 frames

[1176] Scene 3: Consists of 3 frames

[1177] 4. Picture generation:

[1178] Based on the generated layout, the images for each frame are generated, including the character's facial expressions, poses, background, etc.

[1179] Examples:

[1180] Scene 1, Frame 1: The protagonist is thrown into another world with a surprised expression.

[1181] Scene 2, Frame 2: The vast landscape of the magic academy

[1182] Scene 3, Frame 3: Dialogue with a new friend

[1183] 5. Composition and presentation of the manga:

[1184] The illustrations for each scene are combined with the story to create the final manga, which is then checked for errors. The finished manga is then provided in electronic format.

[1185] Examples:

[1186] The final manga is provided to users in PDF format, which they can download and view.

[1187] Prototyping:

[1188] This system allows users to easily generate high-quality manga, share them with other users, and receive feedback. The server utilizes a generative AI model to efficiently analyze, generate, and organize data. Specifically, OpenAI's generative AI model and cloud server technology are used to achieve a highly accurate generation process in real time.

[1189] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1190] Program processing steps

[1191] Step 1:

[1192] A user inputs a summary of a manga using a terminal. The input summary (prompt sentence) is received by the server. The server performs an initial error check on this summary.

[1193] Input: The prompt entered by the user (e.g., "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy.")

[1194] Data processing: Check for errors to ensure the summary is entered correctly.

[1195] Output: Prompt statement that passes error checking

[1196] Step 2:

[1197] The server passes the prompt sentences that pass error checks to a generative AI model, which then generates a detailed storyline.

[1198] Input: A prompt that passes error checking

[1199] Data computation: Generative AI models are used to generate detailed storylines (determine the specific events for each scene).

[1200] Output: A detailed storyline generated

[1201] Step 3:

[1202] The server automatically generates the layout of each page and frame based on the generated story development.

[1203] Input: A detailed generated storyline

[1204] Data calculation: An algorithm is used to automatically generate the layout (panel division, size, and placement) of each page and panel.

[1205] Output: The layout of each generated page and frame

[1206] Step 4:

[1207] The server automatically generates pictures for each frame based on the generated layout information.

[1208] Input: The layout of each generated page and frame

[1209] Data calculation: Uses generative AI models to draw character expressions, poses, backgrounds, etc.

[1210] Output: Picture of each frame

[1211] Step 5:

[1212] The server combines the pictures of each scene with the story to create the final manga and performs error checks.

[1213] Input: Pictures of each frame and detailed storyline

[1214] Data processing: Combining the data and checking for errors in the final manga.

[1215] Output: Final cartoon

[1216] Step 6:

[1217] The server uploads the completed manga to a content platform where other users can view, rate, and comment on it.

[1218] Input: Final cartoon

[1219] Data Calculation: Processing the upload and applying publishing settings on the content platform.

[1220] Output: Manga published on a content platform

[1221] Step 7:

[1222] Other users view the comics created on the content platform and provide ratings and comments.

[1223] Input: Manga published on content platforms

[1224] Data calculation: Performs the process of accepting ratings and comments from other users.

[1225] Output: Manga with ratings and comments

[1226] In this way, a system has been realized in which users, servers, and content platforms work together to efficiently generate, share, and evaluate high-quality manga.

[1227] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1228] System Overview

[1229] This invention is a system in which a generation AI automatically generates a detailed storyline, page and panel layout, and illustrations for each panel based on a manga outline entered by the user, composing the final manga and providing it to the user. Furthermore, by combining it with an emotion engine, the system is characterized by recognizing the user's emotions and adjusting the content of the story and illustrations accordingly. The system is centered around a server, which receives user input, utilizes the generation AI and emotion engine to perform various processes, and provides the final product to the user.

[1230] Program Overview

[1231] The program of this system executes a series of processes in a specific order. Below we will explain the process in brief, with specific examples.

[1232] 1. Summary input and emotion recognition

[1233] A user accesses the system and inputs a rough outline of a manga. The user enters a summary of the story, character settings, and major events in text format into the input fields of a web form or application, and then presses the submit button. At the same time, the emotion engine analyzes the user's emotions as they type.

[1234] Example: When the user inputs a summary of the story in which the protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy, the emotion engine recognizes that the user is smiling happily.

[1235] 2. Receiving and analyzing the summary

[1236] The server receives the summary sent by the user and saves the text data. It performs an initial error check on the saved text data to ensure that the format and content are correct. At the same time, it saves the emotion data analyzed by the emotion engine.

[1237] 3. Storyline generation

[1238] The server analyzes the summary text and emotion data and passes it to the generation AI, which then generates a detailed storyline based on the summary and the user's emotions. This storyline includes specific events and character actions in each scene, and the content is adjusted based on the emotion data.

[1239] Examples:

[1240] Scene 1: The scene where the protagonist gets excited and is reincarnated into another world.

[1241] Scene 2: The exciting first lesson at the Magic Academy

[1242] Scene 3: Meeting a fun-loving friend

[1243] 4. Generate layout

[1244] The server automatically generates the layout of each page and frame based on the generated storyline. The AI ​​determines the frame layout, size, and placement for each scene.

[1245] Examples:

[1246] Scene 1: Consists of four frames (each expressing excitement)

[1247] Scene 2: 6 frames (emphasizes the fun of class)

[1248] Scene 3: Consists of 3 frames (depicting a friendly conversation with a friend)

[1249] 5. Image Generation

[1250] The server automatically generates the images for each frame based on the generated layout information, using a generative AI to depict visual elements such as character expressions, poses, and backgrounds based on emotional data.

[1251] Examples:

[1252] Scene 1, Frame 1: The protagonist is thrown into another world with a smile on his face.

[1253] Scene 2, Frame 2: A vast and fun view of the magic academy

[1254] Scene 3, Frame 3: The protagonist chats happily with a new friend

[1255] 6. Creation and Delivery of Deliverables

[1256] The server combines the illustrations and story of each scene to create the final manga. It then performs error checks to ensure that each page is generated correctly and that there are no problems with the images or text.

[1257] 7. Providing Results

[1258] The server converts the final comic into a downloadable format (e.g. PDF) and provides it to the user, who clicks on the download link to retrieve the generated comic.

[1259] 8. Emotional feedback during execution

[1260] Even while a user is browsing a manga, the emotion engine can analyze the user's emotions in real time, and the generation AI can fine-tune the content based on the emotions.

[1261] Specific examples of work processes

[1262] Below is a concrete example of generating a manga in which the protagonist, a high school student, is reincarnated into another world and becomes active at a magic academy.

[1263] 1. The user enters and submits a summary: "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy." At the same time, the emotion engine recognizes a happy, smiling face.

[1264] 2. The server receives the summary and checks for errors.

[1265] 3. The server passes the synopsis to the generation AI and emotion engine, which generates the following detailed storyline:

[1266] A scene where the main character gets excited and is reincarnated into another world

[1267] Meet exciting new friends at the Magic Academy

[1268] 4. The server analyzes each scene and determines the frame layout (scene 1 is 4 frames, scene 2 is 6 frames, scene 3 is 3 frames).

[1269] 5. The server uses AI to generate pictures for each frame (a smiling protagonist and scenes of fun school life).

[1270] 6. The server combines the pictures and story, generates the final manga, and performs error checking.

[1271] 7. The server provides the final manga to the user in PDF format, which the user can download and view.

[1272] 8. Users can view the manga and the emotion engine can provide real-time feedback and regenerate it as needed.

[1273] In this way, the system of the present invention, which is combined with an emotion engine, can efficiently provide high-quality manga that reflects the user's emotions.

[1274] The processing flow will be explained below.

[1275] Step 1:

[1276] A user accesses the system and inputs a rough outline of a manga. The user enters a summary of the story, character settings, and major events in text format into the input fields of a web form or application, and then presses the submit button. At the same time, the emotion engine analyzes the user's facial expressions and voice to recognize the emotion they are expressing as they type.

[1277] Step 2:

[1278] The server receives the summary sent by the user and saves the text data. At the same time, the emotion data analyzed by the emotion engine is also sent to the server and saved. An initial error check is performed on the saved text data to ensure that the correct format and content have been entered.

[1279] Step 3:

[1280] The server analyzes the summary text and emotion data and passes it to the generation AI, which then generates a detailed storyline based on the summary and the user's emotions. This storyline includes specific events and character actions in each scene, and the content is adjusted based on the emotion data.

[1281] Step 4:

[1282] The server automatically generates the layout of each page and frame based on the generated storyline. The AI ​​determines the frame layout, size, and placement for each scene.

[1283] Step 5:

[1284] The server automatically generates the images for each frame based on the generated layout information, using a generative AI to depict visual elements such as character expressions, poses, and backgrounds based on emotional data.

[1285] Step 6:

[1286] The server combines the generated pictures of each panel with the storyline to create the final manga. It then performs error checks to ensure that each page is generated correctly and that there are no problems with the images or text.

[1287] Step 7:

[1288] The server converts the final comic into a downloadable format (e.g. PDF) and provides it to the user, who clicks on the download link to retrieve the generated comic.

[1289] Step 8:

[1290] As users browse the manga, the emotion engine analyzes their emotions in real time. Depending on their emotions, the generative AI readjusts the content and updates the storyline and frame art until the user is satisfied.

[1291] Example 2

[1292] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1293] Modern manga production requires a lot of time and effort, creating an efficient automatic generation system. However, existing automatic generation systems only generate stories and layouts based on user input, and are unable to adjust content to reflect the user's emotions. Therefore, a system that can recognize the user's emotions and adjust content based on them is needed. Furthermore, they lack error checking capabilities at each stage of story generation, panel layout, image generation, and delivery of the final product, which can result in incomplete manga.

[1294] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1295] In this invention, the server includes: a means for a user to input a summary of a manga; an emotion recognition means for analyzing the user's emotions at the time of input; a generation AI for generating a detailed storyline based on the input summary and emotion data; a means for automatically generating a layout for each page and frame based on the generated storyline; a means for automatically generating pictures for each frame based on the generated layout; and a means for constructing and providing the final manga to the user. This allows for efficient automatic generation of high-quality manga that reflects the user's emotions and allows for fine-tuning of the content in real time in response to the user's emotions. Furthermore, error checking at each step allows for a highly refined final product.

[1296] "User" refers to a person who accesses the system and inputs a summary of a manga.

[1297] "Emotion recognition means" refers to technology that analyzes the user's emotions while the user is entering a summary.

[1298] "Generative AI" refers to artificial intelligence that generates detailed storylines based on a summary and emotional data entered by the user.

[1299] "Story development" refers to the detailed narrative progression generated by the generative AI, including specific events and character actions in each scene.

[1300] "Layout" refers to the design that determines the placement and size of each page and panel based on the generated story development.

[1301] "Automatic picture generation means" refers to a technology that automatically generates pictures for each frame based on the generated layout information.

[1302] "Error checking" refers to the process of checking whether generated data or final products are fraudulent or incorrect.

[1303] "Final product" refers to the finished manga that is created by integrating the generated story, layout, and illustrations.

[1304] "Delivery means" refers to the technology by which the final product is delivered to users in a downloadable format.

[1305] "Real-time feedback" refers to the process in which the emotion engine analyzes the user's emotions in real time while they are reading the manga, and the generative AI fine-tunes the content.

[1306] System Overview

[1307] This invention is a system in which a generation AI automatically generates a detailed storyline, page and panel layout, and the illustrations for each panel based on a manga outline entered by the user, and provides the final manga to the user. This system is characterized by recognizing the user's emotions by combining it with an emotion engine, and adjusting the content of the story and illustrations accordingly. The system is configured around a server, which receives user input, uses the generation AI and emotion engine to perform various processes, and provides the final product to the user.

[1308] Hardware and software used

[1309] This system uses the following hardware and software:

[1310] Server: The central location for receiving, storing, analyzing, and generating data. This is where the emotion engine and generative AI models run.

[1311] Generative AI model: Generates detailed storylines and frame illustrations based on user input and emotional data.

[1312] Emotion Engine: Recognizes user emotions in real time and provides emotional data.

[1313] User interface: A web form or application for users to enter a synopsis of their manga.

[1314] Program processing

[1315] 1. Summary input and emotion recognition

[1316] Users enter a summary of their manga through a web form or application, and the emotion engine analyzes their facial expressions and tone to capture emotional data.

[1317] Example: A user enters the summary "The protagonist is a high school student who is reincarnated in another world and becomes active at a magic academy," and the emotion engine recognizes that the user is smiling happily.

[1318] 2. Receiving and analyzing the summary

[1319] The server receives the summary sent by the user and saves the text data. It performs an initial error check on the saved text data to ensure that it is in the correct format and content. At the same time, it saves the emotion data obtained from the emotion engine.

[1320] 3. Storyline generation

[1321] The server analyzes the summary text and emotional data and passes it to the generation AI, which then generates a detailed storyline based on the summary and emotional data, including specific events and character actions in each scene, and adjusts the content based on the emotional data.

[1322] Examples:

[1323] Scene 1: The scene where the protagonist gets excited and is reincarnated into another world.

[1324] Scene 2: The exciting first lesson at the magic academy

[1325] Scene 3: Meeting a fun-loving friend

[1326] 4. Generate layout

[1327] The server automatically generates the layout of each page and frame based on the storyline. The AI ​​determines the frame layout, size, and placement for each scene.

[1328] Examples:

[1329] Scene 1: 4-frame composition (expresses excitement)

[1330] Scene 2: 6 frames (emphasizes the fun of class)

[1331] Scene 3: Consists of 3 frames (depicting a friendly conversation with a friend)

[1332] 5. Image Generation

[1333] The server automatically generates the images for each frame based on the generated layout information. The AI ​​then depicts visual elements such as character expressions, poses, and backgrounds based on emotional data.

[1334] Examples:

[1335] Scene 1, Frame 1: The protagonist is thrown into another world with a smile on his face.

[1336] Scene 2, Frame 2: A vast and fun view of the magic academy

[1337] Scene 3, Frame 3: The protagonist chats happily with a new friend

[1338] 6. Creation and Delivery of Deliverables

[1339] The server combines the pictures and story of each scene to create the final manga. It checks each page to ensure it is generated correctly and that there are no problems with the images or text. It converts the final manga into a downloadable format (e.g., PDF) and provides it to the user. The user clicks the download link to obtain the generated manga.

[1340] 7. Emotional feedback during execution

[1341] While the user is viewing the manga, the emotion engine analyzes the user's emotions in real time and provides feedback to the generation AI, allowing the content to be fine-tuned.

[1342] Specific examples of work processes

[1343] Below is a concrete example of generating a manga in which the protagonist, a high school student, is reincarnated into another world and becomes active at a magic academy.

[1344] Prompt statement:

[1345] 1. The user enters a summary such as "The protagonist is a high school student who is reincarnated in another world and becomes active at a magic academy" and submits it. The emotion engine recognizes a happy, smiling face.

[1346] 2. The server receives the summary and performs error checking.

[1347] 3. The server passes the synopsis to the generation AI and emotion engine, which generates the following detailed storyline:

[1348] Scene 1: The scene where the protagonist gets excited and is reincarnated into another world.

[1349] Scene 2: Meeting exciting new friends at the Magic Academy

[1350] 4. The server analyzes each scene and determines the frame layout (scene 1 is 4 frames, scene 2 is 6 frames, scene 3 is 3 frames).

[1351] 5. The server uses AI to generate pictures for each frame (a smiling protagonist and scenes of fun school life).

[1352] 6. The server combines the pictures and story, generates the final manga, and performs error checking.

[1353] 7. The server provides the final manga to the user in PDF format, which the user can download and view.

[1354] 8. Users can view the manga and the emotion engine can provide real-time feedback and regenerate it as needed.

[1355] As a result, the system of the present invention can efficiently provide high-quality manga that reflects the user's emotions.

[1356] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1357] Step 1:

[1358] The user enters a summary of the manga through a web form or application. The user enters a summary of the story, character settings, and key events in text format and presses the submit button. This entered text data becomes the input. At the same time, the emotion engine analyzes the user's facial expressions and tone to obtain emotion data (data based on facial expressions and tone). This becomes the output of emotion recognition data.

[1359] Specific operation: The user enters "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy" into the text field and presses the send button. The emotion engine analyzes the user's happy facial expression and tone of voice via the camera and microphone, and obtains the emotion data of a "happy smile."

[1360] Step 2:

[1361] The server receives the summary text data and emotion recognition data sent by the user and stores them in a database. An initial error check is performed on the received text data to ensure that the format and content are correct. The results of this error check are output. At the same time, the acquired emotion data is also stored in the database.

[1362] Specific operation: The server receives and saves the text data "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy." It then checks the text for consistency using an error checking tool. It also saves the emotion data "a happy smile."

[1363] Step 3:

[1364] The server analyzes the stored text data and emotion data and passes it to the generation AI. The analyzed data is the output as a result of this analysis. The generation AI generates a detailed storyline based on the analyzed data it receives. This generated storyline is the output.

[1365] Specific operation: The generation AI takes into account the emotional data of a "happy smile" and generates the following story development.

[1366] Scene 1: The scene where the protagonist gets excited and is reincarnated into another world.

[1367] Scene 2: The exciting first lesson at the magic academy

[1368] Scene 3: Meeting a fun-loving friend

[1369] Step 4:

[1370] The server receives the generated storyline data and generates the layout for each page and frame. The layout data is the output of this layout generation process. The generation AI determines the frame layout, size, and placement for each scene.

[1371] Specific operation: The server determines the following frame layout based on the story development.

[1372] Scene 1: 4-frame composition (expresses excitement)

[1373] Scene 2: 6 frames (emphasizes the fun of class)

[1374] Scene 3: Consists of 3 frames (depicting a friendly conversation with a friend)

[1375] Step 5:

[1376] The server automatically generates the images for each frame based on the generated layout data. The output of this process is image data for each frame. The generation AI depicts visual elements such as character expressions, poses, and backgrounds based on the emotional data.

[1377] Specific operation: The generation AI generates the following image.

[1378] Scene 1, Frame 1: The protagonist is thrown into another world with a smile on his face.

[1379] Scene 2, Frame 2: A vast and fun view of the magic academy

[1380] Scene 3, Frame 3: The protagonist chats happily with a new friend

[1381] Step 6:

[1382] The server integrates the illustrations of each scene with the storyline to create the final manga. The output of this integration process is the final manga data. An error check is performed on the final product. The results of the error check are also output.

[1383] What it does: Ensures that all pages are generated correctly and checks for missing images and text errors.

[1384] Step 7:

[1385] The server converts the final manga into a downloadable format such as PDF. The result of this conversion process is a downloadable file. The user clicks on the provided download link to get the final manga.

[1386] Specific operation: The server converts the generated manga into a PDF and displays a download link in an email or on a web page. The user clicks the link to download the PDF.

[1387] Step 8:

[1388] While the user is browsing the manga, the emotion engine analyzes the user's emotions in real time and provides feedback to the generation AI. As a result of this feedback process, adjustment instruction data is output, which allows the generation AI to fine-tune the content and improve user satisfaction.

[1389] Specific operation: The emotion engine analyzes the user's emotions through the camera and microphone, and instructs the generation AI to regenerate or adjust as necessary.

[1390] (Application example 2)

[1391] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1392] In the past, creating high-quality manga based on users' ideas required drawing skills and storytelling ability, which required a great deal of time and effort. It was also difficult to create manga that reflected the user's emotions, and there was no easy way to create a work that expressed one's own feelings and thoughts. For this reason, many people gave up on turning their ideas into manga.

[1393] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input a summary of a manga; a means for a generation AI to generate a detailed story development based on the input summary; a means for automatically generating a layout for each page and frame based on the generated story development; a means for automatically generating pictures for each frame based on the generated layout; a means for composing the final manga and providing it to the user; a means for analyzing the user's emotions at the time of input using an emotion recognition engine; a means for adjusting the content of the generated story and pictures based on the user's emotions; and a means for saving the generated manga in cloud storage and providing a URL from which the user can download it. This allows users to easily generate high-quality manga that reflect their own emotions and instantly download and share them.

[1394] "User" refers to an individual or organization that uses the system to create manga.

[1395] A "manga synopsis" is a brief description of the manga's story, character settings, major events, etc. that the user enters into the system.

[1396] "Generative AI" is an artificial intelligence that automatically generates detailed story development, page and panel layouts, and pictures for each panel based on an outline entered by the user.

[1397] A "detailed storyline" is a story that includes specific scenes and character actions, which the generation AI generates from the outline entered by the user.

[1398] "Layout" refers to the design of each page and panel, size, placement, etc., which is determined based on the story development generated by the generation AI.

[1399] An "emotion recognition engine" is a system that analyzes the emotions expressed by a user when they input information and acquires the user's emotional data.

[1400] "Cloud storage" refers to an internet storage service used to store generated manga data and provide an accessible URL.

[1401] "URL" refers to a link indicating the location of the manga data on cloud storage, which is used by the user to download the generated manga.

[1402] The present invention is a system that comprises a means for a user to input an outline of a manga, a means for a generation AI to generate a detailed story, a means for automatically generating the layout of each page and panel, a means for automatically generating pictures for each panel, a means for composing the final manga and providing it to the user, a means for analyzing the user's emotions using an emotion recognition engine, a means for adjusting the content of the generated story and pictures based on the emotions, and a means for saving the generated manga in cloud storage and providing a URL from which the user can download it.

[1403] System Configuration

[1404] The system is implemented using the following hardware and software.

[1405] Hardware:

[1406] Smartphone: Used by users to access the system, input synopsis and download generated manga.

[1407] Server: A cloud-based server (e.g., AWS, GCP) runs the generative AI model and emotion recognition engine.

[1408] software:

[1409] Generative AI model: Automatically generates storyline, layout, and illustrations based on a user-supplied outline.

[1410] Emotion recognition engine: Analyzes the emotions of the user when they input and provides feedback to the generative AI.

[1411] Cloud Storage: Store generated comics and manage download links.

[1412] Python: Responsible for data processing and calculations, and libraries used for API calls and image processing (Requests, PIL, etc.).

[1413] Specific examples of processing

[1414] Overview input and emotion recognition

[1415] The user inputs a summary of the manga using the system's smartphone application. For example,

[1416] "The protagonist, a high school student, is reincarnated into another world and becomes active at a magic academy."

[1417] At this time, the emotion recognition engine analyzes the user's emotion and acquires the corresponding emotion data. As a result of the analysis, emotion data such as "it looks fun" is acquired.

[1418] Storyline generation

[1419] The server receives the summary and emotion data sent by the user and passes it to the generative AI model as a prompt.

[1420] "The story is about a high school student protagonist who is reincarnated into another world and becomes active at a magic academy. The emotions recognized by the emotion engine seem fun."

[1421] This will generate a detailed storyline, for example the following scene structure:

[1422] Scene 1: The scene where the protagonist gets excited and is reincarnated into another world.

[1423] Scene 2: The exciting first lesson at the Magic Academy

[1424] Scene 3: Meeting a fun-loving friend

[1425] Layout and picture generation

[1426] The generative AI model determines the layout of each page and panel based on the generated storyline. After each scene and panel layout is determined, the illustrations for each panel are automatically generated. For example:

[1427] Scene 1, Frame 1: The protagonist is thrown into another world with a smile on his face.

[1428] Scene 2, Frame 2: A vast and fun view of the magic academy

[1429] Scene 3, Frame 3: The protagonist chats happily with a new friend

[1430] Preserving and providing manga

[1431] The server stores the generated manga in cloud storage and provides a URL for users to download. Using this download link, users can view and share the generated manga.

[1432] In this way, the present invention provides a system that allows users to easily generate high-quality, emotionally-reflective cartoons.

[1433] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1434] Step 1:

[1435] A user uses a smartphone application to input and submit a summary of a manga. For example, they might input, "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy." At this time, the summary text data is sent to the system. The user's face is also analyzed by a camera, and an emotion recognition engine obtains the user's emotional data. For example, it may recognize that the manga is "fun."

[1436] Step 2:

[1437] The server receives the summary text and emotion data sent by the user. It performs an initial error check on the received text data to ensure that it is in the correct format and content. If there are no errors, it prepares it to be passed as a prompt to the generative AI model along with the emotion data (e.g., "Looks fun").

[1438] Step 3:

[1439] The server passes the summary text and sentiment data to a generative AI model, which generates a detailed storyline using the following prompt:

[1440] "The story is about a high school student protagonist who is reincarnated into another world and becomes active at a magic academy. The emotions recognized by the emotion engine seem fun."

[1441] The generative AI model creates a detailed storyline based on these prompts, outputting story data including, for example, specific events and character actions for each scene.

[1442] Step 4:

[1443] The server receives the story data output from the generative AI model and then generates the layout of each page and frame. Based on the generated story development, the generative AI model automatically generates the frame division, size, and placement of each scene. For example, it outputs layout data such as Scene 1 consisting of four frames, Scene 2 consisting of six frames, etc.

[1444] Step 5:

[1445] The server again uses the generative AI model to generate the image for each frame based on the layout data. The image for each frame includes visual elements such as the character's facial expression, pose, and background, and is adjusted based on the emotional data. For example, in Scene 1, Frame 1, an image of the protagonist smiling and being thrown into a different world is generated. In this way, the image for each frame is output.

[1446] Step 6:

[1447] The server then combines the generated images and story data to create the final manga. After combining, it checks the images and text for errors to ensure the final manga has been generated correctly. After the error check is complete, it saves the generated manga to cloud storage.

[1448] Step 7:

[1449] The server generates a download URL for the manga stored in the cloud storage and provides it to the user. The user can use the URL to download, view, and share the generated manga. In addition, an emotion recognition engine provides real-time emotional feedback to the user as they view the manga, and the manga can be regenerated as needed.

[1450] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1451] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1452] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1453] [Fourth embodiment]

[1454] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1455] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1456] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1457] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1458] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1459] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1460] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1461] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1462] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1463] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1464] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1465] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1466] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1467] System Overview

[1468] This invention is a system in which a generation AI automatically generates a detailed storyline, page and frame layouts, and the pictures for each frame based on a manga outline entered by a user, and then constructs and provides the final manga to the user. The system is configured around a server, receives user input, uses the generation AI to perform various processes, and provides the final product to the user.

[1469] Program Overview

[1470] The program of this system executes a series of processes in a specific order. Below we will explain the process in brief, with specific examples.

[1471] 1. Enter the summary

[1472] A user accesses the system and inputs a rough outline of the manga. The input information is received and stored by the server.

[1473] Example: A high school student protagonist is reincarnated into another world and becomes active at a magic academy.

[1474] 2. Storyline generation

[1475] The server passes the user-entered synopsis to a generation AI, which then generates a detailed storyline based on the synopsis and builds out the specific events that occur in each scene.

[1476] Examples:

[1477] Scene 1: The scene where the protagonist is reincarnated into another world

[1478] Scene 2: First lesson at the Magic Academy

[1479] Scene 3: Confronting a Sassy Rival

[1480] 3. Generate layout

[1481] The server automatically generates the layout of each page and frame based on the generated storyline, using a generation AI to determine the frame layout, size, and placement for each scene.

[1482] Examples:

[1483] Scene 1: 4 frames

[1484] Scene 2: 6 frames

[1485] Scene 3: Consists of 3 frames

[1486] 4. Image Generation

[1487] The server automatically generates the images for each frame based on the generated layout information. The AI ​​then draws the character's facial expressions, poses, background, and other elements.

[1488] Examples:

[1489] Scene 1, Frame 1: The protagonist is thrown into another world with a surprised expression.

[1490] Scene 2, Frame 2: The vast landscape of the magic academy

[1491] Scene 3, Frame 3: Dialogue with a new friend

[1492] 5. Creation and Delivery of Deliverables

[1493] The server combines the illustrations and story of each scene to create the final manga, which it then checks for errors and makes available to users in a downloadable format.

[1494] Examples:

[1495] Deliver the final manga in PDF format or other suitable format

[1496] Users can download, view, and print

[1497] Specific examples of work processes

[1498] Below is a concrete example of generating a manga in which the protagonist, a high school student, is reincarnated into another world and becomes active at a magic academy.

[1499] 1. The user enters the synopsis: "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy."

[1500] 2. The server receives the summary and checks for errors.

[1501] 3. The server passes the outline to the generation AI, which generates the following detailed storyline:

[1502] The scene where the main character is reincarnated into another world

[1503] Meeting new friends at the magic academy

[1504] Magical training and rivalry

[1505] 4. The server analyzes each scene and determines the frame layout (scene 1 is 4 frames, scene 2 is 6 frames, scene 3 is 3 frames).

[1506] 5. The server uses AI to generate the images for each frame (the surprised main character, the magic effects, the school background).

[1507] 6. The server combines the pictures and story, generates the final manga, and performs error checking.

[1508] 7. The server provides the final manga to the user in PDF format, which the user can download and view.

[1509] In this way, the system of the present invention significantly reduces the effort required to create manga and efficiently provides high-quality manga.

[1510] The processing flow will be explained below.

[1511] Step 1:

[1512] A user accesses the system and enters a rough outline of a manga. The user enters a text summary of the story, character settings, and major events into the input fields of a web form or application, and then presses the submit button.

[1513] Step 2:

[1514] The server receives the summary sent by the user and saves the text data. It performs an initial error check on the saved text data to ensure that the correct format and content are entered.

[1515] Step 3:

[1516] The server analyzes the summary text and passes it to the generation AI, which then generates a detailed storyline based on the summary, including specific events and character actions in each scene.

[1517] Step 4:

[1518] The server automatically generates the layout of each page and frame based on the generated storyline. The AI ​​determines the frame layout, size, and placement for each scene.

[1519] Step 5:

[1520] The server automatically generates the images for each frame based on the generated layout information, using a generative AI to depict visual elements such as character expressions, poses, and backgrounds.

[1521] Step 6:

[1522] The server combines the illustrations of each panel with the storyline to create the final manga. It then performs error checks to ensure that each page is generated correctly and that there are no problems with the images or text.

[1523] Step 7:

[1524] The server converts the final comic into a downloadable format (e.g. PDF) and provides it to the user, who clicks on the download link to retrieve the generated comic.

[1525] Step 8:

[1526] The user can view the downloaded manga and, if necessary, make edits or print it.

[1527] Example 1

[1528] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1529] Traditional manga production requires a great deal of time and effort, and many manual processes result in low productivity. In particular, there is a lot of creative work involved, such as developing the storyline, setting the layout of pages and panels, and creating the illustrations, which can lead to variations in quality. There is a need for a system that can solve these problems and automatically generate high-quality manga efficiently.

[1530] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1531] In this invention, the server includes: means for a user to input a summary of a manga; means for a generative AI model to generate a detailed storyline based on the input summary; means for automatically generating a layout for each page and frame based on the generated storyline; means for automatically generating pictures for each frame based on the generated layout; means for combining the pictures for each frame and the storyline to create a final manga and perform error checking; and means for providing the final manga to the user. This makes it possible to automatically generate and quickly provide a high-quality manga efficiently based on the summary input by the user.

[1532] "User" refers to a person who uses the system to input a manga outline and receive the final manga.

[1533] A "manga summary" refers to information that indicates the general content and structure of the story.

[1534] A "generative AI model" refers to an artificial intelligence model that generates storylines and images based on input information.

[1535] A "detailed storyline" refers to a progression of the story that includes specific scenes and events generated based on the outline.

[1536] "Page and panel layout" refers to the composition information such as the panel layout, size, and placement for each page of a manga.

[1537] "Artwork in each panel" refers to the characters, background, and other visual elements that appear in each panel of a manga.

[1538] "Error checking" refers to the process of checking the consistency and quality of the generated manga and detecting errors.

[1539] "PDF format" is an abbreviation for Portable Document Format and refers to one of the formats for electronic documents.

[1540] "Means for providing to the user" refers to a method for providing the final generated manga in a form that is accessible to the user.

[1541] System Overview

[1542] This invention is a system in which a generative AI model automatically generates a detailed storyline, page and frame layouts, and the pictures for each frame based on a manga outline entered by a user, composing the final manga and providing it to the user. The system is centered around a server, which receives user input, uses the generative AI model to perform various processes, and provides the final product to the user.

[1543] Program Overview

[1544] The system executes a series of steps in a specific order to generate a manga based on a user-supplied synopsis, as detailed below.

[1545] Enter a summary

[1546] A user accesses the system through a terminal and inputs a rough outline of the manga. The terminal sends the input information to the server, which then stores it in a database.

[1547] Examples:

[1548] The user enters a summary such as "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy."

[1549] Storyline generation

[1550] The server then passes the saved summary data to a generative AI model, which uses this summary to generate a detailed storyline and build specific events for each scene. This model uses OpenAI's GPT-4 and other AI models.

[1551] Examples:

[1552] Scene 1: The scene where the protagonist is reincarnated into another world

[1553] Scene 2: First lesson at the Magic Academy

[1554] Scene 3: A confrontation with a nasty rival

[1555] Generate layout

[1556] The server automatically generates the layout of each page and frame based on the generated storyline. It uses a generative AI model to determine the frame layout, size, and placement for each scene. OpenAI's DALL-E and other tools are used here.

[1557] Examples:

[1558] Scene 1: 4 frames

[1559] Scene 2: 6 frames

[1560] Scene 3: Consists of 3 frames

[1561] Picture generation

[1562] The server automatically generates the images for each frame based on the generated layout information. It uses a generative AI model to draw the character's facial expressions, poses, background, etc. DALL-E and Stable Diffusion are used here.

[1563] Examples:

[1564] Scene 1, Frame 1: The protagonist is thrown into another world with a surprised expression.

[1565] Scene 2, Frame 2: The vast landscape of the magic academy

[1566] Scene 3, Frame 3: Dialogue with a new friend

[1567] Generate and deliver deliverables

[1568] The server then combines the illustrations and story of each scene to create the final manga, performs error checking, and delivers it to the user in a format that could be PDF or EPUB.

[1569] Examples:

[1570] The final manga will be provided to users in PDF format, which they can download, view, and print.

[1571] Example prompt

[1572] Below is a specific example of a prompt sentence to be passed to the generative AI model.

[1573] Storyline generation prompt:

[1574] "Generate a detailed storyline based on the outline of a high school student protagonist who is reincarnated into another world and becomes active at a magic academy."

[1575] Layout generation prompt:

[1576] "Generate a page and frame layout where Scene 1 is 4 frames, Scene 2 is 6 frames, and Scene 3 is 3 frames."

[1577] Picture generation prompt:

[1578] "In Scene 1, Frame 1, please draw the main character with a surprised expression as he is thrown into another world."

[1579] The system of the present invention efficiently and automatically generates manga from an outline entered by a user based on the above procedures and elements, enabling high-quality and rapid provision.

[1580] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1581] System program processing flow

[1582] Step 1: Enter a summary

[1583] The user accesses the system through a terminal and inputs a story outline, including themes and major events of the manga.

[1584] Input: Manga summary (text)

[1585] Output: Summary data (sent to server)

[1586] Specific operation: The user accesses the input form in a web browser and enters a summary such as "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy." The device detects a click on the "Submit" button and sends the input data to the server as a POST request. The server receives the POST request and saves the entered summary data in a database.

[1587] Step 2: Analyze the summary and check for errors

[1588] The server parses the received summary data and performs some initial error checking to ensure there are no typos or missing information.

[1589] Input: Summary data (fetched from database)

[1590] Output: Verified summary data

[1591] What it does: The server fetches summary data from the database and uses a text analysis module to detect grammatical errors and missing information. If errors are found, it sends feedback to the user to prompt them to make corrections.

[1592] Step 3: Generate a storyline

[1593] The server passes the error-free summary data to a generative AI model (e.g., GPT-4), which generates a detailed storyline based on the prompt.

[1594] Input: Validated summary data

[1595] Output: Detailed storyline

[1596] Specific operation: The server generates a prompt and sends an API request to the generative AI model saying, "Please generate a detailed storyline based on the outline that the high school protagonist is reincarnated in another world and becomes active at a magic academy." The generative AI model generates a detailed storyline and returns it to the server. The server temporarily stores the received storyline data.

[1597] Step 4: Generate the layout

[1598] The server automatically generates the layout of each page and frame based on the generated storyline. It uses a generative AI model (e.g., DALL-E) to determine the frame layout, size, and placement for each scene.

[1599] Input: Detailed storyline

[1600] Output: Page and frame layout

[1601] Specific operation: The server analyzes the story development data and sends an API request to the generation AI model with the prompt "Please generate a page and frame layout where Scene 1 is 4 frames, Scene 2 is 6 frames, and Scene 3 is 3 frames." The generation AI model generates the page and frame layout and returns it to the server. The server temporarily stores the generated layout information.

[1602] Step 5: Generate the image

[1603] The server automatically generates the images for each frame based on the generated layout information, using a generative AI model (such as DALL-E or Stable Diffusion) to draw the character's facial expressions, poses, background, and other details.

[1604] Input: Page and frame layout

[1605] Output: Picture of each frame

[1606] Specific operation: The server fetches layout information, generates a prompt for each frame: "In Scene 1, Frame 1, please draw the main character with a surprised expression as he is thrown into another world," and sends an API request to the generative AI model. The generative AI model generates an image for each frame and returns it to the server. The server temporarily stores the received image data.

[1607] Step 6: Generate and deliver artifacts

[1608] The server combines the illustrations of each scene with the story to create the final manga, which is then verified and checked for errors before being delivered to the user.

[1609] Input: Pictures of each frame and detailed storyline

[1610] Output: Finished manga (PDF format, etc.)

[1611] Specific operation: The server integrates the saved storyline data and illustration data, performs error checks and quality inspections (checking page layout and illustration consistency), converts the final manga into PDF or other formats, and provides a download link to the user. The user clicks the download link on their device to obtain the work.

[1612] In this way, this system uses a series of automated processes to efficiently generate and quickly deliver high-quality manga based on the outline entered by the user.

[1613] (Application example 1)

[1614] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1615] Traditional manga production requires a lot of time and effort, and there is no way to automate the entire process, from story development to layout and illustration creation. This makes it difficult for individual creators or small teams to quickly produce high-quality manga. There is also a lack of an effective platform for sharing and evaluating the manga. Furthermore, there are challenges with error checking and providing the manga in digital format.

[1616] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1617] In this invention, the server includes: a means for a user to input a summary of a manga; a means for a generation AI to generate a detailed storyline based on the input summary; a means for automatically generating a layout for each page and frame based on the generated storyline; a means for automatically generating pictures for each frame based on the generated layout; a means for composing the final manga and providing it to the user; a means for uploading the generated manga to a content platform and sharing it with other users; and a means for other users to rate and comment on the generated manga. This allows users to easily generate high-quality manga, share it with other users, and receive ratings. Error checking and provision in electronic format can also be efficiently performed.

[1618] To match the application example, we will create definitions of the important words included in the patent claims in the following format.

[1619] A "user" is an individual or entity that uses the system to input a comic outline and receive the generated comic.

[1620] "Generative AI" is an artificial intelligence technology that automatically generates detailed story development, page and panel layouts, and pictures for each panel from an input outline.

[1621] A "storyline" is a sequence of specific events and scenes that the generative AI generates based on a summary entered by the user.

[1622] "Layout" refers to the structure including the placement and size of each page and frame, and the relationships between frames.

[1623] "Pictures" refer to visual elements such as the character's facial expression, pose, and background drawn in each panel.

[1624] The "content platform" is an online system that allows users to share their manga with other users and receive ratings.

[1625] "Sharing" is the act of sharing the created manga with other users.

[1626] "Evaluation" is the act of other users providing feedback on the generated comic.

[1627] "Error checking" is the process of checking the user-entered synopsis and generated content for defects and errors.

[1628] "Electronic format" refers to the method of providing the final manga in a digital data format such as PDF.

[1629] MODE FOR CARRYING OUT THE INVENTION

[1630] This invention relates to a system in which a generation AI automatically generates a detailed storyline, page and frame layouts, and frame art based on a manga outline entered by a user, and then constructs the final manga and provides it to the user. Specific embodiments for carrying out this invention are described below.

[1631] System configuration

[1632] The system of this invention consists of a terminal used by a user, a server that runs a generative AI model, and a content platform.

[1633] Device: A mobile device such as a smartphone or tablet is used. Users input the outline of the manga through this device.

[1634] Server: A generative AI model is installed on a cloud server that analyzes the outline, generates detailed story development, determines the layout, and generates the images for each frame.

[1635] Content Platform: The manga will be uploaded to a content platform where other users can view and rate it.

[1636] Program processing

[1637] The server proceeds with the following steps:

[1638] 1. Enter summary and check for errors:

[1639] The user inputs a summary of the manga using a terminal, and the server receives this summary and performs some initial error checking.

[1640] Example: The prompt is "The high school student protagonist is reincarnated into another world and becomes active at a magic academy."

[1641] 2. Storyline generation:

[1642] The server passes the received synopsis to a generative AI model to generate a detailed storyline.

[1643] Examples:

[1644] Scene 1: The scene where the protagonist is reincarnated into another world

[1645] Scene 2: First lesson at the Magic Academy

[1646] Scene 3: Confronting a Sassy Rival

[1647] 3. Generate layout:

[1648] The layout of each page and panel is automatically generated based on the story development. Panel layout, size, and placement are determined.

[1649] Examples:

[1650] Scene 1: 4 frames

[1651] Scene 2: 6 frames

[1652] Scene 3: Consists of 3 frames

[1653] 4. Picture generation:

[1654] Based on the generated layout, the images for each frame are generated, including the character's facial expressions, poses, background, etc.

[1655] Examples:

[1656] Scene 1, Frame 1: The protagonist is thrown into another world with a surprised expression.

[1657] Scene 2, Frame 2: The vast landscape of the magic academy

[1658] Scene 3, Frame 3: Dialogue with a new friend

[1659] 5. Composition and presentation of the manga:

[1660] The illustrations for each scene are combined with the story to create the final manga, which is then checked for errors. The finished manga is then provided in electronic format.

[1661] Examples:

[1662] The final manga is provided to users in PDF format, which they can download and view.

[1663] Prototyping:

[1664] This system allows users to easily generate high-quality manga, share them with other users, and receive feedback. The server utilizes a generative AI model to efficiently analyze, generate, and organize data. Specifically, OpenAI's generative AI model and cloud server technology are used to achieve a highly accurate generation process in real time.

[1665] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1666] Program processing steps

[1667] Step 1:

[1668] A user inputs a summary of a manga using a terminal. The input summary (prompt sentence) is received by the server. The server performs an initial error check on this summary.

[1669] Input: The prompt entered by the user (e.g., "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy.")

[1670] Data processing: Check for errors to ensure the summary is entered correctly.

[1671] Output: Prompt statement that passes error checking

[1672] Step 2:

[1673] The server passes the prompt sentences that pass error checks to a generative AI model, which then generates a detailed storyline.

[1674] Input: A prompt that passes error checking

[1675] Data computation: Generative AI models are used to generate detailed storylines (determine the specific events for each scene).

[1676] Output: A detailed storyline generated

[1677] Step 3:

[1678] The server automatically generates the layout of each page and frame based on the generated story development.

[1679] Input: A detailed generated storyline

[1680] Data calculation: An algorithm is used to automatically generate the layout (panel division, size, and placement) of each page and panel.

[1681] Output: The layout of each generated page and frame

[1682] Step 4:

[1683] The server automatically generates pictures for each frame based on the generated layout information.

[1684] Input: The layout of each generated page and frame

[1685] Data calculation: Uses generative AI models to draw character expressions, poses, backgrounds, etc.

[1686] Output: Picture of each frame

[1687] Step 5:

[1688] The server combines the pictures of each scene with the story to create the final manga and performs error checks.

[1689] Input: Pictures of each frame and detailed storyline

[1690] Data processing: Combining the data and checking for errors in the final manga.

[1691] Output: Final cartoon

[1692] Step 6:

[1693] The server uploads the completed manga to a content platform where other users can view, rate, and comment on it.

[1694] Input: Final cartoon

[1695] Data Calculation: Processing the upload and applying publishing settings on the content platform.

[1696] Output: Manga published on a content platform

[1697] Step 7:

[1698] Other users view the comics created on the content platform and provide ratings and comments.

[1699] Input: Manga published on content platforms

[1700] Data calculation: Performs the process of accepting ratings and comments from other users.

[1701] Output: Manga with ratings and comments

[1702] In this way, a system has been realized in which users, servers, and content platforms work together to efficiently generate, share, and evaluate high-quality manga.

[1703] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1704] System Overview

[1705] This invention is a system in which a generation AI automatically generates a detailed storyline, page and panel layout, and illustrations for each panel based on a manga outline entered by the user, composing the final manga and providing it to the user. Furthermore, by combining it with an emotion engine, the system is characterized by recognizing the user's emotions and adjusting the content of the story and illustrations accordingly. The system is centered around a server, which receives user input, utilizes the generation AI and emotion engine to perform various processes, and provides the final product to the user.

[1706] Program Overview

[1707] The program of this system executes a series of processes in a specific order. Below we will explain the process in brief, with specific examples.

[1708] 1. Summary input and emotion recognition

[1709] A user accesses the system and inputs a rough outline of a manga. The user enters a summary of the story, character settings, and major events in text format into the input fields of a web form or application, and then presses the submit button. At the same time, the emotion engine analyzes the user's emotions as they type.

[1710] Example: When the user inputs a summary of the story in which the protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy, the emotion engine recognizes that the user is smiling happily.

[1711] 2. Receiving and analyzing the summary

[1712] The server receives the summary sent by the user and saves the text data. It performs an initial error check on the saved text data to ensure that the format and content are correct. At the same time, it saves the emotion data analyzed by the emotion engine.

[1713] 3. Storyline generation

[1714] The server analyzes the summary text and emotion data and passes it to the generation AI, which then generates a detailed storyline based on the summary and the user's emotions. This storyline includes specific events and character actions in each scene, and the content is adjusted based on the emotion data.

[1715] Examples:

[1716] Scene 1: The scene where the protagonist gets excited and is reincarnated into another world.

[1717] Scene 2: The exciting first lesson at the Magic Academy

[1718] Scene 3: Meeting a fun-loving friend

[1719] 4. Generate layout

[1720] The server automatically generates the layout of each page and frame based on the generated storyline. The AI ​​determines the frame layout, size, and placement for each scene.

[1721] Examples:

[1722] Scene 1: Consists of four frames (each expressing excitement)

[1723] Scene 2: 6 frames (emphasizes the fun of class)

[1724] Scene 3: Consists of 3 frames (depicting a friendly conversation with a friend)

[1725] 5. Image Generation

[1726] The server automatically generates the images for each frame based on the generated layout information, using a generative AI to depict visual elements such as character expressions, poses, and backgrounds based on emotional data.

[1727] Examples:

[1728] Scene 1, Frame 1: The protagonist is thrown into another world with a smile on his face.

[1729] Scene 2, Frame 2: A vast and fun view of the magic academy

[1730] Scene 3, Frame 3: The protagonist chats happily with a new friend

[1731] 6. Creation and Delivery of Deliverables

[1732] The server combines the illustrations and story of each scene to create the final manga. It then performs error checks to ensure that each page is generated correctly and that there are no problems with the images or text.

[1733] 7. Providing Results

[1734] The server converts the final comic into a downloadable format (e.g. PDF) and provides it to the user, who clicks on the download link to retrieve the generated comic.

[1735] 8. Emotional feedback during execution

[1736] Even while a user is browsing a manga, the emotion engine can analyze the user's emotions in real time, and the generation AI can fine-tune the content based on the emotions.

[1737] Specific examples of work processes

[1738] Below is a concrete example of generating a manga in which the protagonist, a high school student, is reincarnated into another world and becomes active at a magic academy.

[1739] 1. The user enters and submits a summary: "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy." At the same time, the emotion engine recognizes a happy, smiling face.

[1740] 2. The server receives the summary and checks for errors.

[1741] 3. The server passes the synopsis to the generation AI and emotion engine, which generates the following detailed storyline:

[1742] A scene where the main character gets excited and is reincarnated into another world

[1743] Meet exciting new friends at the Magic Academy

[1744] 4. The server analyzes each scene and determines the frame layout (scene 1 is 4 frames, scene 2 is 6 frames, scene 3 is 3 frames).

[1745] 5. The server uses AI to generate pictures for each frame (a smiling protagonist and scenes of fun school life).

[1746] 6. The server combines the pictures and story, generates the final manga, and performs error checking.

[1747] 7. The server provides the final manga to the user in PDF format, which the user can download and view.

[1748] 8. Users can view the manga and the emotion engine can provide real-time feedback and regenerate it as needed.

[1749] In this way, the system of the present invention, which is combined with an emotion engine, can efficiently provide high-quality manga that reflects the user's emotions.

[1750] The processing flow will be explained below.

[1751] Step 1:

[1752] A user accesses the system and inputs a rough outline of a manga. The user enters a summary of the story, character settings, and major events in text format into the input fields of a web form or application, and then presses the submit button. At the same time, the emotion engine analyzes the user's facial expressions and voice to recognize the emotion they are expressing as they type.

[1753] Step 2:

[1754] The server receives the summary sent by the user and saves the text data. At the same time, the emotion data analyzed by the emotion engine is also sent to the server and saved. An initial error check is performed on the saved text data to ensure that the correct format and content have been entered.

[1755] Step 3:

[1756] The server analyzes the summary text and emotion data and passes it to the generation AI, which then generates a detailed storyline based on the summary and the user's emotions. This storyline includes specific events and character actions in each scene, and the content is adjusted based on the emotion data.

[1757] Step 4:

[1758] The server automatically generates the layout of each page and frame based on the generated storyline. The AI ​​determines the frame layout, size, and placement for each scene.

[1759] Step 5:

[1760] The server automatically generates the images for each frame based on the generated layout information, using a generative AI to depict visual elements such as character expressions, poses, and backgrounds based on emotional data.

[1761] Step 6:

[1762] The server combines the generated pictures of each panel with the storyline to create the final manga. It then performs error checks to ensure that each page is generated correctly and that there are no problems with the images or text.

[1763] Step 7:

[1764] The server converts the final comic into a downloadable format (e.g. PDF) and provides it to the user, who clicks on the download link to retrieve the generated comic.

[1765] Step 8:

[1766] As users browse the manga, the emotion engine analyzes their emotions in real time. Depending on their emotions, the generative AI readjusts the content and updates the storyline and frame art until the user is satisfied.

[1767] Example 2

[1768] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1769] Modern manga production requires a lot of time and effort, creating an efficient automatic generation system. However, existing automatic generation systems only generate stories and layouts based on user input, and are unable to adjust content to reflect the user's emotions. Therefore, a system that can recognize the user's emotions and adjust content based on them is needed. Furthermore, they lack error checking capabilities at each stage of story generation, panel layout, image generation, and delivery of the final product, which can result in incomplete manga.

[1770] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1771] In this invention, the server includes: a means for a user to input a summary of a manga; an emotion recognition means for analyzing the user's emotions at the time of input; a generation AI for generating a detailed storyline based on the input summary and emotion data; a means for automatically generating a layout for each page and frame based on the generated storyline; a means for automatically generating pictures for each frame based on the generated layout; and a means for constructing and providing the final manga to the user. This allows for efficient automatic generation of high-quality manga that reflects the user's emotions and allows for fine-tuning of the content in real time in response to the user's emotions. Furthermore, error checking at each step allows for a highly refined final product.

[1772] "User" refers to a person who accesses the system and inputs a summary of a manga.

[1773] "Emotion recognition means" refers to technology that analyzes the user's emotions while the user is entering a summary.

[1774] "Generative AI" refers to artificial intelligence that generates detailed storylines based on a summary and emotional data entered by the user.

[1775] "Story development" refers to the detailed narrative progression generated by the generative AI, including specific events and character actions in each scene.

[1776] "Layout" refers to the design that determines the placement and size of each page and panel based on the generated story development.

[1777] "Automatic picture generation means" refers to a technology that automatically generates pictures for each frame based on the generated layout information.

[1778] "Error checking" refers to the process of checking whether generated data or final products are fraudulent or incorrect.

[1779] "Final product" refers to the finished manga that is created by integrating the generated story, layout, and illustrations.

[1780] "Delivery means" refers to the technology by which the final product is delivered to users in a downloadable format.

[1781] "Real-time feedback" refers to the process in which the emotion engine analyzes the user's emotions in real time while they are reading the manga, and the generative AI fine-tunes the content.

[1782] System Overview

[1783] This invention is a system in which a generation AI automatically generates a detailed storyline, page and panel layout, and the illustrations for each panel based on a manga outline entered by the user, and provides the final manga to the user. This system is characterized by recognizing the user's emotions by combining it with an emotion engine, and adjusting the content of the story and illustrations accordingly. The system is configured around a server, which receives user input, uses the generation AI and emotion engine to perform various processes, and provides the final product to the user.

[1784] Hardware and software used

[1785] This system uses the following hardware and software:

[1786] Server: The central location for receiving, storing, analyzing, and generating data. This is where the emotion engine and generative AI models run.

[1787] Generative AI model: Generates detailed storylines and frame illustrations based on user input and emotional data.

[1788] Emotion Engine: Recognizes user emotions in real time and provides emotional data.

[1789] User interface: A web form or application for users to enter a synopsis of their manga.

[1790] Program processing

[1791] 1. Summary input and emotion recognition

[1792] Users enter a summary of their manga through a web form or application, and the emotion engine analyzes their facial expressions and tone to capture emotional data.

[1793] Example: A user enters the summary "The protagonist is a high school student who is reincarnated in another world and becomes active at a magic academy," and the emotion engine recognizes that the user is smiling happily.

[1794] 2. Receiving and analyzing the summary

[1795] The server receives the summary sent by the user and saves the text data. It performs an initial error check on the saved text data to ensure that it is in the correct format and content. At the same time, it saves the emotion data obtained from the emotion engine.

[1796] 3. Storyline generation

[1797] The server analyzes the summary text and emotional data and passes it to the generation AI, which then generates a detailed storyline based on the summary and emotional data, including specific events and character actions in each scene, and adjusts the content based on the emotional data.

[1798] Examples:

[1799] Scene 1: The scene where the protagonist gets excited and is reincarnated into another world.

[1800] Scene 2: The exciting first lesson at the magic academy

[1801] Scene 3: Meeting a fun-loving friend

[1802] 4. Generate layout

[1803] The server automatically generates the layout of each page and frame based on the storyline. The AI ​​determines the frame layout, size, and placement for each scene.

[1804] Examples:

[1805] Scene 1: 4-frame composition (expresses excitement)

[1806] Scene 2: 6 frames (emphasizes the fun of class)

[1807] Scene 3: Consists of 3 frames (depicting a friendly conversation with a friend)

[1808] 5. Image Generation

[1809] The server automatically generates the images for each frame based on the generated layout information. The AI ​​then depicts visual elements such as character expressions, poses, and backgrounds based on emotional data.

[1810] Examples:

[1811] Scene 1, Frame 1: The protagonist is thrown into another world with a smile on his face.

[1812] Scene 2, Frame 2: A vast and fun view of the magic academy

[1813] Scene 3, Frame 3: The protagonist chats happily with a new friend

[1814] 6. Creation and Delivery of Deliverables

[1815] The server combines the pictures and story of each scene to create the final manga. It checks each page to ensure it is generated correctly and that there are no problems with the images or text. It converts the final manga into a downloadable format (e.g., PDF) and provides it to the user. The user clicks the download link to obtain the generated manga.

[1816] 7. Emotional feedback during execution

[1817] While the user is viewing the manga, the emotion engine analyzes the user's emotions in real time and provides feedback to the generation AI, allowing the content to be fine-tuned.

[1818] Specific examples of work processes

[1819] Below is a concrete example of generating a manga in which the protagonist, a high school student, is reincarnated into another world and becomes active at a magic academy.

[1820] Prompt statement:

[1821] 1. The user enters a summary such as "The protagonist is a high school student who is reincarnated in another world and becomes active at a magic academy" and submits it. The emotion engine recognizes a happy, smiling face.

[1822] 2. The server receives the summary and performs error checking.

[1823] 3. The server passes the synopsis to the generation AI and emotion engine, which generates the following detailed storyline:

[1824] Scene 1: The scene where the protagonist gets excited and is reincarnated into another world.

[1825] Scene 2: Meeting exciting new friends at the Magic Academy

[1826] 4. The server analyzes each scene and determines the frame layout (scene 1 is 4 frames, scene 2 is 6 frames, scene 3 is 3 frames).

[1827] 5. The server uses AI to generate pictures for each frame (a smiling protagonist and scenes of fun school life).

[1828] 6. The server combines the pictures and story, generates the final manga, and performs error checking.

[1829] 7. The server provides the final manga to the user in PDF format, which the user can download and view.

[1830] 8. Users can view the manga and the emotion engine can provide real-time feedback and regenerate it as needed.

[1831] As a result, the system of the present invention can efficiently provide high-quality manga that reflects the user's emotions.

[1832] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1833] Step 1:

[1834] The user enters a summary of the manga through a web form or application. The user enters a summary of the story, character settings, and key events in text format and presses the submit button. This entered text data becomes the input. At the same time, the emotion engine analyzes the user's facial expressions and tone to obtain emotion data (data based on facial expressions and tone). This becomes the output of emotion recognition data.

[1835] Specific operation: The user enters "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy" into the text field and presses the send button. The emotion engine analyzes the user's happy facial expression and tone of voice via the camera and microphone, and obtains the emotion data of a "happy smile."

[1836] Step 2:

[1837] The server receives the summary text data and emotion recognition data sent by the user and stores them in a database. An initial error check is performed on the received text data to ensure that the format and content are correct. The results of this error check are output. At the same time, the acquired emotion data is also stored in the database.

[1838] Specific operation: The server receives and saves the text data "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy." It then checks the text for consistency using an error checking tool. It also saves the emotion data "a happy smile."

[1839] Step 3:

[1840] The server analyzes the stored text data and emotion data and passes it to the generation AI. The analyzed data is the output as a result of this analysis. The generation AI generates a detailed storyline based on the analyzed data it receives. This generated storyline is the output.

[1841] Specific operation: The generation AI takes into account the emotional data of a "happy smile" and generates the following story development.

[1842] Scene 1: The scene where the protagonist gets excited and is reincarnated into another world.

[1843] Scene 2: The exciting first lesson at the magic academy

[1844] Scene 3: Meeting a fun-loving friend

[1845] Step 4:

[1846] The server receives the generated storyline data and generates the layout for each page and frame. The layout data is the output of this layout generation process. The generation AI determines the frame layout, size, and placement for each scene.

[1847] Specific operation: The server determines the following frame layout based on the story development.

[1848] Scene 1: 4-frame composition (expresses excitement)

[1849] Scene 2: 6 frames (emphasizes the fun of class)

[1850] Scene 3: Consists of 3 frames (depicting a friendly conversation with a friend)

[1851] Step 5:

[1852] The server automatically generates the images for each frame based on the generated layout data. The output of this process is image data for each frame. The generation AI depicts visual elements such as character expressions, poses, and backgrounds based on the emotional data.

[1853] Specific operation: The generation AI generates the following image.

[1854] Scene 1, Frame 1: The protagonist is thrown into another world with a smile on his face.

[1855] Scene 2, Frame 2: A vast and fun view of the magic academy

[1856] Scene 3, Frame 3: The protagonist chats happily with a new friend

[1857] Step 6:

[1858] The server integrates the illustrations of each scene with the storyline to create the final manga. The output of this integration process is the final manga data. An error check is performed on the final product. The results of the error check are also output.

[1859] What it does: Ensures that all pages are generated correctly and checks for missing images and text errors.

[1860] Step 7:

[1861] The server converts the final manga into a downloadable format such as PDF. The result of this conversion process is a downloadable file. The user clicks on the provided download link to get the final manga.

[1862] Specific operation: The server converts the generated manga into a PDF and displays a download link in an email or on a web page. The user clicks the link to download the PDF.

[1863] Step 8:

[1864] While the user is browsing the manga, the emotion engine analyzes the user's emotions in real time and provides feedback to the generation AI. As a result of this feedback process, adjustment instruction data is output, which allows the generation AI to fine-tune the content and improve user satisfaction.

[1865] Specific operation: The emotion engine analyzes the user's emotions through the camera and microphone, and instructs the generation AI to regenerate or adjust as necessary.

[1866] (Application example 2)

[1867] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1868] In the past, creating high-quality manga based on users' ideas required drawing skills and storytelling ability, which required a great deal of time and effort. It was also difficult to create manga that reflected the user's emotions, and there was no easy way to create a work that expressed one's own feelings and thoughts. For this reason, many people gave up on turning their ideas into manga.

[1869] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input a summary of a manga; a means for a generation AI to generate a detailed story development based on the input summary; a means for automatically generating a layout for each page and frame based on the generated story development; a means for automatically generating pictures for each frame based on the generated layout; a means for composing the final manga and providing it to the user; a means for analyzing the user's emotions at the time of input using an emotion recognition engine; a means for adjusting the content of the generated story and pictures based on the user's emotions; and a means for saving the generated manga in cloud storage and providing a URL from which the user can download it. This allows users to easily generate high-quality manga that reflect their own emotions and instantly download and share them.

[1870] "User" refers to an individual or organization that uses the system to create manga.

[1871] A "manga synopsis" is a brief description of the manga's story, character settings, major events, etc. that the user enters into the system.

[1872] "Generative AI" is an artificial intelligence that automatically generates detailed story development, page and panel layouts, and pictures for each panel based on an outline entered by the user.

[1873] A "detailed storyline" is a story that includes specific scenes and character actions, which the generation AI generates from the outline entered by the user.

[1874] "Layout" refers to the design of each page and panel, size, placement, etc., which is determined based on the story development generated by the generation AI.

[1875] An "emotion recognition engine" is a system that analyzes the emotions expressed by a user when they input information and acquires the user's emotional data.

[1876] "Cloud storage" refers to an internet storage service used to store generated manga data and provide an accessible URL.

[1877] "URL" refers to a link indicating the location of the manga data on cloud storage, which is used by the user to download the generated manga.

[1878] The present invention is a system that comprises a means for a user to input an outline of a manga, a means for a generation AI to generate a detailed story, a means for automatically generating the layout of each page and panel, a means for automatically generating pictures for each panel, a means for composing the final manga and providing it to the user, a means for analyzing the user's emotions using an emotion recognition engine, a means for adjusting the content of the generated story and pictures based on the emotions, and a means for saving the generated manga in cloud storage and providing a URL from which the user can download it.

[1879] System Configuration

[1880] The system is implemented using the following hardware and software.

[1881] Hardware:

[1882] Smartphone: Used by users to access the system, input synopsis and download generated manga.

[1883] Server: A cloud-based server (e.g., AWS, GCP) runs the generative AI model and emotion recognition engine.

[1884] software:

[1885] Generative AI model: Automatically generates storyline, layout, and illustrations based on a user-supplied outline.

[1886] Emotion recognition engine: Analyzes the emotions of the user when they input and provides feedback to the generative AI.

[1887] Cloud Storage: Store generated comics and manage download links.

[1888] Python: Responsible for data processing and calculations, and libraries used for API calls and image processing (Requests, PIL, etc.).

[1889] Specific examples of processing

[1890] Overview input and emotion recognition

[1891] The user inputs a summary of the manga using the system's smartphone application. For example,

[1892] "The protagonist, a high school student, is reincarnated into another world and becomes active at a magic academy."

[1893] At this time, the emotion recognition engine analyzes the user's emotion and acquires the corresponding emotion data. As a result of the analysis, emotion data such as "it looks fun" is acquired.

[1894] Storyline generation

[1895] The server receives the summary and emotion data sent by the user and passes it to the generative AI model as a prompt.

[1896] "The story is about a high school student protagonist who is reincarnated into another world and becomes active at a magic academy. The emotions recognized by the emotion engine seem fun."

[1897] This will generate a detailed storyline, for example the following scene structure:

[1898] Scene 1: The scene where the protagonist gets excited and is reincarnated into another world.

[1899] Scene 2: The exciting first lesson at the Magic Academy

[1900] Scene 3: Meeting a fun-loving friend

[1901] Layout and picture generation

[1902] The generative AI model determines the layout of each page and panel based on the generated storyline. After each scene and panel layout is determined, the illustrations for each panel are automatically generated. For example:

[1903] Scene 1, Frame 1: The protagonist is thrown into another world with a smile on his face.

[1904] Scene 2, Frame 2: A vast and fun view of the magic academy

[1905] Scene 3, Frame 3: The protagonist chats happily with a new friend

[1906] Preserving and providing manga

[1907] The server stores the generated manga in cloud storage and provides a URL for users to download. Using this download link, users can view and share the generated manga.

[1908] In this way, the present invention provides a system that allows users to easily generate high-quality, emotionally-reflective cartoons.

[1909] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1910] Step 1:

[1911] A user uses a smartphone application to input and submit a summary of a manga. For example, they might input, "The protagonist, a high school student, is reincarnated in another world and becomes active at a magic academy." At this time, the summary text data is sent to the system. The user's face is also analyzed by a camera, and an emotion recognition engine obtains the user's emotional data. For example, it may recognize that the manga is "fun."

[1912] Step 2:

[1913] The server receives the summary text and emotion data sent by the user. It performs an initial error check on the received text data to ensure that it is in the correct format and content. If there are no errors, it prepares it to be passed as a prompt to the generative AI model along with the emotion data (e.g., "Looks fun").

[1914] Step 3:

[1915] The server passes the summary text and sentiment data to a generative AI model, which generates a detailed storyline using the following prompt:

[1916] "The story is about a high school student protagonist who is reincarnated into another world and becomes active at a magic academy. The emotions recognized by the emotion engine seem fun."

[1917] The generative AI model creates a detailed storyline based on these prompts, outputting story data including, for example, specific events and character actions for each scene.

[1918] Step 4:

[1919] The server receives the story data output from the generative AI model and then generates the layout of each page and frame. Based on the generated story development, the generative AI model automatically generates the frame division, size, and placement of each scene. For example, it outputs layout data such as Scene 1 consisting of four frames, Scene 2 consisting of six frames, etc.

[1920] Step 5:

[1921] The server again uses the generative AI model to generate the image for each frame based on the layout data. The image for each frame includes visual elements such as the character's facial expression, pose, and background, and is adjusted based on the emotional data. For example, in Scene 1, Frame 1, an image of the protagonist smiling and being thrown into a different world is generated. In this way, the image for each frame is output.

[1922] Step 6:

[1923] The server then combines the generated images and story data to create the final manga. After combining, it checks the images and text for errors to ensure the final manga has been generated correctly. After the error check is complete, it saves the generated manga to cloud storage.

[1924] Step 7:

[1925] The server generates a download URL for the manga stored in the cloud storage and provides it to the user. The user can use the URL to download, view, and share the generated manga. In addition, an emotion recognition engine provides real-time emotional feedback to the user as they view the manga, and the manga can be regenerated as needed.

[1926] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1927] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1928] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1929] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1930] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1931] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1932] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1933] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1934] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1935] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1936] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1937] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1938] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1939] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1940] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1941] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1942] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1943] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1944] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1945] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1946] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1947] The following is further disclosed regarding the above embodiment.

[1948] (Claim 1)

[1949] A means for a user to input a summary of the manga;

[1950] A means for the generation AI to generate a detailed storyline based on the input synopsis;

[1951] A means for automatically generating a layout of each page and frame based on the generated story development;

[1952] A means for automatically generating pictures for each frame based on the generated layout;

[1953] A means for constructing the final manga and providing it to users;

[1954] A system including:

[1955] (Claim 2)

[1956] 10. The system of claim 1, further comprising means for parsing the user-entered synopsis and performing initial error checking.

[1957] (Claim 3)

[1958] 10. The system of claim 1, further comprising means for providing the final comic to the user in PDF format or other suitable format.

[1959] "Example 1"

[1960] (Claim 1)

[1961] A means for a user to input a summary of the manga;

[1962] a means for the generative AI model to generate a detailed storyline based on the input synopsis; and

[1963] A means for automatically generating a layout of each page and frame based on the generated story development;

[1964] A means for automatically generating pictures for each frame based on the generated layout;

[1965] A method for combining the pictures of each frame and the story development to create the final manga and check for errors.

[1966] A means for providing the final manga to the user;

[1967] A system including:

[1968] (Claim 2)

[1969] 10. The system of claim 1, further comprising means for parsing the user-entered synopsis and performing initial error checking.

[1970] (Claim 3)

[1971] 10. The system of claim 1, further comprising means for providing the final comic to the user in PDF format or other suitable format.

[1972] "Application Example 1"

[1973] We will rewrite the original patent claims based on the invention details and application examples you provide.

[1974] Claims

[1975] (Claim 1)

[1976] A means for a user to input a summary of the manga;

[1977] A means for the generation AI to generate a detailed storyline based on the input synopsis;

[1978] A means for automatically generating a layout of each page and frame based on the generated story development;

[1979] A means for automatically generating pictures for each frame based on the generated layout;

[1980] A means for constructing the final manga and providing it to users;

[1981] A means to upload the generated manga to a content platform and share it with other users;

[1982] means for other users to rate and comment on the generated comic;

[1983] A system including:

[1984] (Claim 2)

[1985] 10. The system of claim 1, further comprising means for parsing the user-entered synopsis and performing initial error checking.

[1986] (Claim 3)

[1987] 10. The system of claim 1, further comprising means for providing the final cartoon to a user in electronic form.

[1988] "Example 2: Combining Emotion Engines"

[1989] (Claim 1)

[1990] A means for a user to input a summary of the manga;

[1991] emotion recognition means for analyzing a user's emotion at the time of input;

[1992] A means for the generative AI to generate a detailed storyline based on the input summary and emotional data; and

[1993] A means for automatically generating a layout of each page and frame based on the generated story development;

[1994] A means for automatically generating pictures for each frame based on the generated layout;

[1995] A means for constructing the final manga and providing it to users;

[1996] A system including:

[1997] (Claim 2)

[1998] 10. The system of claim 1, further comprising means for parsing the user-entered synopsis and performing initial error checking.

[1999] (Claim 3)

[2000] 10. The system of claim 1, further comprising means for providing the final comic to the user in PDF format or other suitable format.

[2001] (Claim 4)

[2002] The system of claim 1, further comprising means for analyzing a user's emotions in real time while the user is viewing the manga, and for the generating AI to fine-tune the content.

[2003] "Application example 2 when combining emotion engines"

[2004] (Claim 1)

[2005] A means for a user to input a summary of the manga;

[2006] A means for the generation AI to generate a detailed storyline based on the input synopsis;

[2007] A means for automatically generating a layout of each page and frame based on the generated story development;

[2008] A means for automatically generating pictures for each frame based on the generated layout;

[2009] A means for constructing the final manga and providing it to users;

[2010] means for analyzing the emotion of a user's input using an emotion recognition engine;

[2011] A means for adjusting the content of the generated story and pictures based on the user's emotions;

[2012] A means of saving the generated manga to cloud storage and providing a URL for users to download it;

[2013] A system including:

[2014] (Claim 2)

[2015] 10. The system of claim 1, further comprising means for parsing the user-entered synopsis and performing initial error checking.

[2016] (Claim 3)

[2017] 10. The system of claim 1, further comprising means for providing the final comic to the user in PDF format or other suitable format. [Explanation of symbols]

[2018] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for a user to input a summary of the manga; A means for the generation AI to generate a detailed storyline based on the input synopsis; A means for automatically generating a layout of each page and frame based on the generated story development; A means for automatically generating pictures for each frame based on the generated layout; A means for constructing the final manga and providing it to users; A system including:

2. 10. The system of claim 1, further comprising means for parsing the user-entered synopsis and performing initial error checking.

3. 10. The system of claim 1, further comprising means for providing the final comic to the user in PDF format or other suitable format.

Images