system

Abstract

Provide a system. [Solution] A system including a scenario input means, a natural language analysis means for analyzing input scenario data, an image generation means for generating manga images based on the analyzed data, a preview means for displaying a preview of the image generated by the image generation means, a correction input means for inputting user correction instructions for the image displayed by the preview means, a regeneration means for regenerating the image based on the correction instructions input by the correction input means, a storage means for saving the final manga generated by the regeneration means, a multilingual translation means for translating the manga saved by the storage means into multiple languages, an upload means for uploading the manga translated by the multilingual translation means to a sales platform, and an additional content generation means for generating additional content for the manga uploaded by the upload means.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional manga production is time-consuming and labor-intensive, and even more resources are required for multilingual support and domestic and international sales. Furthermore, expanding popular manga into additional content (such as stamps or games) also requires significant new development resources. There is a need for a method to solve these issues, allowing users to easily create high-quality manga, support multiple languages, expand sales channels, and further expand content. [Means for solving the problem]

[0005] In order to solve the above-mentioned problems, the present invention provides the following system, which includes a scenario input means, a natural language analysis means for analyzing scenario data input from the scenario input means, an image generation means for generating manga images based on the data analyzed by the natural language analysis means, a preview means for displaying a preview of the image generated by the image generation means, a correction input means for inputting a user's correction instructions for the image displayed by the preview means, a regeneration means for regenerating the image based on the correction instructions input by the correction input means, a storage means for saving the final manga generated by the regeneration means, a multilingual translation means for translating the manga saved by the storage means into multiple languages, an upload means for uploading the manga translated by the multilingual translation means to a sales platform, and an additional content generation means for generating additional content for the manga uploaded by the upload means.

[0006] The "scenario input means" is an interface that allows the user to input the story and dialogue of the manga in text format.

[0007] The "natural language analysis means" is a module that uses natural language processing technology to analyze text data entered by the user and extract information about scenes and characters.

[0008] The "image generation means" is a technology that automatically generates illustrations of manga scenes and characters based on data analyzed by the natural language analysis means.

[0009] The "preview means" is a function that displays each page of the generated manga so that the user can check it.

[0010] The "modification input means" is an interface that allows the user to input the parts and content of modifications to be made to the generated manga.

[0011] The "regeneration means" is a function that regenerates manga scenes and characters based on the user's correction instructions.

[0012] "Storage means" refers to the function of saving the final manga data in the cloud or local storage.

[0013] The "multilingual translation means" is an engine for translating stored manga into multiple languages.

[0014] "Uploading means" refers to the function of uploading translated manga to an online sales platform.

[0015] The "additional content generation means" is a function for generating additional content such as stamps and games using characters and scenes from the manga. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention relates to a manga production system that utilizes generation AI, and an embodiment thereof will be described in detail below.

[0038] System Overview

[0039] This system uses AI technology to automatically generate manga based on a manga scenario entered by the user, and also handles multilingual support, sales development, and the generation of additional content. The system includes user, terminal, and server components.

[0040] System Component Description

[0041] 1. Scenario input method

[0042] The user uses the terminal to input a manga scenario. The scenario is input in text format. For example, the user can input a story in which the protagonist, a high school student, is summoned to another world.

[0043] 2. Natural language analysis means

[0044] The device sends the input scenario to the server, which then analyzes it using natural language processing technology. This analysis extracts detailed information about each scene and character in the story. For example, the background, character expressions, and location of the "summoning scene" can be identified.

[0045] 3. Image Generation Method

[0046] Based on the analysis results, the server uses AI technology to automatically generate each scene in the manga. During this generation process, character designs and backgrounds are drawn. For example, an otherworldly landscape is drawn as the background for the "summoning scene."

[0047] 4. Preview Method

[0048] The image data of the generated manga is sent to the terminal, and the user can check the preview. The preview is displayed in slide format, and the user can check each page of the manga in order.

[0049] 5. Correction input method

[0050] Users are presented with an interface to view a preview and input corrections and improvements, such as specific instructions like "change the character's facial expression to look more surprised."

[0051] 6. Regeneration means

[0052] The user's correction instructions are sent to the server, and the server again uses the image generation means to generate a revised version of the manga page. For example, the facial expression of the character may be changed in accordance with the user's instructions.

[0053] 7. Preservation means

[0054] Once the manga is created to the user's satisfaction, the data is saved on the device or in the cloud. For example, the completed manga can be saved in cloud storage.

[0055] 8. Multilingual Translation Tools

[0056] The saved manga is translated into multiple languages ​​using a translation engine, allowing the same manga to be published in multiple languages, for example, English, Chinese, and Spanish.

[0057] 9. Upload Method

[0058] The translated manga data is uploaded to a sales platform and a sales page is generated, for example, the manga is listed and sold in an online store.

[0059] 10. Additional Content Generation Methods

[0060] If a manga becomes popular, the server will generate additional content such as stamps and games using its characters and scenes. For example, the main character's character will be converted into a LINE stamp.

[0061] Specific examples

[0062] User A uses the scenario input means to input a scenario he or she has created, in which "the protagonist, a high school student, is summoned to another world." The device sends the scenario data to the server, which analyzes it using natural language analysis means. The image generation means then automatically generates each page of the manga, and a preview is displayed on the device. User A checks the preview and inputs corrections to the character's facial expressions. The server generates the images again and sends the corrected versions to the device, repeating this process until User A is satisfied. When the manga is finally completed, it is saved in the cloud using the saving means and translated into multiple languages ​​using the multilingual translation means. The translated manga is uploaded to a sales platform using the uploading means, and popular characters are then sold as stamps using the additional content generation means.

[0063] This allows users to easily create high-quality manga, support multiple languages, expand sales channels, and generate additional content.

[0064] The processing flow will be explained below.

[0065] Step 1:

[0066] The user inputs a scenario. The terminal displays a text area for the user to input the scenario. The user inputs the scenario into the text box and clicks the "Submit" button when complete.

[0067] Step 2:

[0068] When the send button is clicked, the device converts the scenario data into JSON format, which is then sent to the server as an API request.

[0069] Step 3:

[0070] The server analyzes the received scenario data using natural language analysis means, and extracts story scenes and character information as the analysis results.

[0071] Step 4:

[0072] The server uses image generation tools to generate each scene of the manga based on the analysis results. AI technology is used to depict characters and backgrounds, and each page is generated as image data.

[0073] Step 5:

[0074] The server sends the image data of the generated manga page to the terminal, which then decodes the received image data and displays a preview to the user.

[0075] Step 6:

[0076] The user checks the displayed preview and specifies the corrections to be made. The terminal provides an interface for corrections, and the user inputs the corrections.

[0077] Step 7:

[0078] The terminal transmits the inputted correction instructions to the server, which receives the correction instructions and generates a corrected version using the image generating means again.

[0079] Step 8:

[0080] The server sends the image data of the corrected manga page to the terminal, which then displays the preview again for the user to reconfirm.

[0081] Step 9:

[0082] The process from step 6 to step 8 is repeated until the user is satisfied. Finally, a manga that satisfies the user is generated.

[0083] Step 10:

[0084] The user clicks the Finish button, and the terminal uses the storage means to store the final manga data in cloud storage.

[0085] Step 11:

[0086] The server translates the saved manga data using a multilingual translation tool, and it is translated into multiple languages, including English, Chinese, and Spanish.

[0087] Step 12:

[0088] The server uploads the translated manga data to the sales platform using the uploading means, and a sales page is generated and published.

[0089] Step 13:

[0090] The server analyzes the sales data and generates content such as stamps and games for popular manga using additional content generation means.

[0091] This allows users to easily create manga, make it multilingual, sell it, and expand it further.

[0092] Example 1

[0093] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0094] Conventional manga production systems have difficulty automating the entire process, from generating a manga after a user inputs a scenario to multilingual translation, sales development, and the generation of additional content. In particular, they lack the functionality to regenerate manga content in real time according to user correction instructions, and multilingual support and uploading to sales platforms are labor-intensive tasks. Furthermore, generative AI models have not been fully utilized for high-quality image generation or scenario analysis.

[0095] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0096] In this invention, the server includes a scenario input means, a natural language analysis means, an image generation means, a preview means, a correction input means, a regeneration means, a storage means, a multilingual translation means, an upload means, an additional content generation means, a cloud storage means, an analysis means that uses a generative AI model, and a prompt sentence processing means. This allows a user to simply input a scenario, automatically generating a high-quality manga, and enabling real-time regeneration in response to user correction instructions, multilingual support, sales development, and the generation of additional content.

[0097] "Scenario input means" refers to a device or software that allows a user to input a manga scenario in text format.

[0098] "Natural language analysis means" refers to a device or software that analyzes input scenario data using natural language processing technology and extracts information about each scene and character in the story.

[0099] "Image generation means" refers to a device or software for generating images of each scene and character in a manga based on data analyzed by a natural language analysis means.

[0100] The "preview means" refers to a device or software that displays the generated image so that the user can check it.

[0101] The "modification input means" refers to a device or software that provides an interface for a user to input instructions for modification or improvement of a preview.

[0102] The "regeneration means" refers to a device or software that regenerates a manga image based on the correction instructions input by the correction input means.

[0103] "Storage means" refers to a device or software that records completed manga data for long-term retention.

[0104] "Multilingual translation means" refers to a device or software for translating stored manga into multiple languages.

[0105] "Uploading means" refers to the device or software used to upload translated manga to an online sales platform and generate a sales page.

[0106] "Additional content generation means" refers to a device or software for generating additional content such as stamps and games using manga characters and scenes.

[0107] "Cloud storage means" refers to a device or software that stores generated manga in cloud storage and makes it easily accessible.

[0108] "Analysis means utilizing a generative AI model" refers to a device or software that uses a generative AI model to analyze scenario data and obtain appropriate output.

[0109] "Prompt sentence processing means" refers to a device or software for generating and using prompt sentences for scenario data to perform analysis and image generation.

[0110] The present invention relates to a manga production system that utilizes generation AI, and specific embodiments will be described in detail.

[0111] System Overview

[0112] This system uses AI technology to automatically generate manga based on a manga scenario entered by the user, and also handles multilingual support, sales development, and the generation of additional content. The system includes user, terminal, and server components.

[0113] System Component Description

[0114] 1. Scenario input method

[0115] The user uses the terminal to input a manga scenario. The scenario is input in text format. For example, the user can input a story in which the protagonist, a high school student, is summoned to another world.

[0116] 2. Natural language analysis means

[0117] The device sends the input scenario to a server, which then analyzes the scenario using a generative AI model (e.g., GPT-4 (registered trademark)). This analysis extracts detailed information about each scene and character in the story. For example, the background, character expressions, and location of the "summoning scene" are identified.

[0118] 3. Image Generation Method

[0119] Based on the analysis results, the server uses AI technology (such as DALL-E or MidJourney) to automatically generate each scene in the manga. During this generation process, character designs and backgrounds are drawn. For example, an otherworldly landscape is drawn as the background for the "summoning scene."

[0120] 4. Preview Method

[0121] The image data of the generated manga is sent to the terminal, and the user can check the preview. The preview is displayed in slide format, and the user can check each page of the manga in order.

[0122] 5. Correction input method

[0123] Users are presented with an interface to view a preview and input corrections and improvements, such as specific instructions like "change the character's facial expression to look more surprised."

[0124] 6. Regeneration means

[0125] The user's modification instructions are sent from the terminal to the server, and the server uses the image generation means to generate a modified version of the manga page. For example, the facial expression of the character may be changed in response to the user's instructions.

[0126] 7. Preservation means

[0127] Once a manga that satisfies the user is finally created, the data is stored in cloud storage. For example, the completed manga is stored in cloud storage.

[0128] 8. Multilingual Translation Tools

[0129] The server translates the stored manga into multiple languages ​​using a translation engine (e.g., Google (registered trademark) Translate API). This allows the same manga to be offered in multiple languages, for example, English, Chinese, and Spanish.

[0130] 9. Upload Method

[0131] The translated manga data is uploaded to an online sales platform and a sales page is generated, for example, the manga is listed and sold in an online store.

[0132] 10. Additional Content Generation Methods

[0133] If a manga becomes popular, the server will generate additional content such as stamps and games using its characters and scenes. For example, the main character's character could be converted into a stamp.

[0134] Specific examples

[0135] For example, User A uses the scenario input means to input a scenario he or she has created, in which "the protagonist, a high school student, is summoned to another world." The device sends the scenario data to the server, which performs natural language analysis using a generative AI model. The image generation means then automatically generates each page of the manga, and a preview is displayed on the device. User A checks the preview and inputs corrections to the character's facial expressions. The server generates the images again and sends the corrected version to the device, repeating this process until User A is satisfied. Once the manga is finally completed, it is stored in the cloud using the storage means and translated into multiple languages ​​using the multilingual translation means. The translated manga is uploaded to a sales platform using the upload means, and popular characters are sold as stamps using the additional content generation means. This allows users to easily create high-quality manga, support multiple languages, expand sales channels, and generate additional content.

[0136] Prompt Sentence Examples

[0137] 1. "Describe a scene in which the protagonist is summoned to another world, using a fantasy-style landscape as the background."

[0138] 2. "Please make the character's expression look a little more surprised."

[0139] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0140] Step 1:

[0141] Scenario input

[0142] The user uses the device to input the manga scenario in text format. For example, they can input the content, "The protagonist, a high school student, is summoned to another world." The input scenario is saved as text data on the device.

[0143] Input: A textual scenario entered by the user.

[0144] Output: Scenario data saved on the device

[0145] Step 2:

[0146] Scenario transmission

[0147] The device sends the input scenario data to the server, efficiently transmitting the data using HTTP requests.

[0148] Input: Scenario data entered

[0149] Output: Scenario data sent to the server

[0150] Step 3:

[0151] Scenario Analysis

[0152] The server analyzes the received scenario using a generative AI model (e.g., GPT-4), which extracts key elements of the scenario (scenes, character traits, etc.).

[0153] Input: Scenario data sent to the server

[0154] Output: Information about each scene and character in the analyzed story

[0155] How it works: The generative AI model analyzes each part of the scenario and extracts the necessary text information through tokenization, grammatical analysis, and semantic analysis.

[0156] Step 4:

[0157] Image generation

[0158] The server uses AI technology (e.g., DALL-E and MidJourney) to automatically generate each scene in the manga based on the analysis results. For example, the background of the "summoning scene" is an otherworldly landscape.

[0159] Input: Information about each scene and character in the analyzed story

[0160] Output: Images of each scene in the generated manga

[0161] How it works: AI technology generates images based on text information and draws designs for each scene and character.

[0162] Step 5:

[0163] Preview Generation

[0164] The server compiles the generated images and creates preview data, which is formatted in a format that can be easily viewed by the user (slideshow format).

[0165] Input: Each scene image of the generated manga

[0166] Output: Preview data

[0167] Specific operation: The generated images are processed into a continuous slideshow format and converted into a format that the user can view sequentially.

[0168] Step 6:

[0169] Preview display

[0170] The terminal receives the preview data sent from the server and displays it to the user, who then checks the preview in slide format.

[0171] Input: Preview data

[0172] Output: Preview screen displayed on the device

[0173] Specific operation: The device loads the preview data and displays it in a slideshow format that is easy for the user to view.

[0174] Step 7:

[0175] Correction instruction input

[0176] Users can view the preview and input specific complaints and requests for corrections, such as "change the character's facial expression to look more surprised."

[0177] Input: User correction instructions

[0178] Output: Correction instruction data saved on the device

[0179] Specific operation: The terminal receives the user's input and stores it as correction instruction data.

[0180] Step 8:

[0181] Send correction instructions

[0182] The device sends the user's correction instructions to the server via an HTTP request.

[0183] Input: User correction instruction data

[0184] Output: Correction instruction data sent to the server

[0185] Specific operation: The terminal creates an HTTP request and sends correction instruction data to the server.

[0186] Step 9:

[0187] Regeneration Process

[0188] The server then uses the image generation means to generate a revised manga page based on the received revision instructions. For example, the facial expression of the character may be changed in response to the user's instructions.

[0189] Input: Correction instruction data sent to the server

[0190] Output: Regenerated manga scene images

[0191] Specific operation: Using AI technology, the regenerated image is recreated while reflecting correction instructions.

[0192] Step 10:

[0193] final save

[0194] The server stores the manga that the user is finally satisfied with in cloud storage.

[0195] Input: Regenerated manga scene images

[0196] Output: Manga data saved in cloud storage

[0197] Specific operation: Manga data is uploaded to a remote server through the API of a cloud storage service and stored for a long period of time.

[0198] Step 11:

[0199] Multilingual Translation

[0200] The server translates the stored manga data into multiple languages ​​using a translation engine (e.g., Google Translate API), allowing it to be expanded into multiple languages ​​such as English, Chinese, and Spanish.

[0201] Input: Manga data stored in cloud storage

[0202] Output: Manga data translated into multiple languages

[0203] What it does: The translation engine converts the text into each language and applies it to the appropriate manga text section.

[0204] Step 12:

[0205] Upload to sales platform

[0206] The server uploads the translated manga data to an online sales platform (e.g., Amazon Kindle Direct Publishing) and automatically generates a sales page.

[0207] Input: Manga data translated into multiple languages

[0208] Output: Sales page on sales platform

[0209] Specific operations: Use the sales platform's API to upload translated manga data and set up the product page.

[0210] Step 13:

[0211] Additional content generation

[0212] The server analyzes the popularity of the manga and generates additional content such as stamps and games based on popular characters and scenes.

[0213] Input: Manga data stored in cloud storage and popularity analysis results

[0214] Output: Generated additional content (e.g. stamps, games)

[0215] Specific operation: Generate stamps and simple game materials based on data on popular characters and scenes.

[0216] (Application example 1)

[0217] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0218] Traditional manga production methods have the problem of being time-consuming and labor-intensive, as much of the work is done manually. Additionally, processes such as supporting different languages, uploading to sales platforms, and generating additional content are performed separately, reducing overall work efficiency. There was a need for a method to solve these problems and achieve more efficient and faster manga production, translation, distribution, and additional content generation.

[0219] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0220] In this invention, the server includes a scenario input means, a natural language analysis means, an image generation means, a preview means, a correction input means, a regeneration means, a saving means, a multilingual translation means, an uploading means, an additional content generation means, a processing means using a generative AI model, and a prompt sentence generation means. This allows for a process that starts with a scenario input by a user, followed by automatic generation of manga pages using a generative AI model, natural language analysis, image generation, preview and correction, regeneration, saving, multilingual translation, uploading to a sales platform, and even automatic generation of additional content.

[0221] The "scenario input means" is an interface that allows the user to input a scenario in text format.

[0222] The "natural language analysis means" is a function that analyzes input scenario data and extracts detailed information about each scene and character in the story based on that content.

[0223] The "image generation means" is a technology for automatically generating each scene of a manga based on data analyzed by the natural language analysis means.

[0224] The "preview means" is a display means for allowing the user to check the image data of the generated manga.

[0225] The "correction input means" is an interface that allows the user to view the preview and input corrections or improvements.

[0226] The "regeneration means" is a function for regenerating an image based on correction instructions from the user.

[0227] The "storage means" is a function for saving the final generated manga data.

[0228] A "multilingual translation means" is an engine for translating stored manga into multiple different languages.

[0229] "Uploading means" is a function for uploading translated manga data to the sales platform.

[0230] The "additional content generating means" is a means for generating additional content related to the manga (for example, character stamps, etc.).

[0231] A "generative AI model" is an artificial intelligence model that generates manga pages based on a scenario provided by the user.

[0232] A "prompt sentence generation means" is a technology for generating and providing appropriate prompt sentences to a generative AI model.

[0233] This invention relates to a manga production system that utilizes generative AI. Users input a scenario, and AI technology automatically generates a manga, translating it, selling it, and generating additional content. The system includes user, terminal, and server components.

[0234] The main components of the system are as follows:

[0235] 1. Scenario input method

[0236] The user inputs the scenario using a smartphone or a head-mounted display. The scenario is input in text format.

[0237] 2. Natural language analysis means

[0238] The scenario data sent from the device to the server is analyzed using natural language processing engines such as OpenNLP to extract details about scenes and characters.

[0239] 3. Image Generation Method

[0240] Based on the results of natural language analysis, the server uses a generative AI model (e.g., a model based on TENSORFLOW® or GPT-3®) to generate images of each scene in the manga. The generated images are then stored in cloud storage.

[0241] 4. Preview Method

[0242] The generated manga image is sent to the device, and the user can preview it from a React Native-based UI.

[0243] 5. Correction input method

[0244] Users can view the preview and use an interface to provide corrections and improvements, which is also implemented using React Native.

[0245] 6. Regeneration means

[0246] Once the correction instructions are sent to the server, the server runs the AI ​​model again to generate the corrected image.

[0247] 7. Preservation means

[0248] Once the user is finally satisfied with the manga, it is saved in cloud storage.

[0249] 8. Multilingual Translation Tools

[0250] The saved manga can be automatically translated into multiple languages ​​using a multilingual translation tool (e.g., Google Translate API).

[0251] 9. Upload Method

[0252] The translated manga is uploaded using the sales platform's API (e.g., Amazon Kindle Publisher API).

[0253] 10. Additional Content Generation Methods

[0254] If a manga becomes popular, additional content (such as stamps or games) will be generated based on the characters and scenes, also using AI technology.

[0255] Specific examples

[0256] The user inputs the scenario "An adventurous girl explores ancient ruins." This scenario is sent from the device to the server, and natural language analysis extracts the information "adventure," "ancient ruins," and "exploring girl." The generative AI model then generates an image based on the following prompt:

[0257] Example prompts

[0258] "Scenario: An adventurous girl explores ancient ruins. The first scene shows the girl standing at the entrance to the ruins, with a green jungle behind her. In this scene, the girl has a surprised expression, and a broken stone statue can be seen in the background. Generate a manga page based on this scene."

[0259] The generated image is sent to the user as a preview, and the user can input correction instructions, such as "I want the girl's expression to look more surprised." The server then runs the generative AI model again to generate a corrected image. This process is repeated until the user is finally satisfied.

[0260] This allows users to complete the entire process from entering a scenario to creating the final manga, saving, translating, selling, and generating additional content all in one system.

[0261] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0262] Step 1:

[0263] The user inputs a scenario using a smartphone or head-mounted display. This scenario is input in text format, converted to JSON format from the device, and sent to the server. The input data is the scenario thought up by the user, and the output is the JSON-format scenario data received by the server.

[0264] Step 2:

[0265] The server analyzes the received scenario data using natural language analysis methods. A natural language processing engine such as OpenNLP is used for the analysis, and detailed information about each scene and character in the story is extracted from the scenario. The input is the scenario data received by the server, and the output is detailed information about the extracted scenes and characters. Scene classifications and character characteristics are identified through data analysis processing.

[0266] Step 3:

[0267] Based on the analyzed information, the server uses an AI model (for example, a model based on TensorFlow or GPT-3) to automatically generate images of each scene in the manga. At this time, a prompt is generated and supplied to the generative AI model. The input is the extracted scene and character details, as well as the generated prompt, and the output is image data of the generated manga. The AI ​​model generates images based on the prompt, and the images are stored in cloud storage.

[0268] Step 4:

[0269] The generated manga image is sent to the device, and the user can check the preview through a React Native-based UI. Here, the user can check the generated manga page step by step. The input is the generated manga image data, and the output is the preview screen displayed on the user's device. The preview display allows the user to examine the image in detail.

[0270] Step 5:

[0271] While viewing the preview, the user uses an interface to input corrections and improvements. This interface is also implemented using React Native. The input is the corrections specified by the user, and the output is the correction instruction data sent to the server. The specific corrections are specified by the user's interface operations.

[0272] Step 6:

[0273] When the server receives the correction instructions, it runs the AI ​​model again to generate the corrected image. The input is the correction instruction data from the user, and the output is the corrected manga image data. This causes the generative AI model to run again and generate an image according to the user's corrections.

[0274] Step 7:

[0275] The manga image that the user is finally satisfied with is saved in cloud storage. The input is the final manga image data after corrections, and the output is the manga data saved in the cloud. Cloud storage ensures that the data is saved permanently.

[0276] Step 8:

[0277] The stored manga data is translated into multiple languages ​​using a multilingual translation tool (for example, Google Translate API). The input is the manga data stored in cloud storage, and the output is the translated manga data in multiple languages. Multilingual support is achieved using an AI translation engine.

[0278] Step 9:

[0279] The translated manga is uploaded online using the sales platform's API (e.g., Amazon Kindle Publisher API). The input is the translated manga data, and the output is the manga page published on the sales platform. Uploading makes it accessible to a wide range of users.

[0280] Step 10:

[0281] If a manga becomes popular, the server generates additional content based on its characters and scenes. This process is also carried out using AI technology. The input is the saved manga data, and the output is data for additional content (e.g., character stamps). The generated additional content expands the range of manga-related products.

[0282] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0283] The present invention combines an emotion engine that recognizes the user's emotions in a manga production system that utilizes generative AI, and an embodiment of this system will be described in detail below.

[0284] System Overview

[0285] This system uses AI technology to automatically generate manga based on a manga scenario entered by the user, and also handles multilingual support, sales development, and the generation of additional content. By combining it with an emotion engine, it can detect the user's emotional state and provide a more user-friendly interface and feedback. The system includes user, terminal, and server components.

[0286] System Component Description

[0287] 1. Scenario input method

[0288] The user uses the device to input a manga scenario. The scenario is input in text format. The emotion engine analyzes the user's facial expressions and voice as they input, and can obtain their emotional status. For example, when inputting a story in which the protagonist, a high school student, is summoned to another world, the user's emotions of surprise and excitement are analyzed.

[0289] 2. Natural language analysis means

[0290] The device sends the input scenario to the server, which then analyzes it using natural language processing technology to extract detailed information about each scene and character in the story.

[0291] 3. Image Generation Method

[0292] Based on the analysis results, the server uses AI technology to automatically generate each scene of the manga. The AI ​​also adjusts the generation process based on the user's emotional status. For example, if the user is excited when inputting their emotion, the generated design will emphasize action scenes.

[0293] 4. Preview Method

[0294] The image data of the generated manga is sent to the device, where the user can check a preview. During the preview, the emotion engine analyzes the user's facial expressions and voice to obtain emotional feedback. For example, if the user smiles while looking at the preview, that emotion is fed back.

[0295] 5. Correction input method

[0296] Users are presented with an interface to preview the edit and input corrections and improvements. The emotion engine analyzes the user's emotions as they input their corrections and makes suggestions accordingly. For example, if the user has a questioning expression, the engine will suggest corrections for that part.

[0297] 6. Regeneration means

[0298] The user's correction instructions are sent to the server, which then uses the image generation means to generate a revised version of the manga page. The regeneration process also takes the emotional status into consideration.

[0299] 7. Preservation means

[0300] Once a manga that satisfies the user is finally created, the data is saved on the device or in the cloud.

[0301] 8. Multilingual Translation Tools

[0302] The saved manga is translated into multiple languages ​​using a translation engine, allowing the same manga to be published in multiple languages.

[0303] 9. Upload Method

[0304] The translated manga data is uploaded to a sales platform and a sales page is generated.

[0305] 10. Additional Content Generation Methods

[0306] If the manga proves popular, the server will generate additional content such as stamps and games using its characters and scenes.

[0307] Specific examples

[0308] User B inputs a scenario he or she has thought up, "A story about a magical girl who saves the world," using the scenario input means. The device sends the scenario data to the server, which analyzes it using natural language analysis means. The image generation means then automatically generates each page of the manga, and a preview is displayed on the device. During the preview, the emotion engine analyzes User B's facial expressions and obtains his or her emotional status. If User B expresses surprise at a scene, the scene is emphasized or suggested for revision. User B inputs correction instructions, and the server re-generates images and sends the revised version to the device. Once a manga that satisfies User B has finally been generated, it is stored in the cloud using the storage means and translated into multiple languages ​​using the multilingual translation means. The translated manga is uploaded to the sales platform using the upload means, and popular characters are then converted into stamps using the additional content generation means.

[0309] This allows users to create manga using emotional feedback, providing a more intuitive and interactive production experience. It also makes it easy to support multiple languages, expand sales channels, and generate additional content, making it a valuable system for users.

[0310] The processing flow will be explained below.

[0311] Step 1:

[0312] The user inputs a scenario. The device provides the user with a text area for scenario input and the ability to analyze the user's facial expressions and voice using an emotion engine. The user inputs the scenario into the text box and clicks the "Submit" button when complete.

[0313] Step 2:

[0314] When the device receives a click of the send button, it converts the scenario data into JSON format. At the same time, the emotion engine obtains the emotional status of the user at the time of input and sends it to the server along with the scenario data.

[0315] Step 3:

[0316] The server analyzes the received scenario data and emotional status using natural language analysis. When extracting scene and character information as analysis results, the emotional status is also taken into consideration. For example, if the user is excited, a scene will be created that reflects that excitement.

[0317] Step 4:

[0318] The server uses image generation means to generate each scene of the manga based on the analysis results. During the generation process, the character's facial expressions and background are set based on the user's emotional status. For example, action scenes can be emphasized to reflect excitement.

[0319] Step 5:

[0320] The server sends the image data of the generated manga page to the device. The device decodes the received image data and displays a preview to the user. The emotion engine analyzes the user's facial expressions and voice during the preview and sends emotional feedback to the server.

[0321] Step 6:

[0322] The user checks the displayed preview and specifies the areas to be corrected. The device provides an interface for correction, and the user inputs the corrections. The emotion engine analyzes the user's emotions when inputting the corrections and makes correction suggestions as necessary.

[0323] Step 7:

[0324] The terminal transmits the inputted correction instructions and emotion feedback to the server, which receives the correction instructions and emotion feedback and generates a corrected version using the image generation means again.

[0325] Step 8:

[0326] The server sends the image data of the revised manga page to the device. The device displays the preview again, and the user can recheck the revised version. If necessary, steps 6 to 8 are repeated.

[0327] Step 9:

[0328] The process from step 6 to step 8 is repeated until the user is satisfied. When a manga that satisfies the user is finally generated, the user clicks the done button and the terminal saves the final manga data in cloud storage using the storage means.

[0329] Step 10:

[0330] The server translates the saved manga data using a multilingual translation tool, and it is translated into multiple languages, including English, Chinese, and Spanish.

[0331] Step 11:

[0332] The server uploads the translated manga data to the sales platform using the uploading means, and a sales page is generated and published.

[0333] Step 12:

[0334] The server analyzes the sales data and generates content such as stamps and games for popular manga using additional content generation means.

[0335] Step 13:

[0336] The user uses the emotion engine to check the generated stamps and game content and provide instructions for corrections if necessary. The server then repeats the generation process, and the final content is completed.

[0337] Through these steps, users can create manga using emotional feedback and enjoy an intuitive and interactive production experience. Furthermore, the system offers great value to users, as it easily supports multiple languages, expands sales channels, and allows for the creation of additional content.

[0338] Example 2

[0339] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0340] Conventional manga production systems do not reflect the user's emotional state when the user inputs a manga scenario and checks each scene of the generated manga to give instructions for correction. This makes it difficult to generate manga that is in line with the user's intentions and emotions, and the generated manga sometimes does not meet the user's expectations. Furthermore, because the user's emotions could not be taken into account when creating multilingual content or additional content, it was difficult to increase user satisfaction.

[0341] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0342] In this invention, the server includes a scenario input means, a natural language analysis means for analyzing the scenario data input from the scenario input means and the user's emotional status, and an image generation means for generating a manga image based on the data analyzed by the natural language analysis means, thereby enabling the generation of a manga that reflects the user's emotional state.

[0343] The "scenario input means" is a means for a user to input a manga scenario in text format, and also includes a function for acquiring the user's emotional status.

[0344] The "natural language analysis means" is a means for analyzing the scenario data input from the scenario input means and the user's emotional status, and extracting each element of the story (scene and character information).

[0345] "Image generation means" refers to a means for generating manga images using a generative AI model based on data analyzed by a natural language analysis means.

[0346] The "preview means" is a means for displaying the image generated by the image generating means so that the user can check it.

[0347] The "emotion feedback acquisition means" is a means for acquiring an emotional status from the user's facial expression and voice when checking the image displayed by the preview means.

[0348] The "modification input means" is a means for providing an interface for the user to input instructions for modifying the preview image, and for suggesting modifications based on the acquired emotional status.

[0349] The "regeneration means" is a means for generating a revised version of the manga by operating the image generation means again based on the correction instructions and emotion feedback input by the correction input means.

[0350] "Storage means" refers to the means for storing the final manga, including cloud storage and storage locations on the device.

[0351] The "multilingual translation means" is a means for translating the manga stored by the storage means into multiple languages.

[0352] The "uploading means" refers to the means for uploading the translated manga to the sales platform and generating a sales page.

[0353] "Additional content generation means" refers to a means for generating additional content such as stamps and games based on the characters and scenes of manga uploaded to the sales platform.

[0354] The present invention is a system that automatically generates manga using a generative AI model, and combines it with an emotion engine that detects user emotions. Specific embodiments of the present invention will be described in detail below.

[0355] System Overview

[0356] This system uses AI technology to automatically generate manga based on a manga scenario entered by the user, and handles multilingual support, sales promotion, and the generation of additional content. It detects the user's emotional state and provides an interface that allows smooth revisions based on that feedback. The system consists of user, terminal, and server components.

[0357] System Component Description

[0358] 1. Scenario input method

[0359] The user inputs the manga scenario in text format using the device. At this time, the emotion engine analyzes the user's facial expressions and voice to obtain their emotional status. For example, when inputting a story in which the protagonist, a high school student, is summoned to another world, the user's emotions of surprise and excitement are analyzed.

[0360] 2. Natural language analysis means

[0361] The device sends the input scenario data to the server, which then analyzes the received scenario using natural language processing technology (e.g., Python's NLTK library) and extracts each element of the story (scene and character information).

[0362] 3. Image Generation Method

[0363] The server automatically generates each scene of the manga using a generative AI model (e.g., GPT-4) based on the analysis results and the user's emotional status. If the user is excited, the design is adjusted to emphasize action scenes.

[0364] 4. Preview Method

[0365] The image data of the generated manga is sent to the device, and the user can check the preview via a dedicated viewer app.

[0366] 5. Means of obtaining emotional feedback

[0367] During the preview, the device uses an emotion engine to analyze the user's facial expressions and voice to obtain emotional feedback data. For example, if the user smiles while looking at the preview, that emotion is fed back.

[0368] 6. Correction input method

[0369] Users use an interface to view the preview and input corrections and improvements. The emotion engine analyzes the user's emotions and makes appropriate suggestions for corrections. For example, if a user inputs "Make this scene more action-packed," and the emotion engine detects an excited state, it instructs the server to emphasize the action scenes.

[0370] 7. Regeneration means

[0371] The terminal sends the user's correction instructions to the server, and the server regenerates the corrected version of the manga page using the image generation means. The emotional status is also taken into account in this regeneration process.

[0372] 8. Preservation means

[0373] When a manga that satisfies the user is finally created, the data is stored in cloud storage (e.g., AWS (registered trademark) S3) or the like.

[0374] 9. Multilingual Translation Tools

[0375] The server translates the stored manga using a multilingual translation engine (e.g., DeepL API or Google Translate API), allowing the same manga to be published in multiple languages.

[0376] 10. Upload Method

[0377] The translated manga data is uploaded to a sales platform (e.g., Amazon Kindle Direct Publishing) and a sales page is generated.

[0378] 11. Additional Content Generation Methods

[0379] The server will generate additional content such as stamps and games using characters and scenes from popular manga, including the creation of LINE stamps featuring popular characters and the development of mini-games.

[0380] Specific usage examples and prompt sentence examples

[0381] User B inputs a scenario he or she has thought up, "A story about a young magical girl who saves the world," through a scenario input means. The device sends the scenario data to a server, which analyzes it using natural language analysis means. Manga image data is automatically generated using a generative AI model, and a preview is sent to the device. If User B smiles at the preview, that emotional data is sent to the server and reflected as an emphasis point in the next scene. User B inputs correction instructions, and the server generates new illustrations. The completed manga is saved in the cloud, then translated into multiple languages ​​using a multilingual translation means and uploaded to a sales platform. If the manga proves popular, LINE stamps of popular characters are created using an additional content generation means.

[0382] Prompt Sentence Examples

[0383] "Write a story about a high school boy who is summoned to another world and fights with magical powers. Emphasize the emotions of surprise and excitement."

[0384] "Generate a manga-style story about a young wizard girl who saves the world. Adjust the story based on the emotional feedback provided by the user."

[0385] This will allow users to enjoy an intuitive and interactive manga creation experience, and the system will also make it easy to support multiple languages ​​and generate additional content.

[0386] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0387] Specific flow of program processing

[0388] Below, we will explain the processing flow of the program of this system step by step, clearly indicating the input, output, and specific operation of each step.

[0389] Step 1: Scenario Input

[0390] The user inputs a scenario in text format using the scenario input interface of the terminal.

[0391] Input: User-entered scenario text

[0392] Output: Scenario text data and user's emotional status

[0393] Specific operation: When a user inputs "A story about a magical girl who saves the world," the emotion engine analyzes the user's facial expressions and voice in real time through the device's built-in camera and microphone, and obtains emotional status such as surprise or excitement.

[0394] Step 2: Scenario submission

[0395] The terminal transmits the input scenario data to the server.

[0396] Input: Scenario text data and emotional status

[0397] Output: Scenario data and emotional status sent to the server

[0398] Specific operation: The terminal converts the scenario data into JSON format and sends a POST request to the server using the HTTP protocol.

[0399] Step 3: Scenario analysis

[0400] The server analyzes the scenario data and the emotional status using natural language analysis means.

[0401] Input: Scenario data and emotional status sent to the server

[0402] Output: Detailed information and character information for each scene

[0403] Specific operation: The server uses Python's NLTK library to parse the text of the scenario and extract keywords for each element of the story (characters, scenes, etc.).

[0404] Step 4: Manga Generation

[0405] Based on the analysis results, the server uses a generative AI model to automatically generate each scene of the manga.

[0406] Input: Detailed information and emotional status of the analyzed scenario

[0407] Output: Generated manga image data

[0408] Specific operation: The server uses a generative AI model (e.g., GPT-4) to generate illustrations for each scene, emphasizing action scenes if the user is excited.

[0409] Step 5: Preview

[0410] The server transmits the generated manga image data to the terminal, and the user previews it.

[0411] Input: Manga image data

[0412] Output: Preview image displayed on the device

[0413] Specific operation: The server transfers the generated manga image data to the device, and the user previews it using a dedicated viewer app on the device.

[0414] Step 6: Obtaining emotional feedback

[0415] During the preview, the device analyzes the user's facial expressions and voice, obtains emotional feedback data, and sends it to the server.

[0416] Input: Facial expression and voice data of the user during preview

[0417] Output: Emotion feedback data

[0418] Specific operation: The device analyzes the user's facial expressions and voice in real time through the camera and microphone, and sends emotional status such as smile, surprise, or confusion to the server.

[0419] Step 7: Correction Instructions

[0420] The user inputs the corrections and their details using a dedicated interface, and the system makes correction suggestions based on the user's emotional status.

[0421] Input: User correction instructions and emotional feedback data

[0422] Output: Correction instruction data sent to the server

[0423] Specific operation: The user inputs "I want more action in this scene," and the server makes suggestions for revisions based on emotional feedback.

[0424] Step 8: Regenerate

[0425] The server regenerates the image based on the correction instructions and emotional feedback.

[0426] Input: User correction instructions and emotional feedback data

[0427] Output: Regenerated manga image data

[0428] Specific operation: The server generates a new illustration that reflects the correction instructions and creates a new manga image.

[0429] Step 9: Save

[0430] The server then stores the final manga on the device or in cloud storage.

[0431] Input: Final manga image data

[0432] Output: Data stored in cloud storage or on device

[0433] Specific operation: The final manga data is uploaded and saved to cloud storage such as AWS S3.

[0434] Step 10: Multilingual Translation

[0435] The server translates the stored manga into multiple languages.

[0436] Input: Saved manga image data

[0437] Output: Translated manga image data

[0438] Specific operation: The server uses the DeepL API or Google Translate API to translate the manga text into multiple languages.

[0439] Step 11: Upload

[0440] The server uploads the translated manga data to the sales platform.

[0441] Input: Manga data translated into multiple languages

[0442] Output: Manga data uploaded to the sales platform

[0443] Specific operation: The server uploads manga data to Amazon Kindle Direct Publishing or other sales platforms via API.

[0444] Step 12: Generate additional content

[0445] The server generates additional content using characters and scenes from popular manga.

[0446] Input: Sales data from the sales platform and manga character information

[0447] Output: Stamps and mini-games as additional content

[0448] Specific operations: The server develops LINE stamps incorporating popular characters and mini-games using Unity.

[0449] The above is the specific flow and operation of each processing step in this system.

[0450] (Application example 2)

[0451] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0452] Conventional manga production systems lacked the technology to analyze user emotions in real time and dynamically adjust the scenes displayed during preview. This made it difficult for users to create manga that accurately reflected their emotions, and there was room for improvement in the quality of manga and user satisfaction. Despite the demand for providing an interactive viewing experience, especially in physical stores, these technical challenges had not been resolved.

[0453] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0454] In this invention, the server includes a scenario input means, a natural language analysis means, an image generation means, a preview means, an emotion analysis means, a dynamic adjustment means, a correction input means, a regeneration means, a storage means, a multilingual translation means, an upload means, and an additional content generation means. This makes it possible to analyze a user's emotions in real time and dynamically adjust manga scenes based on those emotions. Furthermore, it is possible to realize an interactive manga viewing experience in a physical store and improve user satisfaction.

[0455] The "scenario input means" is a device or interface that allows a user to input a manga scenario in text or audio format.

[0456] The "natural language analysis means" is a technology for analyzing the scenario data input from the scenario input means and extracting detailed information about each scene and character in the story.

[0457] The "image generation means" is a device or program for automatically generating each scene of the manga based on the data analyzed by the natural language analysis means.

[0458] The "preview means" is a device or interface that displays the manga image generated by the image generation means to the user and allows the user to view it.

[0459] The "emotion analysis means" is a technology for analyzing the facial expressions and voice of a user viewing a manga using the preview means, and obtaining the user's emotional status in real time.

[0460] The "dynamic adjustment means" is a technique for dynamically adjusting manga scenes based on the emotion data analyzed by the emotion analysis means.

[0461] The "modification input means" is a device or interface for inputting a user's instructions for modifying the image displayed by the preview means.

[0462] The "regeneration means" is a technique for generating a revised version of the manga by operating the image generation means again based on the correction instructions and emotion data input by the correction input means.

[0463] The "storage means" is a device or program for storing the final manga generated by the reproduction means in a digital format.

[0464] The "multilingual translation means" is a technology for translating the manga stored by the storage means into multiple languages.

[0465] The "uploading means" is a technology for uploading the manga translated by the multilingual translation means to a sales platform and publishing or selling it.

[0466] The "additional content generation means" is a technology for generating additional content such as stamps and games based on characters and scenes from the manga uploaded by the upload means.

[0467] This invention relates to an interactive manga production and viewing system that utilizes generative AI models, and in particular provides technology that analyzes user emotional feedback in real time in physical stores and dynamically adjusts manga scenes.

[0468] System Overview

[0469] The system consists of the following components:

[0470] 1. Scenario input means: A device or interface that allows the user to input the manga scenario in text or voice format.

[0471] 2. Natural language analysis method: Technology that analyzes input scenario data and extracts detailed information about each scene and character in the story.

[0472] 3. Image generation method: Technology that automatically generates each scene of a manga based on analyzed data.

[0473] 4. Preview means: An interface that displays and allows users to view the generated manga image.

[0474] 5. Emotion analysis means: Technology that analyzes the user's facial expressions and voice to obtain the user's emotional status in real time.

[0475] 6. Dynamic Adjustment Method: A technique for dynamically adjusting manga scenes based on emotion data.

[0476] 7. Correction input means: An interface for inputting user correction instructions for the displayed image.

[0477] 8. Regeneration method: Technology to regenerate manga based on correction instructions and emotion data.

[0478] 9. Storage means: A device or program that stores the final manga in digital form.

[0479] 10. Multilingual translation tools: Technology for translating archived manga into multiple languages.

[0480] 11. Uploading method: The technology to upload the translated manga to the sales platform.

[0481] 12. Additional content generation means: Technology for generating additional content for uploaded manga.

[0482] Specific operation of the system

[0483] The system generates manga through the following process, providing users with a personalized viewing experience:

[0484] 1. Scenario input and analysis:

[0485] The user uses smart glasses to input the manga scenario by voice.

[0486] The terminal transmits the input scenario data to the server, and the server analyzes the scenario using natural language analysis means.

[0487] 2. Automatic Manga Generation:

[0488] The server automatically generates each scene of the manga based on the analysis results using an image generation method. For example, OpenAI's (registered trademark) GPT-3 is used as the generation AI model.

[0489] 3. Preview and Sentiment Analysis:

[0490] The terminal displays the generated manga on the smart glasses for the user to preview.

[0491] Using the camera and microphone built into the smart glasses, the emotion analysis means analyzes the user's facial expressions and voice to obtain the user's emotional status.

[0492] 4. Dynamic Adjustment:

[0493] Based on the emotion analysis results, the server reactivates the image generation means using a dynamic adjustment means to adjust the manga scenes in real time.

[0494] 5. Fix and Regenerate:

[0495] The user specifies the part to be corrected by voice or text and sends the correction instruction to the server.

[0496] The server operates the regeneration means based on the correction instructions and emotion data to generate a revised version of the manga.

[0497] 6. Save, translate and upload:

[0498] The server then stores the final manga in the cloud.

[0499] The multilingual translation means translates the stored manga into multiple languages ​​and uploads it to the sales platform using the upload means.

[0500] 7. Generate additional content:

[0501] The server generates additional content such as stamps and games based on the uploaded manga.

[0502] Specific examples

[0503] When a user uses smart glasses to speak the scenario "A story about a hero who defeats a dragon," the system operates in the following process:

[0504] 1. Scenario input: Voice input: "Please generate a scene in which a hero defeats a dragon. Please emphasize the surprising elements."

[0505] 2. Analysis and automatic generation: The server analyzes the scenario and automatically generates scenes using image generation means.

[0506] 3. Preview and Emotion Analysis: The user previews the video, and the smart glasses' camera and microphone analyze the user's facial expressions and voice.

[0507] 4. Dynamic Adjustment: The scene is dynamically adjusted based on the sentiment analysis results. If the user expresses surprise, the scene is regenerated to emphasize the surprise element more.

[0508] In this way, it is possible to provide an interactive manga viewing experience that can reflect the user's emotions in real time.

[0509] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0510] Step 1:

[0511] Entering and sending a scenario

[0512] The user inputs the scenario in voice or text format. Specifically, the user inputs through the smart glasses, "Please generate a scene in which the hero defeats the dragon. Please emphasize the surprising elements in particular." The input scenario data is sent to the server by the device. This provides the server with the input data to begin analysis.

[0513] Step 2:

[0514] Scenario data analysis

[0515] The server then uses natural language analysis to analyze the received scenario data. During the analysis, detailed information about each scene and character in the story is extracted. Specifically, elements such as "hero," "dragon," and "battle scene" are extracted from the scenario. This provides the basic data needed to generate manga scenes.

[0516] Step 3:

[0517] Manga scene generation

[0518] The server uses an image generation method based on the analyzed data to automatically generate each scene of the manga. This uses a generative AI model (e.g., OpenAI GPT-3). Specifically, the server generates the scene using the prompt "Generate a scene in which a hero defeats a dragon. Please emphasize the surprising elements in particular." as input. The output is image data of the generated manga.

[0519] Step 4:

[0520] Show Preview

[0521] The device displays the generated manga image data on the smart glasses, allowing the user to preview it. Specifically, the user views the manga through the smart glasses, allowing the user to check the generated scenes in real time.

[0522] Step 5:

[0523] Emotion analysis and capture

[0524] Using the camera and microphone built into the smart glasses, the emotion analysis means analyzes the user's facial expressions and voice and obtains their emotional status in real time. Specifically, when surprise is detected from the user's facial expression, the data is sent to the server, thereby obtaining the user's emotional data.

[0525] Step 6:

[0526] Dynamic Scene Adjustment

[0527] The server uses dynamic adjustment methods based on the acquired emotional data to adjust the manga scene in real time. For example, if the user expresses surprise, the server regenerates the scene to further emphasize that emotion. Specifically, the server re-inputs the prompt sentence into the generative AI model and regenerates the scene. This results in image data of the emphasized scene.

[0528] Step 7:

[0529] Enter and submit corrections

[0530] The user specifies the parts of the scene they have previewed that they want to edit by voice or text, and sends the edit instructions to the server via their device. For example, the user might say, "Make this a more intense battle scene." This sends the edit instruction data to the server.

[0531] Step 8:

[0532] Running Regeneration

[0533] The server regenerates the manga using the regeneration means based on the modification instructions and emotion data. Specifically, the server operates the image generation means again to generate a modified version of the scene. This results in image data of the modified manga.

[0534] Step 9:

[0535] Save the final manga

[0536] The server stores the final manga after correction and dynamic adjustment in the cloud. Specifically, the data of the manga is stored in a digital format using a storage means, so that the final manga data is stored safely.

[0537] Step 10:

[0538] Multilingual translation and uploading

[0539] The server translates the stored manga into multiple languages ​​using a multilingual translation means and uploads it to the sales platform using an uploading means. Specifically, the server translates the manga into English, French, etc. using a translation engine and uploads it to the online sales site. This allows the manga to be provided to users in various languages.

[0540] Step 11:

[0541] Generating additional content

[0542] The server generates additional content such as stamps and games based on the uploaded manga. Specifically, the server uses the additional content generation means to create a stamp set using the manga characters, thereby providing manga fans with additional products.

[0543] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0544] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0545] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0546] [Second embodiment]

[0547] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0548] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0549] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0550] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0551] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0552] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0553] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0554] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0555] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0556] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0557] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0558] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0559] The present invention relates to a manga production system that utilizes generation AI, and an embodiment thereof will be described in detail below.

[0560] System Overview

[0561] This system uses AI technology to automatically generate manga based on a manga scenario entered by the user, and also handles multilingual support, sales development, and the generation of additional content. The system includes user, terminal, and server components.

[0562] System Component Description

[0563] 1. Scenario input method

[0564] The user uses the terminal to input a manga scenario. The scenario is input in text format. For example, the user can input a story in which the protagonist, a high school student, is summoned to another world.

[0565] 2. Natural language analysis means

[0566] The device sends the input scenario to the server, which then analyzes it using natural language processing technology. This analysis extracts detailed information about each scene and character in the story. For example, the background, character expressions, and location of the "summoning scene" can be identified.

[0567] 3. Image Generation Method

[0568] Based on the analysis results, the server uses AI technology to automatically generate each scene in the manga. During this generation process, character designs and backgrounds are drawn. For example, an otherworldly landscape is drawn as the background for the "summoning scene."

[0569] 4. Preview Method

[0570] The image data of the generated manga is sent to the terminal, and the user can check the preview. The preview is displayed in slide format, and the user can check each page of the manga in order.

[0571] 5. Correction input method

[0572] Users are presented with an interface to view a preview and input corrections and improvements, such as specific instructions like "change the character's facial expression to look more surprised."

[0573] 6. Regeneration means

[0574] The user's correction instructions are sent to the server, and the server again uses the image generation means to generate a revised version of the manga page. For example, the facial expression of the character may be changed in accordance with the user's instructions.

[0575] 7. Preservation means

[0576] Once the manga is created to the user's satisfaction, the data is saved on the device or in the cloud. For example, the completed manga can be saved in cloud storage.

[0577] 8. Multilingual Translation Tools

[0578] The saved manga is translated into multiple languages ​​using a translation engine, allowing the same manga to be published in multiple languages, for example, English, Chinese, and Spanish.

[0579] 9. Upload Method

[0580] The translated manga data is uploaded to a sales platform and a sales page is generated, for example, the manga is listed and sold in an online store.

[0581] 10. Additional Content Generation Methods

[0582] If a manga becomes popular, the server will generate additional content such as stamps and games using its characters and scenes. For example, the main character's character will be converted into a LINE stamp.

[0583] Specific examples

[0584] User A uses the scenario input means to input a scenario he or she has created, in which "the protagonist, a high school student, is summoned to another world." The device sends the scenario data to the server, which analyzes it using natural language analysis means. The image generation means then automatically generates each page of the manga, and a preview is displayed on the device. User A checks the preview and inputs corrections to the character's facial expressions. The server generates the images again and sends the corrected versions to the device, repeating this process until User A is satisfied. When the manga is finally completed, it is saved in the cloud using the saving means and translated into multiple languages ​​using the multilingual translation means. The translated manga is uploaded to a sales platform using the uploading means, and popular characters are then sold as stamps using the additional content generation means.

[0585] This allows users to easily create high-quality manga, support multiple languages, expand sales channels, and generate additional content.

[0586] The processing flow will be explained below.

[0587] Step 1:

[0588] The user inputs a scenario. The terminal displays a text area for the user to input the scenario. The user inputs the scenario into the text box and clicks the "Submit" button when complete.

[0589] Step 2:

[0590] When the send button is clicked, the device converts the scenario data into JSON format, which is then sent to the server as an API request.

[0591] Step 3:

[0592] The server analyzes the received scenario data using natural language analysis means, and extracts story scenes and character information as the analysis results.

[0593] Step 4:

[0594] The server uses image generation tools to generate each scene of the manga based on the analysis results. AI technology is used to depict characters and backgrounds, and each page is generated as image data.

[0595] Step 5:

[0596] The server sends the image data of the generated manga page to the terminal, which then decodes the received image data and displays a preview to the user.

[0597] Step 6:

[0598] The user checks the displayed preview and specifies the corrections to be made. The terminal provides an interface for corrections, and the user inputs the corrections.

[0599] Step 7:

[0600] The terminal transmits the inputted correction instructions to the server, which receives the correction instructions and generates a corrected version using the image generating means again.

[0601] Step 8:

[0602] The server sends the image data of the corrected manga page to the terminal, which then displays the preview again for the user to reconfirm.

[0603] Step 9:

[0604] The process from step 6 to step 8 is repeated until the user is satisfied. Finally, a manga that satisfies the user is generated.

[0605] Step 10:

[0606] The user clicks the Finish button, and the terminal uses the storage means to store the final manga data in cloud storage.

[0607] Step 11:

[0608] The server translates the saved manga data using a multilingual translation tool, and it is translated into multiple languages, including English, Chinese, and Spanish.

[0609] Step 12:

[0610] The server uploads the translated manga data to the sales platform using the uploading means, and a sales page is generated and published.

[0611] Step 13:

[0612] The server analyzes the sales data and generates content such as stamps and games for popular manga using additional content generation means.

[0613] This allows users to easily create manga, make it multilingual, sell it, and expand it further.

[0614] Example 1

[0615] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0616] Conventional manga production systems have difficulty automating the entire process, from generating a manga after a user inputs a scenario to multilingual translation, sales development, and the generation of additional content. In particular, they lack the functionality to regenerate manga content in real time according to user correction instructions, and multilingual support and uploading to sales platforms are labor-intensive tasks. Furthermore, generative AI models have not been fully utilized for high-quality image generation or scenario analysis.

[0617] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0618] In this invention, the server includes a scenario input means, a natural language analysis means, an image generation means, a preview means, a correction input means, a regeneration means, a storage means, a multilingual translation means, an upload means, an additional content generation means, a cloud storage means, an analysis means that uses a generative AI model, and a prompt sentence processing means. This allows a user to simply input a scenario, automatically generating a high-quality manga, and enabling real-time regeneration in response to user correction instructions, multilingual support, sales development, and the generation of additional content.

[0619] "Scenario input means" refers to a device or software that allows a user to input a manga scenario in text format.

[0620] "Natural language analysis means" refers to a device or software that analyzes input scenario data using natural language processing technology and extracts information about each scene and character in the story.

[0621] "Image generation means" refers to a device or software for generating images of each scene and character in a manga based on data analyzed by a natural language analysis means.

[0622] The "preview means" refers to a device or software that displays the generated image so that the user can check it.

[0623] The "modification input means" refers to a device or software that provides an interface for a user to input instructions for modification or improvement of a preview.

[0624] The "regeneration means" refers to a device or software that regenerates a manga image based on the correction instructions input by the correction input means.

[0625] "Storage means" refers to a device or software that records completed manga data for long-term retention.

[0626] "Multilingual translation means" refers to a device or software for translating stored manga into multiple languages.

[0627] "Uploading means" refers to the device or software used to upload translated manga to an online sales platform and generate a sales page.

[0628] "Additional content generation means" refers to a device or software for generating additional content such as stamps and games using manga characters and scenes.

[0629] "Cloud storage means" refers to a device or software that stores generated manga in cloud storage and makes it easily accessible.

[0630] "Analysis means utilizing a generative AI model" refers to a device or software that uses a generative AI model to analyze scenario data and obtain appropriate output.

[0631] "Prompt sentence processing means" refers to a device or software for generating and using prompt sentences for scenario data to perform analysis and image generation.

[0632] The present invention relates to a manga production system that utilizes generation AI, and specific embodiments will be described in detail.

[0633] System Overview

[0634] This system uses AI technology to automatically generate manga based on a manga scenario entered by the user, and also handles multilingual support, sales development, and the generation of additional content. The system includes user, terminal, and server components.

[0635] System Component Description

[0636] 1. Scenario input method

[0637] The user uses the terminal to input a manga scenario. The scenario is input in text format. For example, the user can input a story in which the protagonist, a high school student, is summoned to another world.

[0638] 2. Natural language analysis means

[0639] The device sends the input scenario to a server, which then analyzes it using a generative AI model (e.g., GPT-4). This analysis extracts detailed information about each scene and character in the story. For example, the background, character expressions, and location of the "summoning scene" are identified.

[0640] 3. Image Generation Method

[0641] Based on the analysis results, the server uses AI technology (such as DALL-E or MidJourney) to automatically generate each scene in the manga. During this generation process, character designs and backgrounds are drawn. For example, an otherworldly landscape is drawn as the background for the "summoning scene."

[0642] 4. Preview Method

[0643] The image data of the generated manga is sent to the terminal, and the user can check the preview. The preview is displayed in slide format, and the user can check each page of the manga in order.

[0644] 5. Correction input method

[0645] Users are presented with an interface to view a preview and input corrections and improvements, such as specific instructions like "change the character's facial expression to look more surprised."

[0646] 6. Regeneration means

[0647] The user's modification instructions are sent from the terminal to the server, and the server uses the image generation means to generate a modified version of the manga page. For example, the facial expression of the character may be changed in response to the user's instructions.

[0648] 7. Preservation means

[0649] Once a manga that satisfies the user is finally created, the data is stored in cloud storage. For example, the completed manga is stored in cloud storage.

[0650] 8. Multilingual Translation Tools

[0651] The server translates the stored manga into multiple languages ​​using a translation engine (e.g., Google Translate API). This allows the same manga to be published in multiple languages, for example, English, Chinese, and Spanish.

[0652] 9. Upload Method

[0653] The translated manga data is uploaded to an online sales platform and a sales page is generated, for example, the manga is listed and sold in an online store.

[0654] 10. Additional Content Generation Methods

[0655] If a manga becomes popular, the server will generate additional content such as stamps and games using its characters and scenes. For example, the main character's character could be converted into a stamp.

[0656] Specific examples

[0657] For example, User A uses the scenario input means to input a scenario he or she has created, in which "the protagonist, a high school student, is summoned to another world." The device sends the scenario data to the server, which performs natural language analysis using a generative AI model. The image generation means then automatically generates each page of the manga, and a preview is displayed on the device. User A checks the preview and inputs corrections to the character's facial expressions. The server generates the images again and sends the corrected version to the device, repeating this process until User A is satisfied. Once the manga is finally completed, it is stored in the cloud using the storage means and translated into multiple languages ​​using the multilingual translation means. The translated manga is uploaded to a sales platform using the upload means, and popular characters are sold as stamps using the additional content generation means. This allows users to easily create high-quality manga, support multiple languages, expand sales channels, and generate additional content.

[0658] Prompt Sentence Examples

[0659] 1. "Describe a scene in which the protagonist is summoned to another world, using a fantasy-style landscape as the background."

[0660] 2. "Please make the character's expression look a little more surprised."

[0661] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0662] Step 1:

[0663] Scenario input

[0664] The user uses the device to input the manga scenario in text format. For example, they can input the content, "The protagonist, a high school student, is summoned to another world." The input scenario is saved as text data on the device.

[0665] Input: A textual scenario entered by the user.

[0666] Output: Scenario data saved on the device

[0667] Step 2:

[0668] Scenario transmission

[0669] The device sends the input scenario data to the server, efficiently transmitting the data using HTTP requests.

[0670] Input: Scenario data entered

[0671] Output: Scenario data sent to the server

[0672] Step 3:

[0673] Scenario Analysis

[0674] The server analyzes the received scenario using a generative AI model (e.g., GPT-4), which extracts key elements of the scenario (scenes, character traits, etc.).

[0675] Input: Scenario data sent to the server

[0676] Output: Information about each scene and character in the analyzed story

[0677] How it works: The generative AI model analyzes each part of the scenario and extracts the necessary text information through tokenization, grammatical analysis, and semantic analysis.

[0678] Step 4:

[0679] Image generation

[0680] The server uses AI technology (e.g., DALL-E and MidJourney) to automatically generate each scene in the manga based on the analysis results. For example, the background of the "summoning scene" is an otherworldly landscape.

[0681] Input: Information about each scene and character in the analyzed story

[0682] Output: Images of each scene in the generated manga

[0683] How it works: AI technology generates images based on text information and draws designs for each scene and character.

[0684] Step 5:

[0685] Preview Generation

[0686] The server compiles the generated images and creates preview data, which is formatted in a format that can be easily viewed by the user (slideshow format).

[0687] Input: Each scene image of the generated manga

[0688] Output: Preview data

[0689] Specific operation: The generated images are processed into a continuous slideshow format and converted into a format that the user can view sequentially.

[0690] Step 6:

[0691] Preview display

[0692] The terminal receives the preview data sent from the server and displays it to the user, who then checks the preview in slide format.

[0693] Input: Preview data

[0694] Output: Preview screen displayed on the device

[0695] Specific operation: The device loads the preview data and displays it in a slideshow format that is easy for the user to view.

[0696] Step 7:

[0697] Correction instruction input

[0698] Users can view the preview and input specific complaints and requests for corrections, such as "change the character's facial expression to look more surprised."

[0699] Input: User correction instructions

[0700] Output: Correction instruction data saved on the device

[0701] Specific operation: The terminal receives the user's input and stores it as correction instruction data.

[0702] Step 8:

[0703] Send correction instructions

[0704] The device sends the user's correction instructions to the server via an HTTP request.

[0705] Input: User correction instruction data

[0706] Output: Correction instruction data sent to the server

[0707] Specific operation: The terminal creates an HTTP request and sends correction instruction data to the server.

[0708] Step 9:

[0709] Regeneration Process

[0710] The server then uses the image generation means to generate a revised manga page based on the received revision instructions. For example, the facial expression of the character may be changed in response to the user's instructions.

[0711] Input: Correction instruction data sent to the server

[0712] Output: Regenerated manga scene images

[0713] Specific operation: Using AI technology, the regenerated image is recreated while reflecting correction instructions.

[0714] Step 10:

[0715] final save

[0716] The server stores the manga that the user is finally satisfied with in cloud storage.

[0717] Input: Regenerated manga scene images

[0718] Output: Manga data saved in cloud storage

[0719] Specific operation: Manga data is uploaded to a remote server through the API of a cloud storage service and stored for a long period of time.

[0720] Step 11:

[0721] Multilingual Translation

[0722] The server translates the stored manga data into multiple languages ​​using a translation engine (e.g., Google Translate API), allowing it to be expanded into multiple languages ​​such as English, Chinese, and Spanish.

[0723] Input: Manga data stored in cloud storage

[0724] Output: Manga data translated into multiple languages

[0725] What it does: The translation engine converts the text into each language and applies it to the appropriate manga text section.

[0726] Step 12:

[0727] Upload to sales platform

[0728] The server uploads the translated manga data to an online sales platform (e.g., Amazon Kindle Direct Publishing) and automatically generates a sales page.

[0729] Input: Manga data translated into multiple languages

[0730] Output: Sales page on sales platform

[0731] Specific operations: Use the sales platform's API to upload translated manga data and set up the product page.

[0732] Step 13:

[0733] Additional content generation

[0734] The server analyzes the popularity of the manga and generates additional content such as stamps and games based on popular characters and scenes.

[0735] Input: Manga data stored in cloud storage and popularity analysis results

[0736] Output: Generated additional content (e.g. stamps, games)

[0737] Specific operation: Generate stamps and simple game materials based on data on popular characters and scenes.

[0738] (Application example 1)

[0739] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0740] Traditional manga production methods have the problem of being time-consuming and labor-intensive, as much of the work is done manually. Additionally, processes such as supporting different languages, uploading to sales platforms, and generating additional content are performed separately, reducing overall work efficiency. There was a need for a method to solve these problems and achieve more efficient and faster manga production, translation, distribution, and additional content generation.

[0741] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0742] In this invention, the server includes a scenario input means, a natural language analysis means, an image generation means, a preview means, a correction input means, a regeneration means, a saving means, a multilingual translation means, an uploading means, an additional content generation means, a processing means using a generative AI model, and a prompt sentence generation means. This allows for a process that starts with a scenario input by a user, followed by automatic generation of manga pages using a generative AI model, natural language analysis, image generation, preview and correction, regeneration, saving, multilingual translation, uploading to a sales platform, and even automatic generation of additional content.

[0743] The "scenario input means" is an interface that allows the user to input a scenario in text format.

[0744] The "natural language analysis means" is a function that analyzes input scenario data and extracts detailed information about each scene and character in the story based on that content.

[0745] The "image generation means" is a technology for automatically generating each scene of a manga based on data analyzed by the natural language analysis means.

[0746] The "preview means" is a display means for allowing the user to check the image data of the generated manga.

[0747] The "correction input means" is an interface that allows the user to view the preview and input corrections or improvements.

[0748] The "regeneration means" is a function for regenerating an image based on correction instructions from the user.

[0749] The "storage means" is a function for saving the final generated manga data.

[0750] A "multilingual translation means" is an engine for translating stored manga into multiple different languages.

[0751] "Uploading means" is a function for uploading translated manga data to the sales platform.

[0752] The "additional content generating means" is a means for generating additional content related to the manga (for example, character stamps, etc.).

[0753] A "generative AI model" is an artificial intelligence model that generates manga pages based on a scenario provided by the user.

[0754] A "prompt sentence generation means" is a technology for generating and providing appropriate prompt sentences to a generative AI model.

[0755] This invention relates to a manga production system that utilizes generative AI. Users input a scenario, and AI technology automatically generates a manga, translating it, selling it, and generating additional content. The system includes user, terminal, and server components.

[0756] The main components of the system are as follows:

[0757] 1. Scenario input method

[0758] The user inputs the scenario using a smartphone or a head-mounted display. The scenario is input in text format.

[0759] 2. Natural language analysis means

[0760] The scenario data sent from the device to the server is analyzed using natural language processing engines such as OpenNLP to extract details about scenes and characters.

[0761] 3. Image Generation Method

[0762] Based on the results of natural language analysis, the server uses a generative AI model (e.g., a TensorFlow or GPT-3-based model) to generate images of each scene in the manga, which are then stored in cloud storage.

[0763] 4. Preview Method

[0764] The generated manga image is sent to the device, and the user can preview it from a React Native-based UI.

[0765] 5. Correction input method

[0766] Users can view the preview and use an interface to provide corrections and improvements, which is also implemented using React Native.

[0767] 6. Regeneration means

[0768] Once the correction instructions are sent to the server, the server runs the AI ​​model again to generate the corrected image.

[0769] 7. Preservation means

[0770] Once the user is finally satisfied with the manga, it is saved in cloud storage.

[0771] 8. Multilingual Translation Tools

[0772] The saved manga can be automatically translated into multiple languages ​​using a multilingual translation tool (e.g., Google Translate API).

[0773] 9. Upload Method

[0774] The translated manga is uploaded using the sales platform's API (e.g., Amazon Kindle Publisher API).

[0775] 10. Additional Content Generation Methods

[0776] If a manga becomes popular, additional content (such as stamps or games) will be generated based on the characters and scenes, also using AI technology.

[0777] Specific examples

[0778] The user inputs the scenario "An adventurous girl explores ancient ruins." This scenario is sent from the device to the server, and natural language analysis extracts the information "adventure," "ancient ruins," and "exploring girl." The generative AI model then generates an image based on the following prompt:

[0779] Example prompts

[0780] "Scenario: An adventurous girl explores ancient ruins. The first scene shows the girl standing at the entrance to the ruins, with a green jungle behind her. In this scene, the girl has a surprised expression, and a broken stone statue can be seen in the background. Generate a manga page based on this scene."

[0781] The generated image is sent to the user as a preview, and the user can input correction instructions, such as "I want the girl's expression to look more surprised." The server then runs the generative AI model again to generate a corrected image. This process is repeated until the user is finally satisfied.

[0782] This allows users to complete the entire process from entering a scenario to creating the final manga, saving, translating, selling, and generating additional content all in one system.

[0783] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0784] Step 1:

[0785] The user inputs a scenario using a smartphone or head-mounted display. This scenario is input in text format, converted to JSON format from the device, and sent to the server. The input data is the scenario thought up by the user, and the output is the JSON-format scenario data received by the server.

[0786] Step 2:

[0787] The server analyzes the received scenario data using natural language analysis methods. A natural language processing engine such as OpenNLP is used for the analysis, and detailed information about each scene and character in the story is extracted from the scenario. The input is the scenario data received by the server, and the output is detailed information about the extracted scenes and characters. Scene classifications and character characteristics are identified through data analysis processing.

[0788] Step 3:

[0789] Based on the analyzed information, the server uses an AI model (for example, a model based on TensorFlow or GPT-3) to automatically generate images of each scene in the manga. At this time, a prompt is generated and supplied to the generative AI model. The input is the extracted scene and character details, as well as the generated prompt, and the output is image data of the generated manga. The AI ​​model generates images based on the prompt, and the images are stored in cloud storage.

[0790] Step 4:

[0791] The generated manga image is sent to the device, and the user can check the preview through a React Native-based UI. Here, the user can check the generated manga page step by step. The input is the generated manga image data, and the output is the preview screen displayed on the user's device. The preview display allows the user to examine the image in detail.

[0792] Step 5:

[0793] While viewing the preview, the user uses an interface to input corrections and improvements. This interface is also implemented using React Native. The input is the corrections specified by the user, and the output is the correction instruction data sent to the server. The specific corrections are specified by the user's interface operations.

[0794] Step 6:

[0795] When the server receives the correction instructions, it runs the AI ​​model again to generate the corrected image. The input is the correction instruction data from the user, and the output is the corrected manga image data. This causes the generative AI model to run again and generate an image according to the user's corrections.

[0796] Step 7:

[0797] The manga image that the user is finally satisfied with is saved in cloud storage. The input is the final manga image data after corrections, and the output is the manga data saved in the cloud. Cloud storage ensures that the data is saved permanently.

[0798] Step 8:

[0799] The stored manga data is translated into multiple languages ​​using a multilingual translation tool (for example, Google Translate API). The input is the manga data stored in cloud storage, and the output is the translated manga data in multiple languages. Multilingual support is achieved using an AI translation engine.

[0800] Step 9:

[0801] The translated manga is uploaded online using the sales platform's API (e.g., Amazon Kindle Publisher API). The input is the translated manga data, and the output is the manga page published on the sales platform. Uploading makes it accessible to a wide range of users.

[0802] Step 10:

[0803] If a manga becomes popular, the server generates additional content based on its characters and scenes. This process is also carried out using AI technology. The input is the saved manga data, and the output is data for additional content (e.g., character stamps). The generated additional content expands the range of manga-related products.

[0804] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0805] The present invention combines an emotion engine that recognizes the user's emotions in a manga production system that utilizes generative AI, and an embodiment of this system will be described in detail below.

[0806] System Overview

[0807] This system uses AI technology to automatically generate manga based on a manga scenario entered by the user, and also handles multilingual support, sales development, and the generation of additional content. By combining it with an emotion engine, it can detect the user's emotional state and provide a more user-friendly interface and feedback. The system includes user, terminal, and server components.

[0808] System Component Description

[0809] 1. Scenario input method

[0810] The user uses the device to input a manga scenario. The scenario is input in text format. The emotion engine analyzes the user's facial expressions and voice as they input, and can obtain their emotional status. For example, when inputting a story in which the protagonist, a high school student, is summoned to another world, the user's emotions of surprise and excitement are analyzed.

[0811] 2. Natural language analysis means

[0812] The device sends the input scenario to the server, which then analyzes it using natural language processing technology to extract detailed information about each scene and character in the story.

[0813] 3. Image Generation Method

[0814] Based on the analysis results, the server uses AI technology to automatically generate each scene of the manga. The AI ​​also adjusts the generation process based on the user's emotional status. For example, if the user is excited when inputting their emotion, the generated design will emphasize action scenes.

[0815] 4. Preview Method

[0816] The image data of the generated manga is sent to the device, where the user can check a preview. During the preview, the emotion engine analyzes the user's facial expressions and voice to obtain emotional feedback. For example, if the user smiles while looking at the preview, that emotion is fed back.

[0817] 5. Correction input method

[0818] Users are presented with an interface to preview the edit and input corrections and improvements. The emotion engine analyzes the user's emotions as they input their corrections and makes suggestions accordingly. For example, if the user has a questioning expression, the engine will suggest corrections for that part.

[0819] 6. Regeneration means

[0820] The user's correction instructions are sent to the server, which then uses the image generation means to generate a revised version of the manga page. The regeneration process also takes the emotional status into consideration.

[0821] 7. Preservation means

[0822] Once a manga that satisfies the user is finally created, the data is saved on the device or in the cloud.

[0823] 8. Multilingual Translation Tools

[0824] The saved manga is translated into multiple languages ​​using a translation engine, allowing the same manga to be published in multiple languages.

[0825] 9. Upload Method

[0826] The translated manga data is uploaded to a sales platform and a sales page is generated.

[0827] 10. Additional Content Generation Methods

[0828] If the manga proves popular, the server will generate additional content such as stamps and games using its characters and scenes.

[0829] Specific examples

[0830] User B inputs a scenario he or she has thought up, "A story about a magical girl who saves the world," using the scenario input means. The device sends the scenario data to the server, which analyzes it using natural language analysis means. The image generation means then automatically generates each page of the manga, and a preview is displayed on the device. During the preview, the emotion engine analyzes User B's facial expressions and obtains his or her emotional status. If User B expresses surprise at a scene, the scene is emphasized or suggested for revision. User B inputs correction instructions, and the server re-generates images and sends the revised version to the device. Once a manga that satisfies User B has finally been generated, it is stored in the cloud using the storage means and translated into multiple languages ​​using the multilingual translation means. The translated manga is uploaded to the sales platform using the upload means, and popular characters are then converted into stamps using the additional content generation means.

[0831] This allows users to create manga using emotional feedback, providing a more intuitive and interactive production experience. It also makes it easy to support multiple languages, expand sales channels, and generate additional content, making it a valuable system for users.

[0832] The processing flow will be explained below.

[0833] Step 1:

[0834] The user inputs a scenario. The device provides the user with a text area for scenario input and the ability to analyze the user's facial expressions and voice using an emotion engine. The user inputs the scenario into the text box and clicks the "Submit" button when complete.

[0835] Step 2:

[0836] When the device receives a click of the send button, it converts the scenario data into JSON format. At the same time, the emotion engine obtains the emotional status of the user at the time of input and sends it to the server along with the scenario data.

[0837] Step 3:

[0838] The server analyzes the received scenario data and emotional status using natural language analysis. When extracting scene and character information as analysis results, the emotional status is also taken into consideration. For example, if the user is excited, a scene will be created that reflects that excitement.

[0839] Step 4:

[0840] The server uses image generation means to generate each scene of the manga based on the analysis results. During the generation process, the character's facial expressions and background are set based on the user's emotional status. For example, action scenes can be emphasized to reflect excitement.

[0841] Step 5:

[0842] The server sends the image data of the generated manga page to the device. The device decodes the received image data and displays a preview to the user. The emotion engine analyzes the user's facial expressions and voice during the preview and sends emotional feedback to the server.

[0843] Step 6:

[0844] The user checks the displayed preview and specifies the areas to be corrected. The device provides an interface for correction, and the user inputs the corrections. The emotion engine analyzes the user's emotions when inputting the corrections and makes correction suggestions as necessary.

[0845] Step 7:

[0846] The terminal transmits the inputted correction instructions and emotion feedback to the server, which receives the correction instructions and emotion feedback and generates a corrected version using the image generation means again.

[0847] Step 8:

[0848] The server sends the image data of the revised manga page to the device. The device displays the preview again, and the user can recheck the revised version. If necessary, steps 6 to 8 are repeated.

[0849] Step 9:

[0850] The process from step 6 to step 8 is repeated until the user is satisfied. When a manga that satisfies the user is finally generated, the user clicks the done button and the terminal saves the final manga data in cloud storage using the storage means.

[0851] Step 10:

[0852] The server translates the saved manga data using a multilingual translation tool, and it is translated into multiple languages, including English, Chinese, and Spanish.

[0853] Step 11:

[0854] The server uploads the translated manga data to the sales platform using the uploading means, and a sales page is generated and published.

[0855] Step 12:

[0856] The server analyzes the sales data and generates content such as stamps and games for popular manga using additional content generation means.

[0857] Step 13:

[0858] The user uses the emotion engine to check the generated stamps and game content and provide instructions for corrections if necessary. The server then repeats the generation process, and the final content is completed.

[0859] Through these steps, users can create manga using emotional feedback and enjoy an intuitive and interactive production experience. Furthermore, the system offers great value to users, as it easily supports multiple languages, expands sales channels, and allows for the creation of additional content.

[0860] Example 2

[0861] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0862] Conventional manga production systems do not reflect the user's emotional state when the user inputs a manga scenario and checks each scene of the generated manga to give instructions for correction. This makes it difficult to generate manga that is in line with the user's intentions and emotions, and the generated manga sometimes does not meet the user's expectations. Furthermore, because the user's emotions could not be taken into account when creating multilingual content or additional content, it was difficult to increase user satisfaction.

[0863] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0864] In this invention, the server includes a scenario input means, a natural language analysis means for analyzing the scenario data input from the scenario input means and the user's emotional status, and an image generation means for generating a manga image based on the data analyzed by the natural language analysis means, thereby enabling the generation of a manga that reflects the user's emotional state.

[0865] The "scenario input means" is a means for a user to input a manga scenario in text format, and also includes a function for acquiring the user's emotional status.

[0866] The "natural language analysis means" is a means for analyzing the scenario data input from the scenario input means and the user's emotional status, and extracting each element of the story (scene and character information).

[0867] "Image generation means" refers to a means for generating manga images using a generative AI model based on data analyzed by a natural language analysis means.

[0868] The "preview means" is a means for displaying the image generated by the image generating means so that the user can check it.

[0869] The "emotion feedback acquisition means" is a means for acquiring an emotional status from the user's facial expression and voice when checking the image displayed by the preview means.

[0870] The "modification input means" is a means for providing an interface for the user to input instructions for modifying the preview image, and for suggesting modifications based on the acquired emotional status.

[0871] The "regeneration means" is a means for generating a revised version of the manga by operating the image generation means again based on the correction instructions and emotion feedback input by the correction input means.

[0872] "Storage means" refers to the means for storing the final manga, including cloud storage and storage locations on the device.

[0873] The "multilingual translation means" is a means for translating the manga stored by the storage means into multiple languages.

[0874] The "uploading means" refers to the means for uploading the translated manga to the sales platform and generating a sales page.

[0875] "Additional content generation means" refers to a means for generating additional content such as stamps and games based on the characters and scenes of manga uploaded to the sales platform.

[0876] The present invention is a system that automatically generates manga using a generative AI model, and combines it with an emotion engine that detects user emotions. Specific embodiments of the present invention will be described in detail below.

[0877] System Overview

[0878] This system uses AI technology to automatically generate manga based on a manga scenario entered by the user, and handles multilingual support, sales promotion, and the generation of additional content. It detects the user's emotional state and provides an interface that allows smooth revisions based on that feedback. The system consists of user, terminal, and server components.

[0879] System Component Description

[0880] 1. Scenario input method

[0881] The user inputs the manga scenario in text format using the device. At this time, the emotion engine analyzes the user's facial expressions and voice to obtain their emotional status. For example, when inputting a story in which the protagonist, a high school student, is summoned to another world, the user's emotions of surprise and excitement are analyzed.

[0882] 2. Natural language analysis means

[0883] The device sends the input scenario data to the server, which then analyzes the received scenario using natural language processing technology (e.g., Python's NLTK library) and extracts each element of the story (scene and character information).

[0884] 3. Image Generation Method

[0885] The server automatically generates each scene of the manga using a generative AI model (e.g., GPT-4) based on the analysis results and the user's emotional status. If the user is excited, the design is adjusted to emphasize action scenes.

[0886] 4. Preview Method

[0887] The image data of the generated manga is sent to the device, and the user can check the preview via a dedicated viewer app.

[0888] 5. Means of obtaining emotional feedback

[0889] During the preview, the device uses an emotion engine to analyze the user's facial expressions and voice to obtain emotional feedback data. For example, if the user smiles while looking at the preview, that emotion is fed back.

[0890] 6. Correction input method

[0891] Users use an interface to view the preview and input corrections and improvements. The emotion engine analyzes the user's emotions and makes appropriate suggestions for corrections. For example, if a user inputs "Make this scene more action-packed," and the emotion engine detects an excited state, it instructs the server to emphasize the action scenes.

[0892] 7. Regeneration means

[0893] The terminal sends the user's correction instructions to the server, and the server regenerates the corrected version of the manga page using the image generation means. The emotional status is also taken into account in this regeneration process.

[0894] 8. Preservation means

[0895] Once a manga that satisfies the user is finally generated, the data is stored in cloud storage (e.g., AWS S3).

[0896] 9. Multilingual Translation Tools

[0897] The server translates the stored manga using a multilingual translation engine (e.g., DeepL API or Google Translate API), allowing the same manga to be published in multiple languages.

[0898] 10. Upload Method

[0899] The translated manga data is uploaded to a sales platform (e.g., Amazon Kindle Direct Publishing) and a sales page is generated.

[0900] 11. Additional Content Generation Methods

[0901] The server will generate additional content such as stamps and games using characters and scenes from popular manga, including the creation of LINE stamps featuring popular characters and the development of mini-games.

[0902] Specific usage examples and prompt sentence examples

[0903] User B inputs a scenario he or she has thought up, "A story about a young magical girl who saves the world," through a scenario input means. The device sends the scenario data to a server, which analyzes it using natural language analysis means. Manga image data is automatically generated using a generative AI model, and a preview is sent to the device. If User B smiles at the preview, that emotional data is sent to the server and reflected as an emphasis point in the next scene. User B inputs correction instructions, and the server generates new illustrations. The completed manga is saved in the cloud, then translated into multiple languages ​​using a multilingual translation means and uploaded to a sales platform. If the manga proves popular, LINE stamps of popular characters are created using an additional content generation means.

[0904] Prompt Sentence Examples

[0905] "Write a story about a high school boy who is summoned to another world and fights with magical powers. Emphasize the emotions of surprise and excitement."

[0906] "Generate a manga-style story about a young wizard girl who saves the world. Adjust the story based on the emotional feedback provided by the user."

[0907] This will allow users to enjoy an intuitive and interactive manga creation experience, and the system will also make it easy to support multiple languages ​​and generate additional content.

[0908] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0909] Specific flow of program processing

[0910] Below, we will explain the processing flow of the program of this system step by step, clearly indicating the input, output, and specific operation of each step.

[0911] Step 1: Scenario Input

[0912] The user inputs a scenario in text format using the scenario input interface of the terminal.

[0913] Input: User-entered scenario text

[0914] Output: Scenario text data and user's emotional status

[0915] Specific operation: When a user inputs "A story about a magical girl who saves the world," the emotion engine analyzes the user's facial expressions and voice in real time through the device's built-in camera and microphone, and obtains emotional status such as surprise or excitement.

[0916] Step 2: Scenario submission

[0917] The terminal transmits the input scenario data to the server.

[0918] Input: Scenario text data and emotional status

[0919] Output: Scenario data and emotional status sent to the server

[0920] Specific operation: The terminal converts the scenario data into JSON format and sends a POST request to the server using the HTTP protocol.

[0921] Step 3: Scenario analysis

[0922] The server analyzes the scenario data and the emotional status using natural language analysis means.

[0923] Input: Scenario data and emotional status sent to the server

[0924] Output: Detailed information and character information for each scene

[0925] Specific operation: The server uses Python's NLTK library to parse the text of the scenario and extract keywords for each element of the story (characters, scenes, etc.).

[0926] Step 4: Manga Generation

[0927] Based on the analysis results, the server uses a generative AI model to automatically generate each scene of the manga.

[0928] Input: Detailed information and emotional status of the analyzed scenario

[0929] Output: Generated manga image data

[0930] Specific operation: The server uses a generative AI model (e.g., GPT-4) to generate illustrations for each scene, emphasizing action scenes if the user is excited.

[0931] Step 5: Preview

[0932] The server transmits the generated manga image data to the terminal, and the user previews it.

[0933] Input: Manga image data

[0934] Output: Preview image displayed on the device

[0935] Specific operation: The server transfers the generated manga image data to the device, and the user previews it using a dedicated viewer app on the device.

[0936] Step 6: Obtaining emotional feedback

[0937] During the preview, the device analyzes the user's facial expressions and voice, obtains emotional feedback data, and sends it to the server.

[0938] Input: Facial expression and voice data of the user during preview

[0939] Output: Emotion feedback data

[0940] Specific operation: The device analyzes the user's facial expressions and voice in real time through the camera and microphone, and sends emotional status such as smile, surprise, or confusion to the server.

[0941] Step 7: Correction Instructions

[0942] The user inputs the corrections and their details using a dedicated interface, and the system makes correction suggestions based on the user's emotional status.

[0943] Input: User correction instructions and emotional feedback data

[0944] Output: Correction instruction data sent to the server

[0945] Specific operation: The user inputs "I want more action in this scene," and the server makes suggestions for revisions based on emotional feedback.

[0946] Step 8: Regenerate

[0947] The server regenerates the image based on the correction instructions and emotional feedback.

[0948] Input: User correction instructions and emotional feedback data

[0949] Output: Regenerated manga image data

[0950] Specific operation: The server generates a new illustration that reflects the correction instructions and creates a new manga image.

[0951] Step 9: Save

[0952] The server then stores the final manga on the device or in cloud storage.

[0953] Input: Final manga image data

[0954] Output: Data stored in cloud storage or on device

[0955] Specific operation: The final manga data is uploaded and saved to cloud storage such as AWS S3.

[0956] Step 10: Multilingual Translation

[0957] The server translates the stored manga into multiple languages.

[0958] Input: Saved manga image data

[0959] Output: Translated manga image data

[0960] Specific operation: The server uses the DeepL API or Google Translate API to translate the manga text into multiple languages.

[0961] Step 11: Upload

[0962] The server uploads the translated manga data to the sales platform.

[0963] Input: Manga data translated into multiple languages

[0964] Output: Manga data uploaded to the sales platform

[0965] Specific operation: The server uploads manga data to Amazon Kindle Direct Publishing or other sales platforms via API.

[0966] Step 12: Generate additional content

[0967] The server generates additional content using characters and scenes from popular manga.

[0968] Input: Sales data from the sales platform and manga character information

[0969] Output: Stamps and mini-games as additional content

[0970] Specific operations: The server develops LINE stamps incorporating popular characters and mini-games using Unity.

[0971] The above is the specific flow and operation of each processing step in this system.

[0972] (Application example 2)

[0973] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0974] Conventional manga production systems lacked the technology to analyze user emotions in real time and dynamically adjust the scenes displayed during preview. This made it difficult for users to create manga that accurately reflected their emotions, and there was room for improvement in the quality of manga and user satisfaction. Despite the demand for providing an interactive viewing experience, especially in physical stores, these technical challenges had not been resolved.

[0975] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0976] In this invention, the server includes a scenario input means, a natural language analysis means, an image generation means, a preview means, an emotion analysis means, a dynamic adjustment means, a correction input means, a regeneration means, a storage means, a multilingual translation means, an upload means, and an additional content generation means. This makes it possible to analyze a user's emotions in real time and dynamically adjust manga scenes based on those emotions. Furthermore, it is possible to realize an interactive manga viewing experience in a physical store and improve user satisfaction.

[0977] The "scenario input means" is a device or interface that allows a user to input a manga scenario in text or audio format.

[0978] The "natural language analysis means" is a technology for analyzing the scenario data input from the scenario input means and extracting detailed information about each scene and character in the story.

[0979] The "image generation means" is a device or program for automatically generating each scene of the manga based on the data analyzed by the natural language analysis means.

[0980] The "preview means" is a device or interface that displays the manga image generated by the image generation means to the user and allows the user to view it.

[0981] The "emotion analysis means" is a technology for analyzing the facial expressions and voice of a user viewing a manga using the preview means, and obtaining the user's emotional status in real time.

[0982] The "dynamic adjustment means" is a technique for dynamically adjusting manga scenes based on the emotion data analyzed by the emotion analysis means.

[0983] The "modification input means" is a device or interface for inputting a user's instructions for modifying the image displayed by the preview means.

[0984] The "regeneration means" is a technique for generating a revised version of the manga by operating the image generation means again based on the correction instructions and emotion data input by the correction input means.

[0985] The "storage means" is a device or program for storing the final manga generated by the reproduction means in a digital format.

[0986] The "multilingual translation means" is a technology for translating the manga stored by the storage means into multiple languages.

[0987] The "uploading means" is a technology for uploading the manga translated by the multilingual translation means to a sales platform and publishing or selling it.

[0988] The "additional content generation means" is a technology for generating additional content such as stamps and games based on characters and scenes from the manga uploaded by the upload means.

[0989] This invention relates to an interactive manga production and viewing system that utilizes generative AI models, and in particular provides technology that analyzes user emotional feedback in real time in physical stores and dynamically adjusts manga scenes.

[0990] System Overview

[0991] The system consists of the following components:

[0992] 1. Scenario input means: A device or interface that allows the user to input the manga scenario in text or voice format.

[0993] 2. Natural language analysis method: Technology that analyzes input scenario data and extracts detailed information about each scene and character in the story.

[0994] 3. Image generation method: Technology that automatically generates each scene of a manga based on analyzed data.

[0995] 4. Preview means: An interface that displays and allows users to view the generated manga image.

[0996] 5. Emotion analysis means: Technology that analyzes the user's facial expressions and voice to obtain the user's emotional status in real time.

[0997] 6. Dynamic Adjustment Method: A technique for dynamically adjusting manga scenes based on emotion data.

[0998] 7. Correction input means: An interface for inputting user correction instructions for the displayed image.

[0999] 8. Regeneration method: Technology to regenerate manga based on correction instructions and emotion data.

[1000] 9. Storage means: A device or program that stores the final manga in digital form.

[1001] 10. Multilingual translation tools: Technology for translating archived manga into multiple languages.

[1002] 11. Uploading method: The technology to upload the translated manga to the sales platform.

[1003] 12. Additional content generation means: Technology for generating additional content for uploaded manga.

[1004] Specific operation of the system

[1005] The system generates manga through the following process, providing users with a personalized viewing experience:

[1006] 1. Scenario input and analysis:

[1007] The user uses smart glasses to input the manga scenario by voice.

[1008] The terminal transmits the input scenario data to the server, and the server analyzes the scenario using natural language analysis means.

[1009] 2. Automatic Manga Generation:

[1010] The server automatically generates each scene of the manga using image generation methods based on the analysis results. For example, OpenAI's GPT-3 is used as the generation AI model.

[1011] 3. Preview and Sentiment Analysis:

[1012] The terminal displays the generated manga on the smart glasses for the user to preview.

[1013] Using the camera and microphone built into the smart glasses, the emotion analysis means analyzes the user's facial expressions and voice to obtain the user's emotional status.

[1014] 4. Dynamic Adjustment:

[1015] Based on the emotion analysis results, the server reactivates the image generation means using a dynamic adjustment means to adjust the manga scenes in real time.

[1016] 5. Fix and Regenerate:

[1017] The user specifies the part to be corrected by voice or text and sends the correction instruction to the server.

[1018] The server operates the regeneration means based on the correction instructions and emotion data to generate a revised version of the manga.

[1019] 6. Save, translate and upload:

[1020] The server then stores the final manga in the cloud.

[1021] The multilingual translation means translates the stored manga into multiple languages ​​and uploads it to the sales platform using the upload means.

[1022] 7. Generate additional content:

[1023] The server generates additional content such as stamps and games based on the uploaded manga.

[1024] Specific examples

[1025] When a user uses smart glasses to speak the scenario "A story about a hero who defeats a dragon," the system operates in the following process:

[1026] 1. Scenario input: Voice input: "Please generate a scene in which a hero defeats a dragon. Please emphasize the surprising elements."

[1027] 2. Analysis and automatic generation: The server analyzes the scenario and automatically generates scenes using image generation means.

[1028] 3. Preview and Emotion Analysis: The user previews the video, and the smart glasses' camera and microphone analyze the user's facial expressions and voice.

[1029] 4. Dynamic Adjustment: The scene is dynamically adjusted based on the sentiment analysis results. If the user expresses surprise, the scene is regenerated to emphasize the surprise element more.

[1030] In this way, it is possible to provide an interactive manga viewing experience that can reflect the user's emotions in real time.

[1031] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1032] Step 1:

[1033] Entering and sending a scenario

[1034] The user inputs the scenario in voice or text format. Specifically, the user inputs through the smart glasses, "Please generate a scene in which the hero defeats the dragon. Please emphasize the surprising elements in particular." The input scenario data is sent to the server by the device. This provides the server with the input data to begin analysis.

[1035] Step 2:

[1036] Scenario data analysis

[1037] The server then uses natural language analysis to analyze the received scenario data. During the analysis, detailed information about each scene and character in the story is extracted. Specifically, elements such as "hero," "dragon," and "battle scene" are extracted from the scenario. This provides the basic data needed to generate manga scenes.

[1038] Step 3:

[1039] Manga scene generation

[1040] The server uses an image generation method based on the analyzed data to automatically generate each scene of the manga. This uses a generative AI model (e.g., OpenAI GPT-3). Specifically, the server generates the scene using the prompt "Generate a scene in which a hero defeats a dragon. Please emphasize the surprising elements in particular." as input. The output is image data of the generated manga.

[1041] Step 4:

[1042] Show Preview

[1043] The device displays the generated manga image data on the smart glasses, allowing the user to preview it. Specifically, the user views the manga through the smart glasses, allowing the user to check the generated scenes in real time.

[1044] Step 5:

[1045] Emotion analysis and capture

[1046] Using the camera and microphone built into the smart glasses, the emotion analysis means analyzes the user's facial expressions and voice and obtains their emotional status in real time. Specifically, when surprise is detected from the user's facial expression, the data is sent to the server, thereby obtaining the user's emotional data.

[1047] Step 6:

[1048] Dynamic Scene Adjustment

[1049] The server uses dynamic adjustment methods based on the acquired emotional data to adjust the manga scene in real time. For example, if the user expresses surprise, the server regenerates the scene to further emphasize that emotion. Specifically, the server re-inputs the prompt sentence into the generative AI model and regenerates the scene. This results in image data of the emphasized scene.

[1050] Step 7:

[1051] Enter and submit corrections

[1052] The user specifies the parts of the scene they have previewed that they want to edit by voice or text, and sends the edit instructions to the server via their device. For example, the user might say, "Make this a more intense battle scene." This sends the edit instruction data to the server.

[1053] Step 8:

[1054] Running Regeneration

[1055] The server regenerates the manga using the regeneration means based on the modification instructions and emotion data. Specifically, the server operates the image generation means again to generate a modified version of the scene. This results in image data of the modified manga.

[1056] Step 9:

[1057] Save the final manga

[1058] The server stores the final manga after correction and dynamic adjustment in the cloud. Specifically, the data of the manga is stored in a digital format using a storage means, so that the final manga data is stored safely.

[1059] Step 10:

[1060] Multilingual translation and uploading

[1061] The server translates the stored manga into multiple languages ​​using a multilingual translation means and uploads it to the sales platform using an uploading means. Specifically, the server translates the manga into English, French, etc. using a translation engine and uploads it to the online sales site. This allows the manga to be provided to users in various languages.

[1062] Step 11:

[1063] Generating additional content

[1064] The server generates additional content such as stamps and games based on the uploaded manga. Specifically, the server uses the additional content generation means to create a stamp set using the manga characters, thereby providing manga fans with additional products.

[1065] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1066] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1067] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1068] [Third embodiment]

[1069] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1070] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1071] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1072] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1073] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1074] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1075] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1076] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1077] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1078] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1079] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1080] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1081] The present invention relates to a manga production system that utilizes generation AI, and an embodiment thereof will be described in detail below.

[1082] System Overview

[1083] This system uses AI technology to automatically generate manga based on a manga scenario entered by the user, and also handles multilingual support, sales development, and the generation of additional content. The system includes user, terminal, and server components.

[1084] System Component Description

[1085] 1. Scenario input method

[1086] The user uses the terminal to input a manga scenario. The scenario is input in text format. For example, the user can input a story in which the protagonist, a high school student, is summoned to another world.

[1087] 2. Natural language analysis means

[1088] The device sends the input scenario to the server, which then analyzes it using natural language processing technology. This analysis extracts detailed information about each scene and character in the story. For example, the background, character expressions, and location of the "summoning scene" can be identified.

[1089] 3. Image Generation Method

[1090] Based on the analysis results, the server uses AI technology to automatically generate each scene in the manga. During this generation process, character designs and backgrounds are drawn. For example, an otherworldly landscape is drawn as the background for the "summoning scene."

[1091] 4. Preview Method

[1092] The image data of the generated manga is sent to the terminal, and the user can check the preview. The preview is displayed in slide format, and the user can check each page of the manga in order.

[1093] 5. Correction input method

[1094] Users are presented with an interface to view a preview and input corrections and improvements, such as specific instructions like "change the character's facial expression to look more surprised."

[1095] 6. Regeneration means

[1096] The user's correction instructions are sent to the server, and the server again uses the image generation means to generate a revised version of the manga page. For example, the facial expression of the character may be changed in accordance with the user's instructions.

[1097] 7. Preservation means

[1098] Once the manga is created to the user's satisfaction, the data is saved on the device or in the cloud. For example, the completed manga can be saved in cloud storage.

[1099] 8. Multilingual Translation Tools

[1100] The saved manga is translated into multiple languages ​​using a translation engine, allowing the same manga to be published in multiple languages, for example, English, Chinese, and Spanish.

[1101] 9. Upload Method

[1102] The translated manga data is uploaded to a sales platform and a sales page is generated, for example, the manga is listed and sold in an online store.

[1103] 10. Additional Content Generation Methods

[1104] If a manga becomes popular, the server will generate additional content such as stamps and games using its characters and scenes. For example, the main character's character will be converted into a LINE stamp.

[1105] Specific examples

[1106] User A uses the scenario input means to input a scenario he or she has created, in which "the protagonist, a high school student, is summoned to another world." The device sends the scenario data to the server, which analyzes it using natural language analysis means. The image generation means then automatically generates each page of the manga, and a preview is displayed on the device. User A checks the preview and inputs corrections to the character's facial expressions. The server generates the images again and sends the corrected versions to the device, repeating this process until User A is satisfied. When the manga is finally completed, it is saved in the cloud using the saving means and translated into multiple languages ​​using the multilingual translation means. The translated manga is uploaded to a sales platform using the uploading means, and popular characters are then sold as stamps using the additional content generation means.

[1107] This allows users to easily create high-quality manga, support multiple languages, expand sales channels, and generate additional content.

[1108] The processing flow will be explained below.

[1109] Step 1:

[1110] The user inputs a scenario. The terminal displays a text area for the user to input the scenario. The user inputs the scenario into the text box and clicks the "Submit" button when complete.

[1111] Step 2:

[1112] When the send button is clicked, the device converts the scenario data into JSON format, which is then sent to the server as an API request.

[1113] Step 3:

[1114] The server analyzes the received scenario data using natural language analysis means, and extracts story scenes and character information as the analysis results.

[1115] Step 4:

[1116] The server uses image generation tools to generate each scene of the manga based on the analysis results. AI technology is used to depict characters and backgrounds, and each page is generated as image data.

[1117] Step 5:

[1118] The server sends the image data of the generated manga page to the terminal, which then decodes the received image data and displays a preview to the user.

[1119] Step 6:

[1120] The user checks the displayed preview and specifies the corrections to be made. The terminal provides an interface for corrections, and the user inputs the corrections.

[1121] Step 7:

[1122] The terminal transmits the inputted correction instructions to the server, which receives the correction instructions and generates a corrected version using the image generating means again.

[1123] Step 8:

[1124] The server sends the image data of the corrected manga page to the terminal, which then displays the preview again for the user to reconfirm.

[1125] Step 9:

[1126] The process from step 6 to step 8 is repeated until the user is satisfied. Finally, a manga that satisfies the user is generated.

[1127] Step 10:

[1128] The user clicks the Finish button, and the terminal uses the storage means to store the final manga data in cloud storage.

[1129] Step 11:

[1130] The server translates the saved manga data using a multilingual translation tool, and it is translated into multiple languages, including English, Chinese, and Spanish.

[1131] Step 12:

[1132] The server uploads the translated manga data to the sales platform using the uploading means, and a sales page is generated and published.

[1133] Step 13:

[1134] The server analyzes the sales data and generates content such as stamps and games for popular manga using additional content generation means.

[1135] This allows users to easily create manga, make it multilingual, sell it, and expand it further.

[1136] Example 1

[1137] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1138] Conventional manga production systems have difficulty automating the entire process, from generating a manga after a user inputs a scenario to multilingual translation, sales development, and the generation of additional content. In particular, they lack the functionality to regenerate manga content in real time according to user correction instructions, and multilingual support and uploading to sales platforms are labor-intensive tasks. Furthermore, generative AI models have not been fully utilized for high-quality image generation or scenario analysis.

[1139] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1140] In this invention, the server includes a scenario input means, a natural language analysis means, an image generation means, a preview means, a correction input means, a regeneration means, a storage means, a multilingual translation means, an upload means, an additional content generation means, a cloud storage means, an analysis means that uses a generative AI model, and a prompt sentence processing means. This allows a user to simply input a scenario, automatically generating a high-quality manga, and enabling real-time regeneration in response to user correction instructions, multilingual support, sales development, and the generation of additional content.

[1141] "Scenario input means" refers to a device or software that allows a user to input a manga scenario in text format.

[1142] "Natural language analysis means" refers to a device or software that analyzes input scenario data using natural language processing technology and extracts information about each scene and character in the story.

[1143] "Image generation means" refers to a device or software for generating images of each scene and character in a manga based on data analyzed by a natural language analysis means.

[1144] The "preview means" refers to a device or software that displays the generated image so that the user can check it.

[1145] The "modification input means" refers to a device or software that provides an interface for a user to input instructions for modification or improvement of a preview.

[1146] The "regeneration means" refers to a device or software that regenerates a manga image based on the correction instructions input by the correction input means.

[1147] "Storage means" refers to a device or software that records completed manga data for long-term retention.

[1148] "Multilingual translation means" refers to a device or software for translating stored manga into multiple languages.

[1149] "Uploading means" refers to the device or software used to upload translated manga to an online sales platform and generate a sales page.

[1150] "Additional content generation means" refers to a device or software for generating additional content such as stamps and games using manga characters and scenes.

[1151] "Cloud storage means" refers to a device or software that stores generated manga in cloud storage and makes it easily accessible.

[1152] "Analysis means utilizing a generative AI model" refers to a device or software that uses a generative AI model to analyze scenario data and obtain appropriate output.

[1153] "Prompt sentence processing means" refers to a device or software for generating and using prompt sentences for scenario data to perform analysis and image generation.

[1154] The present invention relates to a manga production system that utilizes generation AI, and specific embodiments will be described in detail.

[1155] System Overview

[1156] This system uses AI technology to automatically generate manga based on a manga scenario entered by the user, and also handles multilingual support, sales development, and the generation of additional content. The system includes user, terminal, and server components.

[1157] System Component Description

[1158] 1. Scenario input method

[1159] The user uses the terminal to input a manga scenario. The scenario is input in text format. For example, the user can input a story in which the protagonist, a high school student, is summoned to another world.

[1160] 2. Natural language analysis means

[1161] The device sends the input scenario to a server, which then analyzes it using a generative AI model (e.g., GPT-4). This analysis extracts detailed information about each scene and character in the story. For example, the background, character expressions, and location of the "summoning scene" are identified.

[1162] 3. Image Generation Method

[1163] Based on the analysis results, the server uses AI technology (such as DALL-E or MidJourney) to automatically generate each scene in the manga. During this generation process, character designs and backgrounds are drawn. For example, an otherworldly landscape is drawn as the background for the "summoning scene."

[1164] 4. Preview Method

[1165] The image data of the generated manga is sent to the terminal, and the user can check the preview. The preview is displayed in slide format, and the user can check each page of the manga in order.

[1166] 5. Correction input method

[1167] Users are presented with an interface to view a preview and input corrections and improvements, such as specific instructions like "change the character's facial expression to look more surprised."

[1168] 6. Regeneration means

[1169] The user's modification instructions are sent from the terminal to the server, and the server uses the image generation means to generate a modified version of the manga page. For example, the facial expression of the character may be changed in response to the user's instructions.

[1170] 7. Preservation means

[1171] Once a manga that satisfies the user is finally created, the data is stored in cloud storage. For example, the completed manga is stored in cloud storage.

[1172] 8. Multilingual Translation Tools

[1173] The server translates the stored manga into multiple languages ​​using a translation engine (e.g., Google Translate API). This allows the same manga to be published in multiple languages, for example, English, Chinese, and Spanish.

[1174] 9. Upload Method

[1175] The translated manga data is uploaded to an online sales platform and a sales page is generated, for example, the manga is listed and sold in an online store.

[1176] 10. Additional Content Generation Methods

[1177] If a manga becomes popular, the server will generate additional content such as stamps and games using its characters and scenes. For example, the main character's character could be converted into a stamp.

[1178] Specific examples

[1179] For example, User A uses the scenario input means to input a scenario he or she has created, in which "the protagonist, a high school student, is summoned to another world." The device sends the scenario data to the server, which performs natural language analysis using a generative AI model. The image generation means then automatically generates each page of the manga, and a preview is displayed on the device. User A checks the preview and inputs corrections to the character's facial expressions. The server generates the images again and sends the corrected version to the device, repeating this process until User A is satisfied. Once the manga is finally completed, it is stored in the cloud using the storage means and translated into multiple languages ​​using the multilingual translation means. The translated manga is uploaded to a sales platform using the upload means, and popular characters are sold as stamps using the additional content generation means. This allows users to easily create high-quality manga, support multiple languages, expand sales channels, and generate additional content.

[1180] Prompt Sentence Examples

[1181] 1. "Describe a scene in which the protagonist is summoned to another world, using a fantasy-style landscape as the background."

[1182] 2. "Please make the character's expression look a little more surprised."

[1183] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1184] Step 1:

[1185] Scenario input

[1186] The user uses the device to input the manga scenario in text format. For example, they can input the content, "The protagonist, a high school student, is summoned to another world." The input scenario is saved as text data on the device.

[1187] Input: A textual scenario entered by the user.

[1188] Output: Scenario data saved on the device

[1189] Step 2:

[1190] Scenario transmission

[1191] The device sends the input scenario data to the server, efficiently transmitting the data using HTTP requests.

[1192] Input: Scenario data entered

[1193] Output: Scenario data sent to the server

[1194] Step 3:

[1195] Scenario Analysis

[1196] The server analyzes the received scenario using a generative AI model (e.g., GPT-4), which extracts key elements of the scenario (scenes, character traits, etc.).

[1197] Input: Scenario data sent to the server

[1198] Output: Information about each scene and character in the analyzed story

[1199] How it works: The generative AI model analyzes each part of the scenario and extracts the necessary text information through tokenization, grammatical analysis, and semantic analysis.

[1200] Step 4:

[1201] Image generation

[1202] The server uses AI technology (e.g., DALL-E and MidJourney) to automatically generate each scene in the manga based on the analysis results. For example, the background of the "summoning scene" is an otherworldly landscape.

[1203] Input: Information about each scene and character in the analyzed story

[1204] Output: Images of each scene in the generated manga

[1205] How it works: AI technology generates images based on text information and draws designs for each scene and character.

[1206] Step 5:

[1207] Preview Generation

[1208] The server compiles the generated images and creates preview data, which is formatted in a format that can be easily viewed by the user (slideshow format).

[1209] Input: Each scene image of the generated manga

[1210] Output: Preview data

[1211] Specific operation: The generated images are processed into a continuous slideshow format and converted into a format that the user can view sequentially.

[1212] Step 6:

[1213] Preview display

[1214] The terminal receives the preview data sent from the server and displays it to the user, who then checks the preview in slide format.

[1215] Input: Preview data

[1216] Output: Preview screen displayed on the device

[1217] Specific operation: The device loads the preview data and displays it in a slideshow format that is easy for the user to view.

[1218] Step 7:

[1219] Correction instruction input

[1220] Users can view the preview and input specific complaints and requests for corrections, such as "change the character's facial expression to look more surprised."

[1221] Input: User correction instructions

[1222] Output: Correction instruction data saved on the device

[1223] Specific operation: The terminal receives the user's input and stores it as correction instruction data.

[1224] Step 8:

[1225] Send correction instructions

[1226] The device sends the user's correction instructions to the server via an HTTP request.

[1227] Input: User correction instruction data

[1228] Output: Correction instruction data sent to the server

[1229] Specific operation: The terminal creates an HTTP request and sends correction instruction data to the server.

[1230] Step 9:

[1231] Regeneration Process

[1232] The server then uses the image generation means to generate a revised manga page based on the received revision instructions. For example, the facial expression of the character may be changed in response to the user's instructions.

[1233] Input: Correction instruction data sent to the server

[1234] Output: Regenerated manga scene images

[1235] Specific operation: Using AI technology, the regenerated image is recreated while reflecting correction instructions.

[1236] Step 10:

[1237] final save

[1238] The server stores the manga that the user is finally satisfied with in cloud storage.

[1239] Input: Regenerated manga scene images

[1240] Output: Manga data saved in cloud storage

[1241] Specific operation: Manga data is uploaded to a remote server through the API of a cloud storage service and stored for a long period of time.

[1242] Step 11:

[1243] Multilingual Translation

[1244] The server translates the stored manga data into multiple languages ​​using a translation engine (e.g., Google Translate API), allowing it to be expanded into multiple languages ​​such as English, Chinese, and Spanish.

[1245] Input: Manga data stored in cloud storage

[1246] Output: Manga data translated into multiple languages

[1247] What it does: The translation engine converts the text into each language and applies it to the appropriate manga text section.

[1248] Step 12:

[1249] Upload to sales platform

[1250] The server uploads the translated manga data to an online sales platform (e.g., Amazon Kindle Direct Publishing) and automatically generates a sales page.

[1251] Input: Manga data translated into multiple languages

[1252] Output: Sales page on sales platform

[1253] Specific operations: Use the sales platform's API to upload translated manga data and set up the product page.

[1254] Step 13:

[1255] Additional content generation

[1256] The server analyzes the popularity of the manga and generates additional content such as stamps and games based on popular characters and scenes.

[1257] Input: Manga data stored in cloud storage and popularity analysis results

[1258] Output: Generated additional content (e.g. stamps, games)

[1259] Specific operation: Generate stamps and simple game materials based on data on popular characters and scenes.

[1260] (Application example 1)

[1261] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1262] Traditional manga production methods have the problem of being time-consuming and labor-intensive, as much of the work is done manually. Additionally, processes such as supporting different languages, uploading to sales platforms, and generating additional content are performed separately, reducing overall work efficiency. There was a need for a method to solve these problems and achieve more efficient and faster manga production, translation, distribution, and additional content generation.

[1263] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1264] In this invention, the server includes a scenario input means, a natural language analysis means, an image generation means, a preview means, a correction input means, a regeneration means, a saving means, a multilingual translation means, an uploading means, an additional content generation means, a processing means using a generative AI model, and a prompt sentence generation means. This allows for a process that starts with a scenario input by a user, followed by automatic generation of manga pages using a generative AI model, natural language analysis, image generation, preview and correction, regeneration, saving, multilingual translation, uploading to a sales platform, and even automatic generation of additional content.

[1265] The "scenario input means" is an interface that allows the user to input a scenario in text format.

[1266] The "natural language analysis means" is a function that analyzes input scenario data and extracts detailed information about each scene and character in the story based on that content.

[1267] The "image generation means" is a technology for automatically generating each scene of a manga based on data analyzed by the natural language analysis means.

[1268] The "preview means" is a display means for allowing the user to check the image data of the generated manga.

[1269] The "correction input means" is an interface that allows the user to view the preview and input corrections or improvements.

[1270] The "regeneration means" is a function for regenerating an image based on correction instructions from the user.

[1271] The "storage means" is a function for saving the final generated manga data.

[1272] A "multilingual translation means" is an engine for translating stored manga into multiple different languages.

[1273] "Uploading means" is a function for uploading translated manga data to the sales platform.

[1274] The "additional content generating means" is a means for generating additional content related to the manga (for example, character stamps, etc.).

[1275] A "generative AI model" is an artificial intelligence model that generates manga pages based on a scenario provided by the user.

[1276] A "prompt sentence generation means" is a technology for generating and providing appropriate prompt sentences to a generative AI model.

[1277] This invention relates to a manga production system that utilizes generative AI. Users input a scenario, and AI technology automatically generates a manga, translating it, selling it, and generating additional content. The system includes user, terminal, and server components.

[1278] The main components of the system are as follows:

[1279] 1. Scenario input method

[1280] The user inputs the scenario using a smartphone or a head-mounted display. The scenario is input in text format.

[1281] 2. Natural language analysis means

[1282] The scenario data sent from the device to the server is analyzed using natural language processing engines such as OpenNLP to extract details about scenes and characters.

[1283] 3. Image Generation Method

[1284] Based on the results of natural language analysis, the server uses a generative AI model (e.g., a TensorFlow or GPT-3-based model) to generate images of each scene in the manga, which are then stored in cloud storage.

[1285] 4. Preview Method

[1286] The generated manga image is sent to the device, and the user can preview it from a React Native-based UI.

[1287] 5. Correction input method

[1288] Users can view the preview and use an interface to provide corrections and improvements, which is also implemented using React Native.

[1289] 6. Regeneration means

[1290] Once the correction instructions are sent to the server, the server runs the AI ​​model again to generate the corrected image.

[1291] 7. Preservation means

[1292] Once the user is finally satisfied with the manga, it is saved in cloud storage.

[1293] 8. Multilingual Translation Tools

[1294] The saved manga can be automatically translated into multiple languages ​​using a multilingual translation tool (e.g., Google Translate API).

[1295] 9. Upload Method

[1296] The translated manga is uploaded using the sales platform's API (e.g., Amazon Kindle Publisher API).

[1297] 10. Additional Content Generation Methods

[1298] If a manga becomes popular, additional content (such as stamps or games) will be generated based on the characters and scenes, also using AI technology.

[1299] Specific examples

[1300] The user inputs the scenario "An adventurous girl explores ancient ruins." This scenario is sent from the device to the server, and natural language analysis extracts the information "adventure," "ancient ruins," and "exploring girl." The generative AI model then generates an image based on the following prompt:

[1301] Example prompts

[1302] "Scenario: An adventurous girl explores ancient ruins. The first scene shows the girl standing at the entrance to the ruins, with a green jungle behind her. In this scene, the girl has a surprised expression, and a broken stone statue can be seen in the background. Generate a manga page based on this scene."

[1303] The generated image is sent to the user as a preview, and the user can input correction instructions, such as "I want the girl's expression to look more surprised." The server then runs the generative AI model again to generate a corrected image. This process is repeated until the user is finally satisfied.

[1304] This allows users to complete the entire process from entering a scenario to creating the final manga, saving, translating, selling, and generating additional content all in one system.

[1305] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1306] Step 1:

[1307] The user inputs a scenario using a smartphone or head-mounted display. This scenario is input in text format, converted to JSON format from the device, and sent to the server. The input data is the scenario thought up by the user, and the output is the JSON-format scenario data received by the server.

[1308] Step 2:

[1309] The server analyzes the received scenario data using natural language analysis methods. A natural language processing engine such as OpenNLP is used for the analysis, and detailed information about each scene and character in the story is extracted from the scenario. The input is the scenario data received by the server, and the output is detailed information about the extracted scenes and characters. Scene classifications and character characteristics are identified through data analysis processing.

[1310] Step 3:

[1311] Based on the analyzed information, the server uses an AI model (for example, a model based on TensorFlow or GPT-3) to automatically generate images of each scene in the manga. At this time, a prompt is generated and supplied to the generative AI model. The input is the extracted scene and character details, as well as the generated prompt, and the output is image data of the generated manga. The AI ​​model generates images based on the prompt, and the images are stored in cloud storage.

[1312] Step 4:

[1313] The generated manga image is sent to the device, and the user can check the preview through a React Native-based UI. Here, the user can check the generated manga page step by step. The input is the generated manga image data, and the output is the preview screen displayed on the user's device. The preview display allows the user to examine the image in detail.

[1314] Step 5:

[1315] While viewing the preview, the user uses an interface to input corrections and improvements. This interface is also implemented using React Native. The input is the corrections specified by the user, and the output is the correction instruction data sent to the server. The specific corrections are specified by the user's interface operations.

[1316] Step 6:

[1317] When the server receives the correction instructions, it runs the AI ​​model again to generate the corrected image. The input is the correction instruction data from the user, and the output is the corrected manga image data. This causes the generative AI model to run again and generate an image according to the user's corrections.

[1318] Step 7:

[1319] The manga image that the user is finally satisfied with is saved in cloud storage. The input is the final manga image data after corrections, and the output is the manga data saved in the cloud. Cloud storage ensures that the data is saved permanently.

[1320] Step 8:

[1321] The stored manga data is translated into multiple languages ​​using a multilingual translation tool (for example, Google Translate API). The input is the manga data stored in cloud storage, and the output is the translated manga data in multiple languages. Multilingual support is achieved using an AI translation engine.

[1322] Step 9:

[1323] The translated manga is uploaded online using the sales platform's API (e.g., Amazon Kindle Publisher API). The input is the translated manga data, and the output is the manga page published on the sales platform. Uploading makes it accessible to a wide range of users.

[1324] Step 10:

[1325] If a manga becomes popular, the server generates additional content based on its characters and scenes. This process is also carried out using AI technology. The input is the saved manga data, and the output is data for additional content (e.g., character stamps). The generated additional content expands the range of manga-related products.

[1326] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1327] The present invention combines an emotion engine that recognizes the user's emotions in a manga production system that utilizes generative AI, and an embodiment of this system will be described in detail below.

[1328] System Overview

[1329] This system uses AI technology to automatically generate manga based on a manga scenario entered by the user, and also handles multilingual support, sales development, and the generation of additional content. By combining it with an emotion engine, it can detect the user's emotional state and provide a more user-friendly interface and feedback. The system includes user, terminal, and server components.

[1330] System Component Description

[1331] 1. Scenario input method

[1332] The user uses the device to input a manga scenario. The scenario is input in text format. The emotion engine analyzes the user's facial expressions and voice as they input, and can obtain their emotional status. For example, when inputting a story in which the protagonist, a high school student, is summoned to another world, the user's emotions of surprise and excitement are analyzed.

[1333] 2. Natural language analysis means

[1334] The device sends the input scenario to the server, which then analyzes it using natural language processing technology to extract detailed information about each scene and character in the story.

[1335] 3. Image Generation Method

[1336] Based on the analysis results, the server uses AI technology to automatically generate each scene of the manga. The AI ​​also adjusts the generation process based on the user's emotional status. For example, if the user is excited when inputting their emotion, the generated design will emphasize action scenes.

[1337] 4. Preview Method

[1338] The image data of the generated manga is sent to the device, where the user can check a preview. During the preview, the emotion engine analyzes the user's facial expressions and voice to obtain emotional feedback. For example, if the user smiles while looking at the preview, that emotion is fed back.

[1339] 5. Correction input method

[1340] Users are presented with an interface to preview the edit and input corrections and improvements. The emotion engine analyzes the user's emotions as they input their corrections and makes suggestions accordingly. For example, if the user has a questioning expression, the engine will suggest corrections for that part.

[1341] 6. Regeneration means

[1342] The user's correction instructions are sent to the server, which then uses the image generation means to generate a revised version of the manga page. The regeneration process also takes the emotional status into consideration.

[1343] 7. Preservation means

[1344] Once a manga that satisfies the user is finally created, the data is saved on the device or in the cloud.

[1345] 8. Multilingual Translation Tools

[1346] The saved manga is translated into multiple languages ​​using a translation engine, allowing the same manga to be published in multiple languages.

[1347] 9. Upload Method

[1348] The translated manga data is uploaded to a sales platform and a sales page is generated.

[1349] 10. Additional Content Generation Methods

[1350] If the manga proves popular, the server will generate additional content such as stamps and games using its characters and scenes.

[1351] Specific examples

[1352] User B inputs a scenario he or she has thought up, "A story about a magical girl who saves the world," using the scenario input means. The device sends the scenario data to the server, which analyzes it using natural language analysis means. The image generation means then automatically generates each page of the manga, and a preview is displayed on the device. During the preview, the emotion engine analyzes User B's facial expressions and obtains his or her emotional status. If User B expresses surprise at a scene, the scene is emphasized or suggested for revision. User B inputs correction instructions, and the server re-generates images and sends the revised version to the device. Once a manga that satisfies User B has finally been generated, it is stored in the cloud using the storage means and translated into multiple languages ​​using the multilingual translation means. The translated manga is uploaded to the sales platform using the upload means, and popular characters are then converted into stamps using the additional content generation means.

[1353] This allows users to create manga using emotional feedback, providing a more intuitive and interactive production experience. It also makes it easy to support multiple languages, expand sales channels, and generate additional content, making it a valuable system for users.

[1354] The processing flow will be explained below.

[1355] Step 1:

[1356] The user inputs a scenario. The device provides the user with a text area for scenario input and the ability to analyze the user's facial expressions and voice using an emotion engine. The user inputs the scenario into the text box and clicks the "Submit" button when complete.

[1357] Step 2:

[1358] When the device receives a click of the send button, it converts the scenario data into JSON format. At the same time, the emotion engine obtains the emotional status of the user at the time of input and sends it to the server along with the scenario data.

[1359] Step 3:

[1360] The server analyzes the received scenario data and emotional status using natural language analysis. When extracting scene and character information as analysis results, the emotional status is also taken into consideration. For example, if the user is excited, a scene will be created that reflects that excitement.

[1361] Step 4:

[1362] The server uses image generation means to generate each scene of the manga based on the analysis results. During the generation process, the character's facial expressions and background are set based on the user's emotional status. For example, action scenes can be emphasized to reflect excitement.

[1363] Step 5:

[1364] The server sends the image data of the generated manga page to the device. The device decodes the received image data and displays a preview to the user. The emotion engine analyzes the user's facial expressions and voice during the preview and sends emotional feedback to the server.

[1365] Step 6:

[1366] The user checks the displayed preview and specifies the areas to be corrected. The device provides an interface for correction, and the user inputs the corrections. The emotion engine analyzes the user's emotions when inputting the corrections and makes correction suggestions as necessary.

[1367] Step 7:

[1368] The terminal transmits the inputted correction instructions and emotion feedback to the server, which receives the correction instructions and emotion feedback and generates a corrected version using the image generation means again.

[1369] Step 8:

[1370] The server sends the image data of the revised manga page to the device. The device displays the preview again, and the user can recheck the revised version. If necessary, steps 6 to 8 are repeated.

[1371] Step 9:

[1372] The process from step 6 to step 8 is repeated until the user is satisfied. When a manga that satisfies the user is finally generated, the user clicks the done button and the terminal saves the final manga data in cloud storage using the storage means.

[1373] Step 10:

[1374] The server translates the saved manga data using a multilingual translation tool, and it is translated into multiple languages, including English, Chinese, and Spanish.

[1375] Step 11:

[1376] The server uploads the translated manga data to the sales platform using the uploading means, and a sales page is generated and published.

[1377] Step 12:

[1378] The server analyzes the sales data and generates content such as stamps and games for popular manga using additional content generation means.

[1379] Step 13:

[1380] The user uses the emotion engine to check the generated stamps and game content and provide instructions for corrections if necessary. The server then repeats the generation process, and the final content is completed.

[1381] Through these steps, users can create manga using emotional feedback and enjoy an intuitive and interactive production experience. Furthermore, the system offers great value to users, as it easily supports multiple languages, expands sales channels, and allows for the creation of additional content.

[1382] Example 2

[1383] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1384] Conventional manga production systems do not reflect the user's emotional state when the user inputs a manga scenario and checks each scene of the generated manga to give instructions for correction. This makes it difficult to generate manga that is in line with the user's intentions and emotions, and the generated manga sometimes does not meet the user's expectations. Furthermore, because the user's emotions could not be taken into account when creating multilingual content or additional content, it was difficult to increase user satisfaction.

[1385] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1386] In this invention, the server includes a scenario input means, a natural language analysis means for analyzing the scenario data input from the scenario input means and the user's emotional status, and an image generation means for generating a manga image based on the data analyzed by the natural language analysis means, thereby enabling the generation of a manga that reflects the user's emotional state.

[1387] The "scenario input means" is a means for a user to input a manga scenario in text format, and also includes a function for acquiring the user's emotional status.

[1388] The "natural language analysis means" is a means for analyzing the scenario data input from the scenario input means and the user's emotional status, and extracting each element of the story (scene and character information).

[1389] "Image generation means" refers to a means for generating manga images using a generative AI model based on data analyzed by a natural language analysis means.

[1390] The "preview means" is a means for displaying the image generated by the image generating means so that the user can check it.

[1391] The "emotion feedback acquisition means" is a means for acquiring an emotional status from the user's facial expression and voice when checking the image displayed by the preview means.

[1392] The "modification input means" is a means for providing an interface for the user to input instructions for modifying the preview image, and for suggesting modifications based on the acquired emotional status.

[1393] The "regeneration means" is a means for generating a revised version of the manga by operating the image generation means again based on the correction instructions and emotion feedback input by the correction input means.

[1394] "Storage means" refers to the means for storing the final manga, including cloud storage and storage locations on the device.

[1395] The "multilingual translation means" is a means for translating the manga stored by the storage means into multiple languages.

[1396] The "uploading means" refers to the means for uploading the translated manga to the sales platform and generating a sales page.

[1397] "Additional content generation means" refers to a means for generating additional content such as stamps and games based on the characters and scenes of manga uploaded to the sales platform.

[1398] The present invention is a system that automatically generates manga using a generative AI model, and combines it with an emotion engine that detects user emotions. Specific embodiments of the present invention will be described in detail below.

[1399] System Overview

[1400] This system uses AI technology to automatically generate manga based on a manga scenario entered by the user, and handles multilingual support, sales promotion, and the generation of additional content. It detects the user's emotional state and provides an interface that allows smooth revisions based on that feedback. The system consists of user, terminal, and server components.

[1401] System Component Description

[1402] 1. Scenario input method

[1403] The user inputs the manga scenario in text format using the device. At this time, the emotion engine analyzes the user's facial expressions and voice to obtain their emotional status. For example, when inputting a story in which the protagonist, a high school student, is summoned to another world, the user's emotions of surprise and excitement are analyzed.

[1404] 2. Natural language analysis means

[1405] The device sends the input scenario data to the server, which then analyzes the received scenario using natural language processing technology (e.g., Python's NLTK library) and extracts each element of the story (scene and character information).

[1406] 3. Image Generation Method

[1407] The server automatically generates each scene of the manga using a generative AI model (e.g., GPT-4) based on the analysis results and the user's emotional status. If the user is excited, the design is adjusted to emphasize action scenes.

[1408] 4. Preview Method

[1409] The image data of the generated manga is sent to the device, and the user can check the preview via a dedicated viewer app.

[1410] 5. Means of obtaining emotional feedback

[1411] During the preview, the device uses an emotion engine to analyze the user's facial expressions and voice to obtain emotional feedback data. For example, if the user smiles while looking at the preview, that emotion is fed back.

[1412] 6. Correction input method

[1413] Users use an interface to view the preview and input corrections and improvements. The emotion engine analyzes the user's emotions and makes appropriate suggestions for corrections. For example, if a user inputs "Make this scene more action-packed," and the emotion engine detects an excited state, it instructs the server to emphasize the action scenes.

[1414] 7. Regeneration means

[1415] The terminal sends the user's correction instructions to the server, and the server regenerates the corrected version of the manga page using the image generation means. The emotional status is also taken into account in this regeneration process.

[1416] 8. Preservation means

[1417] Once a manga that satisfies the user is finally generated, the data is stored in cloud storage (e.g., AWS S3).

[1418] 9. Multilingual Translation Tools

[1419] The server translates the stored manga using a multilingual translation engine (e.g., DeepL API or Google Translate API), allowing the same manga to be published in multiple languages.

[1420] 10. Upload Method

[1421] The translated manga data is uploaded to a sales platform (e.g., Amazon Kindle Direct Publishing) and a sales page is generated.

[1422] 11. Additional Content Generation Methods

[1423] The server will generate additional content such as stamps and games using characters and scenes from popular manga, including the creation of LINE stamps featuring popular characters and the development of mini-games.

[1424] Specific usage examples and prompt sentence examples

[1425] User B inputs a scenario he or she has thought up, "A story about a young magical girl who saves the world," through a scenario input means. The device sends the scenario data to a server, which analyzes it using natural language analysis means. Manga image data is automatically generated using a generative AI model, and a preview is sent to the device. If User B smiles at the preview, that emotional data is sent to the server and reflected as an emphasis point in the next scene. User B inputs correction instructions, and the server generates new illustrations. The completed manga is saved in the cloud, then translated into multiple languages ​​using a multilingual translation means and uploaded to a sales platform. If the manga proves popular, LINE stamps of popular characters are created using an additional content generation means.

[1426] Prompt Sentence Examples

[1427] "Write a story about a high school boy who is summoned to another world and fights with magical powers. Emphasize the emotions of surprise and excitement."

[1428] "Generate a manga-style story about a young wizard girl who saves the world. Adjust the story based on the emotional feedback provided by the user."

[1429] This will allow users to enjoy an intuitive and interactive manga creation experience, and the system will also make it easy to support multiple languages ​​and generate additional content.

[1430] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1431] Specific flow of program processing

[1432] Below, we will explain the processing flow of the program of this system step by step, clearly indicating the input, output, and specific operation of each step.

[1433] Step 1: Scenario Input

[1434] The user inputs a scenario in text format using the scenario input interface of the terminal.

[1435] Input: User-entered scenario text

[1436] Output: Scenario text data and user's emotional status

[1437] Specific operation: When a user inputs "A story about a magical girl who saves the world," the emotion engine analyzes the user's facial expressions and voice in real time through the device's built-in camera and microphone, and obtains emotional status such as surprise or excitement.

[1438] Step 2: Scenario submission

[1439] The terminal transmits the input scenario data to the server.

[1440] Input: Scenario text data and emotional status

[1441] Output: Scenario data and emotional status sent to the server

[1442] Specific operation: The terminal converts the scenario data into JSON format and sends a POST request to the server using the HTTP protocol.

[1443] Step 3: Scenario analysis

[1444] The server analyzes the scenario data and the emotional status using natural language analysis means.

[1445] Input: Scenario data and emotional status sent to the server

[1446] Output: Detailed information and character information for each scene

[1447] Specific operation: The server uses Python's NLTK library to parse the text of the scenario and extract keywords for each element of the story (characters, scenes, etc.).

[1448] Step 4: Manga Generation

[1449] Based on the analysis results, the server uses a generative AI model to automatically generate each scene of the manga.

[1450] Input: Detailed information and emotional status of the analyzed scenario

[1451] Output: Generated manga image data

[1452] Specific operation: The server uses a generative AI model (e.g., GPT-4) to generate illustrations for each scene, emphasizing action scenes if the user is excited.

[1453] Step 5: Preview

[1454] The server transmits the generated manga image data to the terminal, and the user previews it.

[1455] Input: Manga image data

[1456] Output: Preview image displayed on the device

[1457] Specific operation: The server transfers the generated manga image data to the device, and the user previews it using a dedicated viewer app on the device.

[1458] Step 6: Obtaining emotional feedback

[1459] During the preview, the device analyzes the user's facial expressions and voice, obtains emotional feedback data, and sends it to the server.

[1460] Input: Facial expression and voice data of the user during preview

[1461] Output: Emotion feedback data

[1462] Specific operation: The device analyzes the user's facial expressions and voice in real time through the camera and microphone, and sends emotional status such as smile, surprise, or confusion to the server.

[1463] Step 7: Correction Instructions

[1464] The user inputs the corrections and their details using a dedicated interface, and the system makes correction suggestions based on the user's emotional status.

[1465] Input: User correction instructions and emotional feedback data

[1466] Output: Correction instruction data sent to the server

[1467] Specific operation: The user inputs "I want more action in this scene," and the server makes suggestions for revisions based on emotional feedback.

[1468] Step 8: Regenerate

[1469] The server regenerates the image based on the correction instructions and emotional feedback.

[1470] Input: User correction instructions and emotional feedback data

[1471] Output: Regenerated manga image data

[1472] Specific operation: The server generates a new illustration that reflects the correction instructions and creates a new manga image.

[1473] Step 9: Save

[1474] The server then stores the final manga on the device or in cloud storage.

[1475] Input: Final manga image data

[1476] Output: Data stored in cloud storage or on device

[1477] Specific operation: The final manga data is uploaded and saved to cloud storage such as AWS S3.

[1478] Step 10: Multilingual Translation

[1479] The server translates the stored manga into multiple languages.

[1480] Input: Saved manga image data

[1481] Output: Translated manga image data

[1482] Specific operation: The server uses the DeepL API or Google Translate API to translate the manga text into multiple languages.

[1483] Step 11: Upload

[1484] The server uploads the translated manga data to the sales platform.

[1485] Input: Manga data translated into multiple languages

[1486] Output: Manga data uploaded to the sales platform

[1487] Specific operation: The server uploads manga data to Amazon Kindle Direct Publishing or other sales platforms via API.

[1488] Step 12: Generate additional content

[1489] The server generates additional content using characters and scenes from popular manga.

[1490] Input: Sales data from the sales platform and manga character information

[1491] Output: Stamps and mini-games as additional content

[1492] Specific operations: The server develops LINE stamps incorporating popular characters and mini-games using Unity.

[1493] The above is the specific flow and operation of each processing step in this system.

[1494] (Application example 2)

[1495] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1496] Conventional manga production systems lacked the technology to analyze user emotions in real time and dynamically adjust the scenes displayed during preview. This made it difficult for users to create manga that accurately reflected their emotions, and there was room for improvement in the quality of manga and user satisfaction. Despite the demand for providing an interactive viewing experience, especially in physical stores, these technical challenges had not been resolved.

[1497] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1498] In this invention, the server includes a scenario input means, a natural language analysis means, an image generation means, a preview means, an emotion analysis means, a dynamic adjustment means, a correction input means, a regeneration means, a storage means, a multilingual translation means, an upload means, and an additional content generation means. This makes it possible to analyze a user's emotions in real time and dynamically adjust manga scenes based on those emotions. Furthermore, it is possible to realize an interactive manga viewing experience in a physical store and improve user satisfaction.

[1499] The "scenario input means" is a device or interface that allows a user to input a manga scenario in text or audio format.

[1500] The "natural language analysis means" is a technology for analyzing the scenario data input from the scenario input means and extracting detailed information about each scene and character in the story.

[1501] The "image generation means" is a device or program for automatically generating each scene of the manga based on the data analyzed by the natural language analysis means.

[1502] The "preview means" is a device or interface that displays the manga image generated by the image generation means to the user and allows the user to view it.

[1503] The "emotion analysis means" is a technology for analyzing the facial expressions and voice of a user viewing a manga using the preview means, and obtaining the user's emotional status in real time.

[1504] The "dynamic adjustment means" is a technique for dynamically adjusting manga scenes based on the emotion data analyzed by the emotion analysis means.

[1505] The "modification input means" is a device or interface for inputting a user's instructions for modifying the image displayed by the preview means.

[1506] The "regeneration means" is a technique for generating a revised version of the manga by operating the image generation means again based on the correction instructions and emotion data input by the correction input means.

[1507] The "storage means" is a device or program for storing the final manga generated by the reproduction means in a digital format.

[1508] The "multilingual translation means" is a technology for translating the manga stored by the storage means into multiple languages.

[1509] The "uploading means" is a technology for uploading the manga translated by the multilingual translation means to a sales platform and publishing or selling it.

[1510] The "additional content generation means" is a technology for generating additional content such as stamps and games based on characters and scenes from the manga uploaded by the upload means.

[1511] This invention relates to an interactive manga production and viewing system that utilizes generative AI models, and in particular provides technology that analyzes user emotional feedback in real time in physical stores and dynamically adjusts manga scenes.

[1512] System Overview

[1513] The system consists of the following components:

[1514] 1. Scenario input means: A device or interface that allows the user to input the manga scenario in text or voice format.

[1515] 2. Natural language analysis method: Technology that analyzes input scenario data and extracts detailed information about each scene and character in the story.

[1516] 3. Image generation method: Technology that automatically generates each scene of a manga based on analyzed data.

[1517] 4. Preview means: An interface that displays and allows users to view the generated manga image.

[1518] 5. Emotion analysis means: Technology that analyzes the user's facial expressions and voice to obtain the user's emotional status in real time.

[1519] 6. Dynamic Adjustment Method: A technique for dynamically adjusting manga scenes based on emotion data.

[1520] 7. Correction input means: An interface for inputting user correction instructions for the displayed image.

[1521] 8. Regeneration method: Technology to regenerate manga based on correction instructions and emotion data.

[1522] 9. Storage means: A device or program that stores the final manga in digital form.

[1523] 10. Multilingual translation tools: Technology for translating archived manga into multiple languages.

[1524] 11. Uploading method: The technology to upload the translated manga to the sales platform.

[1525] 12. Additional content generation means: Technology for generating additional content for uploaded manga.

[1526] Specific operation of the system

[1527] The system generates manga through the following process, providing users with a personalized viewing experience:

[1528] 1. Scenario input and analysis:

[1529] The user uses smart glasses to input the manga scenario by voice.

[1530] The terminal transmits the input scenario data to the server, and the server analyzes the scenario using natural language analysis means.

[1531] 2. Automatic Manga Generation:

[1532] The server automatically generates each scene of the manga using image generation methods based on the analysis results. For example, OpenAI's GPT-3 is used as the generation AI model.

[1533] 3. Preview and Sentiment Analysis:

[1534] The terminal displays the generated manga on the smart glasses for the user to preview.

[1535] Using the camera and microphone built into the smart glasses, the emotion analysis means analyzes the user's facial expressions and voice to obtain the user's emotional status.

[1536] 4. Dynamic Adjustment:

[1537] Based on the emotion analysis results, the server reactivates the image generation means using a dynamic adjustment means to adjust the manga scenes in real time.

[1538] 5. Fix and Regenerate:

[1539] The user specifies the part to be corrected by voice or text and sends the correction instruction to the server.

[1540] The server operates the regeneration means based on the correction instructions and emotion data to generate a revised version of the manga.

[1541] 6. Save, translate and upload:

[1542] The server then stores the final manga in the cloud.

[1543] The multilingual translation means translates the stored manga into multiple languages ​​and uploads it to the sales platform using the upload means.

[1544] 7. Generate additional content:

[1545] The server generates additional content such as stamps and games based on the uploaded manga.

[1546] Specific examples

[1547] When a user uses smart glasses to speak the scenario "A story about a hero who defeats a dragon," the system operates in the following process:

[1548] 1. Scenario input: Voice input: "Please generate a scene in which a hero defeats a dragon. Please emphasize the surprising elements."

[1549] 2. Analysis and automatic generation: The server analyzes the scenario and automatically generates scenes using image generation means.

[1550] 3. Preview and Emotion Analysis: The user previews the video, and the smart glasses' camera and microphone analyze the user's facial expressions and voice.

[1551] 4. Dynamic Adjustment: The scene is dynamically adjusted based on the sentiment analysis results. If the user expresses surprise, the scene is regenerated to emphasize the surprise element more.

[1552] In this way, it is possible to provide an interactive manga viewing experience that can reflect the user's emotions in real time.

[1553] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1554] Step 1:

[1555] Entering and sending a scenario

[1556] The user inputs the scenario in voice or text format. Specifically, the user inputs through the smart glasses, "Please generate a scene in which the hero defeats the dragon. Please emphasize the surprising elements in particular." The input scenario data is sent to the server by the device. This provides the server with the input data to begin analysis.

[1557] Step 2:

[1558] Scenario data analysis

[1559] The server then uses natural language analysis to analyze the received scenario data. During the analysis, detailed information about each scene and character in the story is extracted. Specifically, elements such as "hero," "dragon," and "battle scene" are extracted from the scenario. This provides the basic data needed to generate manga scenes.

[1560] Step 3:

[1561] Manga scene generation

[1562] The server uses an image generation method based on the analyzed data to automatically generate each scene of the manga. This uses a generative AI model (e.g., OpenAI GPT-3). Specifically, the server generates the scene using the prompt "Generate a scene in which a hero defeats a dragon. Please emphasize the surprising elements in particular." as input. The output is image data of the generated manga.

[1563] Step 4:

[1564] Show Preview

[1565] The device displays the generated manga image data on the smart glasses, allowing the user to preview it. Specifically, the user views the manga through the smart glasses, allowing the user to check the generated scenes in real time.

[1566] Step 5:

[1567] Emotion analysis and capture

[1568] Using the camera and microphone built into the smart glasses, the emotion analysis means analyzes the user's facial expressions and voice and obtains their emotional status in real time. Specifically, when surprise is detected from the user's facial expression, the data is sent to the server, thereby obtaining the user's emotional data.

[1569] Step 6:

[1570] Dynamic Scene Adjustment

[1571] The server uses dynamic adjustment methods based on the acquired emotional data to adjust the manga scene in real time. For example, if the user expresses surprise, the server regenerates the scene to further emphasize that emotion. Specifically, the server re-inputs the prompt sentence into the generative AI model and regenerates the scene. This results in image data of the emphasized scene.

[1572] Step 7:

[1573] Enter and submit corrections

[1574] The user specifies the parts of the scene they have previewed that they want to edit by voice or text, and sends the edit instructions to the server via their device. For example, the user might say, "Make this a more intense battle scene." This sends the edit instruction data to the server.

[1575] Step 8:

[1576] Running Regeneration

[1577] The server regenerates the manga using the regeneration means based on the modification instructions and emotion data. Specifically, the server operates the image generation means again to generate a modified version of the scene. This results in image data of the modified manga.

[1578] Step 9:

[1579] Save the final manga

[1580] The server stores the final manga after correction and dynamic adjustment in the cloud. Specifically, the data of the manga is stored in a digital format using a storage means, so that the final manga data is stored safely.

[1581] Step 10:

[1582] Multilingual translation and uploading

[1583] The server translates the stored manga into multiple languages ​​using a multilingual translation means and uploads it to the sales platform using an uploading means. Specifically, the server translates the manga into English, French, etc. using a translation engine and uploads it to the online sales site. This allows the manga to be provided to users in various languages.

[1584] Step 11:

[1585] Generating additional content

[1586] The server generates additional content such as stamps and games based on the uploaded manga. Specifically, the server uses the additional content generation means to create a stamp set using the manga characters, thereby providing manga fans with additional products.

[1587] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1588] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1589] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1590] [Fourth embodiment]

[1591] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1592] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1593] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1594] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1595] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1596] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1597] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1598] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1599] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1600] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1601] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1602] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1603] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1604] The present invention relates to a manga production system that utilizes generation AI, and an embodiment thereof will be described in detail below.

[1605] System Overview

[1606] This system uses AI technology to automatically generate manga based on a manga scenario entered by the user, and also handles multilingual support, sales development, and the generation of additional content. The system includes user, terminal, and server components.

[1607] System Component Description

[1608] 1. Scenario input method

[1609] The user uses the terminal to input a manga scenario. The scenario is input in text format. For example, the user can input a story in which the protagonist, a high school student, is summoned to another world.

[1610] 2. Natural language analysis means

[1611] The device sends the input scenario to the server, which then analyzes it using natural language processing technology. This analysis extracts detailed information about each scene and character in the story. For example, the background, character expressions, and location of the "summoning scene" can be identified.

[1612] 3. Image Generation Method

[1613] Based on the analysis results, the server uses AI technology to automatically generate each scene in the manga. During this generation process, character designs and backgrounds are drawn. For example, an otherworldly landscape is drawn as the background for the "summoning scene."

[1614] 4. Preview Method

[1615] The image data of the generated manga is sent to the terminal, and the user can check the preview. The preview is displayed in slide format, and the user can check each page of the manga in order.

[1616] 5. Correction input method

[1617] Users are presented with an interface to view a preview and input corrections and improvements, such as specific instructions like "change the character's facial expression to look more surprised."

[1618] 6. Regeneration means

[1619] The user's correction instructions are sent to the server, and the server again uses the image generation means to generate a revised version of the manga page. For example, the facial expression of the character may be changed in accordance with the user's instructions.

[1620] 7. Preservation means

[1621] Once the manga is created to the user's satisfaction, the data is saved on the device or in the cloud. For example, the completed manga can be saved in cloud storage.

[1622] 8. Multilingual Translation Tools

[1623] The saved manga is translated into multiple languages ​​using a translation engine, allowing the same manga to be published in multiple languages, for example, English, Chinese, and Spanish.

[1624] 9. Upload Method

[1625] The translated manga data is uploaded to a sales platform and a sales page is generated, for example, the manga is listed and sold in an online store.

[1626] 10. Additional Content Generation Methods

[1627] If a manga becomes popular, the server will generate additional content such as stamps and games using its characters and scenes. For example, the main character's character will be converted into a LINE stamp.

[1628] Specific examples

[1629] User A uses the scenario input means to input a scenario he or she has created, in which "the protagonist, a high school student, is summoned to another world." The device sends the scenario data to the server, which analyzes it using natural language analysis means. The image generation means then automatically generates each page of the manga, and a preview is displayed on the device. User A checks the preview and inputs corrections to the character's facial expressions. The server generates the images again and sends the corrected versions to the device, repeating this process until User A is satisfied. When the manga is finally completed, it is saved in the cloud using the saving means and translated into multiple languages ​​using the multilingual translation means. The translated manga is uploaded to a sales platform using the uploading means, and popular characters are then sold as stamps using the additional content generation means.

[1630] This allows users to easily create high-quality manga, support multiple languages, expand sales channels, and generate additional content.

[1631] The processing flow will be explained below.

[1632] Step 1:

[1633] The user inputs a scenario. The terminal displays a text area for the user to input the scenario. The user inputs the scenario into the text box and clicks the "Submit" button when complete.

[1634] Step 2:

[1635] When the send button is clicked, the device converts the scenario data into JSON format, which is then sent to the server as an API request.

[1636] Step 3:

[1637] The server analyzes the received scenario data using natural language analysis means, and extracts story scenes and character information as the analysis results.

[1638] Step 4:

[1639] The server uses image generation tools to generate each scene of the manga based on the analysis results. AI technology is used to depict characters and backgrounds, and each page is generated as image data.

[1640] Step 5:

[1641] The server sends the image data of the generated manga page to the terminal, which then decodes the received image data and displays a preview to the user.

[1642] Step 6:

[1643] The user checks the displayed preview and specifies the corrections to be made. The terminal provides an interface for corrections, and the user inputs the corrections.

[1644] Step 7:

[1645] The terminal transmits the inputted correction instructions to the server, which receives the correction instructions and generates a corrected version using the image generating means again.

[1646] Step 8:

[1647] The server sends the image data of the corrected manga page to the terminal, which then displays the preview again for the user to reconfirm.

[1648] Step 9:

[1649] The process from step 6 to step 8 is repeated until the user is satisfied. Finally, a manga that satisfies the user is generated.

[1650] Step 10:

[1651] The user clicks the Finish button, and the terminal uses the storage means to store the final manga data in cloud storage.

[1652] Step 11:

[1653] The server translates the saved manga data using a multilingual translation tool, and it is translated into multiple languages, including English, Chinese, and Spanish.

[1654] Step 12:

[1655] The server uploads the translated manga data to the sales platform using the uploading means, and a sales page is generated and published.

[1656] Step 13:

[1657] The server analyzes the sales data and generates content such as stamps and games for popular manga using additional content generation means.

[1658] This allows users to easily create manga, make it multilingual, sell it, and expand it further.

[1659] Example 1

[1660] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1661] Conventional manga production systems have difficulty automating the entire process, from generating a manga after a user inputs a scenario to multilingual translation, sales development, and the generation of additional content. In particular, they lack the functionality to regenerate manga content in real time according to user correction instructions, and multilingual support and uploading to sales platforms are labor-intensive tasks. Furthermore, generative AI models have not been fully utilized for high-quality image generation or scenario analysis.

[1662] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1663] In this invention, the server includes a scenario input means, a natural language analysis means, an image generation means, a preview means, a correction input means, a regeneration means, a storage means, a multilingual translation means, an upload means, an additional content generation means, a cloud storage means, an analysis means that uses a generative AI model, and a prompt sentence processing means. This allows a user to simply input a scenario, automatically generating a high-quality manga, and enabling real-time regeneration in response to user correction instructions, multilingual support, sales development, and the generation of additional content.

[1664] "Scenario input means" refers to a device or software that allows a user to input a manga scenario in text format.

[1665] "Natural language analysis means" refers to a device or software that analyzes input scenario data using natural language processing technology and extracts information about each scene and character in the story.

[1666] "Image generation means" refers to a device or software for generating images of each scene and character in a manga based on data analyzed by a natural language analysis means.

[1667] The "preview means" refers to a device or software that displays the generated image so that the user can check it.

[1668] The "modification input means" refers to a device or software that provides an interface for a user to input instructions for modification or improvement of a preview.

[1669] The "regeneration means" refers to a device or software that regenerates a manga image based on the correction instructions input by the correction input means.

[1670] "Storage means" refers to a device or software that records completed manga data for long-term retention.

[1671] "Multilingual translation means" refers to a device or software for translating stored manga into multiple languages.

[1672] "Uploading means" refers to the device or software used to upload translated manga to an online sales platform and generate a sales page.

[1673] "Additional content generation means" refers to a device or software for generating additional content such as stamps and games using manga characters and scenes.

[1674] "Cloud storage means" refers to a device or software that stores generated manga in cloud storage and makes it easily accessible.

[1675] "Analysis means utilizing a generative AI model" refers to a device or software that uses a generative AI model to analyze scenario data and obtain appropriate output.

[1676] "Prompt sentence processing means" refers to a device or software for generating and using prompt sentences for scenario data to perform analysis and image generation.

[1677] The present invention relates to a manga production system that utilizes generation AI, and specific embodiments will be described in detail.

[1678] System Overview

[1679] This system uses AI technology to automatically generate manga based on a manga scenario entered by the user, and also handles multilingual support, sales development, and the generation of additional content. The system includes user, terminal, and server components.

[1680] System Component Description

[1681] 1. Scenario input method

[1682] The user uses the terminal to input a manga scenario. The scenario is input in text format. For example, the user can input a story in which the protagonist, a high school student, is summoned to another world.

[1683] 2. Natural language analysis means

[1684] The device sends the input scenario to a server, which then analyzes it using a generative AI model (e.g., GPT-4). This analysis extracts detailed information about each scene and character in the story. For example, the background, character expressions, and location of the "summoning scene" are identified.

[1685] 3. Image Generation Method

[1686] Based on the analysis results, the server uses AI technology (such as DALL-E or MidJourney) to automatically generate each scene in the manga. During this generation process, character designs and backgrounds are drawn. For example, an otherworldly landscape is drawn as the background for the "summoning scene."

[1687] 4. Preview Method

[1688] The image data of the generated manga is sent to the terminal, and the user can check the preview. The preview is displayed in slide format, and the user can check each page of the manga in order.

[1689] 5. Correction input method

[1690] Users are presented with an interface to view a preview and input corrections and improvements, such as specific instructions like "change the character's facial expression to look more surprised."

[1691] 6. Regeneration means

[1692] The user's modification instructions are sent from the terminal to the server, and the server uses the image generation means to generate a modified version of the manga page. For example, the facial expression of the character may be changed in response to the user's instructions.

[1693] 7. Preservation means

[1694] Once a manga that satisfies the user is finally created, the data is stored in cloud storage. For example, the completed manga is stored in cloud storage.

[1695] 8. Multilingual Translation Tools

[1696] The server translates the stored manga into multiple languages ​​using a translation engine (e.g., Google Translate API). This allows the same manga to be published in multiple languages, for example, English, Chinese, and Spanish.

[1697] 9. Upload Method

[1698] The translated manga data is uploaded to an online sales platform and a sales page is generated, for example, the manga is listed and sold in an online store.

[1699] 10. Additional Content Generation Methods

[1700] If a manga becomes popular, the server will generate additional content such as stamps and games using its characters and scenes. For example, the main character's character could be converted into a stamp.

[1701] Specific examples

[1702] For example, User A uses the scenario input means to input a scenario he or she has created, in which "the protagonist, a high school student, is summoned to another world." The device sends the scenario data to the server, which performs natural language analysis using a generative AI model. The image generation means then automatically generates each page of the manga, and a preview is displayed on the device. User A checks the preview and inputs corrections to the character's facial expressions. The server generates the images again and sends the corrected version to the device, repeating this process until User A is satisfied. Once the manga is finally completed, it is stored in the cloud using the storage means and translated into multiple languages ​​using the multilingual translation means. The translated manga is uploaded to a sales platform using the upload means, and popular characters are sold as stamps using the additional content generation means. This allows users to easily create high-quality manga, support multiple languages, expand sales channels, and generate additional content.

[1703] Prompt Sentence Examples

[1704] 1. "Describe a scene in which the protagonist is summoned to another world, using a fantasy-style landscape as the background."

[1705] 2. "Please make the character's expression look a little more surprised."

[1706] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1707] Step 1:

[1708] Scenario input

[1709] The user uses the device to input the manga scenario in text format. For example, they can input the content, "The protagonist, a high school student, is summoned to another world." The input scenario is saved as text data on the device.

[1710] Input: A textual scenario entered by the user.

[1711] Output: Scenario data saved on the device

[1712] Step 2:

[1713] Scenario transmission

[1714] The device sends the input scenario data to the server, efficiently transmitting the data using HTTP requests.

[1715] Input: Scenario data entered

[1716] Output: Scenario data sent to the server

[1717] Step 3:

[1718] Scenario Analysis

[1719] The server analyzes the received scenario using a generative AI model (e.g., GPT-4), which extracts key elements of the scenario (scenes, character traits, etc.).

[1720] Input: Scenario data sent to the server

[1721] Output: Information about each scene and character in the analyzed story

[1722] How it works: The generative AI model analyzes each part of the scenario and extracts the necessary text information through tokenization, grammatical analysis, and semantic analysis.

[1723] Step 4:

[1724] Image generation

[1725] The server uses AI technology (e.g., DALL-E and MidJourney) to automatically generate each scene in the manga based on the analysis results. For example, the background of the "summoning scene" is an otherworldly landscape.

[1726] Input: Information about each scene and character in the analyzed story

[1727] Output: Images of each scene in the generated manga

[1728] How it works: AI technology generates images based on text information and draws designs for each scene and character.

[1729] Step 5:

[1730] Preview Generation

[1731] The server compiles the generated images and creates preview data, which is formatted in a format that can be easily viewed by the user (slideshow format).

[1732] Input: Each scene image of the generated manga

[1733] Output: Preview data

[1734] Specific operation: The generated images are processed into a continuous slideshow format and converted into a format that the user can view sequentially.

[1735] Step 6:

[1736] Preview display

[1737] The terminal receives the preview data sent from the server and displays it to the user, who then checks the preview in slide format.

[1738] Input: Preview data

[1739] Output: Preview screen displayed on the device

[1740] Specific operation: The device loads the preview data and displays it in a slideshow format that is easy for the user to view.

[1741] Step 7:

[1742] Correction instruction input

[1743] Users can view the preview and input specific complaints and requests for corrections, such as "change the character's facial expression to look more surprised."

[1744] Input: User correction instructions

[1745] Output: Correction instruction data saved on the device

[1746] Specific operation: The terminal receives the user's input and stores it as correction instruction data.

[1747] Step 8:

[1748] Send correction instructions

[1749] The device sends the user's correction instructions to the server via an HTTP request.

[1750] Input: User correction instruction data

[1751] Output: Correction instruction data sent to the server

[1752] Specific operation: The terminal creates an HTTP request and sends correction instruction data to the server.

[1753] Step 9:

[1754] Regeneration Process

[1755] The server then uses the image generation means to generate a revised manga page based on the received revision instructions. For example, the facial expression of the character may be changed in response to the user's instructions.

[1756] Input: Correction instruction data sent to the server

[1757] Output: Regenerated manga scene images

[1758] Specific operation: Using AI technology, the regenerated image is recreated while reflecting correction instructions.

[1759] Step 10:

[1760] final save

[1761] The server stores the manga that the user is finally satisfied with in cloud storage.

[1762] Input: Regenerated manga scene images

[1763] Output: Manga data saved in cloud storage

[1764] Specific operation: Manga data is uploaded to a remote server through the API of a cloud storage service and stored for a long period of time.

[1765] Step 11:

[1766] Multilingual Translation

[1767] The server translates the stored manga data into multiple languages ​​using a translation engine (e.g., Google Translate API), allowing it to be expanded into multiple languages ​​such as English, Chinese, and Spanish.

[1768] Input: Manga data stored in cloud storage

[1769] Output: Manga data translated into multiple languages

[1770] What it does: The translation engine converts the text into each language and applies it to the appropriate manga text section.

[1771] Step 12:

[1772] Upload to sales platform

[1773] The server uploads the translated manga data to an online sales platform (e.g., Amazon Kindle Direct Publishing) and automatically generates a sales page.

[1774] Input: Manga data translated into multiple languages

[1775] Output: Sales page on sales platform

[1776] Specific operations: Use the sales platform's API to upload translated manga data and set up the product page.

[1777] Step 13:

[1778] Additional content generation

[1779] The server analyzes the popularity of the manga and generates additional content such as stamps and games based on popular characters and scenes.

[1780] Input: Manga data stored in cloud storage and popularity analysis results

[1781] Output: Generated additional content (e.g. stamps, games)

[1782] Specific operation: Generate stamps and simple game materials based on data on popular characters and scenes.

[1783] (Application example 1)

[1784] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1785] Traditional manga production methods have the problem of being time-consuming and labor-intensive, as much of the work is done manually. Additionally, processes such as supporting different languages, uploading to sales platforms, and generating additional content are performed separately, reducing overall work efficiency. There was a need for a method to solve these problems and achieve more efficient and faster manga production, translation, distribution, and additional content generation.

[1786] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1787] In this invention, the server includes a scenario input means, a natural language analysis means, an image generation means, a preview means, a correction input means, a regeneration means, a saving means, a multilingual translation means, an uploading means, an additional content generation means, a processing means using a generative AI model, and a prompt sentence generation means. This allows for a process that starts with a scenario input by a user, followed by automatic generation of manga pages using a generative AI model, natural language analysis, image generation, preview and correction, regeneration, saving, multilingual translation, uploading to a sales platform, and even automatic generation of additional content.

[1788] The "scenario input means" is an interface that allows the user to input a scenario in text format.

[1789] The "natural language analysis means" is a function that analyzes input scenario data and extracts detailed information about each scene and character in the story based on that content.

[1790] The "image generation means" is a technology for automatically generating each scene of a manga based on data analyzed by the natural language analysis means.

[1791] The "preview means" is a display means for allowing the user to check the image data of the generated manga.

[1792] The "correction input means" is an interface that allows the user to view the preview and input corrections or improvements.

[1793] The "regeneration means" is a function for regenerating an image based on correction instructions from the user.

[1794] The "storage means" is a function for saving the final generated manga data.

[1795] A "multilingual translation means" is an engine for translating stored manga into multiple different languages.

[1796] "Uploading means" is a function for uploading translated manga data to the sales platform.

[1797] The "additional content generating means" is a means for generating additional content related to the manga (for example, character stamps, etc.).

[1798] A "generative AI model" is an artificial intelligence model that generates manga pages based on a scenario provided by the user.

[1799] A "prompt sentence generation means" is a technology for generating and providing appropriate prompt sentences to a generative AI model.

[1800] This invention relates to a manga production system that utilizes generative AI. Users input a scenario, and AI technology automatically generates a manga, translating it, selling it, and generating additional content. The system includes user, terminal, and server components.

[1801] The main components of the system are as follows:

[1802] 1. Scenario input method

[1803] The user inputs the scenario using a smartphone or a head-mounted display. The scenario is input in text format.

[1804] 2. Natural language analysis means

[1805] The scenario data sent from the device to the server is analyzed using natural language processing engines such as OpenNLP to extract details about scenes and characters.

[1806] 3. Image Generation Method

[1807] Based on the results of natural language analysis, the server uses a generative AI model (e.g., a TensorFlow or GPT-3-based model) to generate images of each scene in the manga, which are then stored in cloud storage.

[1808] 4. Preview Method

[1809] The generated manga image is sent to the device, and the user can preview it from a React Native-based UI.

[1810] 5. Correction input method

[1811] Users can view the preview and use an interface to provide corrections and improvements, which is also implemented using React Native.

[1812] 6. Regeneration means

[1813] Once the correction instructions are sent to the server, the server runs the AI ​​model again to generate the corrected image.

[1814] 7. Preservation means

[1815] Once the user is finally satisfied with the manga, it is saved in cloud storage.

[1816] 8. Multilingual Translation Tools

[1817] The saved manga can be automatically translated into multiple languages ​​using a multilingual translation tool (e.g., Google Translate API).

[1818] 9. Upload Method

[1819] The translated manga is uploaded using the sales platform's API (e.g., Amazon Kindle Publisher API).

[1820] 10. Additional Content Generation Methods

[1821] If a manga becomes popular, additional content (such as stamps or games) will be generated based on the characters and scenes, also using AI technology.

[1822] Specific examples

[1823] The user inputs the scenario "An adventurous girl explores ancient ruins." This scenario is sent from the device to the server, and natural language analysis extracts the information "adventure," "ancient ruins," and "exploring girl." The generative AI model then generates an image based on the following prompt:

[1824] Example prompts

[1825] "Scenario: An adventurous girl explores ancient ruins. The first scene shows the girl standing at the entrance to the ruins, with a green jungle behind her. In this scene, the girl has a surprised expression, and a broken stone statue can be seen in the background. Generate a manga page based on this scene."

[1826] The generated image is sent to the user as a preview, and the user can input correction instructions, such as "I want the girl's expression to look more surprised." The server then runs the generative AI model again to generate a corrected image. This process is repeated until the user is finally satisfied.

[1827] This allows users to complete the entire process from entering a scenario to creating the final manga, saving, translating, selling, and generating additional content all in one system.

[1828] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1829] Step 1:

[1830] The user inputs a scenario using a smartphone or head-mounted display. This scenario is input in text format, converted to JSON format from the device, and sent to the server. The input data is the scenario thought up by the user, and the output is the JSON-format scenario data received by the server.

[1831] Step 2:

[1832] The server analyzes the received scenario data using natural language analysis methods. A natural language processing engine such as OpenNLP is used for the analysis, and detailed information about each scene and character in the story is extracted from the scenario. The input is the scenario data received by the server, and the output is detailed information about the extracted scenes and characters. Scene classifications and character characteristics are identified through data analysis processing.

[1833] Step 3:

[1834] Based on the analyzed information, the server uses an AI model (for example, a model based on TensorFlow or GPT-3) to automatically generate images of each scene in the manga. At this time, a prompt is generated and supplied to the generative AI model. The input is the extracted scene and character details, as well as the generated prompt, and the output is image data of the generated manga. The AI ​​model generates images based on the prompt, and the images are stored in cloud storage.

[1835] Step 4:

[1836] The generated manga image is sent to the device, and the user can check the preview through a React Native-based UI. Here, the user can check the generated manga page step by step. The input is the generated manga image data, and the output is the preview screen displayed on the user's device. The preview display allows the user to examine the image in detail.

[1837] Step 5:

[1838] While viewing the preview, the user uses an interface to input corrections and improvements. This interface is also implemented using React Native. The input is the corrections specified by the user, and the output is the correction instruction data sent to the server. The specific corrections are specified by the user's interface operations.

[1839] Step 6:

[1840] When the server receives the correction instructions, it runs the AI ​​model again to generate the corrected image. The input is the correction instruction data from the user, and the output is the corrected manga image data. This causes the generative AI model to run again and generate an image according to the user's corrections.

[1841] Step 7:

[1842] The manga image that the user is finally satisfied with is saved in cloud storage. The input is the final manga image data after corrections, and the output is the manga data saved in the cloud. Cloud storage ensures that the data is saved permanently.

[1843] Step 8:

[1844] The stored manga data is translated into multiple languages ​​using a multilingual translation tool (for example, Google Translate API). The input is the manga data stored in cloud storage, and the output is the translated manga data in multiple languages. Multilingual support is achieved using an AI translation engine.

[1845] Step 9:

[1846] The translated manga is uploaded online using the sales platform's API (e.g., Amazon Kindle Publisher API). The input is the translated manga data, and the output is the manga page published on the sales platform. Uploading makes it accessible to a wide range of users.

[1847] Step 10:

[1848] If a manga becomes popular, the server generates additional content based on its characters and scenes. This process is also carried out using AI technology. The input is the saved manga data, and the output is data for additional content (e.g., character stamps). The generated additional content expands the range of manga-related products.

[1849] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1850] The present invention combines an emotion engine that recognizes the user's emotions in a manga production system that utilizes generative AI, and an embodiment of this system will be described in detail below.

[1851] System Overview

[1852] This system uses AI technology to automatically generate manga based on a manga scenario entered by the user, and also handles multilingual support, sales development, and the generation of additional content. By combining it with an emotion engine, it can detect the user's emotional state and provide a more user-friendly interface and feedback. The system includes user, terminal, and server components.

[1853] System Component Description

[1854] 1. Scenario input method

[1855] The user uses the device to input a manga scenario. The scenario is input in text format. The emotion engine analyzes the user's facial expressions and voice as they input, and can obtain their emotional status. For example, when inputting a story in which the protagonist, a high school student, is summoned to another world, the user's emotions of surprise and excitement are analyzed.

[1856] 2. Natural language analysis means

[1857] The device sends the input scenario to the server, which then analyzes it using natural language processing technology to extract detailed information about each scene and character in the story.

[1858] 3. Image Generation Method

[1859] Based on the analysis results, the server uses AI technology to automatically generate each scene of the manga. The AI ​​also adjusts the generation process based on the user's emotional status. For example, if the user is excited when inputting their emotion, the generated design will emphasize action scenes.

[1860] 4. Preview Method

[1861] The image data of the generated manga is sent to the device, where the user can check a preview. During the preview, the emotion engine analyzes the user's facial expressions and voice to obtain emotional feedback. For example, if the user smiles while looking at the preview, that emotion is fed back.

[1862] 5. Correction input method

[1863] Users are presented with an interface to preview the edit and input corrections and improvements. The emotion engine analyzes the user's emotions as they input their corrections and makes suggestions accordingly. For example, if the user has a questioning expression, the engine will suggest corrections for that part.

[1864] 6. Regeneration means

[1865] The user's correction instructions are sent to the server, which then uses the image generation means to generate a revised version of the manga page. The regeneration process also takes the emotional status into consideration.

[1866] 7. Preservation means

[1867] Once a manga that satisfies the user is finally created, the data is saved on the device or in the cloud.

[1868] 8. Multilingual Translation Tools

[1869] The saved manga is translated into multiple languages ​​using a translation engine, allowing the same manga to be published in multiple languages.

[1870] 9. Upload Method

[1871] The translated manga data is uploaded to a sales platform and a sales page is generated.

[1872] 10. Additional Content Generation Methods

[1873] If the manga proves popular, the server will generate additional content such as stamps and games using its characters and scenes.

[1874] Specific examples

[1875] User B inputs a scenario he or she has thought up, "A story about a magical girl who saves the world," using the scenario input means. The device sends the scenario data to the server, which analyzes it using natural language analysis means. The image generation means then automatically generates each page of the manga, and a preview is displayed on the device. During the preview, the emotion engine analyzes User B's facial expressions and obtains his or her emotional status. If User B expresses surprise at a scene, the scene is emphasized or suggested for revision. User B inputs correction instructions, and the server re-generates images and sends the revised version to the device. Once a manga that satisfies User B has finally been generated, it is stored in the cloud using the storage means and translated into multiple languages ​​using the multilingual translation means. The translated manga is uploaded to the sales platform using the upload means, and popular characters are then converted into stamps using the additional content generation means.

[1876] This allows users to create manga using emotional feedback, providing a more intuitive and interactive production experience. It also makes it easy to support multiple languages, expand sales channels, and generate additional content, making it a valuable system for users.

[1877] The processing flow will be explained below.

[1878] Step 1:

[1879] The user inputs a scenario. The device provides the user with a text area for scenario input and the ability to analyze the user's facial expressions and voice using an emotion engine. The user inputs the scenario into the text box and clicks the "Submit" button when complete.

[1880] Step 2:

[1881] When the device receives a click of the send button, it converts the scenario data into JSON format. At the same time, the emotion engine obtains the emotional status of the user at the time of input and sends it to the server along with the scenario data.

[1882] Step 3:

[1883] The server analyzes the received scenario data and emotional status using natural language analysis. When extracting scene and character information as analysis results, the emotional status is also taken into consideration. For example, if the user is excited, a scene will be created that reflects that excitement.

[1884] Step 4:

[1885] The server uses image generation means to generate each scene of the manga based on the analysis results. During the generation process, the character's facial expressions and background are set based on the user's emotional status. For example, action scenes can be emphasized to reflect excitement.

[1886] Step 5:

[1887] The server sends the image data of the generated manga page to the device. The device decodes the received image data and displays a preview to the user. The emotion engine analyzes the user's facial expressions and voice during the preview and sends emotional feedback to the server.

[1888] Step 6:

[1889] The user checks the displayed preview and specifies the areas to be corrected. The device provides an interface for correction, and the user inputs the corrections. The emotion engine analyzes the user's emotions when inputting the corrections and makes correction suggestions as necessary.

[1890] Step 7:

[1891] The terminal transmits the inputted correction instructions and emotion feedback to the server, which receives the correction instructions and emotion feedback and generates a corrected version using the image generation means again.

[1892] Step 8:

[1893] The server sends the image data of the revised manga page to the device. The device displays the preview again, and the user can recheck the revised version. If necessary, steps 6 to 8 are repeated.

[1894] Step 9:

[1895] The process from step 6 to step 8 is repeated until the user is satisfied. When a manga that satisfies the user is finally generated, the user clicks the done button and the terminal saves the final manga data in cloud storage using the storage means.

[1896] Step 10:

[1897] The server translates the saved manga data using a multilingual translation tool, and it is translated into multiple languages, including English, Chinese, and Spanish.

[1898] Step 11:

[1899] The server uploads the translated manga data to the sales platform using the uploading means, and a sales page is generated and published.

[1900] Step 12:

[1901] The server analyzes the sales data and generates content such as stamps and games for popular manga using additional content generation means.

[1902] Step 13:

[1903] The user uses the emotion engine to check the generated stamps and game content and provide instructions for corrections if necessary. The server then repeats the generation process, and the final content is completed.

[1904] Through these steps, users can create manga using emotional feedback and enjoy an intuitive and interactive production experience. Furthermore, the system offers great value to users, as it easily supports multiple languages, expands sales channels, and allows for the creation of additional content.

[1905] Example 2

[1906] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1907] Conventional manga production systems do not reflect the user's emotional state when the user inputs a manga scenario and checks each scene of the generated manga to give instructions for correction. This makes it difficult to generate manga that is in line with the user's intentions and emotions, and the generated manga sometimes does not meet the user's expectations. Furthermore, because the user's emotions could not be taken into account when creating multilingual content or additional content, it was difficult to increase user satisfaction.

[1908] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1909] In this invention, the server includes a scenario input means, a natural language analysis means for analyzing the scenario data input from the scenario input means and the user's emotional status, and an image generation means for generating a manga image based on the data analyzed by the natural language analysis means, thereby enabling the generation of a manga that reflects the user's emotional state.

[1910] The "scenario input means" is a means for a user to input a manga scenario in text format, and also includes a function for acquiring the user's emotional status.

[1911] The "natural language analysis means" is a means for analyzing the scenario data input from the scenario input means and the user's emotional status, and extracting each element of the story (scene and character information).

[1912] "Image generation means" refers to a means for generating manga images using a generative AI model based on data analyzed by a natural language analysis means.

[1913] The "preview means" is a means for displaying the image generated by the image generating means so that the user can check it.

[1914] The "emotion feedback acquisition means" is a means for acquiring an emotional status from the user's facial expression and voice when checking the image displayed by the preview means.

[1915] The "modification input means" is a means for providing an interface for the user to input instructions for modifying the preview image, and for suggesting modifications based on the acquired emotional status.

[1916] The "regeneration means" is a means for generating a revised version of the manga by operating the image generation means again based on the correction instructions and emotion feedback input by the correction input means.

[1917] "Storage means" refers to the means for storing the final manga, including cloud storage and storage locations on the device.

[1918] The "multilingual translation means" is a means for translating the manga stored by the storage means into multiple languages.

[1919] The "uploading means" refers to the means for uploading the translated manga to the sales platform and generating a sales page.

[1920] "Additional content generation means" refers to a means for generating additional content such as stamps and games based on the characters and scenes of manga uploaded to the sales platform.

[1921] The present invention is a system that automatically generates manga using a generative AI model, and combines it with an emotion engine that detects user emotions. Specific embodiments of the present invention will be described in detail below.

[1922] System Overview

[1923] This system uses AI technology to automatically generate manga based on a manga scenario entered by the user, and handles multilingual support, sales promotion, and the generation of additional content. It detects the user's emotional state and provides an interface that allows smooth revisions based on that feedback. The system consists of user, terminal, and server components.

[1924] System Component Description

[1925] 1. Scenario input method

[1926] The user inputs the manga scenario in text format using the device. At this time, the emotion engine analyzes the user's facial expressions and voice to obtain their emotional status. For example, when inputting a story in which the protagonist, a high school student, is summoned to another world, the user's emotions of surprise and excitement are analyzed.

[1927] 2. Natural language analysis means

[1928] The device sends the input scenario data to the server, which then analyzes the received scenario using natural language processing technology (e.g., Python's NLTK library) and extracts each element of the story (scene and character information).

[1929] 3. Image Generation Method

[1930] The server automatically generates each scene of the manga using a generative AI model (e.g., GPT-4) based on the analysis results and the user's emotional status. If the user is excited, the design is adjusted to emphasize action scenes.

[1931] 4. Preview Method

[1932] The image data of the generated manga is sent to the device, and the user can check the preview via a dedicated viewer app.

[1933] 5. Means of obtaining emotional feedback

[1934] During the preview, the device uses an emotion engine to analyze the user's facial expressions and voice to obtain emotional feedback data. For example, if the user smiles while looking at the preview, that emotion is fed back.

[1935] 6. Correction input method

[1936] Users use an interface to view the preview and input corrections and improvements. The emotion engine analyzes the user's emotions and makes appropriate suggestions for corrections. For example, if a user inputs "Make this scene more action-packed," and the emotion engine detects an excited state, it instructs the server to emphasize the action scenes.

[1937] 7. Regeneration means

[1938] The terminal sends the user's correction instructions to the server, and the server regenerates the corrected version of the manga page using the image generation means. The emotional status is also taken into account in this regeneration process.

[1939] 8. Preservation means

[1940] Once a manga that satisfies the user is finally generated, the data is stored in cloud storage (e.g., AWS S3).

[1941] 9. Multilingual Translation Tools

[1942] The server translates the stored manga using a multilingual translation engine (e.g., DeepL API or Google Translate API), allowing the same manga to be published in multiple languages.

[1943] 10. Upload Method

[1944] The translated manga data is uploaded to a sales platform (e.g., Amazon Kindle Direct Publishing) and a sales page is generated.

[1945] 11. Additional Content Generation Methods

[1946] The server will generate additional content such as stamps and games using characters and scenes from popular manga, including the creation of LINE stamps featuring popular characters and the development of mini-games.

[1947] Specific usage examples and prompt sentence examples

[1948] User B inputs a scenario he or she has thought up, "A story about a young magical girl who saves the world," through a scenario input means. The device sends the scenario data to a server, which analyzes it using natural language analysis means. Manga image data is automatically generated using a generative AI model, and a preview is sent to the device. If User B smiles at the preview, that emotional data is sent to the server and reflected as an emphasis point in the next scene. User B inputs correction instructions, and the server generates new illustrations. The completed manga is saved in the cloud, then translated into multiple languages ​​using a multilingual translation means and uploaded to a sales platform. If the manga proves popular, LINE stamps of popular characters are created using an additional content generation means.

[1949] Prompt Sentence Examples

[1950] "Write a story about a high school boy who is summoned to another world and fights with magical powers. Emphasize the emotions of surprise and excitement."

[1951] "Generate a manga-style story about a young wizard girl who saves the world. Adjust the story based on the emotional feedback provided by the user."

[1952] This will allow users to enjoy an intuitive and interactive manga creation experience, and the system will also make it easy to support multiple languages ​​and generate additional content.

[1953] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1954] Specific flow of program processing

[1955] Below, we will explain the processing flow of the program of this system step by step, clearly indicating the input, output, and specific operation of each step.

[1956] Step 1: Scenario Input

[1957] The user inputs a scenario in text format using the scenario input interface of the terminal.

[1958] Input: User-entered scenario text

[1959] Output: Scenario text data and user's emotional status

[1960] Specific operation: When a user inputs "A story about a magical girl who saves the world," the emotion engine analyzes the user's facial expressions and voice in real time through the device's built-in camera and microphone, and obtains emotional status such as surprise or excitement.

[1961] Step 2: Scenario submission

[1962] The terminal transmits the input scenario data to the server.

[1963] Input: Scenario text data and emotional status

[1964] Output: Scenario data and emotional status sent to the server

[1965] Specific operation: The terminal converts the scenario data into JSON format and sends a POST request to the server using the HTTP protocol.

[1966] Step 3: Scenario analysis

[1967] The server analyzes the scenario data and the emotional status using natural language analysis means.

[1968] Input: Scenario data and emotional status sent to the server

[1969] Output: Detailed information and character information for each scene

[1970] Specific operation: The server uses Python's NLTK library to parse the text of the scenario and extract keywords for each element of the story (characters, scenes, etc.).

[1971] Step 4: Manga Generation

[1972] Based on the analysis results, the server uses a generative AI model to automatically generate each scene of the manga.

[1973] Input: Detailed information and emotional status of the analyzed scenario

[1974] Output: Generated manga image data

[1975] Specific operation: The server uses a generative AI model (e.g., GPT-4) to generate illustrations for each scene, emphasizing action scenes if the user is excited.

[1976] Step 5: Preview

[1977] The server transmits the generated manga image data to the terminal, and the user previews it.

[1978] Input: Manga image data

[1979] Output: Preview image displayed on the device

[1980] Specific operation: The server transfers the generated manga image data to the device, and the user previews it using a dedicated viewer app on the device.

[1981] Step 6: Obtaining emotional feedback

[1982] During the preview, the device analyzes the user's facial expressions and voice, obtains emotional feedback data, and sends it to the server.

[1983] Input: Facial expression and voice data of the user during preview

[1984] Output: Emotion feedback data

[1985] Specific operation: The device analyzes the user's facial expressions and voice in real time through the camera and microphone, and sends emotional status such as smile, surprise, or confusion to the server.

[1986] Step 7: Correction Instructions

[1987] The user inputs the corrections and their details using a dedicated interface, and the system makes correction suggestions based on the user's emotional status.

[1988] Input: User correction instructions and emotional feedback data

[1989] Output: Correction instruction data sent to the server

[1990] Specific operation: The user inputs "I want more action in this scene," and the server makes suggestions for revisions based on emotional feedback.

[1991] Step 8: Regenerate

[1992] The server regenerates the image based on the correction instructions and emotional feedback.

[1993] Input: User correction instructions and emotional feedback data

[1994] Output: Regenerated manga image data

[1995] Specific operation: The server generates a new illustration that reflects the correction instructions and creates a new manga image.

[1996] Step 9: Save

[1997] The server then stores the final manga on the device or in cloud storage.

[1998] Input: Final manga image data

[1999] Output: Data stored in cloud storage or on device

[2000] Specific operation: The final manga data is uploaded and saved to cloud storage such as AWS S3.

[2001] Step 10: Multilingual Translation

[2002] The server translates the stored manga into multiple languages.

[2003] Input: Saved manga image data

[2004] Output: Translated manga image data

[2005] Specific operation: The server uses the DeepL API or Google Translate API to translate the manga text into multiple languages.

[2006] Step 11: Upload

[2007] The server uploads the translated manga data to the sales platform.

[2008] Input: Manga data translated into multiple languages

[2009] Output: Manga data uploaded to the sales platform

[2010] Specific operation: The server uploads manga data to Amazon Kindle Direct Publishing or other sales platforms via API.

[2011] Step 12: Generate additional content

[2012] The server generates additional content using characters and scenes from popular manga.

[2013] Input: Sales data from the sales platform and manga character information

[2014] Output: Stamps and mini-games as additional content

[2015] Specific operations: The server develops LINE stamps incorporating popular characters and mini-games using Unity.

[2016] The above is the specific flow and operation of each processing step in this system.

[2017] (Application example 2)

[2018] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2019] Conventional manga production systems lacked the technology to analyze user emotions in real time and dynamically adjust the scenes displayed during preview. This made it difficult for users to create manga that accurately reflected their emotions, and there was room for improvement in the quality of manga and user satisfaction. Despite the demand for providing an interactive viewing experience, especially in physical stores, these technical challenges had not been resolved.

[2020] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2021] In this invention, the server includes a scenario input means, a natural language analysis means, an image generation means, a preview means, an emotion analysis means, a dynamic adjustment means, a correction input means, a regeneration means, a storage means, a multilingual translation means, an upload means, and an additional content generation means. This makes it possible to analyze a user's emotions in real time and dynamically adjust manga scenes based on those emotions. Furthermore, it is possible to realize an interactive manga viewing experience in a physical store and improve user satisfaction.

[2022] The "scenario input means" is a device or interface that allows a user to input a manga scenario in text or audio format.

[2023] The "natural language analysis means" is a technology for analyzing the scenario data input from the scenario input means and extracting detailed information about each scene and character in the story.

[2024] The "image generation means" is a device or program for automatically generating each scene of the manga based on the data analyzed by the natural language analysis means.

[2025] The "preview means" is a device or interface that displays the manga image generated by the image generation means to the user and allows the user to view it.

[2026] The "emotion analysis means" is a technology for analyzing the facial expressions and voice of a user viewing a manga using the preview means, and obtaining the user's emotional status in real time.

[2027] The "dynamic adjustment means" is a technique for dynamically adjusting manga scenes based on the emotion data analyzed by the emotion analysis means.

[2028] The "modification input means" is a device or interface for inputting a user's instructions for modifying the image displayed by the preview means.

[2029] The "regeneration means" is a technique for generating a revised version of the manga by operating the image generation means again based on the correction instructions and emotion data input by the correction input means.

[2030] The "storage me...

Claims

1. a scenario input means; natural language analysis means for analyzing the scenario data input from the scenario input means; an image generation means for generating a manga image based on the data analyzed by the natural language analysis means; preview means for displaying a preview of the image generated by the image generating means; a correction input means for inputting a user's correction instruction for the image displayed by the preview means; a regeneration means for regenerating the image based on the correction instruction input by the correction input means; a storage means for storing the final manga generated by the regeneration means; a multilingual translation means for translating the manga stored by the storage means into multiple languages; an uploading means for uploading the manga translated by the multilingual translation means to a sales platform; additional content generation means for generating additional content for the manga uploaded by the upload means; A system including:

2. 2. The system according to claim 1, wherein said correction input means provides an interface for inputting the location and content of correction designated by the user.

3. 2. The system according to claim 1, wherein said regeneration means operates said image generation means again based on a correction instruction from a user.

Images