System

Abstract

A system is provided.SOLUTION: A system comprising: means for uploading webtoon content from a user; means for temporarily storing the uploaded content; means for extracting text data from the content and translating the text data into a plurality of languages; means for storing the translated content in a database; means for generating a plurality of scenarios and endings and storing the scenarios and endings in the database; means for providing appropriate content based on user selection; means for collecting and analyzing user browsing data; and means for proposing advertisement placement and pay content based on the analysis.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, the global expansion of webtoon content faces language barriers, limiting opportunities for readers of different cultural backgrounds to enjoy the same content. Another issue is that readers tend to tire of the content once they've finished it, making it difficult to maintain interest. Furthermore, creators face few opportunities to receive proper recognition, and there is a lack of efficient monetization methods, creating a lack of a sustainable production environment. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by the following means: a system including a means for uploading webtoon content from users, a means for temporarily storing the uploaded content, a means for extracting text data from the content and translating it into multiple languages, a means for storing the translated content in a database, a means for generating multiple scenarios and endings and storing them in a database, a means for providing appropriate content based on user selections, a means for collecting and analyzing user viewing data, and a means for suggesting advertising placement and paid content based on the analysis. This system automatically translates webtoon content into multiple languages ​​and makes it accessible globally. Furthermore, the system provides content that can be enjoyed repeatedly through multiple scenarios and multiple endings, and utilizes collected data to properly evaluate creators and effectively monetize them.

[0006] "User" refers to the end user who uploads and views webtoon content.

[0007] "Webtoon content" refers to visual content such as serialized comics and illustrations provided in digital format.

[0008] "Uploading" refers to a user sending webtoon content from their device to the platform.

[0009] A "means" refers to a device, method, or part of a system used to accomplish a particular purpose.

[0010] "Temporarily storing" refers to temporarily storing data in storage before post-processing or processing.

[0011] "Text data" refers to the text information used in webtoons.

[0012] "Translate" refers to the transfer of text or content from one language to another without changing its meaning.

[0013] "Database" refers to a collection of electronic data for storing and managing translated content and generated scenarios.

[0014] A "scenario" refers to a collection of written information that describes the progression and development of a story.

[0015] "Ending" refers to the final part of a scenario and can include multiple outcomes or end states.

[0016] "Selection-based" refers to operating in accordance with the selection of language, scenario, etc. made by the user through the interface.

[0017] "Collect" refers to gathering and recording user behavioral data and browsing history.

[0018] "Analyzing" refers to analyzing collected data and deriving meaning and patterns.

[0019] "Ad placement" refers to the display of an advertisement within a specific page or piece of content.

[0020] "Paid Content" means Content that a User can access by purchasing or paying for it. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] This invention relates to a webtoon platform that can be deployed globally by automatically creating multiple languages ​​and scenarios using generative AI. The system allows users to upload webtoon content, translate it into multiple languages, and provide it in the form of multiple scenarios and endings. It also analyzes user viewing data to suggest effective advertising placements and paid content.

[0043] Program processing flow

[0044] 1. A user uploads a webtoon

[0045] Users: Upload webtoon content from their own devices, including image data and text data.

[0046] Terminal: Accepts uploads from users through a web interface, along with meta information such as title, author, and language.

[0047] 2. Receiving and temporarily storing content

[0048] Server: Receives uploaded content and meta information and stores it in temporary storage. The format and size of the stored data are verified, and invalid data is notified to the user with an error message.

[0049] 3. Extraction and translation of text data

[0050] Server: Uses a language analysis module to extract text data from the uploaded webtoon content, then invokes a translation engine to translate the extracted text into multiple languages.

[0051] Example: Translations are made from Japanese to English, Chinese, and Spanish.

[0052] 4. Saving translated content

[0053] Server: Stores the translated webtoon content in a database and updates the metadata for each language version.

[0054] 5. Multi-scenario and multi-ending generation

[0055] Server: Based on the original content, it identifies key branching points and uses a generative AI engine to automatically generate different scenarios and endings, allowing users to enjoy different story developments depending on their choices.

[0056] Example: Multiple scenarios are generated in which the story progresses in different directions depending on the character's choices.

[0057] 6. Content Delivery

[0058] Device: The user selects the desired language and scenario through a web interface. Based on the selection, the server delivers the most appropriate language and scenario content to the device.

[0059] 7. Data Collection and Analysis

[0060] Server: Collects users' browsing history and scenario selection data and records them in a database. Analyzes the collected data to analyze users' interests and browsing patterns.

[0061] Example: Identifying a user's preferences based on the scenarios and endings they frequently choose.

[0062] 8. Ad Placement and Monetization

[0063] Server: Uses collected data to effectively place appropriate advertisements and recommend paid content, providing users with a personalized experience.

[0064] Example: For users who frequently select a particular scenario, advertisements and paid content related to that scenario will be displayed preferentially.

[0065] By automating these processes, the system will facilitate the global expansion of webtoon content, enabling it to be translated into multiple languages ​​and scenarios. It also utilizes user data to suggest optimal advertising placements and paid content, ensuring that creators are properly evaluated and fostering a sustainable production environment.

[0066] The processing flow will be explained below.

[0067] Step 1:

[0068] User: Uploads webtoon content, including image files (e.g., JPEG, PNG) and text files (e.g., TXT, PDF), as well as meta information such as title, author name, and language.

[0069] Step 2:

[0070] Terminal: Receives content and meta information uploaded through the web interface. After receiving, it performs an initial validation to ensure the file format and size are correct. If there are no issues, it stores the data in temporary storage.

[0071] Step 3:

[0072] Server: Analyzes the uploaded content and extracts text data from image files. Converts text in images into digital text using OCR (Optical Character Recognition) technology.

[0073] Step 4:

[0074] Server: The extracted text data is sent to the multilingual translation module, which uses a generative AI engine to automatically translate the text into multiple specified languages. For example, Japanese text is translated into English, Chinese, and Spanish.

[0075] Step 5:

[0076] Server: Reinserts the translated text into the original webtoon image to generate webtoons in each language. Stores the generated content in a database.

[0077] Step 6:

[0078] Server: Runs the multi-scenario / multi-ending generation module to generate multiple scenarios and endings based on the original webtoon content, identifying branching points in the scenario and creating different progressions for the story.

[0079] Step 7:

[0080] Server: Stores the generated multiple scenarios and endings in a database, including metadata for each scenario and ending (e.g., scenario overview, branching points, etc.).

[0081] Step 8:

[0082] Terminal: Provides an interface for users to select language and scenario through a web interface. Users select their preferred settings.

[0083] Step 9:

[0084] Server: Retrieves the appropriate language version and scenario from the database based on the user's selection. Sends the retrieved content to the device.

[0085] Step 10:

[0086] Device: Displays the received content and allows users to view the webtoon.

[0087] Step 11:

[0088] Server: Collects user browsing data, including the scenarios viewed, language selection, browsing time, etc. Stores the collected data in a database.

[0089] Step 12:

[0090] Server: Analyzes the collected data and derives insights to identify user preferences and interests. Based on the results of this analysis, it proposes optimal ad placements and paid content.

[0091] Step 13:

[0092] Server: Based on the analysis results, it sends personalized ads and content recommendations to the device, so that the information most relevant to the user is displayed.

[0093] Step 14:

[0094] On your device: Displaying received advertisements and content recommendations to you. Depending on your interactions, collecting further data to help optimize the platform.

[0095] These are the specific processing steps of the global webtoon platform that uses generative AI to automatically translate webtoons into multiple languages ​​and create multiple scenarios. This system enables webtoon content to be translated into multiple languages ​​and offers multiple scenarios, providing users with a new entertainment experience.

[0096] Example 1

[0097] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0098] Currently, the global expansion of digital comics requires multilingual support and the provision of a variety of scenarios and endings. However, doing this manually is extremely time-consuming and costly. Other issues include the lack of automated systems for providing personalized content tailored to users' tastes and preferences, effective advertising placement, and multilingual support. Unless these issues are resolved, they will become serious obstacles to the global expansion of digital comics.

[0099] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0100] In this invention, the server includes means for uploading digital comic content from users, means for temporarily storing the uploaded content, means for extracting character data from the content and translating it into multiple languages, means for storing the translated content in a database, means for using a generation AI engine to generate multiple scenarios and endings, means for storing the generated scenarios and endings in a database, means for providing content in an appropriate language version and scenario based on user selection, means for collecting and analyzing user viewing data, and means for suggesting advertising placement and paid content based on the analysis. This makes it possible to make digital comics multilingual and multi-scenarios and provide optimal content based on user data.

[0101] "User" refers to a person who uses digital comic content.

[0102] "Digital comic content" refers to comic or manga data provided in electronic format.

[0103] "Upload" refers to the act of transferring data from a user's device to a server.

[0104] "Means" refers to a method or device for achieving a particular purpose.

[0105] "Temporary storage" refers to the process of temporarily storing data.

[0106] "Text data" refers to the text and textual information within the Content.

[0107] "Translating into multiple languages" refers to the process of converting from one language to another.

[0108] "Database" refers to a system for efficiently storing and managing data.

[0109] A "generative AI engine" refers to a program that uses artificial intelligence to process data and generate new content.

[0110] "Scenario" refers to the flow of development of a story or content.

[0111] "Ending" refers to the conclusion or final part of a story or content.

[0112] "Providing" refers to the act of supplying specific services or content to users.

[0113] "Viewing data" refers to recorded information when a user views content.

[0114] "Analysis" refers to the process of examining data in detail and deriving trends and meaning.

[0115] "Ad placement" refers to the act of appropriately placing an advertisement on a user's screen.

[0116] "Paid Content" means additional content that a User is required to purchase or pay for.

[0117] This invention relates to a digital comic platform that can be deployed globally by automatically generating multiple languages ​​and scenarios using generative AI. The main components of this system are a server, terminals, and users. A detailed explanation is provided below.

[0118] Webtoon uploads by users

[0119] Users upload digital comic content from their own devices. This upload includes image data and text data. Users also enter meta information such as the title, author name, and language. For example, a user might enter information such as "Title: Magical Forest," "Author: Yamada Taro," and "Language: Japanese."

[0120] Receiving and temporarily storing content

[0121] The server stores the content and meta information received from the device in temporary storage. The format and size of the stored data are verified, and if there is any invalid data, an error message is displayed to the user. For example, if an image file is corrupted, the user is notified that the image file is invalid.

[0122] Extraction and translation of text data

[0123] The server uses a language analysis module to extract text data from uploaded digital comic content. The extracted text is then translated into multiple languages ​​using a translation engine. For example, Japanese can be translated into English, Chinese, and Spanish. The translation engine used could be software such as Google Translate API or DeepL.

[0124] Storing translated content

[0125] The server stores the translated digital comic content in a database and updates the metadata for each language version. For example, the English and Spanish versions are stored, and metadata such as "Title: The Magical Forest," "Artist: Yamada Taro," and "Language: English" is updated.

[0126] Multi-scenario and multi-ending generation

[0127] The server identifies key branching points based on the original content and automatically generates different scenarios and endings using a generative AI engine (e.g., GPT-3.5Turbo). This allows users to enjoy different story developments. For example, three different scenarios can be generated depending on the character's behavior choices.

[0128] Content Delivery

[0129] The device allows the user to select the desired language and scenario through a web interface. Based on the selection, the server delivers the most appropriate language and scenario content to the device. For example, if the user selects the English version of Scenario 2, that content will be displayed on the device.

[0130] Data collection and analysis

[0131] The server collects the user's browsing history and scenario selection data and records them in a database. The collected data is analyzed to determine the user's interests and browsing patterns. For example, if a user frequently selects a particular scenario, advertisements and paid content related to that scenario are displayed preferentially.

[0132] Ad Placement and Monetization

[0133] The server effectively places appropriate advertisements based on the collected data. It also recommends paid content and provides a personalized experience for users. For example, a user who likes fantasy scenarios will be shown advertisements for fantasy goods and related apps.

[0134] Prompt Sentence Examples

[0135] "Identify the main branching points in the webtoon and generate prompts to generate different scenarios. This prompt should be in the following format:

[0136] Input: A scene where Character A meets Character B

[0137] Output: Think of three different scenarios in which Character A could act after meeting Character B, and describe each scenario in 200 characters or less.

[0138] The above is a specific embodiment for carrying out the present invention. This system automatically supports multiple languages ​​and multiple scenarios for digital comics, and can provide optimal content based on user data.

[0139] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0140] Step 1:

[0141] Users upload digital comic content from their own devices. They provide image data, text data, and meta information (title, author name, language, etc.) as input. Specifically, they use a web interface to enter information such as "Title: Magical Forest," "Author: Yamada Taro," and "Language: Japanese," and then press the upload button.

[0142] Step 2:

[0143] The server saves the content and meta information received from the device in temporary storage. As input, it receives the digital comic file and meta information uploaded by the user. Specifically, the server verifies the format and size of the data, and if there is an abnormality, it generates an error message and notifies the user. As output, only valid data is temporarily stored, and invalid data is returned as an error message such as "The image file is invalid."

[0144] Step 3:

[0145] The server uses a language analysis module to extract text data from uploaded digital comic content. It receives webtoon image and text data as input. Specifically, it converts the text data in the image into text format using OCR (Optical Character Recognition) technology. The extracted text data is then generated as output.

[0146] Step 4:

[0147] The server translates the extracted text data into multiple languages. It receives Japanese character data as input. Specifically, it calls translation engines such as Google Translate API and DeepL to translate the Japanese text into English, Chinese, and Spanish. The translated text data is generated as output.

[0148] Step 5:

[0149] The server stores the translated digital comic content in a database. As input, it receives the translated text data and the original image data. Specific operations include saving the text and images in the corresponding fields of the database and updating the metadata (e.g., "Title: The Magical Forest," "Author: Yamada Taro," "Language: English"). As output, the database update is complete.

[0150] Step 6:

[0151] The server uses a generative AI engine to automatically generate multiple scenarios and endings based on the original content. It receives the original digital comic content and its main branching points as input. Specifically, it uses a generative AI model such as GPT-3.5 Turbo to create prompts for generating different scenarios and endings. As output, multiple scenarios and endings are generated and stored in a database.

[0152] Step 7:

[0153] The terminal allows the user to select the desired language and scenario through a web interface. As input, it receives the user's selected language and scenario information. In concrete terms, after the user selects the language and scenario, it sends the selection to the server. As output, the selection is transmitted to the server.

[0154] Step 8:

[0155] The server delivers the content of the selected language and scenario to the terminal. As input, it receives the language and scenario information selected by the user. Specifically, it retrieves the corresponding content from the database and sends it to the terminal. As output, the digital comic of the language and scenario selected by the user is displayed on the terminal.

[0156] Step 9:

[0157] The server collects users' browsing history and scenario selection data and records them in a database. As input, it receives the users' browsing and selection data. As a specific operation, it stores the collected data in the database in an appropriate format. As output, the recorded data is used for analysis.

[0158] Step 10:

[0159] The server analyzes the collected data and proposes ad placements and paid content. It receives user browsing data and selection data as input. Specifically, it uses data analysis algorithms to identify the user's interests and preferences and selects relevant ads and content. Personalized ads and paid content are proposed and displayed as output.

[0160] (Application example 1)

[0161] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0162] Webtoons are increasingly being enjoyed by a growing number of users, but language barriers and the limited variety of scenarios make their global adoption difficult. It is also challenging to effectively utilize user browsing data to suggest advertising placements and paid content. Furthermore, making them compatible with various devices is also a key issue.

[0163] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0164] In this invention, the server includes a means for uploading webtoon content from users, a means for temporarily storing the uploaded content, and a means for extracting text data from the content and translating it into multiple languages, thereby enabling webtoon content to be made available in multiple languages.

[0165] The system further includes a means for storing translated content in a database, a means for generating multiple scenarios and endings and storing them in a database, and a means for providing appropriate content based on a user's selection, thereby allowing users to enjoy webtoons in different languages ​​and scenarios at their leisure.

[0166] It also includes a means for collecting and analyzing user browsing data and a means for suggesting advertisement placement and paid content based on the analysis, which makes it possible to suggest advertisements and content personalized for each user.

[0167] Furthermore, it also includes a means for providing the system as an application installed on a smartphone, smart glasses, a head-mounted display, or a robot, which enables use on a variety of devices and improves user convenience.

[0168] "User" means an individual end user who uses the Webtoon Content.

[0169] "Webtoon content" refers to web manga or digital comics that are presented in digital format.

[0170] "Means for uploading" refers to a mechanism for transferring digital content owned by a user to a designated server.

[0171] A "temporary storage means" is a storage device or system for temporarily holding uploaded digital content.

[0172] "Means for extracting text data" refers to technology for identifying and obtaining text information from digital images and videos.

[0173] The "means for translating into multiple languages" is a translation engine or software for converting extracted text data into other languages.

[0174] "Means for storing data in a database" refers to a data management system for organizing and systematically storing digital data such as translated content and generated scenarios.

[0175] "Means for generating multiple scenarios and endings" refers to generative AI technology that automatically creates different developments and endings based on the original content.

[0176] "Means for providing appropriate content based on user selection" refers to a system that delivers optimal digital content according to the user's settings and preferences.

[0177] "Means for collecting and analyzing browsing data" refers to technology that records a user's operation history and browsing patterns and analyzes that data.

[0178] The "means for suggesting advertisement placement and paid content" is a system that presents advertisements and paid services that are most relevant to the user based on the analysis results.

[0179] An "application installed on a smartphone" is software that runs on a smartphone and provides various functions.

[0180] "Applications installed on smart glasses" are software that run on smart glasses and provide various functions.

[0181] An "application installed on a head-mounted display" is software that runs on the head-mounted display and provides various functions.

[0182] An "application installed on a robot" is software that runs on a robot and provides various functions.

[0183] This invention relates to a webtoon platform that can automatically translate multiple languages ​​and generate multiple scenarios using AI. The system of the present invention allows users to upload webtoon content, translate it into multiple languages, and provide it in the form of multiple scenarios and endings. It can also analyze users' viewing data to suggest effective advertising placements and paid content.

[0184] The system consists of the following elements: First, users upload webtoon content from their own devices. At this time, they enter meta information such as the title, author's name, and language. The uploaded content is temporarily stored on the server, where its format and size are verified.

[0185] The server then uses a language analysis module to extract text data from the uploaded webtoon content, and calls a translation engine to translate the extracted text into multiple languages, possibly using Google Cloud Translation API or DeepL API.

[0186] The translated webtoon content for each language version is stored in a database, and the metadata for each language version is also updated. A generative AI engine is then used to automatically generate different storylines and endings based on the original content. The generative AI engine uses OpenAI's GPT-4 or a similar natural language processing model.

[0187] Furthermore, user browsing data and scenario selection data are collected by the server and recorded in a database. This data is analyzed and used to analyze user interests and browsing patterns. Data analysis libraries such as Python's pandas and Scikit-learn are used for the analysis.

[0188] Ad placement and paid content suggestions are based on the collected data. An algorithm is implemented to effectively place appropriate ads, making it possible to provide users with a personalized experience.

[0189] Furthermore, this system is provided as an application that can be installed on smartphones, smart glasses, head-mounted displays, or robots, making it possible to use it on a variety of devices and improving user convenience.

[0190] For example, you can upload a Japanese webtoon and translate it into English, Chinese, and Spanish. You can also provide different scenarios depending on the character you choose. Below are some example prompts:

[0191] "If Character A gets lost, generate a scenario in which he goes to a different city. Output multiple endings along with background descriptions of those cities."

[0192] In this way, users can enjoy webtoons in different languages ​​and scenarios of their choice.

[0193] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0194] Step 1:

[0195] The user uploads webtoon content from their device to the server. At this time, the user enters meta information such as the title, author name, and language. The input data includes image files and text data. The server checks the received files and meta information to ensure that there is no invalid data. If the data is valid, it is temporarily stored.

[0196] Step 2:

[0197] The server extracts character data from the uploaded content. To obtain character data from image data, it uses OCR (Optical Character Recognition) technology. Specifically, it uses the Tesseract OCR library to recognize characters in the image and extract them as text data. This text data is then used for the next process.

[0198] Step 3:

[0199] The server translates the extracted text data into multiple languages. To do this, it uses the Google Cloud Translation API or the DeepL API. The extracted text data is sent to the API and translated into multiple target languages ​​(e.g., English, Chinese, Spanish). The translation results are generated in each language.

[0200] Step 4:

[0201] The server stores the translated webtoon content in a database and updates the metadata for each language. The database stores mapping information between the original language and the translated language, as well as text data for each language. This allows for the necessary data to be organized for subsequent scenario generation and distribution.

[0202] Step 5:

[0203] The server automatically generates multiple scenarios and endings based on the uploaded content using a generative AI engine. For example, OpenAI's GPT-4 is used to generate scenarios and endings based on prompts. The generated scenarios and endings are stored in a database.

[0204] Step 6:

[0205] The user accesses the web interface using a terminal and selects the desired language and scenario. The server delivers content in the optimal language and scenario to the terminal based on the user's selection, providing content files corresponding to the selected language and scenario as transmission data.

[0206] Step 7:

[0207] The server collects user browsing data and scenario selection data and records them in a database. The collected data is used for subsequent processing to analyze user interests and browsing patterns. Data analysis libraries such as Python's pandas and Scikit-learn are used to analyze trends in user behavior.

[0208] Step 8:

[0209] Based on the analysis results, the server effectively places appropriate advertisements and suggests paid content. Algorithms are implemented to provide personalized advertisements and content based on each user's preferences and browsing patterns. This increases creators' revenue and improves the user experience.

[0210] In this way, a system that supports multiple languages ​​and generates multiple scenarios can facilitate the global expansion of webtoon content while providing personalized services to users. Below is an example of a prompt sentence as a concrete example.

[0211] "If Character A gets lost, generate a scenario in which he goes to a different city. Output multiple endings along with background descriptions of those cities."

[0212] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0213] This invention relates to a webtoon platform that can be deployed globally by automatically creating multiple languages ​​and scenarios by combining generative AI and an emotion recognition engine. This system allows users to upload webtoon content, translate it into multiple languages, and provide it in the form of multiple scenarios and endings. It also provides content and advertising suggestions based on users' emotions through emotion recognition, and analyzes users' viewing data to evaluate creators and realize efficient monetization.

[0214] Program processing flow

[0215] 1. A user uploads a webtoon

[0216] User: Uploads webtoon content from their device, including image files and text data, and also enters meta information such as title, author name, and language.

[0217] Terminal: Accepts uploads from users via a web interface, including any meta information entered.

[0218] 2. Receiving and storing content

[0219] Server: Receives uploaded content and meta information and stores it in temporary storage. The format and size of the stored data are verified, and invalid data is notified to the user with an error message.

[0220] 3. Extraction and translation of text data

[0221] Server: Uses language analysis module to extract text data from uploaded webtoon content, and converts text in images into digital text using OCR technology.

[0222] Server: The extracted text data is sent to the multilingual translation module for automatic translation into multiple specified languages. For example, Japanese text is translated into English, Chinese, and Spanish.

[0223] 4. Saving translated content

[0224] Server: Reinserts the translated text into the original webtoon image to generate webtoons in each language. Stores the generated content in a database.

[0225] 5. Multi-scenario and multi-ending generation

[0226] Server: Based on the original content, identify key branching points and use a generative AI engine to automatically generate different scenarios and endings. Identify branching points in the scenario and create different progressions of the story.

[0227] Server: Stores the generated multiple scenarios and endings in a database, including metadata for each scenario and ending (e.g., scenario overview, branching points, etc.).

[0228] 6. Introducing Emotion Recognition

[0229] Device: The user's facial expressions and voice are sent to the emotion engine via the user's camera and microphone.

[0230] Server: Uses an emotion engine to analyze the user's emotions in real time from facial expressions and voice data.

[0231] 7. Content Delivery

[0232] Terminal: Provides an interface for users to select their preferred language and scenario through a web interface. Users can select their preferred settings.

[0233] Server: Retrieves the appropriate language version and scenario from the database based on the user's selection and the recognized emotion. Sends the retrieved content to the device.

[0234] 8. Data Collection and Analysis

[0235] Server: Collects user browsing and emotional data, including viewed scenarios, selected language, browsing time, perceived emotional state, etc. Stores the collected data in a database.

[0236] Server: Analyzes the collected data and derives insights to identify user preferences, interests, and emotional trends. Based on the results of this analysis, it proposes emotionally-based ad placements and paid content.

[0237] 9. Ad Placement and Monetization

[0238] Server: Based on the analysis results, it sends personalized ads and content recommendations to the device, so that the information most relevant to the user is displayed.

[0239] Example: If the user is recognized as "happy" by the emotion engine, relevant ads and content with upbeat content will be displayed preferentially.

[0240] 10. Feedback and Optimization

[0241] Server: Based on user interactions, further data is collected and used to improve the accuracy of the generative AI algorithms and emotion engine, thereby continuously optimizing the entire system.

[0242] By automating these processes, the system enables multilingual and multi-scenario webtoon content, and provides a personalized experience based on the user's emotions. The introduction of emotion recognition provides a more immersive experience, creating a new entertainment experience.

[0243] The processing flow will be explained below.

[0244] Step 1:

[0245] Users: Upload webtoon content from their devices, including image files (e.g., JPEG, PNG) and text files (e.g., TXT, PDF), along with meta information such as title, author name, and language.

[0246] Step 2:

[0247] Terminal: Receives content uploaded via the web interface and any meta information entered. Performs initial validation for proper format and size. If successful, stores the content in temporary storage.

[0248] Step 3:

[0249] Server: Analyzes the uploaded file and extracts text data from the image file, using OCR (Optical Character Recognition) technology to convert the text in the image into digital text.

[0250] Step 4:

[0251] Server: The extracted text data is sent to a multilingual translation module for automatic translation into multiple languages. For example, Japanese text is translated into English, Chinese, and Spanish.

[0252] Step 5:

[0253] Server: Reinserts the translated text into the original webtoon image to generate webtoons in each language. Stores the generated content in a database.

[0254] Step 6:

[0255] Server: Runs the multi-scenario / multi-ending generation module, automatically generating multiple scenarios and endings based on the original content. Identifies branching points in the scenario and creates different story progressions for each branch.

[0256] Step 7:

[0257] Server: The generated multiple scenarios and endings are saved in a database, including metadata for each scenario and ending (e.g., scenario overview, branching points, etc.).

[0258] Step 8:

[0259] Device: The user's facial expressions and voice are sent to the emotion engine via the user's camera and microphone. This data is used to analyze the user's emotions in real time.

[0260] Step 9:

[0261] Server: Analyzes the user's emotions from facial expressions and voice data using an emotion engine. The analysis results are used to provide content based on the user's emotional state.

[0262] Step 10:

[0263] Terminal: Provides an interface for users to select their preferred language and scenario through a web interface. Users select their preferred settings.

[0264] Step 11:

[0265] Server: Based on the user's selection and analyzed emotions, retrieves the appropriate language version and scenario from the database. Sends the retrieved content to the device.

[0266] Step 12:

[0267] Device: Displays the received content and allows users to view the webtoon.

[0268] Step 13:

[0269] Server: Collects user browsing and emotional data, such as viewed scenarios, selected language, browsing time, and perceived emotional state. Stores the collected data in a database.

[0270] Step 14:

[0271] Server: Analyzes the collected data and derives insights to identify user preferences, interests, and emotional trends. Based on the analysis results, it proposes emotionally-based ad placements and paid content.

[0272] Step 15:

[0273] Server: Based on the analysis results, sends personalized ads and content recommendations to the device, so that the user sees the most relevant information.

[0274] Step 16:

[0275] Device: Displaying received advertisements and content recommendations to you, and collecting further data based on your responses to help optimize the system.

[0276] The system automates these processes, enabling webtoon content to be multilingual and multi-scenario compatible, and also provides a personalized content experience through user sentiment analysis, creating new value in entertainment.

[0277] Example 2

[0278] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0279] Conventional webtoon platforms struggled with multilingual support and multiple scenarios, and lacked the ability to provide personalized content based on users' individual emotions and preferences. Effective advertising placement and monetization were also difficult. This has led to a demand for improved user experience and efficient monetization for creators.

[0280] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for uploading content from a user, a means for temporarily storing the uploaded content, a means including a language analysis module and a translation module for extracting character data from the content and translating it into multiple languages, a means for storing the translated content in a database, a means including a generation AI engine for generating different scenarios and endings, a means for storing the generated multiple scenarios and endings in a database, a means including an emotion recognition engine for collecting emotion data from the user's camera and microphone and performing emotion analysis, a means for providing optimal content based on the user's selection and the emotion analysis results, a means for collecting user browsing data and emotion data and storing them in a database, a means for analyzing the data and suggesting advertisement placement and paid content based on the analysis results, and a means for improving the accuracy of the generation AI algorithm and the emotion recognition engine using the collected and analyzed data. This facilitates multilingual support and multi-scenario development of webtoon content, enabling personalized experiences based on individual emotions and preferences. In addition, it also achieves effective advertisement placement and monetization.

[0281] "User" means an individual or organization that uses this system to upload and view content.

[0282] "Content" refers to the image files and text data of webtoons that users upload to the system.

[0283] The "temporary storage means" is a function for temporarily storing uploaded content in server storage or cloud storage for use in subsequent processing.

[0284] "Language Analysis Module" is a software unit for extracting text data from uploaded content. Specifically, it has the function of extracting text from images using OCR technology.

[0285] A "translation module" is a software unit that automatically translates extracted text data into multiple languages, for example, using natural language processing techniques.

[0286] "Generative AI engine" is a general term for machine learning models and algorithms that automatically generate different scenarios and endings based on the original content.

[0287] An "emotion recognition engine" is a software unit that analyzes a user's facial expressions and voice data collected through a camera and microphone to determine the user's emotional state in real time.

[0288] The "database" is an information management system for systematically storing various data used within the system (uploaded content, translated text, generated scenarios and endings, user emotional data and browsing data, etc.).

[0289] "Data analysis" is the process of analyzing collected user browsing data and sentiment data using statistical methods and machine learning techniques to derive insights.

[0290] "Ad placement" is the process of displaying advertisements that match the user's interests and emotions at the optimal time based on the analysis results.

[0291] "Paid content suggestion" is the process of recommending paid content that matches a user's interests and preferences based on data analysis.

[0292] This invention is a system that combines a generative AI model and an emotion recognition engine to realize multilingual and multi-scenario webtoon content, providing a personalized experience based on the user's emotions and preferences, and enabling efficient ad placement and monetization.

[0293] System configuration

[0294] The system includes the following major components:

[0295] 1. User Device:

[0296] Hardware: Devices such as PCs, smartphones, and tablets

[0297] Software: Web browser, camera, microphone

[0298] 2. Server:

[0299] Hardware: Cloud server, database server

[0300] Software: OCR technology, translation module, generative AI engine, emotion recognition engine, data analysis module

[0301] Processing flow

[0302] The system operates as follows:

[0303] 1. User uploads content:

[0304] User: Upload webtoon content from their own device and enter meta information.

[0305] Terminal: Accepts uploads from users through a web interface.

[0306] Example: A user opens a browser, clicks the "Choose File" button on a website's upload screen, selects an image file and text data, and clicks the "Upload" button.

[0307] 2. The server receives and stores the content:

[0308] Server: Receives uploaded content and meta information and temporarily stores it in cloud storage. Validates the format and size of the stored data and notifies the user of any invalid data with an error message.

[0309] Example: The server checks that the image file received is in JPEG format and that the file size is within 10MB, and if there are no problems, it saves it to cloud storage.

[0310] 3. Extract and translate the text data of the content:

[0311] Server: Extracts text data from uploaded webtoon content using OCR technology and sends it to the translation module for automatic translation into multiple languages.

[0312] Example: Use Amazon Textract to perform OCR analysis, and then use the Google Cloud Translation API to translate the acquired Japanese text data into English, Chinese, and Spanish.

[0313] 4. Saving translated content:

[0314] Server: The translated text is reinserted into the original webtoon image, and webtoons in each language are generated and stored in the database.

[0315] Example: Using Adobe Photoshop API to remap translated text onto webtoon images for each language version, and then storing the generated images in cloud storage, and storing information including metadata in a database.

[0316] 5. Multi-Scenario and Multi-Ending Generation:

[0317] Server: Using a generative AI engine, different scenarios and endings are automatically generated based on the original content and stored in a database.

[0318] Example: Using OpenAI's GPT-4 to identify branching points in a story and generate different scenarios and endings.

[0319] 6. Introducing Emotion Recognition:

[0320] Terminal: Collects emotional data through the user's camera and microphone and sends it to the server.

[0321] Server: Performs real-time emotion analysis using an emotion recognition engine and stores the results in a database.

[0322] Example: Use your smartphone's camera and microphone to capture facial expressions and voice, then use the Emotion API to analyze emotions in real time.

[0323] 7. Content Distribution:

[0324] Terminal: Allows users to select their preferred language and scenario through a web interface.

[0325] Server: Based on the user's selection and the results of sentiment analysis, retrieves appropriate content from the database and sends it to the device.

[0326] Example: A user selects a language and scenario on a website, and that information is sent to a server. The server retrieves the selected language and scenario and sends them to the device.

[0327] 8. Data Collection and Analysis:

[0328] Server: Collects user browsing data and emotion data and stores them in a database.

[0329] Server: Uses the data analysis module to analyze the collected data and derive insights.

[0330] Example: Use Google Analytics to collect browsing data, store emotion recognition data in a database, and use Apache Spark for analysis.

[0331] 9. Ad Placement and Monetization:

[0332] Server: Proposes ad placement and paid content based on the analysis results.

[0333] Example: Displaying personalized ads to users whose emotions are identified as "joy" through emotion recognition.

[0334] 10. Feedback and optimization:

[0335] Server: Based on user interactions, collects additional data to improve the accuracy of generative AI algorithms and emotion recognition engines.

[0336] Example: Continuously collect new browsing and sentiment data to retrain machine learning models.

[0337] Prompt Sentence Examples

[0338] "Design a system to translate the following Japanese text into English, Chinese, and Spanish, and generate different scenarios and endings. The system should allow users to enjoy content that matches their mood of the day through emotion recognition. The system should be uploaded by the user using their smartphone, and emotions should be recognized using the camera and microphone."

[0339] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0340] Step 1:

[0341] User uploads content:

[0342] Input: Image files, text data, and meta information (title, author name, language, etc.) of webtoons selected by the user from their device.

[0343] Process: A user opens the website's upload screen in a browser, clicks the "Choose File" button, selects an image file and text data, and clicks the "Upload" button.

[0344] Output: The device receives the uploaded file and meta information.

[0345] Step 2:

[0346] The server receives and stores the content:

[0347] Input: Webtoon image files, text data, and meta information sent from the device.

[0348] Processing: The server receives the uploaded content and meta information, temporarily stores it in cloud storage, validates the format (JPEG, PNG, etc.) and file size of the received image file, and sends an error message to the user if there is invalid data.

[0349] Output: Saved content and meta information.

[0350] Step 3:

[0351] Extract and translate text data from content:

[0352] Input: Webtoon image files stored in cloud storage.

[0353] Processing: The server uses OCR technology (e.g., Amazon Textract) to extract the text data in the image, and then sends the text data to a multilingual translation module (e.g., Google Cloud Translation API) to automatically translate the Japanese text into English, Chinese, and Spanish.

[0354] Output: Translated text data in multiple languages.

[0355] Step 4:

[0356] Saving translated content:

[0357] Input: translated text data, original webtoon images.

[0358] Processing: The server reinserts the translated text into the original webtoon image to generate the webtoon in each language (e.g., using Adobe Photoshop API). The generated webtoon in each language is then stored in a database.

[0359] Output: Multilingual webtoon content stored in a database.

[0360] Step 5:

[0361] Multi-scenario and multi-ending generation:

[0362] Input: Original webtoon content.

[0363] Processing: The server uses a generative AI engine (e.g., OpenAI's GPT-4) to automatically generate different scenarios and endings based on the original content and store them in a database.

[0364] Output: Multiple scenarios and endings stored in a database.

[0365] Step 6:

[0366] Introducing emotion recognition:

[0367] Input: User facial and voice data collected through the device's camera and microphone.

[0368] Processing: The server receives facial expression and voice data sent from the user's device and analyzes it in real time using an emotion recognition engine (e.g., Emotion API).

[0369] Output: Parsed emotion data.

[0370] Step 7:

[0371] Content Delivery:

[0372] Input: User choices (language, scenario), parsed emotion data.

[0373] Processing: The user selects the desired language and scenario through a web interface, and that information is sent to the server. Based on the user's selection and the results of sentiment analysis, the server retrieves appropriate content from a database and sends it to the device.

[0374] Output: The appropriate content is delivered to the user's device.

[0375] Step 8:

[0376] Data collection and analysis:

[0377] Input: User browsing data, emotion data.

[0378] Processing: The server collects user browsing data and sentiment data and stores it in a database. A data analysis module (e.g., Apache Spark) is used to analyze the collected data and derive insights into user preferences and sentiment trends.

[0379] Output: Collected and analyzed data, analytical insights.

[0380] Step 9:

[0381] Ad Placement and Monetization:

[0382] Input: Analytics Insights.

[0383] Processing: The server displays personalized advertisements to the user based on the analysis results and suggests relevant paid content.

[0384] Output: Advertisements and paid content suggestions displayed to the user.

[0385] Step 10:

[0386] Feedback and optimization:

[0387] Input: Collected browsing data, sentiment data, and interaction data.

[0388] Processing: The server uses this data to retrain the generative AI algorithms and emotion recognition engines to improve the accuracy of the system.

[0389] Output: Optimized generative AI model and emotion recognition engine.

[0390] (Application example 2)

[0391] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0392] In addition to providing a webtoon platform that supports multiple languages ​​and allows for individual scenario selection, there is a need for a platform that can provide optimal content and advertisements in real time based on user emotions. However, current systems have difficulty generating flexible scenarios that respond to diverse user emotions and preferences, and providing personalized advertisements. Therefore, an effective system is needed that can simultaneously improve user experience and monetize creators and advertisers.

[0393] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for uploading image content from a user, means for temporarily storing the uploaded content, means for extracting text data from the content and translating it into multiple languages, means for storing the translated content in a database, means for identifying major branching points and generating multiple scenarios and endings, means for recognizing the user's emotions through a camera or microphone, means for providing optimal content and advertisements based on the recognized emotions, means for collecting and analyzing user viewing data and emotion data, and means for suggesting advertisement placement and paid content based on the above analysis. This makes it possible to provide a personalized experience based on the user's emotions and optimize content and advertisements.

[0394] "User" refers to an individual or corporation that uses the system to upload, view, select, and perform other operations on webtoons.

[0395] "Image content" refers to digital data that is primarily composed of visual elements, such as webtoons, comic pages, and illustrations.

[0396] "Means for uploading" refers to the functions and interfaces that allow users to send image content to the server.

[0397] "Temporary storage" means memory or storage for short-term storage of uploaded content.

[0398] "Means for extracting text data" refers to OCR (optical character recognition) technology or software for extracting text information from image content.

[0399] "Means for translating into multiple languages" refers to a translation engine or API for translating extracted text data into multiple languages.

[0400] "Means for storing in a database" refers to a data management system for long-term storage of translated content and generated scenarios.

[0401] "Branch point identification methods" refers to algorithms or software that automatically identify important choices and directions in the development of a story.

[0402] "Means for generating scenarios and endings" refers to generative AI technology that generates different story developments and endings based on branching points.

[0403] "Means of recognition through cameras and microphones" refers to hardware and software for capturing the user's facial expressions and voice and analyzing their emotions.

[0404] "Means for providing content based on recognized emotions" refers to a system for displaying optimal content and advertisements based on analyzed emotional data.

[0405] "Means for collecting and analyzing browsing data and emotional data" refers to a data processing system for recording and analyzing a user's content operation history and recognized emotional data.

[0406] "Means for suggesting advertising placements and paid content" refers to algorithms and systems that recommend optimal advertising and paid content to users based on collected data.

[0407] The present invention relates to a webtoon platform that supports multiple languages ​​and allows individual scenario selection, and is a system that provides optimal content and advertisements based on user emotions. Specific embodiments will be described below.

[0408] System Program Overview

[0409] This system implements a program that includes the following main functions:

[0410] 1. Upload and save webtoons

[0411] The server provides an interface for users to upload image content (e.g., webtoons), which is then temporarily stored.

[0412] 2. Extraction and translation of text data

[0413] The server uses an OCR (Optical Character Recognition) engine to extract text data from image content, which is then translated into multiple languages ​​using the Google Translate API.

[0414] 3. Identifying and generating scenario branching points

[0415] The server uses the extracted text data to identify key story branching points, and uses a generative AI model to automatically generate different scenarios and endings, which are then stored in a database.

[0416] 4. Emotion recognition and personalized content delivery

[0417] It uses the camera and microphone on the device (e.g., smartphone) to recognize the user's emotions in real time. It uses the EmotionRecognizer module to analyze the user's emotional state from their facial expressions and voice. Based on the analysis results, it provides optimal content and personalized advertisements.

[0418] 5. Data Collection and Analysis

[0419] The server collects user browsing data and emotional data and stores it in a database. This data is analyzed to gain insights into user preferences and behavioral patterns. This data is used to optimize ad placement and suggest paid content.

[0420] Hardware and software used

[0421] Hardware

[0422] Devices such as smartphones and tablets: Used to upload and view content from users.

[0423] Camera and microphone: Captures the user's facial expressions and voice for emotion recognition.

[0424] software

[0425] OCR engine: Used to extract text from images.

[0426] Google Translate API: Used to translate the extracted text into multiple languages.

[0427] EmotionRecognizer: A software module for recognizing user emotions.

[0428] Generative AI models: Used to automatically generate different scenarios and endings.

[0429] Specific examples

[0430] For example, if a user opens the app at night and emotion recognition detects the emotion of "joy," the server will prioritize displaying cheerful and upbeat stories and related ads, providing the best possible experience for the user.

[0431] Prompt Sentence Examples

[0432] An example of a specific prompt for the generative AI model is, "Please generate what kind of story development would be best for a user who is in a joyful mood." Based on this prompt, the generative AI model automatically generates multiple story developments that suit the user's emotional state.

[0433] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0434] Step 1:

[0435] Users upload image content (e.g., webtoons) from their devices. They also enter meta information such as the title, author's name, and language used. The server receives this input data and temporarily stores it. Specifically, data is sent via a web interface, and the server temporarily stores the uploaded image file and meta information.

[0436] Step 2:

[0437] The server extracts text data from the stored image content, using an OCR engine (e.g., Google Cloud Vision API) to convert the text in the image into digital text. This allows the character information embedded in the image to be obtained. The input is the temporarily stored image data, and the output is digital text.

[0438] Step 3:

[0439] The server translates the extracted text data into multiple languages. Specifically, it uses the Google Translate API to translate, for example, Japanese into English, Chinese, Spanish, etc. This process generates translated text (output) into each target language based on the original text (input).

[0440] Step 4:

[0441] The server reinserts the translated text into the original webtoon image to generate the webtoon in each language. The generated content for each language is stored in a database. The input is the translated text and the original image file, and the output is a multilingual webtoon image. In this reinsertion process, the text is embedded into the image using image editing software (e.g., Python's Pillow library).

[0442] Step 5:

[0443] The server identifies key branching points based on the original content. Based on the identified branching points, it uses a generative AI model to automatically generate different scenarios and endings. In this process, multiple scenarios and endings (outputs) are generated from the original content (input) and stored in a database. Text analysis algorithms and generative AI algorithms are used for identification and generation.

[0444] Step 6:

[0445] The device captures the user's facial expressions and voice through the user's camera and microphone and sends them to the server. The server then uses the EmotionRecognizer module to analyze the user's emotions in real time from the transmitted facial and voice data. The input is the captured multimedia data, and the output is the analyzed emotional information.

[0446] Step 7:

[0447] The server provides optimal content and personalized advertisements based on the analyzed emotional data. This involves retrieving webtoons with the appropriate language version and scenario from a database and sending them to the user's device. The input is the emotional data and user selection information, and the output is the appropriate content and advertisements.

[0448] Step 8:

[0449] The server collects users' browsing data and emotional data and stores it in a database. The collected data is analyzed to derive insights that identify users' preferences, interests, and emotional trends. The input is user interaction data, and the output is the analysis results. Data mining techniques and machine learning algorithms are used for the analysis.

[0450] Step 9:

[0451] The server then uses the analysis results to suggest ad placements and paid content, thereby displaying the most relevant ads and content to the user. The input is the analysis results, and the output is a list of recommended ads and paid content. A recommendation system algorithm is used in this process.

[0452] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0453] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0454] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0455] [Second embodiment]

[0456] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0457] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0458] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0459] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0460] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0461] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0462] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0463] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0464] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0465] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0466] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0467] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0468] This invention relates to a webtoon platform that can be deployed globally by automatically creating multiple languages ​​and scenarios using generative AI. The system allows users to upload webtoon content, translate it into multiple languages, and provide it in the form of multiple scenarios and endings. It also analyzes user viewing data to suggest effective advertising placements and paid content.

[0469] Program processing flow

[0470] 1. A user uploads a webtoon

[0471] Users: Upload webtoon content from their own devices, including image data and text data.

[0472] Terminal: Accepts uploads from users through a web interface, along with meta information such as title, author, and language.

[0473] 2. Receiving and temporarily storing content

[0474] Server: Receives uploaded content and meta information and stores it in temporary storage. The format and size of the stored data are verified, and invalid data is notified to the user with an error message.

[0475] 3. Extraction and translation of text data

[0476] Server: Uses a language analysis module to extract text data from the uploaded webtoon content, then invokes a translation engine to translate the extracted text into multiple languages.

[0477] Example: Translations are made from Japanese to English, Chinese, and Spanish.

[0478] 4. Saving translated content

[0479] Server: Stores the translated webtoon content in a database and updates the metadata for each language version.

[0480] 5. Multi-scenario and multi-ending generation

[0481] Server: Based on the original content, it identifies key branching points and uses a generative AI engine to automatically generate different scenarios and endings, allowing users to enjoy different story developments depending on their choices.

[0482] Example: Multiple scenarios are generated in which the story progresses in different directions depending on the character's choices.

[0483] 6. Content Delivery

[0484] Device: The user selects the desired language and scenario through a web interface. Based on the selection, the server delivers the most appropriate language and scenario content to the device.

[0485] 7. Data Collection and Analysis

[0486] Server: Collects users' browsing history and scenario selection data and records them in a database. Analyzes the collected data to analyze users' interests and browsing patterns.

[0487] Example: Identifying a user's preferences based on the scenarios and endings they frequently choose.

[0488] 8. Ad Placement and Monetization

[0489] Server: Uses collected data to effectively place appropriate advertisements and recommend paid content, providing users with a personalized experience.

[0490] Example: For users who frequently select a particular scenario, advertisements and paid content related to that scenario will be displayed preferentially.

[0491] By automating these processes, the system will facilitate the global expansion of webtoon content, enabling it to be translated into multiple languages ​​and scenarios. It also utilizes user data to suggest optimal advertising placements and paid content, ensuring that creators are properly evaluated and fostering a sustainable production environment.

[0492] The processing flow will be explained below.

[0493] Step 1:

[0494] User: Uploads webtoon content, including image files (e.g., JPEG, PNG) and text files (e.g., TXT, PDF), as well as meta information such as title, author name, and language.

[0495] Step 2:

[0496] Terminal: Receives content and meta information uploaded through the web interface. After receiving, it performs an initial validation to ensure the file format and size are correct. If there are no issues, it stores the data in temporary storage.

[0497] Step 3:

[0498] Server: Analyzes the uploaded content and extracts text data from image files. Converts text in images into digital text using OCR (Optical Character Recognition) technology.

[0499] Step 4:

[0500] Server: The extracted text data is sent to the multilingual translation module, which uses a generative AI engine to automatically translate the text into multiple specified languages. For example, Japanese text is translated into English, Chinese, and Spanish.

[0501] Step 5:

[0502] Server: Reinserts the translated text into the original webtoon image to generate webtoons in each language. Stores the generated content in a database.

[0503] Step 6:

[0504] Server: Runs the multi-scenario / multi-ending generation module to generate multiple scenarios and endings based on the original webtoon content, identifying branching points in the scenario and creating different progressions for the story.

[0505] Step 7:

[0506] Server: Stores the generated multiple scenarios and endings in a database, including metadata for each scenario and ending (e.g., scenario overview, branching points, etc.).

[0507] Step 8:

[0508] Terminal: Provides an interface for users to select language and scenario through a web interface. Users select their preferred settings.

[0509] Step 9:

[0510] Server: Retrieves the appropriate language version and scenario from the database based on the user's selection. Sends the retrieved content to the device.

[0511] Step 10:

[0512] Device: Displays the received content and allows users to view the webtoon.

[0513] Step 11:

[0514] Server: Collects user browsing data, including the scenarios viewed, language selection, browsing time, etc. Stores the collected data in a database.

[0515] Step 12:

[0516] Server: Analyzes the collected data and derives insights to identify user preferences and interests. Based on the results of this analysis, it proposes optimal ad placements and paid content.

[0517] Step 13:

[0518] Server: Based on the analysis results, it sends personalized ads and content recommendations to the device, so that the information most relevant to the user is displayed.

[0519] Step 14:

[0520] On your device: Displaying received advertisements and content recommendations to you. Depending on your interactions, collecting further data to help optimize the platform.

[0521] These are the specific processing steps of the global webtoon platform that uses generative AI to automatically translate webtoons into multiple languages ​​and create multiple scenarios. This system enables webtoon content to be translated into multiple languages ​​and offers multiple scenarios, providing users with a new entertainment experience.

[0522] Example 1

[0523] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0524] Currently, the global expansion of digital comics requires multilingual support and the provision of a variety of scenarios and endings. However, doing this manually is extremely time-consuming and costly. Other issues include the lack of automated systems for providing personalized content tailored to users' tastes and preferences, effective advertising placement, and multilingual support. Unless these issues are resolved, they will become serious obstacles to the global expansion of digital comics.

[0525] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0526] In this invention, the server includes means for uploading digital comic content from users, means for temporarily storing the uploaded content, means for extracting character data from the content and translating it into multiple languages, means for storing the translated content in a database, means for using a generation AI engine to generate multiple scenarios and endings, means for storing the generated scenarios and endings in a database, means for providing content in an appropriate language version and scenario based on user selection, means for collecting and analyzing user viewing data, and means for suggesting advertising placement and paid content based on the analysis. This makes it possible to make digital comics multilingual and multi-scenarios and provide optimal content based on user data.

[0527] "User" refers to a person who uses digital comic content.

[0528] "Digital comic content" refers to comic or manga data provided in electronic format.

[0529] "Upload" refers to the act of transferring data from a user's device to a server.

[0530] "Means" refers to a method or device for achieving a particular purpose.

[0531] "Temporary storage" refers to the process of temporarily storing data.

[0532] "Text data" refers to the text and textual information within the Content.

[0533] "Translating into multiple languages" refers to the process of converting from one language to another.

[0534] "Database" refers to a system for efficiently storing and managing data.

[0535] A "generative AI engine" refers to a program that uses artificial intelligence to process data and generate new content.

[0536] "Scenario" refers to the flow of development of a story or content.

[0537] "Ending" refers to the conclusion or final part of a story or content.

[0538] "Providing" refers to the act of supplying specific services or content to users.

[0539] "Viewing data" refers to recorded information when a user views content.

[0540] "Analysis" refers to the process of examining data in detail and deriving trends and meaning.

[0541] "Ad placement" refers to the act of appropriately placing an advertisement on a user's screen.

[0542] "Paid Content" means additional content that a User is required to purchase or pay for.

[0543] This invention relates to a digital comic platform that can be deployed globally by automatically generating multiple languages ​​and scenarios using generative AI. The main components of this system are a server, terminals, and users. A detailed explanation is provided below.

[0544] Webtoon uploads by users

[0545] Users upload digital comic content from their own devices. This upload includes image data and text data. Users also enter meta information such as the title, author name, and language. For example, a user might enter information such as "Title: Magical Forest," "Author: Yamada Taro," and "Language: Japanese."

[0546] Receiving and temporarily storing content

[0547] The server stores the content and meta information received from the device in temporary storage. The format and size of the stored data are verified, and if there is any invalid data, an error message is displayed to the user. For example, if an image file is corrupted, the user is notified that the image file is invalid.

[0548] Extraction and translation of text data

[0549] The server uses a language analysis module to extract text data from uploaded digital comic content. The extracted text is then translated into multiple languages ​​using a translation engine. For example, Japanese can be translated into English, Chinese, and Spanish. The translation engine used could be software such as Google Translate API or DeepL.

[0550] Storing translated content

[0551] The server stores the translated digital comic content in a database and updates the metadata for each language version. For example, the English and Spanish versions are stored, and metadata such as "Title: The Magical Forest," "Artist: Yamada Taro," and "Language: English" is updated.

[0552] Multi-scenario and multi-ending generation

[0553] The server identifies key branching points based on the original content and automatically generates different scenarios and endings using a generative AI engine (e.g., GPT-3.5Turbo). This allows users to enjoy different story developments. For example, three different scenarios can be generated depending on the character's behavior choices.

[0554] Content Delivery

[0555] The device allows the user to select the desired language and scenario through a web interface. Based on the selection, the server delivers the most appropriate language and scenario content to the device. For example, if the user selects the English version of Scenario 2, that content will be displayed on the device.

[0556] Data collection and analysis

[0557] The server collects the user's browsing history and scenario selection data and records them in a database. The collected data is analyzed to determine the user's interests and browsing patterns. For example, if a user frequently selects a particular scenario, advertisements and paid content related to that scenario are displayed preferentially.

[0558] Ad Placement and Monetization

[0559] The server effectively places appropriate advertisements based on the collected data. It also recommends paid content and provides a personalized experience for users. For example, a user who likes fantasy scenarios will be shown advertisements for fantasy goods and related apps.

[0560] Prompt Sentence Examples

[0561] "Identify the main branching points in the webtoon and generate prompts to generate different scenarios. This prompt should be in the following format:

[0562] Input: A scene where Character A meets Character B

[0563] Output: Think of three different scenarios in which Character A could act after meeting Character B, and describe each scenario in 200 characters or less.

[0564] The above is a specific embodiment for carrying out the present invention. This system automatically supports multiple languages ​​and creates multiple scenarios for digital comics, and can provide optimal content based on user data.

[0565] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0566] Step 1:

[0567] Users upload digital comic content from their own devices. They provide image data, text data, and meta information (title, author name, language, etc.) as input. Specifically, they use a web interface to enter information such as "Title: Magical Forest," "Author: Yamada Taro," and "Language: Japanese," and then press the upload button.

[0568] Step 2:

[0569] The server saves the content and meta information received from the device in temporary storage. As input, it receives the digital comic file and meta information uploaded by the user. Specifically, the server verifies the format and size of the data, and if there is an abnormality, it generates an error message and notifies the user. As output, only valid data is temporarily stored, and invalid data is returned as an error message such as "The image file is invalid."

[0570] Step 3:

[0571] The server uses a language analysis module to extract text data from uploaded digital comic content. It receives webtoon image and text data as input. Specifically, it converts the text data in the image into text format using OCR (Optical Character Recognition) technology. The extracted text data is then generated as output.

[0572] Step 4:

[0573] The server translates the extracted text data into multiple languages. It receives Japanese character data as input. Specifically, it calls translation engines such as Google Translate API and DeepL to translate the Japanese text into English, Chinese, and Spanish. The translated text data is generated as output.

[0574] Step 5:

[0575] The server stores the translated digital comic content in a database. As input, it receives the translated text data and the original image data. Specific operations include saving the text and images in the corresponding fields of the database and updating the metadata (e.g., "Title: The Magical Forest," "Author: Yamada Taro," "Language: English"). As output, the database update is complete.

[0576] Step 6:

[0577] The server uses a generative AI engine to automatically generate multiple scenarios and endings based on the original content. It receives the original digital comic content and its main branching points as input. Specifically, it uses a generative AI model such as GPT-3.5 Turbo to create prompts for generating different scenarios and endings. As output, multiple scenarios and endings are generated and stored in a database.

[0578] Step 7:

[0579] The terminal allows the user to select the desired language and scenario through a web interface. As input, it receives the user's selected language and scenario information. In concrete terms, after the user selects the language and scenario, it sends the selection to the server. As output, the selection is transmitted to the server.

[0580] Step 8:

[0581] The server delivers the content of the selected language and scenario to the terminal. As input, it receives the language and scenario information selected by the user. Specifically, it retrieves the corresponding content from the database and sends it to the terminal. As output, the digital comic of the language and scenario selected by the user is displayed on the terminal.

[0582] Step 9:

[0583] The server collects users' browsing history and scenario selection data and records them in a database. As input, it receives the users' browsing and selection data. As a specific operation, it stores the collected data in the database in an appropriate format. As output, the recorded data is used for analysis.

[0584] Step 10:

[0585] The server analyzes the collected data and proposes ad placements and paid content. It receives user browsing data and selection data as input. Specifically, it uses data analysis algorithms to identify the user's interests and preferences and selects relevant ads and content. Personalized ads and paid content are proposed and displayed as output.

[0586] (Application example 1)

[0587] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0588] Webtoons are increasingly being enjoyed by a growing number of users, but language barriers and the limited variety of scenarios make their global adoption difficult. It is also challenging to effectively utilize user browsing data to suggest advertising placements and paid content. Furthermore, making them compatible with various devices is also a key issue.

[0589] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0590] In this invention, the server includes a means for uploading webtoon content from users, a means for temporarily storing the uploaded content, and a means for extracting text data from the content and translating it into multiple languages, thereby enabling webtoon content to be made available in multiple languages.

[0591] The system further includes a means for storing translated content in a database, a means for generating multiple scenarios and endings and storing them in a database, and a means for providing appropriate content based on a user's selection, thereby allowing users to enjoy webtoons in different languages ​​and scenarios at their leisure.

[0592] It also includes a means for collecting and analyzing user browsing data and a means for suggesting advertisement placement and paid content based on the analysis, which makes it possible to suggest advertisements and content personalized for each user.

[0593] Furthermore, it also includes a means for providing the system as an application installed on a smartphone, smart glasses, a head-mounted display, or a robot, which enables use on a variety of devices and improves user convenience.

[0594] "User" means an individual end user who uses the Webtoon Content.

[0595] "Webtoon content" refers to web manga or digital comics that are presented in digital format.

[0596] "Means for uploading" refers to a mechanism for transferring digital content owned by a user to a designated server.

[0597] A "temporary storage means" is a storage device or system for temporarily holding uploaded digital content.

[0598] "Means for extracting text data" refers to technology for identifying and obtaining text information from digital images and videos.

[0599] The "means for translating into multiple languages" is a translation engine or software for converting extracted text data into other languages.

[0600] "Means for storing data in a database" refers to a data management system for organizing and systematically storing digital data such as translated content and generated scenarios.

[0601] "Means for generating multiple scenarios and endings" refers to generative AI technology that automatically creates different developments and endings based on the original content.

[0602] "Means for providing appropriate content based on user selection" refers to a system that delivers optimal digital content according to the user's settings and preferences.

[0603] "Means for collecting and analyzing browsing data" refers to technology that records a user's operation history and browsing patterns and analyzes that data.

[0604] The "means for suggesting advertisement placement and paid content" is a system that presents advertisements and paid services that are most relevant to the user based on the analysis results.

[0605] An "application installed on a smartphone" is software that runs on a smartphone and provides various functions.

[0606] "Applications installed on smart glasses" are software that run on smart glasses and provide various functions.

[0607] An "application installed on a head-mounted display" is software that runs on the head-mounted display and provides various functions.

[0608] An "application installed on a robot" is software that runs on a robot and provides various functions.

[0609] This invention relates to a webtoon platform that can automatically translate multiple languages ​​and generate multiple scenarios using AI. The system of the present invention allows users to upload webtoon content, translate it into multiple languages, and provide it in the form of multiple scenarios and endings. It can also analyze users' viewing data to suggest effective advertising placements and paid content.

[0610] The system consists of the following elements: First, users upload webtoon content from their own devices. At this time, they enter meta information such as the title, author's name, and language. The uploaded content is temporarily stored on the server, where its format and size are verified.

[0611] The server then uses a language analysis module to extract text data from the uploaded webtoon content, and calls a translation engine to translate the extracted text into multiple languages, possibly using Google Cloud Translation API or DeepL API.

[0612] The translated webtoon content for each language version is stored in a database, and the metadata for each language version is also updated. A generative AI engine is then used to automatically generate different storylines and endings based on the original content. The generative AI engine uses OpenAI's GPT-4 or a similar natural language processing model.

[0613] Furthermore, user browsing data and scenario selection data are collected by the server and recorded in a database. This data is analyzed and used to analyze user interests and browsing patterns. Data analysis libraries such as Python's pandas and Scikit-learn are used for the analysis.

[0614] Ad placement and paid content suggestions are based on the collected data. An algorithm is implemented to effectively place appropriate ads, making it possible to provide users with a personalized experience.

[0615] Furthermore, this system is provided as an application that can be installed on smartphones, smart glasses, head-mounted displays, or robots, making it possible to use it on a variety of devices and improving user convenience.

[0616] For example, you can upload a Japanese webtoon and translate it into English, Chinese, and Spanish. You can also provide different scenarios depending on the character you choose. Below are some example prompts:

[0617] "If Character A gets lost, generate a scenario in which he goes to a different city. Output multiple endings along with background descriptions of those cities."

[0618] In this way, users can enjoy webtoons in different languages ​​and scenarios of their choice.

[0619] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0620] Step 1:

[0621] The user uploads webtoon content from their device to the server. At this time, the user enters meta information such as the title, author name, and language. The input data includes image files and text data. The server checks the received files and meta information to ensure that there is no invalid data. If the data is valid, it is temporarily stored.

[0622] Step 2:

[0623] The server extracts character data from the uploaded content. To obtain character data from image data, it uses OCR (Optical Character Recognition) technology. Specifically, it uses the Tesseract OCR library to recognize characters in the image and extract them as text data. This text data is then used for the next process.

[0624] Step 3:

[0625] The server translates the extracted text data into multiple languages. To do this, it uses the Google Cloud Translation API or the DeepL API. The extracted text data is sent to the API and translated into multiple target languages ​​(e.g., English, Chinese, Spanish). The translation results are generated in each language.

[0626] Step 4:

[0627] The server stores the translated webtoon content in a database and updates the metadata for each language. The database stores mapping information between the original language and the translated language, as well as text data for each language. This allows for the necessary data to be organized for subsequent scenario generation and distribution.

[0628] Step 5:

[0629] The server automatically generates multiple scenarios and endings based on the uploaded content using a generative AI engine. For example, OpenAI's GPT-4 is used to generate scenarios and endings based on prompts. The generated scenarios and endings are stored in a database.

[0630] Step 6:

[0631] The user accesses the web interface using a terminal and selects the desired language and scenario. The server delivers content in the optimal language and scenario to the terminal based on the user's selection, providing content files corresponding to the selected language and scenario as transmission data.

[0632] Step 7:

[0633] The server collects user browsing data and scenario selection data and records them in a database. The collected data is used for subsequent processing to analyze user interests and browsing patterns. Data analysis libraries such as Python's pandas and Scikit-learn are used to analyze trends in user behavior.

[0634] Step 8:

[0635] Based on the analysis results, the server effectively places appropriate advertisements and suggests paid content. Algorithms are implemented to provide personalized advertisements and content based on each user's preferences and browsing patterns. This increases creators' revenue and improves the user experience.

[0636] In this way, a system that supports multiple languages ​​and generates multiple scenarios can facilitate the global expansion of webtoon content while providing personalized services to users. Below is an example of a prompt sentence as a concrete example.

[0637] "If Character A gets lost, generate a scenario in which he goes to a different city. Output multiple endings along with background descriptions of those cities."

[0638] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0639] This invention relates to a webtoon platform that can be deployed globally by automatically creating multiple languages ​​and scenarios by combining generative AI and an emotion recognition engine. This system allows users to upload webtoon content, translate it into multiple languages, and provide it in the form of multiple scenarios and endings. It also provides content and advertising suggestions based on users' emotions through emotion recognition, and analyzes users' viewing data to evaluate creators and realize efficient monetization.

[0640] Program processing flow

[0641] 1. A user uploads a webtoon

[0642] User: Uploads webtoon content from their device, including image files and text data, and also enters meta information such as title, author name, and language.

[0643] Terminal: Accepts uploads from users via a web interface, including any meta information entered.

[0644] 2. Receiving and storing content

[0645] Server: Receives uploaded content and meta information and stores it in temporary storage. The format and size of the stored data are verified, and invalid data is notified to the user with an error message.

[0646] 3. Extraction and translation of text data

[0647] Server: Uses language analysis module to extract text data from uploaded webtoon content, and converts text in images into digital text using OCR technology.

[0648] Server: The extracted text data is sent to the multilingual translation module for automatic translation into multiple specified languages. For example, Japanese text is translated into English, Chinese, and Spanish.

[0649] 4. Saving translated content

[0650] Server: Reinserts the translated text into the original webtoon image to generate webtoons in each language. Stores the generated content in a database.

[0651] 5. Multi-scenario and multi-ending generation

[0652] Server: Based on the original content, identify key branching points and use a generative AI engine to automatically generate different scenarios and endings. Identify branching points in the scenario and create different progressions of the story.

[0653] Server: Stores the generated multiple scenarios and endings in a database, including metadata for each scenario and ending (e.g., scenario overview, branching points, etc.).

[0654] 6. Introducing Emotion Recognition

[0655] Device: The user's facial expressions and voice are sent to the emotion engine via the user's camera and microphone.

[0656] Server: Uses an emotion engine to analyze the user's emotions in real time from facial expressions and voice data.

[0657] 7. Content Delivery

[0658] Terminal: Provides an interface for users to select their preferred language and scenario through a web interface. Users can select their preferred settings.

[0659] Server: Retrieves the appropriate language version and scenario from the database based on the user's selection and the recognized emotion. Sends the retrieved content to the device.

[0660] 8. Data Collection and Analysis

[0661] Server: Collects user browsing and emotional data, including viewed scenarios, selected language, browsing time, perceived emotional state, etc. Stores the collected data in a database.

[0662] Server: Analyzes the collected data and derives insights to identify user preferences, interests, and emotional trends. Based on the results of this analysis, it proposes emotionally-based ad placements and paid content.

[0663] 9. Ad Placement and Monetization

[0664] Server: Based on the analysis results, it sends personalized ads and content recommendations to the device, so that the information most relevant to the user is displayed.

[0665] Example: If the user is recognized as "happy" by the emotion engine, relevant ads and content with upbeat content will be displayed preferentially.

[0666] 10. Feedback and Optimization

[0667] Server: Based on user interactions, further data is collected and used to improve the accuracy of the generative AI algorithms and emotion engine, thereby continuously optimizing the entire system.

[0668] By automating these processes, the system enables multilingual and multi-scenario webtoon content, and provides a personalized experience based on the user's emotions. The introduction of emotion recognition provides a more immersive experience, creating a new entertainment experience.

[0669] The processing flow will be explained below.

[0670] Step 1:

[0671] Users: Upload webtoon content from their devices, including image files (e.g., JPEG, PNG) and text files (e.g., TXT, PDF), along with meta information such as title, author name, and language.

[0672] Step 2:

[0673] Terminal: Receives content uploaded via the web interface and any meta information entered. Performs initial validation for proper format and size. If successful, stores the content in temporary storage.

[0674] Step 3:

[0675] Server: Analyzes the uploaded file and extracts text data from the image file, using OCR (Optical Character Recognition) technology to convert the text in the image into digital text.

[0676] Step 4:

[0677] Server: The extracted text data is sent to a multilingual translation module for automatic translation into multiple languages. For example, Japanese text is translated into English, Chinese, and Spanish.

[0678] Step 5:

[0679] Server: Reinserts the translated text into the original webtoon image to generate webtoons in each language. Stores the generated content in a database.

[0680] Step 6:

[0681] Server: Runs the multi-scenario / multi-ending generation module, automatically generating multiple scenarios and endings based on the original content. Identifies branching points in the scenario and creates different story progressions for each branch.

[0682] Step 7:

[0683] Server: The generated multiple scenarios and endings are saved in a database, including metadata for each scenario and ending (e.g., scenario overview, branching points, etc.).

[0684] Step 8:

[0685] Device: The user's facial expressions and voice are sent to the emotion engine via the user's camera and microphone. This data is used to analyze the user's emotions in real time.

[0686] Step 9:

[0687] Server: Analyzes the user's emotions from facial expressions and voice data using an emotion engine. The analysis results are used to provide content based on the user's emotional state.

[0688] Step 10:

[0689] Terminal: Provides an interface for users to select their preferred language and scenario through a web interface. Users select their preferred settings.

[0690] Step 11:

[0691] Server: Based on the user's selection and analyzed emotions, retrieves the appropriate language version and scenario from the database. Sends the retrieved content to the device.

[0692] Step 12:

[0693] Device: Displays the received content and allows users to view the webtoon.

[0694] Step 13:

[0695] Server: Collects user browsing and emotional data, such as viewed scenarios, selected language, browsing time, and perceived emotional state. Stores the collected data in a database.

[0696] Step 14:

[0697] Server: Analyzes the collected data and derives insights to identify user preferences, interests, and emotional trends. Based on the analysis results, it proposes emotionally-based ad placements and paid content.

[0698] Step 15:

[0699] Server: Based on the analysis results, sends personalized ads and content recommendations to the device, so that the user sees the most relevant information.

[0700] Step 16:

[0701] Device: Displaying received advertisements and content recommendations to you, and collecting further data based on your responses to help optimize the system.

[0702] The system automates these processes, enabling webtoon content to be multilingual and multi-scenario compatible, and also provides a personalized content experience through user sentiment analysis, creating new value in entertainment.

[0703] Example 2

[0704] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0705] Conventional webtoon platforms struggled with multilingual support and multiple scenarios, and lacked the ability to provide personalized content based on users' individual emotions and preferences. Effective advertising placement and monetization were also difficult. This has led to a demand for improved user experience and efficient monetization for creators.

[0706] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for uploading content from a user, a means for temporarily storing the uploaded content, a means including a language analysis module and a translation module for extracting character data from the content and translating it into multiple languages, a means for storing the translated content in a database, a means including a generation AI engine for generating different scenarios and endings, a means for storing the generated multiple scenarios and endings in a database, a means including an emotion recognition engine for collecting emotion data from the user's camera and microphone and performing emotion analysis, a means for providing optimal content based on the user's selection and the emotion analysis results, a means for collecting user browsing data and emotion data and storing them in a database, a means for analyzing the data and suggesting advertisement placement and paid content based on the analysis results, and a means for improving the accuracy of the generation AI algorithm and the emotion recognition engine using the collected and analyzed data. This facilitates multilingual support and multi-scenario development of webtoon content, enabling personalized experiences based on individual emotions and preferences. In addition, it also achieves effective advertisement placement and monetization.

[0707] "User" means an individual or organization that uses this system to upload and view content.

[0708] "Content" refers to the image files and text data of webtoons that users upload to the system.

[0709] The "temporary storage means" is a function for temporarily storing uploaded content in server storage or cloud storage for use in subsequent processing.

[0710] "Language Analysis Module" is a software unit for extracting text data from uploaded content. Specifically, it has the function of extracting text from images using OCR technology.

[0711] A "translation module" is a software unit that automatically translates extracted text data into multiple languages, for example, using natural language processing techniques.

[0712] "Generative AI engine" is a general term for machine learning models and algorithms that automatically generate different scenarios and endings based on the original content.

[0713] An "emotion recognition engine" is a software unit that analyzes a user's facial expressions and voice data collected through a camera and microphone to determine the user's emotional state in real time.

[0714] The "database" is an information management system for systematically storing various data used within the system (uploaded content, translated text, generated scenarios and endings, user emotional data and browsing data, etc.).

[0715] "Data analysis" is the process of analyzing collected user browsing data and sentiment data using statistical methods and machine learning techniques to derive insights.

[0716] "Ad placement" is the process of displaying advertisements that match the user's interests and emotions at the optimal time based on the analysis results.

[0717] "Paid content suggestion" is the process of recommending paid content that matches a user's interests and preferences based on data analysis.

[0718] This invention is a system that combines a generative AI model and an emotion recognition engine to realize multilingual and multi-scenario webtoon content, providing a personalized experience based on the user's emotions and preferences, and enabling efficient ad placement and monetization.

[0719] System configuration

[0720] The system includes the following major components:

[0721] 1. User Device:

[0722] Hardware: Devices such as PCs, smartphones, and tablets

[0723] Software: Web browser, camera, microphone

[0724] 2. Server:

[0725] Hardware: Cloud server, database server

[0726] Software: OCR technology, translation module, generative AI engine, emotion recognition engine, data analysis module

[0727] Processing flow

[0728] The system operates as follows:

[0729] 1. User uploads content:

[0730] User: Upload webtoon content from their own device and enter meta information.

[0731] Terminal: Accepts uploads from users through a web interface.

[0732] Example: A user opens a browser, clicks the "Choose File" button on a website's upload screen, selects an image file and text data, and clicks the "Upload" button.

[0733] 2. The server receives and stores the content:

[0734] Server: Receives uploaded content and meta information and temporarily stores it in cloud storage. Validates the format and size of the stored data and notifies the user of any invalid data with an error message.

[0735] Example: The server checks that the image file received is in JPEG format and that the file size is within 10MB, and if there are no problems, it saves it to cloud storage.

[0736] 3. Extract and translate the text data of the content:

[0737] Server: Extracts text data from uploaded webtoon content using OCR technology and sends it to the translation module for automatic translation into multiple languages.

[0738] Example: Use Amazon Textract to perform OCR analysis, and then use the Google Cloud Translation API to translate the acquired Japanese text data into English, Chinese, and Spanish.

[0739] 4. Saving translated content:

[0740] Server: The translated text is reinserted into the original webtoon image, and webtoons in each language are generated and stored in the database.

[0741] Example: Using Adobe Photoshop API to remap translated text onto webtoon images for each language version, and then storing the generated images in cloud storage, and storing information including metadata in a database.

[0742] 5. Multi-Scenario and Multi-Ending Generation:

[0743] Server: Using a generative AI engine, different scenarios and endings are automatically generated based on the original content and stored in a database.

[0744] Example: Using OpenAI's GPT-4 to identify branching points in a story and generate different scenarios and endings.

[0745] 6. Introducing Emotion Recognition:

[0746] Terminal: Collects emotional data through the user's camera and microphone and sends it to the server.

[0747] Server: Performs real-time emotion analysis using an emotion recognition engine and stores the results in a database.

[0748] Example: Use your smartphone's camera and microphone to capture facial expressions and voice, then use the Emotion API to analyze emotions in real time.

[0749] 7. Content Distribution:

[0750] Terminal: Allows users to select their preferred language and scenario through a web interface.

[0751] Server: Based on the user's selection and the results of sentiment analysis, retrieves appropriate content from the database and sends it to the device.

[0752] Example: A user selects a language and scenario on a website, and that information is sent to a server. The server retrieves the selected language and scenario and sends them to the device.

[0753] 8. Data Collection and Analysis:

[0754] Server: Collects user browsing data and emotion data and stores them in a database.

[0755] Server: Uses the data analysis module to analyze the collected data and derive insights.

[0756] Example: Use Google Analytics to collect browsing data, store emotion recognition data in a database, and use Apache Spark for analysis.

[0757] 9. Ad Placement and Monetization:

[0758] Server: Proposes ad placement and paid content based on the analysis results.

[0759] Example: Displaying personalized ads to users whose emotions are identified as "joy" through emotion recognition.

[0760] 10. Feedback and optimization:

[0761] Server: Based on user interactions, collects additional data to improve the accuracy of generative AI algorithms and emotion recognition engines.

[0762] Example: Continuously collect new browsing and sentiment data to retrain machine learning models.

[0763] Prompt Sentence Examples

[0764] "Design a system to translate the following Japanese text into English, Chinese, and Spanish, and generate different scenarios and endings. The system should allow users to enjoy content that matches their mood of the day through emotion recognition. The system should be uploaded by the user using their smartphone, and emotions should be recognized using the camera and microphone."

[0765] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0766] Step 1:

[0767] User uploads content:

[0768] Input: Image files, text data, and meta information (title, author name, language, etc.) of webtoons selected by the user from their device.

[0769] Process: A user opens the website's upload screen in a browser, clicks the "Choose File" button, selects an image file and text data, and clicks the "Upload" button.

[0770] Output: The device receives the uploaded file and meta information.

[0771] Step 2:

[0772] The server receives and stores the content:

[0773] Input: Webtoon image files, text data, and meta information sent from the device.

[0774] Processing: The server receives the uploaded content and meta information, temporarily stores it in cloud storage, validates the format (JPEG, PNG, etc.) and file size of the received image file, and sends an error message to the user if there is invalid data.

[0775] Output: Saved content and meta information.

[0776] Step 3:

[0777] Extract and translate text data from content:

[0778] Input: Webtoon image files stored in cloud storage.

[0779] Processing: The server uses OCR technology (e.g., Amazon Textract) to extract the text data in the image, and then sends the text data to a multilingual translation module (e.g., Google Cloud Translation API) to automatically translate the Japanese text into English, Chinese, and Spanish.

[0780] Output: Translated text data in multiple languages.

[0781] Step 4:

[0782] Saving translated content:

[0783] Input: translated text data, original webtoon images.

[0784] Processing: The server reinserts the translated text into the original webtoon image to generate the webtoon in each language (e.g., using Adobe Photoshop API). The generated webtoon in each language is then stored in a database.

[0785] Output: Multilingual webtoon content stored in a database.

[0786] Step 5:

[0787] Multi-scenario and multi-ending generation:

[0788] Input: Original webtoon content.

[0789] Processing: The server uses a generative AI engine (e.g., OpenAI's GPT-4) to automatically generate different scenarios and endings based on the original content and store them in a database.

[0790] Output: Multiple scenarios and endings stored in a database.

[0791] Step 6:

[0792] Introducing emotion recognition:

[0793] Input: User facial and voice data collected through the device's camera and microphone.

[0794] Processing: The server receives facial expression and voice data sent from the user's device and analyzes it in real time using an emotion recognition engine (e.g., Emotion API).

[0795] Output: Parsed emotion data.

[0796] Step 7:

[0797] Content Delivery:

[0798] Input: User choices (language, scenario), parsed emotion data.

[0799] Processing: The user selects the desired language and scenario through a web interface, and that information is sent to the server. Based on the user's selection and the results of sentiment analysis, the server retrieves appropriate content from a database and sends it to the device.

[0800] Output: The appropriate content is delivered to the user's device.

[0801] Step 8:

[0802] Data collection and analysis:

[0803] Input: User browsing data, emotion data.

[0804] Processing: The server collects user browsing data and sentiment data and stores it in a database. A data analysis module (e.g., Apache Spark) is used to analyze the collected data and derive insights into user preferences and sentiment trends.

[0805] Output: Collected and analyzed data, analytical insights.

[0806] Step 9:

[0807] Ad Placement and Monetization:

[0808] Input: Analytics Insights.

[0809] Processing: The server displays personalized advertisements to the user based on the analysis results and suggests relevant paid content.

[0810] Output: Advertisements and paid content suggestions displayed to the user.

[0811] Step 10:

[0812] Feedback and optimization:

[0813] Input: Collected browsing data, sentiment data, and interaction data.

[0814] Processing: The server uses this data to retrain the generative AI algorithms and emotion recognition engines to improve the accuracy of the system.

[0815] Output: Optimized generative AI model and emotion recognition engine.

[0816] (Application example 2)

[0817] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0818] In addition to providing a webtoon platform that supports multiple languages ​​and allows for individual scenario selection, there is a need for a platform that can provide optimal content and advertisements in real time based on user emotions. However, current systems have difficulty generating flexible scenarios that respond to diverse user emotions and preferences, and providing personalized advertisements. Therefore, an effective system is needed that can simultaneously improve user experience and monetize creators and advertisers.

[0819] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for uploading image content from a user, means for temporarily storing the uploaded content, means for extracting text data from the content and translating it into multiple languages, means for storing the translated content in a database, means for identifying major branching points and generating multiple scenarios and endings, means for recognizing the user's emotions through a camera or microphone, means for providing optimal content and advertisements based on the recognized emotions, means for collecting and analyzing user viewing data and emotion data, and means for suggesting advertisement placement and paid content based on the above analysis. This makes it possible to provide a personalized experience based on the user's emotions and optimize content and advertisements.

[0820] "User" refers to an individual or corporation that uses the system to upload, view, select, and perform other operations on webtoons.

[0821] "Image content" refers to digital data that is primarily composed of visual elements, such as webtoons, comic pages, and illustrations.

[0822] "Means for uploading" refers to the functions and interfaces that allow users to send image content to the server.

[0823] "Temporary storage" means memory or storage for short-term storage of uploaded content.

[0824] "Means for extracting text data" refers to OCR (optical character recognition) technology or software for extracting text information from image content.

[0825] "Means for translating into multiple languages" refers to a translation engine or API for translating extracted text data into multiple languages.

[0826] "Means for storing in a database" refers to a data management system for long-term storage of translated content and generated scenarios.

[0827] "Branch point identification methods" refers to algorithms or software that automatically identify important choices and directions in the development of a story.

[0828] "Means for generating scenarios and endings" refers to generative AI technology that generates different story developments and endings based on branching points.

[0829] "Means of recognition through cameras and microphones" refers to hardware and software for capturing the user's facial expressions and voice and analyzing their emotions.

[0830] "Means for providing content based on recognized emotions" refers to a system for displaying optimal content and advertisements based on analyzed emotional data.

[0831] "Means for collecting and analyzing browsing data and emotional data" refers to a data processing system for recording and analyzing a user's content operation history and recognized emotional data.

[0832] "Means for suggesting advertising placements and paid content" refers to algorithms and systems that recommend optimal advertising and paid content to users based on collected data.

[0833] The present invention relates to a webtoon platform that supports multiple languages ​​and allows individual scenario selection, and is a system that provides optimal content and advertisements based on user emotions. Specific embodiments will be described below.

[0834] System Program Overview

[0835] This system implements a program that includes the following main functions:

[0836] 1. Upload and save webtoons

[0837] The server provides an interface for users to upload image content (e.g., webtoons), which is then temporarily stored.

[0838] 2. Extraction and translation of text data

[0839] The server uses an OCR (Optical Character Recognition) engine to extract text data from image content, which is then translated into multiple languages ​​using the Google Translate API.

[0840] 3. Identifying and generating scenario branching points

[0841] The server uses the extracted text data to identify key story branching points, and uses a generative AI model to automatically generate different scenarios and endings, which are then stored in a database.

[0842] 4. Emotion recognition and personalized content delivery

[0843] It uses the camera and microphone on the device (e.g., smartphone) to recognize the user's emotions in real time. It uses the EmotionRecognizer module to analyze the user's emotional state from their facial expressions and voice. Based on the analysis results, it provides optimal content and personalized advertisements.

[0844] 5. Data Collection and Analysis

[0845] The server collects user browsing data and emotional data and stores it in a database. This data is analyzed to gain insights into user preferences and behavioral patterns. This data is used to optimize ad placement and suggest paid content.

[0846] Hardware and software used

[0847] Hardware

[0848] Devices such as smartphones and tablets: Used to upload and view content from users.

[0849] Camera and microphone: Captures the user's facial expressions and voice for emotion recognition.

[0850] software

[0851] OCR engine: Used to extract text from images.

[0852] Google Translate API: Used to translate the extracted text into multiple languages.

[0853] EmotionRecognizer: A software module for recognizing user emotions.

[0854] Generative AI models: Used to automatically generate different scenarios and endings.

[0855] Specific examples

[0856] For example, if a user opens the app at night and emotion recognition detects the emotion of "joy," the server will prioritize displaying cheerful and upbeat stories and related ads, providing the best possible experience for the user.

[0857] Prompt Sentence Examples

[0858] An example of a specific prompt for the generative AI model is, "Please generate what kind of story development would be best for a user who is in a joyful mood." Based on this prompt, the generative AI model automatically generates multiple story developments that suit the user's emotional state.

[0859] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0860] Step 1:

[0861] Users upload image content (e.g., webtoons) from their devices. They also enter meta information such as the title, author's name, and language used. The server receives this input data and temporarily stores it. Specifically, data is sent via a web interface, and the server temporarily stores the uploaded image file and meta information.

[0862] Step 2:

[0863] The server extracts text data from the stored image content, using an OCR engine (e.g., Google Cloud Vision API) to convert the text in the image into digital text. This allows the character information embedded in the image to be obtained. The input is the temporarily stored image data, and the output is digital text.

[0864] Step 3:

[0865] The server translates the extracted text data into multiple languages. Specifically, it uses the Google Translate API to translate, for example, Japanese into English, Chinese, Spanish, etc. This process generates translated text (output) into each target language based on the original text (input).

[0866] Step 4:

[0867] The server reinserts the translated text into the original webtoon image to generate the webtoon in each language. The generated content for each language is stored in a database. The input is the translated text and the original image file, and the output is a multilingual webtoon image. In this reinsertion process, the text is embedded into the image using image editing software (e.g., Python's Pillow library).

[0868] Step 5:

[0869] The server identifies key branching points based on the original content. Based on the identified branching points, it uses a generative AI model to automatically generate different scenarios and endings. In this process, multiple scenarios and endings (outputs) are generated from the original content (input) and stored in a database. Text analysis algorithms and generative AI algorithms are used for identification and generation.

[0870] Step 6:

[0871] The device captures the user's facial expressions and voice through the user's camera and microphone and sends them to the server. The server then uses the EmotionRecognizer module to analyze the user's emotions in real time from the transmitted facial and voice data. The input is the captured multimedia data, and the output is the analyzed emotional information.

[0872] Step 7:

[0873] The server provides optimal content and personalized advertisements based on the analyzed emotional data. This involves retrieving webtoons with the appropriate language version and scenario from a database and sending them to the user's device. The input is the emotional data and user selection information, and the output is the appropriate content and advertisements.

[0874] Step 8:

[0875] The server collects users' browsing data and emotional data and stores it in a database. The collected data is analyzed to derive insights that identify users' preferences, interests, and emotional trends. The input is user interaction data, and the output is the analysis results. Data mining techniques and machine learning algorithms are used for the analysis.

[0876] Step 9:

[0877] The server then uses the analysis results to suggest ad placements and paid content, thereby displaying the most relevant ads and content to the user. The input is the analysis results, and the output is a list of recommended ads and paid content. A recommendation system algorithm is used in this process.

[0878] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0879] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0880] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0881] [Third embodiment]

[0882] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0883] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0884] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0885] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0886] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0887] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0888] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0889] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0890] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0891] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0892] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0893] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0894] This invention relates to a webtoon platform that can be deployed globally by automatically creating multiple languages ​​and scenarios using generative AI. The system allows users to upload webtoon content, translate it into multiple languages, and provide it in the form of multiple scenarios and endings. It also analyzes user viewing data to suggest effective advertising placements and paid content.

[0895] Program processing flow

[0896] 1. A user uploads a webtoon

[0897] Users: Upload webtoon content from their own devices, including image data and text data.

[0898] Terminal: Accepts uploads from users through a web interface, along with meta information such as title, author, and language.

[0899] 2. Receiving and temporarily storing content

[0900] Server: Receives uploaded content and meta information and stores it in temporary storage. The format and size of the stored data are verified, and invalid data is notified to the user with an error message.

[0901] 3. Extraction and translation of text data

[0902] Server: Uses a language analysis module to extract text data from the uploaded webtoon content, then invokes a translation engine to translate the extracted text into multiple languages.

[0903] Example: Translations are made from Japanese to English, Chinese, and Spanish.

[0904] 4. Saving translated content

[0905] Server: Stores the translated webtoon content in a database and updates the metadata for each language version.

[0906] 5. Multi-scenario and multi-ending generation

[0907] Server: Based on the original content, it identifies key branching points and uses a generative AI engine to automatically generate different scenarios and endings, allowing users to enjoy different story developments depending on their choices.

[0908] Example: Multiple scenarios are generated in which the story progresses in different directions depending on the character's choices.

[0909] 6. Content Delivery

[0910] Device: The user selects the desired language and scenario through a web interface. Based on the selection, the server delivers the most appropriate language and scenario content to the device.

[0911] 7. Data Collection and Analysis

[0912] Server: Collects users' browsing history and scenario selection data and records them in a database. Analyzes the collected data to analyze users' interests and browsing patterns.

[0913] Example: Identifying a user's preferences based on the scenarios and endings they frequently choose.

[0914] 8. Ad Placement and Monetization

[0915] Server: Uses collected data to effectively place appropriate advertisements and recommend paid content, providing users with a personalized experience.

[0916] Example: For users who frequently select a particular scenario, advertisements and paid content related to that scenario will be displayed preferentially.

[0917] By automating these processes, the system will facilitate the global expansion of webtoon content, enabling it to be translated into multiple languages ​​and scenarios. It also utilizes user data to suggest optimal advertising placements and paid content, ensuring that creators are properly evaluated and fostering a sustainable production environment.

[0918] The processing flow will be explained below.

[0919] Step 1:

[0920] User: Uploads webtoon content, including image files (e.g., JPEG, PNG) and text files (e.g., TXT, PDF), as well as meta information such as title, author name, and language.

[0921] Step 2:

[0922] Terminal: Receives content and meta information uploaded through the web interface. After receiving, it performs an initial validation to ensure the file format and size are correct. If there are no issues, it stores the data in temporary storage.

[0923] Step 3:

[0924] Server: Analyzes the uploaded content and extracts text data from image files. Converts text in images into digital text using OCR (Optical Character Recognition) technology.

[0925] Step 4:

[0926] Server: The extracted text data is sent to the multilingual translation module, which uses a generative AI engine to automatically translate the text into multiple specified languages. For example, Japanese text is translated into English, Chinese, and Spanish.

[0927] Step 5:

[0928] Server: Reinserts the translated text into the original webtoon image to generate webtoons in each language. Stores the generated content in a database.

[0929] Step 6:

[0930] Server: Runs the multi-scenario / multi-ending generation module to generate multiple scenarios and endings based on the original webtoon content, identifying branching points in the scenario and creating different progressions for the story.

[0931] Step 7:

[0932] Server: Stores the generated multiple scenarios and endings in a database, including metadata for each scenario and ending (e.g., scenario overview, branching points, etc.).

[0933] Step 8:

[0934] Terminal: Provides an interface for users to select language and scenario through a web interface. Users select their preferred settings.

[0935] Step 9:

[0936] Server: Retrieves the appropriate language version and scenario from the database based on the user's selection. Sends the retrieved content to the device.

[0937] Step 10:

[0938] Device: Displays the received content and allows users to view the webtoon.

[0939] Step 11:

[0940] Server: Collects user browsing data, including the scenarios viewed, language selection, browsing time, etc. Stores the collected data in a database.

[0941] Step 12:

[0942] Server: Analyzes the collected data and derives insights to identify user preferences and interests. Based on the results of this analysis, it proposes optimal ad placements and paid content.

[0943] Step 13:

[0944] Server: Based on the analysis results, it sends personalized ads and content recommendations to the device, so that the information most relevant to the user is displayed.

[0945] Step 14:

[0946] On your device: Displaying received advertisements and content recommendations to you. Depending on your interactions, collecting further data to help optimize the platform.

[0947] These are the specific processing steps of the global webtoon platform that uses generative AI to automatically translate webtoons into multiple languages ​​and create multiple scenarios. This system enables webtoon content to be translated into multiple languages ​​and offers multiple scenarios, providing users with a new entertainment experience.

[0948] Example 1

[0949] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0950] Currently, the global expansion of digital comics requires multilingual support and the provision of a variety of scenarios and endings. However, doing this manually is extremely time-consuming and costly. Other issues include the lack of automated systems for providing personalized content tailored to users' tastes and preferences, effective advertising placement, and multilingual support. Unless these issues are resolved, they will become serious obstacles to the global expansion of digital comics.

[0951] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0952] In this invention, the server includes means for uploading digital comic content from users, means for temporarily storing the uploaded content, means for extracting character data from the content and translating it into multiple languages, means for storing the translated content in a database, means for using a generation AI engine to generate multiple scenarios and endings, means for storing the generated scenarios and endings in a database, means for providing content in an appropriate language version and scenario based on user selection, means for collecting and analyzing user viewing data, and means for suggesting advertising placement and paid content based on the analysis. This makes it possible to make digital comics multilingual and multi-scenarios and provide optimal content based on user data.

[0953] "User" refers to a person who uses digital comic content.

[0954] "Digital comic content" refers to comic or manga data provided in electronic format.

[0955] "Upload" refers to the act of transferring data from a user's device to a server.

[0956] "Means" refers to a method or device for achieving a particular purpose.

[0957] "Temporary storage" refers to the process of temporarily storing data.

[0958] "Text data" refers to the text and textual information within the Content.

[0959] "Translating into multiple languages" refers to the process of converting from one language to another.

[0960] "Database" refers to a system for efficiently storing and managing data.

[0961] A "generative AI engine" refers to a program that uses artificial intelligence to process data and generate new content.

[0962] "Scenario" refers to the flow of development of a story or content.

[0963] "Ending" refers to the conclusion or final part of a story or content.

[0964] "Providing" refers to the act of supplying specific services or content to users.

[0965] "Viewing data" refers to recorded information when a user views content.

[0966] "Analysis" refers to the process of examining data in detail and deriving trends and meaning.

[0967] "Ad placement" refers to the act of appropriately placing an advertisement on a user's screen.

[0968] "Paid Content" means additional content that a User is required to purchase or pay for.

[0969] This invention relates to a digital comic platform that can be deployed globally by automatically generating multiple languages ​​and scenarios using generative AI. The main components of this system are a server, terminals, and users. A detailed explanation is provided below.

[0970] Webtoon uploads by users

[0971] Users upload digital comic content from their own devices. This upload includes image data and text data. Users also enter meta information such as the title, author name, and language. For example, a user might enter information such as "Title: Magical Forest," "Author: Yamada Taro," and "Language: Japanese."

[0972] Receiving and temporarily storing content

[0973] The server stores the content and meta information received from the device in temporary storage. The format and size of the stored data are verified, and if there is any invalid data, an error message is displayed to the user. For example, if an image file is corrupted, the user is notified that the image file is invalid.

[0974] Extraction and translation of text data

[0975] The server uses a language analysis module to extract text data from uploaded digital comic content. The extracted text is then translated into multiple languages ​​using a translation engine. For example, Japanese can be translated into English, Chinese, and Spanish. The translation engine used could be software such as Google Translate API or DeepL.

[0976] Storing translated content

[0977] The server stores the translated digital comic content in a database and updates the metadata for each language version. For example, the English and Spanish versions are stored, and metadata such as "Title: The Magical Forest," "Artist: Yamada Taro," and "Language: English" is updated.

[0978] Multi-scenario and multi-ending generation

[0979] The server identifies key branching points based on the original content and automatically generates different scenarios and endings using a generative AI engine (e.g., GPT-3.5Turbo). This allows users to enjoy different story developments. For example, three different scenarios can be generated depending on the character's behavior choices.

[0980] Content Delivery

[0981] The device allows the user to select the desired language and scenario through a web interface. Based on the selection, the server delivers the most appropriate language and scenario content to the device. For example, if the user selects the English version of Scenario 2, that content will be displayed on the device.

[0982] Data collection and analysis

[0983] The server collects the user's browsing history and scenario selection data and records them in a database. The collected data is analyzed to determine the user's interests and browsing patterns. For example, if a user frequently selects a particular scenario, advertisements and paid content related to that scenario are displayed preferentially.

[0984] Ad Placement and Monetization

[0985] The server effectively places appropriate advertisements based on the collected data. It also recommends paid content and provides a personalized experience for users. For example, a user who likes fantasy scenarios will be shown advertisements for fantasy goods and related apps.

[0986] Prompt Sentence Examples

[0987] "Identify the main branching points in the webtoon and generate prompts to generate different scenarios. This prompt should be in the following format:

[0988] Input: A scene where Character A meets Character B

[0989] Output: Think of three different scenarios in which Character A could act after meeting Character B, and describe each scenario in 200 characters or less.

[0990] The above is a specific embodiment for carrying out the present invention. This system automatically supports multiple languages ​​and multiple scenarios for digital comics, and can provide optimal content based on user data.

[0991] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0992] Step 1:

[0993] Users upload digital comic content from their own devices. They provide image data, text data, and meta information (title, author name, language, etc.) as input. Specifically, they use a web interface to enter information such as "Title: Magical Forest," "Author: Yamada Taro," and "Language: Japanese," and then press the upload button.

[0994] Step 2:

[0995] The server saves the content and meta information received from the device in temporary storage. As input, it receives the digital comic file and meta information uploaded by the user. Specifically, the server verifies the format and size of the data, and if there is an abnormality, it generates an error message and notifies the user. As output, only valid data is temporarily stored, and invalid data is returned as an error message such as "The image file is invalid."

[0996] Step 3:

[0997] The server uses a language analysis module to extract text data from uploaded digital comic content. It receives webtoon image and text data as input. Specifically, it converts the text data in the image into text format using OCR (Optical Character Recognition) technology. The extracted text data is then generated as output.

[0998] Step 4:

[0999] The server translates the extracted text data into multiple languages. It receives Japanese character data as input. Specifically, it calls translation engines such as Google Translate API and DeepL to translate the Japanese text into English, Chinese, and Spanish. The translated text data is generated as output.

[1000] Step 5:

[1001] The server stores the translated digital comic content in a database. As input, it receives the translated text data and the original image data. Specific operations include saving the text and images in the corresponding fields of the database and updating the metadata (e.g., "Title: The Magical Forest," "Author: Yamada Taro," "Language: English"). As output, the database update is complete.

[1002] Step 6:

[1003] The server uses a generative AI engine to automatically generate multiple scenarios and endings based on the original content. It receives the original digital comic content and its main branching points as input. Specifically, it uses a generative AI model such as GPT-3.5 Turbo to create prompts for generating different scenarios and endings. As output, multiple scenarios and endings are generated and stored in a database.

[1004] Step 7:

[1005] The terminal allows the user to select the desired language and scenario through a web interface. As input, it receives the user's selected language and scenario information. In concrete terms, after the user selects the language and scenario, it sends the selection to the server. As output, the selection is transmitted to the server.

[1006] Step 8:

[1007] The server delivers the content of the selected language and scenario to the terminal. As input, it receives the language and scenario information selected by the user. Specifically, it retrieves the corresponding content from the database and sends it to the terminal. As output, the digital comic of the language and scenario selected by the user is displayed on the terminal.

[1008] Step 9:

[1009] The server collects users' browsing history and scenario selection data and records them in a database. As input, it receives the users' browsing and selection data. As a specific operation, it stores the collected data in the database in an appropriate format. As output, the recorded data is used for analysis.

[1010] Step 10:

[1011] The server analyzes the collected data and proposes ad placements and paid content. It receives user browsing data and selection data as input. Specifically, it uses data analysis algorithms to identify the user's interests and preferences and selects relevant ads and content. Personalized ads and paid content are proposed and displayed as output.

[1012] (Application example 1)

[1013] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1014] Webtoons are increasingly being enjoyed by a growing number of users, but language barriers and the limited variety of scenarios make their global adoption difficult. It is also challenging to effectively utilize user browsing data to suggest advertising placements and paid content. Furthermore, making them compatible with various devices is also a key issue.

[1015] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1016] In this invention, the server includes a means for uploading webtoon content from users, a means for temporarily storing the uploaded content, and a means for extracting text data from the content and translating it into multiple languages, thereby enabling webtoon content to be made available in multiple languages.

[1017] The system further includes a means for storing translated content in a database, a means for generating multiple scenarios and endings and storing them in a database, and a means for providing appropriate content based on a user's selection, thereby allowing users to enjoy webtoons in different languages ​​and scenarios at their leisure.

[1018] It also includes a means for collecting and analyzing user browsing data and a means for suggesting advertisement placement and paid content based on the analysis, which makes it possible to suggest advertisements and content personalized for each user.

[1019] Furthermore, it also includes a means for providing the system as an application installed on a smartphone, smart glasses, a head-mounted display, or a robot, which enables use on a variety of devices and improves user convenience.

[1020] "User" means an individual end user who uses the Webtoon Content.

[1021] "Webtoon content" refers to web manga or digital comics that are presented in digital format.

[1022] "Means for uploading" refers to a mechanism for transferring digital content owned by a user to a designated server.

[1023] A "temporary storage means" is a storage device or system for temporarily holding uploaded digital content.

[1024] "Means for extracting text data" refers to technology for identifying and obtaining text information from digital images and videos.

[1025] The "means for translating into multiple languages" is a translation engine or software for converting extracted text data into other languages.

[1026] "Means for storing data in a database" refers to a data management system for organizing and systematically storing digital data such as translated content and generated scenarios.

[1027] "Means for generating multiple scenarios and endings" refers to generative AI technology that automatically creates different developments and endings based on the original content.

[1028] "Means for providing appropriate content based on user selection" refers to a system that delivers optimal digital content according to the user's settings and preferences.

[1029] "Means for collecting and analyzing browsing data" refers to technology that records a user's operation history and browsing patterns and analyzes that data.

[1030] The "means for suggesting advertisement placement and paid content" is a system that presents advertisements and paid services that are most relevant to the user based on the analysis results.

[1031] An "application installed on a smartphone" is software that runs on a smartphone and provides various functions.

[1032] "Applications installed on smart glasses" are software that run on smart glasses and provide various functions.

[1033] An "application installed on a head-mounted display" is software that runs on the head-mounted display and provides various functions.

[1034] An "application installed on a robot" is software that runs on a robot and provides various functions.

[1035] This invention relates to a webtoon platform that can automatically translate multiple languages ​​and generate multiple scenarios using AI. The system of the present invention allows users to upload webtoon content, translate it into multiple languages, and provide it in the form of multiple scenarios and endings. It can also analyze users' viewing data to suggest effective advertising placements and paid content.

[1036] The system consists of the following elements: First, users upload webtoon content from their own devices. At this time, they enter meta information such as the title, author's name, and language. The uploaded content is temporarily stored on the server, where its format and size are verified.

[1037] The server then uses a language analysis module to extract text data from the uploaded webtoon content, and calls a translation engine to translate the extracted text into multiple languages, possibly using Google Cloud Translation API or DeepL API.

[1038] The translated webtoon content for each language version is stored in a database, and the metadata for each language version is also updated. A generative AI engine is then used to automatically generate different storylines and endings based on the original content. The generative AI engine uses OpenAI's GPT-4 or a similar natural language processing model.

[1039] Furthermore, user browsing data and scenario selection data are collected by the server and recorded in a database. This data is analyzed and used to analyze user interests and browsing patterns. Data analysis libraries such as Python's pandas and Scikit-learn are used for the analysis.

[1040] Ad placement and paid content suggestions are based on the collected data. An algorithm is implemented to effectively place appropriate ads, making it possible to provide users with a personalized experience.

[1041] Furthermore, this system is provided as an application that can be installed on smartphones, smart glasses, head-mounted displays, or robots, making it possible to use it on a variety of devices and improving user convenience.

[1042] For example, you can upload a Japanese webtoon and translate it into English, Chinese, and Spanish. You can also provide different scenarios depending on the character you choose. Below are some example prompts:

[1043] "If Character A gets lost, generate a scenario in which he goes to a different city. Output multiple endings along with background descriptions of those cities."

[1044] In this way, users can enjoy webtoons in different languages ​​and scenarios of their choice.

[1045] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1046] Step 1:

[1047] The user uploads webtoon content from their device to the server. At this time, the user enters meta information such as the title, author name, and language. The input data includes image files and text data. The server checks the received files and meta information to ensure that there is no invalid data. If the data is valid, it is temporarily stored.

[1048] Step 2:

[1049] The server extracts character data from the uploaded content. To obtain character data from image data, it uses OCR (Optical Character Recognition) technology. Specifically, it uses the Tesseract OCR library to recognize characters in the image and extract them as text data. This text data is then used for the next process.

[1050] Step 3:

[1051] The server translates the extracted text data into multiple languages. To do this, it uses the Google Cloud Translation API or the DeepL API. The extracted text data is sent to the API and translated into multiple target languages ​​(e.g., English, Chinese, Spanish). The translation results are generated in each language.

[1052] Step 4:

[1053] The server stores the translated webtoon content in a database and updates the metadata for each language. The database stores mapping information between the original language and the translated language, as well as text data for each language. This allows for the necessary data to be organized for subsequent scenario generation and distribution.

[1054] Step 5:

[1055] The server automatically generates multiple scenarios and endings based on the uploaded content using a generative AI engine. For example, OpenAI's GPT-4 is used to generate scenarios and endings based on prompts. The generated scenarios and endings are stored in a database.

[1056] Step 6:

[1057] The user accesses the web interface using a terminal and selects the desired language and scenario. The server delivers content in the optimal language and scenario to the terminal based on the user's selection, providing content files corresponding to the selected language and scenario as transmission data.

[1058] Step 7:

[1059] The server collects user browsing data and scenario selection data and records them in a database. The collected data is used for subsequent processing to analyze user interests and browsing patterns. Data analysis libraries such as Python's pandas and Scikit-learn are used to analyze trends in user behavior.

[1060] Step 8:

[1061] Based on the analysis results, the server effectively places appropriate advertisements and suggests paid content. Algorithms are implemented to provide personalized advertisements and content based on each user's preferences and browsing patterns. This increases creators' revenue and improves the user experience.

[1062] In this way, a system that supports multiple languages ​​and generates multiple scenarios can facilitate the global expansion of webtoon content while providing personalized services to users. Below is an example of a prompt sentence as a concrete example.

[1063] "If Character A gets lost, generate a scenario in which he goes to a different city. Output multiple endings along with background descriptions of those cities."

[1064] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1065] This invention relates to a webtoon platform that can be deployed globally by automatically creating multiple languages ​​and scenarios by combining generative AI and an emotion recognition engine. This system allows users to upload webtoon content, translate it into multiple languages, and provide it in the form of multiple scenarios and endings. It also provides content and advertising suggestions based on users' emotions through emotion recognition, and analyzes users' viewing data to evaluate creators and realize efficient monetization.

[1066] Program processing flow

[1067] 1. A user uploads a webtoon

[1068] User: Uploads webtoon content from their device, including image files and text data, and also enters meta information such as title, author name, and language.

[1069] Terminal: Accepts uploads from users via a web interface, including any meta information entered.

[1070] 2. Receiving and storing content

[1071] Server: Receives uploaded content and meta information and stores it in temporary storage. The format and size of the stored data are verified, and invalid data is notified to the user with an error message.

[1072] 3. Extraction and translation of text data

[1073] Server: Uses language analysis module to extract text data from uploaded webtoon content, and converts text in images into digital text using OCR technology.

[1074] Server: The extracted text data is sent to the multilingual translation module for automatic translation into multiple specified languages. For example, Japanese text is translated into English, Chinese, and Spanish.

[1075] 4. Saving translated content

[1076] Server: Reinserts the translated text into the original webtoon image to generate webtoons in each language. Stores the generated content in a database.

[1077] 5. Multi-scenario and multi-ending generation

[1078] Server: Based on the original content, identify key branching points and use a generative AI engine to automatically generate different scenarios and endings. Identify branching points in the scenario and create different progressions of the story.

[1079] Server: Stores the generated multiple scenarios and endings in a database, including metadata for each scenario and ending (e.g., scenario overview, branching points, etc.).

[1080] 6. Introducing Emotion Recognition

[1081] Device: The user's facial expressions and voice are sent to the emotion engine via the user's camera and microphone.

[1082] Server: Uses an emotion engine to analyze the user's emotions in real time from facial expressions and voice data.

[1083] 7. Content Delivery

[1084] Terminal: Provides an interface for users to select their preferred language and scenario through a web interface. Users can select their preferred settings.

[1085] Server: Retrieves the appropriate language version and scenario from the database based on the user's selection and the recognized emotion. Sends the retrieved content to the device.

[1086] 8. Data Collection and Analysis

[1087] Server: Collects user browsing and emotional data, including viewed scenarios, selected language, browsing time, perceived emotional state, etc. Stores the collected data in a database.

[1088] Server: Analyzes the collected data and derives insights to identify user preferences, interests, and emotional trends. Based on the results of this analysis, it proposes emotionally-based ad placements and paid content.

[1089] 9. Ad Placement and Monetization

[1090] Server: Based on the analysis results, it sends personalized ads and content recommendations to the device, so that the information most relevant to the user is displayed.

[1091] Example: If the user is recognized as "happy" by the emotion engine, relevant ads and content with upbeat content will be displayed preferentially.

[1092] 10. Feedback and Optimization

[1093] Server: Based on user interactions, further data is collected and used to improve the accuracy of the generative AI algorithms and emotion engine, thereby continuously optimizing the entire system.

[1094] By automating these processes, the system enables multilingual and multi-scenario webtoon content, and provides a personalized experience based on the user's emotions. The introduction of emotion recognition provides a more immersive experience, creating a new entertainment experience.

[1095] The processing flow will be explained below.

[1096] Step 1:

[1097] Users: Upload webtoon content from their devices, including image files (e.g., JPEG, PNG) and text files (e.g., TXT, PDF), along with meta information such as title, author name, and language.

[1098] Step 2:

[1099] Terminal: Receives content uploaded via the web interface and any meta information entered. Performs initial validation for proper format and size. If successful, stores the content in temporary storage.

[1100] Step 3:

[1101] Server: Analyzes the uploaded file and extracts text data from the image file, using OCR (Optical Character Recognition) technology to convert the text in the image into digital text.

[1102] Step 4:

[1103] Server: The extracted text data is sent to a multilingual translation module for automatic translation into multiple languages. For example, Japanese text is translated into English, Chinese, and Spanish.

[1104] Step 5:

[1105] Server: Reinserts the translated text into the original webtoon image to generate webtoons in each language. Stores the generated content in a database.

[1106] Step 6:

[1107] Server: Runs the multi-scenario / multi-ending generation module, automatically generating multiple scenarios and endings based on the original content. Identifies branching points in the scenario and creates different story progressions for each branch.

[1108] Step 7:

[1109] Server: The generated multiple scenarios and endings are saved in a database, including metadata for each scenario and ending (e.g., scenario overview, branching points, etc.).

[1110] Step 8:

[1111] Device: The user's facial expressions and voice are sent to the emotion engine via the user's camera and microphone. This data is used to analyze the user's emotions in real time.

[1112] Step 9:

[1113] Server: Analyzes the user's emotions from facial expressions and voice data using an emotion engine. The analysis results are used to provide content based on the user's emotional state.

[1114] Step 10:

[1115] Terminal: Provides an interface for users to select their preferred language and scenario through a web interface. Users select their preferred settings.

[1116] Step 11:

[1117] Server: Based on the user's selection and analyzed emotions, retrieves the appropriate language version and scenario from the database. Sends the retrieved content to the device.

[1118] Step 12:

[1119] Device: Displays the received content and allows users to view the webtoon.

[1120] Step 13:

[1121] Server: Collects user browsing and emotional data, such as viewed scenarios, selected language, browsing time, and perceived emotional state. Stores the collected data in a database.

[1122] Step 14:

[1123] Server: Analyzes the collected data and derives insights to identify user preferences, interests, and emotional trends. Based on the analysis results, it proposes emotionally-based ad placements and paid content.

[1124] Step 15:

[1125] Server: Based on the analysis results, sends personalized ads and content recommendations to the device, so that the user sees the most relevant information.

[1126] Step 16:

[1127] Device: Displaying received advertisements and content recommendations to you, and collecting further data based on your responses to help optimize the system.

[1128] The system automates these processes, enabling webtoon content to be multilingual and multi-scenario compatible, and also provides a personalized content experience through user sentiment analysis, creating new value in entertainment.

[1129] Example 2

[1130] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1131] Conventional webtoon platforms struggled with multilingual support and multiple scenarios, and lacked the ability to provide personalized content based on users' individual emotions and preferences. Effective advertising placement and monetization were also difficult. This has led to a demand for improved user experience and efficient monetization for creators.

[1132] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for uploading content from a user, a means for temporarily storing the uploaded content, a means including a language analysis module and a translation module for extracting character data from the content and translating it into multiple languages, a means for storing the translated content in a database, a means including a generation AI engine for generating different scenarios and endings, a means for storing the generated multiple scenarios and endings in a database, a means including an emotion recognition engine for collecting emotion data from the user's camera and microphone and performing emotion analysis, a means for providing optimal content based on the user's selection and the emotion analysis results, a means for collecting user browsing data and emotion data and storing them in a database, a means for analyzing the data and suggesting advertisement placement and paid content based on the analysis results, and a means for improving the accuracy of the generation AI algorithm and the emotion recognition engine using the collected and analyzed data. This facilitates multilingual support and multi-scenario development of webtoon content, enabling personalized experiences based on individual emotions and preferences. In addition, it also achieves effective advertisement placement and monetization.

[1133] "User" means an individual or organization that uses this system to upload and view content.

[1134] "Content" refers to the image files and text data of webtoons that users upload to the system.

[1135] The "temporary storage means" is a function for temporarily storing uploaded content in server storage or cloud storage for use in subsequent processing.

[1136] "Language Analysis Module" is a software unit for extracting text data from uploaded content. Specifically, it has the function of extracting text from images using OCR technology.

[1137] A "translation module" is a software unit that automatically translates extracted text data into multiple languages, for example, using natural language processing techniques.

[1138] "Generative AI engine" is a general term for machine learning models and algorithms that automatically generate different scenarios and endings based on the original content.

[1139] An "emotion recognition engine" is a software unit that analyzes a user's facial expressions and voice data collected through a camera and microphone to determine the user's emotional state in real time.

[1140] The "database" is an information management system for systematically storing various data used within the system (uploaded content, translated text, generated scenarios and endings, user emotional data and browsing data, etc.).

[1141] "Data analysis" is the process of analyzing collected user browsing data and sentiment data using statistical methods and machine learning techniques to derive insights.

[1142] "Ad placement" is the process of displaying advertisements that match the user's interests and emotions at the optimal time based on the analysis results.

[1143] "Paid content suggestion" is the process of recommending paid content that matches a user's interests and preferences based on data analysis.

[1144] This invention is a system that combines a generative AI model and an emotion recognition engine to realize multilingual and multi-scenario webtoon content, providing a personalized experience based on the user's emotions and preferences, and enabling efficient ad placement and monetization.

[1145] System configuration

[1146] The system includes the following major components:

[1147] 1. User Device:

[1148] Hardware: Devices such as PCs, smartphones, and tablets

[1149] Software: Web browser, camera, microphone

[1150] 2. Server:

[1151] Hardware: Cloud server, database server

[1152] Software: OCR technology, translation module, generative AI engine, emotion recognition engine, data analysis module

[1153] Processing flow

[1154] The system operates as follows:

[1155] 1. User uploads content:

[1156] User: Upload webtoon content from their own device and enter meta information.

[1157] Terminal: Accepts uploads from users through a web interface.

[1158] Example: A user opens a browser, clicks the "Choose File" button on a website's upload screen, selects an image file and text data, and clicks the "Upload" button.

[1159] 2. The server receives and stores the content:

[1160] Server: Receives uploaded content and meta information and temporarily stores it in cloud storage. Validates the format and size of the stored data and notifies the user of any invalid data with an error message.

[1161] Example: The server checks that the image file received is in JPEG format and that the file size is within 10MB, and if there are no problems, it saves it to cloud storage.

[1162] 3. Extract and translate the text data of the content:

[1163] Server: Extracts text data from uploaded webtoon content using OCR technology and sends it to the translation module for automatic translation into multiple languages.

[1164] Example: Use Amazon Textract to perform OCR analysis, and then use the Google Cloud Translation API to translate the acquired Japanese text data into English, Chinese, and Spanish.

[1165] 4. Saving translated content:

[1166] Server: The translated text is reinserted into the original webtoon image, and webtoons in each language are generated and stored in the database.

[1167] Example: Using Adobe Photoshop API to remap translated text onto webtoon images for each language version, and then storing the generated images in cloud storage, and storing information including metadata in a database.

[1168] 5. Multi-Scenario and Multi-Ending Generation:

[1169] Server: Using a generative AI engine, different scenarios and endings are automatically generated based on the original content and stored in a database.

[1170] Example: Using OpenAI's GPT-4 to identify branching points in a story and generate different scenarios and endings.

[1171] 6. Introducing Emotion Recognition:

[1172] Terminal: Collects emotional data through the user's camera and microphone and sends it to the server.

[1173] Server: Performs real-time emotion analysis using an emotion recognition engine and stores the results in a database.

[1174] Example: Use your smartphone's camera and microphone to capture facial expressions and voice, then use the Emotion API to analyze emotions in real time.

[1175] 7. Content Distribution:

[1176] Terminal: Allows users to select their preferred language and scenario through a web interface.

[1177] Server: Based on the user's selection and the results of sentiment analysis, retrieves appropriate content from the database and sends it to the device.

[1178] Example: A user selects a language and scenario on a website, and that information is sent to a server. The server retrieves the selected language and scenario and sends them to the device.

[1179] 8. Data Collection and Analysis:

[1180] Server: Collects user browsing data and emotion data and stores them in a database.

[1181] Server: Uses the data analysis module to analyze the collected data and derive insights.

[1182] Example: Use Google Analytics to collect browsing data, store emotion recognition data in a database, and use Apache Spark for analysis.

[1183] 9. Ad Placement and Monetization:

[1184] Server: Proposes ad placement and paid content based on the analysis results.

[1185] Example: Displaying personalized ads to users whose emotions are identified as "joy" through emotion recognition.

[1186] 10. Feedback and optimization:

[1187] Server: Based on user interactions, collects additional data to improve the accuracy of generative AI algorithms and emotion recognition engines.

[1188] Example: Continuously collect new browsing and sentiment data to retrain machine learning models.

[1189] Prompt Sentence Examples

[1190] "Design a system to translate the following Japanese text into English, Chinese, and Spanish, and generate different scenarios and endings. The system should allow users to enjoy content that matches their mood of the day through emotion recognition. The system should be uploaded by the user using their smartphone, and emotions should be recognized using the camera and microphone."

[1191] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1192] Step 1:

[1193] User uploads content:

[1194] Input: Image files, text data, and meta information (title, author name, language, etc.) of webtoons selected by the user from their device.

[1195] Process: A user opens the website's upload screen in a browser, clicks the "Choose File" button, selects an image file and text data, and clicks the "Upload" button.

[1196] Output: The device receives the uploaded file and meta information.

[1197] Step 2:

[1198] The server receives and stores the content:

[1199] Input: Webtoon image files, text data, and meta information sent from the device.

[1200] Processing: The server receives the uploaded content and meta information, temporarily stores it in cloud storage, validates the format (JPEG, PNG, etc.) and file size of the received image file, and sends an error message to the user if there is invalid data.

[1201] Output: Saved content and meta information.

[1202] Step 3:

[1203] Extract and translate text data from content:

[1204] Input: Webtoon image files stored in cloud storage.

[1205] Processing: The server uses OCR technology (e.g., Amazon Textract) to extract the text data in the image, and then sends the text data to a multilingual translation module (e.g., Google Cloud Translation API) to automatically translate the Japanese text into English, Chinese, and Spanish.

[1206] Output: Translated text data in multiple languages.

[1207] Step 4:

[1208] Saving translated content:

[1209] Input: translated text data, original webtoon images.

[1210] Processing: The server reinserts the translated text into the original webtoon image to generate the webtoon in each language (e.g., using Adobe Photoshop API). The generated webtoon in each language is then stored in a database.

[1211] Output: Multilingual webtoon content stored in a database.

[1212] Step 5:

[1213] Multi-scenario and multi-ending generation:

[1214] Input: Original webtoon content.

[1215] Processing: The server uses a generative AI engine (e.g., OpenAI's GPT-4) to automatically generate different scenarios and endings based on the original content and store them in a database.

[1216] Output: Multiple scenarios and endings stored in a database.

[1217] Step 6:

[1218] Introducing emotion recognition:

[1219] Input: User facial and voice data collected through the device's camera and microphone.

[1220] Processing: The server receives facial expression and voice data sent from the user's device and analyzes it in real time using an emotion recognition engine (e.g., Emotion API).

[1221] Output: Parsed emotion data.

[1222] Step 7:

[1223] Content Delivery:

[1224] Input: User choices (language, scenario), parsed emotion data.

[1225] Processing: The user selects the desired language and scenario through a web interface, and that information is sent to the server. Based on the user's selection and the results of sentiment analysis, the server retrieves appropriate content from a database and sends it to the device.

[1226] Output: The appropriate content is delivered to the user's device.

[1227] Step 8:

[1228] Data collection and analysis:

[1229] Input: User browsing data, emotion data.

[1230] Processing: The server collects user browsing data and sentiment data and stores it in a database. A data analysis module (e.g., Apache Spark) is used to analyze the collected data and derive insights into user preferences and sentiment trends.

[1231] Output: Collected and analyzed data, analytical insights.

[1232] Step 9:

[1233] Ad Placement and Monetization:

[1234] Input: Analytics Insights.

[1235] Processing: The server displays personalized advertisements to the user based on the analysis results and suggests relevant paid content.

[1236] Output: Advertisements and paid content suggestions displayed to the user.

[1237] Step 10:

[1238] Feedback and optimization:

[1239] Input: Collected browsing data, sentiment data, and interaction data.

[1240] Processing: The server uses this data to retrain the generative AI algorithms and emotion recognition engines to improve the accuracy of the system.

[1241] Output: Optimized generative AI model and emotion recognition engine.

[1242] (Application example 2)

[1243] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1244] In addition to providing a webtoon platform that supports multiple languages ​​and allows for individual scenario selection, there is a need for a platform that can provide optimal content and advertisements in real time based on user emotions. However, current systems have difficulty generating flexible scenarios that respond to diverse user emotions and preferences, and providing personalized advertisements. Therefore, an effective system is needed that can simultaneously improve user experience and monetize creators and advertisers.

[1245] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for uploading image content from a user, means for temporarily storing the uploaded content, means for extracting text data from the content and translating it into multiple languages, means for storing the translated content in a database, means for identifying major branching points and generating multiple scenarios and endings, means for recognizing the user's emotions through a camera or microphone, means for providing optimal content and advertisements based on the recognized emotions, means for collecting and analyzing user viewing data and emotion data, and means for suggesting advertisement placement and paid content based on the above analysis. This makes it possible to provide a personalized experience based on the user's emotions and optimize content and advertisements.

[1246] "User" refers to an individual or corporation that uses the system to upload, view, select, and perform other operations on webtoons.

[1247] "Image content" refers to digital data that is primarily composed of visual elements, such as webtoons, comic pages, and illustrations.

[1248] "Means for uploading" refers to the functions and interfaces that allow users to send image content to the server.

[1249] "Temporary storage" means memory or storage for short-term storage of uploaded content.

[1250] "Means for extracting text data" refers to OCR (optical character recognition) technology or software for extracting text information from image content.

[1251] "Means for translating into multiple languages" refers to a translation engine or API for translating extracted text data into multiple languages.

[1252] "Means for storing in a database" refers to a data management system for long-term storage of translated content and generated scenarios.

[1253] "Branch point identification methods" refers to algorithms or software that automatically identify important choices and directions in the development of a story.

[1254] "Means for generating scenarios and endings" refers to generative AI technology that generates different story developments and endings based on branching points.

[1255] "Means of recognition through cameras and microphones" refers to hardware and software for capturing the user's facial expressions and voice and analyzing their emotions.

[1256] "Means for providing content based on recognized emotions" refers to a system for displaying optimal content and advertisements based on analyzed emotional data.

[1257] "Means for collecting and analyzing browsing data and emotional data" refers to a data processing system for recording and analyzing a user's content operation history and recognized emotional data.

[1258] "Means for suggesting advertising placements and paid content" refers to algorithms and systems that recommend optimal advertising and paid content to users based on collected data.

[1259] The present invention relates to a webtoon platform that supports multiple languages ​​and allows individual scenario selection, and is a system that provides optimal content and advertisements based on user emotions. Specific embodiments will be described below.

[1260] System Program Overview

[1261] This system implements a program that includes the following main functions:

[1262] 1. Upload and save webtoons

[1263] The server provides an interface for users to upload image content (e.g., webtoons), which is then temporarily stored.

[1264] 2. Extraction and translation of text data

[1265] The server uses an OCR (Optical Character Recognition) engine to extract text data from image content, which is then translated into multiple languages ​​using the Google Translate API.

[1266] 3. Identifying and generating scenario branching points

[1267] The server uses the extracted text data to identify key story branching points, and uses a generative AI model to automatically generate different scenarios and endings, which are then stored in a database.

[1268] 4. Emotion recognition and personalized content delivery

[1269] It uses the camera and microphone on the device (e.g., smartphone) to recognize the user's emotions in real time. It uses the EmotionRecognizer module to analyze the user's emotional state from their facial expressions and voice. Based on the analysis results, it provides optimal content and personalized advertisements.

[1270] 5. Data Collection and Analysis

[1271] The server collects user browsing data and emotional data and stores it in a database. This data is analyzed to gain insights into user preferences and behavioral patterns. This data is used to optimize ad placement and suggest paid content.

[1272] Hardware and software used

[1273] Hardware

[1274] Devices such as smartphones and tablets: Used to upload and view content from users.

[1275] Camera and microphone: Captures the user's facial expressions and voice for emotion recognition.

[1276] software

[1277] OCR engine: Used to extract text from images.

[1278] Google Translate API: Used to translate the extracted text into multiple languages.

[1279] EmotionRecognizer: A software module for recognizing user emotions.

[1280] Generative AI models: Used to automatically generate different scenarios and endings.

[1281] Specific examples

[1282] For example, if a user opens the app at night and emotion recognition detects the emotion of "joy," the server will prioritize displaying cheerful and upbeat stories and related ads, providing the best possible experience for the user.

[1283] Prompt Sentence Examples

[1284] An example of a specific prompt for the generative AI model is, "Please generate what kind of story development would be best for a user who is in a joyful mood." Based on this prompt, the generative AI model automatically generates multiple story developments that suit the user's emotional state.

[1285] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1286] Step 1:

[1287] Users upload image content (e.g., webtoons) from their devices. They also enter meta information such as the title, author's name, and language used. The server receives this input data and temporarily stores it. Specifically, data is sent via a web interface, and the server temporarily stores the uploaded image file and meta information.

[1288] Step 2:

[1289] The server extracts text data from the stored image content, using an OCR engine (e.g., Google Cloud Vision API) to convert the text in the image into digital text. This allows the character information embedded in the image to be obtained. The input is the temporarily stored image data, and the output is digital text.

[1290] Step 3:

[1291] The server translates the extracted text data into multiple languages. Specifically, it uses the Google Translate API to translate, for example, Japanese into English, Chinese, Spanish, etc. This process generates translated text (output) into each target language based on the original text (input).

[1292] Step 4:

[1293] The server reinserts the translated text into the original webtoon image to generate the webtoon in each language. The generated content for each language is stored in a database. The input is the translated text and the original image file, and the output is a multilingual webtoon image. In this reinsertion process, the text is embedded into the image using image editing software (e.g., Python's Pillow library).

[1294] Step 5:

[1295] The server identifies key branching points based on the original content. Based on the identified branching points, it uses a generative AI model to automatically generate different scenarios and endings. In this process, multiple scenarios and endings (outputs) are generated from the original content (input) and stored in a database. Text analysis algorithms and generative AI algorithms are used for identification and generation.

[1296] Step 6:

[1297] The device captures the user's facial expressions and voice through the user's camera and microphone and sends them to the server. The server then uses the EmotionRecognizer module to analyze the user's emotions in real time from the transmitted facial and voice data. The input is the captured multimedia data, and the output is the analyzed emotional information.

[1298] Step 7:

[1299] The server provides optimal content and personalized advertisements based on the analyzed emotional data. This involves retrieving webtoons with the appropriate language version and scenario from a database and sending them to the user's device. The input is the emotional data and user selection information, and the output is the appropriate content and advertisements.

[1300] Step 8:

[1301] The server collects users' browsing data and emotional data and stores it in a database. The collected data is analyzed to derive insights that identify users' preferences, interests, and emotional trends. The input is user interaction data, and the output is the analysis results. Data mining techniques and machine learning algorithms are used for the analysis.

[1302] Step 9:

[1303] The server then uses the analysis results to suggest ad placements and paid content, thereby displaying the most relevant ads and content to the user. The input is the analysis results, and the output is a list of recommended ads and paid content. A recommendation system algorithm is used in this process.

[1304] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1305] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1306] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1307] [Fourth embodiment]

[1308] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1309] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1310] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1311] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1312] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1313] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1314] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1315] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1316] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1317] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1318] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1319] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1320] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1321] This invention relates to a webtoon platform that can be deployed globally by automatically creating multiple languages ​​and scenarios using generative AI. The system allows users to upload webtoon content, translate it into multiple languages, and provide it in the form of multiple scenarios and endings. It also analyzes user viewing data to suggest effective advertising placements and paid content.

[1322] Program processing flow

[1323] 1. A user uploads a webtoon

[1324] Users: Upload webtoon content from their own devices, including image data and text data.

[1325] Terminal: Accepts uploads from users through a web interface, along with meta information such as title, author, and language.

[1326] 2. Receiving and temporarily storing content

[1327] Server: Receives uploaded content and meta information and stores it in temporary storage. The format and size of the stored data are verified, and invalid data is notified to the user with an error message.

[1328] 3. Extraction and translation of text data

[1329] Server: Uses a language analysis module to extract text data from the uploaded webtoon content, then invokes a translation engine to translate the extracted text into multiple languages.

[1330] Example: Translations are made from Japanese to English, Chinese, and Spanish.

[1331] 4. Saving translated content

[1332] Server: Stores the translated webtoon content in a database and updates the metadata for each language version.

[1333] 5. Multi-scenario and multi-ending generation

[1334] Server: Based on the original content, it identifies key branching points and uses a generative AI engine to automatically generate different scenarios and endings, allowing users to enjoy different story developments depending on their choices.

[1335] Example: Multiple scenarios are generated in which the story progresses in different directions depending on the character's choices.

[1336] 6. Content Delivery

[1337] Device: The user selects the desired language and scenario through a web interface. Based on the selection, the server delivers the most appropriate language and scenario content to the device.

[1338] 7. Data Collection and Analysis

[1339] Server: Collects users' browsing history and scenario selection data and records them in a database. Analyzes the collected data to analyze users' interests and browsing patterns.

[1340] Example: Identifying a user's preferences based on the scenarios and endings they frequently choose.

[1341] 8. Ad Placement and Monetization

[1342] Server: Uses collected data to effectively place appropriate advertisements and recommend paid content, providing users with a personalized experience.

[1343] Example: For users who frequently select a particular scenario, advertisements and paid content related to that scenario will be displayed preferentially.

[1344] By automating these processes, the system will facilitate the global expansion of webtoon content, enabling it to be translated into multiple languages ​​and scenarios. It also utilizes user data to suggest optimal advertising placements and paid content, ensuring that creators are properly evaluated and fostering a sustainable production environment.

[1345] The processing flow will be explained below.

[1346] Step 1:

[1347] User: Uploads webtoon content, including image files (e.g., JPEG, PNG) and text files (e.g., TXT, PDF), as well as meta information such as title, author name, and language.

[1348] Step 2:

[1349] Terminal: Receives content and meta information uploaded through the web interface. After receiving, it performs an initial validation to ensure the file format and size are correct. If there are no issues, it stores the data in temporary storage.

[1350] Step 3:

[1351] Server: Analyzes the uploaded content and extracts text data from image files. Converts text in images into digital text using OCR (Optical Character Recognition) technology.

[1352] Step 4:

[1353] Server: The extracted text data is sent to the multilingual translation module, which uses a generative AI engine to automatically translate the text into multiple specified languages. For example, Japanese text is translated into English, Chinese, and Spanish.

[1354] Step 5:

[1355] Server: Reinserts the translated text into the original webtoon image to generate webtoons in each language. Stores the generated content in a database.

[1356] Step 6:

[1357] Server: Runs the multi-scenario / multi-ending generation module to generate multiple scenarios and endings based on the original webtoon content, identifying branching points in the scenario and creating different progressions for the story.

[1358] Step 7:

[1359] Server: Stores the generated multiple scenarios and endings in a database, including metadata for each scenario and ending (e.g., scenario overview, branching points, etc.).

[1360] Step 8:

[1361] Terminal: Provides an interface for users to select language and scenario through a web interface. Users select their preferred settings.

[1362] Step 9:

[1363] Server: Retrieves the appropriate language version and scenario from the database based on the user's selection. Sends the retrieved content to the device.

[1364] Step 10:

[1365] Device: Displays the received content and allows users to view the webtoon.

[1366] Step 11:

[1367] Server: Collects user browsing data, including the scenarios viewed, language selection, browsing time, etc. Stores the collected data in a database.

[1368] Step 12:

[1369] Server: Analyzes the collected data and derives insights to identify user preferences and interests. Based on the results of this analysis, it proposes optimal ad placements and paid content.

[1370] Step 13:

[1371] Server: Based on the analysis results, it sends personalized ads and content recommendations to the device, so that the information most relevant to the user is displayed.

[1372] Step 14:

[1373] On your device: Displaying received advertisements and content recommendations to you. Depending on your interactions, collecting further data to help optimize the platform.

[1374] These are the specific processing steps of the global webtoon platform that uses generative AI to automatically translate webtoons into multiple languages ​​and create multiple scenarios. This system enables webtoon content to be translated into multiple languages ​​and offers multiple scenarios, providing users with a new entertainment experience.

[1375] Example 1

[1376] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1377] Currently, the global expansion of digital comics requires multilingual support and the provision of a variety of scenarios and endings. However, doing this manually is extremely time-consuming and costly. Other issues include the lack of automated systems for providing personalized content tailored to users' tastes and preferences, effective advertising placement, and multilingual support. Unless these issues are resolved, they will become serious obstacles to the global expansion of digital comics.

[1378] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1379] In this invention, the server includes means for uploading digital comic content from users, means for temporarily storing the uploaded content, means for extracting character data from the content and translating it into multiple languages, means for storing the translated content in a database, means for using a generation AI engine to generate multiple scenarios and endings, means for storing the generated scenarios and endings in a database, means for providing content in an appropriate language version and scenario based on user selection, means for collecting and analyzing user viewing data, and means for suggesting advertising placement and paid content based on the analysis. This makes it possible to make digital comics multilingual and multi-scenarios and provide optimal content based on user data.

[1380] "User" refers to a person who uses digital comic content.

[1381] "Digital comic content" refers to comic or manga data provided in electronic format.

[1382] "Upload" refers to the act of transferring data from a user's device to a server.

[1383] "Means" refers to a method or device for achieving a particular purpose.

[1384] "Temporary storage" refers to the process of temporarily storing data.

[1385] "Text data" refers to the text and textual information within the Content.

[1386] "Translating into multiple languages" refers to the process of converting from one language to another.

[1387] "Database" refers to a system for efficiently storing and managing data.

[1388] A "generative AI engine" refers to a program that uses artificial intelligence to process data and generate new content.

[1389] "Scenario" refers to the flow of development of a story or content.

[1390] "Ending" refers to the conclusion or final part of a story or content.

[1391] "Providing" refers to the act of supplying specific services or content to users.

[1392] "Viewing data" refers to recorded information when a user views content.

[1393] "Analysis" refers to the process of examining data in detail and deriving trends and meaning.

[1394] "Ad placement" refers to the act of appropriately placing an advertisement on a user's screen.

[1395] "Paid Content" means additional content that a User is required to purchase or pay for.

[1396] This invention relates to a digital comic platform that can be deployed globally by automatically generating multiple languages ​​and scenarios using generative AI. The main components of this system are a server, terminals, and users. A detailed explanation is provided below.

[1397] Webtoon uploads by users

[1398] Users upload digital comic content from their own devices. This upload includes image data and text data. Users also enter meta information such as the title, author name, and language. For example, a user might enter information such as "Title: Magical Forest," "Author: Yamada Taro," and "Language: Japanese."

[1399] Receiving and temporarily storing content

[1400] The server stores the content and meta information received from the device in temporary storage. The format and size of the stored data are verified, and if there is any invalid data, an error message is displayed to the user. For example, if an image file is corrupted, the user is notified that the image file is invalid.

[1401] Extraction and translation of text data

[1402] The server uses a language analysis module to extract text data from uploaded digital comic content. The extracted text is then translated into multiple languages ​​using a translation engine. For example, Japanese can be translated into English, Chinese, and Spanish. The translation engine used could be software such as Google Translate API or DeepL.

[1403] Storing translated content

[1404] The server stores the translated digital comic content in a database and updates the metadata for each language version. For example, the English and Spanish versions are stored, and metadata such as "Title: The Magical Forest," "Artist: Yamada Taro," and "Language: English" is updated.

[1405] Multi-scenario and multi-ending generation

[1406] The server identifies key branching points based on the original content and automatically generates different scenarios and endings using a generative AI engine (e.g., GPT-3.5Turbo). This allows users to enjoy different story developments. For example, three different scenarios can be generated depending on the character's behavior choices.

[1407] Content Delivery

[1408] The device allows the user to select the desired language and scenario through a web interface. Based on the selection, the server delivers the most appropriate language and scenario content to the device. For example, if the user selects the English version of Scenario 2, that content will be displayed on the device.

[1409] Data collection and analysis

[1410] The server collects the user's browsing history and scenario selection data and records them in a database. The collected data is analyzed to determine the user's interests and browsing patterns. For example, if a user frequently selects a particular scenario, advertisements and paid content related to that scenario are displayed preferentially.

[1411] Ad Placement and Monetization

[1412] The server effectively places appropriate advertisements based on the collected data. It also recommends paid content and provides a personalized experience for users. For example, a user who likes fantasy scenarios will be shown advertisements for fantasy goods and related apps.

[1413] Prompt Sentence Examples

[1414] "Identify the main branching points in the webtoon and generate prompts to generate different scenarios. This prompt should be in the following format:

[1415] Input: A scene where Character A meets Character B

[1416] Output: Think of three different scenarios in which Character A could act after meeting Character B, and describe each scenario in 200 characters or less.

[1417] The above is a specific embodiment for carrying out the present invention. This system automatically supports multiple languages ​​and multiple scenarios for digital comics, and can provide optimal content based on user data.

[1418] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1419] Step 1:

[1420] Users upload digital comic content from their own devices. They provide image data, text data, and meta information (title, author name, language, etc.) as input. Specifically, they use a web interface to enter information such as "Title: Magical Forest," "Author: Yamada Taro," and "Language: Japanese," and then press the upload button.

[1421] Step 2:

[1422] The server saves the content and meta information received from the device in temporary storage. As input, it receives the digital comic file and meta information uploaded by the user. Specifically, the server verifies the format and size of the data, and if there is an abnormality, it generates an error message and notifies the user. As output, only valid data is temporarily stored, and invalid data is returned as an error message such as "The image file is invalid."

[1423] Step 3:

[1424] The server uses a language analysis module to extract text data from uploaded digital comic content. It receives webtoon image and text data as input. Specifically, it converts the text data in the image into text format using OCR (Optical Character Recognition) technology. The extracted text data is then generated as output.

[1425] Step 4:

[1426] The server translates the extracted text data into multiple languages. It receives Japanese character data as input. Specifically, it calls translation engines such as Google Translate API and DeepL to translate the Japanese text into English, Chinese, and Spanish. The translated text data is generated as output.

[1427] Step 5:

[1428] The server stores the translated digital comic content in a database. As input, it receives the translated text data and the original image data. Specific operations include saving the text and images in the corresponding fields of the database and updating the metadata (e.g., "Title: The Magical Forest," "Author: Yamada Taro," "Language: English"). As output, the database update is complete.

[1429] Step 6:

[1430] The server uses a generative AI engine to automatically generate multiple scenarios and endings based on the original content. It receives the original digital comic content and its main branching points as input. Specifically, it uses a generative AI model such as GPT-3.5 Turbo to create prompts for generating different scenarios and endings. As output, multiple scenarios and endings are generated and stored in a database.

[1431] Step 7:

[1432] The terminal allows the user to select the desired language and scenario through a web interface. As input, it receives the user's selected language and scenario information. In concrete terms, after the user selects the language and scenario, it sends the selection to the server. As output, the selection is transmitted to the server.

[1433] Step 8:

[1434] The server delivers the content of the selected language and scenario to the terminal. As input, it receives the language and scenario information selected by the user. Specifically, it retrieves the corresponding content from the database and sends it to the terminal. As output, the digital comic of the language and scenario selected by the user is displayed on the terminal.

[1435] Step 9:

[1436] The server collects users' browsing history and scenario selection data and records them in a database. As input, it receives the users' browsing and selection data. As a specific operation, it stores the collected data in the database in an appropriate format. As output, the recorded data is used for analysis.

[1437] Step 10:

[1438] The server analyzes the collected data and proposes ad placements and paid content. It receives user browsing data and selection data as input. Specifically, it uses data analysis algorithms to identify the user's interests and preferences and selects relevant ads and content. Personalized ads and paid content are proposed and displayed as output.

[1439] (Application example 1)

[1440] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1441] Webtoons are increasingly being enjoyed by a growing number of users, but language barriers and the limited variety of scenarios make their global adoption difficult. It is also challenging to effectively utilize user browsing data to suggest advertising placements and paid content. Furthermore, making them compatible with various devices is also a key issue.

[1442] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1443] In this invention, the server includes a means for uploading webtoon content from users, a means for temporarily storing the uploaded content, and a means for extracting text data from the content and translating it into multiple languages, thereby enabling webtoon content to be made available in multiple languages.

[1444] The system further includes a means for storing translated content in a database, a means for generating multiple scenarios and endings and storing them in a database, and a means for providing appropriate content based on a user's selection, thereby allowing users to enjoy webtoons in different languages ​​and scenarios at their leisure.

[1445] It also includes a means for collecting and analyzing user browsing data and a means for suggesting advertisement placement and paid content based on the analysis, which makes it possible to suggest advertisements and content personalized for each user.

[1446] Furthermore, it also includes a means for providing the system as an application installed on a smartphone, smart glasses, a head-mounted display, or a robot, which enables use on a variety of devices and improves user convenience.

[1447] "User" means an individual end user who uses the Webtoon Content.

[1448] "Webtoon content" refers to web manga or digital comics that are presented in digital format.

[1449] "Means for uploading" refers to a mechanism for transferring digital content owned by a user to a designated server.

[1450] A "temporary storage means" is a storage device or system for temporarily holding uploaded digital content.

[1451] "Means for extracting text data" refers to technology for identifying and obtaining text information from digital images and videos.

[1452] The "means for translating into multiple languages" is a translation engine or software for converting extracted text data into other languages.

[1453] "Means for storing data in a database" refers to a data management system for organizing and systematically storing digital data such as translated content and generated scenarios.

[1454] "Means for generating multiple scenarios and endings" refers to generative AI technology that automatically creates different developments and endings based on the original content.

[1455] "Means for providing appropriate content based on user selection" refers to a system that delivers optimal digital content according to the user's settings and preferences.

[1456] "Means for collecting and analyzing browsing data" refers to technology that records a user's operation history and browsing patterns and analyzes that data.

[1457] The "means for suggesting advertisement placement and paid content" is a system that presents advertisements and paid services that are most relevant to the user based on the analysis results.

[1458] An "application installed on a smartphone" is software that runs on a smartphone and provides various functions.

[1459] "Applications installed on smart glasses" are software that run on smart glasses and provide various functions.

[1460] An "application installed on a head-mounted display" is software that runs on the head-mounted display and provides various functions.

[1461] An "application installed on a robot" is software that runs on a robot and provides various functions.

[1462] This invention relates to a webtoon platform that can automatically translate multiple languages ​​and generate multiple scenarios using AI. The system of the present invention allows users to upload webtoon content, translate it into multiple languages, and provide it in the form of multiple scenarios and endings. It can also analyze users' viewing data to suggest effective advertising placements and paid content.

[1463] The system consists of the following elements: First, users upload webtoon content from their own devices. At this time, they enter meta information such as the title, author's name, and language. The uploaded content is temporarily stored on the server, where its format and size are verified.

[1464] The server then uses a language analysis module to extract text data from the uploaded webtoon content, and calls a translation engine to translate the extracted text into multiple languages, possibly using Google Cloud Translation API or DeepL API.

[1465] The translated webtoon content for each language version is stored in a database, and the metadata for each language version is also updated. A generative AI engine is then used to automatically generate different storylines and endings based on the original content. The generative AI engine uses OpenAI's GPT-4 or a similar natural language processing model.

[1466] Furthermore, user browsing data and scenario selection data are collected by the server and recorded in a database. This data is analyzed and used to analyze user interests and browsing patterns. Data analysis libraries such as Python's pandas and Scikit-learn are used for the analysis.

[1467] Ad placement and paid content suggestions are based on the collected data. An algorithm is implemented to effectively place appropriate ads, making it possible to provide users with a personalized experience.

[1468] Furthermore, this system is provided as an application that can be installed on smartphones, smart glasses, head-mounted displays, or robots, making it possible to use it on a variety of devices and improving user convenience.

[1469] For example, you can upload a Japanese webtoon and translate it into English, Chinese, and Spanish. You can also provide different scenarios depending on the character you choose. Below are some example prompts:

[1470] "If Character A gets lost, generate a scenario in which he goes to a different city. Output multiple endings along with background descriptions of those cities."

[1471] In this way, users can enjoy webtoons in different languages ​​and scenarios of their choice.

[1472] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1473] Step 1:

[1474] The user uploads webtoon content from their device to the server. At this time, the user enters meta information such as the title, author name, and language. The input data includes image files and text data. The server checks the received files and meta information to ensure that there is no invalid data. If the data is valid, it is temporarily stored.

[1475] Step 2:

[1476] The server extracts character data from the uploaded content. To obtain character data from image data, it uses OCR (Optical Character Recognition) technology. Specifically, it uses the Tesseract OCR library to recognize characters in the image and extract them as text data. This text data is then used for the next process.

[1477] Step 3:

[1478] The server translates the extracted text data into multiple languages. To do this, it uses the Google Cloud Translation API or the DeepL API. The extracted text data is sent to the API and translated into multiple target languages ​​(e.g., English, Chinese, Spanish). The translation results are generated in each language.

[1479] Step 4:

[1480] The server stores the translated webtoon content in a database and updates the metadata for each language. The database stores mapping information between the original language and the translated language, as well as text data for each language. This allows for the necessary data to be organized for subsequent scenario generation and distribution.

[1481] Step 5:

[1482] The server automatically generates multiple scenarios and endings based on the uploaded content using a generative AI engine. For example, OpenAI's GPT-4 is used to generate scenarios and endings based on prompts. The generated scenarios and endings are stored in a database.

[1483] Step 6:

[1484] The user accesses the web interface using a terminal and selects the desired language and scenario. The server delivers content in the optimal language and scenario to the terminal based on the user's selection, providing content files corresponding to the selected language and scenario as transmission data.

[1485] Step 7:

[1486] The server collects user browsing data and scenario selection data and records them in a database. The collected data is used for subsequent processing to analyze user interests and browsing patterns. Data analysis libraries such as Python's pandas and Scikit-learn are used to analyze trends in user behavior.

[1487] Step 8:

[1488] Based on the analysis results, the server effectively places appropriate advertisements and suggests paid content. Algorithms are implemented to provide personalized advertisements and content based on each user's preferences and browsing patterns. This increases creators' revenue and improves the user experience.

[1489] In this way, a system that supports multiple languages ​​and generates multiple scenarios can facilitate the global expansion of webtoon content while providing personalized services to users. Below is an example of a prompt sentence as a concrete example.

[1490] "If Character A gets lost, generate a scenario in which he goes to a different city. Output multiple endings along with background descriptions of those cities."

[1491] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1492] This invention relates to a webtoon platform that can be deployed globally by automatically creating multiple languages ​​and scenarios by combining generative AI and an emotion recognition engine. This system allows users to upload webtoon content, translate it into multiple languages, and provide it in the form of multiple scenarios and endings. It also provides content and advertising suggestions based on users' emotions through emotion recognition, and analyzes users' viewing data to evaluate creators and realize efficient monetization.

[1493] Program processing flow

[1494] 1. A user uploads a webtoon

[1495] User: Uploads webtoon content from their device, including image files and text data, and also enters meta information such as title, author name, and language.

[1496] Terminal: Accepts uploads from users via a web interface, including any meta information entered.

[1497] 2. Receiving and storing content

[1498] Server: Receives uploaded content and meta information and stores it in temporary storage. The format and size of the stored data are verified, and invalid data is notified to the user with an error message.

[1499] 3. Extraction and translation of text data

[1500] Server: Uses language analysis module to extract text data from uploaded webtoon content, and converts text in images into digital text using OCR technology.

[1501] Server: The extracted text data is sent to the multilingual translation module for automatic translation into multiple specified languages. For example, Japanese text is translated into English, Chinese, and Spanish.

[1502] 4. Saving translated content

[1503] Server: Reinserts the translated text into the original webtoon image to generate webtoons in each language. Stores the generated content in a database.

[1504] 5. Multi-scenario and multi-ending generation

[1505] Server: Based on the original content, identify key branching points and use a generative AI engine to automatically generate different scenarios and endings. Identify branching points in the scenario and create different progressions of the story.

[1506] Server: Stores the generated multiple scenarios and endings in a database, including metadata for each scenario and ending (e.g., scenario overview, branching points, etc.).

[1507] 6. Introducing Emotion Recognition

[1508] Device: The user's facial expressions and voice are sent to the emotion engine via the user's camera and microphone.

[1509] Server: Uses an emotion engine to analyze the user's emotions in real time from facial expressions and voice data.

[1510] 7. Content Delivery

[1511] Terminal: Provides an interface for users to select their preferred language and scenario through a web interface. Users can select their preferred settings.

[1512] Server: Retrieves the appropriate language version and scenario from the database based on the user's selection and the recognized emotion. Sends the retrieved content to the device.

[1513] 8. Data Collection and Analysis

[1514] Server: Collects user browsing and emotional data, including viewed scenarios, selected language, browsing time, perceived emotional state, etc. Stores the collected data in a database.

[1515] Server: Analyzes the collected data and derives insights to identify user preferences, interests, and emotional trends. Based on the results of this analysis, it proposes emotionally-based ad placements and paid content.

[1516] 9. Ad Placement and Monetization

[1517] Server: Based on the analysis results, it sends personalized ads and content recommendations to the device, so that the information most relevant to the user is displayed.

[1518] Example: If the user is recognized as "happy" by the emotion engine, relevant ads and content with upbeat content will be displayed preferentially.

[1519] 10. Feedback and Optimization

[1520] Server: Based on user interactions, further data is collected and used to improve the accuracy of the generative AI algorithms and emotion engine, thereby continuously optimizing the entire system.

[1521] By automating these processes, the system enables multilingual and multi-scenario webtoon content, and provides a personalized experience based on the user's emotions. The introduction of emotion recognition provides a more immersive experience, creating a new entertainment experience.

[1522] The processing flow will be explained below.

[1523] Step 1:

[1524] Users: Upload webtoon content from their devices, including image files (e.g., JPEG, PNG) and text files (e.g., TXT, PDF), along with meta information such as title, author name, and language.

[1525] Step 2:

[1526] Terminal: Receives content uploaded via the web interface and any meta information entered. Performs initial validation for proper format and size. If successful, stores the content in temporary storage.

[1527] Step 3:

[1528] Server: Analyzes the uploaded file and extracts text data from the image file, using OCR (Optical Character Recognition) technology to convert the text in the image into digital text.

[1529] Step 4:

[1530] Server: The extracted text data is sent to a multilingual translation module for automatic translation into multiple languages. For example, Japanese text is translated into English, Chinese, and Spanish.

[1531] Step 5:

[1532] Server: Reinserts the translated text into the original webtoon image to generate webtoons in each language. Stores the generated content in a database.

[1533] Step 6:

[1534] Server: Runs the multi-scenario / multi-ending generation module, automatically generating multiple scenarios and endings based on the original content. Identifies branching points in the scenario and creates different story progressions for each branch.

[1535] Step 7:

[1536] Server: The generated multiple scenarios and endings are saved in a database, including metadata for each scenario and ending (e.g., scenario overview, branching points, etc.).

[1537] Step 8:

[1538] Device: The user's facial expressions and voice are sent to the emotion engine via the user's camera and microphone. This data is used to analyze the user's emotions in real time.

[1539] Step 9:

[1540] Server: Analyzes the user's emotions from facial expressions and voice data using an emotion engine. The analysis results are used to provide content based on the user's emotional state.

[1541] Step 10:

[1542] Terminal: Provides an interface for users to select their preferred language and scenario through a web interface. Users select their preferred settings.

[1543] Step 11:

[1544] Server: Based on the user's selection and analyzed emotions, retrieves the appropriate language version and scenario from the database. Sends the retrieved content to the device.

[1545] Step 12:

[1546] Device: Displays the received content and allows users to view the webtoon.

[1547] Step 13:

[1548] Server: Collects user browsing and emotional data, such as viewed scenarios, selected language, browsing time, and perceived emotional state. Stores the collected data in a database.

[1549] Step 14:

[1550] Server: Analyzes the collected data and derives insights to identify user preferences, interests, and emotional trends. Based on the analysis results, it proposes emotionally-based ad placements and paid content.

[1551] Step 15:

[1552] Server: Based on the analysis results, sends personalized ads and content recommendations to the device, so that the user sees the most relevant information.

[1553] Step 16:

[1554] Device: Displaying received advertisements and content recommendations to you, and collecting further data based on your responses to help optimize the system.

[1555] The system automates these processes, enabling webtoon content to be multilingual and multi-scenario compatible, and also provides a personalized content experience through user sentiment analysis, creating new value in entertainment.

[1556] Example 2

[1557] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1558] Conventional webtoon platforms struggled with multilingual support and multiple scenarios, and lacked the ability to provide personalized content based on users' individual emotions and preferences. Effective advertising placement and monetization were also difficult. This has led to a demand for improved user experience and efficient monetization for creators.

[1559] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for uploading content from a user, a means for temporarily storing the uploaded content, a means including a language analysis module and a translation module for extracting character data from the content and translating it into multiple languages, a means for storing the translated content in a database, a means including a generation AI engine for generating different scenarios and endings, a means for storing the generated multiple scenarios and endings in a database, a means including an emotion recognition engine for collecting emotion data from the user's camera and microphone and performing emotion analysis, a means for providing optimal content based on the user's selection and the emotion analysis results, a means for collecting user browsing data and emotion data and storing them in a database, a means for analyzing the data and suggesting advertisement placement and paid content based on the analysis results, and a means for improving the accuracy of the generation AI algorithm and the emotion recognition engine using the collected and analyzed data. This facilitates multilingual support and multi-scenario development of webtoon content, enabling personalized experiences based on individual emotions and preferences. In addition, it also achieves effective advertisement placement and monetization.

[1560] "User" means an individual or organization that uses this system to upload and view content.

[1561] "Content" refers to the image files and text data of webtoons that users upload to the system.

[1562] The "temporary storage means" is a function for temporarily storing uploaded content in server storage or cloud storage for use in subsequent processing.

[1563] "Language Analysis Module" is a software unit for extracting text data from uploaded content. Specifically, it has the function of extracting text from images using OCR technology.

[1564] A "translation module" is a software unit that automatically translates extracted text data into multiple languages, for example, using natural language processing techniques.

[1565] "Generative AI engine" is a general term for machine learning models and algorithms that automatically generate different scenarios and endings based on the original content.

[1566] An "emotion recognition engine" is a software unit that analyzes a user's facial expressions and voice data collected through a camera and microphone to determine the user's emotional state in real time.

[1567] The "database" is an information management system for systematically storing various data used within the system (uploaded content, translated text, generated scenarios and endings, user emotional data and browsing data, etc.).

[1568] "Data analysis" is the process of analyzing collected user browsing data and sentiment data using statistical methods and machine learning techniques to derive insights.

[1569] "Ad placement" is the process of displaying advertisements that match the user's interests and emotions at the optimal time based on the analysis results.

[1570] "Paid content suggestion" is the process of recommending paid content that matches a user's interests and preferences based on data analysis.

[1571] This invention is a system that combines a generative AI model and an emotion recognition engine to realize multilingual and multi-scenario webtoon content, providing a personalized experience based on the user's emotions and preferences, and enabling efficient ad placement and monetization.

[1572] System configuration

[1573] The system includes the following major components:

[1574] 1. User Device:

[1575] Hardware: Devices such as PCs, smartphones, and tablets

[1576] Software: Web browser, camera, microphone

[1577] 2. Server:

[1578] Hardware: Cloud server, database server

[1579] Software: OCR technology, translation module, generative AI engine, emotion recognition engine, data analysis module

[1580] Processing flow

[1581] The system operates as follows:

[1582] 1. User uploads content:

[1583] User: Upload webtoon content from their own device and enter meta information.

[1584] Terminal: Accepts uploads from users through a web interface.

[1585] Example: A user opens a browser, clicks the "Choose File" button on a website's upload screen, selects an image file and text data, and clicks the "Upload" button.

[1586] 2. The server receives and stores the content:

[1587] Server: Receives uploaded content and meta information and temporarily stores it in cloud storage. Validates the format and size of the stored data and notifies the user of any invalid data with an error message.

[1588] Example: The server checks that the image file received is in JPEG format and that the file size is within 10MB, and if there are no problems, it saves it to cloud storage.

[1589] 3. Extract and translate the text data of the content:

[1590] Server: Extracts text data from uploaded webtoon content using OCR technology and sends it to the translation module for automatic translation into multiple languages.

[1591] Example: Use Amazon Textract to perform OCR analysis, and then use the Google Cloud Translation API to translate the acquired Japanese text data into English, Chinese, and Spanish.

[1592] 4. Saving translated content:

[1593] Server: The translated text is reinserted into the original webtoon image, and webtoons in each language are generated and stored in the database.

[1594] Example: Using Adobe Photoshop API to remap translated text onto webtoon images for each language version, and then storing the generated images in cloud storage, and storing information including metadata in a database.

[1595] 5. Multi-Scenario and Multi-Ending Generation:

[1596] Server: Using a generative AI engine, different scenarios and endings are automatically generated based on the original content and stored in a database.

[1597] Example: Using OpenAI's GPT-4 to identify branching points in a story and generate different scenarios and endings.

[1598] 6. Introducing Emotion Recognition:

[1599] Terminal: Collects emotional data through the user's camera and microphone and sends it to the server.

[1600] Server: Performs real-time emotion analysis using an emotion recognition engine and stores the results in a database.

[1601] Example: Use your smartphone's camera and microphone to capture facial expressions and voice, then use the Emotion API to analyze emotions in real time.

[1602] 7. Content Distribution:

[1603] Terminal: Allows users to select their preferred language and scenario through a web interface.

[1604] Server: Based on the user's selection and the results of sentiment analysis, retrieves appropriate content from the database and sends it to the device.

[1605] Example: A user selects a language and scenario on a website, and that information is sent to a server. The server retrieves the selected language and scenario and sends them to the device.

[1606] 8. Data Collection and Analysis:

[1607] Server: Collects user browsing data and emotion data and stores them in a database.

[1608] Server: Uses the data analysis module to analyze the collected data and derive insights.

[1609] Example: Use Google Analytics to collect browsing data, store emotion recognition data in a database, and use Apache Spark for analysis.

[1610] 9. Ad Placement and Monetization:

[1611] Server: Proposes ad placement and paid content based on the analysis results.

[1612] Example: Displaying personalized ads to users whose emotions are identified as "joy" through emotion recognition.

[1613] 10. Feedback and optimization:

[1614] Server: Based on user interactions, collects additional data to improve the accuracy of generative AI algorithms and emotion recognition engines.

[1615] Example: Continuously collect new browsing and sentiment data to retrain machine learning models.

[1616] Prompt Sentence Examples

[1617] "Design a system to translate the following Japanese text into English, Chinese, and Spanish, and generate different scenarios and endings. The system should allow users to enjoy content that matches their mood of the day through emotion recognition. The system should be uploaded by the user using their smartphone, and emotions should be recognized using the camera and microphone."

[1618] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1619] Step 1:

[1620] User uploads content:

[1621] Input: Image files, text data, and meta information (title, author name, language, etc.) of webtoons selected by the user from their device.

[1622] Process: A user opens the website's upload screen in a browser, clicks the "Choose File" button, selects an image file and text data, and clicks the "Upload" button.

[1623] Output: The device receives the uploaded file and meta information.

[1624] Step 2:

[1625] The server receives and stores the content:

[1626] Input: Webtoon image files, text data, and meta information sent from the device.

[1627] Processing: The server receives the uploaded content and meta information, temporarily stores it in cloud storage, validates the format (JPEG, PNG, etc.) and file size of the received image file, and sends an error message to the user if there is invalid data.

[1628] Output: Saved content and meta information.

[1629] Step 3:

[1630] Extract and translate text data from content:

[1631] Input: Webtoon image files stored in cloud storage.

[1632] Processing: The server uses OCR technology (e.g., Amazon Textract) to extract the text data in the image, and then sends the text data to a multilingual translation module (e.g., Google Cloud Translation API) to automatically translate the Japanese text into English, Chinese, and Spanish.

[1633] Output: Translated text data in multiple languages.

[1634] Step 4:

[1635] Saving translated content:

[1636] Input: translated text data, original webtoon images.

[1637] Processing: The server reinserts the translated text into the original webtoon image to generate the webtoon in each language (e.g., using Adobe Photoshop API). The generated webtoon in each language is then stored in a database.

[1638] Output: Multilingual webtoon content stored in a database.

[1639] Step 5:

[1640] Multi-scenario and multi-ending generation:

[1641] Input: Original webtoon content.

[1642] Processing: The server uses a generative AI engine (e.g., OpenAI's GPT-4) to automatically generate different scenarios and endings based on the original content and store them in a database.

[1643] Output: Multiple scenarios and endings stored in a database.

[1644] Step 6:

[1645] Introducing emotion recognition:

[1646] Input: User facial and voice data collected through the device's camera and microphone.

[1647] Processing: The server receives facial expression and voice data sent from the user's device and analyzes it in real time using an emotion recognition engine (e.g., Emotion API).

[1648] Output: Parsed emotion data.

[1649] Step 7:

[1650] Content Delivery:

[1651] Input: User choices (language, scenario), parsed emotion data.

[1652] Processing: The user selects the desired language and scenario through a web interface, and that information is sent to the server. Based on the user's selection and the results of sentiment analysis, the server retrieves appropriate content from a database and sends it to the device.

[1653] Output: The appropriate content is delivered to the user's device.

[1654] Step 8:

[1655] Data collection and analysis:

[1656] Input: User browsing data, emotion data.

[1657] Processing: The server collects user browsing data and sentiment data and stores it in a database. A data analysis module (e.g., Apache Spark) is used to analyze the collected data and derive insights into user preferences and sentiment trends.

[1658] Output: Collected and analyzed data, analytical insights.

[1659] Step 9:

[1660] Ad Placement and Monetization:

[1661] Input: Analytics Insights.

[1662] Processing: The server displays personalized advertisements to the user based on the analysis results and suggests relevant paid content.

[1663] Output: Advertisements and paid content suggestions displayed to the user.

[1664] Step 10:

[1665] Feedback and optimization:

[1666] Input: Collected browsing data, sentiment data, and interaction data.

[1667] Processing: The server uses this data to retrain the generative AI algorithms and emotion recognition engines to improve the accuracy of the system.

[1668] Output: Optimized generative AI model and emotion recognition engine.

[1669] (Application example 2)

[1670] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1671] In addition to providing a webtoon platform that supports multiple languages ​​and allows for individual scenario selection, there is a need for a platform that can provide optimal content and advertisements in real time based on user emotions. However, current systems have difficulty generating flexible scenarios that respond to diverse user emotions and preferences, and providing personalized advertisements. Therefore, an effective system is needed that can simultaneously improve user experience and monetize creators and advertisers.

[1672] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for uploading image content from a user, means for temporarily storing the uploaded content, means for extracting text data from the content and translating it into multiple languages, means for storing the translated content in a database, means for identifying major branching points and generating multiple scenarios and endings, means for recognizing the user's emotions through a camera or microphone, means for providing optimal content and advertisements based on the recognized emotions, means for collecting and analyzing user viewing data and emotion data, and means for suggesting advertisement placement and paid content based on the above analysis. This makes it possible to provide a personalized experience based on the user's emotions and optimize content and advertisements.

[1673] "User" refers to an individual or corporation that uses the system to upload, view, select, and perform other operations on webtoons.

[1674] "Image content" refers to digital data that is primarily composed of visual elements, such as webtoons, comic pages, and illustrations.

[1675] "Means for uploading" refers to the functions and interfaces that allow users to send image content to the server.

[1676] "Temporary storage" means memory or storage for short-term storage of uploaded content.

[1677] "Means for extracting text data" refers to OCR (optical character recognition) technology or software for extracting text information from image content.

[1678] "Means for translating into multiple languages" refers to a translation engine or API for translating extracted text data into multiple languages.

[1679] "Means for storing in a database" refers to a data management system for long-term storage of translated content and generated scenarios.

[1680] "Branch point identification methods" refers to algorithms or software that automatically identify important choices and directions in the development of a story.

[1681] "Means for generating scenarios and endings" refers to generative AI technology that generates different story developments and endings based on branching points.

[1682] "Means of recognition through cameras and microphones" refers to hardware and software for capturing the user's facial expressions and voice and analyzing their emotions.

[1683] "Means for providing content based on recognized emotions" refers to a system for displaying optimal content and advertisements based on analyzed emotional data.

[1684] "Means for collecting and analyzing browsing data and emotional data" refers to a data processing system for recording and analyzing a user's content operation history and recognized emotional data.

[1685] "Means for suggesting advertising placements and paid content" refers to algorithms and systems that recommend optimal advertising and paid content to users based on collected data.

[1686] The present invention relates to a webtoon platform that supports multiple languages ​​and allows individual scenario selection, and is a system that provides optimal content and advertisements based on user emotions. Specific embodiments will be described below.

[1687] System Program Overview

[1688] This system implements a program that includes the following main functions:

[1689] 1. Upload and save webtoons

[1690] The server provides an interface for users to upload image content (e.g., webtoons), which is then temporarily stored.

[1691] 2. Extraction and translation of text data

[1692] The server uses an OCR (Optical Character Recognition) engine to extract text data from image content, which is then translated into multiple languages ​​using the Google Translate API.

[1693] 3. Identifying and generating scenario branching points

[1694] The server uses the extracted text data to identify key story branching points, and uses a generative AI model to automatically generate different scenarios and endings, which are then stored in a database.

[1695] 4. Emotion recognition and personalized content delivery

[1696] It uses the camera and microphone on the device (e.g., smartphone) to recognize the user's emotions in real time. It uses the EmotionRecognizer module to analyze the user's emotional state from their facial expressions and voice. Based on the analysis results, it provides optimal content and personalized advertisements.

[1697] 5. Data Collection and Analysis

[1698] The server collects user browsing data and emotional data and stores it in a database. This data is analyzed to gain insights into user preferences and behavioral patterns. This data is used to optimize ad placement and suggest paid content.

[1699] Hardware and software used

[1700] Hardware

[1701] Devices such as smartphones and tablets: Used to upload and view content from users.

[1702] Camera and microphone: Captures the user's facial expressions and voice for emotion recognition.

[1703] software

[1704] OCR engine: Used to extract text from images.

[1705] Google Translate API: Used to translate the extracted text into multiple languages.

[1706] EmotionRecognizer: A software module for recognizing user emotions.

[1707] Generative AI models: Used to automatically generate different scenarios and endings.

[1708] Specific examples

[1709] For example, if a user opens the app at night and emotion recognition detects the emotion of "joy," the server will prioritize displaying cheerful and upbeat stories and related ads, providing the best possible experience for the user.

[1710] Prompt Sentence Examples

[1711] An example of a specific prompt for the generative AI model is, "Please generate what kind of story development would be best for a user who is in a joyful mood." Based on this prompt, the generative AI model automatically generates multiple story developments that suit the user's emotional state.

[1712] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1713] Step 1:

[1714] Users upload image content (e.g., webtoons) from their devices. They also enter meta information such as the title, author's name, and language used. The server receives this input data and temporarily stores it. Specifically, data is sent via a web interface, and the server temporarily stores the uploaded image file and meta information.

[1715] Step 2:

[1716] The server extracts text data from the stored image content, using an OCR engine (e.g., Google Cloud Vision API) to convert the text in the image into digital text. This allows the character information embedded in the image to be obtained. The input is the temporarily stored image data, and the output is digital text.

[1717] Step 3:

[1718] The server translates the extracted text data into multiple languages. Specifically, it uses the Google Translate API to translate, for example, Japanese into English, Chinese, Spanish, etc. This process generates translated text (output) into each target language based on the original text (input).

[1719] Step 4:

[1720] The server reinserts the translated text into the original webtoon image to generate the webtoon in each language. The generated content for each language is stored in a database. The input is the translated text and the original image file, and the output is a multilingual webtoon image. In this reinsertion process, the text is embedded into the image using image editing software (e.g., Python's Pillow library).

[1721] Step 5:

[1722] The server identifies key branching points based on the original content. Based on the identified branching points, it uses a generative AI model to automatically generate different scenarios and endings. In this process, multiple scenarios and endings (outputs) are generated from the original content (input) and stored in a database. Text analysis algorithms and generative AI algorithms are used for identification and generation.

[1723] Step 6:

[1724] The device captures the user's facial expressions and voice through the user's camera and microphone and sends them to the server. The server then uses the EmotionRecognizer module to analyze the user's emotions in real time from the transmitted facial and voice data. The input is the captured multimedia data, and the output is the analyzed emotional information.

[1725] Step 7:

[1726] The server provides optimal content and personalized advertisements based on the analyzed emotional data. This involves retrieving webtoons with the appropriate language version and scenario from a database and sending them to the user's device. The input is the emotional data and user selection information, and the output is the appropriate content and advertisements.

[1727] Step 8:

[1728] The server collects users' browsing data and emotional data and stores it in a database. The collected data is analyzed to derive insights that identify users' preferences, interests, and emotional trends. The input is user interaction data, and the output is the analysis results. Data mining techniques and machine learning algorithms are used for the analysis.

[1729] Step 9:

[1730] The server then uses the analysis results to suggest ad placements and paid content, thereby displaying the most relevant ads and content to the user. The input is the analysis results, and the output is a list of recommended ads and paid content. A recommendation system algorithm is used in this process.

[1731] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1732] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1733] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1734] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1735] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1736] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1737] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1738] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1739] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1740] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1741] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1742] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1743] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1744] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1745] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1746] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1747] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1748] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1749] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1750] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1751] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1752] The following is further disclosed regarding the above embodiment.

[1753] (Claim 1)

[1754] A means for users to upload webtoon content;

[1755] a means for temporarily storing uploaded content;

[1756] A means for extracting text data from the content and translating it into multiple languages;

[1757] A means of storing the translated content in a database;

[1758] A means for generating multiple scenarios and endings and storing them in a database;

[1759] A means for providing appropriate content based on user selection;

[1760] means for collecting and analyzing user browsing data;

[1761] A system that includes a means for suggesting advertisement placement and paid content based on the above analysis.

[1762] (Claim 2)

[1763] 10. The system of claim 1, which provides optimal content based on a language and scenario selected by a user.

[1764] (Claim 3)

[1765] The system of claim 1, wherein the collected user browsing data is utilized to improve the accuracy of the generating AI algorithm.

[1766] "Example 1"

[1767] (Claim 1)

[1768] a means for users to upload digital comic content;

[1769] a means for temporarily storing uploaded content;

[1770] A means for extracting text data from the content and translating it into multiple languages;

[1771] A means of storing the translated content in a database;

[1772] A means for using a generation AI engine that generates multiple scenarios and endings;

[1773] A means for storing the generated scenario and ending in a database;

[1774] A means for providing appropriate language and scenario content based on the user's selection;

[1775] means for collecting and analyzing user browsing data;

[1776] A system that includes a means for suggesting advertisement placement and paid content based on the above analysis.

[1777] (Claim 2)

[1778] 10. The system of claim 1, which provides optimal content based on a language and scenario selected by a user.

[1779] (Claim 3)

[1780] The system of claim 1, wherein the collected user browsing data is utilized to improve the accuracy of the generating AI algorithm.

[1781] "Application Example 1"

[1782] New Claims

[1783] (Claim 1)

[1784] A means for users to upload webtoon content;

[1785] a means for temporarily storing uploaded content;

[1786] A means for extracting text data from the content and translating it into multiple languages;

[1787] A means of storing the translated content in a database;

[1788] A means for generating multiple scenarios and endings and storing them in a database;

[1789] A means for providing appropriate content based on user selection;

[1790] means for collecting and analyzing user browsing data;

[1791] A means for proposing advertisement placement and paid content based on the above analysis;

[1792] A system including a means for providing the system as an application installed on a smartphone, smart glasses, a head-mounted display, or a robot.

[1793] (Claim 2)

[1794] 10. The system of claim 1, which provides optimal content based on a language and scenario selected by a user.

[1795] (Claim 3)

[1796] The system of claim 1, wherein the collected user browsing data is utilized to improve the accuracy of the generating AI algorithm.

[1797] "Example 2: Combining Emotion Engines"

[1798] (Claim 1)

[1799] a means for uploading content from users;

[1800] a means for temporarily storing uploaded content;

[1801] means including a language analysis module and a translation module for extracting character data from content and translating the data into multiple languages;

[1802] A means of storing the translated content in a database;

[1803] a generating AI engine for generating different scenarios and endings;

[1804] A means for storing the generated multiple scenarios and endings in a database;

[1805] A means including an emotion recognition engine for collecting emotion data through a user's camera and microphone and performing emotion analysis;

[1806] A means for providing optimal content based on user selection and sentiment analysis results;

[1807] A means for collecting and storing user browsing data and emotion data in a database;

[1808] A means of analyzing data and proposing advertising placement and paid content based on the analysis results;

[1809] A system including means for using collected and analyzed data to improve the accuracy of generative AI algorithms and emotion recognition engines.

[1810] (Claim 2)

[1811] 10. The system of claim 1, which provides optimal content based on a language and scenario selected by a user.

[1812] (Claim 3)

[1813] The system of claim 1, wherein the collected user browsing data and emotion data are utilized to improve the accuracy of the generative AI algorithm and emotion recognition engine.

[1814] "Application example 2 when combining emotion engines"

[1815] (Claim 1)

[1816] a means for uploading image content from a user;

[1817] a means for temporarily storing uploaded content;

[1818] A means for extracting text data from the content and translating it into multiple languages;

[1819] A means of storing the translated content in a database;

[1820] A means to identify key branching points and generate multiple scenarios and endings;

[1821] A means of recognizing the user's emotions through a camera or microphone,

[1822] A means to provide optimal content and advertisements based on recognized emotions, and

[1823] means for collecting and analyzing user browsing data and emotional data;

[1824] A system including a means for suggesting advertisement placement and paid content based on the above analysis.

[1825] (Claim 2)

[1826] 10. The system of claim 1, which provides optimal content based on a language and scenario selected by a user.

[1827] (Claim 3)

[1828] The system of claim 1, wherein the collected user browsing data and sentiment data are utilized to improve the accuracy of the generative AI algorithm. [Explanation of symbols]

[1829] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for users to upload webtoon content; a means for temporarily storing uploaded content; A means for extracting text data from the content and translating it into multiple languages; A means of storing the translated content in a database; A means for generating multiple scenarios and endings and storing them in a database; A means for providing appropriate content based on user selection; means for collecting and analyzing user browsing data; A system that includes a means for suggesting advertisement placement and paid content based on the above analysis.

2. The system according to claim 1, wherein the system provides optimal content based on a language and scenario selected by a user.

3. The system of claim 1, wherein the collected user browsing data is utilized to improve the accuracy of the generating AI algorithm.

Images