system

Abstract

We provide the system. [Solution] An input means for receiving the genre, theme, and related characteristics of content from the user, An analysis method that analyzes received information and compares it with past content, A generation method for generating new scripts while referencing past content, A means of creating storyboards based on the generated script, A means of sending the generated script and storyboard to the user, A means of regenerating the script and storyboards based on user feedback, A system that includes this.

Description

Technical Field

[0005] ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In content production, there is a demand for a system that can efficiently and easily generate new and creative scripts and picture contents. However, the conventional technology only imitates past works and has difficulty in generating content rich in novelty and originality. Also, for individuals and small production teams, cost and technical knowledge often become hurdles when using existing tools, making it difficult to create new stories and visuals.

Means for Solving the Problems

[0005] This invention provides a generation means that analyzes user-input content, such as genre and theme, using natural language processing and machine learning, and generates original scripts while referencing a past content database. It also includes a creation means that automatically generates storyboards from the generated scripts, thereby solving conventional problems. Furthermore, it provides a revision means that allows for the iterative generation of better content by receiving and incorporating feedback through user interaction. In this way, it becomes possible to automatically generate efficient and highly original content that meets the creative needs of creators.

[0006] A "user" refers to an individual or group that wants to use the system to generate scripts or storyboards for content.

[0007] "Content" is a general term referring to creative works, including stories, storyboards, and scripts.

[0008] "Genre" refers to the criteria used to classify themes, styles, and categories in screenplays and content.

[0009] "Theme" is a concept that refers to the central subject or message of a story or content.

[0010] "Analysis means" refers to a function or process within a system that understands information input by the user and performs necessary processing based on that information.

[0011] "Generation means" refers to a function or process within a system used to automatically create new scripts or stories based on analyzed information.

[0012] "Creation method" refers to a function or process within a system used to automatically create storyboards and visual materials based on the generated script.

[0013] "Transmission means" refers to the methods and functions for delivering the generated and created scripts and storyboards to the user.

[0014] "Correction measures" refer to functions or processes within the system used to improve generated scripts and storyboards based on user feedback.

[0015] "System" refers to the overall mechanism for generating and providing content through the means and processes employed in this invention. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the language used in the following description will be explained.

[0019] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] The system of this invention is designed to allow users to easily create new and original content. Through an interface that operates on various devices, users can input details about the genre, theme, and characters of their story. This allows users to indicate a specific direction for their content through the system.

[0038] When the server receives a request from a user, it uses natural language processing (NLP) to analyze the input information. This analysis helps the server understand the characteristics of the content the user is seeking and references it to a database of past content stored within the system. Based on the reference results, the system creates a new scenario using a specific generation algorithm.

[0039] The generated scenario is visualized as a related storyboard. Storyboard creation utilizes AI technology to automatically translate the scenario's content into a visual format. This allows users to obtain not only the written script but also storyboards that aid visual understanding.

[0040] The generated content is sent to the user's device via the internet. The user can view the content on their device and submit feedback as needed. This feedback is sent to the server, and the system then uses it to revise and update the content, creating a continuous cycle. This entire process allows users to interactively create content and obtain the final product they desire.

[0041] For example, if a user requests an "adventure story in a fantasy world," the server can refer to past works that fit that theme and generate a new scenario and imaginative visuals for the story. In this way, the present invention is a system that flexibly responds to the needs of a wide range of users, from professionals to amateurs, and contributes to the creation of creative content.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] The user inputs the genre, theme, character settings, and other details of the content they want to generate through the device's interface. This information is prepared as structured request data.

[0045] Step 2:

[0046] The terminal sends the request data received from the user to the server. This request contains all the necessary input information.

[0047] Step 3:

[0048] The server applies natural language processing (NLP) to analyze the received request data. This process extracts key information and requirements contained in the request.

[0049] Step 4:

[0050] Based on the analysis results, the server searches its internal database to refer to similar past content. This identifies patterns and elements of highly relevant content.

[0051] Step 5:

[0052] The server's AI model utilizes referenced past content information to generate new scenarios tailored to user requests. These scenarios include unique storylines and character plots.

[0053] Step 6:

[0054] The server automatically creates storyboards based on the generated scenario. A visual generation algorithm defines the visual representation for each scene.

[0055] Step 7:

[0056] The server then packages the final generated scenario and storyboards and prepares them for transmission to the terminal.

[0057] Step 8:

[0058] The terminal receives data sent from the server and displays the scenario and storyboards for easy viewing by the user. The user can then review this content.

[0059] Step 9:

[0060] Users provide feedback on the generated content and submit requests for corrections or improvements if necessary.

[0061] Step 10:

[0062] The server analyzes user feedback and modifies or regenerates the content. This process is repeated until the user is satisfied.

[0063] (Example 1)

[0064] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0065] Conventional content generation systems have a problem where users have to manually edit and adjust many things when creating new stories or visual content on their own themes, making the creative process very time-consuming and laborious. Furthermore, the process of incorporating feedback on the generated content is also complicated, making it difficult to respond quickly to user needs and expectations. This invention aims to solve these problems and realize a system that allows users to generate content easily and instantly and easily modify it as needed.

[0066] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0067] In this invention, the server includes input means for receiving content categories, themes, and related attributes from a user; analysis means for analyzing the received information using natural language processing technology and comparing it with past information data; and generation means for generating a new story using a generative AI model while referring to past information data. This enables users to easily create and visualize their own content.

[0068] "Input means" refers to a device or method for receiving content categories, subjects, and related attributes from a user.

[0069] "Analysis means" refers to a device or method for analyzing received information using natural language processing technology and comparing it with past information data.

[0070] A "generative AI model" is a model that uses artificial intelligence technology to generate new narratives while referencing past information data.

[0071] "Generative means" refers to a device or method for creating a new story using a generative AI model.

[0072] "Creation means" refers to a device or method for automatically creating visualized content based on a generated narrative.

[0073] "Transmission means" refers to a device or method for transmitting generated narrative and visualization content to a user.

[0074] "Modification means" refers to a device or method for receiving user evaluation information and regenerating narrative and visualization content.

[0075] "Search means" refers to a device or method for searching multiple historical data sets and identifying relevant data based on a user's request.

[0076] "Display means" refers to a device or method that provides a user interface for easily displaying and editing generated narratives and visualization content.

[0077] In this embodiment of the invention, the system consists of three main components: a user, a server, and a terminal. The user uses the terminal to access the system's interface and input details such as the content category, subject, and characters. This allows the user to specifically instruct the server on the direction of the content they desire. An example of a prompt might be, "Please generate an adventure story set in a fantasy world. The main characters are a brave knight and a wizard."

[0078] When the server receives a request from this user, it analyzes the input information using natural language processing (NLP) techniques. NLP, also known as a parsing engine, interprets the meaning of the input and compares it to historical data. In this process, the server refers to a database of previously collected and stored information to generate content best suited to the user's request.

[0079] Next, the server uses a generative AI model to generate a new story. This generative AI model incorporates the intended themes and characters from the prompt text, constructing original and novel content that aligns with the user's requests. This process utilizes advanced machine learning algorithms, aiming to ensure that the generated story exceeds user expectations.

[0080] Furthermore, based on the generated story, the server uses AI technology to create visualized content. This creation process allows users to obtain not only text content but also visual information. This enables users to perceive the story in a more three-dimensional way.

[0081] Finally, the server sends this generated story and visualized content to the user's device. The user can review the content via their device and provide feedback to the server as needed. The server receives this feedback and regenerates or modifies the content. Through this cycle, the user can gradually evolve their own creation towards its final form.

[0082] In this embodiment, technologies and processes are incorporated to quickly and efficiently fulfill the user's creative requirements, resulting in a system that can meet a wide range of needs and also contribute to improving the quality of content creation.

[0083] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0084] Step 1:

[0085] The user inputs information such as content category, subject, and character details through the terminal's interface. This input data is sent to the server in JSON or text format. At this stage, the terminal organizes the information according to the user's instructions and generates prompts. For example, the prompt might read, "Please create an adventure story in a fantasy world. The main characters are a brave knight and a wizard."

[0086] Step 2:

[0087] The server analyzes the prompt message received from the terminal using natural language processing techniques. This analysis involves keyword extraction and theme identification, and data processing to clearly understand the user's request. Based on the input prompt message, the server searches its historical information database to retrieve relevant data. The output is an analysis result of the detailed requirements for the content the user is requesting.

[0088] Step 3:

[0089] The server generates a new story using a generative AI model based on the analysis results. The generative AI model utilizes machine learning algorithms to construct a story suitable for the user's requested themes and characters. The analysis results are used as input, and the generation process outputs a creative and original story. This generated story includes elements such as the plot and character dialogue.

[0090] Step 4:

[0091] The server creates visualized content based on the generated story. In this step, AI technology is used to convert the story into storyboards or visual storyboards. The generated story is the input, and after the visualization process, visual information is obtained as output. Specifically, character illustrations and scene descriptions are automatically generated.

[0092] Step 5:

[0093] The server sends the final content (story and visualized content) to the user's device via the internet. The user can review the sent content on their device and provide feedback, including evaluations and modifications. At this stage, the device organizes the feedback information provided by the user and sends it back to the server.

[0094] Step 6:

[0095] The server receives user feedback and uses it to regenerate and modify story and visualization content. User feedback information is input, and data calculations and modifications are performed based on it. At this stage, the generated story and visual elements are re-evaluated as needed, and new output is created. This allows users to obtain more satisfying content.

[0096] (Application Example 1)

[0097] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0098] When users create original and novel content, specialized skills and knowledge are required, and generating and visualizing the content is difficult. Furthermore, the limited means of quickly distributing and sharing the generated content can potentially detract from the user experience.

[0099] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0100] In this invention, the server includes an input means for receiving content types and themes from a user, an analysis means for analyzing the received information and comparing it with existing information, a generation means for generating a new story while referring to past information resources, and a creation means for creating visual content based on the generated story. This enables users to easily generate their own content without requiring specialized knowledge and to quickly distribute and share it.

[0101] A "user" is an individual or group that uses the system to create original content.

[0102] "Content" refers to stories, visual materials, and other creative works created by users.

[0103] A "type" is a category or classification used to identify the nature or style of content.

[0104] "Subject matter" refers to the elements specified by the user as the theme or subject of the content.

[0105] "Attributes" refer to information about the detailed settings of characters and stories, and serve as the basis for content creation.

[0106] An "input method" is an interface that allows users to provide information such as categories, subjects, and attributes to the system.

[0107] "Analysis means" refers to a method for processing received information and evaluating its relationship to existing information.

[0108] A "generative means" is a mechanism for creating new narratives based on past information.

[0109] "Means of creation" refers to the means of visually representing the generated story, which involves forming a visual storyboard.

[0110] "Correction methods" refer to ways of updating the story and visual materials based on user feedback.

[0111] "Visual content" refers to visual deliverables created to be easily viewed and shared by users.

[0112] This invention is a system that automates the process of users creating their own content and visually representing it. Users input the content type, subject matter, and related attributes via a smartphone or computer. This input information is received by a server.

[0113] The server analyzes the received data using natural language processing (NLP) techniques (e.g., spaCy or GPT APIs). The analysis results are then compared to a database of past content. This identifies relevant information resources, and a generative AI model (e.g., OpenAI®'s GPT) is used to create a new narrative.

[0114] Next, the server creates a visual storyboard based on the generated narrative. This utilizes AI-driven visualization tools. The visual storyboard is designed to make the content easier for the user to understand.

[0115] Users can easily review the generated visual content through an interface on their smartphone or computer and submit feedback as needed. The server receives this feedback, restarts the generation process, and improves the content.

[0116] For example, if the user selects "A story about solving a mystery case at school," the server will generate a mystery-solving scenario using characters and settings in a style related to school stories. An example of a prompt to the generating AI model is "Genre: School Mystery, Theme: Detective Club and Mystery."

[0117] This entire process allows users to create high-quality, original content and share it quickly, without requiring specialized technical skills.

[0118] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0119] Step 1:

[0120] The terminal receives content type, subject matter, and attributes as input from the user. The input data is formatted into the required format and sent to the server. This prepares the server to correctly retrieve the information necessary for analysis.

[0121] Step 2:

[0122] The server receives the data as input and performs analysis using natural language processing (NLP) techniques. Specifically, it analyzes the input keywords and phrases to understand the user's intent. Next, it uses the analysis results to compare with existing databases and identifies relevant past content as output.

[0123] Step 3:

[0124] The server generates a new story using a generative AI model based on identified past content information. A prompt (e.g., "Genre: School Mystery, Theme: Detective Club and Mystery") is input to the generative AI model, which then outputs a story scenario. In this step, the AI ​​model utilizes past data to create a creative scenario.

[0125] Step 4:

[0126] The server creates a visual storyboard based on the outputted narrative. At this stage, an AI-driven visualization tool is used to visually represent the scenes of the story. The visual storyboard is output as images and illustrations that the user can intuitively understand.

[0127] Step 5:

[0128] Users receive visual content generated through their device as input and review it. This interface allows users to input and submit feedback as needed. This feedback is sent to the server and serves as input for further improvement.

[0129] Step 6:

[0130] The server receives user feedback as input and regenerates narratives and visual content. Specifically, it analyzes the feedback, updates the prompt text, and inputs it back into the generation AI model to output an improved scenario. This cycle results in content that meets the user's expectations.

[0131] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0132] This invention relates to a system that recognizes user emotions and generates content that reflects those emotions. This system personalizes the generated content by incorporating user emotions as an intervention element.

[0133] Users can use an interface on their device to input the genre, theme, and character details of the content they desire, as well as provide feedback on the situation, including emotions. The user's device is equipped with a camera and sensors, and the data collected through these is sent to a server for analysis by an emotion engine.

[0134] The server analyzes the input data and uses an emotion engine to determine the user's emotions. Based on this information, it references relevant information from a past content database and generates a new script. The tone and style of the generated content are dynamically adjusted according to the user's emotions, allowing users to access content that resonates more deeply with them.

[0135] The generated script is visualized as a storyboard using AI. In this process as well, emotional information is reflected in the scene composition and character expressions, enabling more emotionally responsive depictions.

[0136] For example, if a user requests an "exciting adventure story" and their emotion is recognized as "excitement," the server can refer to past works that reflect similar emotions and generate a scenario with a higher level of tension. In this way, by utilizing emotional information, content creation that is more responsive to user requests can be achieved.

[0137] In this system, users can continuously send feedback that includes emotional data, which the server analyzes with an emotion engine to improve the script and storyboards. This makes the creation process more interactive and dynamic, rather than merely a one-way street.

[0138] The following describes the processing flow.

[0139] Step 1:

[0140] Users input content genre, theme, character details, and their current emotional state from their device. The emotional state is recognized by monitoring the user's facial expressions, heart rate, and other data acquired through the device's camera and sensors.

[0141] Step 2:

[0142] The device sends the user's portrait data and physiological data along with the entered information to the server. This data is necessary for the emotion engine's analysis.

[0143] Step 3:

[0144] The server analyzes the received information, using natural language processing (NLP) to decode the text information while simultaneously activating an emotion engine to analyze emotional data. This analysis identifies the user's current emotions.

[0145] Step 4:

[0146] Based on the analyzed information, the server searches its internal database for similar past content. Based on the search results, it collects story elements that match the user's desired genre and theme.

[0147] Step 5:

[0148] The server's AI model generates new scenarios using reference results and user sentiment data. The tone and plot points of the script are adjusted to match the sentiment data.

[0149] Step 6:

[0150] Based on the generated script, the server creates storyboards. At this time, the character's facial expressions and the atmosphere of the scene are set according to the situation, reflecting the results of the emotion engine.

[0151] Step 7:

[0152] The server sends the generated script and storyboards to the user's terminal and provides visuals and text for the user to review the results.

[0153] Step 8:

[0154] Users can review the displayed script and storyboards and send feedback from their device if necessary. This feedback can also include additional emotional data.

[0155] Step 9:

[0156] The server re-analyzes the received feedback and revises the script and storyboards. This allows for content improvements that address new emotions and requests.

[0157] (Example 2)

[0158] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0159] There is a need to generate content that accurately reflects user emotions, enhance content personalization, and enable interactive communication with users. However, conventional systems have been unable to fully utilize user emotions, resulting in uniform content, which has been a problem.

[0160] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0161] In this invention, the server includes acquisition means for acquiring diverse content attributes and emotion-related information from the user, emotion analysis means for analyzing the acquired information and determining the user's emotions, and generation means for creating new content by referring to past information based on the determined emotion information. This makes it possible to generate personalized content that responds to the user's emotions.

[0162] "Acquisition methods" refer to mechanisms for collecting content attributes and sentiment-related information from users.

[0163] An "emotion analysis tool" is a system that analyzes acquired information to determine the user's emotions.

[0164] "Generation method" refers to the process of creating new content by referencing past information based on identified emotional information.

[0165] "Visualization methods" refer to technologies for representing generated scripts or content as visual information.

[0166] "Distribution method" refers to the method used to deliver generated content to users.

[0167] A "reconstruction method" is a function that reshapes and improves visual information based on user feedback.

[0168] "Search methods" refer to techniques that utilize sentiment data to efficiently search past information and identify relevant information.

[0169] A "display means" is an interface that presents generated visual information to the user and enables editing.

[0170] This invention is a system for generating personalized content that reflects the user's emotions. The embodiments thereof are described below.

[0171] Users input the attributes of their desired content using their device and provide sentiment data. This sentiment data is acquired through cameras and sensors and transmitted to the server in real time. The device also has a function to collect user feedback.

[0172] The server is equipped with emotion analysis capabilities to analyze the acquired information. Here, image recognition technology is used to determine emotions from facial expressions. To generate content tailored to users with specific emotions, the server uses generation capabilities to refer to past information and create new content. This process utilizes a generation AI model; for example, if a user requests an exciting story, high-energy content will be generated.

[0173] The generated content is visualized in storyboard format using visualization tools. Character expressions and scene tones are adjusted based on the user's emotions. This visual information is provided to the user via distribution tools and can be easily viewed and edited through their device.

[0174] User feedback, continuously provided, is analyzed using restructuring methods and used to improve content. This interactive cycle ensures that content is always optimized for the user.

[0175] For example, if a user requests an "exciting adventure story" and their emotion is determined to be "excitement," the server can send the prompt message "Generate an adventure story centered on excitement" to the AI ​​model, thereby creating content that meets the user's expectations. In this way, the present invention realizes user-centered content generation through the utilization of emotion data.

[0176] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0177] Step 1:

[0178] Users input desired content attributes, such as genre or theme, through their device. The device's camera and sensors also capture emotional data from facial expressions and voice. Input information includes user text instructions and emotional data from sensors. This information is then sent to a server for analysis.

[0179] Step 2:

[0180] The server receives text information and emotion data from the user. Here, using emotion analysis tools, image recognition technology identifies emotions such as "excitement," "joy," and "sadness" from facial expressions. The input data is classified into emotion categories during the analysis process and output as a dataset representing the user's emotional state.

[0181] Step 3:

[0182] The server generates new content using a generation method based on the analyzed emotional information. During this process, it compares the data with past databases and incorporates suitable story elements. A generation AI model is used to create prompts and send instructions to the AI. For example, an instruction such as "Generate an adventure story centered on excitement" might be used. The output is a personalized scenario tailored to the user's emotions.

[0183] Step 4:

[0184] The server converts the generated scenario into a storyboard format using visualization tools. Based on emotional information, character expressions and background tones are dynamically adjusted. The input is the generated scenario, and the output is visualized content display data.

[0185] Step 5:

[0186] The user receives visual content generated through the device. The device displays the visual content and provides a user interface for the user to review and edit the content. User feedback is also collected through the device and sent to the server.

[0187] Step 6:

[0188] The server analyzes user feedback using a reconstruction mechanism and makes revisions to the content and storyboards. The recalculated output, based on the input data obtained from the feedback, is improved new content. This continuously optimizes the content provided to the user.

[0189] (Application Example 2)

[0190] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0191] Providing personalized content that responds to user emotions is a crucial challenge in content delivery services. However, traditional methods make it difficult to analyze user emotions in real time and instantly generate content that responds to those emotions. This results in a problem where the user experience is not sufficiently improved.

[0192] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0193] In this invention, the server includes acquisition means for receiving content genre, theme, and related features from the user; sensor means for detecting the user's emotions and acquiring emotion data; and analysis means for analyzing the acquired information and comparing it with past data. This enables the dynamic adjustment of tone and style according to the user's emotions, and the provision of personalized content in real time.

[0194] "Acquisition means" refers to a mechanism for collecting content genre, theme, and related characteristics from users.

[0195] A "sensor device" is a device used to detect a user's emotions and collect that emotional data.

[0196] "Analysis means" refers to functions for analyzing acquired information and comparing it with past data sets.

[0197] "Generative means" refers to methods for creating new scripts or content based on analysis results.

[0198] "Creation means" refers to a device or software for constructing scene designs based on the generated script.

[0199] "Transmission means" refers to communication means for transferring generated content to the user.

[0200] A "correction mechanism" is a system for receiving user feedback and regenerating the script and scene design.

[0201] "Past information" refers to a database of content that has been collected or generated to date.

[0202] "Dynamically adjusting tone and style" means changing the style and expression of generated content in response to the user's emotional information.

[0203] "Interactive content" refers to content that can change its content in response to user input or emotions.

[0204] The system of this invention operates in conjunction with a user's terminal and a central server. The user's terminal is equipped with an interface for acquiring necessary information and a sensor for detecting emotions. The terminal first acquires the content genre, theme, and related features from the user. This information forms the basis for what kind of content the user desires.

[0205] Next, the device uses its camera and microphone to detect the user's emotions in real time and sends that data to the server. Specifically, it recognizes emotions by analyzing facial expressions from camera images and analyzing the tone of voice. For this, software such as Amazon Rekognition can be used for image analysis, and Google Cloud Speech-to-Text can be used for voice analysis.

[0206] The server processes the received user request and emotional data using analytical tools, and generates new content that dynamically adjusts the tone and style to match the user's emotions, while referencing past data. This involves creating prompts using a generative AI model and forming the content scenario. For example, if the user wants to relax, the server will generate a prompt such as, "Please create a story that includes a relaxing scene."

[0207] The generated content is sent from the server to the user's device, allowing the user to experience the visualized content. If the user provides feedback, that data is sent back to the server and used in the next generation process. This feedback loop allows the content to become more personalized to the user.

[0208] The overall configuration of this system provides users with new experiences and enables innovative content delivery that resonates with their emotions.

[0209] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0210] Step 1:

[0211] The user enters the content genre, theme, and related characteristics into the device. The entered data is acquired through the device's input method and stored in memory as basic information about the user's preferences. After acquiring this data, the system is ready to proceed to sentiment detection.

[0212] Step 2:

[0213] The device uses a camera and microphone to detect the user's emotions. This involves facial recognition and voice tone analysis, as well as data processing to collect emotional data in real time. The emotional data is analyzed using specific algorithms (e.g., OpenCV or TENSORFLOW®), and the analysis results are sent to a server.

[0214] Step 3:

[0215] The server processes the received user request information and sentiment data using analytical tools. In this step, it searches through past data sets and performs data calculations to extract relevant information. Based on the analysis results, it creates prompt sentences using a generative AI model and generates new content scenarios.

[0216] Step 4:

[0217] Through the generation mechanism, the server generates content based on prompt messages. For example, using the prompt message "Create a story that includes a relaxing landscape," a scenario with a tone and style that matches the user's emotions is generated. This generated scenario is then converted into data as visual content.

[0218] Step 5:

[0219] The server sends the generated content to the terminal. The terminal receives the transmitted data and displays it in a way that the user can visually access. The visualized content provides the user with a new experience.

[0220] Step 6:

[0221] Users input feedback about the content they experienced into their device. This feedback is sent back to the server, analyzed by a correction system, and incorporated into future content creation. Through this process, content is further personalized based on user feedback.

[0222] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0223] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0224] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0225] [Second Embodiment]

[0226] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0227] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0228] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0229] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0230] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0231] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0232] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0233] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0234] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0235] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0236] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0237] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0238] The system of this invention is designed to allow users to easily create new and original content. Through an interface that operates on various devices, users can input details about the genre, theme, and characters of their story. This allows users to indicate a specific direction for their content through the system.

[0239] When the server receives a request from a user, it uses natural language processing (NLP) to analyze the input information. This analysis helps the server understand the characteristics of the content the user is seeking and references it to a database of past content stored within the system. Based on the reference results, the system creates a new scenario using a specific generation algorithm.

[0240] The generated scenario is visualized as a related storyboard. Storyboard creation utilizes AI technology to automatically translate the scenario's content into a visual format. This allows users to obtain not only the written script but also storyboards that aid visual understanding.

[0241] The generated content is sent to the user's device via the internet. The user can view the content on their device and submit feedback as needed. This feedback is sent to the server, and the system then uses it to revise and update the content, creating a continuous cycle. This entire process allows users to interactively create content and obtain the final product they desire.

[0242] For example, if a user requests an "adventure story in a fantasy world," the server can refer to past works that fit that theme and generate a new scenario and imaginative visuals for the story. In this way, the present invention is a system that flexibly responds to the needs of a wide range of users, from professionals to amateurs, and contributes to the creation of creative content.

[0243] The following describes the processing flow.

[0244] Step 1:

[0245] The user inputs the genre, theme, character settings, and other details of the content they want to generate through the device's interface. This information is prepared as structured request data.

[0246] Step 2:

[0247] The terminal sends the request data received from the user to the server. This request contains all the necessary input information.

[0248] Step 3:

[0249] The server applies natural language processing (NLP) to analyze the received request data. This process extracts key information and requirements contained in the request.

[0250] Step 4:

[0251] Based on the analysis results, the server searches its internal database to refer to similar past content. This identifies patterns and elements of highly relevant content.

[0252] Step 5:

[0253] The server's AI model utilizes referenced past content information to generate new scenarios tailored to user requests. These scenarios include unique storylines and character plots.

[0254] Step 6:

[0255] The server automatically creates storyboards based on the generated scenario. A visual generation algorithm defines the visual representation for each scene.

[0256] Step 7:

[0257] The server then packages the final generated scenario and storyboards and prepares them for transmission to the terminal.

[0258] Step 8:

[0259] The terminal receives data sent from the server and displays the scenario and storyboards for easy viewing by the user. The user can then review this content.

[0260] Step 9:

[0261] Users provide feedback on the generated content and submit requests for corrections or improvements if necessary.

[0262] Step 10:

[0263] The server analyzes user feedback and modifies or regenerates the content. This process is repeated until the user is satisfied.

[0264] (Example 1)

[0265] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0266] Conventional content generation systems have a problem where users have to manually edit and adjust many things when creating new stories or visual content on their own themes, making the creative process very time-consuming and laborious. Furthermore, the process of incorporating feedback on the generated content is also complicated, making it difficult to respond quickly to user needs and expectations. This invention aims to solve these problems and realize a system that allows users to generate content easily and instantly and easily modify it as needed.

[0267] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0268] In this invention, the server includes input means for receiving content categories, themes, and related attributes from a user; analysis means for analyzing the received information using natural language processing technology and comparing it with past information data; and generation means for generating a new story using a generative AI model while referring to past information data. This enables users to easily create and visualize their own content.

[0269] "Input means" refers to a device or method for receiving content categories, subjects, and related attributes from a user.

[0270] "Analysis means" refers to a device or method for analyzing received information using natural language processing technology and comparing it with past information data.

[0271] A "generative AI model" is a model that uses artificial intelligence technology to generate new narratives while referencing past information data.

[0272] "Generative means" refers to a device or method for creating a new story using a generative AI model.

[0273] "Creation means" refers to a device or method for automatically creating visualized content based on a generated narrative.

[0274] "Transmission means" refers to a device or method for transmitting generated narrative and visualization content to a user.

[0275] "Modification means" refers to a device or method for receiving user evaluation information and regenerating narrative and visualization content.

[0276] "Search means" refers to a device or method for searching multiple historical data sets and identifying relevant data based on a user's request.

[0277] "Display means" refers to a device or method that provides a user interface for easily displaying and editing generated narratives and visualization content.

[0278] In this embodiment of the invention, the system consists of three main components: a user, a server, and a terminal. The user uses the terminal to access the system's interface and input details such as the content category, subject, and characters. This allows the user to specifically instruct the server on the direction of the content they desire. An example of a prompt might be, "Please generate an adventure story set in a fantasy world. The main characters are a brave knight and a wizard."

[0279] When the server receives a request from this user, it analyzes the input information using natural language processing (NLP) techniques. NLP, also known as a parsing engine, interprets the meaning of the input and compares it to historical data. In this process, the server refers to a database of previously collected and stored information to generate content best suited to the user's request.

[0280] Next, the server uses a generative AI model to generate a new story. This generative AI model incorporates the intended themes and characters from the prompt text, constructing original and novel content that aligns with the user's requests. This process utilizes advanced machine learning algorithms, aiming to ensure that the generated story exceeds user expectations.

[0281] Furthermore, based on the generated story, the server uses AI technology to create visualized content. This creation process allows users to obtain not only text content but also visual information. This enables users to perceive the story in a more three-dimensional way.

[0282] Finally, the server sends this generated story and visualization content to the user's terminal. The user can view the content via the terminal and provide feedback to the server if necessary. The server receives this feedback and regenerates or modifies the content. Through this cycle, the user can gradually evolve their original creation into a complete form.

[0283] In this embodiment, technologies and processes for quickly and efficiently realizing the creative requirements of users are incorporated, resulting in a system that meets a wide range of needs and also contributes to improving the quality of content production.

[0284] The flow of specific processing in Example 1 will be described using FIG. 11.

[0285] Step 1:

[0286] The user inputs information such as the category, theme, and details of characters of the content through the interface of the terminal. These input data are sent to the server in JSON format or text format. At this stage, the terminal sorts the information according to the user's instructions and generates a prompt sentence. For example, input a prompt sentence like "I want to create an adventure story in a fantasy world. The main characters are a brave knight and a wizard."

[0287] Step 2:

[0288] The server analyzes the prompt sentence received from the terminal using natural language processing technology. In this analysis, keyword extraction and theme identification are performed, and data processing is carried out to clearly understand the user's requirements. Based on the input prompt sentence, the server searches the past information database to extract relevant data. As output, the analysis result of the detailed requirements of the content required by the user is obtained.

[0289] Step 3:

[0290] The server generates a new story using a generative AI model based on the analysis results. The generative AI model utilizes machine learning algorithms to construct a story suitable for the user's requested themes and characters. The analysis results are used as input, and the generation process outputs a creative and original story. This generated story includes elements such as the plot and character dialogue.

[0291] Step 4:

[0292] The server creates visualized content based on the generated story. In this step, AI technology is used to convert the story into storyboards or visual storyboards. The generated story is the input, and after the visualization process, visual information is obtained as output. Specifically, character illustrations and scene descriptions are automatically generated.

[0293] Step 5:

[0294] The server sends the final content (story and visualized content) to the user's device via the internet. The user can review the sent content on their device and provide feedback, including evaluations and modifications. At this stage, the device organizes the feedback information provided by the user and sends it back to the server.

[0295] Step 6:

[0296] The server receives user feedback and uses it to regenerate and modify story and visualization content. User feedback information is input, and data calculations and modifications are performed based on it. At this stage, the generated story and visual elements are re-evaluated as needed, and new output is created. This allows users to obtain more satisfying content.

[0297] (Application Example 1)

[0298] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0299] When users create original and novel content, specialized skills and knowledge are required, and generating and visualizing the content is difficult. Furthermore, the limited means of quickly distributing and sharing the generated content can potentially detract from the user experience.

[0300] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0301] In this invention, the server includes an input means for receiving content types and themes from a user, an analysis means for analyzing the received information and comparing it with existing information, a generation means for generating a new story while referring to past information resources, and a creation means for creating visual content based on the generated story. This enables users to easily generate their own content without requiring specialized knowledge and to quickly distribute and share it.

[0302] A "user" is an individual or group that uses the system to create original content.

[0303] "Content" refers to stories, visual materials, and other creative works created by users.

[0304] A "type" is a category or classification used to identify the nature or style of content.

[0305] "Subject matter" refers to the elements specified by the user as the theme or subject of the content.

[0306] "Attributes" refer to information about the detailed settings of characters and stories, and serve as the basis for content creation.

[0307] The "input means" is an interface for the user to provide information such as type, theme, attributes, etc. to the system.

[0308] The "analysis means" is a method for processing the received information and evaluating its relevance to existing information.

[0309] The "generation means" is a mechanism for creating new stories based on past information.

[0310] The "creation means" is a means for visually expressing the generated story and forms a visual storyboard.

[0311] The "modification means" is a method for updating the story and visual materials by reflecting the opinions from the user.

[0312] The "visual content" is a visual product created so that the user can easily view and share it.

[0313] This invention is a system that automates the process for the user to create their own content and visually represent it. The user inputs the type, theme, and related attributes of the content via a smartphone or computer. This input information is received by the server.

[0314] The server analyzes the received data using natural language processing (NLP) technologies (such as the APIs of spaCy or GPT). As a result of the analysis, the information is compared with the past content database. Thereby, relevant information resources are identified, and a new story is created using a generative AI model (such as OpenAI's GPT).

[0315] Next, the server creates a visual storyboard based on the generated story. For this, an AI-driven visualization tool is utilized. The visual storyboard is for making the content easier for the user to understand.

[0316] Users can easily review the generated visual content through an interface on their smartphone or computer and submit feedback as needed. The server receives this feedback, restarts the generation process, and improves the content.

[0317] For example, if the user selects "A story about solving a mystery case at school," the server will generate a mystery-solving scenario using characters and settings in a style related to school stories. An example of a prompt to the generating AI model is "Genre: School Mystery, Theme: Detective Club and Mystery."

[0318] This entire process allows users to create high-quality, original content and share it quickly, without requiring specialized technical skills.

[0319] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0320] Step 1:

[0321] The terminal receives content type, subject matter, and attributes as input from the user. The input data is formatted into the required format and sent to the server. This prepares the server to correctly retrieve the information necessary for analysis.

[0322] Step 2:

[0323] The server receives the data as input and performs analysis using natural language processing (NLP) techniques. Specifically, it analyzes the input keywords and phrases to understand the user's intent. Next, it uses the analysis results to compare with existing databases and identifies relevant past content as output.

[0324] Step 3:

[0325] The server generates a new story using a generative AI model based on identified past content information. A prompt (e.g., "Genre: School Mystery, Theme: Detective Club and Mystery") is input to the generative AI model, which then outputs a story scenario. In this step, the AI ​​model utilizes past data to create a creative scenario.

[0326] Step 4:

[0327] The server creates a visual storyboard based on the outputted narrative. At this stage, an AI-driven visualization tool is used to visually represent the scenes of the story. The visual storyboard is output as images and illustrations that the user can intuitively understand.

[0328] Step 5:

[0329] Users receive visual content generated through their device as input and review it. This interface allows users to input and submit feedback as needed. This feedback is sent to the server and serves as input for further improvement.

[0330] Step 6:

[0331] The server receives user feedback as input and regenerates narratives and visual content. Specifically, it analyzes the feedback, updates the prompt text, and inputs it back into the generation AI model to output an improved scenario. This cycle results in content that meets the user's expectations.

[0332] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0333] This invention relates to a system that recognizes user emotions and generates content that reflects those emotions. This system personalizes the generated content by incorporating user emotions as an intervention element.

[0334] Users can use an interface on their device to input the genre, theme, and character details of the content they desire, as well as provide feedback on the situation, including emotions. The user's device is equipped with a camera and sensors, and the data collected through these is sent to a server for analysis by an emotion engine.

[0335] The server analyzes the input data and uses an emotion engine to determine the user's emotions. Based on this information, it references relevant information from a past content database and generates a new script. The tone and style of the generated content are dynamically adjusted according to the user's emotions, allowing users to access content that resonates more deeply with them.

[0336] The generated script is visualized as a storyboard using AI. In this process as well, emotional information is reflected in the scene composition and character expressions, enabling more emotionally responsive depictions.

[0337] For example, if a user requests an "exciting adventure story" and their emotion is recognized as "excitement," the server can refer to past works that reflect similar emotions and generate a scenario with a higher level of tension. In this way, by utilizing emotional information, content creation that is more responsive to user requests can be achieved.

[0338] In this system, users can continuously send feedback that includes emotional data, which the server analyzes with an emotion engine to improve the script and storyboards. This makes the creation process more interactive and dynamic, rather than merely a one-way street.

[0339] The following describes the processing flow.

[0340] Step 1:

[0341] Users input content genre, theme, character details, and their current emotional state from their device. The emotional state is recognized by monitoring the user's facial expressions, heart rate, and other data acquired through the device's camera and sensors.

[0342] Step 2:

[0343] The device sends the user's portrait data and physiological data along with the entered information to the server. This data is necessary for the emotion engine's analysis.

[0344] Step 3:

[0345] The server analyzes the received information, using natural language processing (NLP) to decode the text information while simultaneously activating an emotion engine to analyze emotional data. This analysis identifies the user's current emotions.

[0346] Step 4:

[0347] Based on the analyzed information, the server searches its internal database for similar past content. Based on the search results, it collects story elements that match the user's desired genre and theme.

[0348] Step 5:

[0349] The server's AI model generates new scenarios using reference results and user sentiment data. The tone and plot points of the script are adjusted to match the sentiment data.

[0350] Step 6:

[0351] Based on the generated script, the server creates storyboards. At this time, the character's facial expressions and the atmosphere of the scene are set according to the situation, reflecting the results of the emotion engine.

[0352] Step 7:

[0353] The server sends the generated script and storyboards to the user's terminal and provides visuals and text for the user to review the results.

[0354] Step 8:

[0355] Users can review the displayed script and storyboards and send feedback from their device if necessary. This feedback can also include additional emotional data.

[0356] Step 9:

[0357] The server re-analyzes the received feedback and revises the script and storyboards. This allows for content improvements that address new emotions and requests.

[0358] (Example 2)

[0359] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0360] There is a need to generate content that accurately reflects user emotions, enhance content personalization, and enable interactive communication with users. However, conventional systems have been unable to fully utilize user emotions, resulting in uniform content, which has been a problem.

[0361] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0362] In this invention, the server includes acquisition means for acquiring diverse content attributes and emotion-related information from the user, emotion analysis means for analyzing the acquired information and determining the user's emotions, and generation means for creating new content by referring to past information based on the determined emotion information. This makes it possible to generate personalized content that responds to the user's emotions.

[0363] "Acquisition methods" refer to mechanisms for collecting content attributes and sentiment-related information from users.

[0364] An "emotion analysis tool" is a system that analyzes acquired information to determine the user's emotions.

[0365] "Generation method" refers to the process of creating new content by referencing past information based on identified emotional information.

[0366] "Visualization methods" refer to technologies for representing generated scripts or content as visual information.

[0367] "Distribution method" refers to the method used to deliver generated content to users.

[0368] A "reconstruction method" is a function that reshapes and improves visual information based on user feedback.

[0369] "Search methods" refer to techniques that utilize sentiment data to efficiently search past information and identify relevant information.

[0370] A "display means" is an interface that presents generated visual information to the user and enables editing.

[0371] This invention is a system for generating personalized content that reflects the user's emotions. The embodiments thereof are described below.

[0372] Users input the attributes of their desired content using their device and provide sentiment data. This sentiment data is acquired through cameras and sensors and transmitted to the server in real time. The device also has a function to collect user feedback.

[0373] The server is equipped with emotion analysis capabilities to analyze the acquired information. Here, image recognition technology is used to determine emotions from facial expressions. To generate content tailored to users with specific emotions, the server uses generation capabilities to refer to past information and create new content. This process utilizes a generation AI model; for example, if a user requests an exciting story, high-energy content will be generated.

[0374] The generated content is visualized in storyboard format using visualization tools. Character expressions and scene tones are adjusted based on the user's emotions. This visual information is provided to the user via distribution tools and can be easily viewed and edited through their device.

[0375] User feedback, continuously provided, is analyzed using restructuring methods and used to improve content. This interactive cycle ensures that content is always optimized for the user.

[0376] For example, if a user requests an "exciting adventure story" and their emotion is determined to be "excitement," the server can send the prompt message "Generate an adventure story centered on excitement" to the AI ​​model, thereby creating content that meets the user's expectations. In this way, the present invention realizes user-centered content generation through the utilization of emotion data.

[0377] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0378] Step 1:

[0379] Users input desired content attributes, such as genre or theme, through their device. The device's camera and sensors also capture emotional data from facial expressions and voice. Input information includes user text instructions and emotional data from sensors. This information is then sent to a server for analysis.

[0380] Step 2:

[0381] The server receives text information and emotion data from the user. Here, using emotion analysis tools, image recognition technology identifies emotions such as "excitement," "joy," and "sadness" from facial expressions. The input data is classified into emotion categories during the analysis process and output as a dataset representing the user's emotional state.

[0382] Step 3:

[0383] The server generates new content using a generation method based on the analyzed emotional information. During this process, it compares the data with past databases and incorporates suitable story elements. A generation AI model is used to create prompts and send instructions to the AI. For example, an instruction such as "Generate an adventure story centered on excitement" might be used. The output is a personalized scenario tailored to the user's emotions.

[0384] Step 4:

[0385] The server converts the generated scenario into a storyboard format using visualization tools. Based on emotional information, character expressions and background tones are dynamically adjusted. The input is the generated scenario, and the output is visualized content display data.

[0386] Step 5:

[0387] The user receives visual content generated through the device. The device displays the visual content and provides a user interface for the user to review and edit the content. User feedback is also collected through the device and sent to the server.

[0388] Step 6:

[0389] The server analyzes user feedback using a reconstruction mechanism and makes revisions to the content and storyboards. The recalculated output, based on the input data obtained from the feedback, is improved new content. This continuously optimizes the content provided to the user.

[0390] (Application Example 2)

[0391] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0392] Providing personalized content that responds to user emotions is a crucial challenge in content delivery services. However, traditional methods make it difficult to analyze user emotions in real time and instantly generate content that responds to those emotions. This results in a problem where the user experience is not sufficiently improved.

[0393] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0394] In this invention, the server includes acquisition means for receiving content genre, theme, and related features from the user; sensor means for detecting the user's emotions and acquiring emotion data; and analysis means for analyzing the acquired information and comparing it with past data. This enables the dynamic adjustment of tone and style according to the user's emotions, and the provision of personalized content in real time.

[0395] "Acquisition means" refers to a mechanism for collecting content genre, theme, and related characteristics from users.

[0396] A "sensor device" is a device used to detect a user's emotions and collect that emotional data.

[0397] "Analysis means" refers to functions for analyzing acquired information and comparing it with past data sets.

[0398] "Generative means" refers to methods for creating new scripts or content based on analysis results.

[0399] "Creation means" refers to a device or software for constructing scene designs based on the generated script.

[0400] "Transmission means" refers to communication means for transferring generated content to the user.

[0401] A "correction mechanism" is a system for receiving user feedback and regenerating the script and scene design.

[0402] "Past information" refers to a database of content that has been collected or generated to date.

[0403] "Dynamically adjusting tone and style" means changing the style and expression of generated content in response to the user's emotional information.

[0404] "Interactive content" refers to content that can change its content in response to user input or emotions.

[0405] The system of this invention operates in conjunction with a user's terminal and a central server. The user's terminal is equipped with an interface for acquiring necessary information and a sensor for detecting emotions. The terminal first acquires the content genre, theme, and related features from the user. This information forms the basis for what kind of content the user desires.

[0406] Next, the device uses its camera and microphone to detect the user's emotions in real time and sends that data to a server. Specifically, it recognizes emotions by analyzing facial expressions from camera images and analyzing the tone of voice. For this process, software such as Amazon Rekognition can be used for image analysis, and Google Cloud Speech-to-Text can be used for voice analysis.

[0407] The server processes the received user request and emotional data using analytical tools, and generates new content that dynamically adjusts the tone and style to match the user's emotions, while referencing past data. This involves creating prompts using a generative AI model and forming the content scenario. For example, if the user wants to relax, the server will generate a prompt such as, "Please create a story that includes a relaxing scene."

[0408] The generated content is sent from the server to the user's device, allowing the user to experience the visualized content. If the user provides feedback, that data is sent back to the server and used in the next generation process. This feedback loop allows the content to become more personalized to the user.

[0409] The overall configuration of this system provides users with new experiences and enables innovative content delivery that resonates with their emotions.

[0410] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0411] Step 1:

[0412] The user enters the content genre, theme, and related characteristics into the device. The entered data is acquired through the device's input method and stored in memory as basic information about the user's preferences. After acquiring this data, the system is ready to proceed to sentiment detection.

[0413] Step 2:

[0414] The device uses its camera and microphone to detect the user's emotions. This involves facial recognition and voice tone analysis, as well as data processing to collect emotional data in real time. The emotional data is analyzed using specific algorithms (e.g., OpenCV or TensorFlow), and the analysis results are sent to a server.

[0415] Step 3:

[0416] The server processes the received user request information and sentiment data using analytical tools. In this step, it searches through past data sets and performs data calculations to extract relevant information. Based on the analysis results, it creates prompt sentences using a generative AI model and generates new content scenarios.

[0417] Step 4:

[0418] Through the generation mechanism, the server generates content based on prompt messages. For example, using the prompt message "Create a story that includes a relaxing landscape," a scenario with a tone and style that matches the user's emotions is generated. This generated scenario is then converted into data as visual content.

[0419] Step 5:

[0420] The server sends the generated content to the terminal. The terminal receives the transmitted data and displays it in a way that the user can visually access. The visualized content provides the user with a new experience.

[0421] Step 6:

[0422] Users input feedback about the content they experienced into their device. This feedback is sent back to the server, analyzed by a correction system, and incorporated into future content creation. Through this process, content is further personalized based on user feedback.

[0423] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0424] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0425] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0426] [Third Embodiment]

[0427] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0428] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0429] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0430] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0431] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0432] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0433] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0434] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0435] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0436] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0437] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0438] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0439] The system of this invention is designed to allow users to easily create new and original content. Through an interface that operates on various devices, users can input details about the genre, theme, and characters of their story. This allows users to indicate a specific direction for their content through the system.

[0440] When the server receives a request from a user, it uses natural language processing (NLP) to analyze the input information. This analysis helps the server understand the characteristics of the content the user is seeking and references it to a database of past content stored within the system. Based on the reference results, the system creates a new scenario using a specific generation algorithm.

[0441] The generated scenario is visualized as a related storyboard. Storyboard creation utilizes AI technology to automatically translate the scenario's content into a visual format. This allows users to obtain not only the written script but also storyboards that aid visual understanding.

[0442] The generated content is sent to the user's device via the internet. The user can view the content on their device and submit feedback as needed. This feedback is sent to the server, and the system then uses it to revise and update the content, creating a continuous cycle. This entire process allows users to interactively create content and obtain the final product they desire.

[0443] For example, if a user requests an "adventure story in a fantasy world," the server can refer to past works that fit that theme and generate a new scenario and imaginative visuals for the story. In this way, the present invention is a system that flexibly responds to the needs of a wide range of users, from professionals to amateurs, and contributes to the creation of creative content.

[0444] The following describes the processing flow.

[0445] Step 1:

[0446] The user inputs the genre, theme, character settings, and other details of the content they want to generate through the device's interface. This information is prepared as structured request data.

[0447] Step 2:

[0448] The terminal sends the request data received from the user to the server. This request contains all the necessary input information.

[0449] Step 3:

[0450] The server applies natural language processing (NLP) to analyze the received request data. This process extracts key information and requirements contained in the request.

[0451] Step 4:

[0452] Based on the analysis results, the server searches its internal database to refer to similar past content. This identifies patterns and elements of highly relevant content.

[0453] Step 5:

[0454] The server's AI model utilizes referenced past content information to generate new scenarios tailored to user requests. These scenarios include unique storylines and character plots.

[0455] Step 6:

[0456] The server automatically creates storyboards based on the generated scenario. A visual generation algorithm defines the visual representation for each scene.

[0457] Step 7:

[0458] The server then packages the final generated scenario and storyboards and prepares them for transmission to the terminal.

[0459] Step 8:

[0460] The terminal receives data sent from the server and displays the scenario and storyboards for easy viewing by the user. The user can then review this content.

[0461] Step 9:

[0462] Users provide feedback on the generated content and submit requests for corrections or improvements if necessary.

[0463] Step 10:

[0464] The server analyzes user feedback and modifies or regenerates the content. This process is repeated until the user is satisfied.

[0465] (Example 1)

[0466] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0467] Conventional content generation systems have a problem where users have to manually edit and adjust many things when creating new stories or visual content on their own themes, making the creative process very time-consuming and laborious. Furthermore, the process of incorporating feedback on the generated content is also complicated, making it difficult to respond quickly to user needs and expectations. This invention aims to solve these problems and realize a system that allows users to generate content easily and instantly and easily modify it as needed.

[0468] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0469] In this invention, the server includes input means for receiving content categories, themes, and related attributes from a user; analysis means for analyzing the received information using natural language processing technology and comparing it with past information data; and generation means for generating a new story using a generative AI model while referring to past information data. This enables users to easily create and visualize their own content.

[0470] "Input means" refers to a device or method for receiving content categories, subjects, and related attributes from a user.

[0471] "Analysis means" refers to a device or method for analyzing received information using natural language processing technology and comparing it with past information data.

[0472] A "generative AI model" is a model that uses artificial intelligence technology to generate new narratives while referencing past information data.

[0473] "Generative means" refers to a device or method for creating a new story using a generative AI model.

[0474] "Creation means" refers to a device or method for automatically creating visualized content based on a generated narrative.

[0475] "Transmission means" refers to a device or method for transmitting generated narrative and visualization content to a user.

[0476] "Modification means" refers to a device or method for receiving user evaluation information and regenerating narrative and visualization content.

[0477] "Search means" refers to a device or method for searching multiple historical data sets and identifying relevant data based on a user's request.

[0478] "Display means" refers to a device or method that provides a user interface for easily displaying and editing generated narratives and visualization content.

[0479] In this embodiment of the invention, the system consists of three main components: a user, a server, and a terminal. The user uses the terminal to access the system's interface and input details such as the content category, subject, and characters. This allows the user to specifically instruct the server on the direction of the content they desire. An example of a prompt might be, "Please generate an adventure story set in a fantasy world. The main characters are a brave knight and a wizard."

[0480] When the server receives a request from this user, it analyzes the input information using natural language processing (NLP) techniques. NLP, also known as a parsing engine, interprets the meaning of the input and compares it to historical data. In this process, the server refers to a database of previously collected and stored information to generate content best suited to the user's request.

[0481] Next, the server uses a generative AI model to generate a new story. This generative AI model incorporates the intended themes and characters from the prompt text, constructing original and novel content that aligns with the user's requests. This process utilizes advanced machine learning algorithms, aiming to ensure that the generated story exceeds user expectations.

[0482] Furthermore, based on the generated story, the server uses AI technology to create visualized content. This creation process allows users to obtain not only text content but also visual information. This enables users to perceive the story in a more three-dimensional way.

[0483] Finally, the server sends this generated story and visualized content to the user's device. The user can review the content via their device and provide feedback to the server as needed. The server receives this feedback and regenerates or modifies the content. Through this cycle, the user can gradually evolve their own creation towards its final form.

[0484] In this embodiment, technologies and processes are incorporated to quickly and efficiently fulfill the user's creative requirements, resulting in a system that can meet a wide range of needs and also contribute to improving the quality of content creation.

[0485] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0486] Step 1:

[0487] The user inputs information such as content category, subject, and character details through the terminal's interface. This input data is sent to the server in JSON or text format. At this stage, the terminal organizes the information according to the user's instructions and generates prompts. For example, the prompt might read, "Please create an adventure story in a fantasy world. The main characters are a brave knight and a wizard."

[0488] Step 2:

[0489] The server analyzes the prompt message received from the terminal using natural language processing techniques. This analysis involves keyword extraction and theme identification, and data processing to clearly understand the user's request. Based on the input prompt message, the server searches its historical information database to retrieve relevant data. The output is an analysis result of the detailed requirements for the content the user is requesting.

[0490] Step 3:

[0491] The server generates a new story using a generative AI model based on the analysis results. The generative AI model utilizes machine learning algorithms to construct a story suitable for the user's requested themes and characters. The analysis results are used as input, and the generation process outputs a creative and original story. This generated story includes elements such as the plot and character dialogue.

[0492] Step 4:

[0493] The server creates visualized content based on the generated story. In this step, AI technology is used to convert the story into storyboards or visual storyboards. The generated story is the input, and after the visualization process, visual information is obtained as output. Specifically, character illustrations and scene descriptions are automatically generated.

[0494] Step 5:

[0495] The server sends the final content (story and visualized content) to the user's device via the internet. The user can review the sent content on their device and provide feedback, including evaluations and modifications. At this stage, the device organizes the feedback information provided by the user and sends it back to the server.

[0496] Step 6:

[0497] The server receives user feedback and uses it to regenerate and modify story and visualization content. User feedback information is input, and data calculations and modifications are performed based on it. At this stage, the generated story and visual elements are re-evaluated as needed, and new output is created. This allows users to obtain more satisfying content.

[0498] (Application Example 1)

[0499] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0500] When users create original and novel content, specialized skills and knowledge are required, and generating and visualizing the content is difficult. Furthermore, the limited means of quickly distributing and sharing the generated content can potentially detract from the user experience.

[0501] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0502] In this invention, the server includes an input means for receiving content types and themes from a user, an analysis means for analyzing the received information and comparing it with existing information, a generation means for generating a new story while referring to past information resources, and a creation means for creating visual content based on the generated story. This enables users to easily generate their own content without requiring specialized knowledge and to quickly distribute and share it.

[0503] A "user" is an individual or group that uses the system to create original content.

[0504] "Content" refers to stories, visual materials, and other creative works created by users.

[0505] A "type" is a category or classification used to identify the nature or style of content.

[0506] "Subject matter" refers to the elements specified by the user as the theme or subject of the content.

[0507] "Attributes" refer to information about the detailed settings of characters and stories, and serve as the basis for content creation.

[0508] An "input method" is an interface that allows users to provide information such as categories, subjects, and attributes to the system.

[0509] "Analysis means" refers to a method for processing received information and evaluating its relationship to existing information.

[0510] A "generative means" is a mechanism for creating new narratives based on past information.

[0511] "Means of creation" refers to the means of visually representing the generated story, which involves forming a visual storyboard.

[0512] "Correction methods" refer to ways of updating the story and visual materials based on user feedback.

[0513] "Visual content" refers to visual deliverables created to be easily viewed and shared by users.

[0514] This invention is a system that automates the process of users creating their own content and visually representing it. Users input the content type, subject matter, and related attributes via a smartphone or computer. This input information is received by a server.

[0515] The server analyzes the received data using natural language processing (NLP) techniques (e.g., spaCy or GPT APIs). The analysis results are then compared to a database of past content. This identifies relevant information resources, and a generative AI model (e.g., OpenAI's GPT) is used to create a new narrative.

[0516] Next, the server creates a visual storyboard based on the generated narrative. This utilizes AI-driven visualization tools. The visual storyboard is designed to make the content easier for the user to understand.

[0517] Users can easily review the generated visual content through an interface on their smartphone or computer and submit feedback as needed. The server receives this feedback, restarts the generation process, and improves the content.

[0518] For example, if the user selects "A story about solving a mystery case at school," the server will generate a mystery-solving scenario using characters and settings in a style related to school stories. An example of a prompt to the generating AI model is "Genre: School Mystery, Theme: Detective Club and Mystery."

[0519] This entire process allows users to create high-quality, original content and share it quickly, without requiring specialized technical skills.

[0520] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0521] Step 1:

[0522] The terminal receives content type, subject matter, and attributes as input from the user. The input data is formatted into the required format and sent to the server. This prepares the server to correctly retrieve the information necessary for analysis.

[0523] Step 2:

[0524] The server receives the data as input and performs analysis using natural language processing (NLP) techniques. Specifically, it analyzes the input keywords and phrases to understand the user's intent. Next, it uses the analysis results to compare with existing databases and identifies relevant past content as output.

[0525] Step 3:

[0526] The server generates a new story using a generative AI model based on identified past content information. A prompt (e.g., "Genre: School Mystery, Theme: Detective Club and Mystery") is input to the generative AI model, which then outputs a story scenario. In this step, the AI ​​model utilizes past data to create a creative scenario.

[0527] Step 4:

[0528] The server creates a visual storyboard based on the outputted narrative. At this stage, an AI-driven visualization tool is used to visually represent the scenes of the story. The visual storyboard is output as images and illustrations that the user can intuitively understand.

[0529] Step 5:

[0530] Users receive visual content generated through their device as input and review it. This interface allows users to input and submit feedback as needed. This feedback is sent to the server and serves as input for further improvement.

[0531] Step 6:

[0532] The server receives user feedback as input and regenerates narratives and visual content. Specifically, it analyzes the feedback, updates the prompt text, and inputs it back into the generation AI model to output an improved scenario. This cycle results in content that meets the user's expectations.

[0533] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0534] This invention relates to a system that recognizes user emotions and generates content that reflects those emotions. This system personalizes the generated content by incorporating user emotions as an intervention element.

[0535] Users can use an interface on their device to input the genre, theme, and character details of the content they desire, as well as provide feedback on the situation, including emotions. The user's device is equipped with a camera and sensors, and the data collected through these is sent to a server for analysis by an emotion engine.

[0536] The server analyzes the input data and uses an emotion engine to determine the user's emotions. Based on this information, it references relevant information from a past content database and generates a new script. The tone and style of the generated content are dynamically adjusted according to the user's emotions, allowing users to access content that resonates more deeply with them.

[0537] The generated script is visualized as a storyboard using AI. In this process as well, emotional information is reflected in the scene composition and character expressions, enabling more emotionally responsive depictions.

[0538] For example, if a user requests an "exciting adventure story" and their emotion is recognized as "excitement," the server can refer to past works that reflect similar emotions and generate a scenario with a higher level of tension. In this way, by utilizing emotional information, content creation that is more responsive to user requests can be achieved.

[0539] In this system, users can continuously send feedback that includes emotional data, which the server analyzes with an emotion engine to improve the script and storyboards. This makes the creation process more interactive and dynamic, rather than merely a one-way street.

[0540] The following describes the processing flow.

[0541] Step 1:

[0542] Users input content genre, theme, character details, and their current emotional state from their device. The emotional state is recognized by monitoring the user's facial expressions, heart rate, and other data acquired through the device's camera and sensors.

[0543] Step 2:

[0544] The device sends the user's portrait data and physiological data along with the entered information to the server. This data is necessary for the emotion engine's analysis.

[0545] Step 3:

[0546] The server analyzes the received information, using natural language processing (NLP) to decode the text information while simultaneously activating an emotion engine to analyze emotional data. This analysis identifies the user's current emotions.

[0547] Step 4:

[0548] Based on the analyzed information, the server searches its internal database for similar past content. Based on the search results, it collects story elements that match the user's desired genre and theme.

[0549] Step 5:

[0550] The server's AI model generates new scenarios using reference results and user sentiment data. The tone and plot points of the script are adjusted to match the sentiment data.

[0551] Step 6:

[0552] Based on the generated script, the server creates storyboards. At this time, the character's facial expressions and the atmosphere of the scene are set according to the situation, reflecting the results of the emotion engine.

[0553] Step 7:

[0554] The server sends the generated script and storyboards to the user's terminal and provides visuals and text for the user to review the results.

[0555] Step 8:

[0556] Users can review the displayed script and storyboards and send feedback from their device if necessary. This feedback can also include additional emotional data.

[0557] Step 9:

[0558] The server re-analyzes the received feedback and revises the script and storyboards. This allows for content improvements that address new emotions and requests.

[0559] (Example 2)

[0560] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0561] There is a need to generate content that accurately reflects user emotions, enhance content personalization, and enable interactive communication with users. However, conventional systems have been unable to fully utilize user emotions, resulting in uniform content, which has been a problem.

[0562] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0563] In this invention, the server includes acquisition means for acquiring diverse content attributes and emotion-related information from the user, emotion analysis means for analyzing the acquired information and determining the user's emotions, and generation means for creating new content by referring to past information based on the determined emotion information. This makes it possible to generate personalized content that responds to the user's emotions.

[0564] "Acquisition methods" refer to mechanisms for collecting content attributes and sentiment-related information from users.

[0565] An "emotion analysis tool" is a system that analyzes acquired information to determine the user's emotions.

[0566] "Generation method" refers to the process of creating new content by referencing past information based on identified emotional information.

[0567] "Visualization methods" refer to technologies for representing generated scripts or content as visual information.

[0568] "Distribution method" refers to the method used to deliver generated content to users.

[0569] A "reconstruction method" is a function that reshapes and improves visual information based on user feedback.

[0570] "Search methods" refer to techniques that utilize sentiment data to efficiently search past information and identify relevant information.

[0571] A "display means" is an interface that presents generated visual information to the user and enables editing.

[0572] This invention is a system for generating personalized content that reflects the user's emotions. The embodiments thereof are described below.

[0573] Users input the attributes of their desired content using their device and provide sentiment data. This sentiment data is acquired through cameras and sensors and transmitted to the server in real time. The device also has a function to collect user feedback.

[0574] The server is equipped with emotion analysis capabilities to analyze the acquired information. Here, image recognition technology is used to determine emotions from facial expressions. To generate content tailored to users with specific emotions, the server uses generation capabilities to refer to past information and create new content. This process utilizes a generation AI model; for example, if a user requests an exciting story, high-energy content will be generated.

[0575] The generated content is visualized in storyboard format using visualization tools. Character expressions and scene tones are adjusted based on the user's emotions. This visual information is provided to the user via distribution tools and can be easily viewed and edited through their device.

[0576] User feedback, continuously provided, is analyzed using restructuring methods and used to improve content. This interactive cycle ensures that content is always optimized for the user.

[0577] For example, if a user requests an "exciting adventure story" and their emotion is determined to be "excitement," the server can send the prompt message "Generate an adventure story centered on excitement" to the AI ​​model, thereby creating content that meets the user's expectations. In this way, the present invention realizes user-centered content generation through the utilization of emotion data.

[0578] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0579] Step 1:

[0580] Users input desired content attributes, such as genre or theme, through their device. The device's camera and sensors also capture emotional data from facial expressions and voice. Input information includes user text instructions and emotional data from sensors. This information is then sent to a server for analysis.

[0581] Step 2:

[0582] The server receives text information and emotion data from the user. Here, using emotion analysis tools, image recognition technology identifies emotions such as "excitement," "joy," and "sadness" from facial expressions. The input data is classified into emotion categories during the analysis process and output as a dataset representing the user's emotional state.

[0583] Step 3:

[0584] The server generates new content using a generation method based on the analyzed emotional information. During this process, it compares the data with past databases and incorporates suitable story elements. A generation AI model is used to create prompts and send instructions to the AI. For example, an instruction such as "Generate an adventure story centered on excitement" might be used. The output is a personalized scenario tailored to the user's emotions.

[0585] Step 4:

[0586] The server converts the generated scenario into a storyboard format using visualization tools. Based on emotional information, character expressions and background tones are dynamically adjusted. The input is the generated scenario, and the output is visualized content display data.

[0587] Step 5:

[0588] The user receives visual content generated through the device. The device displays the visual content and provides a user interface for the user to review and edit the content. User feedback is also collected through the device and sent to the server.

[0589] Step 6:

[0590] The server analyzes user feedback using a reconstruction mechanism and makes revisions to the content and storyboards. The recalculated output, based on the input data obtained from the feedback, is improved new content. This continuously optimizes the content provided to the user.

[0591] (Application Example 2)

[0592] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0593] Providing personalized content that responds to user emotions is a crucial challenge in content delivery services. However, traditional methods make it difficult to analyze user emotions in real time and instantly generate content that responds to those emotions. This results in a problem where the user experience is not sufficiently improved.

[0594] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0595] In this invention, the server includes acquisition means for receiving content genre, theme, and related features from the user; sensor means for detecting the user's emotions and acquiring emotion data; and analysis means for analyzing the acquired information and comparing it with past data. This enables the dynamic adjustment of tone and style according to the user's emotions, and the provision of personalized content in real time.

[0596] "Acquisition means" refers to a mechanism for collecting content genre, theme, and related characteristics from users.

[0597] A "sensor device" is a device used to detect a user's emotions and collect that emotional data.

[0598] "Analysis means" refers to functions for analyzing acquired information and comparing it with past data sets.

[0599] "Generative means" refers to methods for creating new scripts or content based on analysis results.

[0600] "Creation means" refers to a device or software for constructing scene designs based on the generated script.

[0601] "Transmission means" refers to communication means for transferring generated content to the user.

[0602] A "correction mechanism" is a system for receiving user feedback and regenerating the script and scene design.

[0603] "Past information" refers to a database of content that has been collected or generated to date.

[0604] "Dynamically adjusting tone and style" means changing the style and expression of generated content in response to the user's emotional information.

[0605] "Interactive content" refers to content that can change its content in response to user input or emotions.

[0606] The system of this invention operates in conjunction with a user's terminal and a central server. The user's terminal is equipped with an interface for acquiring necessary information and a sensor for detecting emotions. The terminal first acquires the content genre, theme, and related features from the user. This information forms the basis for what kind of content the user desires.

[0607] Next, the device uses its camera and microphone to detect the user's emotions in real time and sends that data to a server. Specifically, it recognizes emotions by analyzing facial expressions from camera images and analyzing the tone of voice. For this process, software such as Amazon Rekognition can be used for image analysis, and Google Cloud Speech-to-Text can be used for voice analysis.

[0608] The server processes the received user request and emotional data using analytical tools, and generates new content that dynamically adjusts the tone and style to match the user's emotions, while referencing past data. This involves creating prompts using a generative AI model and forming the content scenario. For example, if the user wants to relax, the server will generate a prompt such as, "Please create a story that includes a relaxing scene."

[0609] The generated content is sent from the server to the user's device, allowing the user to experience the visualized content. If the user provides feedback, that data is sent back to the server and used in the next generation process. This feedback loop allows the content to become more personalized to the user.

[0610] The overall configuration of this system provides users with new experiences and enables innovative content delivery that resonates with their emotions.

[0611] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0612] Step 1:

[0613] The user enters the content genre, theme, and related characteristics into the device. The entered data is acquired through the device's input method and stored in memory as basic information about the user's preferences. After acquiring this data, the system is ready to proceed to sentiment detection.

[0614] Step 2:

[0615] The device uses its camera and microphone to detect the user's emotions. This involves facial recognition and voice tone analysis, as well as data processing to collect emotional data in real time. The emotional data is analyzed using specific algorithms (e.g., OpenCV or TensorFlow), and the analysis results are sent to a server.

[0616] Step 3:

[0617] The server processes the received user request information and sentiment data using analytical tools. In this step, it searches through past data sets and performs data calculations to extract relevant information. Based on the analysis results, it creates prompt sentences using a generative AI model and generates new content scenarios.

[0618] Step 4:

[0619] Through the generation mechanism, the server generates content based on prompt messages. For example, using the prompt message "Create a story that includes a relaxing landscape," a scenario with a tone and style that matches the user's emotions is generated. This generated scenario is then converted into data as visual content.

[0620] Step 5:

[0621] The server sends the generated content to the terminal. The terminal receives the transmitted data and displays it in a way that the user can visually access. The visualized content provides the user with a new experience.

[0622] Step 6:

[0623] Users input feedback about the content they experienced into their device. This feedback is sent back to the server, analyzed by a correction system, and incorporated into future content creation. Through this process, content is further personalized based on user feedback.

[0624] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0625] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0626] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0627] [Fourth Embodiment]

[0628] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0629] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0630] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0631] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0632] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0633] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0634] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0635] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0636] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0637] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0638] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0639] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0640] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0641] The system of this invention is designed to allow users to easily create new and original content. Through an interface that operates on various devices, users can input details about the genre, theme, and characters of their story. This allows users to indicate a specific direction for their content through the system.

[0642] When the server receives a request from a user, it uses natural language processing (NLP) to analyze the input information. This analysis helps the server understand the characteristics of the content the user is seeking and references it to a database of past content stored within the system. Based on the reference results, the system creates a new scenario using a specific generation algorithm.

[0643] The generated scenario is visualized as a related storyboard. Storyboard creation utilizes AI technology to automatically translate the scenario's content into a visual format. This allows users to obtain not only the written script but also storyboards that aid visual understanding.

[0644] The generated content is sent to the user's device via the internet. The user can view the content on their device and submit feedback as needed. This feedback is sent to the server, and the system then uses it to revise and update the content, creating a continuous cycle. This entire process allows users to interactively create content and obtain the final product they desire.

[0645] For example, if a user requests an "adventure story in a fantasy world," the server can refer to past works that fit that theme and generate a new scenario and imaginative visuals for the story. In this way, the present invention is a system that flexibly responds to the needs of a wide range of users, from professionals to amateurs, and contributes to the creation of creative content.

[0646] The following describes the processing flow.

[0647] Step 1:

[0648] The user inputs the genre, theme, character settings, and other details of the content they want to generate through the device's interface. This information is prepared as structured request data.

[0649] Step 2:

[0650] The terminal sends the request data received from the user to the server. This request contains all the necessary input information.

[0651] Step 3:

[0652] The server applies natural language processing (NLP) to analyze the received request data. This process extracts key information and requirements contained in the request.

[0653] Step 4:

[0654] Based on the analysis results, the server searches its internal database to refer to similar past content. This identifies patterns and elements of highly relevant content.

[0655] Step 5:

[0656] The server's AI model utilizes referenced past content information to generate new scenarios tailored to user requests. These scenarios include unique storylines and character plots.

[0657] Step 6:

[0658] The server automatically creates storyboards based on the generated scenario. A visual generation algorithm defines the visual representation for each scene.

[0659] Step 7:

[0660] The server then packages the final generated scenario and storyboards and prepares them for transmission to the terminal.

[0661] Step 8:

[0662] The terminal receives data sent from the server and displays the scenario and storyboards for easy viewing by the user. The user can then review this content.

[0663] Step 9:

[0664] Users provide feedback on the generated content and submit requests for corrections or improvements if necessary.

[0665] Step 10:

[0666] The server analyzes user feedback and modifies or regenerates the content. This process is repeated until the user is satisfied.

[0667] (Example 1)

[0668] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0669] Conventional content generation systems have a problem where users have to manually edit and adjust many things when creating new stories or visual content on their own themes, making the creative process very time-consuming and laborious. Furthermore, the process of incorporating feedback on the generated content is also complicated, making it difficult to respond quickly to user needs and expectations. This invention aims to solve these problems and realize a system that allows users to generate content easily and instantly and easily modify it as needed.

[0670] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0671] In this invention, the server includes input means for receiving content categories, themes, and related attributes from a user; analysis means for analyzing the received information using natural language processing technology and comparing it with past information data; and generation means for generating a new story using a generative AI model while referring to past information data. This enables users to easily create and visualize their own content.

[0672] "Input means" refers to a device or method for receiving content categories, subjects, and related attributes from a user.

[0673] "Analysis means" refers to a device or method for analyzing received information using natural language processing technology and comparing it with past information data.

[0674] A "generative AI model" is a model that uses artificial intelligence technology to generate new narratives while referencing past information data.

[0675] "Generative means" refers to a device or method for creating a new story using a generative AI model.

[0676] "Creation means" refers to a device or method for automatically creating visualized content based on a generated narrative.

[0677] "Transmission means" refers to a device or method for transmitting generated narrative and visualization content to a user.

[0678] "Modification means" refers to a device or method for receiving user evaluation information and regenerating narrative and visualization content.

[0679] "Search means" refers to a device or method for searching multiple historical data sets and identifying relevant data based on a user's request.

[0680] "Display means" refers to a device or method that provides a user interface for easily displaying and editing generated narratives and visualization content.

[0681] In this embodiment of the invention, the system consists of three main components: a user, a server, and a terminal. The user uses the terminal to access the system's interface and input details such as the content category, subject, and characters. This allows the user to specifically instruct the server on the direction of the content they desire. An example of a prompt might be, "Please generate an adventure story set in a fantasy world. The main characters are a brave knight and a wizard."

[0682] When the server receives a request from this user, it analyzes the input information using natural language processing (NLP) techniques. NLP, also known as a parsing engine, interprets the meaning of the input and compares it to historical data. In this process, the server refers to a database of previously collected and stored information to generate content best suited to the user's request.

[0683] Next, the server uses a generative AI model to generate a new story. This generative AI model incorporates the intended themes and characters from the prompt text, constructing original and novel content that aligns with the user's requests. This process utilizes advanced machine learning algorithms, aiming to ensure that the generated story exceeds user expectations.

[0684] Furthermore, based on the generated story, the server uses AI technology to create visualized content. This creation process allows users to obtain not only text content but also visual information. This enables users to perceive the story in a more three-dimensional way.

[0685] Finally, the server sends this generated story and visualized content to the user's device. The user can review the content via their device and provide feedback to the server as needed. The server receives this feedback and regenerates or modifies the content. Through this cycle, the user can gradually evolve their own creation towards its final form.

[0686] In this embodiment, technologies and processes are incorporated to quickly and efficiently fulfill the user's creative requirements, resulting in a system that can meet a wide range of needs and also contribute to improving the quality of content creation.

[0687] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0688] Step 1:

[0689] The user inputs information such as content category, subject, and character details through the terminal's interface. This input data is sent to the server in JSON or text format. At this stage, the terminal organizes the information according to the user's instructions and generates prompts. For example, the prompt might read, "Please create an adventure story in a fantasy world. The main characters are a brave knight and a wizard."

[0690] Step 2:

[0691] The server analyzes the prompt message received from the terminal using natural language processing techniques. This analysis involves keyword extraction and theme identification, and data processing to clearly understand the user's request. Based on the input prompt message, the server searches its historical information database to retrieve relevant data. The output is an analysis result of the detailed requirements for the content the user is requesting.

[0692] Step 3:

[0693] The server generates a new story using a generative AI model based on the analysis results. The generative AI model utilizes machine learning algorithms to construct a story suitable for the user's requested themes and characters. The analysis results are used as input, and the generation process outputs a creative and original story. This generated story includes elements such as the plot and character dialogue.

[0694] Step 4:

[0695] The server creates visualized content based on the generated story. In this step, AI technology is used to convert the story into storyboards or visual storyboards. The generated story is the input, and after the visualization process, visual information is obtained as output. Specifically, character illustrations and scene descriptions are automatically generated.

[0696] Step 5:

[0697] The server sends the final content (story and visualized content) to the user's device via the internet. The user can review the sent content on their device and provide feedback, including evaluations and modifications. At this stage, the device organizes the feedback information provided by the user and sends it back to the server.

[0698] Step 6:

[0699] The server receives user feedback and uses it to regenerate and modify story and visualization content. User feedback information is input, and data calculations and modifications are performed based on it. At this stage, the generated story and visual elements are re-evaluated as needed, and new output is created. This allows users to obtain more satisfying content.

[0700] (Application Example 1)

[0701] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0702] When users create original and novel content, specialized skills and knowledge are required, and generating and visualizing the content is difficult. Furthermore, the limited means of quickly distributing and sharing the generated content can potentially detract from the user experience.

[0703] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0704] In this invention, the server includes an input means for receiving content types and themes from a user, an analysis means for analyzing the received information and comparing it with existing information, a generation means for generating a new story while referring to past information resources, and a creation means for creating visual content based on the generated story. This enables users to easily generate their own content without requiring specialized knowledge and to quickly distribute and share it.

[0705] A "user" is an individual or group that uses the system to create original content.

[0706] "Content" refers to stories, visual materials, and other creative works created by users.

[0707] A "type" is a category or classification used to identify the nature or style of content.

[0708] "Subject matter" refers to the elements specified by the user as the theme or subject of the content.

[0709] "Attributes" refer to information about the detailed settings of characters and stories, and serve as the basis for content creation.

[0710] An "input method" is an interface that allows users to provide information such as categories, subjects, and attributes to the system.

[0711] "Analysis means" refers to a method for processing received information and evaluating its relationship to existing information.

[0712] A "generative means" is a mechanism for creating new narratives based on past information.

[0713] "Means of creation" refers to the means of visually representing the generated story, which involves forming a visual storyboard.

[0714] "Correction methods" refer to ways of updating the story and visual materials based on user feedback.

[0715] "Visual content" refers to visual deliverables created to be easily viewed and shared by users.

[0716] This invention is a system that automates the process of users creating their own content and visually representing it. Users input the content type, subject matter, and related attributes via a smartphone or computer. This input information is received by a server.

[0717] The server analyzes the received data using natural language processing (NLP) techniques (e.g., spaCy or GPT APIs). The analysis results are then compared to a database of past content. This identifies relevant information resources, and a generative AI model (e.g., OpenAI's GPT) is used to create a new narrative.

[0718] Next, the server creates a visual storyboard based on the generated narrative. This utilizes AI-driven visualization tools. The visual storyboard is designed to make the content easier for the user to understand.

[0719] Users can easily review the generated visual content through an interface on their smartphone or computer and submit feedback as needed. The server receives this feedback, restarts the generation process, and improves the content.

[0720] For example, if the user selects "A story about solving a mystery case at school," the server will generate a mystery-solving scenario using characters and settings in a style related to school stories. An example of a prompt to the generating AI model is "Genre: School Mystery, Theme: Detective Club and Mystery."

[0721] This entire process allows users to create high-quality, original content and share it quickly, without requiring specialized technical skills.

[0722] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0723] Step 1:

[0724] The terminal receives content type, subject matter, and attributes as input from the user. The input data is formatted into the required format and sent to the server. This prepares the server to correctly retrieve the information necessary for analysis.

[0725] Step 2:

[0726] The server receives the data as input and performs analysis using natural language processing (NLP) techniques. Specifically, it analyzes the input keywords and phrases to understand the user's intent. Next, it uses the analysis results to compare with existing databases and identifies relevant past content as output.

[0727] Step 3:

[0728] The server generates a new story using a generative AI model based on identified past content information. A prompt (e.g., "Genre: School Mystery, Theme: Detective Club and Mystery") is input to the generative AI model, which then outputs a story scenario. In this step, the AI ​​model utilizes past data to create a creative scenario.

[0729] Step 4:

[0730] The server creates a visual storyboard based on the outputted narrative. At this stage, an AI-driven visualization tool is used to visually represent the scenes of the story. The visual storyboard is output as images and illustrations that the user can intuitively understand.

[0731] Step 5:

[0732] Users receive visual content generated through their device as input and review it. This interface allows users to input and submit feedback as needed. This feedback is sent to the server and serves as input for further improvement.

[0733] Step 6:

[0734] The server receives user feedback as input and regenerates narratives and visual content. Specifically, it analyzes the feedback, updates the prompt text, and inputs it back into the generation AI model to output an improved scenario. This cycle results in content that meets the user's expectations.

[0735] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0736] This invention relates to a system that recognizes user emotions and generates content that reflects those emotions. This system personalizes the generated content by incorporating user emotions as an intervention element.

[0737] Users can use an interface on their device to input the genre, theme, and character details of the content they desire, as well as provide feedback on the situation, including emotions. The user's device is equipped with a camera and sensors, and the data collected through these is sent to a server for analysis by an emotion engine.

[0738] The server analyzes the input data and uses an emotion engine to determine the user's emotions. Based on this information, it references relevant information from a past content database and generates a new script. The tone and style of the generated content are dynamically adjusted according to the user's emotions, allowing users to access content that resonates more deeply with them.

[0739] The generated script is visualized as a storyboard using AI. In this process as well, emotional information is reflected in the scene composition and character expressions, enabling more emotionally responsive depictions.

[0740] For example, if a user requests an "exciting adventure story" and their emotion is recognized as "excitement," the server can refer to past works that reflect similar emotions and generate a scenario with a higher level of tension. In this way, by utilizing emotional information, content creation that is more responsive to user requests can be achieved.

[0741] In this system, users can continuously send feedback that includes emotional data, which the server analyzes with an emotion engine to improve the script and storyboards. This makes the creation process more interactive and dynamic, rather than merely a one-way street.

[0742] The following describes the processing flow.

[0743] Step 1:

[0744] Users input content genre, theme, character details, and their current emotional state from their device. The emotional state is recognized by monitoring the user's facial expressions, heart rate, and other data acquired through the device's camera and sensors.

[0745] Step 2:

[0746] The device sends the user's portrait data and physiological data along with the entered information to the server. This data is necessary for the emotion engine's analysis.

[0747] Step 3:

[0748] The server analyzes the received information, using natural language processing (NLP) to decode the text information while simultaneously activating an emotion engine to analyze emotional data. This analysis identifies the user's current emotions.

[0749] Step 4:

[0750] Based on the analyzed information, the server searches its internal database for similar past content. Based on the search results, it collects story elements that match the user's desired genre and theme.

[0751] Step 5:

[0752] The server's AI model generates new scenarios using reference results and user sentiment data. The tone and plot points of the script are adjusted to match the sentiment data.

[0753] Step 6:

[0754] Based on the generated script, the server creates storyboards. At this time, the character's facial expressions and the atmosphere of the scene are set according to the situation, reflecting the results of the emotion engine.

[0755] Step 7:

[0756] The server sends the generated script and storyboards to the user's terminal and provides visuals and text for the user to review the results.

[0757] Step 8:

[0758] Users can review the displayed script and storyboards and send feedback from their device if necessary. This feedback can also include additional emotional data.

[0759] Step 9:

[0760] The server re-analyzes the received feedback and revises the script and storyboards. This allows for content improvements that address new emotions and requests.

[0761] (Example 2)

[0762] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0763] There is a need to generate content that accurately reflects user emotions, enhance content personalization, and enable interactive communication with users. However, conventional systems have been unable to fully utilize user emotions, resulting in uniform content, which has been a problem.

[0764] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0765] In this invention, the server includes acquisition means for acquiring diverse content attributes and emotion-related information from the user, emotion analysis means for analyzing the acquired information and determining the user's emotions, and generation means for creating new content by referring to past information based on the determined emotion information. This makes it possible to generate personalized content that responds to the user's emotions.

[0766] "Acquisition methods" refer to mechanisms for collecting content attributes and sentiment-related information from users.

[0767] An "emotion analysis tool" is a system that analyzes acquired information to determine the user's emotions.

[0768] "Generation method" refers to the process of creating new content by referencing past information based on identified emotional information.

[0769] "Visualization methods" refer to technologies for representing generated scripts or content as visual information.

[0770] "Distribution method" refers to the method used to deliver generated content to users.

[0771] A "reconstruction method" is a function that reshapes and improves visual information based on user feedback.

[0772] "Search methods" refer to techniques that utilize sentiment data to efficiently search past information and identify relevant information.

[0773] A "display means" is an interface that presents generated visual information to the user and enables editing.

[0774] This invention is a system for generating personalized content that reflects the user's emotions. The embodiments thereof are described below.

[0775] Users input the attributes of their desired content using their device and provide sentiment data. This sentiment data is acquired through cameras and sensors and transmitted to the server in real time. The device also has a function to collect user feedback.

[0776] The server is equipped with emotion analysis capabilities to analyze the acquired information. Here, image recognition technology is used to determine emotions from facial expressions. To generate content tailored to users with specific emotions, the server uses generation capabilities to refer to past information and create new content. This process utilizes a generation AI model; for example, if a user requests an exciting story, high-energy content will be generated.

[0777] The generated content is visualized in storyboard format using visualization tools. Character expressions and scene tones are adjusted based on the user's emotions. This visual information is provided to the user via distribution tools and can be easily viewed and edited through their device.

[0778] User feedback, continuously provided, is analyzed using restructuring methods and used to improve content. This interactive cycle ensures that content is always optimized for the user.

[0779] For example, if a user requests an "exciting adventure story" and their emotion is determined to be "excitement," the server can send the prompt message "Generate an adventure story centered on excitement" to the AI ​​model, thereby creating content that meets the user's expectations. In this way, the present invention realizes user-centered content generation through the utilization of emotion data.

[0780] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0781] Step 1:

[0782] Users input desired content attributes, such as genre or theme, through their device. The device's camera and sensors also capture emotional data from facial expressions and voice. Input information includes user text instructions and emotional data from sensors. This information is then sent to a server for analysis.

[0783] Step 2:

[0784] The server receives text information and emotion data from the user. Here, using emotion analysis tools, image recognition technology identifies emotions such as "excitement," "joy," and "sadness" from facial expressions. The input data is classified into emotion categories during the analysis process and output as a dataset representing the user's emotional state.

[0785] Step 3:

[0786] The server generates new content using a generation method based on the analyzed emotional information. During this process, it compares the data with past databases and incorporates suitable story elements. A generation AI model is used to create prompts and send instructions to the AI. For example, an instruction such as "Generate an adventure story centered on excitement" might be used. The output is a personalized scenario tailored to the user's emotions.

[0787] Step 4:

[0788] The server converts the generated scenario into a storyboard format using visualization tools. Based on emotional information, character expressions and background tones are dynamically adjusted. The input is the generated scenario, and the output is visualized content display data.

[0789] Step 5:

[0790] The user receives visual content generated through the device. The device displays the visual content and provides a user interface for the user to review and edit the content. User feedback is also collected through the device and sent to the server.

[0791] Step 6:

[0792] The server analyzes user feedback using a reconstruction mechanism and makes revisions to the content and storyboards. The recalculated output, based on the input data obtained from the feedback, is improved new content. This continuously optimizes the content provided to the user.

[0793] (Application Example 2)

[0794] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0795] Providing personalized content that responds to user emotions is a crucial challenge in content delivery services. However, traditional methods make it difficult to analyze user emotions in real time and instantly generate content that responds to those emotions. This results in a problem where the user experience is not sufficiently improved.

[0796] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0797] In this invention, the server includes acquisition means for receiving content genre, theme, and related features from the user; sensor means for detecting the user's emotions and acquiring emotion data; and analysis means for analyzing the acquired information and comparing it with past data. This enables the dynamic adjustment of tone and style according to the user's emotions, and the provision of personalized content in real time.

[0798] "Acquisition means" refers to a mechanism for collecting content genre, theme, and related characteristics from users.

[0799] A "sensor device" is a device used to detect a user's emotions and collect that emotional data.

[0800] "Analysis means" refers to functions for analyzing acquired information and comparing it with past data sets.

[0801] "Generative means" refers to methods for creating new scripts or content based on analysis results.

[0802] "Creation means" refers to a device or software for constructing scene designs based on the generated script.

[0803] "Transmission means" refers to communication means for transferring generated content to the user.

[0804] A "correction mechanism" is a system for receiving user feedback and regenerating the script and scene design.

[0805] "Past information" refers to a database of content that has been collected or generated to date.

[0806] "Dynamically adjusting tone and style" means changing the style and expression of generated content in response to the user's emotional information.

[0807] "Interactive content" refers to content that can change its content in response to user input or emotions.

[0808] The system of this invention operates in conjunction with a user's terminal and a central server. The user's terminal is equipped with an interface for acquiring necessary information and a sensor for detecting emotions. The terminal first acquires the content genre, theme, and related features from the user. This information forms the basis for what kind of content the user desires.

[0809] Next, the device uses its camera and microphone to detect the user's emotions in real time and sends that data to a server. Specifically, it recognizes emotions by analyzing facial expressions from camera images and analyzing the tone of voice. For this process, software such as Amazon Rekognition can be used for image analysis, and Google Cloud Speech-to-Text can be used for voice analysis.

[0810] The server processes the received user request and emotional data using analytical tools, and generates new content that dynamically adjusts the tone and style to match the user's emotions, while referencing past data. This involves creating prompts using a generative AI model and forming the content scenario. For example, if the user wants to relax, the server will generate a prompt such as, "Please create a story that includes a relaxing scene."

[0811] The generated content is sent from the server to the user's device, allowing the user to experience the visualized content. If the user provides feedback, that data is sent back to the server and used in the next generation process. This feedback loop allows the content to become more personalized to the user.

[0812] The overall configuration of this system provides users with new experiences and enables innovative content delivery that resonates with their emotions.

[0813] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0814] Step 1:

[0815] The user enters the content genre, theme, and related characteristics into the device. The entered data is acquired through the device's input method and stored in memory as basic information about the user's preferences. After acquiring this data, the system is ready to proceed to sentiment detection.

[0816] Step 2:

[0817] The device uses its camera and microphone to detect the user's emotions. This involves facial recognition and voice tone analysis, as well as data processing to collect emotional data in real time. The emotional data is analyzed using specific algorithms (e.g., OpenCV or TensorFlow), and the analysis results are sent to a server.

[0818] Step 3:

[0819] The server processes the received user request information and sentiment data using analytical tools. In this step, it searches through past data sets and performs data calculations to extract relevant information. Based on the analysis results, it creates prompt sentences using a generative AI model and generates new content scenarios.

[0820] Step 4:

[0821] Through the generation mechanism, the server generates content based on prompt messages. For example, using the prompt message "Create a story that includes a relaxing landscape," a scenario with a tone and style that matches the user's emotions is generated. This generated scenario is then converted into data as visual content.

[0822] Step 5:

[0823] The server sends the generated content to the terminal. The terminal receives the transmitted data and displays it in a way that the user can visually access. The visualized content provides the user with a new experience.

[0824] Step 6:

[0825] Users input feedback about the content they experienced into their device. This feedback is sent back to the server, analyzed by a correction system, and incorporated into future content creation. Through this process, content is further personalized based on user feedback.

[0826] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0827] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0828] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0829] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0830] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0831] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0832] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0833] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0834] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0835] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0836] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0837] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0838] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0839] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0840] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0841] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0842] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0843] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0844] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0845] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0846] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0847] The following is further disclosed regarding the embodiments described above.

[0848] (Claim 1)

[0849] An input means for receiving the genre, theme, and related characteristics of content from the user,

[0850] An analysis method that analyzes received information and compares it with past content,

[0851] A generation method for generating new scripts while referencing past content,

[0852] A means of creating storyboards based on the generated script,

[0853] A means of sending the generated script and storyboard to the user,

[0854] A means of regenerating the script and storyboards based on user feedback,

[0855] A system that includes this.

[0856] (Claim 2)

[0857] The system according to claim 1, further comprising a search means for searching multiple past content information based on a user's request and identifying relevant information.

[0858] (Claim 3)

[0859] The system according to claim 1, further comprising display means that provide an interface for easily displaying and editing generated scripts and storyboards.

[0860] "Example 1"

[0861] (Claim 1)

[0862] An input means for receiving content categories, subjects, and related attributes from the user,

[0863] An analysis means that analyzes the received information using natural language processing technology and compares it with past information data,

[0864] A generation method that generates new stories using a generative AI model while referring to past information data,

[0865] A means of creating visualized content based on a generated story,

[0866] A means of sending generated stories and visualization content to the user,

[0867] A means of modification that receives user evaluation information and regenerates narrative and visualization content,

[0868] A system that includes this.

[0869] (Claim 2)

[0870] The system according to claim 1, further comprising a search means for searching multiple historical data sets and identifying related data based on user requests.

[0871] (Claim 3)

[0872] The system according to claim 1, further comprising display means that provide a user interface for easily displaying and editing generated narratives and visualization content.

[0873] "Application Example 1"

[0874] (Claim 1)

[0875] An input means for receiving content type, subject matter, and related attributes from the user,

[0876] An analysis means that analyzes the received information and compares it with existing content,

[0877] A means of generating new narratives while referencing past information resources,

[0878] A means of creating a visual storyboard based on a generated narrative,

[0879] A means of sending generated stories and visual storyboards to the user,

[0880] A means of regenerating the story and visual storyboards based on feedback from users,

[0881] A means for easily viewing and sharing visual content generated on a user's device,

[0882] A system that includes this.

[0883] (Claim 2)

[0884] The system according to claim 1, further comprising a search means for searching multiple existing information resources and identifying relevant information based on a user's request.

[0885] (Claim 3)

[0886] The system according to claim 1, further comprising display means that provide an interface for easily displaying and adjusting generated narratives and visual storyboards.

[0887] "Example 2 of combining an emotion engine"

[0888] (Claim 1)

[0889] A means of acquiring diverse content attributes and sentiment-related information from users,

[0890] A sentiment analysis method that analyzes acquired information to determine the user's emotions,

[0891] A generation means that creates new content by referencing past information based on identified emotional information,

[0892] A visualization means that represents the generated content as visual information,

[0893] A distribution method for delivering generated content to users,

[0894] A reconstruction method that receives emotional feedback from users and reconstructs visual information,

[0895] A system that includes this.

[0896] (Claim 2)

[0897] The system according to claim 1, further comprising a search means for searching multiple past information using user sentiment data and identifying related information.

[0898] (Claim 3)

[0899] The system according to claim 1, further comprising display means for easily displaying generated visual information and providing an interface for editing.

[0900] "Application example 2 when combining with an emotional engine"

[0901] (Claim 1)

[0902] A means for receiving the genre, theme, and related characteristics of content from a user,

[0903] A sensor means for detecting the user's emotions and acquiring that emotion data,

[0904] An analysis method that analyzes the acquired information and compares it with past data sets,

[0905] Based on the analysis results, a means to dynamically adjust the tone and style of the content generated according to emotions,

[0906] A generation method for generating a new script while referring to past information sets,

[0907] A means of creating a scene design based on the generated script,

[0908] A means of transmission that visualizes the generated content and sends it to the user,

[0909] A means of regenerating the script and scene design based on user feedback,

[0910] A system that includes this.

[0911] (Claim 2)

[0912] The system according to claim 1, further comprising a search means for searching multiple sets of past information, identifying related information, and generating interactive content using sentiment information.

[0913] (Claim 3)

[0914] The system according to claim 1, further comprising display means that provide an interface for easily displaying generated scripts and scene designs and enabling emotion-based adjustments. [Explanation of Symbols]

[0915] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. An input means for receiving the genre, theme, and related characteristics of content from the user, An analysis method that analyzes received information and compares it with past content, A generation method for generating new scripts while referencing past content, A means of creating storyboards based on the generated script, A means of sending the generated script and storyboard to the user, A means of regenerating the script and storyboards based on user feedback, A system that includes this.

2. The system according to claim 1, further comprising a search means for searching multiple past content information based on a user's request and identifying relevant information.

3. The system according to claim 1, further comprising display means that provide an interface for easily displaying and editing generated scripts and storyboards.

Images