system

Abstract

We provide the system. [Solution] A means for analyzing input documents and generating an emotional arc to visualize emotional fluctuations, A means of proposing to optimize the overall story structure of a document based on the generated emotional arc, A means of analyzing the character settings and relationships to adjust them in order to improve empathy, A means of providing a filter that adjusts documents to suit the cultural sentiments of a target region, A means of providing users with suggestions for improving scene settings and character development in real time, based on scripts uploaded to the content distribution system, in order to increase emotional depth. A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In creating a scenario of a video work, it is to solve the problem that it is difficult to efficiently create a high-quality script that effectively conveys emotions to viewers and obtains empathy in a multi-cultural environment. Also, there is a need to construct a system in which the settings and relationships of the characters in the script are attractively conveyed across cultures.

Means for Solving the Problems

[0005] According to this invention, an emotional arc is generated to visualize emotional fluctuations by analyzing an input document. Furthermore, based on the generated emotional arc, suggestions are made to optimize the overall story structure of the document, and the settings and relationships of the characters are analyzed to make adjustments to improve empathy. In addition, by providing a filter that adjusts the document to suit the cultural emotional expressions of the target region, empathy in multicultural areas is achieved. These means improve production efficiency and enable successful global expansion.

[0006] "Input documents" refer to text data that users provide to the system for analysis.

[0007] An "emotion arc" is a graph or diagram that visually represents the fluctuations and changes in emotions within a document.

[0008] "Story structure" refers to the development of a narrative in a document, the arrangement of scenes, and the overall flow that results from their combination.

[0009] "Character settings" refer to detailed information about the characters defined within the story, such as their personalities, roles, and backgrounds.

[0010] "Relationships" refer to the connections and interactions that exist between characters, and are an important element in the development of the story.

[0011] "Empathy level" refers to the degree to which viewers can emotionally connect with the characters and story.

[0012] A "filter" is a means of selecting or transforming input data based on specific conditions, and in this context, its purpose is to adapt cultural emotional expressions.

[0013] "Viewer" refers to an individual or group that appreciates and evaluates a work through this system. [Brief explanation of the drawing]

[0014] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention provides a method for specifically implementing an emotional resonance-type scenario creation system utilizing generative AI. The following describes how the system operates.

[0036] Users create scenario scripts and upload them to the system via their terminals. Upon receiving these scripts, the server uses natural language processing (NLP) to analyze the text and generate an emotion arc. This arc is extracted from the context of the sentences, the dialogue, and the characters' actions, and is represented as a graph to visualize the emotional fluctuations.

[0037] Based on the generated emotional arc, the server proposes the most effective story structure. This includes optimizing the pacing of the narrative, scene placement, and plot twists. The server also analyzes the character settings and relationships between characters in detail, creating specific adjustments to improve empathy. This includes revising dialogue and depicting the characters' inner growth.

[0038] Furthermore, the server uses a cultural sentiment filter to identify sentimental expressions appropriate to the target region's culture and adjust the document accordingly. This filter takes into account the nuances of sentimental expressions across different cultural spheres, aiming to elicit empathy in a multicultural environment.

[0039] For example, if a scenario is set in the near future and the protagonist confronts societal challenges, the server will suggest scene placements that emphasize the protagonist's emotional journey and key points of conflict. The terminal will then present these suggestions to the user, helping the work resonate more strongly with the target audience.

[0040] Thus, this system enhances the emotional depth of scripts, supports efficient scenario creation, and assists in the global distribution of works.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] The user uploads the scenario script to the system as a text file via their terminal. This prepares the system for analysis.

[0044] Step 2:

[0045] The server receives the uploaded script and tokenizes the text data using natural language processing (NLP) algorithms. This involves breaking down sentences into words and phrases and assigning sentiment metadata to each word.

[0046] Step 3:

[0047] The server generates an emotion arc from tokenized data. This visualizes the emotional changes throughout the entire script in chronological order, showing the rise and fall of emotions in each scene.

[0048] Step 4:

[0049] The server generates suggestions for the optimal story structure based on the emotional arc. Specifically, this includes rearranging the order of scenes, adding new plot points, and emphasizing certain aspects of the characters' emotional expressions.

[0050] Step 5:

[0051] The server analyzes the character settings and relationships between characters, and suggests adjustments to dialogue and actions to improve empathy. Here, the focus is on the psychological growth of each character and the deepening of relationships.

[0052] Step 6:

[0053] The server applies a cultural emotion filter, adjusting emotional expressions to align with the culture of the target region. This process takes into account differences in how emotions are perceived across cultures and makes necessary adjustments.

[0054] Step 7:

[0055] The terminal presents the user with suggestions and adjustments from the server. The user uses this information to revise the scenario and improve its completeness.

[0056] Step 8:

[0057] Users review the final scenario through their devices and provide feedback to the server as needed. This feedback is stored within the system and used to improve future models.

[0058] (Example 1)

[0059] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0060] In recent years, with the increasing diversity of digital content, creating stories that resonate emotionally with consumers has become increasingly important. However, conventional methods have struggled to adequately analyze the emotional impact on viewers and automatically propose effective story structures. This invention aims to solve these problems and generate content that more effectively appeals to the hearts of viewers.

[0061] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0062] In this invention, the server includes means for analyzing input text and generating an emotional structure to visually represent emotional fluctuations; means for suggesting an optimization of the overall narrative structure of the document based on the generated emotional structure; and means for analyzing the characteristics and relationships of the characters and performing characteristic adjustments to enhance empathy. This makes it possible to efficiently create content that provides an effective emotional experience for the viewer and enhances the emotional depth of the story and its global cultural adaptability.

[0063] "Emotional structure" refers to an abstract or concrete data representation used to analyze input text and visually show the fluctuations in emotions expressed within it.

[0064] "Narrative structure" refers to optimizing the overall flow and arrangement of a document, and designing its placement and development to take into account the emotional impact on the viewer.

[0065] "Characteristic adjustment" refers to the act of analyzing the characteristics and relationships of characters and modifying their settings and other elements to increase their relatability.

[0066] A "filter" is a method or tool used to adjust data or representations to suit a specific purpose, particularly to adapt them to culturally sensitive expressions.

[0067] A "computational procedure" is a series of algorithms or processes performed to achieve a specific purpose, and is used to optimize the interaction and arrangement of scenes in a document.

[0068] The "opinion management function" refers to a function that accumulates the input opinions and uses them to improve the accuracy of the analysis model.

[0069] This invention relates to an emotional resonance-type scenario creation system utilizing generative AI. This system analyzes the script of a scenario created by a user and generates an emotional structure, thereby proposing a more effective story structure.

[0070] The server receives the script of the scenario uploaded by the user via their terminal. At this stage, NLP (Natural Language Processing) technology is used to analyze the input text. This analysis process includes capturing emotional fluctuations within the text using an emotion analysis API. The resulting emotional structure is visualized as a graph generated based on the dialogue of each scene and character in the script.

[0071] Based on this emotional structure, the server proposes the optimal narrative structure. Specifically, it uses an AI algorithm to analyze and optimize the story's pace, scene placement, and foreshadowing. The server also analyzes character traits and relationships in detail and proposes adjustments to improve empathy.

[0072] Furthermore, to account for cultural diversity, the server applies a cultural sentiment filter, adjusting the document to express sentiments appropriate for the target region. This filter takes into account the subtle differences in sentiment expression across different cultural spheres, enabling emotional empathy in multicultural environments.

[0073] For example, in a scenario set in the near future where the protagonist confronts society, the server can refer to the emotional structure and suggest scene arrangements that emphasize the protagonist's emotional fluctuations and points of challenge. The terminal then presents these suggestions to the user, supporting the creation of a script that resonates strongly with the target audience.

[0074] An example of a prompt for a generative AI model is, "Please suggest the emotional structure and narrative structure of a script set in a near-future city, in which the protagonist confronts social inequality." In this way, users can efficiently create emotionally rich works that are suitable for global distribution, utilizing advanced AI technology.

[0075] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0076] Step 1:

[0077] Users create scenario scripts and upload them to the server via their terminal. Text files are used as input. Output is data sent to the server, ready for analysis. Operation on the terminal includes drag-and-drop and file selection buttons.

[0078] Step 2:

[0079] The server receives script files submitted by users and analyzes the text using natural language processing (NLP) techniques. The input is the received text data, and the output is the analysis result, with emotional fluctuations extracted as data. The server performs data calculations through tokenization, morphological analysis, and the use of sentiment analysis APIs.

[0080] Step 3:

[0081] The server generates an emotional structure based on the analysis results. The input is the analysis results obtained in step 2, and the output is a computational graph showing the emotional fluctuations for each scene in the script. Specifically, the process involves graphing the emotional ups and downs based on the context and character actions within the scenario.

[0082] Step 4:

[0083] The server uses emotional structures to propose an optimized story structure. The input is the generated emotional structure, and the output is the optimized story structure proposal. The server applies an AI algorithm to optimize the story's pace, scene placement, and plot twists.

[0084] Step 5:

[0085] The server analyzes character traits and relationships, and generates adjustment suggestions to enhance empathy. The input is character information from the script, and the output is a suggested adjustment to character settings aimed at improving empathy. It re-evaluates dialogue and actions, and proposes revisions as needed.

[0086] Step 6:

[0087] The server applies a cultural sentiment filter to adjust the document to suit the culture of the target region. The input is the current script data, and the output is the adjusted final document. The filter makes fine adjustments to gain empathy in different cultural spheres.

[0088] Step 7:

[0089] The terminal displays suggestions from the server to the user. Input consists of optimization and adjustment suggestions from the server, while output is information presented in a user-friendly format. The terminal assists users in easily making modifications based on the suggestions using a visual editor and dashboard.

[0090] (Application Example 1)

[0091] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0092] In modern content distribution, viewers tend to prioritize storytelling and emotional depth. However, screenwriters and creators face challenges in efficiently analyzing vast amounts of scenario data and producing works that resonate with audiences. Furthermore, they are required to adjust emotional expression to suit the cultural background of their target regions. There is a lack of support to meet these complex requirements, and new methods for creating richer, more globally relevant content are needed.

[0093] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0094] This invention includes a server that analyzes an input document and generates an emotional arc to visualize emotional fluctuations, a means to suggest optimizing the overall story structure of the document based on the generated emotional arc, a means to provide a filter to adjust the document to suit the cultural emotional expressions of a target region, and a means to provide users in real time with suggestions for improving scene settings and character development to enhance emotional depth, based on scripts uploaded to the content distribution system. This enables screenwriters and creators to efficiently and effectively construct stories that resonate with viewers and produce works that are compatible with different cultural backgrounds.

[0095] A "document" refers to organized and formalized text data, and can mean the content of a script or story.

[0096] An "emotion arc" refers to a diagram or data structure that visually represents the changes and flow of emotions within a document.

[0097] "Story structure" refers to the overall framework and sequence of events in a narrative, including the arrangement of scenes and events.

[0098] "Empathy level" is a measure that indicates the degree to which viewers are likely to emotionally connect with the characters and situations in a story.

[0099] A "filter" is a mechanism used to adjust a document, referring to the criteria or tools used to modify content according to specific cultural or emotional nuances.

[0100] A "content distribution system" refers to the technical infrastructure and platform used to deliver digital content to viewers.

[0101] "Real-time" refers to a time-constrained operation or function where input or processing occurs immediately.

[0102] An "improvement plan" refers to specific methods of modification or improvement proposed to make the current situation better.

[0103] "Internal growth" refers to the process by which a character's actions and values ​​develop through experiences within the story.

[0104] The system that realizes this invention functions through close cooperation between the user's terminal and the server. The user uploads a document they have created to their terminal, and this data is sent to the server. The server analyzes the content of the document using a natural language processing (NLP) library and generates an emotion arc that visualizes the fluctuations of emotions.

[0105] Specifically, the server utilizes a generative AI model to identify and visualize emotional shifts within a document. This allows users to intuitively understand the emotional flow of the story. Furthermore, based on the generated emotional arc, the server suggests an optimal story structure. This suggestion includes improvements to scene placement and character development, and is presented to the user in real time.

[0106] The server also helps create content suitable for a global audience by providing cultural filters that adjust emotional expression to suit the culture of the target region. The filters accurately analyze emotional expression across cultures and automatically make the necessary adjustments.

[0107] For example, it's possible to adapt the script of a science fiction drama set in the near future to suit the cultural background of the audience and maximize the emotional impact of the story.

[0108] An example of a prompt using a generative AI model is: "How can we depict the heightened emotions and deepening friendships in a scene where the protagonist confronts society on a new planet?"

[0109] This system is a powerful support tool for users in content creation, helping them to evoke empathy from viewers and enabling them to create works that appeal to audiences with diverse cultural backgrounds.

[0110] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0111] Step 1:

[0112] The user uploads the document they created to the terminal. The terminal then sends this document data to the server. Here, the input is the user's document, and the output is the transmission to the server.

[0113] Step 2:

[0114] The server uses a natural language processing (NLP) library to analyze the received document. In this step, emotions are extracted from the document's content, and changes and intensity of emotions are obtained as data. The input is the document received from the terminal, and the output is the analyzed emotion data.

[0115] Step 3:

[0116] The server utilizes a generative AI model to generate emotion arcs based on emotion data. These are graphs that visually represent the flow of emotions within a document, and in this process, extracted emotion data is taken as input and emotion arcs are output.

[0117] Step 4:

[0118] The server proposes an optimal story structure based on the generated emotional arc. Specifically, it generates suggestions for improvements regarding scene placement, narrative pacing, and character development. The input for this step is the emotional arc, and the output is the story proposal.

[0119] Step 5:

[0120] The server applies cultural filters to adjust emotional expressions according to the culture of the target region. This generates documents that take cross-cultural empathy into consideration. Here, the original story and its emotional content are taken as input, and a multiculturally adjusted document is output.

[0121] Step 6:

[0122] The server returns the generated proposals and refined documents to the terminal and presents them to the user in real time. Based on this, the user can further edit and review the documents. The input for this step is the story proposals and refined documents generated by the server, and the output is data displayed on the user's terminal as feedback.

[0123] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0124] This invention combines an emotion engine with an emotion-resonance type scenario creation system to dynamically improve the user experience of a video production's scenario by utilizing real-time user emotion data. This system is operated in the following manner.

[0125] First, the user uploads the scenario script to the system via their device. The server receives this script and analyzes the text using natural language processing (NLP) algorithms. During the analysis process, an emotion arc is generated. This emotion arc visually represents the changes in emotions within the script over time.

[0126] The added emotion engine acquires real-time emotion data from the user's facial expressions, tone of voice, gaze, and other factors. This emotion data is sent from the user's device to the server for analysis. Based on this emotion data, the server generates suggestions to dynamically adjust elements of the story. Specifically, it optimizes scene composition, character dialogue, and story development in accordance with the user's emotional responses. For example, if the user shows strong emotion towards a particular scene, a suggestion will be made to describe that scene in more detail.

[0127] The server also accumulates user emotional patterns over the long term to improve predictive models. This data will enable the system to predict what kind of story developments users will prefer in the future, allowing for more personalized experiences.

[0128] As a concrete example, if a scenario is set in a fantasy world where the protagonist overcomes trials, the emotion engine analyzes the user's reaction to the tension of the scenes. If the user shows excitement or tension, the server will use that reaction to suggest adding scenes that further heighten the tension or deepening the character's internal conflict. In this way, the system aims to enhance immersion and empathy for the work by recognizing the user's emotions.

[0129] The following describes the processing flow.

[0130] Step 1:

[0131] The user uploads the scenario script to the system using their device. This transfers the text data to be analyzed to the server.

[0132] Step 2:

[0133] The server receives the uploaded script and analyzes the text using natural language processing (NLP) algorithms. Here, sentence structure and emotional nuances are extracted, and an emotional arc is generated based on this analysis.

[0134] Step 3:

[0135] Based on the emotion arc generated by the server, we will propose ways to optimize the overall story structure of the script. These proposals may include rearranging scenes, strengthening dialogue, and adding plot points.

[0136] Step 4:

[0137] An emotion engine installed in the user's device collects real-time emotional data from the user's facial expressions, tone of voice, gaze, and other factors. This data provides a detailed record of the user's emotional responses.

[0138] Step 5:

[0139] The server receives and analyzes emotional data sent from the device. Based on this data, the server creates suggestions for dynamically adjusting elements of the story. For example, if a user expresses surprise in a particular scene, the server will provide a suggested structure to further emphasize that scene.

[0140] Step 6:

[0141] The server sends suggestions based on the user's emotions to the user's device and presents them to the user. The user can then use these suggestions to modify the scenario.

[0142] Step 7:

[0143] Users provide feedback on the final scenario and send it to the server via their device. This feedback is stored on the server and used to improve the accuracy of the sentiment engine and story optimization algorithms.

[0144] Step 8:

[0145] The server updates its predictive model based on the collected feedback, improving its ability to predict users' future preferences with greater accuracy. This ensures that future scenario creation will be more effective.

[0146] (Example 2)

[0147] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0148] There is a problem in that user-provided scenarios do not adequately adapt to the dynamic and personalized experience required to maximize viewer emotions. Furthermore, adjusting the story to take into account users' real-time emotional responses is difficult, and existing story developments are fixed, creating a need for improved immersion. Additionally, the lack of effective mechanisms for accumulating and utilizing feedback data makes long-term improvement of the user experience a challenge.

[0149] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0150] In this invention, the server includes means for receiving a scenario from a terminal, analyzing the document using natural language processing technology, and generating an emotion arc that indicates changes in emotion; means for suggesting a dynamic optimization of story elements based on the generated emotion arc and real-time emotion data obtained from the terminal; and means for obtaining and analyzing emotion data based on the user's facial expressions and tone of voice to make adjustments to improve the user experience. This makes it possible to reflect the user's real-time emotional responses in the story, provide a personalized and immersive experience, and improve long-term predictive models by utilizing feedback data.

[0151] A "scenario" is a document that describes the plot and development of a story, and is used as part of a script in film or theater.

[0152] "Natural language processing technology" refers to the technology that enables computers to understand, generate, and process human language, and is applied to areas such as text analysis and sentiment analysis.

[0153] An "emotional arc" is a chronological representation of emotional changes in a story or scenario, visually depicting the emotional fluctuations of the characters and the audience.

[0154] A "terminal" is an electronic device used by a user to access a system, and includes computers, smartphones, tablets, and other similar devices.

[0155] "Story elements" refer to the individual elements necessary to construct a narrative in a scenario, such as characters, dialogue, setting, and plot.

[0156] "Real-time emotional data" refers to data that reflects the user's current emotional state, and is acquired based on facial expressions, tone of voice, eye contact, and other factors.

[0157] "Dynamic" refers to something that changes adaptively depending on the situation or conditions, meaning it is not fixed but can be flexibly modified.

[0158] A "generative AI model" is a type of artificial intelligence that takes a prompt as input and generates output in natural language, and is applied to language generation tasks.

[0159] A "predictive model" is a mathematical or machine learning algorithm used to estimate future outcomes or trends based on past data.

[0160] This system provides a dynamic scenario experience that responds to the user's emotions. An embodiment of this system is shown below.

[0161] First, the user uploads a written scenario to the system using their device. The scenario is prepared in text file format and can be easily selected and uploaded through the device's interface. A user interface (UI) is used to facilitate file selection and submission during this process.

[0162] Next, the server processes the received scenario data. The server uses natural language processing (NLP) techniques to analyze the text and generate emotion arcs within the script. This analysis uses Python-based NLP libraries (e.g., NLTK or spaCy) to tokenize the text and classify the emotions. The generated emotion arcs are then visually displayed using data visualization libraries such as Matplotlib.

[0163] Furthermore, the device's emotion engine collects the user's facial expressions, voice tone, and gaze in real time to acquire emotion data. Computer vision technology (e.g., OpenCV) and speech analysis technology (e.g., TENSORFLOW® speech analysis module) are used to collect this data, and the analyzed emotion data is sent to a server.

[0164] The server makes suggestions to dynamically adjust elements of the story based on real-time emotion data and generated emotion arcs. These suggestions are communicated via WebSocket using Python scripts and defined as prompts sent to a generative AI model (e.g., GPT-3®). A concrete example of a prompt might be, "The user has shown strong excitement, so please generate a new scene to further increase the sense of tension." This prompt is sent to the generative AI model, which then generates a story scene appropriate to the user's emotions.

[0165] As a concrete example, in a fantasy scenario set in another world, if there is a scene where the protagonist overcomes a trial, the emotion engine generates prompt messages to be sent from the server to the AI ​​model when the user shows tension or excitement, enhancing the visual depiction of the scene. In this way, the user experience is personalized, resulting in an immersive scenario experience.

[0166] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0167] Step 1:

[0168] The user uploads a scenario text file to the system via their terminal. Specifically, they select the desired scenario file using the file selection interface on the terminal and press the upload button. At this point, the input is a text file, which is sent to the server. The output of this process is the scenario data that has reached the server.

[0169] Step 2:

[0170] The server receives the uploaded scenario and analyzes the text using natural language processing techniques. Specifically, it tokenizes the document using a Python NLP library and extracts emotion keywords. The input is the scenario data sent in step 1, and the output is a dataset for forming an emotion arc. This prepares the server to visualize the emotional changes in the story over time.

[0171] Step 3:

[0172] The server generates emotion arcs from the analyzed data. Specifically, it uses data visualization libraries such as Matplotlib to create graphs that represent changes in emotion. The input for this step is the dataset obtained in step 2, and the output is the visualized emotion arcs. This graph is displayed on the terminal in a way that the user can visually confirm.

[0173] Step 4:

[0174] The device's emotion engine captures the user's facial expressions, voice, and gaze to obtain real-time emotion data. Specifically, it collects data using a camera and microphone device and applies computer vision and speech analysis algorithms. The input is the user's real-time behavior, and the output is evaluated emotion data. This data is immediately transmitted to the server.

[0175] Step 5:

[0176] The server receives real-time sentiment data sent from the terminal and generates suggestions for story adjustments. Specifically, it performs data analysis and creates prompt sentences based on the user's sentiment response. The input for this step is the sentiment data obtained in step 4, and the output is the prompt sentence sent to the generating AI model.

[0177] Step 6:

[0178] The server uses a generative AI model to generate story adjustment suggestions based on the created prompt text. Specifically, it sends prompt text to the AI ​​model and generates text that optimizes elements of the story and dialogue. The input for this step is the prompt text created in step 5, and the output is specific story adjustment suggestions to improve the user experience. These adjustment suggestions are sent back to the user's device and presented to the user visually.

[0179] (Application Example 2)

[0180] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0181] In the viewing experience of video works and content, there is a challenge in that it is difficult to dynamically respond to the viewer's emotions, making it difficult to enhance emotional immersion and empathy. Furthermore, there is a need for highly accurate predictions to understand what kind of story development viewers prefer and to provide individualized viewing experiences.

[0182] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0183] In this invention, the server includes means for analyzing input documents and generating an emotional arc to visualize emotional fluctuations; means for analyzing the audience's emotions in real time and dynamically adjusting the narrative of the video medium based on that data; and means for accumulating viewer emotional data and predicting future emotional responses to provide a personalized experience. This makes it possible to provide a dynamic and individualized viewing experience of video works that is in line with the viewer's emotions.

[0184] An "emotion arc" is a visual representation that shows the changes over time, generated to visualize the emotional fluctuations of an input document.

[0185] An "emotion engine" is a technology that analyzes the audience's facial expressions, tone of voice, and gaze to acquire real-time emotional data.

[0186] "Dynamic adjustment" is the process of changing the scene structure and character dialogue of a story in a visual medium based on the audience's emotional response.

[0187] A "personalized experience" is a unique viewing experience of video content that is optimized based on the emotional patterns and reactions of individual viewers.

[0188] The "feedback management function" is a feature that accumulates emotional feedback obtained from viewers and uses it to improve the accuracy of the analysis model.

[0189] The main components of this system are a user terminal, a server, and a real-time sentiment analysis engine. The user terminal displays video media and provides an interface for acquiring the user's emotions. The hardware used includes a camera and microphone, while the software utilizes the video and audio processing library "OpenCV" and a machine learning model for sentiment analysis.

[0190] The server receives emotional data transmitted from user terminals, analyzes that data, and dynamically adjusts the narrative of the video medium in real time. Specifically, a natural language processing algorithm on the server generates emotional arcs and adjusts the narrative to take emotional changes into account.

[0191] User sentiment data is collected and used in the long term to predict viewer preferences and reactions. This makes it possible to provide users with a more personalized experience.

[0192] As a concrete example, in the climax scene of an isekai fantasy work, if the server detects tension from the user's facial expression, the scenario will be modified to enhance the tension of that scene. Conversely, if a relaxed reaction is observed, a comical interlude can be inserted.

[0193] An example of a prompt message is: "Imagine the climax scene of an isekai fantasy story. If the user's expression shows tension, how would you develop the scene? Add story elements that will double the sense of tension." In this way, generative AI models can be used to support the dynamic adjustment of narratives.

[0194] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0195] Step 1:

[0196] The user terminal uses a camera and microphone to capture the user's facial expressions and voice tone. This allows the terminal to acquire video frames and audio data as input. This data is processed by an emotion analysis algorithm, which outputs emotion data (such as excitement level and happiness level) representing the user's real-time emotional state.

[0197] Step 2:

[0198] The user terminal sends acquired emotional data to the server. The server receives and stores this emotional data. Next, it analyzes the past emotional data stored in the database with the newly received data to perform data calculations to understand the user's emotional patterns. Finally, it outputs real-time analysis results based on this analysis.

[0199] Step 3:

[0200] The server uses natural language processing (NLP) algorithms to analyze the script of the video medium. It uses the uploaded script as input. It generates an emotion arc and outputs data that visualizes the emotional changes within the script. This process allows for an understanding of the overall emotional flow of the story.

[0201] Step 4:

[0202] The server dynamically adjusts the narrative of the video medium based on emotional arcs and real-time user emotion data. Specifically, it adds slapstick elements to tense scenes and inserts elements that further heighten the tension. The input to this process is emotional arcs and user emotion data, and the output is optimized scenario data.

[0203] Step 5:

[0204] The user terminal plays video media based on optimized scenario data received from the server. As a result, the user can experience personalized video content. The input for this step is the scenario data from the server, and the output is the video experience viewed by the user.

[0205] Step 6:

[0206] The server accumulates user feedback over the long term and uses it to improve its analytical model, which is then used to enhance future video experiences. The input is user feedback data, and the output is the updated feedback model. As a result, it becomes possible to more accurately predict user preferences and emotional responses.

[0207] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0208] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0209] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0210] [Second Embodiment]

[0211] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0212] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0213] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0214] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0215] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0216] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0217] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0218] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0219] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0220] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0221] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0222] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0223] This invention provides a method for specifically implementing an emotional resonance-type scenario creation system utilizing generative AI. The following describes how the system operates.

[0224] Users create scenario scripts and upload them to the system via their terminals. Upon receiving these scripts, the server uses natural language processing (NLP) to analyze the text and generate an emotion arc. This arc is extracted from the context of the sentences, the dialogue, and the characters' actions, and is represented as a graph to visualize the emotional fluctuations.

[0225] Based on the generated emotional arc, the server proposes the most effective story structure. This includes optimizing the pacing of the narrative, scene placement, and plot twists. The server also analyzes the character settings and relationships between characters in detail, creating specific adjustments to improve empathy. This includes revising dialogue and depicting the characters' inner growth.

[0226] Furthermore, the server uses a cultural sentiment filter to identify sentimental expressions appropriate to the target region's culture and adjust the document accordingly. This filter takes into account the nuances of sentimental expressions across different cultural spheres, aiming to elicit empathy in a multicultural environment.

[0227] For example, if a scenario is set in the near future and the protagonist confronts societal challenges, the server will suggest scene placements that emphasize the protagonist's emotional journey and key points of conflict. The terminal will then present these suggestions to the user, helping the work resonate more strongly with the target audience.

[0228] Thus, this system enhances the emotional depth of scripts, supports efficient scenario creation, and assists in the global distribution of works.

[0229] The following describes the processing flow.

[0230] Step 1:

[0231] The user uploads the scenario script to the system as a text file via their terminal. This prepares the system for analysis.

[0232] Step 2:

[0233] The server receives the uploaded script and tokenizes the text data using natural language processing (NLP) algorithms. This involves breaking down sentences into words and phrases and assigning sentiment metadata to each word.

[0234] Step 3:

[0235] The server generates an emotion arc from tokenized data. This visualizes the emotional changes throughout the entire script in chronological order, showing the rise and fall of emotions in each scene.

[0236] Step 4:

[0237] The server generates suggestions for the optimal story structure based on the emotional arc. Specifically, this includes rearranging the order of scenes, adding new plot points, and emphasizing certain aspects of the characters' emotional expressions.

[0238] Step 5:

[0239] The server analyzes the character settings and relationships between characters, and suggests adjustments to dialogue and actions to improve empathy. Here, the focus is on the psychological growth of each character and the deepening of relationships.

[0240] Step 6:

[0241] The server applies a cultural emotion filter, adjusting emotional expressions to align with the culture of the target region. This process takes into account differences in how emotions are perceived across cultures and makes necessary adjustments.

[0242] Step 7:

[0243] The terminal presents the user with suggestions and adjustments from the server. The user uses this information to revise the scenario and improve its completeness.

[0244] Step 8:

[0245] Users review the final scenario through their devices and provide feedback to the server as needed. This feedback is stored within the system and used to improve future models.

[0246] (Example 1)

[0247] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0248] In recent years, with the increasing diversity of digital content, creating stories that resonate emotionally with consumers has become increasingly important. However, conventional methods have struggled to adequately analyze the emotional impact on viewers and automatically propose effective story structures. This invention aims to solve these problems and generate content that more effectively appeals to the hearts of viewers.

[0249] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0250] In this invention, the server includes means for analyzing input text and generating an emotional structure to visually represent emotional fluctuations; means for suggesting an optimization of the overall narrative structure of the document based on the generated emotional structure; and means for analyzing the characteristics and relationships of the characters and performing characteristic adjustments to enhance empathy. This makes it possible to efficiently create content that provides an effective emotional experience for the viewer and enhances the emotional depth of the story and its global cultural adaptability.

[0251] "Emotional structure" refers to an abstract or concrete data representation used to analyze input text and visually show the fluctuations in emotions expressed within it.

[0252] "Narrative structure" refers to optimizing the overall flow and arrangement of a document, and designing its placement and development to take into account the emotional impact on the viewer.

[0253] "Characteristic adjustment" refers to the act of analyzing the characteristics and relationships of characters and modifying their settings and other elements to increase their relatability.

[0254] A "filter" is a method or tool used to adjust data or representations to suit a specific purpose, particularly to adapt them to culturally sensitive expressions.

[0255] A "computational procedure" is a series of algorithms or processes performed to achieve a specific purpose, and is used to optimize the interaction and arrangement of scenes in a document.

[0256] The "opinion management function" refers to a function that accumulates the input opinions and uses them to improve the accuracy of the analysis model.

[0257] This invention relates to an emotional resonance-type scenario creation system utilizing generative AI. This system analyzes the script of a scenario created by a user and generates an emotional structure, thereby proposing a more effective story structure.

[0258] The server receives the script of the scenario uploaded by the user via their terminal. At this stage, NLP (Natural Language Processing) technology is used to analyze the input text. This analysis process includes capturing emotional fluctuations within the text using an emotion analysis API. The resulting emotional structure is visualized as a graph generated based on the dialogue of each scene and character in the script.

[0259] Based on this emotional structure, the server proposes the optimal narrative structure. Specifically, it uses an AI algorithm to analyze and optimize the story's pace, scene placement, and foreshadowing. The server also analyzes character traits and relationships in detail and proposes adjustments to improve empathy.

[0260] Furthermore, to account for cultural diversity, the server applies a cultural sentiment filter, adjusting the document to express sentiments appropriate for the target region. This filter takes into account the subtle differences in sentiment expression across different cultural spheres, enabling emotional empathy in multicultural environments.

[0261] For example, in a scenario set in the near future where the protagonist confronts society, the server can refer to the emotional structure and suggest scene arrangements that emphasize the protagonist's emotional fluctuations and points of challenge. The terminal then presents these suggestions to the user, supporting the creation of a script that resonates strongly with the target audience.

[0262] An example of a prompt for a generative AI model is, "Please suggest the emotional structure and narrative structure of a script set in a near-future city, in which the protagonist confronts social inequality." In this way, users can efficiently create emotionally rich works that are suitable for global distribution, utilizing advanced AI technology.

[0263] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0264] Step 1:

[0265] Users create scenario scripts and upload them to the server via their terminal. Text files are used as input. Output is data sent to the server, ready for analysis. Operation on the terminal includes drag-and-drop and file selection buttons.

[0266] Step 2:

[0267] The server receives script files submitted by users and analyzes the text using natural language processing (NLP) techniques. The input is the received text data, and the output is the analysis result, with emotional fluctuations extracted as data. The server performs data calculations through tokenization, morphological analysis, and the use of sentiment analysis APIs.

[0268] Step 3:

[0269] The server generates an emotional structure based on the analysis results. The input is the analysis results obtained in step 2, and the output is a computational graph showing the emotional fluctuations for each scene in the script. Specifically, the process involves graphing the emotional ups and downs based on the context and character actions within the scenario.

[0270] Step 4:

[0271] The server uses emotional structures to propose an optimized story structure. The input is the generated emotional structure, and the output is the optimized story structure proposal. The server applies an AI algorithm to optimize the story's pace, scene placement, and plot twists.

[0272] Step 5:

[0273] The server analyzes character traits and relationships, and generates adjustment suggestions to enhance empathy. The input is character information from the script, and the output is a suggested adjustment to character settings aimed at improving empathy. It re-evaluates dialogue and actions, and proposes revisions as needed.

[0274] Step 6:

[0275] The server applies a cultural sentiment filter to adjust the document to suit the culture of the target region. The input is the current script data, and the output is the adjusted final document. The filter makes fine adjustments to gain empathy in different cultural spheres.

[0276] Step 7:

[0277] The terminal displays suggestions from the server to the user. Input consists of optimization and adjustment suggestions from the server, while output is information presented in a user-friendly format. The terminal assists users in easily making modifications based on the suggestions using a visual editor and dashboard.

[0278] (Application Example 1)

[0279] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart glasses 214 are referred to as a "terminal".

[0280] In modern content distribution, viewers tend to value the storytelling and emotional depth of a narrative. However, it is difficult for screenwriters and creators to efficiently analyze a vast amount of scenario data and produce works that evoke empathy from viewers. Additionally, adjustments to emotional expressions that suit the cultural background of the target region are also required. There is a lack of support for meeting such complex conditions, and new methods for more abundant and globally suitable content production are being sought.

[0281] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0282] In this invention, the server includes means for analyzing an input document to generate an emotional arc for visualizing emotional fluctuations, means for making a proposal to optimize the story structure of the entire document based on the generated emotional arc, means for providing a filter for adjusting the document to conform to the cultural emotional expressions of the target region, and means for providing, in real time, improvement plans for scene settings and inner character growth to increase the emotional depth for users based on the script uploaded in the content distribution system. As a result, screenwriters and creators can construct stories that efficiently and effectively evoke empathy from viewers, and it becomes possible to produce works that can also handle different cultural backgrounds.

[0283] A "document" refers to text data that has been organized and formalized to convey information, and it means the content of a script or a story.

[0284] An "emotional arc" refers to a diagram or data structure that visually represents the changes and flow of emotions within a document.

[0285] "Story structure" refers to the overall framework and sequence of development of a story, meaning a system that includes the arrangement of scenes and events.

[0286] "Sympathy level" is a measure indicating the degree to which viewers are likely to resonate emotionally with characters and situations through a story.

[0287] "Filter" is a mechanism used for document adjustment, referring to criteria and tools for modifying content according to specific cultural or emotional nuances.

[0288] "Content delivery system" represents the technical infrastructure and platform for delivering digital works to viewers.

[0289] "Real-time" means an operation or function with time constraints where input and processing are performed immediately.

[0290] "Improvement plan" refers to specific methods of modification and improvement proposed to make the current situation better.

[0291] "Inner growth" represents the process through which a character experiences and develops actions and values through a story.

[0292] The system that realizes this invention functions by the close cooperation of the user's terminal and the server. The user uploads a document created by themselves to the terminal, and this data is sent to the server. The server analyzes the content of the document using a natural language processing (NLP) library and generates an emotion arc that visualizes the emotional ups and downs.

[0293] [[ID=NO=31]] Specifically, the server utilizes a generative AI model to identify and visualize the emotional changes within the document. As a result, the user can intuitively understand the emotional flow of the story. Furthermore, based on the generated emotion arc, the server proposes an optimal story structure. This proposal includes improvement plans for the arrangement of scenes and the inner growth of characters, and is presented to the user in real-time.

[0294] The server also helps create content suitable for a global audience by providing cultural filters that adjust emotional expression to suit the culture of the target region. The filters accurately analyze emotional expression across cultures and automatically make the necessary adjustments.

[0295] For example, it's possible to adapt the script of a science fiction drama set in the near future to suit the cultural background of the audience and maximize the emotional impact of the story.

[0296] An example of a prompt using a generative AI model is: "How can we depict the heightened emotions and deepening friendships in a scene where the protagonist confronts society on a new planet?"

[0297] This system is a powerful support tool for users in content creation, helping them to evoke empathy from viewers and enabling them to create works that appeal to audiences with diverse cultural backgrounds.

[0298] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0299] Step 1:

[0300] The user uploads the document they created to the terminal. The terminal then sends this document data to the server. Here, the input is the user's document, and the output is the transmission to the server.

[0301] Step 2:

[0302] The server uses a natural language processing (NLP) library to analyze the received document. In this step, emotions are extracted from the document's content, and changes and intensity of emotions are obtained as data. The input is the document received from the terminal, and the output is the analyzed emotion data.

[0303] Step 3:

[0304] The server utilizes a generative AI model to generate an emotion arc based on emotion data. This is a graph that visually represents the flow of emotion within a document. In this process, the extracted emotion data is used as input, and an emotion arc is outputted.

[0305] Step 4:

[0306] The server proposes an optimal story structure based on the generated emotion arc. Specifically, it generates improvement suggestions regarding the arrangement of scenes, the pacing distribution of the story, and the internal growth of characters. The input for this step is the emotion arc, and the output is a story proposal.

[0307] Step 5:

[0308] The server applies a cultural filter to adjust the emotional expressions according to the culture of the target region. As a result, a document that takes into account cross - cultural empathy is generated. Here, the original story and its emotional content are used as input, and a multi - cultural - adapted and adjusted document is outputted.

[0309] Step 6:

[0310] The server sends the generated proposals and adjusted documents back to the terminal and presents them to the user in real - time. Based on this, the user can further edit and review the document. The input for this step is the story proposal and adjusted document generated by the server, and the output is the data as feedback displayed on the user's terminal.

[0311] Furthermore, an emotion engine for estimating the user's emotion may be combined. That is, the specific processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform specific processing using the user's emotion.

[0312] This invention combines an emotion engine with an emotion-resonance type scenario creation system to dynamically improve the user experience of a video production's scenario by utilizing real-time user emotion data. This system is operated in the following manner.

[0313] First, the user uploads the scenario script to the system via their device. The server receives this script and analyzes the text using natural language processing (NLP) algorithms. During the analysis process, an emotion arc is generated. This emotion arc visually represents the changes in emotions within the script over time.

[0314] The added emotion engine acquires real-time emotion data from the user's facial expressions, tone of voice, gaze, and other factors. This emotion data is sent from the user's device to the server for analysis. Based on this emotion data, the server generates suggestions to dynamically adjust elements of the story. Specifically, it optimizes scene composition, character dialogue, and story development in accordance with the user's emotional responses. For example, if the user shows strong emotion towards a particular scene, a suggestion will be made to describe that scene in more detail.

[0315] The server also accumulates user emotional patterns over the long term to improve predictive models. This data will enable the system to predict what kind of story developments users will prefer in the future, allowing for more personalized experiences.

[0316] As a concrete example, if a scenario is set in a fantasy world where the protagonist overcomes trials, the emotion engine analyzes the user's reaction to the tension of the scenes. If the user shows excitement or tension, the server will use that reaction to suggest adding scenes that further heighten the tension or deepening the character's internal conflict. In this way, the system aims to enhance immersion and empathy for the work by recognizing the user's emotions.

[0317] The following describes the processing flow.

[0318] Step 1:

[0319] The user uploads the scenario script to the system using their device. This transfers the text data to be analyzed to the server.

[0320] Step 2:

[0321] The server receives the uploaded script and analyzes the text using natural language processing (NLP) algorithms. Here, sentence structure and emotional nuances are extracted, and an emotional arc is generated based on this analysis.

[0322] Step 3:

[0323] Based on the emotion arc generated by the server, we will propose ways to optimize the overall story structure of the script. These proposals may include rearranging scenes, strengthening dialogue, and adding plot points.

[0324] Step 4:

[0325] An emotion engine installed in the user's device collects real-time emotional data from the user's facial expressions, tone of voice, gaze, and other factors. This data provides a detailed record of the user's emotional responses.

[0326] Step 5:

[0327] The server receives and analyzes emotional data sent from the device. Based on this data, the server creates suggestions for dynamically adjusting elements of the story. For example, if a user expresses surprise in a particular scene, the server will provide a suggested structure to further emphasize that scene.

[0328] Step 6:

[0329] The server sends suggestions based on the user's emotions to the user's device and presents them to the user. The user can then use these suggestions to modify the scenario.

[0330] Step 7:

[0331] Users provide feedback on the final scenario and send it to the server via their device. This feedback is stored on the server and used to improve the accuracy of the sentiment engine and story optimization algorithms.

[0332] Step 8:

[0333] The server updates its predictive model based on the collected feedback, improving its ability to predict users' future preferences with greater accuracy. This ensures that future scenario creation will be more effective.

[0334] (Example 2)

[0335] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0336] There is a problem in that user-provided scenarios do not adequately adapt to the dynamic and personalized experience required to maximize viewer emotions. Furthermore, adjusting the story to take into account users' real-time emotional responses is difficult, and existing story developments are fixed, creating a need for improved immersion. Additionally, the lack of effective mechanisms for accumulating and utilizing feedback data makes long-term improvement of the user experience a challenge.

[0337] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0338] In this invention, the server includes means for receiving a scenario from a terminal, analyzing the document using natural language processing technology, and generating an emotion arc that indicates changes in emotion; means for suggesting a dynamic optimization of story elements based on the generated emotion arc and real-time emotion data obtained from the terminal; and means for obtaining and analyzing emotion data based on the user's facial expressions and tone of voice to make adjustments to improve the user experience. This makes it possible to reflect the user's real-time emotional responses in the story, provide a personalized and immersive experience, and improve long-term predictive models by utilizing feedback data.

[0339] A "scenario" is a document that describes the plot and development of a story, and is used as part of a script in film or theater.

[0340] "Natural language processing technology" refers to the technology that enables computers to understand, generate, and process human language, and is applied to areas such as text analysis and sentiment analysis.

[0341] An "emotional arc" is a chronological representation of emotional changes in a story or scenario, visually depicting the emotional fluctuations of the characters and the audience.

[0342] A "terminal" is an electronic device used by a user to access a system, and includes computers, smartphones, tablets, and other similar devices.

[0343] "Story elements" refer to the individual elements necessary to construct a narrative in a scenario, such as characters, dialogue, setting, and plot.

[0344] "Real-time emotional data" refers to data that reflects the user's current emotional state, and is acquired based on facial expressions, tone of voice, eye contact, and other factors.

[0345] "Dynamic" refers to something that changes adaptively depending on the situation or conditions, meaning it is not fixed but can be flexibly modified.

[0346] A "generative AI model" is a type of artificial intelligence that takes a prompt as input and generates output in natural language, and is applied to language generation tasks.

[0347] A "predictive model" is a mathematical or machine learning algorithm used to estimate future outcomes or trends based on past data.

[0348] This system provides a dynamic scenario experience that responds to the user's emotions. An embodiment of this system is shown below.

[0349] First, the user uploads a written scenario to the system using their device. The scenario is prepared in text file format and can be easily selected and uploaded through the device's interface. A user interface (UI) is used to facilitate file selection and submission during this process.

[0350] Next, the server processes the received scenario data. The server uses natural language processing (NLP) techniques to analyze the text and generate emotion arcs within the script. This analysis uses Python-based NLP libraries (e.g., NLTK or spaCy) to tokenize the text and classify the emotions. The generated emotion arcs are then visually displayed using data visualization libraries such as Matplotlib.

[0351] Furthermore, the device's emotion engine collects the user's facial expressions, voice tone, and gaze in real time to acquire emotion data. Computer vision technology (e.g., OpenCV) and speech analysis technology (e.g., TensorFlow's speech analysis module) are used to collect this data, and the analyzed emotion data is sent to a server.

[0352] The server makes suggestions to dynamically adjust elements of the story based on real-time emotion data and generated emotion arcs. These suggestions are communicated via WebSocket using Python scripts and defined as prompts sent to a generative AI model (e.g., GPT-3). A concrete example of a prompt might be, "The user has shown strong excitement, so please generate a new scene to further increase the tension." This prompt is sent to the generative AI model, which then generates a story scene appropriate to the user's emotions.

[0353] As a concrete example, in a fantasy scenario set in another world, if there is a scene where the protagonist overcomes a trial, the emotion engine generates prompt messages to be sent from the server to the AI ​​model when the user shows tension or excitement, enhancing the visual depiction of the scene. In this way, the user experience is personalized, resulting in an immersive scenario experience.

[0354] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0355] Step 1:

[0356] The user uploads a scenario text file to the system via their terminal. Specifically, they select the desired scenario file using the file selection interface on the terminal and press the upload button. At this point, the input is a text file, which is sent to the server. The output of this process is the scenario data that has reached the server.

[0357] Step 2:

[0358] The server receives the uploaded scenario and analyzes the text using natural language processing techniques. Specifically, it tokenizes the document using a Python NLP library and extracts emotion keywords. The input is the scenario data sent in step 1, and the output is a dataset for forming an emotion arc. This prepares the server to visualize the emotional changes in the story over time.

[0359] Step 3:

[0360] The server generates emotion arcs from the analyzed data. Specifically, it uses data visualization libraries such as Matplotlib to create graphs that represent changes in emotion. The input for this step is the dataset obtained in step 2, and the output is the visualized emotion arcs. This graph is displayed on the terminal in a way that the user can visually confirm.

[0361] Step 4:

[0362] The device's emotion engine captures the user's facial expressions, voice, and gaze to obtain real-time emotion data. Specifically, it collects data using a camera and microphone device and applies computer vision and speech analysis algorithms. The input is the user's real-time behavior, and the output is evaluated emotion data. This data is immediately transmitted to the server.

[0363] Step 5:

[0364] The server receives real-time sentiment data sent from the terminal and generates suggestions for story adjustments. Specifically, it performs data analysis and creates prompt sentences based on the user's sentiment response. The input for this step is the sentiment data obtained in step 4, and the output is the prompt sentence sent to the generating AI model.

[0365] Step 6:

[0366] The server uses a generative AI model to generate story adjustment suggestions based on the created prompt text. Specifically, it sends prompt text to the AI ​​model and generates text that optimizes elements of the story and dialogue. The input for this step is the prompt text created in step 5, and the output is specific story adjustment suggestions to improve the user experience. These adjustment suggestions are sent back to the user's device and presented to the user visually.

[0367] (Application Example 2)

[0368] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0369] In the viewing experience of video works and content, there is a challenge in that it is difficult to dynamically respond to the viewer's emotions, making it difficult to enhance emotional immersion and empathy. Furthermore, there is a need for highly accurate predictions to understand what kind of story development viewers prefer and to provide individualized viewing experiences.

[0370] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0371] In this invention, the server includes means for analyzing input documents and generating an emotional arc to visualize emotional fluctuations; means for analyzing the audience's emotions in real time and dynamically adjusting the narrative of the video medium based on that data; and means for accumulating viewer emotional data and predicting future emotional responses to provide a personalized experience. This makes it possible to provide a dynamic and individualized viewing experience of video works that is in line with the viewer's emotions.

[0372] An "emotion arc" is a visual representation that shows the changes over time, generated to visualize the emotional fluctuations of an input document.

[0373] An "emotion engine" is a technology that analyzes the audience's facial expressions, tone of voice, and gaze to acquire real-time emotional data.

[0374] "Dynamic adjustment" is the process of changing the scene structure and character dialogue of a story in a visual medium based on the audience's emotional response.

[0375] A "personalized experience" is a unique viewing experience of video content that is optimized based on the emotional patterns and reactions of individual viewers.

[0376] The "feedback management function" is a feature that accumulates emotional feedback obtained from viewers and uses it to improve the accuracy of the analysis model.

[0377] The main components of this system are a user terminal, a server, and a real-time sentiment analysis engine. The user terminal displays video media and provides an interface for acquiring the user's emotions. The hardware used includes a camera and microphone, while the software utilizes the video and audio processing library "OpenCV" and a machine learning model for sentiment analysis.

[0378] The server receives emotional data transmitted from user terminals, analyzes that data, and dynamically adjusts the narrative of the video medium in real time. Specifically, a natural language processing algorithm on the server generates emotional arcs and adjusts the narrative to take emotional changes into account.

[0379] User sentiment data is collected and used in the long term to predict viewer preferences and reactions. This makes it possible to provide users with a more personalized experience.

[0380] As a concrete example, in the climax scene of an isekai fantasy work, if the server detects tension from the user's facial expression, the scenario will be modified to enhance the tension of that scene. Conversely, if a relaxed reaction is observed, a comical interlude can be inserted.

[0381] An example of a prompt message is: "Imagine the climax scene of an isekai fantasy story. If the user's expression shows tension, how would you develop the scene? Add story elements that will double the sense of tension." In this way, generative AI models can be used to support the dynamic adjustment of narratives.

[0382] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0383] Step 1:

[0384] The user terminal uses a camera and microphone to capture the user's facial expressions and voice tone. This allows the terminal to acquire video frames and audio data as input. This data is processed by an emotion analysis algorithm, which outputs emotion data (such as excitement level and happiness level) representing the user's real-time emotional state.

[0385] Step 2:

[0386] The user terminal sends acquired emotional data to the server. The server receives and stores this emotional data. Next, it analyzes the past emotional data stored in the database with the newly received data to perform data calculations to understand the user's emotional patterns. Finally, it outputs real-time analysis results based on this analysis.

[0387] Step 3:

[0388] The server uses natural language processing (NLP) algorithms to analyze the script of the video medium. It uses the uploaded script as input. It generates an emotion arc and outputs data that visualizes the emotional changes within the script. This process allows for an understanding of the overall emotional flow of the story.

[0389] Step 4:

[0390] The server dynamically adjusts the narrative of the video medium based on emotional arcs and real-time user emotion data. Specifically, it adds slapstick elements to tense scenes and inserts elements that further heighten the tension. The input to this process is emotional arcs and user emotion data, and the output is optimized scenario data.

[0391] Step 5:

[0392] The user terminal plays video media based on optimized scenario data received from the server. As a result, the user can experience personalized video content. The input for this step is the scenario data from the server, and the output is the video experience viewed by the user.

[0393] Step 6:

[0394] The server accumulates user feedback over the long term and uses it to improve its analytical model, which is then used to enhance future video experiences. The input is user feedback data, and the output is the updated feedback model. As a result, it becomes possible to more accurately predict user preferences and emotional responses.

[0395] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0396] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0397] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0398] [Third Embodiment]

[0399] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0400] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0401] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0402] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0403] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0404] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0405] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0406] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0407] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0408] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0409] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0410] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0411] This invention provides a method for specifically implementing an emotional resonance-type scenario creation system utilizing generative AI. The following describes how the system operates.

[0412] Users create scenario scripts and upload them to the system via their terminals. Upon receiving these scripts, the server uses natural language processing (NLP) to analyze the text and generate an emotion arc. This arc is extracted from the context of the sentences, the dialogue, and the characters' actions, and is represented as a graph to visualize the emotional fluctuations.

[0413] Based on the generated emotional arc, the server proposes the most effective story structure. This includes optimizing the pacing of the narrative, scene placement, and plot twists. The server also analyzes the character settings and relationships between characters in detail, creating specific adjustments to improve empathy. This includes revising dialogue and depicting the characters' inner growth.

[0414] Furthermore, the server uses a cultural sentiment filter to identify sentimental expressions appropriate to the target region's culture and adjust the document accordingly. This filter takes into account the nuances of sentimental expressions across different cultural spheres, aiming to elicit empathy in a multicultural environment.

[0415] For example, if a scenario is set in the near future and the protagonist confronts societal challenges, the server will suggest scene placements that emphasize the protagonist's emotional journey and key points of conflict. The terminal will then present these suggestions to the user, helping the work resonate more strongly with the target audience.

[0416] Thus, this system enhances the emotional depth of scripts, supports efficient scenario creation, and assists in the global distribution of works.

[0417] The following describes the processing flow.

[0418] Step 1:

[0419] The user uploads the scenario script to the system as a text file via their terminal. This prepares the system for analysis.

[0420] Step 2:

[0421] The server receives the uploaded script and tokenizes the text data using natural language processing (NLP) algorithms. This involves breaking down sentences into words and phrases and assigning sentiment metadata to each word.

[0422] Step 3:

[0423] The server generates an emotion arc from tokenized data. This visualizes the emotional changes throughout the entire script in chronological order, showing the rise and fall of emotions in each scene.

[0424] Step 4:

[0425] The server generates suggestions for the optimal story structure based on the emotional arc. Specifically, this includes rearranging the order of scenes, adding new plot points, and emphasizing certain aspects of the characters' emotional expressions.

[0426] Step 5:

[0427] The server analyzes the character settings and relationships between characters, and suggests adjustments to dialogue and actions to improve empathy. Here, the focus is on the psychological growth of each character and the deepening of relationships.

[0428] Step 6:

[0429] The server applies a cultural emotion filter, adjusting emotional expressions to align with the culture of the target region. This process takes into account differences in how emotions are perceived across cultures and makes necessary adjustments.

[0430] Step 7:

[0431] The terminal presents the user with suggestions and adjustments from the server. The user uses this information to revise the scenario and improve its completeness.

[0432] Step 8:

[0433] Users review the final scenario through their devices and provide feedback to the server as needed. This feedback is stored within the system and used to improve future models.

[0434] (Example 1)

[0435] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0436] In recent years, with the increasing diversity of digital content, creating stories that resonate emotionally with consumers has become increasingly important. However, conventional methods have struggled to adequately analyze the emotional impact on viewers and automatically propose effective story structures. This invention aims to solve these problems and generate content that more effectively appeals to the hearts of viewers.

[0437] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0438] In this invention, the server includes means for analyzing input text and generating an emotional structure to visually represent emotional fluctuations; means for suggesting an optimization of the overall narrative structure of the document based on the generated emotional structure; and means for analyzing the characteristics and relationships of the characters and performing characteristic adjustments to enhance empathy. This makes it possible to efficiently create content that provides an effective emotional experience for the viewer and enhances the emotional depth of the story and its global cultural adaptability.

[0439] "Emotional structure" refers to an abstract or concrete data representation used to analyze input text and visually show the fluctuations in emotions expressed within it.

[0440] "Narrative structure" refers to optimizing the overall flow and arrangement of a document, and designing its placement and development to take into account the emotional impact on the viewer.

[0441] "Characteristic adjustment" refers to the act of analyzing the characteristics and relationships of characters and modifying their settings and other elements to increase their relatability.

[0442] A "filter" is a method or tool used to adjust data or representations to suit a specific purpose, particularly to adapt them to culturally sensitive expressions.

[0443] A "computational procedure" is a series of algorithms or processes performed to achieve a specific purpose, and is used to optimize the interaction and arrangement of scenes in a document.

[0444] The "opinion management function" refers to a function that accumulates the input opinions and uses them to improve the accuracy of the analysis model.

[0445] This invention relates to an emotional resonance-type scenario creation system utilizing generative AI. This system analyzes the script of a scenario created by a user and generates an emotional structure, thereby proposing a more effective story structure.

[0446] The server receives the script of the scenario uploaded by the user via their terminal. At this stage, NLP (Natural Language Processing) technology is used to analyze the input text. This analysis process includes capturing emotional fluctuations within the text using an emotion analysis API. The resulting emotional structure is visualized as a graph generated based on the dialogue of each scene and character in the script.

[0447] Based on this emotional structure, the server proposes the optimal narrative structure. Specifically, it uses an AI algorithm to analyze and optimize the story's pace, scene placement, and foreshadowing. The server also analyzes character traits and relationships in detail and proposes adjustments to improve empathy.

[0448] Furthermore, to account for cultural diversity, the server applies a cultural sentiment filter, adjusting the document to express sentiments appropriate for the target region. This filter takes into account the subtle differences in sentiment expression across different cultural spheres, enabling emotional empathy in multicultural environments.

[0449] For example, in a scenario set in the near future where the protagonist confronts society, the server can refer to the emotional structure and suggest scene arrangements that emphasize the protagonist's emotional fluctuations and points of challenge. The terminal then presents these suggestions to the user, supporting the creation of a script that resonates strongly with the target audience.

[0450] An example of a prompt for a generative AI model is, "Please suggest the emotional structure and narrative structure of a script set in a near-future city, in which the protagonist confronts social inequality." In this way, users can efficiently create emotionally rich works that are suitable for global distribution, utilizing advanced AI technology.

[0451] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0452] Step 1:

[0453] Users create scenario scripts and upload them to the server via their terminal. Text files are used as input. Output is data sent to the server, ready for analysis. Operation on the terminal includes drag-and-drop and file selection buttons.

[0454] Step 2:

[0455] The server receives script files submitted by users and analyzes the text using natural language processing (NLP) techniques. The input is the received text data, and the output is the analysis result, with emotional fluctuations extracted as data. The server performs data calculations through tokenization, morphological analysis, and the use of sentiment analysis APIs.

[0456] Step 3:

[0457] The server generates an emotional structure based on the analysis results. The input is the analysis results obtained in step 2, and the output is a computational graph showing the emotional fluctuations for each scene in the script. Specifically, the process involves graphing the emotional ups and downs based on the context and character actions within the scenario.

[0458] Step 4:

[0459] The server uses emotional structures to propose an optimized story structure. The input is the generated emotional structure, and the output is the optimized story structure proposal. The server applies an AI algorithm to optimize the story's pace, scene placement, and plot twists.

[0460] Step 5:

[0461] The server analyzes character traits and relationships, and generates adjustment suggestions to enhance empathy. The input is character information from the script, and the output is a suggested adjustment to character settings aimed at improving empathy. It re-evaluates dialogue and actions, and proposes revisions as needed.

[0462] Step 6:

[0463] The server applies a cultural sentiment filter to adjust the document to suit the culture of the target region. The input is the current script data, and the output is the adjusted final document. The filter makes fine adjustments to gain empathy in different cultural spheres.

[0464] Step 7:

[0465] The terminal displays suggestions from the server to the user. Input consists of optimization and adjustment suggestions from the server, while output is information presented in a user-friendly format. The terminal assists users in easily making modifications based on the suggestions using a visual editor and dashboard.

[0466] (Application Example 1)

[0467] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0468] In modern content distribution, viewers tend to prioritize storytelling and emotional depth. However, screenwriters and creators face challenges in efficiently analyzing vast amounts of scenario data and producing works that resonate with audiences. Furthermore, they are required to adjust emotional expression to suit the cultural background of their target regions. There is a lack of support to meet these complex requirements, and new methods for creating richer, more globally relevant content are needed.

[0469] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0470] This invention includes a server that analyzes an input document and generates an emotional arc to visualize emotional fluctuations, a means to suggest optimizing the overall story structure of the document based on the generated emotional arc, a means to provide a filter to adjust the document to suit the cultural emotional expressions of a target region, and a means to provide users in real time with suggestions for improving scene settings and character development to enhance emotional depth, based on scripts uploaded to the content distribution system. This enables screenwriters and creators to efficiently and effectively construct stories that resonate with viewers and produce works that are compatible with different cultural backgrounds.

[0471] A "document" refers to organized and formalized text data, and can mean the content of a script or story.

[0472] An "emotion arc" refers to a diagram or data structure that visually represents the changes and flow of emotions within a document.

[0473] "Story structure" refers to the overall framework and sequence of events in a narrative, including the arrangement of scenes and events.

[0474] "Empathy level" is a measure that indicates the degree to which viewers are likely to emotionally connect with the characters and situations in a story.

[0475] A "filter" is a mechanism used to adjust a document, referring to the criteria or tools used to modify content according to specific cultural or emotional nuances.

[0476] A "content distribution system" refers to the technical infrastructure and platform used to deliver digital content to viewers.

[0477] "Real-time" refers to a time-constrained operation or function where input or processing occurs immediately.

[0478] An "improvement plan" refers to specific methods of modification or improvement proposed to make the current situation better.

[0479] "Internal growth" refers to the process by which a character's actions and values ​​develop through experiences within the story.

[0480] The system that realizes this invention functions through close cooperation between the user's terminal and the server. The user uploads a document they have created to their terminal, and this data is sent to the server. The server analyzes the content of the document using a natural language processing (NLP) library and generates an emotion arc that visualizes the fluctuations of emotions.

[0481] Specifically, the server utilizes a generative AI model to identify and visualize emotional shifts within a document. This allows users to intuitively understand the emotional flow of the story. Furthermore, based on the generated emotional arc, the server suggests an optimal story structure. This suggestion includes improvements to scene placement and character development, and is presented to the user in real time.

[0482] The server also helps create content suitable for a global audience by providing cultural filters that adjust emotional expression to suit the culture of the target region. The filters accurately analyze emotional expression across cultures and automatically make the necessary adjustments.

[0483] For example, it's possible to adapt the script of a science fiction drama set in the near future to suit the cultural background of the audience and maximize the emotional impact of the story.

[0484] An example of a prompt using a generative AI model is: "How can we depict the heightened emotions and deepening friendships in a scene where the protagonist confronts society on a new planet?"

[0485] This system is a powerful support tool for users in content creation, helping them to evoke empathy from viewers and enabling them to create works that appeal to audiences with diverse cultural backgrounds.

[0486] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0487] Step 1:

[0488] The user uploads the document they created to the terminal. The terminal then sends this document data to the server. Here, the input is the user's document, and the output is the transmission to the server.

[0489] Step 2:

[0490] The server uses a natural language processing (NLP) library to analyze the received document. In this step, emotions are extracted from the document's content, and changes and intensity of emotions are obtained as data. The input is the document received from the terminal, and the output is the analyzed emotion data.

[0491] Step 3:

[0492] The server utilizes a generative AI model to generate emotion arcs based on emotion data. These are graphs that visually represent the flow of emotions within a document, and in this process, extracted emotion data is taken as input and emotion arcs are output.

[0493] Step 4:

[0494] The server proposes an optimal story structure based on the generated emotional arc. Specifically, it generates suggestions for improvements regarding scene placement, narrative pacing, and character development. The input for this step is the emotional arc, and the output is the story proposal.

[0495] Step 5:

[0496] The server applies cultural filters to adjust emotional expressions according to the culture of the target region. This generates documents that take cross-cultural empathy into consideration. Here, the original story and its emotional content are taken as input, and a multiculturally adjusted document is output.

[0497] Step 6:

[0498] The server returns the generated proposals and refined documents to the terminal and presents them to the user in real time. Based on this, the user can further edit and review the documents. The input for this step is the story proposals and refined documents generated by the server, and the output is data displayed on the user's terminal as feedback.

[0499] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0500] This invention combines an emotion engine with an emotion-resonance type scenario creation system to dynamically improve the user experience of a video production's scenario by utilizing real-time user emotion data. This system is operated in the following manner.

[0501] First, the user uploads the scenario script to the system via their device. The server receives this script and analyzes the text using natural language processing (NLP) algorithms. During the analysis process, an emotion arc is generated. This emotion arc visually represents the changes in emotions within the script over time.

[0502] The added emotion engine acquires real-time emotion data from the user's facial expressions, tone of voice, gaze, and other factors. This emotion data is sent from the user's device to the server for analysis. Based on this emotion data, the server generates suggestions to dynamically adjust elements of the story. Specifically, it optimizes scene composition, character dialogue, and story development in accordance with the user's emotional responses. For example, if the user shows strong emotion towards a particular scene, a suggestion will be made to describe that scene in more detail.

[0503] The server also accumulates user emotional patterns over the long term to improve predictive models. This data will enable the system to predict what kind of story developments users will prefer in the future, allowing for more personalized experiences.

[0504] As a concrete example, if a scenario is set in a fantasy world where the protagonist overcomes trials, the emotion engine analyzes the user's reaction to the tension of the scenes. If the user shows excitement or tension, the server will use that reaction to suggest adding scenes that further heighten the tension or deepening the character's internal conflict. In this way, the system aims to enhance immersion and empathy for the work by recognizing the user's emotions.

[0505] The following describes the processing flow.

[0506] Step 1:

[0507] The user uploads the scenario script to the system using their device. This transfers the text data to be analyzed to the server.

[0508] Step 2:

[0509] The server receives the uploaded script and analyzes the text using natural language processing (NLP) algorithms. Here, sentence structure and emotional nuances are extracted, and an emotional arc is generated based on this analysis.

[0510] Step 3:

[0511] Based on the emotion arc generated by the server, we will propose ways to optimize the overall story structure of the script. These proposals may include rearranging scenes, strengthening dialogue, and adding plot points.

[0512] Step 4:

[0513] An emotion engine installed in the user's device collects real-time emotional data from the user's facial expressions, tone of voice, gaze, and other factors. This data provides a detailed record of the user's emotional responses.

[0514] Step 5:

[0515] The server receives and analyzes emotional data sent from the device. Based on this data, the server creates suggestions for dynamically adjusting elements of the story. For example, if a user expresses surprise in a particular scene, the server will provide a suggested structure to further emphasize that scene.

[0516] Step 6:

[0517] The server sends suggestions based on the user's emotions to the user's device and presents them to the user. The user can then use these suggestions to modify the scenario.

[0518] Step 7:

[0519] Users provide feedback on the final scenario and send it to the server via their device. This feedback is stored on the server and used to improve the accuracy of the sentiment engine and story optimization algorithms.

[0520] Step 8:

[0521] The server updates its predictive model based on the collected feedback, improving its ability to predict users' future preferences with greater accuracy. This ensures that future scenario creation will be more effective.

[0522] (Example 2)

[0523] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0524] There is a problem in that user-provided scenarios do not adequately adapt to the dynamic and personalized experience required to maximize viewer emotions. Furthermore, adjusting the story to take into account users' real-time emotional responses is difficult, and existing story developments are fixed, creating a need for improved immersion. Additionally, the lack of effective mechanisms for accumulating and utilizing feedback data makes long-term improvement of the user experience a challenge.

[0525] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0526] In this invention, the server includes means for receiving a scenario from a terminal, analyzing the document using natural language processing technology, and generating an emotion arc that indicates changes in emotion; means for suggesting a dynamic optimization of story elements based on the generated emotion arc and real-time emotion data obtained from the terminal; and means for obtaining and analyzing emotion data based on the user's facial expressions and tone of voice to make adjustments to improve the user experience. This makes it possible to reflect the user's real-time emotional responses in the story, provide a personalized and immersive experience, and improve long-term predictive models by utilizing feedback data.

[0527] A "scenario" is a document that describes the plot and development of a story, and is used as part of a script in film or theater.

[0528] "Natural language processing technology" refers to the technology that enables computers to understand, generate, and process human language, and is applied to areas such as text analysis and sentiment analysis.

[0529] An "emotional arc" is a chronological representation of emotional changes in a story or scenario, visually depicting the emotional fluctuations of the characters and the audience.

[0530] A "terminal" is an electronic device used by a user to access a system, and includes computers, smartphones, tablets, and other similar devices.

[0531] "Story elements" refer to the individual elements necessary to construct a narrative in a scenario, such as characters, dialogue, setting, and plot.

[0532] "Real-time emotional data" refers to data that reflects the user's current emotional state, and is acquired based on facial expressions, tone of voice, eye contact, and other factors.

[0533] "Dynamic" refers to something that changes adaptively depending on the situation or conditions, meaning it is not fixed but can be flexibly modified.

[0534] A "generative AI model" is a type of artificial intelligence that takes a prompt as input and generates output in natural language, and is applied to language generation tasks.

[0535] A "predictive model" is a mathematical or machine learning algorithm used to estimate future outcomes or trends based on past data.

[0536] This system provides a dynamic scenario experience that responds to the user's emotions. An embodiment of this system is shown below.

[0537] First, the user uploads a written scenario to the system using their device. The scenario is prepared in text file format and can be easily selected and uploaded through the device's interface. A user interface (UI) is used to facilitate file selection and submission during this process.

[0538] Next, the server processes the received scenario data. The server uses natural language processing (NLP) techniques to analyze the text and generate emotion arcs within the script. This analysis uses Python-based NLP libraries (e.g., NLTK or spaCy) to tokenize the text and classify the emotions. The generated emotion arcs are then visually displayed using data visualization libraries such as Matplotlib.

[0539] Furthermore, the device's emotion engine collects the user's facial expressions, voice tone, and gaze in real time to acquire emotion data. Computer vision technology (e.g., OpenCV) and speech analysis technology (e.g., TensorFlow's speech analysis module) are used to collect this data, and the analyzed emotion data is sent to a server.

[0540] The server makes suggestions to dynamically adjust elements of the story based on real-time emotion data and generated emotion arcs. These suggestions are communicated via WebSocket using Python scripts and defined as prompts sent to a generative AI model (e.g., GPT-3). A concrete example of a prompt might be, "The user has shown strong excitement, so please generate a new scene to further increase the tension." This prompt is sent to the generative AI model, which then generates a story scene appropriate to the user's emotions.

[0541] As a concrete example, in a fantasy scenario set in another world, if there is a scene where the protagonist overcomes a trial, the emotion engine generates prompt messages to be sent from the server to the AI ​​model when the user shows tension or excitement, enhancing the visual depiction of the scene. In this way, the user experience is personalized, resulting in an immersive scenario experience.

[0542] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0543] Step 1:

[0544] The user uploads a scenario text file to the system via their terminal. Specifically, they select the desired scenario file using the file selection interface on the terminal and press the upload button. At this point, the input is a text file, which is sent to the server. The output of this process is the scenario data that has reached the server.

[0545] Step 2:

[0546] The server receives the uploaded scenario and analyzes the text using natural language processing techniques. Specifically, it tokenizes the document using a Python NLP library and extracts emotion keywords. The input is the scenario data sent in step 1, and the output is a dataset for forming an emotion arc. This prepares the server to visualize the emotional changes in the story over time.

[0547] Step 3:

[0548] The server generates emotion arcs from the analyzed data. Specifically, it uses data visualization libraries such as Matplotlib to create graphs that represent changes in emotion. The input for this step is the dataset obtained in step 2, and the output is the visualized emotion arcs. This graph is displayed on the terminal in a way that the user can visually confirm.

[0549] Step 4:

[0550] The device's emotion engine captures the user's facial expressions, voice, and gaze to obtain real-time emotion data. Specifically, it collects data using a camera and microphone device and applies computer vision and speech analysis algorithms. The input is the user's real-time behavior, and the output is evaluated emotion data. This data is immediately transmitted to the server.

[0551] Step 5:

[0552] The server receives real-time sentiment data sent from the terminal and generates suggestions for story adjustments. Specifically, it performs data analysis and creates prompt sentences based on the user's sentiment response. The input for this step is the sentiment data obtained in step 4, and the output is the prompt sentence sent to the generating AI model.

[0553] Step 6:

[0554] The server uses a generative AI model to generate story adjustment suggestions based on the created prompt text. Specifically, it sends prompt text to the AI ​​model and generates text that optimizes elements of the story and dialogue. The input for this step is the prompt text created in step 5, and the output is specific story adjustment suggestions to improve the user experience. These adjustment suggestions are sent back to the user's device and presented to the user visually.

[0555] (Application Example 2)

[0556] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0557] In the viewing experience of video works and content, there is a challenge in that it is difficult to dynamically respond to the viewer's emotions, making it difficult to enhance emotional immersion and empathy. Furthermore, there is a need for highly accurate predictions to understand what kind of story development viewers prefer and to provide individualized viewing experiences.

[0558] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0559] In this invention, the server includes means for analyzing input documents and generating an emotional arc to visualize emotional fluctuations; means for analyzing the audience's emotions in real time and dynamically adjusting the narrative of the video medium based on that data; and means for accumulating viewer emotional data and predicting future emotional responses to provide a personalized experience. This makes it possible to provide a dynamic and individualized viewing experience of video works that is in line with the viewer's emotions.

[0560] An "emotion arc" is a visual representation that shows the changes over time, generated to visualize the emotional fluctuations of an input document.

[0561] An "emotion engine" is a technology that analyzes the audience's facial expressions, tone of voice, and gaze to acquire real-time emotional data.

[0562] "Dynamic adjustment" is the process of changing the scene structure and character dialogue of a story in a visual medium based on the audience's emotional response.

[0563] A "personalized experience" is a unique viewing experience of video content that is optimized based on the emotional patterns and reactions of individual viewers.

[0564] The "feedback management function" is a feature that accumulates emotional feedback obtained from viewers and uses it to improve the accuracy of the analysis model.

[0565] The main components of this system are a user terminal, a server, and a real-time sentiment analysis engine. The user terminal displays video media and provides an interface for acquiring the user's emotions. The hardware used includes a camera and microphone, while the software utilizes the video and audio processing library "OpenCV" and a machine learning model for sentiment analysis.

[0566] The server receives emotional data transmitted from user terminals, analyzes that data, and dynamically adjusts the narrative of the video medium in real time. Specifically, a natural language processing algorithm on the server generates emotional arcs and adjusts the narrative to take emotional changes into account.

[0567] User sentiment data is collected and used in the long term to predict viewer preferences and reactions. This makes it possible to provide users with a more personalized experience.

[0568] As a concrete example, in the climax scene of an isekai fantasy work, if the server detects tension from the user's facial expression, the scenario will be modified to enhance the tension of that scene. Conversely, if a relaxed reaction is observed, a comical interlude can be inserted.

[0569] An example of a prompt message is: "Imagine the climax scene of an isekai fantasy story. If the user's expression shows tension, how would you develop the scene? Add story elements that will double the sense of tension." In this way, generative AI models can be used to support the dynamic adjustment of narratives.

[0570] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0571] Step 1:

[0572] The user terminal uses a camera and microphone to capture the user's facial expressions and voice tone. This allows the terminal to acquire video frames and audio data as input. This data is processed by an emotion analysis algorithm, which outputs emotion data (such as excitement level and happiness level) representing the user's real-time emotional state.

[0573] Step 2:

[0574] The user terminal sends acquired emotional data to the server. The server receives and stores this emotional data. Next, it analyzes the past emotional data stored in the database with the newly received data to perform data calculations to understand the user's emotional patterns. Finally, it outputs real-time analysis results based on this analysis.

[0575] Step 3:

[0576] The server uses natural language processing (NLP) algorithms to analyze the script of the video medium. It uses the uploaded script as input. It generates an emotion arc and outputs data that visualizes the emotional changes within the script. This process allows for an understanding of the overall emotional flow of the story.

[0577] Step 4:

[0578] The server dynamically adjusts the narrative of the video medium based on emotional arcs and real-time user emotion data. Specifically, it adds slapstick elements to tense scenes and inserts elements that further heighten the tension. The input to this process is emotional arcs and user emotion data, and the output is optimized scenario data.

[0579] Step 5:

[0580] The user terminal plays video media based on optimized scenario data received from the server. As a result, the user can experience personalized video content. The input for this step is the scenario data from the server, and the output is the video experience viewed by the user.

[0581] Step 6:

[0582] The server accumulates user feedback over the long term and uses it to improve its analytical model, which is then used to enhance future video experiences. The input is user feedback data, and the output is the updated feedback model. As a result, it becomes possible to more accurately predict user preferences and emotional responses.

[0583] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0584] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0585] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0586] [Fourth Embodiment]

[0587] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0588] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0589] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0590] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0591] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0592] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0593] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0594] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0595] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0596] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0597] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0598] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0599] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0600] This invention provides a method for specifically implementing an emotional resonance-type scenario creation system utilizing generative AI. The following describes how the system operates.

[0601] Users create scenario scripts and upload them to the system via their terminals. Upon receiving these scripts, the server uses natural language processing (NLP) to analyze the text and generate an emotion arc. This arc is extracted from the context of the sentences, the dialogue, and the characters' actions, and is represented as a graph to visualize the emotional fluctuations.

[0602] Based on the generated emotional arc, the server proposes the most effective story structure. This includes optimizing the pacing of the narrative, scene placement, and plot twists. The server also analyzes the character settings and relationships between characters in detail, creating specific adjustments to improve empathy. This includes revising dialogue and depicting the characters' inner growth.

[0603] Furthermore, the server uses a cultural sentiment filter to identify sentimental expressions appropriate to the target region's culture and adjust the document accordingly. This filter takes into account the nuances of sentimental expressions across different cultural spheres, aiming to elicit empathy in a multicultural environment.

[0604] For example, if a scenario is set in the near future and the protagonist confronts societal challenges, the server will suggest scene placements that emphasize the protagonist's emotional journey and key points of conflict. The terminal will then present these suggestions to the user, helping the work resonate more strongly with the target audience.

[0605] Thus, this system enhances the emotional depth of scripts, supports efficient scenario creation, and assists in the global distribution of works.

[0606] The following describes the processing flow.

[0607] Step 1:

[0608] The user uploads the scenario script to the system as a text file via their terminal. This prepares the system for analysis.

[0609] Step 2:

[0610] The server receives the uploaded script and tokenizes the text data using natural language processing (NLP) algorithms. This involves breaking down sentences into words and phrases and assigning sentiment metadata to each word.

[0611] Step 3:

[0612] The server generates an emotion arc from tokenized data. This visualizes the emotional changes throughout the entire script in chronological order, showing the rise and fall of emotions in each scene.

[0613] Step 4:

[0614] The server generates suggestions for the optimal story structure based on the emotional arc. Specifically, this includes rearranging the order of scenes, adding new plot points, and emphasizing certain aspects of the characters' emotional expressions.

[0615] Step 5:

[0616] The server analyzes the character settings and relationships between characters, and suggests adjustments to dialogue and actions to improve empathy. Here, the focus is on the psychological growth of each character and the deepening of relationships.

[0617] Step 6:

[0618] The server applies a cultural emotion filter, adjusting emotional expressions to align with the culture of the target region. This process takes into account differences in how emotions are perceived across cultures and makes necessary adjustments.

[0619] Step 7:

[0620] The terminal presents the user with suggestions and adjustments from the server. The user uses this information to revise the scenario and improve its completeness.

[0621] Step 8:

[0622] Users review the final scenario through their devices and provide feedback to the server as needed. This feedback is stored within the system and used to improve future models.

[0623] (Example 1)

[0624] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0625] In recent years, with the increasing diversity of digital content, creating stories that resonate emotionally with consumers has become increasingly important. However, conventional methods have struggled to adequately analyze the emotional impact on viewers and automatically propose effective story structures. This invention aims to solve these problems and generate content that more effectively appeals to the hearts of viewers.

[0626] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0627] In this invention, the server includes means for analyzing input text and generating an emotional structure to visually represent emotional fluctuations; means for suggesting an optimization of the overall narrative structure of the document based on the generated emotional structure; and means for analyzing the characteristics and relationships of the characters and performing characteristic adjustments to enhance empathy. This makes it possible to efficiently create content that provides an effective emotional experience for the viewer and enhances the emotional depth of the story and its global cultural adaptability.

[0628] "Emotional structure" refers to an abstract or concrete data representation used to analyze input text and visually show the fluctuations in emotions expressed within it.

[0629] "Narrative structure" refers to optimizing the overall flow and arrangement of a document, and designing its placement and development to take into account the emotional impact on the viewer.

[0630] "Characteristic adjustment" refers to the act of analyzing the characteristics and relationships of characters and modifying their settings and other elements to increase their relatability.

[0631] A "filter" is a method or tool used to adjust data or representations to suit a specific purpose, particularly to adapt them to culturally sensitive expressions.

[0632] A "computational procedure" is a series of algorithms or processes performed to achieve a specific purpose, and is used to optimize the interaction and arrangement of scenes in a document.

[0633] The "opinion management function" refers to a function that accumulates the input opinions and uses them to improve the accuracy of the analysis model.

[0634] This invention relates to an emotional resonance-type scenario creation system utilizing generative AI. This system analyzes the script of a scenario created by a user and generates an emotional structure, thereby proposing a more effective story structure.

[0635] The server receives the script of the scenario uploaded by the user via their terminal. At this stage, NLP (Natural Language Processing) technology is used to analyze the input text. This analysis process includes capturing emotional fluctuations within the text using an emotion analysis API. The resulting emotional structure is visualized as a graph generated based on the dialogue of each scene and character in the script.

[0636] Based on this emotional structure, the server proposes the optimal narrative structure. Specifically, it uses an AI algorithm to analyze and optimize the story's pace, scene placement, and foreshadowing. The server also analyzes character traits and relationships in detail and proposes adjustments to improve empathy.

[0637] Furthermore, to account for cultural diversity, the server applies a cultural sentiment filter, adjusting the document to express sentiments appropriate for the target region. This filter takes into account the subtle differences in sentiment expression across different cultural spheres, enabling emotional empathy in multicultural environments.

[0638] For example, in a scenario set in the near future where the protagonist confronts society, the server can refer to the emotional structure and suggest scene arrangements that emphasize the protagonist's emotional fluctuations and points of challenge. The terminal then presents these suggestions to the user, supporting the creation of a script that resonates strongly with the target audience.

[0639] An example of a prompt for a generative AI model is, "Please suggest the emotional structure and narrative structure of a script set in a near-future city, in which the protagonist confronts social inequality." In this way, users can efficiently create emotionally rich works that are suitable for global distribution, utilizing advanced AI technology.

[0640] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0641] Step 1:

[0642] Users create scenario scripts and upload them to the server via their terminal. Text files are used as input. Output is data sent to the server, ready for analysis. Operation on the terminal includes drag-and-drop and file selection buttons.

[0643] Step 2:

[0644] The server receives script files submitted by users and analyzes the text using natural language processing (NLP) techniques. The input is the received text data, and the output is the analysis result, with emotional fluctuations extracted as data. The server performs data calculations through tokenization, morphological analysis, and the use of sentiment analysis APIs.

[0645] Step 3:

[0646] The server generates an emotional structure based on the analysis results. The input is the analysis results obtained in step 2, and the output is a computational graph showing the emotional fluctuations for each scene in the script. Specifically, the process involves graphing the emotional ups and downs based on the context and character actions within the scenario.

[0647] Step 4:

[0648] The server uses emotional structures to propose an optimized story structure. The input is the generated emotional structure, and the output is the optimized story structure proposal. The server applies an AI algorithm to optimize the story's pace, scene placement, and plot twists.

[0649] Step 5:

[0650] The server analyzes character traits and relationships, and generates adjustment suggestions to enhance empathy. The input is character information from the script, and the output is a suggested adjustment to character settings aimed at improving empathy. It re-evaluates dialogue and actions, and proposes revisions as needed.

[0651] Step 6:

[0652] The server applies a cultural sentiment filter to adjust the document to suit the culture of the target region. The input is the current script data, and the output is the adjusted final document. The filter makes fine adjustments to gain empathy in different cultural spheres.

[0653] Step 7:

[0654] The terminal displays suggestions from the server to the user. Input consists of optimization and adjustment suggestions from the server, while output is information presented in a user-friendly format. The terminal assists users in easily making modifications based on the suggestions using a visual editor and dashboard.

[0655] (Application Example 1)

[0656] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0657] In modern content distribution, viewers tend to prioritize storytelling and emotional depth. However, screenwriters and creators face challenges in efficiently analyzing vast amounts of scenario data and producing works that resonate with audiences. Furthermore, they are required to adjust emotional expression to suit the cultural background of their target regions. There is a lack of support to meet these complex requirements, and new methods for creating richer, more globally relevant content are needed.

[0658] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0659] This invention includes a server that analyzes an input document and generates an emotional arc to visualize emotional fluctuations, a means to suggest optimizing the overall story structure of the document based on the generated emotional arc, a means to provide a filter to adjust the document to suit the cultural emotional expressions of a target region, and a means to provide users in real time with suggestions for improving scene settings and character development to enhance emotional depth, based on scripts uploaded to the content distribution system. This enables screenwriters and creators to efficiently and effectively construct stories that resonate with viewers and produce works that are compatible with different cultural backgrounds.

[0660] A "document" refers to organized and formalized text data, and can mean the content of a script or story.

[0661] An "emotion arc" refers to a diagram or data structure that visually represents the changes and flow of emotions within a document.

[0662] "Story structure" refers to the overall framework and sequence of events in a narrative, including the arrangement of scenes and events.

[0663] "Empathy level" is a measure that indicates the degree to which viewers are likely to emotionally connect with the characters and situations in a story.

[0664] A "filter" is a mechanism used to adjust a document, referring to the criteria or tools used to modify content according to specific cultural or emotional nuances.

[0665] A "content distribution system" refers to the technical infrastructure and platform used to deliver digital content to viewers.

[0666] "Real-time" refers to a time-constrained operation or function where input or processing occurs immediately.

[0667] An "improvement plan" refers to specific methods of modification or improvement proposed to make the current situation better.

[0668] "Internal growth" refers to the process by which a character's actions and values ​​develop through experiences within the story.

[0669] The system that realizes this invention functions through close cooperation between the user's terminal and the server. The user uploads a document they have created to their terminal, and this data is sent to the server. The server analyzes the content of the document using a natural language processing (NLP) library and generates an emotion arc that visualizes the fluctuations of emotions.

[0670] Specifically, the server utilizes a generative AI model to identify and visualize emotional shifts within a document. This allows users to intuitively understand the emotional flow of the story. Furthermore, based on the generated emotional arc, the server suggests an optimal story structure. This suggestion includes improvements to scene placement and character development, and is presented to the user in real time.

[0671] The server also helps create content suitable for a global audience by providing cultural filters that adjust emotional expression to suit the culture of the target region. The filters accurately analyze emotional expression across cultures and automatically make the necessary adjustments.

[0672] For example, it's possible to adapt the script of a science fiction drama set in the near future to suit the cultural background of the audience and maximize the emotional impact of the story.

[0673] An example of a prompt using a generative AI model is: "How can we depict the heightened emotions and deepening friendships in a scene where the protagonist confronts society on a new planet?"

[0674] This system is a powerful support tool for users in content creation, helping them to evoke empathy from viewers and enabling them to create works that appeal to audiences with diverse cultural backgrounds.

[0675] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0676] Step 1:

[0677] The user uploads the document they created to the terminal. The terminal then sends this document data to the server. Here, the input is the user's document, and the output is the transmission to the server.

[0678] Step 2:

[0679] The server uses a natural language processing (NLP) library to analyze the received document. In this step, emotions are extracted from the document's content, and changes and intensity of emotions are obtained as data. The input is the document received from the terminal, and the output is the analyzed emotion data.

[0680] Step 3:

[0681] The server utilizes a generative AI model to generate emotion arcs based on emotion data. These are graphs that visually represent the flow of emotions within a document, and in this process, extracted emotion data is taken as input and emotion arcs are output.

[0682] Step 4:

[0683] The server proposes an optimal story structure based on the generated emotional arc. Specifically, it generates suggestions for improvements regarding scene placement, narrative pacing, and character development. The input for this step is the emotional arc, and the output is the story proposal.

[0684] Step 5:

[0685] The server applies cultural filters to adjust emotional expressions according to the culture of the target region. This generates documents that take cross-cultural empathy into consideration. Here, the original story and its emotional content are taken as input, and a multiculturally adjusted document is output.

[0686] Step 6:

[0687] The server returns the generated proposals and refined documents to the terminal and presents them to the user in real time. Based on this, the user can further edit and review the documents. The input for this step is the story proposals and refined documents generated by the server, and the output is data displayed on the user's terminal as feedback.

[0688] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0689] This invention combines an emotion engine with an emotion-resonance type scenario creation system to dynamically improve the user experience of a video production's scenario by utilizing real-time user emotion data. This system is operated in the following manner.

[0690] First, the user uploads the scenario script to the system via their device. The server receives this script and analyzes the text using natural language processing (NLP) algorithms. During the analysis process, an emotion arc is generated. This emotion arc visually represents the changes in emotions within the script over time.

[0691] The added emotion engine acquires real-time emotion data from the user's facial expressions, tone of voice, gaze, and other factors. This emotion data is sent from the user's device to the server for analysis. Based on this emotion data, the server generates suggestions to dynamically adjust elements of the story. Specifically, it optimizes scene composition, character dialogue, and story development in accordance with the user's emotional responses. For example, if the user shows strong emotion towards a particular scene, a suggestion will be made to describe that scene in more detail.

[0692] The server also accumulates user emotional patterns over the long term to improve predictive models. This data will enable the system to predict what kind of story developments users will prefer in the future, allowing for more personalized experiences.

[0693] As a concrete example, if a scenario is set in a fantasy world where the protagonist overcomes trials, the emotion engine analyzes the user's reaction to the tension of the scenes. If the user shows excitement or tension, the server will use that reaction to suggest adding scenes that further heighten the tension or deepening the character's internal conflict. In this way, the system aims to enhance immersion and empathy for the work by recognizing the user's emotions.

[0694] The following describes the processing flow.

[0695] Step 1:

[0696] The user uploads the scenario script to the system using their device. This transfers the text data to be analyzed to the server.

[0697] Step 2:

[0698] The server receives the uploaded script and analyzes the text using natural language processing (NLP) algorithms. Here, sentence structure and emotional nuances are extracted, and an emotional arc is generated based on this analysis.

[0699] Step 3:

[0700] Based on the emotion arc generated by the server, we will propose ways to optimize the overall story structure of the script. These proposals may include rearranging scenes, strengthening dialogue, and adding plot points.

[0701] Step 4:

[0702] An emotion engine installed in the user's device collects real-time emotional data from the user's facial expressions, tone of voice, gaze, and other factors. This data provides a detailed record of the user's emotional responses.

[0703] Step 5:

[0704] The server receives and analyzes emotional data sent from the device. Based on this data, the server creates suggestions for dynamically adjusting elements of the story. For example, if a user expresses surprise in a particular scene, the server will provide a suggested structure to further emphasize that scene.

[0705] Step 6:

[0706] The server sends suggestions based on the user's emotions to the user's device and presents them to the user. The user can then use these suggestions to modify the scenario.

[0707] Step 7:

[0708] Users provide feedback on the final scenario and send it to the server via their device. This feedback is stored on the server and used to improve the accuracy of the sentiment engine and story optimization algorithms.

[0709] Step 8:

[0710] The server updates its predictive model based on the collected feedback, improving its ability to predict users' future preferences with greater accuracy. This ensures that future scenario creation will be more effective.

[0711] (Example 2)

[0712] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0713] There is a problem in that user-provided scenarios do not adequately adapt to the dynamic and personalized experience required to maximize viewer emotions. Furthermore, adjusting the story to take into account users' real-time emotional responses is difficult, and existing story developments are fixed, creating a need for improved immersion. Additionally, the lack of effective mechanisms for accumulating and utilizing feedback data makes long-term improvement of the user experience a challenge.

[0714] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0715] In this invention, the server includes means for receiving a scenario from a terminal, analyzing the document using natural language processing technology, and generating an emotion arc that indicates changes in emotion; means for suggesting a dynamic optimization of story elements based on the generated emotion arc and real-time emotion data obtained from the terminal; and means for obtaining and analyzing emotion data based on the user's facial expressions and tone of voice to make adjustments to improve the user experience. This makes it possible to reflect the user's real-time emotional responses in the story, provide a personalized and immersive experience, and improve long-term predictive models by utilizing feedback data.

[0716] A "scenario" is a document that describes the plot and development of a story, and is used as part of a script in film or theater.

[0717] "Natural language processing technology" refers to the technology that enables computers to understand, generate, and process human language, and is applied to areas such as text analysis and sentiment analysis.

[0718] An "emotional arc" is a chronological representation of emotional changes in a story or scenario, visually depicting the emotional fluctuations of the characters and the audience.

[0719] A "terminal" is an electronic device used by a user to access a system, and includes computers, smartphones, tablets, and other similar devices.

[0720] "Story elements" refer to the individual elements necessary to construct a narrative in a scenario, such as characters, dialogue, setting, and plot.

[0721] "Real-time emotional data" refers to data that reflects the user's current emotional state, and is acquired based on facial expressions, tone of voice, eye contact, and other factors.

[0722] "Dynamic" refers to something that changes adaptively depending on the situation or conditions, meaning it is not fixed but can be flexibly modified.

[0723] A "generative AI model" is a type of artificial intelligence that takes a prompt as input and generates output in natural language, and is applied to language generation tasks.

[0724] A "predictive model" is a mathematical or machine learning algorithm used to estimate future outcomes or trends based on past data.

[0725] This system provides a dynamic scenario experience that responds to the user's emotions. An embodiment of this system is shown below.

[0726] First, the user uploads a written scenario to the system using their device. The scenario is prepared in text file format and can be easily selected and uploaded through the device's interface. A user interface (UI) is used to facilitate file selection and submission during this process.

[0727] Next, the server processes the received scenario data. The server uses natural language processing (NLP) techniques to analyze the text and generate emotion arcs within the script. This analysis uses Python-based NLP libraries (e.g., NLTK or spaCy) to tokenize the text and classify the emotions. The generated emotion arcs are then visually displayed using data visualization libraries such as Matplotlib.

[0728] Furthermore, the device's emotion engine collects the user's facial expressions, voice tone, and gaze in real time to acquire emotion data. Computer vision technology (e.g., OpenCV) and speech analysis technology (e.g., TensorFlow's speech analysis module) are used to collect this data, and the analyzed emotion data is sent to a server.

[0729] The server makes suggestions to dynamically adjust elements of the story based on real-time emotion data and generated emotion arcs. These suggestions are communicated via WebSocket using Python scripts and defined as prompts sent to a generative AI model (e.g., GPT-3). A concrete example of a prompt might be, "The user has shown strong excitement, so please generate a new scene to further increase the tension." This prompt is sent to the generative AI model, which then generates a story scene appropriate to the user's emotions.

[0730] As a concrete example, in a fantasy scenario set in another world, if there is a scene where the protagonist overcomes a trial, the emotion engine generates prompt messages to be sent from the server to the AI ​​model when the user shows tension or excitement, enhancing the visual depiction of the scene. In this way, the user experience is personalized, resulting in an immersive scenario experience.

[0731] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0732] Step 1:

[0733] The user uploads a scenario text file to the system via their terminal. Specifically, they select the desired scenario file using the file selection interface on the terminal and press the upload button. At this point, the input is a text file, which is sent to the server. The output of this process is the scenario data that has reached the server.

[0734] Step 2:

[0735] The server receives the uploaded scenario and analyzes the text using natural language processing techniques. Specifically, it tokenizes the document using a Python NLP library and extracts emotion keywords. The input is the scenario data sent in step 1, and the output is a dataset for forming an emotion arc. This prepares the server to visualize the emotional changes in the story over time.

[0736] Step 3:

[0737] The server generates emotion arcs from the analyzed data. Specifically, it uses data visualization libraries such as Matplotlib to create graphs that represent changes in emotion. The input for this step is the dataset obtained in step 2, and the output is the visualized emotion arcs. This graph is displayed on the terminal in a way that the user can visually confirm.

[0738] Step 4:

[0739] The device's emotion engine captures the user's facial expressions, voice, and gaze to obtain real-time emotion data. Specifically, it collects data using a camera and microphone device and applies computer vision and speech analysis algorithms. The input is the user's real-time behavior, and the output is evaluated emotion data. This data is immediately transmitted to the server.

[0740] Step 5:

[0741] The server receives real-time sentiment data sent from the terminal and generates suggestions for story adjustments. Specifically, it performs data analysis and creates prompt sentences based on the user's sentiment response. The input for this step is the sentiment data obtained in step 4, and the output is the prompt sentence sent to the generating AI model.

[0742] Step 6:

[0743] The server uses a generative AI model to generate story adjustment suggestions based on the created prompt text. Specifically, it sends prompt text to the AI ​​model and generates text that optimizes elements of the story and dialogue. The input for this step is the prompt text created in step 5, and the output is specific story adjustment suggestions to improve the user experience. These adjustment suggestions are sent back to the user's device and presented to the user visually.

[0744] (Application Example 2)

[0745] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0746] In the viewing experience of video works and content, there is a challenge in that it is difficult to dynamically respond to the viewer's emotions, making it difficult to enhance emotional immersion and empathy. Furthermore, there is a need for highly accurate predictions to understand what kind of story development viewers prefer and to provide individualized viewing experiences.

[0747] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0748] In this invention, the server includes means for analyzing input documents and generating an emotional arc to visualize emotional fluctuations; means for analyzing the audience's emotions in real time and dynamically adjusting the narrative of the video medium based on that data; and means for accumulating viewer emotional data and predicting future emotional responses to provide a personalized experience. This makes it possible to provide a dynamic and individualized viewing experience of video works that is in line with the viewer's emotions.

[0749] An "emotion arc" is a visual representation that shows the changes over time, generated to visualize the emotional fluctuations of an input document.

[0750] An "emotion engine" is a technology that analyzes the audience's facial expressions, tone of voice, and gaze to acquire real-time emotional data.

[0751] "Dynamic adjustment" is the process of changing the scene structure and character dialogue of a story in a visual medium based on the audience's emotional response.

[0752] A "personalized experience" is a unique viewing experience of video content that is optimized based on the emotional patterns and reactions of individual viewers.

[0753] The "feedback management function" is a feature that accumulates emotional feedback obtained from viewers and uses it to improve the accuracy of the analysis model.

[0754] The main components of this system are a user terminal, a server, and a real-time sentiment analysis engine. The user terminal displays video media and provides an interface for acquiring the user's emotions. The hardware used includes a camera and microphone, while the software utilizes the video and audio processing library "OpenCV" and a machine learning model for sentiment analysis.

[0755] The server receives emotional data transmitted from user terminals, analyzes that data, and dynamically adjusts the narrative of the video medium in real time. Specifically, a natural language processing algorithm on the server generates emotional arcs and adjusts the narrative to take emotional changes into account.

[0756] User sentiment data is collected and used in the long term to predict viewer preferences and reactions. This makes it possible to provide users with a more personalized experience.

[0757] As a concrete example, in the climax scene of an isekai fantasy work, if the server detects tension from the user's facial expression, the scenario will be modified to enhance the tension of that scene. Conversely, if a relaxed reaction is observed, a comical interlude can be inserted.

[0758] An example of a prompt message is: "Imagine the climax scene of an isekai fantasy story. If the user's expression shows tension, how would you develop the scene? Add story elements that will double the sense of tension." In this way, generative AI models can be used to support the dynamic adjustment of narratives.

[0759] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0760] Step 1:

[0761] The user terminal uses a camera and microphone to capture the user's facial expressions and voice tone. This allows the terminal to acquire video frames and audio data as input. This data is processed by an emotion analysis algorithm, which outputs emotion data (such as excitement level and happiness level) representing the user's real-time emotional state.

[0762] Step 2:

[0763] The user terminal sends acquired emotional data to the server. The server receives and stores this emotional data. Next, it analyzes the past emotional data stored in the database with the newly received data to perform data calculations to understand the user's emotional patterns. Finally, it outputs real-time analysis results based on this analysis.

[0764] Step 3:

[0765] The server uses natural language processing (NLP) algorithms to analyze the script of the video medium. It uses the uploaded script as input. It generates an emotion arc and outputs data that visualizes the emotional changes within the script. This process allows for an understanding of the overall emotional flow of the story.

[0766] Step 4:

[0767] The server dynamically adjusts the narrative of the video medium based on emotional arcs and real-time user emotion data. Specifically, it adds slapstick elements to tense scenes and inserts elements that further heighten the tension. The input to this process is emotional arcs and user emotion data, and the output is optimized scenario data.

[0768] Step 5:

[0769] The user terminal plays video media based on optimized scenario data received from the server. As a result, the user can experience personalized video content. The input for this step is the scenario data from the server, and the output is the video experience viewed by the user.

[0770] Step 6:

[0771] The server accumulates user feedback over the long term and uses it to improve its analytical model, which is then used to enhance future video experiences. The input is user feedback data, and the output is the updated feedback model. As a result, it becomes possible to more accurately predict user preferences and emotional responses.

[0772] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0773] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0774] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0775] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0776] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0777] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0778] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0779] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0780] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0781] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0782] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0783] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0784] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0785] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0786] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0787] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0788] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0789] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0790] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0791] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0792] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0793] The following is further disclosed regarding the embodiments described above.

[0794] (Claim 1)

[0795] A means for analyzing input documents and generating an emotional arc to visualize emotional fluctuations,

[0796] A means of proposing to optimize the overall story structure of a document based on the generated emotional arc,

[0797] A means of analyzing the character settings and relationships to adjust them in order to improve empathy,

[0798] A means of providing a filter that adjusts documents to suit the cultural sentiments of a target region,

[0799] A system that includes this.

[0800] (Claim 2)

[0801] The system according to claim 1, which uses an algorithm to optimize the arrangement of dialogues and scenes within a document to propose creating an effective emotional experience for the viewer.

[0802] (Claim 3)

[0803] The system according to claim 1, which has a feedback management function that accumulates input feedback and uses it to improve the accuracy of the analysis model.

[0804] "Example 1"

[0805] (Claim 1)

[0806] A means for analyzing input text and generating an emotional structure to visually represent emotional fluctuations,

[0807] A means of proposing an optimization of the overall narrative structure of a document based on the generated emotional structure,

[0808] A means of analyzing the characteristics and relationships of the characters and adjusting those characteristics to enhance empathy,

[0809] A means of providing a filter to adjust documents to suit the cultural sentiments of a target region,

[0810] A system that includes this.

[0811] (Claim 2)

[0812] The system according to claim 1, which uses computational procedures to optimize the dialogue and scene arrangement of a document to propose an effective emotional experience for the viewer.

[0813] (Claim 3)

[0814] The system according to claim 1, which has an opinion management function that stores input opinions and uses them to improve the accuracy of the analysis model.

[0815] "Application Example 1"

[0816] (Claim 1)

[0817] A means for analyzing input documents and generating an emotional arc to visualize emotional fluctuations,

[0818] A means of proposing to optimize the overall story structure of a document based on the generated emotional arc,

[0819] A means of analyzing the character settings and relationships to adjust them in order to improve empathy,

[0820] A means of providing a filter that adjusts documents to suit the cultural sentiments of a target region,

[0821] A means of providing users with suggestions for improving scene settings and character development in real time, based on scripts uploaded to the content distribution system, in order to increase emotional depth.

[0822] A system that includes this.

[0823] (Claim 2)

[0824] The system according to claim 1, which uses an algorithm to optimize the arrangement of dialogues and scenes within a document to propose creating an effective emotional experience for the viewer.

[0825] (Claim 3)

[0826] The system according to claim 1, which has a feedback management function that accumulates input feedback and uses it to improve the accuracy of the analysis model.

[0827] "Example 2 of combining an emotion engine"

[0828] (Claim 1)

[0829] A means for receiving a scenario from a terminal, analyzing the document using natural language processing technology, and generating an emotion arc that indicates changes in emotion,

[0830] A means for dynamically optimizing story elements based on generated emotion arcs and real-time emotion data acquired from the device,

[0831] A means of acquiring and analyzing emotional data based on the user's facial expressions and tone of voice to make adjustments to improve the user experience,

[0832] A means to visually represent the generated emotional arc, accumulate user-specific emotional patterns over the long term, and improve the predictive model,

[0833] A system that includes this.

[0834] (Claim 2)

[0835] The system according to claim 1, which dynamically adjusts the story development and dialogue within a document using a generative AI model in response to the user's emotional response, thereby providing the user with an optimal emotional experience.

[0836] (Claim 3)

[0837] The system according to claim 1, which has a feedback management function that accumulates user feedback data and uses it to further improve the accuracy of a predictive model.

[0838] "Application example 2 when combining with an emotional engine"

[0839] (Claim 1)

[0840] A means for analyzing input documents and generating an emotional arc to visualize emotional fluctuations,

[0841] A means of proposing to optimize the overall story structure of a document based on the generated emotional arc,

[0842] A means of analyzing the character settings and relationships to adjust them in order to improve empathy,

[0843] A means of providing a filter that adjusts documents to suit the cultural sentiments of a target region,

[0844] A means of analyzing audience emotions in real time and dynamically adjusting the narrative of the video medium based on that data,

[0845] A means of accumulating viewer emotional data, predicting future emotional responses, and providing personalized experiences,

[0846] A system that includes this.

[0847] (Claim 2)

[0848] The system according to claim 1, which uses an algorithm to optimize the arrangement of dialogues and scenes within a document to propose creating an effective emotional experience for the audience.

[0849] (Claim 3)

[0850] The system according to claim 1, which has a feedback management function that accumulates input emotional feedback and uses it to improve the accuracy of the analysis model. [Explanation of Symbols]

[0851] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for analyzing input documents and generating an emotional arc to visualize emotional fluctuations, A means of proposing to optimize the overall story structure of a document based on the generated emotional arc, A means of analyzing the character settings and relationships and making adjustments to improve empathy, A means of providing a filter that adjusts documents to suit the cultural sentiments of a target region, A means of providing users with suggestions for improving scene settings and character development in real time, based on scripts uploaded to the content distribution system, in order to increase emotional depth. A system that includes this.

2. The system according to claim 1, which uses an algorithm to optimize the arrangement of dialogues and scenes in a document to propose creating an effective emotional experience for the viewer.

3. The system according to claim 1, which has a feedback management function that accumulates input feedback and uses it to improve the accuracy of the analysis model.

Images