system

Abstract

We provide the system. [Solution] A means by which the story branches and events occur through conversations with the character selected by the user, A means by which the system calculates intimacy based on the content of the conversation and displays a special scenario when certain conditions are met, A means for analyzing chat content using natural language processing and generating an appropriate response based on the analysis results, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern computer games and simulations, the interaction between the user and the character is limited, and often only specific options or fixed stories are provided. As a result, the user experience becomes monotonous, and there is a problem of lacking intimacy and emotional continuity with the character. In addition, there are few systems in which stories or events change dynamically in response to user input, and there is a problem that the user experience is limited.

Means for Solving the Problems

[0005] This invention provides a system in which the story branches and events occur through conversations with a character selected by the user. This system includes means for calculating intimacy based on the conversation content and displaying a special scenario when certain conditions are met. Furthermore, by including means for analyzing the chat content using natural language processing and generating an appropriate response based on the analysis results, it is possible to dynamically develop the story in response to the user's choices and inputs. This enriches the user experience and enhances the intimacy and emotional continuity with the character.

[0006] A "user" refers to a person who operates a computer system or application and enjoys the progression of the story through conversations with characters and various choices.

[0007] A "character" refers to a virtual person or entity that a user selects and interacts with through dialogue and events within a computer system.

[0008] "Conversation" refers to the exchange of text and messages between the user and the character, the content of which influences the progression and branching of the story.

[0009] "Story branching" means that the direction of the story changes depending on the user's dialogue and choices.

[0010] An "event" refers to a special occurrence or scene that happens during the progression of the story, and these can influence the branching of the storyline or changes in character intimacy levels.

[0011] "Intimacy" is a quantifiable indicator of the emotional distance and relationship between the user and the character, and it fluctuates depending on conversations and events.

[0012] A "special scenario" refers to a scenario that includes specific important scenes or events that are only displayed when the user's intimacy with a character or certain conditions are met.

[0013] "Natural language processing" refers to the technology that enables computers to understand, analyze, and generate appropriate responses to human language.

[0014] "Analysis results" refer to the information and data that the natural language processing engine was able to obtain by analyzing the user's message.

[0015] "Appropriate response" refers to the character's message or action that is generated based on the analysis results and displayed as a reply to the user's message. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the language used in the following description will be explained.

[0019] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention describes a system in which a story branches and events occur through conversations between a user and a character. This system uses natural language processing to analyze the user's messages and generate appropriate responses, and includes means for calculating intimacy levels and displaying special scenarios.

[0038] Specific program processing

[0039] 1. User Login and Authentication

[0040] Terminal: The user opens the login screen from their terminal and enters their username and password.

[0041] Server: The server compares the username and password received from the terminal with the authentication database. If authentication is successful, it sends the home screen data to the terminal. If authentication fails, it sends an error message.

[0042] 2. Character Selection

[0043] Terminal: Upon successful authentication, the character selection screen will be displayed.

[0044] User: Select your preferred character from the list.

[0045] Terminal: Sends selection information to the server.

[0046] Server: Loads the initial data and background information of the selected character and sends the chat screen data to the terminal.

[0047] 3. Start chat

[0048] User: Enter a message such as "Hello" in the chat screen and send it.

[0049] Terminal: Sends a message to the server.

[0050] Server: Receives messages and parses them using a natural language processing engine. Based on the analysis results, it generates an appropriate response such as "Hello, how are you?" and sends it to the terminal.

[0051] Terminal: Displays the character's reply to the user.

[0052] 4. Story branching and event occurrences

[0053] Server: Analyzes chat content and determines the direction of the story. For example, if the user asks "What are you doing today?", the server selects a scenario where the character answers "I'm planning to go shopping."

[0054] Terminal: Displays new scenes and events received from the server.

[0055] User: Can select messages and options to advance the conversation.

[0056] 5. Intimacy level calculation and special scenario display

[0057] Server: Calculates intimacy based on user input and choices. For example, if a user sends a friendly message to a character, intimacy increases.

[0058] Terminal: When intimacy exceeds a certain threshold, data for special scenarios and events will be loaded and displayed.

[0059] Users can experience special scenarios and deepen their relationships with the characters.

[0060] Specific example

[0061] Making a date from the first meeting

[0062] 1. Login and Character Selection

[0063] The user logs into the system from their terminal and selects "Beautiful Girl Character A" from the displayed list. The server receives this selection, loads the initial information for "Beautiful Girl Character A," and sends the chat screen to the user's terminal.

[0064] 2. Start the conversation

[0065] The user sends the message "Hello, nice to meet you." The server receives this message, parses it using natural language processing, generates the response "Hello, how are you?", and sends it to the terminal. The terminal then displays this response to the user.

[0066] 3. Story progression and event occurrence

[0067] When the user replies, "I'm fine, what are you doing today?", the server analyzes this and branches the scenario to, "I'm planning to go shopping. Would you like to come along?". This response is sent to the terminal, and if the user replies, "Of course, I'll go with you," a date event occurs.

[0068] 4. Intimacy and Special Scenarios

[0069] During a dating event, if the user selects the option "Tell me what you like?", the server will increase the intimacy level. When the intimacy level reaches a certain threshold, "Beautiful Girl Character A" will propose a special scenario to the user, which will then be displayed on the device. The user can then enjoy this special scenario.

[0070] The above are specific examples of embodiments for carrying out the invention. This system allows users to enjoy individual story experiences and emotional continuity through conversations and choices with characters.

[0071] The following describes the processing flow.

[0072] Step 1:

[0073] Terminal: The user launches the application from their terminal, and the login screen is displayed. The user enters their username and password and clicks the "Login" button.

[0074] Step 2:

[0075] Terminal: Sends the entered username and password to the server.

[0076] Step 3:

[0077] Server: The server compares the received username and password with the database and generates an authentication result. If authentication is successful, it prepares the data for the home screen. If authentication fails, it generates an error message.

[0078] Step 4:

[0079] Server: Sends the authentication result (success or failure) to the terminal.

[0080] Step 5:

[0081] Device: If authentication is successful, the home screen will be displayed. If authentication fails, an error message will be displayed on the login screen, prompting the user to re-enter their information.

[0082] Step 6:

[0083] Device: The user selects their preferred character from the character selection menu on the home screen.

[0084] Step 7:

[0085] Terminal: Sends information about the selected character to the server.

[0086] Step 8:

[0087] Server: Loads initial character information and background information from the database and generates data for starting a chat.

[0088] Step 9:

[0089] Server: Sends data to the terminal to start the chat.

[0090] Step 10:

[0091] Terminal: The chat screen appears, and the initial message of the selected character is displayed to the user.

[0092] Step 11:

[0093] User: Type your message in the chat screen and click the "Send" button.

[0094] Step 12:

[0095] Terminal: Sends user messages to the server.

[0096] Step 13:

[0097] Server: The server analyzes received messages using a natural language processing engine and generates an appropriate response based on the analysis results.

[0098] Step 14:

[0099] Server: Sends the generated response to the terminal.

[0100] Step 15:

[0101] Terminal: Displays the received response message to the user.

[0102] Step 16:

[0103] Server: Analyzes user conversations and determines the direction of the story's progression. Loads new scene and event data as needed.

[0104] Step 17:

[0105] Server: Sends necessary scene and event data to the terminal.

[0106] Step 18:

[0107] Terminal: Displays data from received scenes and events.

[0108] Step 19:

[0109] Terminal: The user selects options for new scenes or events and sends the selection results to the server.

[0110] Step 20:

[0111] Server: Updates intimacy based on the selection results. If intimacy reaches a certain condition, loads a special scenario.

[0112] Step 21:

[0113] Server: Sends a special scenario to the terminal based on the level of intimacy.

[0114] Step 22:

[0115] Terminal: Displays special scenarios to the user.

[0116] Step 23:

[0117] User: Experience a special scenario and continue the conversation.

[0118] Step 24:

[0119] Server: Periodically saves user progress data and saves all data at the end of the session.

[0120] Step 25:

[0121] User: Click the logout button to end the session.

[0122] Step 26:

[0123] Terminal: Sends a logout request to the server.

[0124] Step 27:

[0125] Server: Processes the received logout request, terminates the session, and checks the saved data.

[0126] Step 28:

[0127] Terminal: Display a logout completion screen to the user.

[0128] The above describes the specific processing flow in the system of the present invention. This system allows users to enjoy dynamically unfolding stories and events through conversations with characters.

[0129] (Example 1)

[0130] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0131] Traditional character conversation systems often result in a fixed user experience with limited dynamic branching of storylines and events. Furthermore, they restrict the calculation of intimacy levels based on conversation content and the display of special scenarios. This leads to limited user interaction and a lack of long-term appeal.

[0132] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0133] In this invention, the server includes means for a user to log in and authenticate from a terminal; means for a story to branch and events to occur through conversations with a virtual character selected by the user; means for the system to calculate intimacy based on the conversation content and display a special scenario when certain conditions are met; means for analyzing the chat content using natural language processing with a generation AI model and generating an appropriate response based on the analysis results; and means for dynamically loading and displaying scenarios in response to changes in intimacy. This diversifies user interaction and enables dynamic and seamless story progression and event occurrence.

[0134] A "terminal" is a physical or virtual device used by a user to access and operate a system.

[0135] A "server" is a centralized computer system that receives, processes, and analyzes user input information, and then returns the results to the terminal.

[0136] "Logging in" is a series of processes in which a user enters authentication information into a system and verifies and grants access rights.

[0137] "Authentication" is the process of verifying whether the entered user information matches the registered data.

[0138] A "virtual character" is a digital persona designed to interact with users within a system.

[0139] "Story branching" refers to the process where different storylines or events are generated depending on the user's choices and conversations.

[0140] An "event" is a specific action or occurrence that happens during the story and influences the user's choices.

[0141] "Intimacy level" is an indicator that measures the depth of the relationship between the user and the virtual character, and is calculated as points based on specific actions and messages within the system.

[0142] A "special scenario" refers to a specific story or event that is only displayed when certain conditions are met regarding intimacy.

[0143] A "generative AI model" is an engine that uses artificial intelligence to perform natural language processing, and is responsible for analyzing user messages and generating appropriate responses.

[0144] "Natural language processing" is a technology that enables computers to understand and process human language.

[0145] "Analysis results" refer to the information and conclusions obtained when a generative AI model processes user input.

[0146] "Dynamically loading and displaying scenarios" refers to the process by which the system loads new scenarios and events in real time in response to user actions and displays them on the device.

[0147] This embodiment is a system that dynamically generates and branches a story and triggers events through interactive conversations between the user and a virtual character. This system functions through the cooperation of a terminal, a server, and the user.

[0148] The server manages authentication information when a user logs in from a terminal to access the system. This authentication information is stored in a database (e.g., MySQL®) and verified using SQL queries. After the user enters their username and password, the server authenticates them and establishes the login status.

[0149] After logging in, users can access a character selection screen on their device. For example, multiple virtual characters may be displayed as thumbnails, allowing the user to select one. The initial data of the selected character (name, profile, background story, etc.) is sent from the server to the device in JSON format. The device then uses this data to configure and display the chat screen to the user.

[0150] At the start of a chat, the user types and sends a text message. This message is sent from the terminal to the server, where it is analyzed using a generative AI model (e.g., GPT-4®). The generative AI model analyzes the message content and generates an appropriate response. This response is sent to the terminal in JSON format, and the terminal displays the character's reply to the user.

[0151] The story branches based on the conversation, and events occur. For example, if the user asks, "What are you doing today?", the server analyzes this and selects the scenario, "I'm planning to go shopping." Next, information about this scenario is sent to the device, and a new storyline is displayed.

[0152] During the conversation, the system calculates intimacy based on the user's responses. Intimacy increases when favorable messages or specific choices are selected. If this intimacy exceeds a certain threshold, the server loads and sends a special scenario or event to the device. The device then displays the special scenario, allowing the user to enjoy a new experience.

[0153] For example, if a user inputs "Hello, beautiful girl character A. What are your plans for today?", the server uses a generative AI model to analyze the scenario and generate a response such as "I'm planning to go shopping. Would you like to come along?". This response is sent to the terminal and displayed to the user. If the user replies "Of course, I'll go with you," a date event occurs, and the intimacy level between the user and the virtual character increases.

[0154] This system allows users to enjoy personalized story experiences and emotional continuity through conversations with virtual characters.

[0155] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0156] Step 1: User Login

[0157] User: The user opens the login screen from their device and enters their username and password. The login process is completed by pressing the login button.

[0158] Input: Username, Password

[0159] Terminal: Sends the entered username and password to the server.

[0160] Server: The server compares the received username and password with the authentication information in the database (e.g., MySQL). It uses an SQL query to perform the comparison and obtain the authentication result. For example, it executes the query "SELECT user_id FROM users WHERE username='example' AND password='123456'".

[0161] Output: Authentication result (success or failure)

[0162] Server: If authentication is successful, the server sends the user ID and home screen data (in JSON format) to the device. For example, it may be in the format "{"status":"success","user_id":4,"home_data":{"background":"blue","notifications":"on"}}".

[0163] Step 2: Character Selection

[0164] Terminal: After successful authentication, the user is shown a character selection screen. A list of selectable characters (icons with images and names) is displayed.

[0165] User: Select your preferred character from the list and press the select button to complete the input.

[0166] Input: ID of the selected character

[0167] Terminal: Sends the ID of the selected character to the server.

[0168] Server: Retrieves the character's initial data from the database and sends it to the terminal in JSON format. For example, "{"character_id":"A","name":"Beautiful Girl Character A","profile":"Profile Information"}".

[0169] Output: Initial character data (JSON format)

[0170] Terminal: Based on the initial data received, configures and displays the chat screen to the user.

[0171] Step 3: Start chatting

[0172] User: Enter a message such as "Hello" in the chat screen and press the send button to complete the input.

[0173] Input: User message

[0174] Terminal: Sends the entered message to the server.

[0175] Server: Analyzes received messages using a generation AI model (e.g., GPT-4). Generates a prompt and inputs it into the generation AI model. For example, "The user typed 'hello'. Please generate an appropriate response."

[0176] Server: Sends the generated response to the terminal in JSON format. For example, "{"response":"Hello, how are you?"}".

[0177] Output: Generated response (JSON format)

[0178] Terminal: Displays the character's reply message to the user.

[0179] Step 4: Story branching and event occurrence

[0180] Server: Continuously analyzes chat content to determine the direction of the story. If a user types "What are you doing today?", it selects a scenario such as "I'm planning to go shopping" based on pre-set conditions.

[0181] Input: User message, story progression conditions

[0182] Server: Sends the data for the selected scenario to the terminal in JSON format. For example, "{"scene":"I'm planning to go shopping. Would you like to join me?"}".

[0183] Output: Scenario data (JSON format)

[0184] Terminal: Displays new scenes and events received from the server. For example, it displays a scene of a character going shopping using images and text.

[0185] User: Selects responses and options in response to displayed scenes and events, and inputs information to advance the conversation.

[0186] Step 5: Intimacy level calculation and special scenario display

[0187] Server: Applies an algorithm that calculates intimacy based on user input and choices. For example, it adds intimacy points for favorable messages.

[0188] Input: User's message and choices

[0189] Server: When intimacy exceeds a certain threshold, load data for special scenarios or events and send it to the terminal in JSON format. For example, "{'special_scenario':'Special Dating Event'}".

[0190] Output: Data for a special scenario (JSON format)

[0191] Terminal: Displays received special scenarios. For example, it provides the user with a description of a special date scene using diagrams and text.

[0192] User: Experience special scenarios and advance through new choices and conversations to deepen your relationship with the virtual character.

[0193] (Application Example 1)

[0194] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0195] Conventional interactive storytelling systems have limited conversations between users and characters, failing to adequately achieve natural interaction and increased intimacy. Furthermore, seamless, real-time branching of the story based on user choices is difficult, limiting the potential for improving the user experience. This invention solves these problems, providing more natural conversations and dynamic, real-time story branching, while also enabling the calculation of user-character intimacy and the presentation of special scenarios based on that.

[0196] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0197] In this invention, the server includes means for branching the story and triggering events through conversations with a character selected by the user; means for the system to calculate intimacy based on the conversation content and display a special scenario when certain conditions are met; means for analyzing the chat content using natural language processing and generating an appropriate response based on the analysis results; means for providing interactive storytelling through conversations between the user and a character in an application installed on a smartphone, smart glasses, head-mounted display, or robot; and means for generating and displaying new scenes and events in real time based on the conversation content and story progression. As a result, the user can enjoy natural conversations with the character and experience special scenarios according to their intimacy level, enabling a richer interactive story experience.

[0198] A "user" is someone who uses the system to engage in conversations and storytelling with characters.

[0199] A "character" is a virtual character that the user selects within the system and interacts with through conversations and stories.

[0200] "Story branching" is a mechanism where the direction of the story changes depending on the user's choices and the content of the conversations.

[0201] An "event" refers to a specific scene or occurrence that happens during the progression of the story.

[0202] "Intimacy level" is a numerical representation of the depth of the relationship between the user and the character.

[0203] A "special scenario" refers to a special storyline or scene that is only displayed when the level of intimacy reaches certain conditions.

[0204] "Natural language processing" is a technology that enables computers to understand and analyze user messages.

[0205] An "appropriate response" is a reply message to the user that the natural language processing engine generates based on its analysis results.

[0206] A "smartphone" is a portable device capable of connecting to the internet and running various applications.

[0207] "Smart glasses" are glasses-type devices equipped with the ability to display information and run applications.

[0208] A "head-mounted display" is a display device that is worn on the head.

[0209] A "robot" is a mechanical device that is capable of interacting with and being operated by a user.

[0210] "Interactive storytelling" is a technique in which the user and the system interact to advance the narrative.

[0211] "Real-time" refers to the temporality in which a system processes information instantly and responds immediately to user actions and conversations.

[0212] A "new scene" is a new moment that is generated during the progression of the story based on the user's choices and conversations.

[0213] The system for carrying out this invention provides interactive storytelling. Users can access the system using a smartphone or other device and converse with characters. Specific embodiments are described below.

[0214] 1. User login and authentication

[0215] Users access the login screen from a device such as a smartphone and enter their username and password.

[0216] The server verifies the entered authentication information against the authentication database. If authentication is successful, it sends the home screen data to the user's device; otherwise, it sends an error message.

[0217] 2. Character Selection

[0218] After successful authentication, the user selects their preferred character from the character selection screen that appears.

[0219] The server loads initial data and background information corresponding to the selected character and sends the chat screen to the user's device.

[0220] 3. Start chat

[0221] Users type messages in the chat screen and send messages such as "Hello."

[0222] The server receives the message and parses it using a natural language processing engine (e.g., spaCy).

[0223] Based on the analysis results, an appropriate response is generated and sent to the user's device.

[0224] The device displays the generated response to the user.

[0225] 4. Story branching and event occurrences

[0226] The server analyzes the user's chat content and determines the direction of the story's progression.

[0227] For example, if the user asks, "What are you doing today?", a scenario is selected in which the character replies, "I'm planning to go shopping." Information about this new scene or event is then sent to the user's device.

[0228] Users can advance the conversation by selecting messages and options based on the scene or event.

[0229] 5. Intimacy level calculation and special scenario display

[0230] The server calculates the level of intimacy based on the user's input and choices.

[0231] For example, when a user sends a positive message, their intimacy level increases. If the intimacy level exceeds a certain threshold, data for special scenarios or events is loaded and sent to the user's device.

[0232] Users can experience special scenarios and deepen their relationships with the characters.

[0233] Hardware and software to be used

[0234] The following hardware and software will be used to implement this system.

[0235] Hardware: Smartphones, smart glasses, head-mounted displays, robots

[0236] Software: Flask (web framework), MongoDB (database), spaCy (natural language processing engine)

[0237] Specific example

[0238] Consider the case where the user types the message "I'm fine, what are you doing today?".

[0239] The server uses a natural language processing engine to analyze the message and generates a response, "I'm planning to go shopping. Would you like to join me?", which it then sends to the user's device.

[0240] Upon receiving this response, the user replies, "Of course, I'll go with you," and a date event occurs.

[0241] During a dating event, if the user selects "Tell me what you like?", the server will increase the intimacy level. When the intimacy level reaches certain conditions, a special scenario will be displayed.

[0242] Example of a prompt

[0243] The user sends the following message to the system:

[0244] "I'm fine, what are you doing today?"

[0245] This system allows users to enjoy natural conversations and interactive storytelling with characters, and also experience special scenarios that change depending on their level of intimacy with the characters.

[0246] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0247] Step 1:

[0248] The user enters their username and password on the login screen of their smartphone.

[0249] Input: Username and password

[0250] Output: Login request data

[0251] Specific action: The user device sends the entered authentication information to the server.

[0252] Step 2:

[0253] The server receives the login request data and compares it with the authentication database.

[0254] Input: Username and password

[0255] Output: Authentication result (success or failure)

[0256] Specific operation: If authentication is successful, the data from the home screen is sent to the user's device; if it fails, an error message is sent.

[0257] Step 3:

[0258] The user opens the character selection screen on the home screen and selects their preferred character.

[0259] Input: Character selection information

[0260] Output: Character selection data

[0261] Specific operation: The user device sends the selected character information to the server.

[0262] Step 4:

[0263] The server loads initial data and background information corresponding to the selected character and sends the chat screen data to the user's device.

[0264] Input: Character selection data

[0265] Output: Chat screen data

[0266] Specific operation: The server retrieves the character's initial data and background information from the database and generates the chat screen.

[0267] Step 5:

[0268] Users type and send messages in the chat screen.

[0269] Input: User message

[0270] Output: Chat message data

[0271] Specific action: The user device sends the entered message to the server.

[0272] Step 6:

[0273] The server receives chat message data and analyzes it using a natural language processing engine (such as spaCy).

[0274] Input: Chat message data

[0275] Output: Analysis results

[0276] Specific operation: The natural language processing engine analyzes the message and identifies the appropriate response.

[0277] Step 7:

[0278] The server generates an appropriate response based on the analysis results of the natural language processing engine and sends that response to the user device.

[0279] Input: Parsing result

[0280] Output: Response message

[0281] Specific operation: Based on the parsing result, generate a response message and send it to the user device.

[0282] Step 8:

[0283] The user checks the response of the character displayed on the device and continues the conversation.

[0284] Input: Response message

[0285] Output: User's reaction message

[0286] Specific operation: Enter the next message based on the response displayed on the chat screen.

[0287] Step 9:

[0288] The server determines the progress direction of the story based on the user's input, and new scenes and events occur.

[0289] Input: User's reaction message

[0290] Output: Data of new scenes and events

[0291] Specific operation: The system re-analyzes the chat content, generates a branching scenario, and displays it on the user device.

[0292] Step 10:

[0293] The server calculates the intimacy based on the user's input and choices, and when a specific threshold is exceeded, loads the data for displaying special scenarios and events.

[0294] Input: Conversation history and choice data

[0295] Output: Intimacy and special scenario data

[0296] Specific operation: The system calculates intimacy based on an intimacy algorithm and generates a special scenario if the conditions are met.

[0297] Step 11:

[0298] Users can experience special scenarios and deepen their relationships with the characters.

[0299] Input: Special scenario data

[0300] Output: User experience record

[0301] Specific operation: The content of the special scenario is displayed on the user's device, and the user enjoys it.

[0302] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0303] This invention combines an emotion engine with a system in which the story branches and events occur through conversations with a character selected by the user. This system uses natural language processing and emotion recognition technology to provide a more dynamic and emotional story development.

[0304] Specific program processing

[0305] 1. User Login and Authentication

[0306] Terminal: The user launches the application from their terminal, and the login screen is displayed. They enter their username and password and click the "Login" button.

[0307] Terminal: Transmit the entered username and password to the server.

[0308] Server: Compare the received username and password with the database and generate an authentication result. If authentication is successful, prepare the data for the home screen. If authentication fails, generate an error message.

[0309] Server: Transmit the authentication result (success or failure) to the terminal.

[0310] Terminal: If authentication is successful, display the home screen. If authentication fails, display an error message on the login screen and prompt the user to re-enter.

[0311] 2. Character Selection

[0312] Terminal: When authentication is successful, display the character selection screen.

[0313] User: Select a favorite character from the list.

[0314] Terminal: Transmit the selection information to the server.

[0315] Server: Load the initial information and background information of the selected character from the database and generate data for starting the chat.

[0316] Server: Transmit the data for starting the chat to the terminal.

[0317] Terminal: The chat screen is displayed, and the initial message of the selected character is shown to the user.

[0318] 3. Chat Start

[0319] User: Enter a message on the chat screen and click the "Send" button.

[0320] Terminal: Sends user messages to the server.

[0321] Server: Analyzes received messages using a natural language processing engine and generates appropriate responses based on the analysis results. Furthermore, it analyzes the user's emotions using an emotion engine.

[0322] Server: Based on the results of natural language processing and emotion recognition, it generates a refined response and sends it to the terminal.

[0323] Terminal: Displays the character's reply to the user.

[0324] 4. Story branching and event occurrences

[0325] Server: Analyzes chat content and emotion recognition results to determine the direction of the story's progression. For example, if "joy" is recognized from the user's message, it selects a scenario in which the character proposes a "fun event." Based on the emotion recognition results, it seamlessly adjusts events and scenarios.

[0326] Terminal: Displays new scenes and events received from the server.

[0327] User: Can select messages and options to advance the conversation.

[0328] 5. Intimacy level calculation and special scenario display

[0329] Server: Calculates intimacy based on user input, choices, and emotion recognition results. For example, if a user sends a positive message to a character, intimacy increases.

[0330] Terminal: When intimacy exceeds a certain threshold, data for special scenarios and events will be loaded and displayed.

[0331] Users can experience special scenarios and deepen their relationships with the characters.

[0332] Specific example

[0333] Making a date from the first meeting

[0334] 1. Login and Character Selection

[0335] The user logs into the system from their terminal and selects "Character A" from the displayed list. The server receives this selection, loads the initial information for "Character A," and sends the chat screen to the user's terminal.

[0336] 2. Start the conversation

[0337] The user sends the message "Hello, nice to meet you." The server receives this message, analyzes it using natural language processing and an emotion engine, generates the response "Hello, how are you?", and sends it to the terminal. The terminal then displays this response to the user.

[0338] 3. Story progression and event occurrence

[0339] When the user asks, "What are you doing today?", the server analyzes this and, if the emotion engine recognizes an interest, selects a scenario that suggests, "I'm planning to go shopping. Would you like to join me?" This response is sent to the device, and if the user replies, "Of course, I'll go with you," a date event occurs.

[0340] 4. Intimacy and Special Scenarios

[0341] During a dating event, if the user selects the option "Tell me what you like?", the server increases the intimacy level. Furthermore, if the emotion engine detects the user's "enjoyment," a special dating scenario will be loaded, which the user can then experience.

[0342] The above are specific examples of embodiments for carrying out the invention. This system allows users to enjoy individual story experiences and emotional continuity through conversations and choices with characters. The introduction of an emotion engine enables dynamic story development that responds to the user's emotions, providing a richer user experience.

[0343] The following describes the processing flow.

[0344] Step 1:

[0345] Terminal: The user launches the application from their terminal, and the login screen is displayed. They enter their username and password and click the "Login" button.

[0346] Step 2:

[0347] Terminal: Sends the entered username and password to the server.

[0348] Step 3:

[0349] Server: The server compares the received username and password with the database and generates an authentication result. If authentication is successful, it prepares the data for the home screen. If authentication fails, it generates an error message.

[0350] Step 4:

[0351] Server: Sends the authentication result (success or failure) to the terminal.

[0352] Step 5:

[0353] Device: If authentication is successful, the home screen will be displayed. If authentication fails, an error message will be displayed on the login screen, prompting the user to re-enter their information.

[0354] Step 6:

[0355] Device: The user selects their preferred character from the character selection menu on the home screen.

[0356] Step 7:

[0357] Terminal: Sends information about the selected character to the server.

[0358] Step 8:

[0359] Server: Loads initial character information and background information from the database and generates data for starting a chat.

[0360] Step 9:

[0361] Server: Sends data to the terminal to start the chat.

[0362] Step 10:

[0363] Terminal: The chat screen appears, and the initial message of the selected character is displayed to the user.

[0364] Step 11:

[0365] User: Type your message in the chat screen and click the "Send" button.

[0366] Step 12:

[0367] Terminal: Sends user messages to the server.

[0368] Step 13:

[0369] Server: Analyzes received messages using a natural language processing engine. Furthermore, analyzes the user's emotions using an emotion engine.

[0370] Step 14:

[0371] Server: Generates tailored responses based on the results of natural language processing and sentiment recognition.

[0372] Step 15:

[0373] Server: Sends the generated response to the terminal.

[0374] Step 16:

[0375] Terminal: Displays the received response message to the user.

[0376] Step 17:

[0377] Server: Analyzes the user's conversation content and emotion recognition results to determine the direction of the story's progression. For example, if "joy" is recognized from the user's message, it selects a scenario in which the character proposes a "fun event."

[0378] Step 18:

[0379] Server: Loads the necessary scene and event data.

[0380] Step 19:

[0381] Server: Sends loaded scene and event data to the terminal.

[0382] Step 20:

[0383] Terminal: Displays data from received scenes and events.

[0384] Step 21:

[0385] User: Select options for new scenes or events and send the selection results to the server.

[0386] Step 22:

[0387] Server: Updates intimacy based on selection results. Dynamically adjusts intimacy based on the user's emotion recognition results.

[0388] Step 23:

[0389] Server: When intimacy exceeds a certain threshold, it loads data for special scenarios or events.

[0390] Step 24:

[0391] Server: Sends data for special scenarios and events to terminals.

[0392] Step 25:

[0393] Terminal: Displays special scenarios and events.

[0394] Step 26:

[0395] User: Experience special scenarios and events, and continue the conversation.

[0396] Step 27:

[0397] Server: Periodically saves user progress data and saves all data at the end of the session.

[0398] Step 28:

[0399] User: Click the logout button to end the session.

[0400] Step 29:

[0401] Terminal: Sends a logout request to the server.

[0402] Step 30:

[0403] Server: Processes the received logout request, terminates the session, and checks the saved data.

[0404] Step 31:

[0405] Terminal: Display a logout completion screen to the user.

[0406] The above describes the specific processing flow in the system of the present invention. This system allows users to enjoy dynamically unfolding stories and events through conversations and choices with characters. The introduction of an emotion engine enables dynamic story development in response to the user's emotions, providing a richer user experience. For example, when a user asks, "What do you like?", the emotion engine recognizes "affection" and displays a special date scenario.

[0407] (Example 2)

[0408] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0409] Traditional dialogue systems have a problem where conversations with characters are formal, and dynamic story branching and event generation that respond to user emotions and choices cannot be realized. Furthermore, the lack of intimacy level calculations that reflect user emotions and the display of special scenarios have resulted in a limited user experience.

[0410] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0411] In this invention, the server includes means for branching the story and causing events through conversations with a character selected by the user; means for the system to calculate intimacy based on the conversation content and emotion recognition results, and display a special scenario when certain conditions are met; means for analyzing the chat content using natural language processing and generating an appropriate response based on the analysis results and emotion recognition results; means for seamlessly adjusting the story branching and event occurrences based on the user's emotion recognition results; and means for dynamically generating and displaying special scenarios and events based on the calculation of intimacy. This enables a dynamic and emotional story development and a rich user experience based on the user's emotions and choices.

[0412] A "user" is someone who utilizes this system and experiences the story progression and events through interaction with characters.

[0413] A "character" is a virtual person or entity that interacts with the user within the system, and has the role of responding dynamically according to the user's choices and emotional recognition results.

[0414] "Conversation" refers to the exchange of messages between the user and the character, and is generally conducted in text format.

[0415] "Story branching" refers to the change in the direction of the narrative based on the user's choices and emotional responses, resulting in different events and scenarios.

[0416] An "event" refers to a specific scene or occurrence within a story, and its content may change depending on the user's choices and emotions.

[0417] "Intimacy level" is a numerical or metric representation of the depth of the relationship between the user and the character, and it fluctuates in response to the user's actions and statements.

[0418] A "special scenario" refers to a special scene or story that appears when the intimacy level meets certain conditions, and it contains unique content that differs from the regular scenario.

[0419] "Emotion recognition" is a technology that analyzes a user's input and actions to identify their emotions and determine their emotional state, such as joy, interest, or sadness.

[0420] "Natural language processing" is a technology that analyzes user messages to understand their meaning and intent, and involves text analysis and response generation.

[0421] A "response" is a message that a character returns in response to a message from the user, and is generated based on natural language processing and emotion recognition results.

[0422] Embodiments of this invention will be described in detail below. This system has a mechanism in which the story branches and events occur through conversations with a character selected by the user. In addition, it can calculate intimacy based on the conversation content and emotion recognition results, and display a special scenario when certain conditions are met.

[0423] System Overview

[0424] First, let me explain the system's overview. This system consists of three components: a server, a terminal, and a user, all of which interact with each other. The server is responsible for key processes such as story branching, event generation, emotion recognition, intimacy calculation, and natural language processing. The terminal provides the user's interface and communicates with the server. The user operates the terminal to advance conversations with the characters.

[0425] Hardware and software to be used

[0426] This system uses the following hardware and software:

[0427] Server hardware: A server with a high-performance processor, sufficient memory, and fast database access capabilities.

[0428] Database: A relational database management system (RDBMS) such as MySQL.

[0429] Natural language processing engine: High-performance generative AI models such as GPT-4.

[0430] Emotion recognition engine: Emotion analysis tools such as Emotion AI.

[0431] Details of the program processing

[0432] This section provides a detailed explanation of the program processing performed by the system. The specific processing steps for each stage are shown below.

[0433] User login and authentication

[0434] 1. User: Launch the application from your device and enter your username and password on the login screen.

[0435] 2. Terminal: Sends the entered information to the server.

[0436] 3. Server: Matches the information with the database, generates an authentication result, and sends it to the terminal.

[0437] 4. Device: Displays the home screen or an error message based on the authentication result.

[0438] Character Selection

[0439] 1. Device: After successful authentication, the character selection screen will be displayed.

[0440] 2. User: Select a character from the list.

[0441] 3. Terminal: Sends selection information to the server.

[0442] 4. Server: Loads the character's initial information, generates data for starting a chat, and sends it to the terminal.

[0443] 5. Terminal: Displays the chat screen and the character's initial message.

[0444] Start chat

[0445] 1. User: Enter your message.

[0446] 2. Terminal: Sends the entered message to the server.

[0447] 3. Server: Analyzes messages using a natural language processing engine and analyzes emotions using an emotion engine.

[0448] 4. Server: Generates a response and sends it to the terminal.

[0449] 5. Terminal: Displays the character's reply.

[0450] Story branching and event occurrences

[0451] 1. Server: Analyzes chat content and emotion recognition results to determine the direction of the story's progression.

[0452] 2. Terminal: Displays new scenes and events.

[0453] 3. User: Can continue the conversation.

[0454] Intimacy level calculation and special scenario display

[0455] 1. Server: Calculates intimacy levels based on user statements and emotion recognition results.

[0456] 2. Device: When the intimacy level exceeds a certain threshold, a special scenario will be displayed.

[0457] 3. User: Experience a special scenario.

[0458] Specific example

[0459] Specific examples are given below.

[0460] Making a date from the first meeting

[0461] The user logs into the system from their device and selects "Character A". The server receives the selection information, loads the initial information for "Character A", and sends the chat screen to the user's device. When the user sends the message "Hi, nice to meet you", the server analyzes it and generates the response "Hi, how are you?" which is sent to the device. When the user continues with "What are you doing today?", the server analyzes this and, if it recognizes the user's "interest", selects a scenario that suggests "I'm planning to go shopping. Would you like to come along?". This response is sent to the device, and if the user replies "Of course, I'll go with you", a date event occurs. During the date event, if the user selects "Tell me what you like?", the server increases the intimacy level. Furthermore, if the emotion engine detects the user's "enjoyment", a special date scenario is loaded, which the user can experience.

[0462] Example of a prompt

[0463] "Ask Character A what they're doing today and guide them towards a date event."

[0464] In this embodiment of the invention, users can enjoy a dynamic and emotional storyline through emotional interaction with characters. By using an emotion recognition engine and a natural language processing engine together, more personalized responses become possible, significantly improving the user experience.

[0465] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0466] The processing flow of this system's program (explained in steps)

[0467] Step 1: Display the login screen

[0468] Input: The user launches the application.

[0469] Action: The device launches the application and displays the login screen.

[0470] Output: The login screen is displayed to the user.

[0471] Step 2: Enter and submit user authentication information.

[0472] Input: The user enters their username and password and clicks the login button.

[0473] Operation: The terminal sends the entered username and password to the server. The data is encrypted before transmission.

[0474] Output: User authentication information is sent to the server.

[0475] Step 3: Authentication Process

[0476] Input: The username and password received by the server.

[0477] Operation: The server connects to the database and verifies the username and password. If successful, it generates an authentication token and prepares data for the home screen. If it fails, it prepares an error message.

[0478] Output: Authentication results are generated, and an authentication token or error message is generated.

[0479] Step 4: Submit and display authentication results

[0480] Input: Authentication result and authentication token or error message.

[0481] Operation: The server sends the authentication result to the device. Based on the received data, the device displays either the home screen or an error message.

[0482] Output: The home screen is displayed, or an error message is displayed on the login screen.

[0483] Step 5: Display the character selection screen.

[0484] Input: User who successfully authenticated.

[0485] Action: The device displays the character selection screen.

[0486] Output: The character selection screen is displayed to the user.

[0487] Step 6: Submit character selection information

[0488] Input: The user selects a character and clicks the select button.

[0489] Operation: The terminal sends the ID of the selected character to the server.

[0490] Output: Character ID information is sent to the server.

[0491] Step 7: Loading character information

[0492] Input: The character ID received by the server.

[0493] Operation: The server loads initial information for the selected character from the database and generates data to start the chat.

[0494] Output: Data for starting a chat is generated.

[0495] Step 8: Send and display data to start the chat.

[0496] Input: Data to start the chat.

[0497] Operation: The server sends data to the terminal to initiate a chat. The terminal receives the data, displays the chat screen, and shows the character's initial message.

[0498] Output: The chat screen and the character's initial message are displayed.

[0499] Step 9: Enter and send the user message.

[0500] Input: The user enters a message and clicks the send button.

[0501] Operation: The terminal sends the entered message to the server.

[0502] Output: The user message is sent to the server.

[0503] Step 10: Message Analysis

[0504] Input: User messages received by the server.

[0505] Operation: The server analyzes messages using a natural language processing engine (such as GPT-4). It also analyzes the user's emotions using an emotion recognition engine (such as Emotion AI).

[0506] Output: Message analysis results and sentiment recognition results are generated.

[0507] Step 11: Generating a response message

[0508] Input: Message analysis results and sentiment recognition results.

[0509] Operation: Based on these results, the server generates an appropriate response message.

[0510] Output: A response message is generated.

[0511] Step 12: Sending and displaying the response message

[0512] Input: Response message.

[0513] Operation: The server sends a response message to the terminal. The terminal receives the message and displays it on the chat screen.

[0514] Output: The response message is displayed on the chat screen.

[0515] Step 13: Deciding on a story branch

[0516] Input: Chat content and sentiment recognition results.

[0517] Operation: The server determines the direction of the story's progression based on this data.

[0518] Output: The results of the story branching are generated.

[0519] Step 14: Generating new scenes and events

[0520] Input: The result of the story branching.

[0521] Operation: The server generates data for new scenes and events and sends it to the terminal.

[0522] Output: New scene and event data is generated.

[0523] Step 15: Displaying new scenes and events

[0524] Input: Data for new scenes or events.

[0525] Operation: The terminal displays the received data.

[0526] Output: New scenes or events are displayed on the terminal.

[0527] Step 16: Calculating Intimacy Level

[0528] Input: User input, choices, and sentiment recognition results.

[0529] Operation: The server calculates intimacy based on this data.

[0530] Output: An intimacy value is generated.

[0531] Step 17: Generating a Special Scenario

[0532] Input: Intimacy level value.

[0533] Operation: When intimacy exceeds a certain threshold, the server generates special scenarios or events.

[0534] Output: Data for special scenarios and events is generated.

[0535] Step 18: Displaying Special Scenarios

[0536] Input: Data for special scenarios or events.

[0537] Operation: The terminal displays the received data.

[0538] Output: Special scenarios or events will be displayed on the terminal.

[0539] The above describes the detailed processing steps and specific operation of this system's program.

[0540] (Application Example 2)

[0541] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0542] Traditional interactive storytelling systems were unable to dynamically branch stories in response to user emotions, instead relying on fixed story patterns. This resulted in a limited user experience and a lack of entertainment value and immersion. Furthermore, the calculation of intimacy levels and the resulting display of special scenarios were also fixed, making real-time, emotion-based responses difficult.

[0543] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0544] In this invention, the server includes means for branching the story and triggering events through conversations with a character selected by the user; means for the system to calculate intimacy based on the conversation content and display a special scenario when certain conditions are met; means for analyzing the chat content using natural language processing and generating an appropriate response based on the analysis results; means for recognizing the user's emotions and dynamically branching the story according to those emotions; and means for seamlessly displaying special events and scenarios based on intimacy and emotions. This enables flexible and dynamic story development that responds to the user's emotions.

[0545] A "user" is someone who uses the system to enjoy content through conversations with characters.

[0546] A "character" is a virtual entity or person that a user can select to engage in conversation with.

[0547] A "story" refers to a series of events or occurrences that unfold through conversations with characters.

[0548] "Branching" refers to the point where the direction of the story changes depending on the user's choices and emotions.

[0549] An "event" refers to a specific occurrence or action that takes place within a story.

[0550] "Intimacy level" is an indicator that shows the depth of the relationship between the user and the character.

[0551] A "special scenario" is a scenario that appears when certain conditions are met, and is different from the normal storyline.

[0552] "Natural language processing" is the technology that uses computers to understand, analyze, and generate human language.

[0553] "Emotions" refer to the user's feelings or emotional state.

[0554] "Emotion recognition" is a technology that analyzes and identifies emotions from a user's words and actions.

[0555] "Dynamic" refers to the property of changing or adapting in real time according to the situation.

[0556] "Seamlessly" refers to something that is continuous and uninterrupted.

[0557] The system of this invention provides a mechanism in which the story branches and events occur through conversations with characters selected by the user. The program to realize this is structured as follows.

[0558] The system requires a smartphone and a server as hardware. The smartphone is the device that the user directly interacts with, while the server processes and stores various types of data. As for software, the "transformers" library, a natural language processing engine, is used. This library is based on a generative AI model and uses a "sentiment-analysis" model for emotion recognition.

[0559] When a user logs into the system using their smartphone, the server verifies the user's credentials against the database and performs authentication. If authentication is successful, a character selection screen appears on the device. The user selects their preferred character, and the server loads the data for the selected character and prepares to start chatting.

[0560] When a user types and sends a message to a character, the message is sent from the terminal to the server. The server uses the "transformers" library to parse the message and perform emotion recognition. Based on the results, it generates an appropriate response and sends it to the terminal. The terminal then displays this response to the user.

[0561] Based on the chat content and the results of emotion recognition, the server determines the branching of the story. For example, if joy is recognized from the user's message, it selects a scenario that suggests a pleasant event. It also calculates intimacy and, if it exceeds a certain threshold, loads a special scenario and presents it to the user.

[0562] As a concrete example, consider a scenario where a user logs in with the username "example_user" and password "123456," and selects "Character A." When the user sends the message "I'm relieved that work is over," the server recognizes this as "joy" and generates a response asking, "Would you like to share something fun?" This system enables flexible and dynamic story development that responds to the user's emotions, providing a rich user experience.

[0563] Furthermore, the following input can be considered as an example of a prompt statement.

[0564] User's input message: "Every day is tough..."

[0565] Processing result: "Did something upsetting happen?"

[0566] The above describes the embodiments for carrying out this invention.

[0567] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0568] Step 1:

[0569] The user launches the application from their smartphone, and the login screen appears. They enter their username and password and click the "Login" button.

[0570] Input: Username, Password

[0571] Output: Authentication Request

[0572] Step 2:

[0573] The terminal sends the entered username and password to the server. The server compares the received authentication information with its database and generates an authentication result.

[0574] Input: Authentication Request

[0575] Output: Authentication result (success or failure)

[0576] Step 3:

[0577] If authentication is successful, the server prepares the home screen data and sends it to the device. If authentication fails, it generates an error message and sends it to the device. If authentication is successful, the device displays the home screen; if it fails, it displays the error message on the login screen.

[0578] Input: Authentication result

[0579] Output: Home screen data or error messages

[0580] Step 4:

[0581] The user moves from the home screen to the character selection screen and selects a character from the displayed list. The device then sends the selection information to the server.

[0582] Input: Character Selection

[0583] Output: Character selection information

[0584] Step 5:

[0585] Based on the received character selection information, the server loads the selected character's initial and background information from the database, generates data for starting a chat, and sends it to the terminal. The terminal displays the chat screen and shows the selected character's initial message.

[0586] Input: Character selection information

[0587] Output: Chat start data, initial message

[0588] Step 6:

[0589] The user types a message in the chat screen and clicks the "Send" button. The device sends the user's message to the server.

[0590] Input: Message

[0591] Output: Message sending request

[0592] Step 7:

[0593] The server analyzes the received message using the "transformers" library and performs emotion recognition using a generative AI model. Based on the analysis results and emotion recognition results, it generates an appropriate response message and sends it to the terminal.

[0594] Input: Message sending request

[0595] Output: Response message

[0596] Step 8:

[0597] The terminal displays the response message received from the server to the user. The user can then continue chatting.

[0598] Input: Response message

[0599] Output: Display of response message

[0600] Step 9:

[0601] The server analyzes the chat content and emotion recognition results to determine the direction of the story. For example, if it recognizes "joy" from the user's message, it selects a scenario in which the character proposes a "fun event." New scenes and events are generated and sent to the device according to specific conditions.

[0602] Input: Sentiment recognition result

[0603] Output: Event data, new scene data

[0604] Step 10:

[0605] The terminal displays new scenes and events received from the server to the user. The user can then enter further messages or options to continue the conversation.

[0606] Input: New scene data, event data

[0607] Output: Display new scene and events

[0608] Step 11:

[0609] The server calculates intimacy based on user input, choices, and emotion recognition results. For example, if a user sends a positive message to a character, intimacy increases. If intimacy exceeds a certain threshold, a special scenario or event is generated and sent to the terminal.

[0610] Input: User input, emotion recognition results

[0611] Output: Intimacy level, special scenario data

[0612] Step 12:

[0613] The terminal displays special scenarios and events sent from the server, and the user can continue to interact with them.

[0614] Input: Special Scenario Data

[0615] Output: Display of special scenarios

[0616] The above outlines the specific processing steps of this system's program.

[0617] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0618] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0619] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0620] [Second Embodiment]

[0621] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0622] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0623] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0624] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0625] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0626] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0627] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0628] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0629] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0630] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0631] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0632] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0633] This invention describes a system in which a story branches and events occur through conversations between a user and a character. This system uses natural language processing to analyze the user's messages and generate appropriate responses, and includes means for calculating intimacy levels and displaying special scenarios.

[0634] Specific program processing

[0635] 1. User Login and Authentication

[0636] Terminal: The user opens the login screen from their terminal and enters their username and password.

[0637] Server: The server compares the username and password received from the terminal with the authentication database. If authentication is successful, it sends the home screen data to the terminal. If authentication fails, it sends an error message.

[0638] 2. Character Selection

[0639] Terminal: Upon successful authentication, the character selection screen will be displayed.

[0640] User: Select your preferred character from the list.

[0641] Terminal: Sends selection information to the server.

[0642] Server: Loads the initial data and background information of the selected character and sends the chat screen data to the terminal.

[0643] 3. Start chat

[0644] User: Enter a message such as "Hello" in the chat screen and send it.

[0645] Terminal: Sends a message to the server.

[0646] Server: Receives messages and parses them using a natural language processing engine. Based on the analysis results, it generates an appropriate response such as "Hello, how are you?" and sends it to the terminal.

[0647] Terminal: Displays the character's reply to the user.

[0648] 4. Story branching and event occurrences

[0649] Server: Analyzes chat content and determines the direction of the story. For example, if the user asks "What are you doing today?", the server selects a scenario where the character answers "I'm planning to go shopping."

[0650] Terminal: Displays new scenes and events received from the server.

[0651] User: Can select messages and options to advance the conversation.

[0652] 5. Intimacy level calculation and special scenario display

[0653] Server: Calculates intimacy based on user input and choices. For example, if a user sends a friendly message to a character, intimacy increases.

[0654] Terminal: When intimacy exceeds a certain threshold, data for special scenarios and events will be loaded and displayed.

[0655] Users can experience special scenarios and deepen their relationships with the characters.

[0656] Specific example

[0657] Making a date from the first meeting

[0658] 1. Login and Character Selection

[0659] The user logs into the system from their terminal and selects "Beautiful Girl Character A" from the displayed list. The server receives this selection, loads the initial information for "Beautiful Girl Character A," and sends the chat screen to the user's terminal.

[0660] 2. Start the conversation

[0661] The user sends the message "Hello, nice to meet you." The server receives this message, parses it using natural language processing, generates the response "Hello, how are you?", and sends it to the terminal. The terminal then displays this response to the user.

[0662] 3. Story progression and event occurrence

[0663] When the user replies, "I'm fine, what are you doing today?", the server analyzes this and branches the scenario to, "I'm planning to go shopping. Would you like to come along?". This response is sent to the terminal, and if the user replies, "Of course, I'll go with you," a date event occurs.

[0664] 4. Intimacy and Special Scenarios

[0665] During a dating event, if the user selects the option "Tell me what you like?", the server will increase the intimacy level. When the intimacy level reaches a certain threshold, "Beautiful Girl Character A" will propose a special scenario to the user, which will then be displayed on the device. The user can then enjoy this special scenario.

[0666] The above are specific examples of embodiments for carrying out the invention. This system allows users to enjoy individual story experiences and emotional continuity through conversations and choices with characters.

[0667] The following describes the processing flow.

[0668] Step 1:

[0669] Terminal: The user launches the application from their terminal, and the login screen is displayed. The user enters their username and password and clicks the "Login" button.

[0670] Step 2:

[0671] Terminal: Sends the entered username and password to the server.

[0672] Step 3:

[0673] Server: The server compares the received username and password with the database and generates an authentication result. If authentication is successful, it prepares the data for the home screen. If authentication fails, it generates an error message.

[0674] Step 4:

[0675] Server: Sends the authentication result (success or failure) to the terminal.

[0676] Step 5:

[0677] Device: If authentication is successful, the home screen will be displayed. If authentication fails, an error message will be displayed on the login screen, prompting the user to re-enter their information.

[0678] Step 6:

[0679] Device: The user selects their preferred character from the character selection menu on the home screen.

[0680] Step 7:

[0681] Terminal: Sends information about the selected character to the server.

[0682] Step 8:

[0683] Server: Loads initial character information and background information from the database and generates data for starting a chat.

[0684] Step 9:

[0685] Server: Sends data to the terminal to start the chat.

[0686] Step 10:

[0687] Terminal: The chat screen appears, and the initial message of the selected character is displayed to the user.

[0688] Step 11:

[0689] User: Type your message in the chat screen and click the "Send" button.

[0690] Step 12:

[0691] Terminal: Sends user messages to the server.

[0692] Step 13:

[0693] Server: The server analyzes received messages using a natural language processing engine and generates an appropriate response based on the analysis results.

[0694] Step 14:

[0695] Server: Sends the generated response to the terminal.

[0696] Step 15:

[0697] Terminal: Displays the received response message to the user.

[0698] Step 16:

[0699] Server: Analyzes user conversations and determines the direction of the story's progression. Loads new scene and event data as needed.

[0700] Step 17:

[0701] Server: Sends necessary scene and event data to the terminal.

[0702] Step 18:

[0703] Terminal: Displays data from received scenes and events.

[0704] Step 19:

[0705] Terminal: The user selects options for new scenes or events and sends the selection results to the server.

[0706] Step 20:

[0707] Server: Updates intimacy based on the selection results. If intimacy reaches a certain condition, loads a special scenario.

[0708] Step 21:

[0709] Server: Sends a special scenario to the terminal based on the level of intimacy.

[0710] Step 22:

[0711] Terminal: Displays special scenarios to the user.

[0712] Step 23:

[0713] User: Experience a special scenario and continue the conversation.

[0714] Step 24:

[0715] Server: Periodically saves user progress data and saves all data at the end of the session.

[0716] Step 25:

[0717] User: Click the logout button to end the session.

[0718] Step 26:

[0719] Terminal: Sends a logout request to the server.

[0720] Step 27:

[0721] Server: Processes the received logout request, terminates the session, and checks the saved data.

[0722] Step 28:

[0723] Terminal: Display a logout completion screen to the user.

[0724] The above describes the specific processing flow in the system of the present invention. This system allows users to enjoy dynamically unfolding stories and events through conversations with characters.

[0725] (Example 1)

[0726] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0727] Traditional character conversation systems often result in a fixed user experience with limited dynamic branching of storylines and events. Furthermore, they restrict the calculation of intimacy levels based on conversation content and the display of special scenarios. This leads to limited user interaction and a lack of long-term appeal.

[0728] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0729] In this invention, the server includes means for a user to log in and authenticate from a terminal; means for a story to branch and events to occur through conversations with a virtual character selected by the user; means for the system to calculate intimacy based on the conversation content and display a special scenario when certain conditions are met; means for analyzing the chat content using natural language processing with a generation AI model and generating an appropriate response based on the analysis results; and means for dynamically loading and displaying scenarios in response to changes in intimacy. This diversifies user interaction and enables dynamic and seamless story progression and event occurrence.

[0730] A "terminal" is a physical or virtual device used by a user to access and operate a system.

[0731] A "server" is a centralized computer system that receives, processes, and analyzes user input information, and then returns the results to the terminal.

[0732] "Logging in" is a series of processes in which a user enters authentication information into a system and verifies and grants access rights.

[0733] "Authentication" is the process of verifying whether the entered user information matches the registered data.

[0734] A "virtual character" is a digital persona designed to interact with users within a system.

[0735] "Story branching" refers to the process where different storylines or events are generated depending on the user's choices and conversations.

[0736] An "event" is a specific action or occurrence that happens during the story and influences the user's choices.

[0737] "Intimacy level" is an indicator that measures the depth of the relationship between the user and the virtual character, and is calculated as points based on specific actions and messages within the system.

[0738] A "special scenario" refers to a specific story or event that is only displayed when certain conditions are met regarding intimacy.

[0739] A "generative AI model" is an engine that uses artificial intelligence to perform natural language processing, and is responsible for analyzing user messages and generating appropriate responses.

[0740] "Natural language processing" is a technology that enables computers to understand and process human language.

[0741] "Analysis results" refer to the information and conclusions obtained when a generative AI model processes user input.

[0742] "Dynamically loading and displaying scenarios" refers to the process by which the system loads new scenarios and events in real time in response to user actions and displays them on the device.

[0743] This embodiment is a system that dynamically generates and branches a story and triggers events through interactive conversations between the user and a virtual character. This system functions through the cooperation of a terminal, a server, and the user.

[0744] The server manages authentication information when a user logs in from a terminal to access the system. This authentication information is stored in a database (e.g., MySQL) and verified using SQL queries. After the user enters their username and password, the server authenticates them and establishes the login status.

[0745] After logging in, users can access a character selection screen on their device. For example, multiple virtual characters may be displayed as thumbnails, allowing the user to select one. The initial data of the selected character (name, profile, background story, etc.) is sent from the server to the device in JSON format. The device then uses this data to configure and display the chat screen to the user.

[0746] At the start of a chat, the user types and sends a text message. This message is sent from the terminal to the server, where it is analyzed using a generative AI model (e.g., GPT-4). The generative AI model analyzes the message content and generates an appropriate response. This response is sent to the terminal in JSON format, and the terminal displays the character's reply to the user.

[0747] The story branches based on the conversation, and events occur. For example, if the user asks, "What are you doing today?", the server analyzes this and selects the scenario, "I'm planning to go shopping." Next, information about this scenario is sent to the device, and a new storyline is displayed.

[0748] During the conversation, the system calculates intimacy based on the user's responses. Intimacy increases when favorable messages or specific choices are selected. If this intimacy exceeds a certain threshold, the server loads and sends a special scenario or event to the device. The device then displays the special scenario, allowing the user to enjoy a new experience.

[0749] For example, if a user inputs "Hello, beautiful girl character A. What are your plans for today?", the server uses a generative AI model to analyze the scenario and generate a response such as "I'm planning to go shopping. Would you like to come along?". This response is sent to the terminal and displayed to the user. If the user replies "Of course, I'll go with you," a date event occurs, and the intimacy level between the user and the virtual character increases.

[0750] This system allows users to enjoy personalized story experiences and emotional continuity through conversations with virtual characters.

[0751] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0752] Step 1: User Login

[0753] User: The user opens the login screen from their device and enters their username and password. The login process is completed by pressing the login button.

[0754] Input: Username, Password

[0755] Terminal: Sends the entered username and password to the server.

[0756] Server: The server compares the received username and password with the authentication information in the database (e.g., MySQL). It uses an SQL query to perform the comparison and obtain the authentication result. For example, it executes the query "SELECT user_id FROM users WHERE username='example' AND password='123456'".

[0757] Output: Authentication result (success or failure)

[0758] Server: If authentication is successful, the server sends the user ID and home screen data (in JSON format) to the device. For example, it may be in the format "{"status":"success","user_id":4,"home_data":{"background":"blue","notifications":"on"}}".

[0759] Step 2: Character Selection

[0760] Terminal: After successful authentication, the user is shown a character selection screen. A list of selectable characters (icons with images and names) is displayed.

[0761] User: Select your preferred character from the list and press the select button to complete the input.

[0762] Input: ID of the selected character

[0763] Terminal: Sends the ID of the selected character to the server.

[0764] Server: Retrieves the character's initial data from the database and sends it to the terminal in JSON format. For example, "{"character_id":"A","name":"Beautiful Girl Character A","profile":"Profile Information"}".

[0765] Output: Initial character data (JSON format)

[0766] Terminal: Based on the initial data received, configures and displays the chat screen to the user.

[0767] Step 3: Start chatting

[0768] User: Enter a message such as "Hello" in the chat screen and press the send button to complete the input.

[0769] Input: User message

[0770] Terminal: Sends the entered message to the server.

[0771] Server: Analyzes received messages using a generation AI model (e.g., GPT-4). Generates a prompt and inputs it into the generation AI model. For example, "The user typed 'hello'. Please generate an appropriate response."

[0772] Server: Sends the generated response to the terminal in JSON format. For example, "{"response":"Hello, how are you?"}".

[0773] Output: Generated response (JSON format)

[0774] Terminal: Displays the character's reply message to the user.

[0775] Step 4: Story branching and event occurrence

[0776] Server: Continuously analyzes chat content to determine the direction of the story. If a user types "What are you doing today?", it selects a scenario such as "I'm planning to go shopping" based on pre-set conditions.

[0777] Input: User message, story progression conditions

[0778] Server: Sends the data for the selected scenario to the terminal in JSON format. For example, "{"scene":"I'm planning to go shopping. Would you like to join me?"}".

[0779] Output: Scenario data (JSON format)

[0780] Terminal: Displays new scenes and events received from the server. For example, it displays a scene of a character going shopping using images and text.

[0781] User: Selects responses and options in response to displayed scenes and events, and inputs information to advance the conversation.

[0782] Step 5: Intimacy level calculation and special scenario display

[0783] Server: Applies an algorithm that calculates intimacy based on user input and choices. For example, it adds intimacy points for favorable messages.

[0784] Input: User's message and choices

[0785] Server: When intimacy exceeds a certain threshold, load data for special scenarios or events and send it to the terminal in JSON format. For example, "{'special_scenario':'Special Dating Event'}".

[0786] Output: Data for a special scenario (JSON format)

[0787] Terminal: Displays received special scenarios. For example, it provides the user with a description of a special date scene using diagrams and text.

[0788] User: Experience special scenarios and advance through new choices and conversations to deepen your relationship with the virtual character.

[0789] (Application Example 1)

[0790] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0791] Conventional interactive storytelling systems have limited conversations between users and characters, failing to adequately achieve natural interaction and increased intimacy. Furthermore, seamless, real-time branching of the story based on user choices is difficult, limiting the potential for improving the user experience. This invention solves these problems, providing more natural conversations and dynamic, real-time story branching, while also enabling the calculation of user-character intimacy and the presentation of special scenarios based on that.

[0792] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0793] In this invention, the server includes means for branching the story and triggering events through conversations with a character selected by the user; means for the system to calculate intimacy based on the conversation content and display a special scenario when certain conditions are met; means for analyzing the chat content using natural language processing and generating an appropriate response based on the analysis results; means for providing interactive storytelling through conversations between the user and a character in an application installed on a smartphone, smart glasses, head-mounted display, or robot; and means for generating and displaying new scenes and events in real time based on the conversation content and story progression. As a result, the user can enjoy natural conversations with the character and experience special scenarios according to their intimacy level, enabling a richer interactive story experience.

[0794] A "user" is someone who uses the system to engage in conversations and storytelling with characters.

[0795] A "character" is a virtual character that the user selects within the system and interacts with through conversations and stories.

[0796] "Story branching" is a mechanism where the direction of the story changes depending on the user's choices and the content of the conversations.

[0797] An "event" refers to a specific scene or occurrence that happens during the progression of the story.

[0798] "Intimacy level" is a numerical representation of the depth of the relationship between the user and the character.

[0799] A "special scenario" refers to a special storyline or scene that is only displayed when the level of intimacy reaches certain conditions.

[0800] "Natural language processing" is a technology that enables computers to understand and analyze user messages.

[0801] An "appropriate response" is a reply message to the user that the natural language processing engine generates based on its analysis results.

[0802] A "smartphone" is a portable device capable of connecting to the internet and running various applications.

[0803] "Smart glasses" are glasses-type devices equipped with the ability to display information and run applications.

[0804] A "head-mounted display" is a display device that is worn on the head.

[0805] A "robot" is a mechanical device that is capable of interacting with and being operated by a user.

[0806] "Interactive storytelling" is a technique in which the user and the system interact to advance the narrative.

[0807] "Real-time" refers to the temporality in which a system processes information instantly and responds immediately to user actions and conversations.

[0808] A "new scene" is a new moment that is generated during the progression of the story based on the user's choices and conversations.

[0809] The system for carrying out this invention provides interactive storytelling. Users can access the system using a smartphone or other device and converse with characters. Specific embodiments are described below.

[0810] 1. User login and authentication

[0811] Users access the login screen from a device such as a smartphone and enter their username and password.

[0812] The server verifies the entered authentication information against the authentication database. If authentication is successful, it sends the home screen data to the user's device; otherwise, it sends an error message.

[0813] 2. Character Selection

[0814] After successful authentication, the user selects their preferred character from the character selection screen that appears.

[0815] The server loads initial data and background information corresponding to the selected character and sends the chat screen to the user's device.

[0816] 3. Start chat

[0817] Users type messages in the chat screen and send messages such as "Hello."

[0818] The server receives the message and parses it using a natural language processing engine (e.g., spaCy).

[0819] Based on the analysis results, an appropriate response is generated and sent to the user's device.

[0820] The device displays the generated response to the user.

[0821] 4. Story branching and event occurrences

[0822] The server analyzes the user's chat content and determines the direction of the story's progression.

[0823] For example, if the user asks, "What are you doing today?", a scenario is selected in which the character replies, "I'm planning to go shopping." Information about this new scene or event is then sent to the user's device.

[0824] Users can advance the conversation by selecting messages and options based on the scene or event.

[0825] 5. Intimacy level calculation and special scenario display

[0826] The server calculates the level of intimacy based on the user's input and choices.

[0827] For example, when a user sends a positive message, their intimacy level increases. If the intimacy level exceeds a certain threshold, data for special scenarios or events is loaded and sent to the user's device.

[0828] Users can experience special scenarios and deepen their relationships with the characters.

[0829] Hardware and software to be used

[0830] The following hardware and software will be used to implement this system.

[0831] Hardware: Smartphones, smart glasses, head-mounted displays, robots

[0832] Software: Flask (web framework), MongoDB (database), spaCy (natural language processing engine)

[0833] Specific example

[0834] Consider the case where the user types the message "I'm fine, what are you doing today?".

[0835] The server uses a natural language processing engine to analyze the message and generates a response, "I'm planning to go shopping. Would you like to join me?", which it then sends to the user's device.

[0836] Upon receiving this response, the user replies, "Of course, I'll go with you," and a date event occurs.

[0837] During a dating event, if the user selects "Tell me what you like?", the server will increase the intimacy level. When the intimacy level reaches certain conditions, a special scenario will be displayed.

[0838] Example of a prompt

[0839] The user sends the following message to the system:

[0840] "I'm fine, what are you doing today?"

[0841] This system allows users to enjoy natural conversations and interactive storytelling with characters, and also experience special scenarios that change depending on their level of intimacy with the characters.

[0842] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0843] Step 1:

[0844] The user enters their username and password on the login screen of their smartphone.

[0845] Input: Username and password

[0846] Output: Login request data

[0847] Specific action: The user device sends the entered authentication information to the server.

[0848] Step 2:

[0849] The server receives the login request data and compares it with the authentication database.

[0850] Input: Username and password

[0851] Output: Authentication result (success or failure)

[0852] Specific operation: If authentication is successful, the data from the home screen is sent to the user's device; if it fails, an error message is sent.

[0853] Step 3:

[0854] The user opens the character selection screen on the home screen and selects their preferred character.

[0855] Input: Character selection information

[0856] Output: Character selection data

[0857] Specific operation: The user device sends the selected character information to the server.

[0858] Step 4:

[0859] The server loads initial data and background information corresponding to the selected character and sends the chat screen data to the user's device.

[0860] Input: Character selection data

[0861] Output: Chat screen data

[0862] Specific operation: The server retrieves the character's initial data and background information from the database and generates the chat screen.

[0863] Step 5:

[0864] Users type and send messages in the chat screen.

[0865] Input: User message

[0866] Output: Chat message data

[0867] Specific action: The user device sends the entered message to the server.

[0868] Step 6:

[0869] The server receives chat message data and analyzes it using a natural language processing engine (such as spaCy).

[0870] Input: Chat message data

[0871] Output: Analysis results

[0872] Specific operation: The natural language processing engine analyzes the message and identifies the appropriate response.

[0873] Step 7:

[0874] The server generates an appropriate response based on the analysis results of the natural language processing engine and sends that response to the user device.

[0875] Input: Analysis results

[0876] Output: Response message

[0877] Specific operation: Based on the analysis results, a response message is generated and sent to the user device.

[0878] Step 8:

[0879] The user checks the response of the character displayed on the device and continues the conversation.

[0880] Input: Response message

[0881] Output: User response messages

[0882] Specific action: Based on the response displayed on the chat screen, enter the following message.

[0883] Step 9:

[0884] The server determines the direction of the story based on user input, and new scenes and events occur.

[0885] Input: User response message

[0886] Output: Data for new scenes and events

[0887] Specific operation: The system re-analyzes the chat content, generates branching scenarios, and displays them on the user's device.

[0888] Step 10:

[0889] The server calculates intimacy based on user input and choices, and loads data to display special scenarios or events if a certain threshold is exceeded.

[0890] Input: Conversation history and choice data

[0891] Output: Intimacy and special scenario data

[0892] Specific operation: The system calculates intimacy based on an intimacy algorithm and generates a special scenario if the conditions are met.

[0893] Step 11:

[0894] Users can experience special scenarios and deepen their relationships with the characters.

[0895] Input: Special scenario data

[0896] Output: User experience record

[0897] Specific operation: The content of the special scenario is displayed on the user's device, and the user enjoys it.

[0898] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0899] This invention combines an emotion engine with a system in which the story branches and events occur through conversations with a character selected by the user. This system uses natural language processing and emotion recognition technology to provide a more dynamic and emotional story development.

[0900] Specific program processing

[0901] 1. User Login and Authentication

[0902] Terminal: The user launches the application from their terminal, and the login screen is displayed. They enter their username and password and click the "Login" button.

[0903] Terminal: Sends the entered username and password to the server.

[0904] Server: The server compares the received username and password with the database and generates an authentication result. If authentication is successful, it prepares the data for the home screen. If authentication fails, it generates an error message.

[0905] Server: Sends the authentication result (success or failure) to the terminal.

[0906] Device: If authentication is successful, the home screen will be displayed. If authentication fails, an error message will be displayed on the login screen, prompting the user to re-enter their information.

[0907] 2. Character Selection

[0908] Terminal: Upon successful authentication, the character selection screen will be displayed.

[0909] User: Select your preferred character from the list.

[0910] Terminal: Sends selection information to the server.

[0911] Server: Loads the initial and background information of the selected character from the database and generates data for starting a chat.

[0912] Server: Sends data to the terminal to start the chat.

[0913] Terminal: The chat screen appears, and the initial message of the selected character is displayed to the user.

[0914] 3. Start chat

[0915] User: Type your message in the chat screen and click the "Send" button.

[0916] Terminal: Sends user messages to the server.

[0917] Server: Analyzes received messages using a natural language processing engine and generates appropriate responses based on the analysis results. Furthermore, it analyzes the user's emotions using an emotion engine.

[0918] Server: Based on the results of natural language processing and emotion recognition, it generates a refined response and sends it to the terminal.

[0919] Terminal: Displays the character's reply to the user.

[0920] 4. Story branching and event occurrences

[0921] Server: Analyzes chat content and emotion recognition results to determine the direction of the story's progression. For example, if "joy" is recognized from the user's message, it selects a scenario in which the character proposes a "fun event." Based on the emotion recognition results, it seamlessly adjusts events and scenarios.

[0922] Terminal: Displays new scenes and events received from the server.

[0923] User: Can select messages and options to advance the conversation.

[0924] 5. Intimacy level calculation and special scenario display

[0925] Server: Calculates intimacy based on user input, choices, and emotion recognition results. For example, if a user sends a positive message to a character, intimacy increases.

[0926] Terminal: When intimacy exceeds a certain threshold, data for special scenarios and events will be loaded and displayed.

[0927] Users can experience special scenarios and deepen their relationships with the characters.

[0928] Specific example

[0929] Making a date from the first meeting

[0930] 1. Login and Character Selection

[0931] The user logs into the system from their terminal and selects "Character A" from the displayed list. The server receives this selection, loads the initial information for "Character A," and sends the chat screen to the user's terminal.

[0932] 2. Start the conversation

[0933] The user sends the message "Hello, nice to meet you." The server receives this message, analyzes it using natural language processing and an emotion engine, generates the response "Hello, how are you?", and sends it to the terminal. The terminal then displays this response to the user.

[0934] 3. Story progression and event occurrence

[0935] When the user asks, "What are you doing today?", the server analyzes this and, if the emotion engine recognizes an interest, selects a scenario that suggests, "I'm planning to go shopping. Would you like to join me?" This response is sent to the device, and if the user replies, "Of course, I'll go with you," a date event occurs.

[0936] 4. Intimacy and Special Scenarios

[0937] During a dating event, if the user selects the option "Tell me what you like?", the server increases the intimacy level. Furthermore, if the emotion engine detects the user's "enjoyment," a special dating scenario will be loaded, which the user can then experience.

[0938] The above are specific examples of embodiments for carrying out the invention. This system allows users to enjoy individual story experiences and emotional continuity through conversations and choices with characters. The introduction of an emotion engine enables dynamic story development that responds to the user's emotions, providing a richer user experience.

[0939] The following describes the processing flow.

[0940] Step 1:

[0941] Terminal: The user launches the application from their terminal, and the login screen is displayed. They enter their username and password and click the "Login" button.

[0942] Step 2:

[0943] Terminal: Sends the entered username and password to the server.

[0944] Step 3:

[0945] Server: The server compares the received username and password with the database and generates an authentication result. If authentication is successful, it prepares the data for the home screen. If authentication fails, it generates an error message.

[0946] Step 4:

[0947] Server: Sends the authentication result (success or failure) to the terminal.

[0948] Step 5:

[0949] Device: If authentication is successful, the home screen will be displayed. If authentication fails, an error message will be displayed on the login screen, prompting the user to re-enter their information.

[0950] Step 6:

[0951] Device: The user selects their preferred character from the character selection menu on the home screen.

[0952] Step 7:

[0953] Terminal: Sends information about the selected character to the server.

[0954] Step 8:

[0955] Server: Loads initial character information and background information from the database and generates data for starting a chat.

[0956] Step 9:

[0957] Server: Sends data to the terminal to start the chat.

[0958] Step 10:

[0959] Terminal: The chat screen appears, and the initial message of the selected character is displayed to the user.

[0960] Step 11:

[0961] User: Type your message in the chat screen and click the "Send" button.

[0962] Step 12:

[0963] Terminal: Sends user messages to the server.

[0964] Step 13:

[0965] Server: Analyzes received messages using a natural language processing engine. Furthermore, analyzes the user's emotions using an emotion engine.

[0966] Step 14:

[0967] Server: Generates tailored responses based on the results of natural language processing and sentiment recognition.

[0968] Step 15:

[0969] Server: Sends the generated response to the terminal.

[0970] Step 16:

[0971] Terminal: Displays the received response message to the user.

[0972] Step 17:

[0973] Server: Analyzes the user's conversation content and emotion recognition results to determine the direction of the story's progression. For example, if "joy" is recognized from the user's message, it selects a scenario in which the character proposes a "fun event."

[0974] Step 18:

[0975] Server: Loads the necessary scene and event data.

[0976] Step 19:

[0977] Server: Sends loaded scene and event data to the terminal.

[0978] Step 20:

[0979] Terminal: Displays data from received scenes and events.

[0980] Step 21:

[0981] User: Select options for new scenes or events and send the selection results to the server.

[0982] Step 22:

[0983] Server: Updates intimacy based on selection results. Dynamically adjusts intimacy based on the user's emotion recognition results.

[0984] Step 23:

[0985] Server: When intimacy exceeds a certain threshold, it loads data for special scenarios or events.

[0986] Step 24:

[0987] Server: Sends data for special scenarios and events to terminals.

[0988] Step 25:

[0989] Terminal: Displays special scenarios and events.

[0990] Step 26:

[0991] User: Experience special scenarios and events, and continue the conversation.

[0992] Step 27:

[0993] Server: Periodically saves user progress data and saves all data at the end of the session.

[0994] Step 28:

[0995] User: Click the logout button to end the session.

[0996] Step 29:

[0997] Terminal: Sends a logout request to the server.

[0998] Step 30:

[0999] Server: Processes the received logout request, terminates the session, and checks the saved data.

[1000] Step 31:

[1001] Terminal: Display a logout completion screen to the user.

[1002] The above describes the specific processing flow in the system of the present invention. This system allows users to enjoy dynamically unfolding stories and events through conversations and choices with characters. The introduction of an emotion engine enables dynamic story development in response to the user's emotions, providing a richer user experience. For example, when a user asks, "What do you like?", the emotion engine recognizes "affection" and displays a special date scenario.

[1003] (Example 2)

[1004] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[1005] Traditional dialogue systems have a problem where conversations with characters are formal, and dynamic story branching and event generation that respond to user emotions and choices cannot be realized. Furthermore, the lack of intimacy level calculations that reflect user emotions and the display of special scenarios have resulted in a limited user experience.

[1006] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1007] In this invention, the server includes means for branching the story and causing events through conversations with a character selected by the user; means for the system to calculate intimacy based on the conversation content and emotion recognition results, and display a special scenario when certain conditions are met; means for analyzing the chat content using natural language processing and generating an appropriate response based on the analysis results and emotion recognition results; means for seamlessly adjusting the story branching and event occurrences based on the user's emotion recognition results; and means for dynamically generating and displaying special scenarios and events based on the calculation of intimacy. This enables a dynamic and emotional story development and a rich user experience based on the user's emotions and choices.

[1008] A "user" is someone who utilizes this system and experiences the story progression and events through interaction with characters.

[1009] A "character" is a virtual person or entity that interacts with the user within the system, and has the role of responding dynamically according to the user's choices and emotional recognition results.

[1010] "Conversation" refers to the exchange of messages between the user and the character, and is generally conducted in text format.

[1011] "Story branching" refers to the change in the direction of the narrative based on the user's choices and emotional responses, resulting in different events and scenarios.

[1012] An "event" refers to a specific scene or occurrence within a story, and its content may change depending on the user's choices and emotions.

[1013] "Intimacy level" is a numerical or metric representation of the depth of the relationship between the user and the character, and it fluctuates in response to the user's actions and statements.

[1014] A "special scenario" refers to a special scene or story that appears when the intimacy level meets certain conditions, and it contains unique content that differs from the regular scenario.

[1015] "Emotion recognition" is a technology that analyzes a user's input and actions to identify their emotions and determine their emotional state, such as joy, interest, or sadness.

[1016] "Natural language processing" is a technology that analyzes user messages to understand their meaning and intent, and involves text analysis and response generation.

[1017] A "response" is a message that a character returns in response to a message from the user, and is generated based on natural language processing and emotion recognition results.

[1018] Embodiments of this invention will be described in detail below. This system has a mechanism in which the story branches and events occur through conversations with a character selected by the user. In addition, it can calculate intimacy based on the conversation content and emotion recognition results, and display a special scenario when certain conditions are met.

[1019] System Overview

[1020] First, let me explain the system's overview. This system consists of three components: a server, a terminal, and a user, all of which interact with each other. The server is responsible for key processes such as story branching, event generation, emotion recognition, intimacy calculation, and natural language processing. The terminal provides the user's interface and communicates with the server. The user operates the terminal to advance conversations with the characters.

[1021] Hardware and software to be used

[1022] This system uses the following hardware and software:

[1023] Server hardware: A server with a high-performance processor, sufficient memory, and fast database access capabilities.

[1024] Database: A relational database management system (RDBMS) such as MySQL.

[1025] Natural language processing engine: High-performance generative AI models such as GPT-4.

[1026] Emotion recognition engine: Emotion analysis tools such as Emotion AI.

[1027] Details of the program processing

[1028] This section provides a detailed explanation of the program processing performed by the system. The specific processing steps for each stage are shown below.

[1029] User login and authentication

[1030] 1. User: Launch the application from your device and enter your username and password on the login screen.

[1031] 2. Terminal: Sends the entered information to the server.

[1032] 3. Server: Matches the information with the database, generates an authentication result, and sends it to the terminal.

[1033] 4. Device: Displays the home screen or an error message based on the authentication result.

[1034] Character Selection

[1035] 1. Device: After successful authentication, the character selection screen will be displayed.

[1036] 2. User: Select a character from the list.

[1037] 3. Terminal: Sends selection information to the server.

[1038] 4. Server: Loads the character's initial information, generates data for starting a chat, and sends it to the terminal.

[1039] 5. Terminal: Displays the chat screen and the character's initial message.

[1040] Start chat

[1041] 1. User: Enter your message.

[1042] 2. Terminal: Sends the entered message to the server.

[1043] 3. Server: Analyzes messages using a natural language processing engine and analyzes emotions using an emotion engine.

[1044] 4. Server: Generates a response and sends it to the terminal.

[1045] 5. Terminal: Displays the character's reply.

[1046] Story branching and event occurrences

[1047] 1. Server: Analyzes chat content and emotion recognition results to determine the direction of the story's progression.

[1048] 2. Terminal: Displays new scenes and events.

[1049] 3. User: Can continue the conversation.

[1050] Intimacy level calculation and special scenario display

[1051] 1. Server: Calculates intimacy levels based on user statements and emotion recognition results.

[1052] 2. Device: When the intimacy level exceeds a certain threshold, a special scenario will be displayed.

[1053] 3. User: Experience a special scenario.

[1054] Specific example

[1055] Specific examples are given below.

[1056] Making a date from the first meeting

[1057] The user logs into the system from their device and selects "Character A". The server receives the selection information, loads the initial information for "Character A", and sends the chat screen to the user's device. When the user sends the message "Hi, nice to meet you", the server analyzes it and generates the response "Hi, how are you?" which is sent to the device. When the user continues with "What are you doing today?", the server analyzes this and, if it recognizes the user's "interest", selects a scenario that suggests "I'm planning to go shopping. Would you like to come along?". This response is sent to the device, and if the user replies "Of course, I'll go with you", a date event occurs. During the date event, if the user selects "Tell me what you like?", the server increases the intimacy level. Furthermore, if the emotion engine detects the user's "enjoyment", a special date scenario is loaded, which the user can experience.

[1058] Example of a prompt

[1059] "Ask Character A what they're doing today and guide them towards a date event."

[1060] In this embodiment of the invention, users can enjoy a dynamic and emotional storyline through emotional interaction with characters. By using an emotion recognition engine and a natural language processing engine together, more personalized responses become possible, significantly improving the user experience.

[1061] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1062] The processing flow of this system's program (explained in steps)

[1063] Step 1: Display the login screen

[1064] Input: The user launches the application.

[1065] Action: The device launches the application and displays the login screen.

[1066] Output: The login screen is displayed to the user.

[1067] Step 2: Enter and submit user authentication information.

[1068] Input: The user enters their username and password and clicks the login button.

[1069] Operation: The terminal sends the entered username and password to the server. The data is encrypted before transmission.

[1070] Output: User authentication information is sent to the server.

[1071] Step 3: Authentication Process

[1072] Input: The username and password received by the server.

[1073] Operation: The server connects to the database and verifies the username and password. If successful, it generates an authentication token and prepares data for the home screen. If it fails, it prepares an error message.

[1074] Output: Authentication results are generated, and an authentication token or error message is generated.

[1075] Step 4: Submit and display authentication results

[1076] Input: Authentication result and authentication token or error message.

[1077] Operation: The server sends the authentication result to the device. Based on the received data, the device displays either the home screen or an error message.

[1078] Output: The home screen is displayed, or an error message is displayed on the login screen.

[1079] Step 5: Display the character selection screen.

[1080] Input: User who successfully authenticated.

[1081] Action: The device displays the character selection screen.

[1082] Output: The character selection screen is displayed to the user.

[1083] Step 6: Submit character selection information

[1084] Input: The user selects a character and clicks the select button.

[1085] Operation: The terminal sends the ID of the selected character to the server.

[1086] Output: Character ID information is sent to the server.

[1087] Step 7: Loading character information

[1088] Input: The character ID received by the server.

[1089] Operation: The server loads initial information for the selected character from the database and generates data to start the chat.

[1090] Output: Data for starting a chat is generated.

[1091] Step 8: Send and display data to start the chat.

[1092] Input: Data to start the chat.

[1093] Operation: The server sends data to the terminal to initiate a chat. The terminal receives the data, displays the chat screen, and shows the character's initial message.

[1094] Output: The chat screen and the character's initial message are displayed.

[1095] Step 9: Enter and send the user message.

[1096] Input: The user enters a message and clicks the send button.

[1097] Operation: The terminal sends the entered message to the server.

[1098] Output: The user message is sent to the server.

[1099] Step 10: Message Analysis

[1100] Input: User messages received by the server.

[1101] Operation: The server analyzes messages using a natural language processing engine (such as GPT-4). It also analyzes the user's emotions using an emotion recognition engine (such as Emotion AI).

[1102] Output: Message analysis results and sentiment recognition results are generated.

[1103] Step 11: Generating a response message

[1104] Input: Message analysis results and sentiment recognition results.

[1105] Operation: Based on these results, the server generates an appropriate response message.

[1106] Output: A response message is generated.

[1107] Step 12: Sending and displaying the response message

[1108] Input: Response message.

[1109] Operation: The server sends a response message to the terminal. The terminal receives the message and displays it on the chat screen.

[1110] Output: The response message is displayed on the chat screen.

[1111] Step 13: Deciding on a story branch

[1112] Input: Chat content and sentiment recognition results.

[1113] Operation: The server determines the direction of the story's progression based on this data.

[1114] Output: The results of the story branching are generated.

[1115] Step 14: Generating new scenes and events

[1116] Input: The result of the story branching.

[1117] Operation: The server generates data for new scenes and events and sends it to the terminal.

[1118] Output: New scene and event data is generated.

[1119] Step 15: Displaying new scenes and events

[1120] Input: Data for new scenes or events.

[1121] Operation: The terminal displays the received data.

[1122] Output: New scenes or events are displayed on the terminal.

[1123] Step 16: Calculating Intimacy Level

[1124] Input: User input, choices, and sentiment recognition results.

[1125] Operation: The server calculates intimacy based on this data.

[1126] Output: An intimacy value is generated.

[1127] Step 17: Generating a Special Scenario

[1128] Input: Intimacy level value.

[1129] Operation: When intimacy exceeds a certain threshold, the server generates special scenarios or events.

[1130] Output: Data for special scenarios and events is generated.

[1131] Step 18: Displaying Special Scenarios

[1132] Input: Data for special scenarios or events.

[1133] Operation: The terminal displays the received data.

[1134] Output: Special scenarios or events will be displayed on the terminal.

[1135] The above describes the detailed processing steps and specific operation of this system's program.

[1136] (Application Example 2)

[1137] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1138] Traditional interactive storytelling systems were unable to dynamically branch stories in response to user emotions, instead relying on fixed story patterns. This resulted in a limited user experience and a lack of entertainment value and immersion. Furthermore, the calculation of intimacy levels and the resulting display of special scenarios were also fixed, making real-time, emotion-based responses difficult.

[1139] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1140] In this invention, the server includes means for branching the story and triggering events through conversations with a character selected by the user; means for the system to calculate intimacy based on the conversation content and display a special scenario when certain conditions are met; means for analyzing the chat content using natural language processing and generating an appropriate response based on the analysis results; means for recognizing the user's emotions and dynamically branching the story according to those emotions; and means for seamlessly displaying special events and scenarios based on intimacy and emotions. This enables flexible and dynamic story development that responds to the user's emotions.

[1141] A "user" is someone who uses the system to enjoy content through conversations with characters.

[1142] A "character" is a virtual entity or person that a user can select to engage in conversation with.

[1143] A "story" refers to a series of events or occurrences that unfold through conversations with characters.

[1144] "Branching" refers to the point where the direction of the story changes depending on the user's choices and emotions.

[1145] An "event" refers to a specific occurrence or action that takes place within a story.

[1146] "Intimacy level" is an indicator that shows the depth of the relationship between the user and the character.

[1147] A "special scenario" is a scenario that appears when certain conditions are met, and is different from the normal storyline.

[1148] "Natural language processing" is the technology that uses computers to understand, analyze, and generate human language.

[1149] "Emotions" refer to the user's feelings or emotional state.

[1150] "Emotion recognition" is a technology that analyzes and identifies emotions from a user's words and actions.

[1151] "Dynamic" refers to the property of changing or adapting in real time according to the situation.

[1152] "Seamlessly" refers to something that is continuous and uninterrupted.

[1153] The system of this invention provides a mechanism in which the story branches and events occur through conversations with characters selected by the user. The program to realize this is structured as follows.

[1154] The system requires a smartphone and a server as hardware. The smartphone is the device that the user directly interacts with, while the server processes and stores various types of data. As for software, the "transformers" library, a natural language processing engine, is used. This library is based on a generative AI model and uses a "sentiment-analysis" model for emotion recognition.

[1155] When a user logs into the system using their smartphone, the server verifies the user's credentials against the database and performs authentication. If authentication is successful, a character selection screen appears on the device. The user selects their preferred character, and the server loads the data for the selected character and prepares to start chatting.

[1156] When a user types and sends a message to a character, the message is sent from the terminal to the server. The server uses the "transformers" library to parse the message and perform emotion recognition. Based on the results, it generates an appropriate response and sends it to the terminal. The terminal then displays this response to the user.

[1157] Based on the chat content and the results of emotion recognition, the server determines the branching of the story. For example, if joy is recognized from the user's message, it selects a scenario that suggests a pleasant event. It also calculates intimacy and, if it exceeds a certain threshold, loads a special scenario and presents it to the user.

[1158] As a concrete example, consider a scenario where a user logs in with the username "example_user" and password "123456," and selects "Character A." When the user sends the message "I'm relieved that work is over," the server recognizes this as "joy" and generates a response asking, "Would you like to share something fun?" This system enables flexible and dynamic story development that responds to the user's emotions, providing a rich user experience.

[1159] Furthermore, the following input can be considered as an example of a prompt statement.

[1160] User's input message: "Every day is tough..."

[1161] Processing result: "Did something upsetting happen?"

[1162] The above describes the embodiments for carrying out this invention.

[1163] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1164] Step 1:

[1165] The user launches the application from their smartphone, and the login screen appears. They enter their username and password and click the "Login" button.

[1166] Input: Username, Password

[1167] Output: Authentication Request

[1168] Step 2:

[1169] The terminal sends the entered username and password to the server. The server compares the received authentication information with its database and generates an authentication result.

[1170] Input: Authentication Request

[1171] Output: Authentication result (success or failure)

[1172] Step 3:

[1173] If authentication is successful, the server prepares the home screen data and sends it to the device. If authentication fails, it generates an error message and sends it to the device. If authentication is successful, the device displays the home screen; if it fails, it displays the error message on the login screen.

[1174] Input: Authentication result

[1175] Output: Home screen data or error messages

[1176] Step 4:

[1177] The user moves from the home screen to the character selection screen and selects a character from the displayed list. The device then sends the selection information to the server.

[1178] Input: Character Selection

[1179] Output: Character selection information

[1180] Step 5:

[1181] Based on the received character selection information, the server loads the selected character's initial and background information from the database, generates data for starting a chat, and sends it to the terminal. The terminal displays the chat screen and shows the selected character's initial message.

[1182] Input: Character selection information

[1183] Output: Chat start data, initial message

[1184] Step 6:

[1185] The user types a message in the chat screen and clicks the "Send" button. The device sends the user's message to the server.

[1186] Input: Message

[1187] Output: Message sending request

[1188] Step 7:

[1189] The server analyzes the received message using the "transformers" library and performs emotion recognition using a generative AI model. Based on the analysis results and emotion recognition results, it generates an appropriate response message and sends it to the terminal.

[1190] Input: Message sending request

[1191] Output: Response message

[1192] Step 8:

[1193] The terminal displays the response message received from the server to the user. The user can then continue chatting.

[1194] Input: Response message

[1195] Output: Display of response message

[1196] Step 9:

[1197] The server analyzes the chat content and emotion recognition results to determine the direction of the story. For example, if it recognizes "joy" from the user's message, it selects a scenario in which the character proposes a "fun event." New scenes and events are generated and sent to the device according to specific conditions.

[1198] Input: Sentiment recognition result

[1199] Output: Event data, new scene data

[1200] Step 10:

[1201] The terminal displays new scenes and events received from the server to the user. The user can then enter further messages or options to continue the conversation.

[1202] Input: New scene data, event data

[1203] Output: Display new scene and events

[1204] Step 11:

[1205] The server calculates intimacy based on user input, choices, and emotion recognition results. For example, if a user sends a positive message to a character, intimacy increases. If intimacy exceeds a certain threshold, a special scenario or event is generated and sent to the terminal.

[1206] Input: User input, emotion recognition results

[1207] Output: Intimacy level, special scenario data

[1208] Step 12:

[1209] The terminal displays special scenarios and events sent from the server, and the user can continue to interact with them.

[1210] Input: Special Scenario Data

[1211] Output: Display of special scenarios

[1212] The above outlines the specific processing steps of this system's program.

[1213] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1214] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1215] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1216] [Third Embodiment]

[1217] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1218] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1219] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1220] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1221] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1222] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1223] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1224] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1225] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1226] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1227] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1228] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1229] This invention describes a system in which a story branches and events occur through conversations between a user and a character. This system uses natural language processing to analyze the user's messages and generate appropriate responses, and includes means for calculating intimacy levels and displaying special scenarios.

[1230] Specific program processing

[1231] 1. User Login and Authentication

[1232] Terminal: The user opens the login screen from their terminal and enters their username and password.

[1233] Server: The server compares the username and password received from the terminal with the authentication database. If authentication is successful, it sends the home screen data to the terminal. If authentication fails, it sends an error message.

[1234] 2. Character Selection

[1235] Terminal: Upon successful authentication, the character selection screen will be displayed.

[1236] User: Select your preferred character from the list.

[1237] Terminal: Sends selection information to the server.

[1238] Server: Loads the initial data and background information of the selected character and sends the chat screen data to the terminal.

[1239] 3. Start chat

[1240] User: Enter a message such as "Hello" in the chat screen and send it.

[1241] Terminal: Sends a message to the server.

[1242] Server: Receives messages and parses them using a natural language processing engine. Based on the analysis results, it generates an appropriate response such as "Hello, how are you?" and sends it to the terminal.

[1243] Terminal: Displays the character's reply to the user.

[1244] 4. Story branching and event occurrences

[1245] Server: Analyzes chat content and determines the direction of the story. For example, if the user asks "What are you doing today?", the server selects a scenario where the character answers "I'm planning to go shopping."

[1246] Terminal: Displays new scenes and events received from the server.

[1247] User: Can select messages and options to advance the conversation.

[1248] 5. Intimacy level calculation and special scenario display

[1249] Server: Calculates intimacy based on user input and choices. For example, if a user sends a friendly message to a character, intimacy increases.

[1250] Terminal: When intimacy exceeds a certain threshold, data for special scenarios and events will be loaded and displayed.

[1251] Users can experience special scenarios and deepen their relationships with the characters.

[1252] Specific example

[1253] Making a date from the first meeting

[1254] 1. Login and Character Selection

[1255] The user logs into the system from their terminal and selects "Beautiful Girl Character A" from the displayed list. The server receives this selection, loads the initial information for "Beautiful Girl Character A," and sends the chat screen to the user's terminal.

[1256] 2. Start the conversation

[1257] The user sends the message "Hello, nice to meet you." The server receives this message, parses it using natural language processing, generates the response "Hello, how are you?", and sends it to the terminal. The terminal then displays this response to the user.

[1258] 3. Story progression and event occurrence

[1259] When the user replies, "I'm fine, what are you doing today?", the server analyzes this and branches the scenario to, "I'm planning to go shopping. Would you like to come along?". This response is sent to the terminal, and if the user replies, "Of course, I'll go with you," a date event occurs.

[1260] 4. Intimacy and Special Scenarios

[1261] During a dating event, if the user selects the option "Tell me what you like?", the server will increase the intimacy level. When the intimacy level reaches a certain threshold, "Beautiful Girl Character A" will propose a special scenario to the user, which will then be displayed on the device. The user can then enjoy this special scenario.

[1262] The above are specific examples of embodiments for carrying out the invention. This system allows users to enjoy individual story experiences and emotional continuity through conversations and choices with characters.

[1263] The following describes the processing flow.

[1264] Step 1:

[1265] Terminal: The user launches the application from their terminal, and the login screen is displayed. The user enters their username and password and clicks the "Login" button.

[1266] Step 2:

[1267] Terminal: Sends the entered username and password to the server.

[1268] Step 3:

[1269] Server: The server compares the received username and password with the database and generates an authentication result. If authentication is successful, it prepares the data for the home screen. If authentication fails, it generates an error message.

[1270] Step 4:

[1271] Server: Sends the authentication result (success or failure) to the terminal.

[1272] Step 5:

[1273] Device: If authentication is successful, the home screen will be displayed. If authentication fails, an error message will be displayed on the login screen, prompting the user to re-enter their information.

[1274] Step 6:

[1275] Device: The user selects their preferred character from the character selection menu on the home screen.

[1276] Step 7:

[1277] Terminal: Sends information about the selected character to the server.

[1278] Step 8:

[1279] Server: Loads initial character information and background information from the database and generates data for starting a chat.

[1280] Step 9:

[1281] Server: Sends data to the terminal to start the chat.

[1282] Step 10:

[1283] Terminal: The chat screen appears, and the initial message of the selected character is displayed to the user.

[1284] Step 11:

[1285] User: Type your message in the chat screen and click the "Send" button.

[1286] Step 12:

[1287] Terminal: Sends user messages to the server.

[1288] Step 13:

[1289] Server: The server analyzes received messages using a natural language processing engine and generates an appropriate response based on the analysis results.

[1290] Step 14:

[1291] Server: Sends the generated response to the terminal.

[1292] Step 15:

[1293] Terminal: Displays the received response message to the user.

[1294] Step 16:

[1295] Server: Analyzes user conversations and determines the direction of the story's progression. Loads new scene and event data as needed.

[1296] Step 17:

[1297] Server: Sends necessary scene and event data to the terminal.

[1298] Step 18:

[1299] Terminal: Displays data from received scenes and events.

[1300] Step 19:

[1301] Terminal: The user selects options for new scenes or events and sends the selection results to the server.

[1302] Step 20:

[1303] Server: Updates intimacy based on the selection results. If intimacy reaches a certain condition, loads a special scenario.

[1304] Step 21:

[1305] Server: Sends a special scenario to the terminal based on the level of intimacy.

[1306] Step 22:

[1307] Terminal: Displays special scenarios to the user.

[1308] Step 23:

[1309] User: Experience a special scenario and continue the conversation.

[1310] Step 24:

[1311] Server: Periodically saves user progress data and saves all data at the end of the session.

[1312] Step 25:

[1313] User: Click the logout button to end the session.

[1314] Step 26:

[1315] Terminal: Sends a logout request to the server.

[1316] Step 27:

[1317] Server: Processes the received logout request, terminates the session, and checks the saved data.

[1318] Step 28:

[1319] Terminal: Display a logout completion screen to the user.

[1320] The above describes the specific processing flow in the system of the present invention. This system allows users to enjoy dynamically unfolding stories and events through conversations with characters.

[1321] (Example 1)

[1322] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1323] Traditional character conversation systems often result in a fixed user experience with limited dynamic branching of storylines and events. Furthermore, they restrict the calculation of intimacy levels based on conversation content and the display of special scenarios. This leads to limited user interaction and a lack of long-term appeal.

[1324] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1325] In this invention, the server includes means for a user to log in and authenticate from a terminal; means for a story to branch and events to occur through conversations with a virtual character selected by the user; means for the system to calculate intimacy based on the conversation content and display a special scenario when certain conditions are met; means for analyzing the chat content using natural language processing with a generation AI model and generating an appropriate response based on the analysis results; and means for dynamically loading and displaying scenarios in response to changes in intimacy. This diversifies user interaction and enables dynamic and seamless story progression and event occurrence.

[1326] A "terminal" is a physical or virtual device used by a user to access and operate a system.

[1327] A "server" is a centralized computer system that receives, processes, and analyzes user input information, and then returns the results to the terminal.

[1328] "Logging in" is a series of processes in which a user enters authentication information into a system and verifies and grants access rights.

[1329] "Authentication" is the process of verifying whether the entered user information matches the registered data.

[1330] A "virtual character" is a digital persona designed to interact with users within a system.

[1331] "Story branching" refers to the process where different storylines or events are generated depending on the user's choices and conversations.

[1332] An "event" is a specific action or occurrence that happens during the story and influences the user's choices.

[1333] "Intimacy level" is an indicator that measures the depth of the relationship between the user and the virtual character, and is calculated as points based on specific actions and messages within the system.

[1334] A "special scenario" refers to a specific story or event that is only displayed when certain conditions are met regarding intimacy.

[1335] A "generative AI model" is an engine that uses artificial intelligence to perform natural language processing, and is responsible for analyzing user messages and generating appropriate responses.

[1336] "Natural language processing" is a technology that enables computers to understand and process human language.

[1337] "Analysis results" refer to the information and conclusions obtained when a generative AI model processes user input.

[1338] "Dynamically loading and displaying scenarios" refers to the process by which the system loads new scenarios and events in real time in response to user actions and displays them on the device.

[1339] This embodiment is a system that dynamically generates and branches a story and triggers events through interactive conversations between the user and a virtual character. This system functions through the cooperation of a terminal, a server, and the user.

[1340] The server manages authentication information when a user logs in from a terminal to access the system. This authentication information is stored in a database (e.g., MySQL) and verified using SQL queries. After the user enters their username and password, the server authenticates them and establishes the login status.

[1341] After logging in, users can access a character selection screen on their device. For example, multiple virtual characters may be displayed as thumbnails, allowing the user to select one. The initial data of the selected character (name, profile, background story, etc.) is sent from the server to the device in JSON format. The device then uses this data to configure and display the chat screen to the user.

[1342] At the start of a chat, the user types and sends a text message. This message is sent from the terminal to the server, where it is analyzed using a generative AI model (e.g., GPT-4). The generative AI model analyzes the message content and generates an appropriate response. This response is sent to the terminal in JSON format, and the terminal displays the character's reply to the user.

[1343] The story branches based on the conversation, and events occur. For example, if the user asks, "What are you doing today?", the server analyzes this and selects the scenario, "I'm planning to go shopping." Next, information about this scenario is sent to the device, and a new storyline is displayed.

[1344] During the conversation, the system calculates intimacy based on the user's responses. Intimacy increases when favorable messages or specific choices are selected. If this intimacy exceeds a certain threshold, the server loads and sends a special scenario or event to the device. The device then displays the special scenario, allowing the user to enjoy a new experience.

[1345] For example, if a user inputs "Hello, beautiful girl character A. What are your plans for today?", the server uses a generative AI model to analyze the scenario and generate a response such as "I'm planning to go shopping. Would you like to come along?". This response is sent to the terminal and displayed to the user. If the user replies "Of course, I'll go with you," a date event occurs, and the intimacy level between the user and the virtual character increases.

[1346] This system allows users to enjoy personalized story experiences and emotional continuity through conversations with virtual characters.

[1347] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1348] Step 1: User Login

[1349] User: The user opens the login screen from their device and enters their username and password. The login process is completed by pressing the login button.

[1350] Input: Username, Password

[1351] Terminal: Sends the entered username and password to the server.

[1352] Server: The server compares the received username and password with the authentication information in the database (e.g., MySQL). It uses an SQL query to perform the comparison and obtain the authentication result. For example, it executes the query "SELECT user_id FROM users WHERE username='example' AND password='123456'".

[1353] Output: Authentication result (success or failure)

[1354] Server: If authentication is successful, the server sends the user ID and home screen data (in JSON format) to the device. For example, it may be in the format "{"status":"success","user_id":4,"home_data":{"background":"blue","notifications":"on"}}".

[1355] Step 2: Character Selection

[1356] Terminal: After successful authentication, the user is shown a character selection screen. A list of selectable characters (icons with images and names) is displayed.

[1357] User: Select your preferred character from the list and press the select button to complete the input.

[1358] Input: ID of the selected character

[1359] Terminal: Sends the ID of the selected character to the server.

[1360] Server: Retrieves the character's initial data from the database and sends it to the terminal in JSON format. For example, "{"character_id":"A","name":"Beautiful Girl Character A","profile":"Profile Information"}".

[1361] Output: Initial character data (JSON format)

[1362] Terminal: Based on the initial data received, configures and displays the chat screen to the user.

[1363] Step 3: Start chatting

[1364] User: Enter a message such as "Hello" in the chat screen and press the send button to complete the input.

[1365] Input: User message

[1366] Terminal: Sends the entered message to the server.

[1367] Server: Analyzes received messages using a generation AI model (e.g., GPT-4). Generates a prompt and inputs it into the generation AI model. For example, "The user typed 'hello'. Please generate an appropriate response."

[1368] Server: Sends the generated response to the terminal in JSON format. For example, "{"response":"Hello, how are you?"}".

[1369] Output: Generated response (JSON format)

[1370] Terminal: Displays the character's reply message to the user.

[1371] Step 4: Story branching and event occurrence

[1372] Server: Continuously analyzes chat content to determine the direction of the story. If a user types "What are you doing today?", it selects a scenario such as "I'm planning to go shopping" based on pre-set conditions.

[1373] Input: User message, story progression conditions

[1374] Server: Sends the data for the selected scenario to the terminal in JSON format. For example, "{"scene":"I'm planning to go shopping. Would you like to join me?"}".

[1375] Output: Scenario data (JSON format)

[1376] Terminal: Displays new scenes and events received from the server. For example, it displays a scene of a character going shopping using images and text.

[1377] User: Selects responses and options in response to displayed scenes and events, and inputs information to advance the conversation.

[1378] Step 5: Intimacy level calculation and special scenario display

[1379] Server: Applies an algorithm that calculates intimacy based on user input and choices. For example, it adds intimacy points for favorable messages.

[1380] Input: User's message and choices

[1381] Server: When intimacy exceeds a certain threshold, load data for special scenarios or events and send it to the terminal in JSON format. For example, "{'special_scenario':'Special Dating Event'}".

[1382] Output: Data for a special scenario (JSON format)

[1383] Terminal: Displays received special scenarios. For example, it provides the user with a description of a special date scene using diagrams and text.

[1384] User: Experience special scenarios and advance through new choices and conversations to deepen your relationship with the virtual character.

[1385] (Application Example 1)

[1386] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1387] Conventional interactive storytelling systems have limited conversations between users and characters, failing to adequately achieve natural interaction and increased intimacy. Furthermore, seamless, real-time branching of the story based on user choices is difficult, limiting the potential for improving the user experience. This invention solves these problems, providing more natural conversations and dynamic, real-time story branching, while also enabling the calculation of user-character intimacy and the presentation of special scenarios based on that.

[1388] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1389] In this invention, the server includes means for branching the story and triggering events through conversations with a character selected by the user; means for the system to calculate intimacy based on the conversation content and display a special scenario when certain conditions are met; means for analyzing the chat content using natural language processing and generating an appropriate response based on the analysis results; means for providing interactive storytelling through conversations between the user and a character in an application installed on a smartphone, smart glasses, head-mounted display, or robot; and means for generating and displaying new scenes and events in real time based on the conversation content and story progression. As a result, the user can enjoy natural conversations with the character and experience special scenarios according to their intimacy level, enabling a richer interactive story experience.

[1390] A "user" is someone who uses the system to engage in conversations and storytelling with characters.

[1391] A "character" is a virtual character that the user selects within the system and interacts with through conversations and stories.

[1392] "Story branching" is a mechanism where the direction of the story changes depending on the user's choices and the content of the conversations.

[1393] An "event" refers to a specific scene or occurrence that happens during the progression of the story.

[1394] "Intimacy level" is a numerical representation of the depth of the relationship between the user and the character.

[1395] A "special scenario" refers to a special storyline or scene that is only displayed when the level of intimacy reaches certain conditions.

[1396] "Natural language processing" is a technology that enables computers to understand and analyze user messages.

[1397] An "appropriate response" is a reply message to the user that the natural language processing engine generates based on its analysis results.

[1398] A "smartphone" is a portable device capable of connecting to the internet and running various applications.

[1399] "Smart glasses" are glasses-type devices equipped with the ability to display information and run applications.

[1400] A "head-mounted display" is a display device that is worn on the head.

[1401] A "robot" is a mechanical device that is capable of interacting with and being operated by a user.

[1402] "Interactive storytelling" is a technique in which the user and the system interact to advance the narrative.

[1403] "Real-time" refers to the temporality in which a system processes information instantly and responds immediately to user actions and conversations.

[1404] A "new scene" is a new moment that is generated during the progression of the story based on the user's choices and conversations.

[1405] The system for carrying out this invention provides interactive storytelling. Users can access the system using a smartphone or other device and converse with characters. Specific embodiments are described below.

[1406] 1. User login and authentication

[1407] Users access the login screen from a device such as a smartphone and enter their username and password.

[1408] The server verifies the entered authentication information against the authentication database. If authentication is successful, it sends the home screen data to the user's device; otherwise, it sends an error message.

[1409] 2. Character Selection

[1410] After successful authentication, the user selects their preferred character from the character selection screen that appears.

[1411] The server loads initial data and background information corresponding to the selected character and sends the chat screen to the user's device.

[1412] 3. Start chat

[1413] Users type messages in the chat screen and send messages such as "Hello."

[1414] The server receives the message and parses it using a natural language processing engine (e.g., spaCy).

[1415] Based on the analysis results, an appropriate response is generated and sent to the user's device.

[1416] The device displays the generated response to the user.

[1417] 4. Story branching and event occurrences

[1418] The server analyzes the user's chat content and determines the direction of the story's progression.

[1419] For example, if the user asks, "What are you doing today?", a scenario is selected in which the character replies, "I'm planning to go shopping." Information about this new scene or event is then sent to the user's device.

[1420] Users can advance the conversation by selecting messages and options based on the scene or event.

[1421] 5. Intimacy level calculation and special scenario display

[1422] The server calculates the level of intimacy based on the user's input and choices.

[1423] For example, when a user sends a positive message, their intimacy level increases. If the intimacy level exceeds a certain threshold, data for special scenarios or events is loaded and sent to the user's device.

[1424] Users can experience special scenarios and deepen their relationships with the characters.

[1425] Hardware and software to be used

[1426] The following hardware and software will be used to implement this system.

[1427] Hardware: Smartphones, smart glasses, head-mounted displays, robots

[1428] Software: Flask (web framework), MongoDB (database), spaCy (natural language processing engine)

[1429] Specific example

[1430] Consider the case where the user types the message "I'm fine, what are you doing today?".

[1431] The server uses a natural language processing engine to analyze the message and generates a response, "I'm planning to go shopping. Would you like to join me?", which it then sends to the user's device.

[1432] Upon receiving this response, the user replies, "Of course, I'll go with you," and a date event occurs.

[1433] During a dating event, if the user selects "Tell me what you like?", the server will increase the intimacy level. When the intimacy level reaches certain conditions, a special scenario will be displayed.

[1434] Example of a prompt

[1435] The user sends the following message to the system:

[1436] "I'm fine, what are you doing today?"

[1437] This system allows users to enjoy natural conversations and interactive storytelling with characters, and also experience special scenarios that change depending on their level of intimacy with the characters.

[1438] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1439] Step 1:

[1440] The user enters their username and password on the login screen of their smartphone.

[1441] Input: Username and password

[1442] Output: Login request data

[1443] Specific action: The user device sends the entered authentication information to the server.

[1444] Step 2:

[1445] The server receives the login request data and compares it with the authentication database.

[1446] Input: Username and password

[1447] Output: Authentication result (success or failure)

[1448] Specific operation: If authentication is successful, the data from the home screen is sent to the user's device; if it fails, an error message is sent.

[1449] Step 3:

[1450] The user opens the character selection screen on the home screen and selects their preferred character.

[1451] Input: Character selection information

[1452] Output: Character selection data

[1453] Specific operation: The user device sends the selected character information to the server.

[1454] Step 4:

[1455] The server loads initial data and background information corresponding to the selected character and sends the chat screen data to the user's device.

[1456] Input: Character selection data

[1457] Output: Chat screen data

[1458] Specific operation: The server retrieves the character's initial data and background information from the database and generates the chat screen.

[1459] Step 5:

[1460] Users type and send messages in the chat screen.

[1461] Input: User message

[1462] Output: Chat message data

[1463] Specific action: The user device sends the entered message to the server.

[1464] Step 6:

[1465] The server receives chat message data and analyzes it using a natural language processing engine (such as spaCy).

[1466] Input: Chat message data

[1467] Output: Analysis results

[1468] Specific operation: The natural language processing engine analyzes the message and identifies the appropriate response.

[1469] Step 7:

[1470] The server generates an appropriate response based on the analysis results of the natural language processing engine and sends that response to the user device.

[1471] Input: Analysis results

[1472] Output: Response message

[1473] Specific operation: Based on the analysis results, a response message is generated and sent to the user device.

[1474] Step 8:

[1475] The user checks the response of the character displayed on the device and continues the conversation.

[1476] Input: Response message

[1477] Output: User response messages

[1478] Specific action: Based on the response displayed on the chat screen, enter the following message.

[1479] Step 9:

[1480] The server determines the direction of the story based on user input, and new scenes and events occur.

[1481] Input: User response message

[1482] Output: Data for new scenes and events

[1483] Specific operation: The system re-analyzes the chat content, generates branching scenarios, and displays them on the user's device.

[1484] Step 10:

[1485] The server calculates intimacy based on user input and choices, and loads data to display special scenarios or events if a certain threshold is exceeded.

[1486] Input: Conversation history and choice data

[1487] Output: Intimacy and special scenario data

[1488] Specific operation: The system calculates intimacy based on an intimacy algorithm and generates a special scenario if the conditions are met.

[1489] Step 11:

[1490] Users can experience special scenarios and deepen their relationships with the characters.

[1491] Input: Special scenario data

[1492] Output: User experience record

[1493] Specific operation: The content of the special scenario is displayed on the user's device, and the user enjoys it.

[1494] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1495] This invention combines an emotion engine with a system in which the story branches and events occur through conversations with a character selected by the user. This system uses natural language processing and emotion recognition technology to provide a more dynamic and emotional story development.

[1496] Specific program processing

[1497] 1. User Login and Authentication

[1498] Terminal: The user launches the application from their terminal, and the login screen is displayed. They enter their username and password and click the "Login" button.

[1499] Terminal: Sends the entered username and password to the server.

[1500] Server: The server compares the received username and password with the database and generates an authentication result. If authentication is successful, it prepares the data for the home screen. If authentication fails, it generates an error message.

[1501] Server: Sends the authentication result (success or failure) to the terminal.

[1502] Device: If authentication is successful, the home screen will be displayed. If authentication fails, an error message will be displayed on the login screen, prompting the user to re-enter their information.

[1503] 2. Character Selection

[1504] Terminal: Upon successful authentication, the character selection screen will be displayed.

[1505] User: Select your preferred character from the list.

[1506] Terminal: Sends selection information to the server.

[1507] Server: Loads the initial and background information of the selected character from the database and generates data for starting a chat.

[1508] Server: Sends data to the terminal to start the chat.

[1509] Terminal: The chat screen appears, and the initial message of the selected character is displayed to the user.

[1510] 3. Start chat

[1511] User: Type your message in the chat screen and click the "Send" button.

[1512] Terminal: Sends user messages to the server.

[1513] Server: Analyzes received messages using a natural language processing engine and generates appropriate responses based on the analysis results. Furthermore, it analyzes the user's emotions using an emotion engine.

[1514] Server: Based on the results of natural language processing and emotion recognition, it generates a refined response and sends it to the terminal.

[1515] Terminal: Displays the character's reply to the user.

[1516] 4. Story branching and event occurrences

[1517] Server: Analyzes chat content and emotion recognition results to determine the direction of the story's progression. For example, if "joy" is recognized from the user's message, it selects a scenario in which the character proposes a "fun event." Based on the emotion recognition results, it seamlessly adjusts events and scenarios.

[1518] Terminal: Displays new scenes and events received from the server.

[1519] User: Can select messages and options to advance the conversation.

[1520] 5. Intimacy level calculation and special scenario display

[1521] Server: Calculates intimacy based on user input, choices, and emotion recognition results. For example, if a user sends a positive message to a character, intimacy increases.

[1522] Terminal: When intimacy exceeds a certain threshold, data for special scenarios and events will be loaded and displayed.

[1523] Users can experience special scenarios and deepen their relationships with the characters.

[1524] Specific example

[1525] Making a date from the first meeting

[1526] 1. Login and Character Selection

[1527] The user logs into the system from their terminal and selects "Character A" from the displayed list. The server receives this selection, loads the initial information for "Character A," and sends the chat screen to the user's terminal.

[1528] 2. Start the conversation

[1529] The user sends the message "Hello, nice to meet you." The server receives this message, analyzes it using natural language processing and an emotion engine, generates the response "Hello, how are you?", and sends it to the terminal. The terminal then displays this response to the user.

[1530] 3. Story progression and event occurrence

[1531] When the user asks, "What are you doing today?", the server analyzes this and, if the emotion engine recognizes an interest, selects a scenario that suggests, "I'm planning to go shopping. Would you like to join me?" This response is sent to the device, and if the user replies, "Of course, I'll go with you," a date event occurs.

[1532] 4. Intimacy and Special Scenarios

[1533] During a dating event, if the user selects the option "Tell me what you like?", the server increases the intimacy level. Furthermore, if the emotion engine detects the user's "enjoyment," a special dating scenario will be loaded, which the user can then experience.

[1534] The above are specific examples of embodiments for carrying out the invention. This system allows users to enjoy individual story experiences and emotional continuity through conversations and choices with characters. The introduction of an emotion engine enables dynamic story development that responds to the user's emotions, providing a richer user experience.

[1535] The following describes the processing flow.

[1536] Step 1:

[1537] Terminal: The user launches the application from their terminal, and the login screen is displayed. They enter their username and password and click the "Login" button.

[1538] Step 2:

[1539] Terminal: Sends the entered username and password to the server.

[1540] Step 3:

[1541] Server: The server compares the received username and password with the database and generates an authentication result. If authentication is successful, it prepares the data for the home screen. If authentication fails, it generates an error message.

[1542] Step 4:

[1543] Server: Sends the authentication result (success or failure) to the terminal.

[1544] Step 5:

[1545] Device: If authentication is successful, the home screen will be displayed. If authentication fails, an error message will be displayed on the login screen, prompting the user to re-enter their information.

[1546] Step 6:

[1547] Device: The user selects their preferred character from the character selection menu on the home screen.

[1548] Step 7:

[1549] Terminal: Sends information about the selected character to the server.

[1550] Step 8:

[1551] Server: Loads initial character information and background information from the database and generates data for starting a chat.

[1552] Step 9:

[1553] Server: Sends data to the terminal to start the chat.

[1554] Step 10:

[1555] Terminal: The chat screen appears, and the initial message of the selected character is displayed to the user.

[1556] Step 11:

[1557] User: Type your message in the chat screen and click the "Send" button.

[1558] Step 12:

[1559] Terminal: Sends user messages to the server.

[1560] Step 13:

[1561] Server: Analyzes received messages using a natural language processing engine. Furthermore, analyzes the user's emotions using an emotion engine.

[1562] Step 14:

[1563] Server: Generates tailored responses based on the results of natural language processing and sentiment recognition.

[1564] Step 15:

[1565] Server: Sends the generated response to the terminal.

[1566] Step 16:

[1567] Terminal: Displays the received response message to the user.

[1568] Step 17:

[1569] Server: Analyzes the user's conversation content and emotion recognition results to determine the direction of the story's progression. For example, if "joy" is recognized from the user's message, it selects a scenario in which the character proposes a "fun event."

[1570] Step 18:

[1571] Server: Loads the necessary scene and event data.

[1572] Step 19:

[1573] Server: Sends loaded scene and event data to the terminal.

[1574] Step 20:

[1575] Terminal: Displays data from received scenes and events.

[1576] Step 21:

[1577] User: Select options for new scenes or events and send the selection results to the server.

[1578] Step 22:

[1579] Server: Updates intimacy based on selection results. Dynamically adjusts intimacy based on the user's emotion recognition results.

[1580] Step 23:

[1581] Server: When intimacy exceeds a certain threshold, it loads data for special scenarios or events.

[1582] Step 24:

[1583] Server: Sends data for special scenarios and events to terminals.

[1584] Step 25:

[1585] Terminal: Displays special scenarios and events.

[1586] Step 26:

[1587] User: Experience special scenarios and events, and continue the conversation.

[1588] Step 27:

[1589] Server: Periodically saves user progress data and saves all data at the end of the session.

[1590] Step 28:

[1591] User: Click the logout button to end the session.

[1592] Step 29:

[1593] Terminal: Sends a logout request to the server.

[1594] Step 30:

[1595] Server: Processes the received logout request, terminates the session, and checks the saved data.

[1596] Step 31:

[1597] Terminal: Display a logout completion screen to the user.

[1598] The above describes the specific processing flow in the system of the present invention. This system allows users to enjoy dynamically unfolding stories and events through conversations and choices with characters. The introduction of an emotion engine enables dynamic story development in response to the user's emotions, providing a richer user experience. For example, when a user asks, "What do you like?", the emotion engine recognizes "affection" and displays a special date scenario.

[1599] (Example 2)

[1600] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1601] Traditional dialogue systems have a problem where conversations with characters are formal, and dynamic story branching and event generation that respond to user emotions and choices cannot be realized. Furthermore, the lack of intimacy level calculations that reflect user emotions and the display of special scenarios have resulted in a limited user experience.

[1602] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1603] In this invention, the server includes means for branching the story and causing events through conversations with a character selected by the user; means for the system to calculate intimacy based on the conversation content and emotion recognition results, and display a special scenario when certain conditions are met; means for analyzing the chat content using natural language processing and generating an appropriate response based on the analysis results and emotion recognition results; means for seamlessly adjusting the story branching and event occurrences based on the user's emotion recognition results; and means for dynamically generating and displaying special scenarios and events based on the calculation of intimacy. This enables a dynamic and emotional story development and a rich user experience based on the user's emotions and choices.

[1604] A "user" is someone who utilizes this system and experiences the story progression and events through interaction with characters.

[1605] A "character" is a virtual person or entity that interacts with the user within the system, and has the role of responding dynamically according to the user's choices and emotional recognition results.

[1606] "Conversation" refers to the exchange of messages between the user and the character, and is generally conducted in text format.

[1607] "Story branching" refers to the change in the direction of the narrative based on the user's choices and emotional responses, resulting in different events and scenarios.

[1608] An "event" refers to a specific scene or occurrence within a story, and its content may change depending on the user's choices and emotions.

[1609] "Intimacy level" is a numerical or metric representation of the depth of the relationship between the user and the character, and it fluctuates in response to the user's actions and statements.

[1610] A "special scenario" refers to a special scene or story that appears when the intimacy level meets certain conditions, and it contains unique content that differs from the regular scenario.

[1611] "Emotion recognition" is a technology that analyzes a user's input and actions to identify their emotions and determine their emotional state, such as joy, interest, or sadness.

[1612] "Natural language processing" is a technology that analyzes user messages to understand their meaning and intent, and involves text analysis and response generation.

[1613] A "response" is a message that a character returns in response to a message from the user, and is generated based on natural language processing and emotion recognition results.

[1614] Embodiments of this invention will be described in detail below. This system has a mechanism in which the story branches and events occur through conversations with a character selected by the user. In addition, it can calculate intimacy based on the conversation content and emotion recognition results, and display a special scenario when certain conditions are met.

[1615] System Overview

[1616] First, let me explain the system's overview. This system consists of three components: a server, a terminal, and a user, all of which interact with each other. The server is responsible for key processes such as story branching, event generation, emotion recognition, intimacy calculation, and natural language processing. The terminal provides the user's interface and communicates with the server. The user operates the terminal to advance conversations with the characters.

[1617] Hardware and software to be used

[1618] This system uses the following hardware and software:

[1619] Server hardware: A server with a high-performance processor, sufficient memory, and fast database access capabilities.

[1620] Database: A relational database management system (RDBMS) such as MySQL.

[1621] Natural language processing engine: High-performance generative AI models such as GPT-4.

[1622] Emotion recognition engine: Emotion analysis tools such as Emotion AI.

[1623] Details of the program processing

[1624] This section provides a detailed explanation of the program processing performed by the system. The specific processing steps for each stage are shown below.

[1625] User login and authentication

[1626] 1. User: Launch the application from your device and enter your username and password on the login screen.

[1627] 2. Terminal: Sends the entered information to the server.

[1628] 3. Server: Matches the information with the database, generates an authentication result, and sends it to the terminal.

[1629] 4. Device: Displays the home screen or an error message based on the authentication result.

[1630] Character Selection

[1631] 1. Device: After successful authentication, the character selection screen will be displayed.

[1632] 2. User: Select a character from the list.

[1633] 3. Terminal: Sends selection information to the server.

[1634] 4. Server: Loads the character's initial information, generates data for starting a chat, and sends it to the terminal.

[1635] 5. Terminal: Displays the chat screen and the character's initial message.

[1636] Start chat

[1637] 1. User: Enter your message.

[1638] 2. Terminal: Sends the entered message to the server.

[1639] 3. Server: Analyzes messages using a natural language processing engine and analyzes emotions using an emotion engine.

[1640] 4. Server: Generates a response and sends it to the terminal.

[1641] 5. Terminal: Displays the character's reply.

[1642] Story branching and event occurrences

[1643] 1. Server: Analyzes chat content and emotion recognition results to determine the direction of the story's progression.

[1644] 2. Terminal: Displays new scenes and events.

[1645] 3. User: Can continue the conversation.

[1646] Intimacy level calculation and special scenario display

[1647] 1. Server: Calculates intimacy levels based on user statements and emotion recognition results.

[1648] 2. Device: When the intimacy level exceeds a certain threshold, a special scenario will be displayed.

[1649] 3. User: Experience a special scenario.

[1650] Specific example

[1651] Specific examples are given below.

[1652] Making a date from the first meeting

[1653] The user logs into the system from their device and selects "Character A". The server receives the selection information, loads the initial information for "Character A", and sends the chat screen to the user's device. When the user sends the message "Hi, nice to meet you", the server analyzes it and generates the response "Hi, how are you?" which is sent to the device. When the user continues with "What are you doing today?", the server analyzes this and, if it recognizes the user's "interest", selects a scenario that suggests "I'm planning to go shopping. Would you like to come along?". This response is sent to the device, and if the user replies "Of course, I'll go with you", a date event occurs. During the date event, if the user selects "Tell me what you like?", the server increases the intimacy level. Furthermore, if the emotion engine detects the user's "enjoyment", a special date scenario is loaded, which the user can experience.

[1654] Example of a prompt

[1655] "Ask Character A what they're doing today and guide them towards a date event."

[1656] In this embodiment of the invention, users can enjoy a dynamic and emotional storyline through emotional interaction with characters. By using an emotion recognition engine and a natural language processing engine together, more personalized responses become possible, significantly improving the user experience.

[1657] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1658] The processing flow of this system's program (explained in steps)

[1659] Step 1: Display the login screen

[1660] Input: The user launches the application.

[1661] Action: The device launches the application and displays the login screen.

[1662] Output: The login screen is displayed to the user.

[1663] Step 2: Enter and submit user authentication information.

[1664] Input: The user enters their username and password and clicks the login button.

[1665] Operation: The terminal sends the entered username and password to the server. The data is encrypted before transmission.

[1666] Output: User authentication information is sent to the server.

[1667] Step 3: Authentication Process

[1668] Input: The username and password received by the server.

[1669] Operation: The server connects to the database and verifies the username and password. If successful, it generates an authentication token and prepares data for the home screen. If it fails, it prepares an error message.

[1670] Output: Authentication results are generated, and an authentication token or error message is generated.

[1671] Step 4: Submit and display authentication results

[1672] Input: Authentication result and authentication token or error message.

[1673] Operation: The server sends the authentication result to the device. Based on the received data, the device displays either the home screen or an error message.

[1674] Output: The home screen is displayed, or an error message is displayed on the login screen.

[1675] Step 5: Display the character selection screen.

[1676] Input: User who successfully authenticated.

[1677] Action: The device displays the character selection screen.

[1678] Output: The character selection screen is displayed to the user.

[1679] Step 6: Submit character selection information

[1680] Input: The user selects a character and clicks the select button.

[1681] Operation: The terminal sends the ID of the selected character to the server.

[1682] Output: Character ID information is sent to the server.

[1683] Step 7: Loading character information

[1684] Input: The character ID received by the server.

[1685] Operation: The server loads initial information for the selected character from the database and generates data to start the chat.

[1686] Output: Data for starting a chat is generated.

[1687] Step 8: Send and display data to start the chat.

[1688] Input: Data to start the chat.

[1689] Operation: The server sends data to the terminal to initiate a chat. The terminal receives the data, displays the chat screen, and shows the character's initial message.

[1690] Output: The chat screen and the character's initial message are displayed.

[1691] Step 9: Enter and send the user message.

[1692] Input: The user enters a message and clicks the send button.

[1693] Operation: The terminal sends the entered message to the server.

[1694] Output: The user message is sent to the server.

[1695] Step 10: Message Analysis

[1696] Input: User messages received by the server.

[1697] Operation: The server analyzes messages using a natural language processing engine (such as GPT-4). It also analyzes the user's emotions using an emotion recognition engine (such as Emotion AI).

[1698] Output: Message analysis results and sentiment recognition results are generated.

[1699] Step 11: Generating a response message

[1700] Input: Message analysis results and sentiment recognition results.

[1701] Operation: Based on these results, the server generates an appropriate response message.

[1702] Output: A response message is generated.

[1703] Step 12: Sending and displaying the response message

[1704] Input: Response message.

[1705] Operation: The server sends a response message to the terminal. The terminal receives the message and displays it on the chat screen.

[1706] Output: The response message is displayed on the chat screen.

[1707] Step 13: Deciding on a story branch

[1708] Input: Chat content and sentiment recognition results.

[1709] Operation: The server determines the direction of the story's progression based on this data.

[1710] Output: The results of the story branching are generated.

[1711] Step 14: Generating new scenes and events

[1712] Input: The result of the story branching.

[1713] Operation: The server generates data for new scenes and events and sends it to the terminal.

[1714] Output: New scene and event data is generated.

[1715] Step 15: Displaying new scenes and events

[1716] Input: Data for new scenes or events.

[1717] Operation: The terminal displays the received data.

[1718] Output: New scenes or events are displayed on the terminal.

[1719] Step 16: Calculating Intimacy Level

[1720] Input: User input, choices, and sentiment recognition results.

[1721] Operation: The server calculates intimacy based on this data.

[1722] Output: An intimacy value is generated.

[1723] Step 17: Generating a Special Scenario

[1724] Input: Intimacy level value.

[1725] Operation: When intimacy exceeds a certain threshold, the server generates special scenarios or events.

[1726] Output: Data for special scenarios and events is generated.

[1727] Step 18: Displaying Special Scenarios

[1728] Input: Data for special scenarios or events.

[1729] Operation: The terminal displays the received data.

[1730] Output: Special scenarios or events will be displayed on the terminal.

[1731] The above describes the detailed processing steps and specific operation of this system's program.

[1732] (Application Example 2)

[1733] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1734] Traditional interactive storytelling systems were unable to dynamically branch stories in response to user emotions, instead relying on fixed story patterns. This resulted in a limited user experience and a lack of entertainment value and immersion. Furthermore, the calculation of intimacy levels and the resulting display of special scenarios were also fixed, making real-time, emotion-based responses difficult.

[1735] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1736] In this invention, the server includes means for branching the story and triggering events through conversations with a character selected by the user; means for the system to calculate intimacy based on the conversation content and display a special scenario when certain conditions are met; means for analyzing the chat content using natural language processing and generating an appropriate response based on the analysis results; means for recognizing the user's emotions and dynamically branching the story according to those emotions; and means for seamlessly displaying special events and scenarios based on intimacy and emotions. This enables flexible and dynamic story development that responds to the user's emotions.

[1737] A "user" is someone who uses the system to enjoy content through conversations with characters.

[1738] A "character" is a virtual entity or person that a user can select to engage in conversation with.

[1739] A "story" refers to a series of events or occurrences that unfold through conversations with characters.

[1740] "Branching" refers to the point where the direction of the story changes depending on the user's choices and emotions.

[1741] An "event" refers to a specific occurrence or action that takes place within a story.

[1742] "Intimacy level" is an indicator that shows the depth of the relationship between the user and the character.

[1743] A "special scenario" is a scenario that appears when certain conditions are met, and is different from the normal storyline.

[1744] "Natural language processing" is the technology that uses computers to understand, analyze, and generate human language.

[1745] "Emotions" refer to the user's feelings or emotional state.

[1746] "Emotion recognition" is a technology that analyzes and identifies emotions from a user's words and actions.

[1747] "Dynamic" refers to the property of changing or adapting in real time according to the situation.

[1748] "Seamlessly" refers to something that is continuous and uninterrupted.

[1749] The system of this invention provides a mechanism in which the story branches and events occur through conversations with characters selected by the user. The program to realize this is structured as follows.

[1750] The system requires a smartphone and a server as hardware. The smartphone is the device that the user directly interacts with, while the server processes and stores various types of data. As for software, the "transformers" library, a natural language processing engine, is used. This library is based on a generative AI model and uses a "sentiment-analysis" model for emotion recognition.

[1751] When a user logs into the system using their smartphone, the server verifies the user's credentials against the database and performs authentication. If authentication is successful, a character selection screen appears on the device. The user selects their preferred character, and the server loads the data for the selected character and prepares to start chatting.

[1752] When a user types and sends a message to a character, the message is sent from the terminal to the server. The server uses the "transformers" library to parse the message and perform emotion recognition. Based on the results, it generates an appropriate response and sends it to the terminal. The terminal then displays this response to the user.

[1753] Based on the chat content and the results of emotion recognition, the server determines the branching of the story. For example, if joy is recognized from the user's message, it selects a scenario that suggests a pleasant event. It also calculates intimacy and, if it exceeds a certain threshold, loads a special scenario and presents it to the user.

[1754] As a concrete example, consider a scenario where a user logs in with the username "example_user" and password "123456," and selects "Character A." When the user sends the message "I'm relieved that work is over," the server recognizes this as "joy" and generates a response asking, "Would you like to share something fun?" This system enables flexible and dynamic story development that responds to the user's emotions, providing a rich user experience.

[1755] Furthermore, the following input can be considered as an example of a prompt statement.

[1756] User's input message: "Every day is tough..."

[1757] Processing result: "Did something upsetting happen?"

[1758] The above describes the embodiments for carrying out this invention.

[1759] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1760] Step 1:

[1761] The user launches the application from their smartphone, and the login screen appears. They enter their username and password and click the "Login" button.

[1762] Input: Username, Password

[1763] Output: Authentication Request

[1764] Step 2:

[1765] The terminal sends the entered username and password to the server. The server compares the received authentication information with its database and generates an authentication result.

[1766] Input: Authentication Request

[1767] Output: Authentication result (success or failure)

[1768] Step 3:

[1769] If authentication is successful, the server prepares the home screen data and sends it to the device. If authentication fails, it generates an error message and sends it to the device. If authentication is successful, the device displays the home screen; if it fails, it displays the error message on the login screen.

[1770] Input: Authentication result

[1771] Output: Home screen data or error messages

[1772] Step 4:

[1773] The user moves from the home screen to the character selection screen and selects a character from the displayed list. The device then sends the selection information to the server.

[1774] Input: Character Selection

[1775] Output: Character selection information

[1776] Step 5:

[1777] Based on the received character selection information, the server loads the selected character's initial and background information from the database, generates data for starting a chat, and sends it to the terminal. The terminal displays the chat screen and shows the selected character's initial message.

[1778] Input: Character selection information

[1779] Output: Chat start data, initial message

[1780] Step 6:

[1781] The user types a message in the chat screen and clicks the "Send" button. The device sends the user's message to the server.

[1782] Input: Message

[1783] Output: Message sending request

[1784] Step 7:

[1785] The server analyzes the received message using the "transformers" library and performs emotion recognition using a generative AI model. Based on the analysis results and emotion recognition results, it generates an appropriate response message and sends it to the terminal.

[1786] Input: Message sending request

[1787] Output: Response message

[1788] Step 8:

[1789] The terminal displays the response message received from the server to the user. The user can then continue chatting.

[1790] Input: Response message

[1791] Output: Display of response message

[1792] Step 9:

[1793] The server analyzes the chat content and emotion recognition results to determine the direction of the story. For example, if it recognizes "joy" from the user's message, it selects a scenario in which the character proposes a "fun event." New scenes and events are generated and sent to the device according to specific conditions.

[1794] Input: Sentiment recognition result

[1795] Output: Event data, new scene data

[1796] Step 10:

[1797] The terminal displays new scenes and events received from the server to the user. The user can then enter further messages or options to continue the conversation.

[1798] Input: New scene data, event data

[1799] Output: Display new scene and events

[1800] Step 11:

[1801] The server calculates intimacy based on user input, choices, and emotion recognition results. For example, if a user sends a positive message to a character, intimacy increases. If intimacy exceeds a certain threshold, a special scenario or event is generated and sent to the terminal.

[1802] Input: User input, emotion recognition results

[1803] Output: Intimacy level, special scenario data

[1804] Step 12:

[1805] The terminal displays special scenarios and events sent from the server, and the user can continue to interact with them.

[1806] Input: Special Scenario Data

[1807] Output: Display of special scenarios

[1808] The above outlines the specific processing steps of this system's program.

[1809] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1810] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1811] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1812] [Fourth Embodiment]

[1813] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1814] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1815] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1816] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1817] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1818] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1819] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1820] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1821] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1822] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1823] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1824] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1825] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1826] This invention describes a system in which a story branches and events occur through conversations between a user and a character. This system uses natural language processing to analyze the user's messages and generate appropriate responses, and includes means for calculating intimacy levels and displaying special scenarios.

[1827] Specific program processing

[1828] 1. User Login and Authentication

[1829] Terminal: The user opens the login screen from their terminal and enters their username and password.

[1830] Server: The server compares the username and password received from the terminal with the authentication database. If authentication is successful, it sends the home screen data to the terminal. If authentication fails, it sends an error message.

[1831] 2. Character Selection

[1832] Terminal: Upon successful authentication, the character selection screen will be displayed.

[1833] User: Select your preferred character from the list.

[1834] Terminal: Sends selection information to the server.

[1835] Server: Loads the initial data and background information of the selected character and sends the chat screen data to the terminal.

[1836] 3. Start chat

[1837] User: Enter a message such as "Hello" in the chat screen and send it.

[1838] Terminal: Sends a message to the server.

[1839] Server: Receives messages and parses them using a natural language processing engine. Based on the analysis results, it generates an appropriate response such as "Hello, how are you?" and sends it to the terminal.

[1840] Terminal: Displays the character's reply to the user.

[1841] 4. Story branching and event occurrences

[1842] Server: Analyzes chat content and determines the direction of the story. For example, if the user asks "What are you doing today?", the server selects a scenario where the character answers "I'm planning to go shopping."

[1843] Terminal: Displays new scenes and events received from the server.

[1844] User: Can select messages and options to advance the conversation.

[1845] 5. Intimacy level calculation and special scenario display

[1846] Server: Calculates intimacy based on user input and choices. For example, if a user sends a friendly message to a character, intimacy increases.

[1847] Terminal: When intimacy exceeds a certain threshold, data for special scenarios and events will be loaded and displayed.

[1848] Users can experience special scenarios and deepen their relationships with the characters.

[1849] Specific example

[1850] Making a date from the first meeting

[1851] 1. Login and Character Selection

[1852] The user logs into the system from their terminal and selects "Beautiful Girl Character A" from the displayed list. The server receives this selection, loads the initial information for "Beautiful Girl Character A," and sends the chat screen to the user's terminal.

[1853] 2. Start the conversation

[1854] The user sends the message "Hello, nice to meet you." The server receives this message, parses it using natural language processing, generates the response "Hello, how are you?", and sends it to the terminal. The terminal then displays this response to the user.

[1855] 3. Story progression and event occurrence

[1856] When the user replies, "I'm fine, what are you doing today?", the server analyzes this and branches the scenario to, "I'm planning to go shopping. Would you like to come along?". This response is sent to the terminal, and if the user replies, "Of course, I'll go with you," a date event occurs.

[1857] 4. Intimacy and Special Scenarios

[1858] During a dating event, if the user selects the option "Tell me what you like?", the server will increase the intimacy level. When the intimacy level reaches a certain threshold, "Beautiful Girl Character A" will propose a special scenario to the user, which will then be displayed on the device. The user can then enjoy this special scenario.

[1859] The above are specific examples of embodiments for carrying out the invention. This system allows users to enjoy individual story experiences and emotional continuity through conversations and choices with characters.

[1860] The following describes the processing flow.

[1861] Step 1:

[1862] Terminal: The user launches the application from their terminal, and the login screen is displayed. The user enters their username and password and clicks the "Login" button.

[1863] Step 2:

[1864] Terminal: Sends the entered username and password to the server.

[1865] Step 3:

[1866] Server: The server compares the received username and password with the database and generates an authentication result. If authentication is successful, it prepares the data for the home screen. If authentication fails, it generates an error message.

[1867] Step 4:

[1868] Server: Sends the authentication result (success or failure) to the terminal.

[1869] Step 5:

[1870] Device: If authentication is successful, the home screen will be displayed. If authentication fails, an error message will be displayed on the login screen, prompting the user to re-enter their information.

[1871] Step 6:

[1872] Device: The user selects their preferred character from the character selection menu on the home screen.

[1873] Step 7:

[1874] Terminal: Sends information about the selected character to the server.

[1875] Step 8:

[1876] Server: Loads initial character information and background information from the database and generates data for starting a chat.

[1877] Step 9:

[1878] Server: Sends data to the terminal to start the chat.

[1879] Step 10:

[1880] Terminal: The chat screen appears, and the initial message of the selected character is displayed to the user.

[1881] Step 11:

[1882] User: Type your message in the chat screen and click the "Send" button.

[1883] Step 12:

[1884] Terminal: Sends user messages to the server.

[1885] Step 13:

[1886] Server: The server analyzes received messages using a natural language processing engine and generates an appropriate response based on the analysis results.

[1887] Step 14:

[1888] Server: Sends the generated response to the terminal.

[1889] Step 15:

[1890] Terminal: Displays the received response message to the user.

[1891] Step 16:

[1892] Server: Analyzes user conversations and determines the direction of the story's progression. Loads new scene and event data as needed.

[1893] Step 17:

[1894] Server: Sends necessary scene and event data to the terminal.

[1895] Step 18:

[1896] Terminal: Displays data from received scenes and events.

[1897] Step 19:

[1898] Terminal: The user selects options for new scenes or events and sends the selection results to the server.

[1899] Step 20:

[1900] Server: Updates intimacy based on the selection results. If intimacy reaches a certain condition, loads a special scenario.

[1901] Step 21:

[1902] Server: Sends a special scenario to the terminal based on the level of intimacy.

[1903] Step 22:

[1904] Terminal: Displays special scenarios to the user.

[1905] Step 23:

[1906] User: Experience a special scenario and continue the conversation.

[1907] Step 24:

[1908] Server: Periodically saves user progress data and saves all data at the end of the session.

[1909] Step 25:

[1910] User: Click the logout button to end the session.

[1911] Step 26:

[1912] Terminal: Sends a logout request to the server.

[1913] Step 27:

[1914] Server: Processes the received logout request, terminates the session, and checks the saved data.

[1915] Step 28:

[1916] Terminal: Display a logout completion screen to the user.

[1917] The above describes the specific processing flow in the system of the present invention. This system allows users to enjoy dynamically unfolding stories and events through conversations with characters.

[1918] (Example 1)

[1919] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1920] Traditional character conversation systems often result in a fixed user experience with limited dynamic branching of storylines and events. Furthermore, they restrict the calculation of intimacy levels based on conversation content and the display of special scenarios. This leads to limited user interaction and a lack of long-term appeal.

[1921] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1922] In this invention, the server includes means for a user to log in and authenticate from a terminal; means for a story to branch and events to occur through conversations with a virtual character selected by the user; means for the system to calculate intimacy based on the conversation content and display a special scenario when certain conditions are met; means for analyzing the chat content using natural language processing with a generation AI model and generating an appropriate response based on the analysis results; and means for dynamically loading and displaying scenarios in response to changes in intimacy. This diversifies user interaction and enables dynamic and seamless story progression and event occurrence.

[1923] A "terminal" is a physical or virtual device used by a user to access and operate a system.

[1924] A "server" is a centralized computer system that receives, processes, and analyzes user input information, and then returns the results to the terminal.

[1925] "Logging in" is a series of processes in which a user enters authentication information into a system and verifies and grants access rights.

[1926] "Authentication" is the process of verifying whether the entered user information matches the registered data.

[1927] A "virtual character" is a digital persona designed to interact with users within a system.

[1928] "Story branching" refers to the process where different storylines or events are generated depending on the user's choices and conversations.

[1929] An "event" is a specific action or occurrence that happens during the story and influences the user's choices.

[1930] "Intimacy level" is an indicator that measures the depth of the relationship between the user and the virtual character, and is calculated as points based on specific actions and messages within the system.

[1931] A "special scenario" refers to a specific story or event that is only displayed when certain conditions are met regarding intimacy.

[1932] A "generative AI model" is an engine that uses artificial intelligence to perform natural language processing, and is responsible for analyzing user messages and generating appropriate responses.

[1933] "Natural language processing" is a technology that enables computers to understand and process human language.

[1934] "Analysis results" refer to the information and conclusions obtained when a generative AI model processes user input.

[1935] "Dynamically loading and displaying scenarios" refers to the process by which the system loads new scenarios and events in real time in response to user actions and displays them on the device.

[1936] This embodiment is a system that dynamically generates and branches a story and triggers events through interactive conversations between the user and a virtual character. This system functions through the cooperation of a terminal, a server, and the user.

[1937] The server manages authentication information when a user logs in from a terminal to access the system. This authentication information is stored in a database (e.g., MySQL) and verified using SQL queries. After the user enters their username and password, the server authenticates them and establishes the login status.

[1938] After logging in, users can access a character selection screen on their device. For example, multiple virtual characters may be displayed as thumbnails, allowing the user to select one. The initial data of the selected character (name, profile, background story, etc.) is sent from the server to the device in JSON format. The device then uses this data to configure and display the chat screen to the user.

[1939] At the start of a chat, the user types and sends a text message. This message is sent from the terminal to the server, where it is analyzed using a generative AI model (e.g., GPT-4). The generative AI model analyzes the message content and generates an appropriate response. This response is sent to the terminal in JSON format, and the terminal displays the character's reply to the user.

[1940] The story branches based on the conversation, and events occur. For example, if the user asks, "What are you doing today?", the server analyzes this and selects the scenario, "I'm planning to go shopping." Next, information about this scenario is sent to the device, and a new storyline is displayed.

[1941] During the conversation, the system calculates intimacy based on the user's responses. Intimacy increases when favorable messages or specific choices are selected. If this intimacy exceeds a certain threshold, the server loads and sends a special scenario or event to the device. The device then displays the special scenario, allowing the user to enjoy a new experience.

[1942] For example, if a user inputs "Hello, beautiful girl character A. What are your plans for today?", the server uses a generative AI model to analyze the scenario and generate a response such as "I'm planning to go shopping. Would you like to come along?". This response is sent to the terminal and displayed to the user. If the user replies "Of course, I'll go with you," a date event occurs, and the intimacy level between the user and the virtual character increases.

[1943] This system allows users to enjoy personalized story experiences and emotional continuity through conversations with virtual characters.

[1944] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1945] Step 1: User Login

[1946] User: The user opens the login screen from their device and enters their username and password. The login process is completed by pressing the login button.

[1947] Input: Username, Password

[1948] Terminal: Sends the entered username and password to the server.

[1949] Server: The server compares the received username and password with the authentication information in the database (e.g., MySQL). It uses an SQL query to perform the comparison and obtain the authentication result. For example, it executes the query "SELECT user_id FROM users WHERE username='example' AND password='123456'".

[1950] Output: Authentication result (success or failure)

[1951] Server: If authentication is successful, the server sends the user ID and home screen data (in JSON format) to the device. For example, it may be in the format "{"status":"success","user_id":4,"home_data":{"background":"blue","notifications":"on"}}".

[1952] Step 2: Character Selection

[1953] Terminal: After successful authentication, the user is shown a character selection screen. A list of selectable characters (icons with images and names) is displayed.

[1954] User: Select your preferred character from the list and press the select button to complete the input.

[1955] Input: ID of the selected character

[1956] Terminal: Sends the ID of the selected character to the server.

[1957] Server: Retrieves the character's initial data from the database and sends it to the terminal in JSON format. For example, "{"character_id":"A","name":"Beautiful Girl Character A","profile":"Profile Information"}".

[1958] Output: Initial character data (JSON format)

[1959] Terminal: Based on the initial data received, configures and displays the chat screen to the user.

[1960] Step 3: Start chatting

[1961] User: Enter a message such as "Hello" in the chat screen and press the send button to complete the input.

[1962] Input: User message

[1963] Terminal: Sends the entered message to the server.

[1964] Server: Analyzes received messages using a generation AI model (e.g., GPT-4). Generates a prompt and inputs it into the generation AI model. For example, "The user typed 'hello'. Please generate an appropriate response."

[1965] Server: Sends the generated response to the terminal in JSON format. For example, "{"response":"Hello, how are you?"}".

[1966] Output: Generated response (JSON format)

[1967] Terminal: Displays the character's reply message to the user.

[1968] Step 4: Story branching and event occurrence

[1969] Server: Continuously analyzes chat content to determine the direction of the story. If a user types "What are you doing today?", it selects a scenario such as "I'm planning to go shopping" based on pre-set conditions.

[1970] Input: User message, story progression conditions

[1971] Server: Sends the data for the selected scenario to the terminal in JSON format. For example, "{"scene":"I'm planning to go shopping. Would you like to join me?"}".

[1972] Output: Scenario data (JSON format)

[1973] Terminal: Displays new scenes and events received from the server. For example, it displays a scene of a character going shopping using images and text.

[1974] User: Selects responses and options in response to displayed scenes and events, and inputs information to advance the conversation.

[1975] Step 5: Intimacy level calculation and special scenario display

[1976] Server: Applies an algorithm that calculates intimacy based on user input and choices. For example, it adds intimacy points for favorable messages.

[1977] Input: User's message and choices

[1978] Server: When intimacy exceeds a certain threshold, load data for special scenarios or events and send it to the terminal in JSON format. For example, "{'special_scenario':'Special Dating Event'}".

[1979] Output: Data for a special scenario (JSON format)

[1980] Terminal: Displays received special scenarios. For example, it provides the user with a description of a special date scene using diagrams and text.

[1981] User: Experience special scenarios and advance through new choices and conversations to deepen your relationship with the virtual character.

[1982] (Application Example 1)

[1983] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1984] Conventional interactive storytelling systems have limited conversations between users and characters, failing to adequately achieve natural interaction and increased intimacy. Furthermore, seamless, real-time branching of the story based on user choices is difficult, limiting the potential for improving the user experience. This invention solves these problems, providing more natural conversations and dynamic, real-time story branching, while also enabling the calculation of user-character intimacy and the presentation of special scenarios based on that.

[1985] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1986] In this invention, the server includes means for branching the story and triggering events through conversations with a character selected by the user; means for the system to calculate intimacy based on the conversation content and display a special scenario when certain conditions are met; means for analyzing the chat content using natural language processing and generating an appropriate response based on the analysis results; means for providing interactive storytelling through conversations between the user and a character in an application installed on a smartphone, smart glasses, head-mounted display, or robot; and means for generating and displaying new scenes and events in real time based on the conversation content and story progression. As a result, the user can enjoy natural conversations with the character and experience special scenarios according to their intimacy level, enabling a richer interactive story experience.

[1987] A "user" is someone who uses the system to engage in conversations and storytelling with characters.

[1988] A "character" is a virtual character that the user selects within the system and interacts with through conversations and stories.

[1989] "Story branching" is a mechanism where the direction of the story changes depending on the user's choices and the content of the conversations.

[1990] An "event" refers to a specific scene or occurrence that happens during the progression of the story.

[1991] "Intimacy level" is a numerical representation of the depth of the relationship between the user and the character.

[1992] A "special scenario" refers to a special storyline or scene that is only displayed when the level of intimacy reaches certain conditions.

[1993] "Natural language processing" is a technology that enables computers to understand and analyze user messages.

[1994] An "appropriate response" is a reply message to the user that the natural language processing engine generates based on its analysis results.

[1995] A "smartphone" is a portable device capable of connecting to the internet and running various applications.

[1996] "Smart glasses" are glasses-type devices equipped with the ability to display information and run applications.

[1997] A "head-mounted display" is a display device that is worn on the head.

[1998] A "robot" is a mechanical device that is capable of interacting with and being operated by a user.

[1999] "Interactive storytelling" is a technique in which the user and the system interact to advance the narrative.

[2000] "Real-time" refers to the temporality in which a system processes information instantly and responds immediately to user actions and conversations.

[2001] A "new scene" is a new moment that is generated during the progression of the story based on the user's choices and conversations.

[2002] The system for carrying out this invention provides interactive storytelling. Users can access the system using a smartphone or other device and converse with characters. Specific embodiments are described below.

[2003] 1. User login and authentication

[2004] Users access the login screen from a device such as a smartphone and enter their username and password.

[2005] The server verifies the entered authentication information against the authentication database. If authentication is successful, it sends the home screen data to the user's device; otherwise, it sends an error message.

[2006] 2. Character Selection

[2007] After successful authentication, the user selects their preferred character from the character selection screen that appears.

[2008] The server loads initial data and background information corresponding to the selected character and sends the chat screen to the user's device.

[2009] 3. Start chat

[2010] Users type messages in the chat screen and send messages such as "Hello."

[2011] The server receives the message and parses it using a natural language processing engine (e.g., spaCy).

[2012] Based on the analysis results, an appropriate response is generated and sent to the user's device.

[2013] The device displays the generated response to the user.

[2014] 4. Story branching and event occurrences

[2015] The server analyzes the user's chat content and determines the direction of the story's progression.

[2016] For example, if the user asks, "What are you doing today?", a scenario is selected in which the character replies, "I'm planning to go shopping." Information about this new scene or event is then sent to the user's device.

[2017] Users can advance the conversation by selecting messages and options based on the scene or event.

[2018] 5. Intimacy level calculation and special scenario display

[2019] The server calculates the level of intimacy based on the user's input and choices.

[2020] For example, when a user sends a positive message, their intimacy level increases. If the intimacy level exceeds a certain threshold, data for special scenarios or events is loaded and sent to the user's device.

[2021] Users can experience special scenarios and deepen their relationships with the characters.

[2022] Hardware and software to be used

[2023] The following hardware and software will be used to implement this system.

[2024] Hardware: Smartphones, smart glasses, head-mounted displays, robots

[2025] Software: Flask (web framework), MongoDB (database), spaCy (natural language processing engine)

[2026] Specific example

[2027] Consider the case where the user types the message "I'm fine, what are you doing today?".

[2028] The server uses a natural language processing engine to analyze the message and generates a response, "I'm planning to go shopping. Would you like to join me?", which it then sends to the user's device.

[2029] Upon receiving this response, the user replies, "Of course, I'll go with you," and a date event occurs.

[2030] During a dating event, if the user selects "Tell me what you like?", the server will increase the intimacy level. When the intimacy level reaches certain conditions, a special scenario will be displayed.

[2031] Example of a prompt

[2032] The user sends the following message to the system:

[2033] "I'm fine, what are you doing today?"

[2034] This system allows users to enjoy natural conversations and interactive storytelling with characters, and also experience special scenarios that change depending on their level of intimacy with the characters.

[2035] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[2036] Step 1:

[2037] The user enters their username and password on the login screen of their smartphone.

[2038] Input: Username and password

[2039] Output: Login request data

[2040] Specific action: The user device sends the entered authentication information to the server.

[2041] Step 2:

[2042] The server receives the login request data and compares it with the authentication database.

[2043] Input: Username and password

[2044] Output: Authentication result (success or failure)

[2045] Specific operation: If authentication is successful, the data from the home screen is sent to the user's device; if it fails, an error message is sent.

[2046] Step 3:

[2047] The user opens the character selection screen on the home screen and selects their preferred character.

[2048] Input: Character selection information

[2049] Output: Character selection data

[2050] Specific operation: The user device sends the selected character information to the server.

[2051] Step 4:

[2052] The server loads initial data and background information corresponding to the selected character and sends the chat screen data to the user's device.

[2053] Input: Character selection data

[2054] Output: Chat screen data

[2055] Specific operation: The server retrieves the character's initial data and background information from the database and generates the chat screen.

[2056] Step 5:

[2057] Users type and send messages in the chat screen.

[2058] Input: User message

[2059] Output: Chat message data

[2060] Specific action: The user device sends the entered message to the server.

[2061] Step 6:

[2062] The server receives chat message data and analyzes it using a natural language processing engine (such as spaCy).

[2063] Input: Chat message data

[2064] Output: Analysis results

[2065] Specific operation: The natural language processing engine analyzes the message and identifies the appropriate response.

[2066] Step 7:

[2067] The server generates an appropriate response based on the analysis results of the natural language processing engine and sends that response to the user device.

[2068] Input: Analysis results

[2069] Output: Response message

[2070] Specific operation: Based on the analysis results, a response message is generated and sent to the user device.

[2071] Step 8:

[2072] The user checks the response of the character displayed on the device and continues the conversation.

[2073] Input: Response message

[2074] Output: User response messages

[2075] Specific action: Based on the response displayed on the chat screen, enter the following message.

[2076] Step 9:

[2077] The server determines the direction of the story based on user input, and new scenes and events occur.

[2078] Input: User response message

[2079] Output: Data for new scenes and events

[2080] Specific operation: The system re-analyzes the chat content, generates branching scenarios, and displays them on the user's device.

[2081] Step 10:

[2082] The server calculates intimacy based on user input and choices, and loads data to display special scenarios or events if a certain threshold is exceeded.

[2083] Input: Conversation history and choice data

[2084] Output: Intimacy and special scenario data

[2085] Specific operation: The system calculates intimacy based on an intimacy algorithm and generates a special scenario if the conditions are met.

[2086] Step 11:

[2087] Users can experience special scenarios and deepen their relationships with the characters.

[2088] Input: Special scenario data

[2089] Output: User experience record

[2090] Specific operation: The content of the special scenario is displayed on the user's device, and the user enjoys it.

[2091] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[2092] This invention combines an emotion engine with a system in which the story branches and events occur through conversations with a character selected by the user. This system uses natural language processing and emotion recognition technology to provide a more dynamic and emotional story development.

[2093] Specific program processing

[2094] 1. User Login and Authentication

[2095] Terminal: The user launches the application from their terminal, and the login screen is displayed. They enter their username and password and click the "Login" button.

[2096] Terminal: Sends the entered username and password to the server.

[2097] Server: The server compares the received username and password with the database and generates an authentication result. If authentication is successful, it prepares the data for the home screen. If authentication fails, it generates an error message.

[2098] Server: Sends the authentication result (success or failure) to the terminal.

[2099] Device: If authentication is successful, the home screen will be displayed. If authentication fails, an error message will be displayed on the login screen, prompting the user to re-enter their information.

[2100] 2. Character Selection

[2101] Terminal: Upon successful authentication, the character selection screen will be displayed.

[2102] User: Select your preferred character from the list.

[2103] Terminal: Sends selection information to the server.

[2104] Server: Loads the initial and background information of the selected character from the database and generates data for starting a chat.

[2105] Server: Sends data to the terminal to start the chat.

[2106] Terminal: The chat screen appears, and the initial message of the selected character is displayed to the user.

[2107] 3. Start chat

[2108] User: Type your message in the chat screen and click the "Send" button.

[2109] Terminal: Sends user messages to the server.

[2110] Server: Analyzes received messages using a natural language processing engine and generates appropriate responses based on the analysis results. Furthermore, it analyzes the user's emotions using an emotion engine.

[2111] Server: Based on the results of natural language processing and emotion recognition, it generates a refined response and sends it to the terminal.

[2112] Terminal: Displays the character's reply to the user.

[2113] 4. Story branching and event occurrences

[2114] Server: Analyzes chat content and emotion recognition results to determine the direction of the story's progression. For example, if "joy" is recognized from the user's message, it selects a scenario in which the character proposes a "fun event." Based on the emotion recognition results, it seamlessly adjusts events and scenarios.

[2115] Terminal: Displays new scenes and events received from the server.

[2116] User: Can select messages and options to advance the conversation.

[2117] 5. Intimacy level calculation and special scenario display

[2118] Server: Calculates intimacy based on user input, choices, and emotion recognition results. For example, if a user sends a positive message to a character, intimacy increases.

[2119] Terminal: When intimacy exceeds a certain threshold, data for special scenarios and events will be loaded and displayed.

[2120] Users can experience special scenarios and deepen their relationships with the characters.

[2121] Specific example

[2122] Making a date from the first meeting

[2123] 1. Login and Character Selection

[2124] The user logs into the system from their terminal and selects "Character A" from the displayed list. The server receives this selection, loads the initial information for "Character A," and sends the chat screen to the user's terminal.

[2125] 2. Start the conversation

[2126] The user sends the message "Hello, nice to meet you." The server receives this message, analyzes it using natural language processing and an emotion engine, generates the response "Hello, how are you?", and sends it to the terminal. The terminal then displays this response to the user.

[2127] 3. Story progression and event occurrence

[2128] When the user asks, "What are you doing today?", the server analyzes this and, if the emotion engine recognizes an interest, selects a scenario that suggests, "I'm planning to go shopping. Would you like to join me?" This response is sent to the device, and if the user replies, "Of course, I'll go with you," a date event occurs.

[2129] 4. Intimacy and Special Scenarios

[2130] During a dating event, if the user selects the option "Tell me what you like?", the server increases the intimacy level. Furthermore, if the emotion engine detects the user's "enjoyment," a special dating scenario will be loaded, which the user can then experience.

[2131] The above are specific examples of embodiments for carrying out the invention. This system allows users to enjoy individual story experiences and emotional continuity through conversations and choices with characters. The introduction of an emotion engine enables dynamic story development that responds to the user's emotions, providing a richer user experience.

[2132] The following describes the processing flow.

[2133] Step 1:

[2134] Terminal: The user launches the application from their terminal, and the login screen is displayed. They enter their username and password and click the "Login" button.

[2135] Step 2:

[2136] Terminal: Sends the entered username and password to the server.

[2137] Step 3:

[2138] Server: The server compares the received username and password with the database and generates an authentication result. If authentication is successful, it prepares the data for the home screen. If authentication fails, it generates an error message.

[2139] Step 4:

[2140] Server: Sends the authentication result (success or failure) to the terminal.

[2141] Step 5:

[2142] Device: If authentication is successful, the home screen will be displayed. If authentication fails, an error message will be displayed on the login screen, prompting the user to re-enter their information.

[2143] Step 6:

[2144] Device: The user selects their preferred character from the character selection menu on the home screen.

[2145] Step 7:

[2146] Terminal: Sends information about the selected character to the server.

[2147] Step 8:

[2148] Server: Loads initial character information and background information from the database and generates data for starting a chat.

[2149] Step 9:

[2150] Server: Sends data to the terminal to start the chat.

[2151] Step 10:

[2152] Terminal: The chat screen appears, and the initial message of the selected character is displayed to the user.

[2153] Step 11:

[2154] User: Type your message in the chat screen and click the "Send" button.

[2155] Step 12:

[2156] Terminal: Sends user messages to the server.

[2157] Step 13:

[2158] Server: Analyzes received messages using a natural language processing engine. Furthermore, analyzes the user's emotions using an emotion engine.

[2159] Step 14:

[2160] Server: Generates tailored responses based on the results of natural language processing and sentiment recognition.

[2161] Step 15:

[2162] Server: Sends the generated response to the terminal.

[2163] Step 16:

[2164] Terminal: Displays the received response message to the user.

[2165] Step 17:

[2166] Server: Analyzes the user's conversation content and emotion recognition results to determine the direction of the story's progression. For example, if "joy" is recognized from the user's message, it selects a scenario in which the character proposes a "fun event."

[2167] Step 18:

[2168] Server: Loads the necessary scene and event data.

[2169] Step 19:

[2170] Server: Sends loaded scene and event data to the terminal.

[2171] Step 20:

[2172] Terminal: Displays data from received scenes and events.

[2173] Step 21:

[2174] User: Select options for new scenes or events and send the selection results to the server.

[2175] Step 22:

[2176] Server: Updates intimacy based on selection results. Dynamically adjusts intimacy based on the user's emotion recognition results.

[2177] Step 23:

[2178] Server: When intimacy exceeds a certain threshold, it loads data for special scenarios or events.

[2179] Step 24:

[2180] Server: Sends data for special scenarios and events to terminals.

[2181] Step 25:

[2182] Terminal: Displays special scenarios and events.

[2183] Step 26:

[2184] User: Experience special scenarios and events, and continue the conversation.

[2185] Step 27:

[2186] Server: Periodically saves user progress data and saves all data at the end of the session.

[2187] Step 28:

[2188] User: Click the logout button to end the session.

[2189] Step 29:

[2190] Terminal: Sends a logout request to the server.

[2191] Step 30:

[2192] Server: Processes the received logout request, terminates the session, and checks the saved data.

[2193] Step 31:

[2194] Terminal: Display a logout completion screen to the user.

[2195] The above describes the specific processing flow in the system of the present invention. This system allows users to enjoy dynamically unfolding stories and events through conversations and choices with characters. The introduction of an emotion engine enables dynamic story development in response to the user's emotions, providing a richer user experience. For example, when a user asks, "What do you like?", the emotion engine recognizes "affection" and displays a special date scenario.

[2196] (Example 2)

[2197] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2198] Traditional dialogue systems have a problem where conversations with characters are formal, and dynamic story branching and event generation that respond to user emotions and choices cannot be realized. Furthermore, the lack of intimacy level calculations that reflect user emotions and the display of special scenarios have resulted in a limited user experience.

[2199] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[2200] In this invention, the server includes means for branching the story and causing events through conversations with a character selected by the user; means for the system to calculate intimacy based on the conversation content and emotion recognition results, and display a special scenario when certain conditions are met; means for analyzing the chat content using natural language processing and generating an appropriate response based on the analysis results and emotion recognition results; means for seamlessly adjusting the story branching and event occurrences based on the user's emotion recognition results; and means for dynamically generating and displaying special scenarios and events based on the calculation of intimacy. This enables a dynamic and emotional story development and a rich user experience based on the user's emotions and choices.

[2201] A "user" is someone who utilizes this system and experiences the story progression and events through interaction with characters.

[2202] A "character" is a virtual person or entity that interacts with the user within the system, and has the role of responding dynamically according to the user's choices and emotional recognition results.

[2203] "Conversation" refers to the exchange of messages between the user and the character, and is generally conducted in text format.

[2204] "Story branching" refers to the change in the direction of the narrative based on the user's choices and emotional responses, resulting in different events and scenarios.

[2205] An "event" refers to a specific scene or occurrence within a story, and its content may change depending on the user's choices and emotions.

[2206] "Intimacy level" is a numerical or metric representation of the depth of the relationship between the user and the character, and it fluctuates in response to the user's actions and statements.

[2207] A "special scenario" refers to a special scene or story that appears when the intimacy level meets certain conditions, and it contains unique content that differs from the regular scenario.

[2208] "Emotion recognition" is a technology that analyzes a user's input and actions to identify their emotions and determine their emotional state, such as joy, interest, or sadness.

[2209] "Natural language processing" is a technology that analyzes user messages to understand their meaning and intent, and involves text analysis and response generation.

[2210] A "response" is a message that a character returns in response to a message from the user, and is generated based on natural language processing and emotion recognition results.

[2211] Embodiments of this invention will be described in detail below. This system has a mechanism in which the story branches and events occur through conversations with a character selected by the user. In addition, it can calculate intimacy based on the conversation content and emotion recognition results, and display a special scenario when certain conditions are met.

[2212] System Overview

[2213] First, let me explain the system's overview. This system consists of three components: a server, a terminal, and a user, all of which interact with each other. The server is responsible for key processes such as story branching, event generation, emotion recognition, intimacy calculation, and natural language processing. The terminal provides the user's interface and communicates with the server. The user operates the terminal to advance conversations with the characters.

[2214] Hardware and software to be used

[2215] This system uses the following hardware and software:

[2216] Server hardware: A server with a high-performance processor, sufficient memory, and fast database access capabilities.

[2217] Database: A relational database management system (RDBMS) such as MySQL.

[2218] Natural language processing engine: High-performance generative AI models such as GPT-4.

[2219] Emotion recognition engine: Emotion analysis tools such as Emotion AI.

[2220] Details of the program processing

[2221] This section provides a detailed explanation of the program processing performed by the system. The specific processing steps for each stage are shown below.

[2222] User login and authentication

[2223] 1. User: Launch the application from your device and enter your username and password on the login screen.

[2224] 2. Terminal: Sends the entered information to the server.

[2225] 3. Server: Matches the information with the database, generates an authentication result, and sends it to the terminal.

[2226] 4. Device: Displays the home screen or an error message based on the authentication result.

[2227] Character Selection

[2228] 1. Device: After successful authentication, the character selection screen will be displayed.

[2229] 2. User: Select a character from the list.

[2230] 3. Terminal: Sends selection information to the server.

[2231] 4. Server: Loads the character's initial information, generates data for starting a chat, and sends it to the terminal.

[2232] 5. Terminal: Displays the chat screen and the character's initial message.

[2233] Start chat

[2234] 1. User: Enter your message.

[2235] 2. Terminal: Sends the entered message to the server.

[2236] 3. Server: Analyzes messages using a natural language processing engine and analyzes emotions using an emotion engine.

[2237] 4. Server: Generates a response and sends it to the terminal.

[2238] 5. Terminal: Displays the character's reply.

[2239] Story branching and event occurrences

[2240] 1. Server: Analyzes chat content and emotion recognition results to determine the direction of the story's progression.

[2241] 2. Terminal: Displays new scenes and events.

[2242] 3. User: Can continue the conversation.

[2243] Intimacy level calculation and special scenario display

[2244] 1. Server: Calculates intimacy levels based on user statements and emotion recognition results.

[2245] 2. Device: When the intimacy level exceeds a certain threshold, a special scenario will be displayed.

[2246] 3. User: Experience a special scenario.

[2247] Specific example

[2248] Specific examples are given below.

[2249] Making a date from the first meeting

[2250] The user logs into the system from their device and selects "Character A". The server receives the selection information, loads the initial information for "Character A", and sends the chat screen to the user's device. When the user sends the message "Hi, nice to meet you", the server analyzes it and generates the response "Hi, how are you?" which is sent to the device. When the user continues with "What are you doing today?", the server analyzes this and, if it recognizes the user's "interest", selects a scenario that suggests "I'm planning to go shopping. Would you like to come along?". This response is sent to the device, and if the user replies "Of course, I'll go with you", a date event occurs. During the date event, if the user selects "Tell me what you like?", the server increases the intimacy level. Furthermore, if the emotion engine detects the user's "enjoyment", a special date scenario is loaded, which the user can experience.

[2251] Example of a prompt

[2252] "Ask Character A what they're doing today and guide them towards a date event."

[2253] In this embodiment of the invention, users can enjoy a dynamic and emotional storyline through emotional interaction with characters. By using an emotion recognition engine and a natural language processing engine together, more personalized responses become possible, significantly improving the user experience.

[2254] The flow of the specific processing in Example 2 will be explained using Figure 13.

[2255] The processing flow of this system's program (explained in steps)

[2256] Step 1: Display the login screen

[2257] Input: The user launches the application.

[2258] Action: The device launches the application and displays the login screen.

[2259] Output: The login screen is displayed to the user.

[2260] Step 2: Enter and submit user authentication information.

[2261] Input: The user enters their username and password and clicks the login button.

[2262] Operation: The terminal sends the entered username and password to the server. The data is encrypted before transmission.

[2263] Output: User authentication information is sent to the server.

[2264] Step 3: Authentication Process

[2265] Input: The username and password received by the server.

[2266] Operation: The server connects to the database and verifies the username and password. If successful, it generates an authentication token and prepares data for the home screen. If it fails, it prepares an error message.

[2267] Output: Authentication results are generated, and an authentication token or error message is generated.

[2268] Step 4: Submit and display authentication results

[2269] Input: Authentication result and authentication token or error message.

[2270] Operation: The server sends the authentication result to the device. Based on the received data, the device displays either the home screen or an error message.

[2271] Output: The home screen is displayed, or an error message is displayed on the login screen.

[2272] Step 5: Display the character selection screen.

[2273] Input: User who successfully authenticated.

[2274] Action: The device displays the character selection screen.

[2275] Output: The character selection screen is displayed to the user.

[2276] Step 6: Submit character selection information

[2277] Input: The user selects a character and clicks the select button.

[2278] Operation: The terminal sends the ID of the selected character to the server.

[2279] Output: Character ID information is sent to the server.

[2280] Step 7: Loading character information

[2281] Input: The character ID received by the server.

[2282] Operation: The server loads initial information for the selected character from the database and generates data to start the chat.

[2283] Output: Data for starting a chat is generated.

[2284] Step 8: Send and display data to start the chat.

[2285] Input: Data to start the chat.

[2286] Operation: The server sends data to the terminal to initiate a chat. The terminal receives the data, displays the chat screen, and shows the character's initial message.

[2287] Output: The chat screen and the character's initial message are displayed.

[2288] Step 9: Enter and send the user message.

[2289] Input: The user enters a message and clicks the send button.

[2290] Operation: The terminal sends the entered message to the server.

[2291] Output: The user message is sent to the server.

[2292] Step 10: Message Analysis

[2293] Input: User messages received by the server.

[2294] Operation: The server analyzes messages using a natural language processing engine (such as GPT-4). It also analyzes the user's emotions using an emotion recognition engine (such as Emotion AI).

[2295] Output: Message analysis results and sentiment recognition results are generated.

[2296] Step 11: Generating a response message

[2297] Input: Message analysis results and sentiment recognition results.

[2298] Operation: Based on these results, the server generates an appropriate response message.

[2299] Output: A response message is generated.

[2300] Step 12: Sending and displaying the response message

[2301] Input: Response message.

[2302] Operation: The server sends a response message to the terminal. The terminal receives the message and displays it on the chat screen.

[2303] Output: The response message is displayed on the chat screen.

[2304] Step 13: Deciding on a story branch

[2305] Input: Chat content and sentiment recognition results.

[2306] Operation: The server determines the direction of the story's progression based on this data.

[2307] Output: The results of the story branching are generated.

[2308] Step 14: Generating new scenes and events

[2309] Input: The result of the story branching.

[2310] Operation: The server generates data for new scenes and events and sends it to the terminal.

[2311] Output: New scene and event data is generated.

[2312] Step 15: Displaying new scenes and events

[2313] Input: Data for new scenes or events.

[2314] Operation: The terminal displays the received data.

[2315] Output: New scenes or events are displayed on the terminal.

[2316] Step 16: Calculating Intimacy Level

[2317] Input: User input, choices, and sentiment recognition results.

[2318] Operation: The server calculates intimacy based on this data.

[2319] Output: An intimacy value is generated.

[2320] Step 17: Generating a Special Scenario

[2321] Input: Intimacy level value.

[2322] Operation: When intimacy exceeds a certain threshold, the server generates special scenarios or events.

[2323] Output: Data for special scenarios and events is generated.

[2324] Step 18: Displaying Special Scenarios

[2325] Input: Data for special scenarios or events.

[2326] Operation: The terminal displays the received data.

[2327] Output: Special scenarios or events will be displayed on the terminal.

[2328] The above describes the detailed processing steps and specific operation of this system's program.

[2329] (Application Example 2)

[2330] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2331] Traditional interactive storytelling systems were unable to dynamically branch stories in response to user emotions, instead relying on fixed story patterns. This resulted in a limited user experience and a lack of entertainment value and immersion. Furthermore, the calculation of intimacy levels and the resulting display of special scenarios were also fixed, making real-time, emotion-based responses difficult.

[2332] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[2333] In this invention, the server includes means for branching the story and triggering events through conversations with a character selected by the user; means for the system to calculate intimacy based on the conversation content and display a special scenario when certain conditions are met; means for analyzing the chat content using natural language processing and generating an appropriate response based on the analysis results; means for recognizing the user's emotions and dynamically branching the story according to those emotions; and means for seamlessly displaying special events and scenarios based on intimacy and emotions. This enables flexible and dynamic story development that responds to the user's emotions.

[2334] A "user" is someone who uses the system to enjoy content through conversations with characters.

[2335] A "character" is a virtual entity or person that a user can select to engage in conversation with.

[2336] A "story" refers to a series of events or occurrences that unfold through conversations with characters.

[2337] "Branching" refers to the point where the direction of the story changes depending on the user's choices and emotions.

[2338] An "event" refers to a specific occurrence or action that takes place within a story.

[2339] "Intimacy level" is an indicator that shows the depth of the relationship between the user and the character.

[2340] A "special scenario" is a scenario that appears when certain conditions are met, and is different from the normal storyline.

[2341] "Natural language processing" is the technology that uses computers to understand, analyze, and generate human language.

[2342] "Emotions" refer to the user's feelings or emotional state.

[2343] "Emotion recognition" is a technology that analyzes and identifies emotions from a user's words and actions.

[2344] "Dynamic" refers to the property of changing or adapting in real time according to the situation.

[2345] "Seamlessly" refers to something that is continuous and uninterrupted.

[2346] The system of this invention provides a mechanism in which the story branches and events occur through conversations with characters selected by the user. The program to realize this is structured as follows.

[2347] The system requires a smartphone and a server as hardware. The smartphone is the device that the user directly interacts with, while the server processes and stores various types of data. As for software, the "transformers" library, a natural language processing engine, is used. This library is based on a generative AI model and uses a "sentiment-analysis" model for emotion recognition.

[2348] When a user logs into the system using their smartphone, the server verifies the user's credentials against the database and performs authentication. If authentication is successful, a character selection screen appears on the device. The user selects their preferred character, and the server loads the data for the selected character and prepares to start chatting.

[2349] When a user types and sends a message to a character, the message is sent from the terminal to the server. The server uses the "transformers" library to parse the message and perform emotion recognition. Based on the results, it generates an appropriate response and sends it to the terminal. The terminal then displays this response to the user.

[2350] Based on the chat content and the results of emotion recognition, the server determines the branching of the story. For example, if joy is recognized from the user's message, it selects a scenario that suggests a pleasant event. It also calculates intimacy and, if it exceeds a certain threshold, loads a special scenario and presents it to the user.

[2351] As a concrete example, consider a scenario where a user logs in with the username "example_user" and password "123456," and selects "Character A." When the user sends the message "I'm relieved that work is over," the server recognizes this as "joy" and generates a response asking, "Would you like to share something fun?" This system enables flexible and dynamic story development that responds to the user's emotions, providing a rich user experience.

[2352] Furthermore, the following input can be considered as an example of a prompt statement.

[2353] User's input message: "Every day is tough..."

[2354] Processing result: "Did something upsetting happen?"

[2355] The above describes the embodiments for carrying out this invention.

[2356] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[2357] Step 1:

[2358] The user launches the application from their smartphone, and the login screen appears. They enter their username and password and click the "Login" button.

[2359] Input: Username, Password

[2360] Output: Authentication Request

[2361] Step 2:

[2362] The terminal sends the entered username and password to the server. The server compares the received authentication information with its database and generates an authentication result.

[2363] Input: Authentication Request

[2364] Output: Authentication result (success or failure)

[2365] Step 3:

[2366] If authentication is successful, the server prepares the home screen data and sends it to the device. If authentication fails, it generates an error message and sends it to the device. If authentication is successful, the device displays the home screen; if it fails, it displays the error message on the login screen.

[2367] Input: Authentication result

[2368] Output: Home screen data or error messages

[2369] Step 4:

[2370] The user moves from the home screen to the character selection screen and selects a character from the displayed list. The device then sends the selection information to the server.

[2371] Input: Character Selection

[2372] Output: Character selection information

[2373] Step 5:

[2374] Based on the received character selection information, the server loads the selected character's initial and background information from the database, generates data for starting a chat, and sends it to the terminal. The terminal displays the chat screen and shows the selected character's initial message.

[2375] Input: Character selection information

[2376] Output: Chat start data, initial message

[2377] Step 6:

[2378] The user types a message in the chat screen and clicks the "Send" button. The device sends the user's message to the server.

[2379] Input: Message

[2380] Output: Message sending request

[2381] Step 7:

[2382] The server analyzes the received message using the "transformers" library and performs emotion recognition using a generative AI model. Based on the analysis results and emotion recognition results, it generates an appropriate response message and sends it to the terminal.

[2383] Input: Message sending request

[2384] Output: Response message

[2385] Step 8:

[2386] The terminal displays the response message received from the server to the user. The user can then continue chatting.

[2387] Input: Response message

[2388] Output: Display of response message

[2389] Step 9:

[2390] The server analyzes the chat content and emotion recognition results to determine the direction of the story. For example, if it recognizes "joy" from the user's message, it selects a scenario in which the character proposes a "fun event." New scenes and events are generated and sent to the device according to specific conditions.

[2391] Input: Sentiment recognition result

[2392] Output: Event data, new scene data

[2393] Step 10:

[2394] The terminal displays new scenes and events received from the server to the user. The user can then enter further messages or options to continue the conversation.

[2395] Input: New scene data, event data

[2396] Output: Display new scene and events

[2397] Step 11:

[2398] The server calculates intimacy based on user input, choices, and emotion recognition results. For example, if a user sends a positive message to a character, intimacy increases. If intimacy exceeds a certain threshold, a special scenario or event is generated and sent to the terminal.

[2399] Input: User input, emotion recognition results

[2400] Output: Intimacy level, special scenario data

[2401] Step 12:

[2402] The terminal displays special scenarios and events sent from the server, and the user can continue to interact with them.

[2403] Input: Special Scenario Data

[2404] Output: Display of special scenarios

[2405] The above outlines the specific processing steps of this system's program.

[2406] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[2407] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2408] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[2409] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2410] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[2411] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[2412] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[2413] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[2414] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, i...

Claims

1. The story branches and events occur through conversations with the character selected by the user. A means by which the system calculates intimacy based on the content of the conversation and displays a special scenario when certain conditions are met, A means for analyzing chat content using natural language processing and generating an appropriate response based on the analysis results, A system that includes this.

2. The system according to claim 1, wherein the story branches based on the user's choices, and different events occur seamlessly based on those branches.

3. The system according to claim 1, which uses a natural language processing engine to analyze messages from users and dynamically generates a story based on the analysis results.

Images