System

Abstract

To provide a system for achieving interactive story experience based on feeling and selection of a user.SOLUTION: A system includes: means for receiving selection information including information on a character, a situation, a genre, and a reader selected by a user; means for generating an original story by using a generative AI and a prompt sentence instructing to generate an original story based on the selection information; means for identifying emotion of the user by using a camera and an emotion recognition model; means for presenting a plurality of options related to progress of the story to the user in the middle of the story; means for generating a continuation of the story by using the generative AI and a prompt sentence instructing to generate the continuation of the story based on any one of the options selected by the user and the identified emotion of the user; and means for displaying the generated story to the user.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional story-creation apps allow users to change the outcome of the story by selecting how the story progresses, but the changes to the story that can be influenced by the user's choices are limited, making them insufficient to stimulate the user's creativity. [Means for solving the problem]

[0005] The system of the present invention includes the following means to stimulate the user's creativity and enable more free story creation.

[0006] That is, the system includes a means for the user to generate an original story that matches the preferences of the user by selecting a character (the main character's personality), a situation (the era, country, region, or place in which the story is set), a genre (adventure, romance, politics, history, science fiction), and information about the reader (age, gender). The system also includes a means for presenting multiple options during the story and allowing the user to determine the progress of the story through their own selection. This allows the user to customize the story through their own selection and demonstrate their own creativity.

[0007] The system also includes a means for generating a story using natural language processing techniques, called a data generation model, which generates parts of the story based on user selections, providing a more realistic storytelling experience.

[0008] The system also includes a means for displaying updates to the story to the user, allowing the user to see how their choices affect the story, allowing the user to get feedback on the results of their choices and to take a more proactive role in progressing through the story.

[0009] Through the above means, the present invention can solve the problems associated with conventional story creation apps and enable users to freely create stories that stimulate their creativity.

[0010] A "data-generative model" is a machine learning model that uses natural language processing technology to provide an interactive model for generating and progressing a story. A data-generative model generates parts of a story based on the user's selection, providing a more realistic story experience through dialogue with the user. One example of a data-generative model is generative AI.

[0011] "Choices" are multiple options presented to the user during the story. The user can decide the progress of the story by selecting one of these options. Choices are an important element that change the development and ending of the story depending on the user's free choice.

[0012] "Interactively creating a story" means that the user can advance the story through their own choices. The user can select information such as characters, situations, and genres, and can change the outcome of the story by choosing options presented during the story. This allows the user to freely create a story while exerting their own creativity. [Brief description of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] 4 is a sequence diagram showing a process flow of the data processing system according to the first embodiment. FIG. [Figure 12] 11 is a sequence diagram showing a process flow of the data processing system in application example 1. FIG. [Figure 13]FIG. 11 is a sequence diagram showing the flow of processing of the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 11 is a sequence diagram showing the flow of processing in the data processing system in application example 2 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), or an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] An embodiment for implementing the present invention includes the following elements.

[0035] (Element 1: User Interface)

[0036] A user accesses the story creation application via a terminal. An interface for selecting information on characters, situations, genres, and readers is displayed on the terminal. The user selects this information and transmits the selected information to the server via the terminal.

[0037] (Element 2: Server processing)

[0038] The server uses a generative AI to generate a story based on the received selection information. The generative AI uses natural language processing technology to generate a part of the original story based on the user's selection. The server transmits the generated story to the terminal.

[0039] (Element 3: Story presentation and options)

[0040] The device displays the received story and presents the user with options generated by generative AI during the story. The user selects one option from the options presented and sends it to the server via the device.

[0041] (Element 4: Story progression and updating)

[0042] The server updates the story progression using a generative AI based on the user's selection. The generative AI generates a new part of the story according to the selection, and the server transmits the updated story to the device.

[0043] (Element 5: Narrative presentation and feedback)

[0044] The device displays the received story updates, allowing the user to see the results of their choices. Users get feedback on the progress of the story and can freely customize the story development.

[0045] The process of presenting options, updating the progress of the story, and displaying to the user is repeated until the story is completed.

[0046] With these elements, users can create original stories based on their own choices through a story creation application.

[0047] For example, if a user selects a "brave adventurer" as the main character, "medieval Europe" as the setting of the story, and "adventure" as the genre, the generative AI will generate a story based on those selections. In the middle of the story, the user is presented with options such as "enter the cave," "explore the forest," and "talk to the villagers." If the user selects "enter the cave," the story will continue with an adventure in the cave. On the other hand, if the user selects "explore the forest," the story will move on to an adventure in the forest.

[0048] The process flow will be explained below.

[0049] Step 1: When a user accesses the story creation application, the device connects to the server.

[0050] Step 2: An interface is displayed on the terminal to select information about the character, situation, genre, and reader. The user selects this information and sends it to the server via the terminal.

[0051] Step 3: The server uses a generative AI to generate a story based on the received information. The generative AI uses natural language processing techniques to generate parts of an original story based on the user's selections.

[0052] Step 4: The server sends the generated story to the terminal.

[0053] Step 5: The device displays the received story and presents options to the user during the story.

[0054] Step 6: The user selects one of the options presented and sends it to the server via the terminal.

[0055] Step 7: The server updates the story progression using the generative AI based on the received selection information. The generative AI generates a new part of the story according to the selection, and the server sends the updated story to the device.

[0056] Step 8: The device displays the received story update. Present the user with options again during the story. Repeat steps 6 to 7.

[0057] This is the process flow of this system. The user selects information, the terminal sends it to the server, which generates a story, and the terminal displays the story and presents options to the user. The user selects an option, the terminal sends it to the server, which updates the story, and the terminal again displays the story and presents options to the user. As this process is repeated, the user can progress through the story based on their own choices.

[0058] Example 1

[0059] Next, a description will be given of Example 1. In the following description, the data processing device 12 is referred to as a "server" and the smart device 14 is referred to as a "terminal."

[0060] In conventional story creation systems, it was difficult for users to freely create a story based on their own choices. In particular, there was a problem that the interactivity experienced by the user was reduced because the progress and updates of the story according to the user's choices were done manually. In addition, the created stories sometimes lacked consistency, which reduced user satisfaction. It is necessary to provide a story creation system that can solve these problems and is more interactive and flexibly respond to user choices.

[0061] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0062] In this invention, the server includes a means for receiving selection information including information on a character, a situation, a genre, or a reader selected by a user, a means for generating an original story using a generative AI model, a means for presenting options regarding the progress of the story to the user in the middle of the story, a means for updating the progress of the story using the generative AI model based on the options selected by the user, and a means for displaying updated information on the story to the user. This allows the user to enjoy a story that is dynamically generated based on their own selection.

[0063] A "user" is an entity that uses the system to interactively create a story.

[0064] "Selected information" refers to information including the character, situation, genre, and reader selected by the user.

[0065] A "generative AI model" is an artificial intelligence technology that generates original stories based on user-selected information.

[0066] "Story progression" refers to the process and content of how the story unfolds.

[0067] "Choices" are multiple, selectable options for story progression presented to the user.

[0068] An "emotion analysis engine" is a technology that recognizes the user's emotions and adjusts the story progression based on those emotions.

[0069] "Natural language processing" is a language processing technique used when a generative AI model generates sentences using prompts.

[0070] "Updated information" refers to the content of the story that has been updated based on user selections and sentiment analysis.

[0071] The present invention relates to a system that allows users to interactively create stories, which consists of a user interface, a server process, a story display, and updating of the story progress based on user selections.

[0072] User Interface

[0073] A user accesses a story creation application using a terminal such as a smartphone or a PC. The application displays an interface for selecting information on characters, situations, genres, and readers. The user selects this information and transmits the selected information to the server via the terminal.

[0074] Server Processing

[0075] The server receives the selection information selected by the user. The server is built using a programming language such as Python or Java (registered trademark), and inputs the selection information into a generative AI model (e.g., ChatGPT (registered trademark)). The generative AI model uses natural language processing technology to generate an original story based on the user's selection. The generated story is sent from the server to the terminal.

[0076] Presenting the story and giving options

[0077] The device displays the received story on the screen. As the story progresses, the user is presented with options to choose from during the story generated by the generative AI model. For example, options such as "Enter the cave," "Explore the forest," and "Talk to the villagers" are displayed. The user selects one of the options presented and sends the selection information to the server via the device.

[0078] Story progression and updates

[0079] The server receives the user's choice and generates the rest of the story using a generative AI model. The generative AI model generates a story in the following format as a prompt:

[0080] The main character is a brave adventurer, and you must create an adventure story set in medieval Europe. The actions the main character can choose in the next scene are to "enter the cave," "explore the forest," or "talk to the villagers." Generate a sequel for each option.

[0081] The new part of the story that is generated is again sent from the server to the terminal.

[0082] View story updates and give feedback

[0083] The device displays the updated story on the screen and provides feedback to the user, who can review the results of their choices and enjoy the progression of the story. This process of presenting choices and updating the story continues until the story is complete.

[0084] The present invention allows users to enjoy a story that is interactively generated based on their own selections, resulting in consistent stories and improved user satisfaction.

[0085] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0086] Step 1:

[0087] A user accesses a story creation application using a device such as a smartphone or a PC. The device displays a user interface and presents options for selecting information on characters, situations, genres, and readers. The user selects this information. The input is the information selected by the user, and the output is the state in which the selected information is complete.

[0088] Step 2:

[0089] The terminal receives the user's selection information and sends it to the server through an HTTP request. The input is the user's selection information, and the output is the selection information being sent to the server. This passes the selection information to the server.

[0090] Step 3:

[0091] The server sends a prompt to the generative AI model based on the received selection information. The input is the user's selection information, and the output is a prompt to the generative AI model. The server uses Python to generate a prompt like this:

[0092] "Create an adventure story for your hero, a brave adventurer, set in medieval Europe."

[0093] Step 4:

[0094] The generative AI model analyzes the prompt and generates an original part of the story based on the user's specifications. The input is the prompt, and the output is the generated story text. For example, the story generated by the generative AI model is "A brave adventurer arrives in a medieval village and sets out on an unknown adventure."

[0095] Step 5:

[0096] The server sends the generated story to the terminal. The input is the generated story text and the output is the message sent to the terminal. The server converts the story text to JSON format and sends it as an HTTP response.

[0097] Step 6:

[0098] The terminal parses the received story and displays it on the screen. The input is the story text received from the server and the output is the display to the user interface. Once the story is displayed, the user can read it.

[0099] Step 7:

[0100] During the story, the device presents the user with options generated by the generative AI model. The input is part of the generated story, and the output is the options presented to the user. For example, options such as "Enter the cave," "Explore the forest," and "Talk to the villagers" are displayed.

[0101] Step 8:

[0102] The user selects one of the options presented, and the device again sends the selection information to the server. The input is the user's selection information, and the output is the transmission to the server. Once the selection information is passed to the server, the server again uses the generative AI model to generate the next part of the story.

[0103] Step 9:

[0104] The server updates the story progression using a generative AI model based on the user's choices. The input is the user's selection information, and the output is the continuation of the generated story. If the user selects "Enter the cave," the generative AI model generates a story such as "The brave adventurer stepped into the dark cave."

[0105] Step 10:

[0106] The server sends the updated story to the terminal. The input is the generated story continuation, and the output is the message sent to the terminal. The server converts the story text to JSON format and sends it as an HTTP response.

[0107] Step 11:

[0108] The device displays the updated story on the screen and provides feedback to the user. The input is the updated story text received from the server, and the output is the display on the user interface. The user can confirm the results of their selection and enjoy the story progression again. This process of presenting options and updating the story is repeated until the story is completed.

[0109] (Application example 1)

[0110] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0111] Current interactive storytelling systems have difficulty incorporating educational content effectively, and users have limited options to maximize their learning. In addition, there are few examples of systems that dynamically customize the story progression based on user choices to provide educational value.

[0112] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0113] In this invention, the server includes a means for receiving selection information including information on a character, a situation, a genre, or a reader selected by a user, a means for generating an original story using a generative AI model based on the selection information, a means for presenting options regarding the progress of the story to the user during the story, a means for updating the progress of the story using the generative AI model based on the options selected by the user, a means for displaying update information on the story to the user, a means for selecting an educational theme, and a means for customizing the story based on the educational theme. This makes it possible to dynamically generate and update a story based on an educational theme selected by a user, providing an interactive and highly educational story experience.

[0114] A "character" is someone who appears in a story or is set as the protagonist.

[0115] A "situation" is the scene or condition in which a story or event occurs.

[0116] A "genre" is a category that classifies types and themes of stories.

[0117] "Reader information" refers to data and attribute information about the recipients of the story.

[0118] "Selection information" is data including information on the character, situation, genre, or reader selected by the user.

[0119] A "generative AI model" is an algorithm that uses artificial intelligence to generate data and text.

[0120] "Narrative progression" is the process by which a story's plot or storyline unfolds.

[0121] "Choices" are options that a user can choose from to affect the progression of the story.

[0122] "Updates" are story data that are generated or modified based on user selections.

[0123] "Educational Theme" refers to specific content or concepts related to learning or teaching.

[0124] "Customization" is the act of changing and adjusting a story or system according to the user's choices and needs.

[0125] The system that realizes this application example consists of five main parts: the user interface, server processing, story presentation and choice generation, story progress and update, and story display and feedback. Specifically, it takes the following form.

[0126] User Interface

[0127] The server provides an interface that allows users to access the story creation application using a terminal and select characters, situations, genres, and educational themes. The interface is built as a smartphone or tablet application and is developed using React Native.

[0128] Server Processing

[0129] The device sends the user's selections to the server, where requests are received and analyzed on the server side using Python and Flask. The server uses a generative AI model (OpenAI's ChatGPT) to generate an original story based on the selections. During the generation process, the server constructs prompts that match the characters, situations, and educational themes selected by the user, and inputs them into the AI ​​model.

[0130] Presenting a story and generating options

[0131] The device displays the original story and presents the user with options for progressing through the story. Options are displayed according to the situation. For example, in an ancient Egyptian setting, options include "Learn how to build pyramids" and "Explore Egyptian culture."

[0132] Story progression and updates

[0133] After the user selects an option, the selection information is sent to the server again. The server again uses the ChatGPT API to generate and update the story progression based on the selected option, and the prompt text reflects the user's selection.

[0134] Viewing and Feedback on Stories

[0135] The updated story is sent to the terminal, and the user can view the update information to check the progress of the story. If the user makes the right choice based on the story options, an educational learning effect can be obtained. Regarding the specific learning content, the story is appropriately customized according to the user's feedback.

[0136] For example, if you select an "Ancient Scholar" as the character, a "Medieval Castle" as the setting, and "Mathematics Basics" as the theme for a third-grade history lesson, the generative AI model will receive the following prompt as input and generate a story:

[0137] Input prompt example

[0138] "Generate the following story: Character: Ancient Scholar; Situation: Medieval Castle; Theme: Mathematics Basics. The Ancient Scholar teaches a group of children how to design a castle. Include the following options: a) how to calculate area b) how to measure height c) how much stone to use."

[0139] In this way, a system is realized that dynamically generates and updates stories based on conditions selected by the user, providing educational value.

[0140] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0141] Step 1:

[0142] The terminal provides an interface for users to access the story creation application and select characters, situations, genres, and educational themes. The user interface is built with React Native and receives user selection information. This input includes data such as the characters and situations selected by the user.

[0143] Step 2:

[0144] The device sends the user's selection information in JSON format to the server. The server uses Python and Flask to analyze the received selection information and constructs a prompt for the generative AI model (OpenAI's ChatGPT). The prompt specifically reflects the selection information.

[0145] Step 3:

[0146] The server uses a generative AI model to generate an original story based on the prompt sentence. The AI ​​model generates a series of text data, which becomes the initial part of the story. The prompt sentence is given as input, and the story text is obtained as output.

[0147] Step 4:

[0148] The server sends the generated story to the terminal, and the terminal displays the story to the user. At this time, options are inserted at appropriate points in the story so that the user can select options for the progress of the story. The options are also displayed through an interface built with React Native (registered trademark).

[0149] Step 5:

[0150] The user selects one of the options presented and sends the selection information to the server again via the terminal. The server receives this selection information and again uses the generative AI model to generate the progression of the story based on it. Specifically, it constructs a prompt sentence again and inputs it into the AI ​​model. The prompt sentence reflects the user's selection.

[0151] Step 6:

[0152] The server sends the updated story parts to the terminal, and the terminal displays the updated story information to the user. The user checks the updated story and the story progresses interactively. This allows the story progress to change dynamically according to the user's selections.

[0153] Step 7:

[0154] This process is repeated until the end of the story. Each time the user selects an option at each stage of the story, the server updates the story based on the selection and displays the updated information on the terminal. Here, feedback content based on educational themes is also displayed to enhance the user's learning effect.

[0155] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0156] In this case, the embodiment of the present invention further includes the following elements.

[0157] (Element 1: Incorporating an emotion engine)

[0158] This system is equipped with an emotion engine that can analyze information such as the user's facial expressions and voice and recognize the user's emotions.

[0159] (Element 2: Emotionally-driven story development)

[0160] Based on the user's emotions recognized by the emotion engine, the generative AI can adjust the story development. For example, if the user feels joy in the story, the emotion engine can add a happy development to the story.

[0161] (Element 3: Presenting options based on emotions)

[0162] Based on the user's emotions recognized by the emotion engine, the generative AI will adjust the options: for example, if the user feels sadness in the story, the generative AI can present comforting options.

[0163] (Element 4: Emotional Feedback)

[0164] The user's emotions may change as the story progresses or choices are presented. The emotion engine can detect changes in the user's emotions and reflect them in the development of the story or in adjusting choices.

[0165] By combining the above elements with the emotion engine, this system can recognize the user's emotions and adjust the story development and the presentation of options according to the emotions. This allows the user to become more emotionally immersed in the story and experience a more realistic feeling.

[0166] The process flow will be explained below.

[0167] Step 1: When a user accesses the story creation application, the device connects to the server.

[0168] Step 2: An interface for selecting information on the character, situation, genre, and reader is displayed on the terminal. The user selects this information and sends it to the server via the terminal.

[0169] Step 3: The server uses the generative AI to generate part of the story based on the received information. At the same time, the emotion engine recognizes the user's emotions. The generative AI and emotion engine work together to adjust the story development and options according to the user's emotions.

[0170] Step 4: The server sends the generated story and emotion-based options to the device.

[0171] Step 5: The device displays the received story and presents the user with options. At the same time, the emotion engine analyzes the user's emotions and detects changes in the user's emotions.

[0172] Step 6: The user selects one of the options presented and sends it to the server via the terminal. At the same time, the emotion engine detects changes in the user's emotions and sends them to the server.

[0173] Step 7: The server updates the story progression by coordinating the generative AI and emotion engine based on the received selection information and emotion information. The updated story and emotion-based choices are sent to the terminal.

[0174] Step 8: The device displays the received story update information and presents the user with options. At the same time, the emotion engine analyzes the user's emotions and detects changes in the user's emotions. Steps 6 to 7 are repeated.

[0175] This is the process flow of this system. The user selects information, and the device sends it to the server, generating a story, which then displays the story and presents options to the user. The user selects an option, and the device sends it to the server, updating the story, which then again displays the story and presents options to the user. As this process is repeated, the user's emotions are analyzed by the emotion engine and reflected in the development of the story and adjustments to the options.

[0176] Example 2

[0177] Next, a description will be given of Example 2. In the following description, the data processing device 12 is referred to as a "server" and the smart device 14 is referred to as a "terminal."

[0178] Conventional interactive story generation systems have the problem that it is difficult to present story development and options that fully reflect the user's emotions. Therefore, there is a demand for a system that can adjust story development and options in real time according to the user's emotions.

[0179] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving selection information including information on a character, a situation, a genre, or a reader selected by a user, a means for generating an original story using a data generation model based on the selection information, a means for presenting options regarding the progress of the story to the user in the middle of the story, a means for updating the progress of the story using the data generation model based on the options selected by the user, and a means for acquiring the user's facial expression and voice and analyzing the user's emotions using an emotion analysis device in the updating means, a means for sending a prompt sentence to a generation AI model based on the user's emotions and generating the next story development, a means for displaying the generated story development and options to the user, and a means for detecting a change in the user's emotions again and continuously adjusting the story progress and options. This makes it possible to present a dynamic and real-time story development and options according to the user's emotions.

[0180] A "user" is an entity that uses this system to read a story and select options.

[0181] A "character" is someone who appears in a story or is the main character.

[0182] A "situation" refers to a scene in a story or a scene setting selected by the user.

[0183] "Genre" refers to the type or classification of a story, such as fantasy, suspense, romance, etc.

[0184] "Selected information" refers to information such as the character, situation, genre, etc. selected by the user.

[0185] A "data generation model" is a machine learning model that generates new data based on given input information.

[0186] A "narrative" refers to a textual sequence of events or a scenario that is presented to a user.

[0187] "Choices" refer to multiple options from which a user can choose regarding the progression of a story.

[0188] The "means for updating" is a function for moving the story progression to the next stage in response to a user's selection.

[0189] An "emotion analysis device" is a device or software that analyzes a user's facial expressions and voice data to determine their emotions.

[0190] A "generative AI model" is an artificial intelligence algorithm that generates new text based on a given prompt.

[0191] A "prompt sentence" is a sentence that is given to a generative AI model as instructions when it generates new text.

[0192] "Story development" refers to the progression or progress of a story.

[0193] "Presenting options" is the act of displaying options to the user for what to select next.

[0194] "Changes in the user's emotions" refers to changes in the user's state of mind due to the progression of the story or other factors.

[0195] The embodiment of this invention is an interactive story generation system that can analyze the user's emotions in real time and dynamically generate story development and options based on the analysis. This system uses the following main hardware and software:

[0196] Hardware

[0197] 1. Terminal:

[0198] Devices such as PCs, smartphones, tablets, etc. These devices must have a webcam and microphone.

[0199] 2. Server:

[0200] A high-performance computer that acts as the central nerve center of the entire system.

[0201] software

[0202] 1. Emotion analysis device:

[0203] Microsoft(R) Azure(R) Face API

[0204] Microsoft Azure Speech API

[0205] 2. Generative AI model:

[0206] OpenAI ChatGPT

[0207] Data processing and calculation flow

[0208] The server transmits facial expression and voice data acquired in real time from the user's device to the emotion analysis device, which analyzes the user's emotions. Based on the emotion data obtained as a result of the analysis, the server sends a prompt to the generative AI model to generate the next development of the story. This prompt is based on the user's emotions. For example, if the user is expressing the emotion of "joy," the following prompt is generated:

[0209] So you've detected a situation where the user is smiling. What's the next happy twist to add to the story?

[0210] Based on this prompt, the generative AI model generates the next story development and returns it to the server, which then displays the generated story development on the user's device.

[0211] Examples

[0212] Suppose the user starts reading a story. Suppose the story reaches a scene where a character gets happily married. The emotion analyzer (Microsoft Azure Face API) recognizes the user's facial expression as "happiness" and responds to the server with "emotion":"happiness". In response, the server generates the following prompt:

[0213] So you've detected a situation where the user is smiling. What's the next happy twist to add to the story?

[0214] The generative AI model (OpenAI ChatGPT) generates text such as "The character won the lottery" and returns it to the server. The server displays this to the user. In this way, the story unfolds dynamically according to the user's emotions.

[0215] This system allows users to become deeply immersed in the story, creating a more realistic experience. By continuously analyzing the user's emotions and adjusting the story development and choices based on the results, the system provides a story that is in tune with the user's interests and emotions.

[0216] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0217] Step 1: Initialize the system

[0218] 1. The server starts and loads the API credentials and models.

[0219] 2. The user opens a web browser on the terminal and accesses the system's URL.

[0220] 3. The server provides the user interface and the user is presented with a login screen.

[0221] Input: None

[0222] Output: The user interface is displayed in the terminal.

[0223] Specific behavior:

[0224] The server loads the interface module and generates the HTML and JavaScript to display the login screen.

[0225] Step 2: Recognizing User Emotions

[0226] 1. After logging in, the user allows access to the device's webcam and microphone.

[0227] 2. The device uses a camera and microphone to capture the user's facial expressions and voice in real time.

[0228] 3. The server sends the acquired image data to Microsoft Azure's Face API and analyzes emotions from facial expressions.

[0229] 4. The voice data is sent to Microsoft Azure's Speech API and emotion is analyzed from the voice tone.

[0230] Input: User login information, data from camera and microphone

[0231] Output: Emotion data based on facial expressions, emotion data based on voice tones

[0232] Specific behavior:

[0233] The server sends image data to the Face API and receives emotion tags such as "happiness" or "sadness." Similarly, it sends voice data to the Speech API and receives emotion evaluation results.

[0234] Step 3: Creating and developing a story

[0235] 1. The server determines the user's current emotion based on the emotion analysis results.

[0236] 2. The server generates a prompt based on the emotion (e.g., "Since the user is feeling happy, what happy development should we add next?").

[0237] 3. Send the prompt to the generative AI model to get the next story development.

[0238] 4. Display the generated narrative text to the user.

[0239] Input: Sentiment analysis results

[0240] Output: Story development text

[0241] Specific behavior:

[0242] The server processes the analysis results in the analysis module and generates a new prompt sentence. The generative AI model generates a new story paragraph based on this prompt and returns it. The server displays this text in the user interface.

[0243] Step 4: Present options based on emotions

[0244] 1. The server continuously monitors the user's emotional data.

[0245] 2. As the user reads the story, generate prompts that offer options related to the next development (e.g., "If the user is sad, how would you present a comforting scenario?").

[0246] 3. The prompt is sent to a generative AI model to generate appropriate options.

[0247] 4. Display the generated options to the user and let them select.

[0248] Input: Real-time emotion data

[0249] Output: The text of the choice

[0250] Specific behavior:

[0251] The server receives emotion data in real time and generates a prompt based on it. The generative AI model generates the next options based on this prompt and returns them. The server displays these options in the user interface.

[0252] Step 5: Emotional feedback and regulation

[0253] 1. As the story progresses, the server again collects the user's facial expressions and voice and performs emotion analysis.

[0254] 2. Based on the user's new emotional data, the server again sends the next story development and choices to the generative AI model for adjustment.

[0255] 3. Present the user again with the adjusted storyline and options.

[0256] Input: Recaptured facial expression data and voice data

[0257] Output: Newly adjusted story development and choices

[0258] Specific behavior:

[0259] The server regenerates the prompt based on the new analysis and sends it to the generative AI model, which generates new text and sends it back to the user, who sees the updated story.

[0260] By repeating the above steps, the system dynamically generates a story and provides options according to the user's emotions.

[0261] (Application example 2)

[0262] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0263] Conventional interactive story generation systems did not consider story development that reflected the user's emotions. As a result, it was difficult for the user to emotionally empathize with the content of the story, resulting in a lack of immersion. In response to this issue, the present invention aims to provide a deeper immersive and interactive reading experience by reflecting the user's emotions in real time and adjusting the story development and the presentation of options based on the user's emotions.

[0264] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0265] In this invention, the server includes: a means for receiving selection information including information on a character, a situation, a genre, and a reader selected by a user; A means for generating an original story using a prompt sentence that instructs the user to generate an original story based on the selected information and a generation AI; means for identifying the user's emotion using a camera and an emotion recognition model; A means for presenting the user with a plurality of options regarding the progression of the story during the course of the story; A prompt sentence instructing to generate a continuation of the story based on any one of the options selected by the user and the identified user emotion, and a means for generating a continuation of the story using the generation AI; means for displaying the generated story to the user; A means for reading out the content of the story by a voice synthesis engine; Including, The process of presenting the options, generating the continuation of the story, and displaying and reading it to the user is repeated. This allows the story to develop in response to the user's emotions.

[0266] "User" refers to an individual who uses the system to interactively experience a story.

[0267] A "character" is a character or an animal or other entity that appears in a story.

[0268] "Situation" refers to the setting of the story, such as the location, circumstances, and background environment.

[0269] "Genre" refers to the classification or category to which a story belongs, such as fantasy, science fiction, or mystery.

[0270] "Reader" refers to the person who receives the story, i.e. the user or a third party.

[0271] "Selection information" refers to data including information about the character, situation, genre, or reader selected by the user.

[0272] "Generative AI" refers to algorithms or computer programs that use natural language processing to generate original stories based on given input information.

[0273] An "emotion recognition model" refers to an algorithm or computer program that analyzes a user's facial, voice, and other characteristics to identify their emotions.

[0274] "Story development" refers to the progression of the story or changes in the storyline, and is adjusted based on the user's selections and emotions.

[0275] "Choices" are multiple selectable options presented to the user during the course of the story.

[0276] The "updating means" refers to a method or device for dynamically updating the progress of the story based on the user's selection and emotion recognition results.

[0277] "Means for displaying" refers to a display device or interface that visually presents the progress of the story and selection information to the user.

[0278] A "speech synthesis engine" refers to an algorithm or software that converts text data into voice data and reads it aloud.

[0279] "Reading aloud" refers to the act of providing a story or options generated using a speech synthesis engine to the user as audio.

[0280] The present invention provides a system for accepting selection information including information on a character, a situation, a genre, or a reader selected by a user. Next, the configuration and operation of this system will be specifically described.

[0281] System Configuration

[0282] 1. User device:

[0283] Camera (to capture the user's face)

[0284] Display (to show the story and options)

[0285] A microphone (to receive audio input)

[0286] Speaker (to read the story aloud)

[0287] 2. Server:

[0288] Data generation model (algorithm for generating original stories)

[0289] Emotion recognition models (algorithms for identifying user emotions)

[0290] A natural language processing engine (to process the generated narrative in natural language)

[0291] A speech synthesis engine (to convert the generated text into audio data and read it aloud)

[0292] Program Processing

[0293] 1. Acceptance of Selection Information:

[0294] The user terminal transmits information about the character, situation, genre, or reader to the server. This selection information is used as setting information that forms the basis of the story.

[0295] 2. Narrative Generation:

[0296] The data generation model on the server generates a story using a prompt sentence that instructs the data generation model to generate an original story based on the selected information received, and the data generation model (generative AI). This data generation model utilizes a natural language processing engine to dynamically create an original story that the user does not know.

[0297] 3. Emotion recognition:

[0298] The user's face is captured by the camera on the user's device, and the emotion recognition model analyzes the facial expressions to determine the current emotion of the user.

[0299] 4. Story Development Adjustment:

[0300] The server adjusts the storyline when generating the next storyline based on the emotion identified by the emotion recognition model, for example, if the user is showing happy emotion, the storyline is set to take a happy turn.

[0301] 5. Present your options:

[0302] During the story, the server presents the user with multiple options regarding the next development of the story. The user selects one of the options to determine the progress of the story.

[0303] 6. Story Progression and Updates:

[0304] The server generates the continuation of the story using a prompt sentence that instructs the server to generate a continuation of the story based on any one of the options selected by the user and the identified user emotion, and the generation AI. The generated continuation of the story is displayed on the user terminal and is read aloud by a voice synthesis engine.

[0305] Examples

[0306] Suppose a user launches an application, selects the fantasy genre, chooses a "hero" as the character (protagonist), a "village" as the situation, and a "teenage male" as the reader. As the user reads the story, the camera captures the user's happy facial expression, and the emotion recognition model identifies the emotion "joy." In this case, the story develops in a happy way, with the hero defeating the dragon and being thanked by the villagers. Also, in the middle of the story, the options of "saving the village" and "searching for treasure" are presented, and if the user selects "searching for treasure," the story will progress based on that choice.

[0307] An example of an initial prompt for generating a story: "Please generate a story based on the genre "Fantasy", the character (protagonist) "Hero", and the situation "Village". The reader is a "teenage boy". Please generate a story that suits this reader."

[0308] An example of an initial prompt for generating a continuation of a story: "Generate a continuation of the story based on the option "Treasure Search." The reader's emotion is "Joy." Generate a continuation of the story that matches this reader's emotion."

[0309] As described above, the present invention makes it possible to reflect a user's emotions in real time and provide an interactive storytelling experience.

[0310] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0311] Step 1:

[0312] User input of selection information

[0313] The user selects information about the characters, situation, genre, and reader through the application on the terminal. This selection information is sent from the terminal to the server. The input data is received as a string of characters, which becomes the basis for the next story generation.

[0314] Step 2:

[0315] Beginning of story generation

[0316] The server uses a data generation model to generate an original story based on the selection information received in step 1. Specifically, it uses a natural language processing engine to output a story that matches the selection information as text data.

[0317] Step 3:

[0318] User Emotion Recognition

[0319] While the user reads the story, the device's camera captures the user's face. The facial image is sent to the server in real time, and the emotion recognition model analyzes the image. As a result of the analysis, the user's emotion is identified as "happy" or "sad," etc., and the emotion data is output.

[0320] Step 4:

[0321] Adjusting story development based on emotions

[0322] The server adjusts the development of the story by including the emotion data obtained in step 3 in the prompt sentence when generating the rest of the story. For example, if the user shows happy emotion, the server changes the development of the story by including a sentence such as "The reader is feeling happy" in the prompt sentence when generating the rest of the story.

[0323] Step 5:

[0324] Present options for story progression

[0325] During the story, the server presents the user with multiple options for the next development. For example, the server displays the question "Where should the hero go next?" along with options such as "Climb the mountain" or "Cross the river." The user selects an option from the options.

[0326] Step 6:

[0327] Choice-based story updates

[0328] The server receives the choice selected by the user and updates the story progression based on this. The data generation model is then used again to generate the next part of the story based on the choice. For example, if the user selects "Cross the river," a sentence such as "The hero mustered up his courage and decided to cross the river" is generated.

[0329] Step 7:

[0330] View and read stories

[0331] The updated story is displayed on the device's display and is read aloud by a speech synthesis engine. The speech synthesis engine converts the generated text data into speech data and reads it through the speaker. The input is the updated text data, and the output is speech data.

[0332] By repeating the above steps, an interactive storytelling experience based on the user's emotions and choices is realized.

[0333] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0334] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0335] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0336] [Second embodiment]

[0337] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0338] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0339] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0340] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0341] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0342] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0343] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0344] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0345] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0346] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0347] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0348] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0349] An embodiment for implementing the present invention includes the following elements.

[0350] (Element 1: User Interface)

[0351] A user accesses the story creation application via a terminal. An interface for selecting information on characters, situations, genres, and readers is displayed on the terminal. The user selects this information and transmits the selected information to the server via the terminal.

[0352] (Element 2: Server processing)

[0353] The server uses a generative AI to generate a story based on the received selection information. The generative AI uses natural language processing technology to generate a part of the original story based on the user's selection. The server transmits the generated story to the terminal.

[0354] (Element 3: Story presentation and options)

[0355] The device displays the received story and presents the user with options generated by generative AI during the story. The user selects one option from the options presented and sends it to the server via the device.

[0356] (Element 4: Story progression and updating)

[0357] The server updates the story progression using a generative AI based on the user's selection. The generative AI generates a new part of the story according to the selection, and the server transmits the updated story to the device.

[0358] (Element 5: Narrative presentation and feedback)

[0359] The device displays the received story updates, allowing the user to see the results of their choices. Users get feedback on the progress of the story and can freely customize the story development.

[0360] The process of presenting options, updating the progress of the story, and displaying to the user is repeated until the story is completed.

[0361] With these elements, users can create original stories based on their own choices through a story creation application.

[0362] For example, if a user selects a "brave adventurer" as the main character, "medieval Europe" as the setting of the story, and "adventure" as the genre, the generative AI will generate a story based on those selections. In the middle of the story, the user is presented with options such as "enter the cave," "explore the forest," and "talk to the villagers." If the user selects "enter the cave," the story will continue with an adventure in the cave. On the other hand, if the user selects "explore the forest," the story will move on to an adventure in the forest.

[0363] The process flow will be explained below.

[0364] Step 1: When a user accesses the story creation application, the device connects to the server.

[0365] Step 2: An interface is displayed on the terminal to select information about the character, situation, genre, and reader. The user selects this information and sends it to the server via the terminal.

[0366] Step 3: The server uses a generative AI to generate a story based on the received information. The generative AI uses natural language processing techniques to generate parts of an original story based on the user's selections.

[0367] Step 4: The server sends the generated story to the terminal.

[0368] Step 5: The device displays the received story and presents options to the user during the story.

[0369] Step 6: The user selects one of the options presented and sends it to the server via the terminal.

[0370] Step 7: The server updates the story progression using the generative AI based on the received selection information. The generative AI generates a new part of the story according to the selection, and the server sends the updated story to the device.

[0371] Step 8: The device displays the received story update. Present the user with options again during the story. Repeat steps 6 to 7.

[0372] This is the process flow of this system. The user selects information, the terminal sends it to the server, which generates a story, and the terminal displays the story and presents options to the user. The user selects an option, the terminal sends it to the server, which updates the story, and the terminal again displays the story and presents options to the user. As this process is repeated, the user can progress through the story based on their own choices.

[0373] Example 1

[0374] Next, a description will be given of Example 1. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0375] In conventional story creation systems, it was difficult for users to freely create a story based on their own choices. In particular, there was a problem that the interactivity experienced by the user was reduced because the progress and updates of the story according to the user's choices were done manually. In addition, the created stories sometimes lacked consistency, which reduced user satisfaction. It is necessary to provide a story creation system that can solve these problems and is more interactive and flexibly respond to user choices.

[0376] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0377] In this invention, the server includes a means for receiving selection information including information on a character, a situation, a genre, or a reader selected by a user, a means for generating an original story using a generative AI model, a means for presenting options regarding the progress of the story to the user in the middle of the story, a means for updating the progress of the story using the generative AI model based on the options selected by the user, and a means for displaying updated information on the story to the user. This allows the user to enjoy a story that is dynamically generated based on their own selection.

[0378] A "user" is an entity that uses the system to interactively create a story.

[0379] "Selected information" refers to information including the character, situation, genre, and reader selected by the user.

[0380] A "generative AI model" is an artificial intelligence technology that generates original stories based on user-selected information.

[0381] "Story progression" refers to the process and content of how the story unfolds.

[0382] "Choices" are multiple, selectable options for story progression presented to the user.

[0383] An "emotion analysis engine" is a technology that recognizes the user's emotions and adjusts the story progression based on those emotions.

[0384] "Natural language processing" is a language processing technique used when a generative AI model generates sentences using prompts.

[0385] "Updated information" refers to the content of the story that has been updated based on user selections and sentiment analysis.

[0386] The present invention relates to a system that allows users to interactively create stories, which consists of a user interface, a server process, a story display, and updating of the story progress based on user selections.

[0387] User Interface

[0388] A user accesses a story creation application using a terminal such as a smartphone or a PC. The application displays an interface for selecting information on characters, situations, genres, and readers. The user selects this information and transmits the selected information to the server via the terminal.

[0389] Server Processing

[0390] The server receives the selection information selected by the user. The server is built using a programming language such as Python or Java, and inputs the selection information into a generative AI model (e.g., ChatGPT). The generative AI model uses natural language processing technology to generate an original story based on the user's selection. The generated story is sent from the server to the terminal.

[0391] Presenting the story and giving options

[0392] The device displays the received story on the screen. As the story progresses, the user is presented with options to choose from during the story generated by the generative AI model. For example, options such as "Enter the cave," "Explore the forest," and "Talk to the villagers" are displayed. The user selects one of the options presented and sends the selection information to the server via the device.

[0393] Story progression and updates

[0394] The server receives the user's choice and generates the rest of the story using a generative AI model. The generative AI model generates a story in the following format as a prompt:

[0395] The main character is a brave adventurer, and you must create an adventure story set in medieval Europe. The actions the main character can choose in the next scene are to "enter the cave," "explore the forest," or "talk to the villagers." Generate a sequel for each option.

[0396] The new part of the story that is generated is again sent from the server to the terminal.

[0397] View story updates and give feedback

[0398] The device displays the updated story on the screen and provides feedback to the user, who can review the results of their choices and enjoy the progression of the story. This process of presenting choices and updating the story continues until the story is complete.

[0399] The present invention allows users to enjoy a story that is interactively generated based on their own selections, thereby maintaining the consistency of the story and improving user satisfaction.

[0400] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0401] Step 1:

[0402] A user accesses a story creation application using a device such as a smartphone or a PC. The device displays a user interface and presents options for selecting information on characters, situations, genres, and readers. The user selects this information. The input is the information selected by the user, and the output is the state in which the selected information is complete.

[0403] Step 2:

[0404] The terminal receives the user's selection information and sends it to the server through an HTTP request. The input is the user's selection information, and the output is the selection information being sent to the server. This passes the selection information to the server.

[0405] Step 3:

[0406] The server sends a prompt to the generative AI model based on the received selection information. The input is the user's selection information, and the output is a prompt to the generative AI model. The server uses Python to generate a prompt like this:

[0407] "Create an adventure story for your hero, a brave adventurer, set in medieval Europe."

[0408] Step 4:

[0409] The generative AI model analyzes the prompt and generates an original part of the story based on the user's specifications. The input is the prompt, and the output is the generated story text. For example, the story generated by the generative AI model is "A brave adventurer arrives in a medieval village and sets out on an unknown adventure."

[0410] Step 5:

[0411] The server sends the generated story to the terminal. The input is the generated story text and the output is the message sent to the terminal. The server converts the story text to JSON format and sends it as an HTTP response.

[0412] Step 6:

[0413] The terminal parses the received story and displays it on the screen. The input is the story text received from the server and the output is the display to the user interface. Once the story is displayed, the user can read it.

[0414] Step 7:

[0415] During the story, the device presents the user with options generated by the generative AI model. The input is part of the generated story, and the output is the options presented to the user. For example, options such as "Enter the cave," "Explore the forest," and "Talk to the villagers" are displayed.

[0416] Step 8:

[0417] The user selects one of the options presented, and the device again sends the selection information to the server. The input is the user's selection information, and the output is the transmission to the server. Once the selection information is passed to the server, the server again uses the generative AI model to generate the next part of the story.

[0418] Step 9:

[0419] The server updates the story progression using a generative AI model based on the user's choices. The input is the user's selection information, and the output is the continuation of the generated story. If the user selects "Enter the cave," the generative AI model generates a story such as "The brave adventurer stepped into the dark cave."

[0420] Step 10:

[0421] The server sends the updated story to the terminal. The input is the generated story continuation, and the output is the message sent to the terminal. The server converts the story text to JSON format and sends it as an HTTP response.

[0422] Step 11:

[0423] The device displays the updated story on the screen and provides feedback to the user. The input is the updated story text received from the server, and the output is the display on the user interface. The user can confirm the results of their selection and enjoy the story progression again. This process of presenting options and updating the story is repeated until the story is completed.

[0424] (Application example 1)

[0425] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0426] Current interactive storytelling systems have difficulty incorporating educational content effectively, and users have limited options to maximize their learning. In addition, there are few examples of systems that dynamically customize the story progression based on user choices to provide educational value.

[0427] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0428] In this invention, the server includes a means for receiving selection information including information on a character, a situation, a genre, or a reader selected by a user, a means for generating an original story using a generative AI model based on the selection information, a means for presenting options regarding the progress of the story to the user during the story, a means for updating the progress of the story using the generative AI model based on the options selected by the user, a means for displaying update information on the story to the user, a means for selecting an educational theme, and a means for customizing the story based on the educational theme. This makes it possible to dynamically generate and update a story based on an educational theme selected by a user, providing an interactive and highly educational story experience.

[0429] A "character" is someone who appears in a story or is set as the protagonist.

[0430] A "situation" is the scene or condition in which a story or event occurs.

[0431] A "genre" is a category that classifies types and themes of stories.

[0432] "Reader information" refers to data and attribute information about the recipients of the story.

[0433] "Selection information" is data including information on the character, situation, genre, or reader selected by the user.

[0434] A "generative AI model" is an algorithm that uses artificial intelligence to generate data and text.

[0435] "Narrative progression" is the process by which a story's plot or storyline unfolds.

[0436] "Choices" are options that a user can choose from to affect the progression of the story.

[0437] "Updates" are story data that are generated or modified based on user selections.

[0438] "Educational Theme" refers to specific content or concepts related to learning or teaching.

[0439] "Customization" is the act of changing and adjusting a story or system according to the user's choices and needs.

[0440] The system that realizes this application example consists of five main parts: the user interface, server processing, story presentation and choice generation, story progress and update, and story display and feedback. Specifically, it takes the following form.

[0441] User Interface

[0442] The server provides an interface that allows users to access the story creation application using a terminal and select characters, situations, genres, and educational themes. The interface is built as a smartphone or tablet application and is developed using React Native.

[0443] Server Processing

[0444] The device sends the user's selections to the server, where requests are received and analyzed on the server side using Python and Flask. The server uses a generative AI model (OpenAI's ChatGPT) to generate an original story based on the selections. During the generation process, the server constructs prompts that match the characters, situations, and educational themes selected by the user, and inputs them into the AI ​​model.

[0445] Presenting a story and generating options

[0446] The device displays the original story and presents the user with options for progressing through the story. Options are displayed according to the situation. For example, in an ancient Egyptian setting, options include "Learn how to build pyramids" and "Explore Egyptian culture."

[0447] Story progression and updates

[0448] After the user selects an option, the selection information is sent to the server again. The server again uses the ChatGPT API to generate and update the story progression based on the selected option, and the prompt text reflects the user's selection.

[0449] Viewing and Feedback on Stories

[0450] The updated story is sent to the terminal, and the user can view the update information to check the progress of the story. If the user makes the right choice based on the story options, an educational learning effect can be obtained. Regarding the specific learning content, the story is appropriately customized according to the user's feedback.

[0451] For example, if you select an "Ancient Scholar" as the character, a "Medieval Castle" as the setting, and "Mathematics Basics" as the theme for a third-grade history lesson, the generative AI model will receive the following prompt as input and generate a story:

[0452] Input prompt example

[0453] "Generate the following story: Character: Ancient Scholar; Situation: Medieval Castle; Theme: Mathematics Basics. The Ancient Scholar teaches a group of children how to design a castle. Include the following options: a) how to calculate area b) how to measure height c) how much stone to use."

[0454] In this way, a system is realized that dynamically generates and updates stories based on conditions selected by the user, providing educational value.

[0455] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0456] Step 1:

[0457] The terminal provides an interface for users to access the story creation application and select characters, situations, genres, and educational themes. The user interface is built with React Native and receives user selection information. This input includes data such as the characters and situations selected by the user.

[0458] Step 2:

[0459] The device sends the user's selection information in JSON format to the server. The server uses Python and Flask to analyze the received selection information and constructs a prompt for the generative AI model (OpenAI's ChatGPT). The prompt specifically reflects the selection information.

[0460] Step 3:

[0461] The server uses a generative AI model to generate an original story based on the prompt sentence. The AI ​​model generates a series of text data, which becomes the initial part of the story. The prompt sentence is given as input, and the story text is obtained as output.

[0462] Step 4:

[0463] The server sends the generated story to the device, which then displays it to the user, inserting options into the appropriate places in the story so that the user can choose options for the progression of the story. The options are also displayed through an interface built with React Native.

[0464] Step 5:

[0465] The user selects one of the options presented and sends the selection information to the server again via the terminal. The server receives this selection information and again uses the generative AI model to generate the progression of the story based on it. Specifically, it constructs a prompt sentence again and inputs it into the AI ​​model. The prompt sentence reflects the user's selection.

[0466] Step 6:

[0467] The server sends the updated story parts to the terminal, and the terminal displays the updated story information to the user. The user checks the updated story and the story progresses interactively. This allows the story progress to change dynamically according to the user's selections.

[0468] Step 7:

[0469] This process is repeated until the end of the story. Each time the user selects an option at each stage of the story, the server updates the story based on the selection and displays the updated information on the terminal. Here, feedback content based on educational themes is also displayed to enhance the user's learning effect.

[0470] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0471] In this case, the embodiment of the present invention further includes the following elements.

[0472] (Element 1: Incorporating an emotion engine)

[0473] This system is equipped with an emotion engine that can analyze information such as the user's facial expressions and voice and recognize the user's emotions.

[0474] (Element 2: Emotionally-driven story development)

[0475] Based on the user's emotions recognized by the emotion engine, the generative AI can adjust the story development. For example, if the user feels joy in the story, the emotion engine can add a happy development to the story.

[0476] (Element 3: Presenting options based on emotions)

[0477] Based on the user's emotions recognized by the emotion engine, the generative AI will adjust the options: for example, if the user feels sadness in the story, the generative AI can present comforting options.

[0478] (Element 4: Emotional Feedback)

[0479] The user's emotions may change as the story progresses or choices are presented. The emotion engine can detect changes in the user's emotions and reflect them in the development of the story or in adjusting choices.

[0480] By combining the above elements with the emotion engine, this system can recognize the user's emotions and adjust the story development and the presentation of options according to the emotions. This allows the user to become more emotionally immersed in the story and experience a more realistic feeling.

[0481] The process flow will be explained below.

[0482] Step 1: When a user accesses the story creation application, the device connects to the server.

[0483] Step 2: An interface for selecting information on the character, situation, genre, and reader is displayed on the terminal. The user selects this information and sends it to the server via the terminal.

[0484] Step 3: The server uses the generative AI to generate part of the story based on the received information. At the same time, the emotion engine recognizes the user's emotions. The generative AI and emotion engine work together to adjust the story development and options according to the user's emotions.

[0485] Step 4: The server sends the generated story and emotion-based options to the device.

[0486] Step 5: The device displays the received story and presents the user with options. At the same time, the emotion engine analyzes the user's emotions and detects changes in the user's emotions.

[0487] Step 6: The user selects one of the options presented and sends it to the server via the terminal. At the same time, the emotion engine detects changes in the user's emotions and sends them to the server.

[0488] Step 7: The server updates the story progression by coordinating the generative AI and emotion engine based on the received selection information and emotion information. The updated story and emotion-based choices are sent to the terminal.

[0489] Step 8: The device displays the received story update information and presents the user with options. At the same time, the emotion engine analyzes the user's emotions and detects changes in the user's emotions. Steps 6 to 7 are repeated.

[0490] This is the process flow of this system. The user selects information, and the device sends it to the server, generating a story, which then displays the story and presents options to the user. The user selects an option, and the device sends it to the server, updating the story, which then again displays the story and presents options to the user. As this process is repeated, the user's emotions are analyzed by the emotion engine and reflected in the development of the story and adjustments to the options.

[0491] Example 2

[0492] Next, a description will be given of Example 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0493] Conventional interactive story generation systems have the problem that it is difficult to present story development and options that fully reflect the user's emotions. Therefore, there is a demand for a system that can adjust story development and options in real time according to the user's emotions.

[0494] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving selection information including information on a character, a situation, a genre, or a reader selected by a user, a means for generating an original story using a data generation model based on the selection information, a means for presenting options regarding the progress of the story to the user in the middle of the story, a means for updating the progress of the story using the data generation model based on the options selected by the user, and a means for acquiring the user's facial expression and voice and analyzing the user's emotions using an emotion analysis device in the updating means, a means for sending a prompt sentence to a generation AI model based on the user's emotions and generating the next story development, a means for displaying the generated story development and options to the user, and a means for detecting a change in the user's emotions again and continuously adjusting the story progress and options. This makes it possible to present a dynamic and real-time story development and options according to the user's emotions.

[0495] A "user" is an entity that uses this system to read a story and select options.

[0496] A "character" is someone who appears in a story or is the main character.

[0497] A "situation" refers to a scene in a story or a scene setting selected by the user.

[0498] "Genre" refers to the type or classification of a story, such as fantasy, suspense, romance, etc.

[0499] "Selected information" refers to information such as the character, situation, genre, etc. selected by the user.

[0500] A "data generation model" is a machine learning model that generates new data based on given input information.

[0501] A "narrative" refers to a textual sequence of events or a scenario that is presented to a user.

[0502] "Choices" refer to multiple options from which a user can choose regarding the progression of a story.

[0503] The "means for updating" is a function for moving the story progression to the next stage in response to a user's selection.

[0504] An "emotion analysis device" is a device or software that analyzes a user's facial expressions and voice data to determine their emotions.

[0505] A "generative AI model" is an artificial intelligence algorithm that generates new text based on a given prompt.

[0506] A "prompt sentence" is a sentence that is given to a generative AI model as instructions when it generates new text.

[0507] "Story development" refers to the progression or progress of a story.

[0508] "Presenting options" is the act of displaying options to the user for what to select next.

[0509] "Changes in the user's emotions" refers to changes in the user's state of mind due to the progression of the story or other factors.

[0510] The embodiment of this invention is an interactive story generation system that can analyze the user's emotions in real time and dynamically generate story development and options based on the analysis. This system uses the following main hardware and software:

[0511] Hardware

[0512] 1. Terminal:

[0513] Devices such as PCs, smartphones, tablets, etc. These devices must have a webcam and microphone.

[0514] 2. Server:

[0515] A high-performance computer that acts as the central nerve center of the entire system.

[0516] software

[0517] 1. Emotion analysis device:

[0518] Microsoft Azure Face API

[0519] Microsoft Azure Speech API

[0520] 2. Generative AI model:

[0521] OpenAI ChatGPT

[0522] Data processing and calculation flow

[0523] The server transmits facial expression and voice data acquired in real time from the user's device to the emotion analysis device, which analyzes the user's emotions. Based on the emotion data obtained as a result of the analysis, the server sends a prompt to the generative AI model to generate the next development of the story. This prompt is based on the user's emotions. For example, if the user is expressing the emotion of "joy," the following prompt is generated:

[0524] So you've detected a situation where the user is smiling. What's the next happy twist to add to the story?

[0525] Based on this prompt, the generative AI model generates the next story development and returns it to the server, which then displays the generated story development on the user's device.

[0526] Examples

[0527] Suppose the user starts reading a story. Suppose the story reaches a scene where a character gets happily married. The emotion analyzer (Microsoft Azure Face API) recognizes the user's facial expression as "happiness" and responds to the server with "emotion":"happiness". In response, the server generates the following prompt:

[0528] So you've detected a situation where the user is smiling. What's the next happy twist to add to the story?

[0529] The generative AI model (OpenAI ChatGPT) generates text such as "The character won the lottery" and returns it to the server. The server displays this to the user. In this way, the story unfolds dynamically according to the user's emotions.

[0530] This system allows users to become deeply immersed in the story, creating a more realistic experience. By continuously analyzing the user's emotions and adjusting the story development and choices based on the results, the system provides a story that is in tune with the user's interests and emotions.

[0531] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0532] Step 1: Initialize the system

[0533] 1. The server starts and loads the API credentials and models.

[0534] 2. The user opens a web browser on the terminal and accesses the system's URL.

[0535] 3. The server provides the user interface and the user is presented with a login screen.

[0536] Input: None

[0537] Output: The user interface is displayed in the terminal.

[0538] Specific behavior:

[0539] The server loads the interface module and generates the HTML and JavaScript to display the login screen.

[0540] Step 2: Recognizing User Emotions

[0541] 1. After logging in, the user allows access to the device's webcam and microphone.

[0542] 2. The device uses a camera and microphone to capture the user's facial expressions and voice in real time.

[0543] 3. The server sends the acquired image data to Microsoft Azure's Face API and analyzes emotions from facial expressions.

[0544] 4. The voice data is sent to Microsoft Azure's Speech API and emotion is analyzed from the voice tone.

[0545] Input: User login information, data from camera and microphone

[0546] Output: Emotion data based on facial expressions, emotion data based on voice tones

[0547] Specific behavior:

[0548] The server sends image data to the Face API and receives emotion tags such as "happiness" or "sadness." Similarly, it sends voice data to the Speech API and receives emotion evaluation results.

[0549] Step 3: Creating and developing a story

[0550] 1. The server determines the user's current emotion based on the emotion analysis results.

[0551] 2. The server generates a prompt based on the emotion (e.g., "Since the user is feeling happy, what happy development should we add next?").

[0552] 3. Send the prompt to the generative AI model to get the next story development.

[0553] 4. Display the generated narrative text to the user.

[0554] Input: Sentiment analysis results

[0555] Output: Story development text

[0556] Specific behavior:

[0557] The server processes the analysis results in the analysis module and generates a new prompt sentence. The generative AI model generates a new story paragraph based on this prompt and returns it. The server displays this text in the user interface.

[0558] Step 4: Present options based on emotions

[0559] 1. The server continuously monitors the user's emotional data.

[0560] 2. As the user reads the story, generate prompts that offer options related to the next development (e.g., "If the user is sad, how would you present a comforting scenario?").

[0561] 3. The prompt is sent to a generative AI model to generate appropriate options.

[0562] 4. Display the generated options to the user and let them select.

[0563] Input: Real-time emotion data

[0564] Output: The text of the choice

[0565] Specific behavior:

[0566] The server receives emotion data in real time and generates a prompt based on it. The generative AI model generates the next options based on this prompt and returns them. The server displays these options in the user interface.

[0567] Step 5: Emotional feedback and regulation

[0568] 1. As the story progresses, the server again collects the user's facial expressions and voice and performs emotion analysis.

[0569] 2. Based on the user's new emotional data, the server again sends the next story development and choices to the generative AI model for adjustment.

[0570] 3. Present the user again with the adjusted storyline and options.

[0571] Input: Recaptured facial expression data and voice data

[0572] Output: Newly adjusted story development and choices

[0573] Specific behavior:

[0574] The server regenerates the prompt based on the new analysis and sends it to the generative AI model, which generates new text and sends it back to the user, who sees the updated story.

[0575] By repeating the above steps, the system dynamically generates a story and provides options according to the user's emotions.

[0576] (Application example 2)

[0577] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0578] Conventional interactive story generation systems did not consider story development that reflected the user's emotions. As a result, it was difficult for the user to emotionally empathize with the content of the story, resulting in a lack of immersion. In response to this issue, the present invention aims to provide a deeper immersive and interactive reading experience by reflecting the user's emotions in real time and adjusting the story development and the presentation of options based on the user's emotions.

[0579] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0580] In this invention, the server includes: a means for receiving selection information including information on a character, a situation, a genre, and a reader selected by a user; A means for generating an original story using a prompt sentence that instructs the user to generate an original story based on the selected information and a generation AI; means for identifying the user's emotion using a camera and an emotion recognition model; means for presenting the user with a plurality of options regarding the progression of the story during the course of the story; A prompt sentence instructing to generate a continuation of the story based on any one of the options selected by the user and the identified user emotion, and a means for generating a continuation of the story using the generation AI; means for displaying the generated story to the user; a means for reading out the content of the story by a voice synthesis engine; Including, The process of presenting the options, generating the continuation of the story, and displaying and reading it to the user is repeated. This allows the story to develop in response to the user's emotions.

[0581] "User" refers to an individual who uses the system to interactively experience a story.

[0582] A "character" is a character or an animal or other entity that appears in a story.

[0583] "Situation" refers to the setting of the story, such as the location, circumstances, and background environment.

[0584] "Genre" refers to the classification or category to which a story belongs, such as fantasy, science fiction, or mystery.

[0585] "Reader" refers to the person who receives the story, i.e. the user or a third party.

[0586] "Selection information" refers to data including information about the character, situation, genre, or reader selected by the user.

[0587] "Generative AI" refers to algorithms or computer programs that use natural language processing to generate original stories based on given input information.

[0588] An "emotion recognition model" refers to an algorithm or computer program that analyzes a user's facial, voice, and other characteristics to identify their emotions.

[0589] "Story development" refers to the progression of the story or changes in the storyline, and is adjusted based on the user's selections and emotions.

[0590] "Choices" are multiple selectable options presented to the user during the course of the story.

[0591] The "updating means" refers to a method or device for dynamically updating the progress of the story based on the user's selection and emotion recognition results.

[0592] "Means for displaying" refers to a display device or interface that visually presents the progress of the story and selection information to the user.

[0593] A "speech synthesis engine" refers to an algorithm or software that converts text data into voice data and reads it aloud.

[0594] "Reading aloud" refers to the act of providing a story or options generated using a speech synthesis engine to the user as audio.

[0595] The present invention provides a system for accepting selection information including information on a character, a situation, a genre, or a reader selected by a user. Next, the configuration and operation of this system will be specifically described.

[0596] System Configuration

[0597] 1. User device:

[0598] Camera (to capture the user's face)

[0599] Display (to show the story and options)

[0600] A microphone (to receive audio input)

[0601] Speaker (to read the story aloud)

[0602] 2. Server:

[0603] Data generation model (algorithm for generating original stories)

[0604] Emotion recognition models (algorithms for identifying user emotions)

[0605] A natural language processing engine (to process the generated narrative in natural language)

[0606] A speech synthesis engine (to convert the generated text into audio data and read it aloud)

[0607] Program Processing

[0608] 1. Acceptance of Selection Information:

[0609] The user terminal transmits information about the character, situation, genre, or reader to the server. This selection information is used as setting information that forms the basis of the story.

[0610] 2. Narrative Generation:

[0611] The data generation model on the server generates a story using a prompt sentence that instructs the data generation model to generate an original story based on the selected information received, and the data generation model (generative AI). This data generation model utilizes a natural language processing engine to dynamically create an original story that the user does not know.

[0612] 3. Emotion recognition:

[0613] The user's face is captured by the camera on the user's device, and the emotion recognition model analyzes the facial expressions to determine the current emotion of the user.

[0614] 4. Story Development Adjustment:

[0615] The server adjusts the storyline when generating the next storyline based on the emotion identified by the emotion recognition model, for example, if the user is showing happy emotion, the storyline is set to take a happy turn.

[0616] 5. Present your options:

[0617] During the story, the server presents the user with multiple options regarding the next development of the story. The user selects one of the options to determine the progress of the story.

[0618] 6. Story Progression and Updates:

[0619] The server generates the continuation of the story using a prompt sentence that instructs the server to generate a continuation of the story based on any one of the options selected by the user and the identified user emotion, and the generation AI. The generated continuation of the story is displayed on the user terminal and is read aloud by a voice synthesis engine.

[0620] Examples

[0621] Suppose a user launches an application, selects the fantasy genre, chooses a "hero" as the character (protagonist), a "village" as the situation, and a "teenage male" as the reader. As the user reads the story, the camera captures the user's happy facial expression, and the emotion recognition model identifies the emotion "joy." In this case, the story develops in a happy way, with the hero defeating the dragon and being thanked by the villagers. Also, in the middle of the story, the options of "saving the village" and "searching for treasure" are presented, and if the user selects "searching for treasure," the story will progress based on that choice.

[0622] An example of an initial prompt for generating a story: "Please generate a story based on the genre "Fantasy", the character (protagonist) "Hero", and the situation "Village". The reader is a "teenage boy". Please generate a story that suits this reader."

[0623] An example of an initial prompt for generating a continuation of a story: "Generate a continuation of the story based on the option "Treasure Search." The reader's emotion is "Joy." Generate a continuation of the story that matches this reader's emotion."

[0624] As described above, the present invention makes it possible to reflect a user's emotions in real time and provide an interactive storytelling experience.

[0625] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0626] Step 1:

[0627] User input of selection information

[0628] The user selects information about the characters, situation, genre, and reader through the application on the terminal. This selection information is sent from the terminal to the server. The input data is received as a string of characters, which becomes the basis for the next story generation.

[0629] Step 2:

[0630] Beginning of story generation

[0631] The server uses a data generation model to generate an original story based on the selection information received in step 1. Specifically, it uses a natural language processing engine to output a story that matches the selection information as text data.

[0632] Step 3:

[0633] User Emotion Recognition

[0634] While the user reads the story, the device's camera captures the user's face. The facial image is sent to the server in real time, and the emotion recognition model analyzes the image. As a result of the analysis, the user's emotion is identified as "happy" or "sad," etc., and the emotion data is output.

[0635] Step 4:

[0636] Adjusting story development based on emotions

[0637] The server adjusts the development of the story by including the emotion data obtained in step 3 in the prompt sentence when generating the rest of the story. For example, if the user shows happy emotion, the server changes the development of the story by including a sentence such as "The reader is feeling happy" in the prompt sentence when generating the rest of the story.

[0638] Step 5:

[0639] Present options for story progression

[0640] During the story, the server presents the user with multiple options for the next development. For example, the server displays the question "Where should the hero go next?" along with options such as "Climb the mountain" or "Cross the river." The user selects an option from the options.

[0641] Step 6:

[0642] Choice-based story updates

[0643] The server receives the choice selected by the user and updates the story progression based on this. The data generation model is then used again to generate the next part of the story based on the choice. For example, if the user selects "Cross the river," a sentence such as "The hero mustered up his courage and decided to cross the river" is generated.

[0644] Step 7:

[0645] View and read stories

[0646] The updated story is displayed on the device's display and is read aloud by a speech synthesis engine. The speech synthesis engine converts the generated text data into speech data and reads it through the speaker. The input is the updated text data, and the output is speech data.

[0647] By repeating the above steps, an interactive storytelling experience based on the user's emotions and choices is realized.

[0648] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0649] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0650] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0651] [Third embodiment]

[0652] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0653] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0654] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0655] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0656] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0657] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0658] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0659] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0660] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0661] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0662] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0663] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server", and the headset type terminal 314 will be referred to as the "terminal".

[0664] An embodiment for implementing the present invention includes the following elements.

[0665] (Element 1: User Interface)

[0666] A user accesses the story creation application via a terminal. An interface for selecting information on characters, situations, genres, and readers is displayed on the terminal. The user selects this information and transmits the selected information to the server via the terminal.

[0667] (Element 2: Server processing)

[0668] The server uses a generative AI to generate a story based on the received selection information. The generative AI uses natural language processing technology to generate a part of the original story based on the user's selection. The server transmits the generated story to the terminal.

[0669] (Element 3: Story presentation and options)

[0670] The device displays the received story and presents the user with options generated by generative AI during the story. The user selects one option from the options presented and sends it to the server via the device.

[0671] (Element 4: Story progression and updating)

[0672] The server updates the story progression using a generative AI based on the user's selection. The generative AI generates a new part of the story according to the selection, and the server transmits the updated story to the device.

[0673] (Element 5: Narrative presentation and feedback)

[0674] The device displays the received story updates, allowing the user to see the results of their choices. Users get feedback on the progress of the story and can freely customize the story development.

[0675] The process of presenting options, updating the progress of the story, and displaying to the user is repeated until the story is completed.

[0676] With these elements, users can create original stories based on their own choices through a story creation application.

[0677] For example, if a user selects a "brave adventurer" as the main character, "medieval Europe" as the setting of the story, and "adventure" as the genre, the generative AI will generate a story based on those selections. In the middle of the story, the user is presented with options such as "enter the cave," "explore the forest," and "talk to the villagers." If the user selects "enter the cave," the story will continue with an adventure in the cave. On the other hand, if the user selects "explore the forest," the story will move on to an adventure in the forest.

[0678] The process flow will be explained below.

[0679] Step 1: When a user accesses the story creation application, the device connects to the server.

[0680] Step 2: An interface is displayed on the terminal to select information about the character, situation, genre, and reader. The user selects this information and sends it to the server via the terminal.

[0681] Step 3: The server uses a generative AI to generate a story based on the received information. The generative AI uses natural language processing techniques to generate parts of an original story based on the user's selections.

[0682] Step 4: The server sends the generated story to the terminal.

[0683] Step 5: The device displays the received story and presents options to the user during the story.

[0684] Step 6: The user selects one of the options presented and sends it to the server via the terminal.

[0685] Step 7: The server updates the story progression using the generative AI based on the received selection information. The generative AI generates a new part of the story according to the selection, and the server sends the updated story to the device.

[0686] Step 8: The device displays the received story update. Present the user with options again during the story. Repeat steps 6 to 7.

[0687] This is the process flow of this system. The user selects information, the terminal sends it to the server, which generates a story, and the terminal displays the story and presents options to the user. The user selects an option, the terminal sends it to the server, which updates the story, and the terminal again displays the story and presents options to the user. As this process is repeated, the user can progress through the story based on their own choices.

[0688] Example 1

[0689] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[0690] In conventional story creation systems, it was difficult for users to freely create a story based on their own choices. In particular, there was a problem that the interactivity experienced by the user was reduced because the progress and updates of the story according to the user's choices were done manually. In addition, the created stories sometimes lacked consistency, which reduced user satisfaction. It is necessary to provide a story creation system that can solve these problems and is more interactive and flexibly respond to user choices.

[0691] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0692] In this invention, the server includes a means for receiving selection information including information on a character, a situation, a genre, or a reader selected by a user, a means for generating an original story using a generative AI model, a means for presenting options regarding the progress of the story to the user in the middle of the story, a means for updating the progress of the story using the generative AI model based on the options selected by the user, and a means for displaying updated information on the story to the user. This allows the user to enjoy a story that is dynamically generated based on their own selection.

[0693] A "user" is an entity that uses the system to interactively create a story.

[0694] "Selected information" refers to information including the character, situation, genre, and reader selected by the user.

[0695] A "generative AI model" is an artificial intelligence technology that generates original stories based on user-selected information.

[0696] "Story progression" refers to the process and content of how the story unfolds.

[0697] "Choices" are multiple, selectable options for story progression presented to the user.

[0698] An "emotion analysis engine" is a technology that recognizes the user's emotions and adjusts the story progression based on those emotions.

[0699] "Natural language processing" is a language processing technique used when a generative AI model generates sentences using prompts.

[0700] "Updated information" refers to the content of the story that has been updated based on user selections and sentiment analysis.

[0701] The present invention relates to a system that allows users to interactively create stories, which consists of a user interface, a server process, a story display, and updating of the story progress based on user selections.

[0702] User Interface

[0703] A user accesses a story creation application using a terminal such as a smartphone or a PC. The application displays an interface for selecting information on characters, situations, genres, and readers. The user selects this information and transmits the selected information to the server via the terminal.

[0704] Server Processing

[0705] The server receives the selection information selected by the user. The server is built using a programming language such as Python or Java, and inputs the selection information into a generative AI model (e.g., ChatGPT). The generative AI model uses natural language processing technology to generate an original story based on the user's selection. The generated story is sent from the server to the terminal.

[0706] Presenting the story and giving options

[0707] The device displays the received story on the screen. As the story progresses, the user is presented with options to choose from during the story generated by the generative AI model. For example, options such as "Enter the cave," "Explore the forest," and "Talk to the villagers" are displayed. The user selects one of the options presented and sends the selection information to the server via the device.

[0708] Story progression and updates

[0709] The server receives the user's choice and generates the rest of the story using a generative AI model. The generative AI model generates the story as a prompt in the following format:

[0710] The main character is a brave adventurer, and you must create an adventure story set in medieval Europe. The actions the main character can choose in the next scene are to "enter the cave," "explore the forest," or "talk to the villagers." Generate a sequel for each option.

[0711] The new part of the story that is generated is again sent from the server to the terminal.

[0712] View story updates and give feedback

[0713] The device displays the updated story on the screen and provides feedback to the user, who can review the results of their choices and enjoy the progression of the story. This process of presenting choices and updating the story is repeated until the story is completed.

[0714] The present invention allows users to enjoy a story that is interactively generated based on their own selections, resulting in consistent stories and improved user satisfaction.

[0715] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0716] Step 1:

[0717] A user accesses a story creation application using a device such as a smartphone or a PC. The device displays a user interface and presents options for selecting information on characters, situations, genres, and readers. The user selects this information. The input is the information selected by the user, and the output is the state in which the selected information is complete.

[0718] Step 2:

[0719] The terminal receives the user's selection information and sends it to the server through an HTTP request. The input is the user's selection information, and the output is the selection information being sent to the server. This passes the selection information to the server.

[0720] Step 3:

[0721] The server sends a prompt to the generative AI model based on the received selection information. The input is the user's selection information, and the output is a prompt to the generative AI model. The server uses Python to generate a prompt like this:

[0722] "Create an adventure story for your hero, a brave adventurer, set in medieval Europe."

[0723] Step 4:

[0724] The generative AI model analyzes the prompt and generates an original part of the story based on the user's specifications. The input is the prompt, and the output is the generated story text. For example, the story generated by the generative AI model is "A brave adventurer arrives in a medieval village and sets out on an unknown adventure."

[0725] Step 5:

[0726] The server sends the generated story to the terminal. The input is the generated story text and the output is the message sent to the terminal. The server converts the story text to JSON format and sends it as an HTTP response.

[0727] Step 6:

[0728] The terminal parses the received story and displays it on the screen. The input is the story text received from the server and the output is the display to the user interface. Once the story is displayed, the user can read it.

[0729] Step 7:

[0730] During the story, the device presents the user with options generated by the generative AI model. The input is part of the generated story, and the output is the options presented to the user. For example, options such as "Enter the cave," "Explore the forest," and "Talk to the villagers" are displayed.

[0731] Step 8:

[0732] The user selects one of the options presented, and the device again sends the selection information to the server. The input is the user's selection information, and the output is the transmission to the server. Once the selection information is passed to the server, the server again uses the generative AI model to generate the next part of the story.

[0733] Step 9:

[0734] The server updates the story progression using a generative AI model based on the user's choices. The input is the user's selection information, and the output is the continuation of the generated story. If the user selects "Enter the cave," the generative AI model generates a story such as "The brave adventurer stepped into the dark cave."

[0735] Step 10:

[0736] The server sends the updated story to the terminal. The input is the generated story continuation, and the output is the message sent to the terminal. The server converts the story text to JSON format and sends it as an HTTP response.

[0737] Step 11:

[0738] The device displays the updated story on the screen and provides feedback to the user. The input is the updated story text received from the server, and the output is the display on the user interface. The user can confirm the results of their selection and enjoy the story progression again. This process of presenting options and updating the story is repeated until the story is completed.

[0739] (Application example 1)

[0740] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0741] Current interactive storytelling systems have difficulty incorporating educational content effectively, and users have limited options to maximize their learning. In addition, there are few examples of systems that dynamically customize the story progression based on user choices to provide educational value.

[0742] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0743] In this invention, the server includes a means for receiving selection information including information on a character, a situation, a genre, or a reader selected by a user, a means for generating an original story using a generative AI model based on the selection information, a means for presenting options regarding the progress of the story to the user during the story, a means for updating the progress of the story using the generative AI model based on the options selected by the user, a means for displaying update information on the story to the user, a means for selecting an educational theme, and a means for customizing the story based on the educational theme. This makes it possible to dynamically generate and update a story based on an educational theme selected by a user, providing an interactive and highly educational story experience.

[0744] A "character" is someone who appears in a story or is set as the protagonist.

[0745] A "situation" is the scene or condition in which a story or event occurs.

[0746] A "genre" is a category that classifies types and themes of stories.

[0747] "Reader information" refers to data and attribute information about the recipients of the story.

[0748] "Selection information" is data including information on the character, situation, genre, or reader selected by the user.

[0749] A "generative AI model" is an algorithm that uses artificial intelligence to generate data and text.

[0750] "Narrative progression" is the process by which a story's plot or storyline unfolds.

[0751] "Choices" are options that a user can choose from to affect the progression of the story.

[0752] "Updates" are story data that are generated or modified based on user selections.

[0753] "Educational Theme" refers to specific content or concepts related to learning or teaching.

[0754] "Customization" is the act of changing and adjusting a story or system according to the user's choices and needs.

[0755] The system that realizes this application example consists of five main parts: the user interface, server processing, story presentation and choice generation, story progress and update, and story display and feedback. Specifically, it takes the following form.

[0756] User Interface

[0757] The server provides an interface that allows users to access the story creation application using a terminal and select characters, situations, genres, and educational themes. The interface is built as a smartphone or tablet application and is developed using React Native.

[0758] Server Processing

[0759] The device sends the user's selections to the server, where requests are received and analyzed on the server side using Python and Flask. The server uses a generative AI model (OpenAI's ChatGPT) to generate an original story based on the selections. During the generation process, the server constructs prompts that match the characters, situations, and educational themes selected by the user, and inputs them into the AI ​​model.

[0760] Presenting a story and generating options

[0761] The device displays the original story and presents the user with options for progressing through the story. Options are displayed according to the situation. For example, in an ancient Egyptian setting, options include "Learn how to build pyramids" and "Explore Egyptian culture."

[0762] Story progression and updates

[0763] After the user selects an option, the selection information is sent to the server again. The server again uses the ChatGPT API to generate and update the story progression based on the selected option, and the prompt text reflects the user's selection.

[0764] Viewing and Feedback on Stories

[0765] The updated story is sent to the terminal, and the user can view the update information to check the progress of the story. If the user makes the right choice based on the story options, an educational learning effect can be obtained. Regarding the specific learning content, the story is appropriately customized according to the user's feedback.

[0766] For example, if you select an "Ancient Scholar" as the character, a "Medieval Castle" as the setting, and "Mathematics Basics" as the theme for a third-grade history lesson, the generative AI model will receive the following prompt as input and generate a story:

[0767] Input prompt example

[0768] "Generate the following story: Character: Ancient Scholar; Situation: Medieval Castle; Theme: Mathematics Basics. The Ancient Scholar teaches a group of children how to design a castle. Include the following options: a) how to calculate area b) how to measure height c) how much stone to use."

[0769] In this way, a system is realized that dynamically generates and updates stories based on conditions selected by the user, providing educational value.

[0770] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0771] Step 1:

[0772] The terminal provides an interface for users to access the story creation application and select characters, situations, genres, and educational themes. The user interface is built with React Native and receives user selection information. This input includes data such as the characters and situations selected by the user.

[0773] Step 2:

[0774] The device sends the user's selection information in JSON format to the server. The server uses Python and Flask to analyze the received selection information and constructs a prompt for the generative AI model (OpenAI's ChatGPT). The prompt specifically reflects the selection information.

[0775] Step 3:

[0776] The server uses a generative AI model to generate an original story based on the prompt sentence. The AI ​​model generates a series of text data, which becomes the initial part of the story. The prompt sentence is given as input, and the story text is obtained as output.

[0777] Step 4:

[0778] The server sends the generated story to the device, which then displays it to the user, inserting options into the appropriate places in the story so that the user can choose options for the progression of the story. The options are also displayed through an interface built with React Native.

[0779] Step 5:

[0780] The user selects one of the options presented and sends the selection information to the server again via the terminal. The server receives this selection information and again uses the generative AI model to generate the progression of the story based on it. Specifically, it constructs a prompt sentence again and inputs it into the AI ​​model. The prompt sentence reflects the user's selection.

[0781] Step 6:

[0782] The server sends the updated story parts to the terminal, and the terminal displays the updated story information to the user. The user checks the updated story and the story progresses interactively. This allows the story progress to change dynamically according to the user's selections.

[0783] Step 7:

[0784] This process is repeated until the end of the story. Each time the user selects an option at each stage of the story, the server updates the story based on the selection and displays the updated information on the terminal. Here, feedback content based on educational themes is also displayed to enhance the user's learning effect.

[0785] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0786] In this case, the embodiment of the present invention further includes the following elements.

[0787] (Element 1: Incorporating an emotion engine)

[0788] This system is equipped with an emotion engine that can analyze information such as the user's facial expressions and voice and recognize the user's emotions.

[0789] (Element 2: Emotionally-driven story development)

[0790] Based on the user's emotions recognized by the emotion engine, the generative AI can adjust the story development. For example, if the user feels joy in the story, the emotion engine can add a happy development to the story.

[0791] (Element 3: Presenting options based on emotions)

[0792] Based on the user's emotions recognized by the emotion engine, the generative AI will adjust the options: for example, if the user feels sad in the story, the generative AI can present comforting options.

[0793] (Element 4: Emotional Feedback)

[0794] The user's emotions may change as the story progresses or choices are presented. The emotion engine can detect changes in the user's emotions and reflect them in the development of the story or in adjusting choices.

[0795] By combining the above elements with the emotion engine, this system can recognize the user's emotions and adjust the story development and the presentation of options according to the emotions. This allows the user to become more emotionally immersed in the story and experience a more realistic feeling.

[0796] The process flow will be explained below.

[0797] Step 1: When a user accesses the story creation application, the device connects to the server.

[0798] Step 2: An interface for selecting information on the character, situation, genre, and reader is displayed on the terminal. The user selects this information and sends it to the server via the terminal.

[0799] Step 3: The server uses the generative AI to generate part of the story based on the received information. At the same time, the emotion engine recognizes the user's emotions. The generative AI and emotion engine work together to adjust the story development and options according to the user's emotions.

[0800] Step 4: The server sends the generated story and emotion-based options to the device.

[0801] Step 5: The device displays the received story and presents the user with options. At the same time, the emotion engine analyzes the user's emotions and detects changes in the user's emotions.

[0802] Step 6: The user selects one of the options presented and sends it to the server via the terminal. At the same time, the emotion engine detects changes in the user's emotions and sends them to the server.

[0803] Step 7: The server updates the story progression by coordinating the generative AI and emotion engine based on the received selection information and emotion information. The updated story and emotion-based choices are sent to the terminal.

[0804] Step 8: The device displays the received story update information and presents the user with options. At the same time, the emotion engine analyzes the user's emotions and detects changes in the user's emotions. Steps 6 to 7 are repeated.

[0805] This is the process flow of this system. The user selects information, and the device sends it to the server, generating a story, which then displays the story and presents options to the user. The user selects an option, and the device sends it to the server, updating the story, which then again displays the story and presents options to the user. As this process is repeated, the user's emotions are analyzed by the emotion engine and reflected in the development of the story and adjustments to the options.

[0806] Example 2

[0807] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[0808] Conventional interactive story generation systems have the problem that it is difficult to present story development and options that fully reflect the user's emotions. Therefore, there is a demand for a system that can adjust story development and options in real time according to the user's emotions.

[0809] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving selection information including information on a character, a situation, a genre, or a reader selected by a user, a means for generating an original story using a data generation model based on the selection information, a means for presenting options regarding the progress of the story to the user in the middle of the story, a means for updating the progress of the story using the data generation model based on the options selected by the user, and a means for acquiring the user's facial expression and voice and analyzing the user's emotions using an emotion analysis device in the updating means, a means for sending a prompt sentence to a generation AI model based on the user's emotions and generating the next story development, a means for displaying the generated story development and options to the user, and a means for detecting a change in the user's emotions again and continuously adjusting the story progress and options. This makes it possible to present a dynamic and real-time story development and options according to the user's emotions.

[0810] A "user" is an entity that uses this system to read a story and select options.

[0811] A "character" is someone who appears in a story or takes a leading role.

[0812] A "situation" refers to a scene in a story or a scene setting selected by the user.

[0813] "Genre" refers to the type or classification of a story, such as fantasy, suspense, romance, etc.

[0814] "Selected information" refers to information such as the character, situation, genre, etc. selected by the user.

[0815] A "data generation model" is a machine learning model that generates new data based on given input information.

[0816] A "narrative" refers to a textual sequence of events or a scenario that is presented to a user.

[0817] "Choices" refer to multiple options from which a user can choose regarding the progression of a story.

[0818] The "means for updating" is a function for moving the story progression to the next stage in response to a user's selection.

[0819] An "emotion analysis device" is a device or software that analyzes a user's facial expressions and voice data to determine their emotions.

[0820] A "generative AI model" is an artificial intelligence algorithm that generates new text based on a given prompt.

[0821] A "prompt sentence" is a sentence that is given to a generative AI model as instructions when it generates new text.

[0822] "Story development" refers to the progression or progress of a story.

[0823] "Presenting options" is the act of displaying options to the user for what to select next.

[0824] "Changes in the user's emotions" refers to changes in the user's state of mind due to the progression of the story or other factors.

[0825] The embodiment of this invention is an interactive story generation system that can analyze the user's emotions in real time and dynamically generate story development and options based on the analysis. This system uses the following main hardware and software:

[0826] Hardware

[0827] 1. Terminal:

[0828] Devices such as PCs, smartphones, tablets, etc. These devices must have a webcam and microphone.

[0829] 2. Server:

[0830] A high-performance computer that acts as the central nerve center of the entire system.

[0831] software

[0832] 1. Emotion analysis device:

[0833] Microsoft Azure Face API

[0834] Microsoft Azure Speech API

[0835] 2. Generative AI model:

[0836] OpenAI ChatGPT

[0837] Data processing and calculation flow

[0838] The server transmits facial expression and voice data acquired in real time from the user's device to the emotion analysis device, which analyzes the user's emotions. Based on the emotion data obtained as a result of the analysis, the server sends a prompt to the generative AI model to generate the next development of the story. This prompt is based on the user's emotions. For example, if the user is expressing the emotion of "joy," the following prompt is generated:

[0839] So you've detected a situation where the user is smiling. What's the next happy twist to add to the story?

[0840] Based on this prompt, the generative AI model generates the next story development and returns it to the server, which then displays the generated story development on the user's device.

[0841] Examples

[0842] Suppose the user starts reading a story. Suppose the story reaches a scene where a character gets happily married. The emotion analyzer (Microsoft Azure Face API) recognizes the user's facial expression as "happiness" and responds to the server with "emotion":"happiness". In response, the server generates the following prompt:

[0843] So you've detected a situation where the user is smiling. What's the next happy twist to add to the story?

[0844] The generative AI model (OpenAI ChatGPT) generates text such as "The character won the lottery" and returns it to the server. The server displays this to the user. In this way, the story unfolds dynamically according to the user's emotions.

[0845] This system allows users to become deeply immersed in the story, creating a more realistic experience. By continuously analyzing the user's emotions and adjusting the story development and choices based on the results, the system provides a story that is in tune with the user's interests and emotions.

[0846] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0847] Step 1: Initialize the system

[0848] 1. The server starts and loads the API credentials and models.

[0849] 2. The user opens a web browser on the terminal and accesses the system's URL.

[0850] 3. The server provides the user interface and the user is presented with a login screen.

[0851] Input: None

[0852] Output: The user interface is displayed in the terminal.

[0853] Specific behavior:

[0854] The server loads the interface module and generates the HTML and JavaScript to display the login screen.

[0855] Step 2: Recognizing User Emotions

[0856] 1. After logging in, the user allows access to the device's webcam and microphone.

[0857] 2. The device uses a camera and microphone to capture the user's facial expressions and voice in real time.

[0858] 3. The server sends the acquired image data to Microsoft Azure's Face API and analyzes emotions from facial expressions.

[0859] 4. The voice data is sent to Microsoft Azure's Speech API and emotion is analyzed from the voice tone.

[0860] Input: User login information, data from camera and microphone

[0861] Output: Emotion data based on facial expressions, emotion data based on voice tones

[0862] Specific behavior:

[0863] The server sends image data to the Face API and receives emotion tags such as "happiness" or "sadness." Similarly, it sends voice data to the Speech API and receives emotion evaluation results.

[0864] Step 3: Creating and developing a story

[0865] 1. The server determines the user's current emotion based on the emotion analysis results.

[0866] 2. The server generates a prompt based on the emotion (e.g., "Since the user is feeling happy, what happy development should we add next?").

[0867] 3. Send the prompt to the generative AI model to get the next story development.

[0868] 4. Display the generated narrative text to the user.

[0869] Input: Sentiment analysis results

[0870] Output: Story development text

[0871] Specific behavior:

[0872] The server processes the analysis results in the analysis module and generates a new prompt sentence. The generative AI model generates a new story paragraph based on this prompt and returns it. The server displays this text in the user interface.

[0873] Step 4: Present options based on emotions

[0874] 1. The server continuously monitors the user's emotional data.

[0875] 2. As the user reads the story, generate prompts that offer options related to the next development (e.g., "If the user is sad, how would you present a comforting scenario?").

[0876] 3. The prompt is sent to a generative AI model to generate appropriate options.

[0877] 4. Display the generated options to the user and let them select.

[0878] Input: Real-time emotion data

[0879] Output: The text of the choice

[0880] Specific behavior:

[0881] The server receives emotion data in real time and generates a prompt based on it. The generative AI model generates the next options based on this prompt and returns them. The server displays these options in the user interface.

[0882] Step 5: Emotional feedback and regulation

[0883] 1. As the story progresses, the server again collects the user's facial expressions and voice and performs emotion analysis.

[0884] 2. Based on the user's new emotional data, the server again sends the next story development and choices to the generative AI model for adjustment.

[0885] 3. Present the user again with the adjusted storyline and options.

[0886] Input: Recaptured facial expression data and voice data

[0887] Output: Newly adjusted story development and choices

[0888] Specific behavior:

[0889] The server regenerates the prompt based on the new analysis and sends it to the generative AI model, which generates new text and sends it back to the user, who sees the updated story.

[0890] By repeating the above steps, the system dynamically generates a story and provides options according to the user's emotions.

[0891] (Application example 2)

[0892] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[0893] Conventional interactive story generation systems did not consider story development that reflected the user's emotions. As a result, it was difficult for the user to emotionally empathize with the content of the story, resulting in a lack of immersion. In response to this issue, the present invention aims to provide a deeper immersive and interactive reading experience by reflecting the user's emotions in real time and adjusting the story development and the presentation of options based on the user's emotions.

[0894] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0895] In this invention, the server includes: a means for receiving selection information including information on a character, a situation, a genre, and a reader selected by a user; A means for generating an original story using a prompt sentence that instructs the user to generate an original story based on the selected information and a generation AI; means for identifying the user's emotion using a camera and an emotion recognition model; means for presenting the user with a plurality of options regarding the progression of the story during the course of the story; A prompt sentence instructing to generate a continuation of the story based on any one of the options selected by the user and the identified user emotion, and a means for generating a continuation of the story using the generation AI; means for displaying the generated story to the user; a means for reading out the content of the story by a voice synthesis engine; Including, The process of presenting the options, generating the continuation of the story, and displaying and reading it to the user is repeated. This allows the story to develop in response to the user's emotions.

[0896] "User" refers to an individual who uses the system to interactively experience a story.

[0897] A "character" is a character or an animal or other entity that appears in a story.

[0898] "Situation" refers to the setting of the story, such as the location, circumstances, and background environment.

[0899] "Genre" refers to the classification or category to which a story belongs, such as fantasy, science fiction, or mystery.

[0900] "Reader" refers to the person who receives the story, i.e. the user or a third party.

[0901] "Selection information" refers to data including information about the character, situation, genre, or reader selected by the user.

[0902] "Generative AI" refers to algorithms or computer programs that use natural language processing to generate original stories based on given input information.

[0903] An "emotion recognition model" refers to an algorithm or computer program that analyzes a user's facial, voice, and other characteristics to identify their emotions.

[0904] "Story development" refers to the progression of the story or changes in the storyline, and is adjusted based on the user's selections and emotions.

[0905] "Choices" are multiple selectable options presented to the user during the course of the story.

[0906] The "updating means" refers to a method or device for dynamically updating the progress of the story based on the user's selection and emotion recognition results.

[0907] "Means for displaying" refers to a display device or interface that visually presents the progress of the story and selection information to the user.

[0908] A "speech synthesis engine" refers to an algorithm or software that converts text data into voice data and reads it aloud.

[0909] "Reading aloud" refers to the act of providing a story or options generated using a speech synthesis engine to the user as audio.

[0910] The present invention provides a system for accepting selection information including information on a character, a situation, a genre, or a reader selected by a user. Next, the configuration and operation of this system will be specifically described.

[0911] System Configuration

[0912] 1. User device:

[0913] Camera (to capture the user's face)

[0914] Display (to show the story and options)

[0915] A microphone (to receive audio input)

[0916] Speaker (to read the story aloud)

[0917] 2. Server:

[0918] Data generation model (algorithm for generating original stories)

[0919] Emotion recognition models (algorithms for identifying user emotions)

[0920] A natural language processing engine (to process the generated narrative in natural language)

[0921] A speech synthesis engine (to convert the generated text into audio data and read it aloud)

[0922] Program Processing

[0923] 1. Acceptance of Selection Information:

[0924] The user terminal transmits information about the character, situation, genre, or reader to the server. This selection information is used as setting information that forms the basis of the story.

[0925] 2. Narrative Generation:

[0926] The data generation model on the server generates a story using a prompt sentence that instructs the data generation model to generate an original story based on the selected information received, and the data generation model (generative AI). This data generation model utilizes a natural language processing engine to dynamically create an original story that the user does not know.

[0927] 3. Emotion recognition:

[0928] The user's face is captured by the camera on the user's device, and the emotion recognition model analyzes the facial expressions to determine the current emotion of the user.

[0929] 4. Story Development Adjustment:

[0930] The server adjusts the storyline when generating the next storyline based on the emotion identified by the emotion recognition model, for example, if the user is showing happy emotion, the storyline is set to take a happy turn.

[0931] 5. Present your options:

[0932] During the story, the server presents the user with multiple options regarding the next development of the story. The user selects one of the options to determine the progress of the story.

[0933] 6. Story Progression and Updates:

[0934] The server generates the continuation of the story using a prompt sentence that instructs the server to generate a continuation of the story based on any one of the options selected by the user and the identified user emotion, and the generation AI. The generated continuation of the story is displayed on the user terminal and is read aloud by a voice synthesis engine.

[0935] Examples

[0936] Suppose a user launches an application, selects the fantasy genre, chooses a "hero" as the character (protagonist), a "village" as the situation, and a "teenage male" as the reader. As the user reads the story, the camera captures the user's happy facial expression, and the emotion recognition model identifies the emotion "joy." In this case, the story develops in a happy way, with the hero defeating the dragon and being thanked by the villagers. Also, in the middle of the story, the options of "saving the village" and "searching for treasure" are presented, and if the user selects "searching for treasure," the story will progress based on that choice.

[0937] An example of an initial prompt for generating a story: "Please generate a story based on the genre "Fantasy", the character (protagonist) "Hero", and the situation "Village". The reader is a "teenage boy". Please generate a story that suits this reader."

[0938] An example of an initial prompt for generating a continuation of a story: "Generate a continuation of the story based on the option "Treasure Search." The reader's emotion is "Joy." Generate a continuation of the story that matches this reader's emotion."

[0939] As described above, the present invention makes it possible to reflect a user's emotions in real time and provide an interactive storytelling experience.

[0940] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0941] Step 1:

[0942] User input of selection information

[0943] The user selects information about the characters, situation, genre, and reader through the application on the terminal. This selection information is sent from the terminal to the server. The input data is received as a string of characters, which becomes the basis for the next story generation.

[0944] Step 2:

[0945] Beginning of story generation

[0946] The server uses a data generation model to generate an original story based on the selection information received in step 1. Specifically, it uses a natural language processing engine to output a story that matches the selection information as text data.

[0947] Step 3:

[0948] User Emotion Recognition

[0949] While the user reads the story, the device's camera captures the user's face. The facial image is sent to the server in real time, and the emotion recognition model analyzes the image. As a result of the analysis, the user's emotion is identified as "happy" or "sad," etc., and the emotion data is output.

[0950] Step 4:

[0951] Adjusting story development based on emotions

[0952] The server adjusts the development of the story by including the emotion data obtained in step 3 in the prompt sentence when generating the rest of the story. For example, if the user shows happy emotion, the server changes the development of the story by including a sentence such as "The reader is feeling happy" in the prompt sentence when generating the rest of the story.

[0953] Step 5:

[0954] Present options for story progression

[0955] During the story, the server presents the user with multiple options for the next development. For example, the server displays the question "Where should the hero go next?" along with options such as "Climb the mountain" or "Cross the river." The user selects an option from the options.

[0956] Step 6:

[0957] Choice-based story updates

[0958] The server receives the choice selected by the user and updates the story progression based on this. The data generation model is then used again to generate the next part of the story based on the choice. For example, if the user selects "Cross the river," a sentence such as "The hero mustered up his courage and decided to cross the river" is generated.

[0959] Step 7:

[0960] View and read stories

[0961] The updated story is displayed on the device's display and is read aloud by a speech synthesis engine. The speech synthesis engine converts the generated text data into speech data and reads it through the speaker. The input is the updated text data, and the output is speech data.

[0962] By repeating the above steps, an interactive storytelling experience based on the user's emotions and choices is realized.

[0963] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0964] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0965] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[0966] [Fourth embodiment]

[0967] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0968] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0969] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a “computer” according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32.

[0970] The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0971] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0972] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0973] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0974] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0975] The control target 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, legs, etc. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, legs, etc. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0976] Fig. 8 shows an example of main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0977] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0978] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0979] In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0980] Next, a description will be given of the specific processing by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[0981] An embodiment for implementing the present invention includes the following elements.

[0982] (Element 1: User Interface)

[0983] A user accesses the story creation application via a terminal. An interface for selecting information on characters, situations, genres, and readers is displayed on the terminal. The user selects this information and transmits the selected information to the server via the terminal.

[0984] (Element 2: Server processing)

[0985] The server uses a generative AI to generate a story based on the received selection information. The generative AI uses natural language processing technology to generate a part of the original story based on the user's selection. The server transmits the generated story to the terminal.

[0986] (Element 3: Story presentation and options)

[0987] The device displays the received story and presents the user with options generated by generative AI during the story. The user selects one option from the options presented and sends it to the server via the device.

[0988] (Element 4: Story progression and updating)

[0989] The server updates the story progression using a generative AI based on the user's selection. The generative AI generates a new part of the story according to the selection, and the server transmits the updated story to the device.

[0990] (Element 5: Narrative presentation and feedback)

[0991] The device displays the received story updates, allowing the user to see the results of their choices. Users get feedback on the progress of the story and can freely customize the story development.

[0992] The process of presenting options, updating the progress of the story, and displaying to the user is repeated until the story is completed.

[0993] With these elements, users can create original stories based on their own choices through a story creation application.

[0994] For example, if a user selects a "brave adventurer" as the main character, "medieval Europe" as the setting of the story, and "adventure" as the genre, the generative AI will generate a story based on those selections. In the middle of the story, the user is presented with options such as "enter the cave," "explore the forest," and "talk to the villagers." If the user selects "enter the cave," the story will continue with an adventure in the cave. On the other hand, if the user selects "explore the forest," the story will move on to an adventure in the forest.

[0995] The process flow will be explained below.

[0996] Step 1: When a user accesses the story creation application, the device connects to the server.

[0997] Step 2: An interface is displayed on the terminal to select information about the character, situation, genre, and reader. The user selects this information and sends it to the server via the terminal.

[0998] Step 3: The server uses a generative AI to generate a story based on the received information. The generative AI uses natural language processing techniques to generate parts of an original story based on the user's selections.

[0999] Step 4: The server sends the generated story to the terminal.

[1000] Step 5: The device displays the received story and presents options to the user during the story.

[1001] Step 6: The user selects one of the options presented and sends it to the server via the terminal.

[1002] Step 7: The server updates the story progression using the generative AI based on the received selection information. The generative AI generates a new part of the story according to the selection, and the server sends the updated story to the device.

[1003] Step 8: The device displays the received story update. Present the user with options again during the story. Repeat steps 6 to 7.

[1004] This is the process flow of this system. The user selects information, the terminal sends it to the server, which generates a story, and the terminal displays the story and presents options to the user. The user selects an option, the terminal sends it to the server, which updates the story, and the terminal again displays the story and presents options to the user. As this process is repeated, the user can progress through the story based on their own choices.

[1005] Example 1

[1006] Next, a description will be given of Example 1. In the following description, the data processing device 12 is referred to as a "server" and the robot 414 is referred to as a "terminal."

[1007] In conventional story creation systems, it was difficult for users to freely create a story based on their own choices. In particular, there was a problem that the interactivity experienced by the user was reduced because the progress and updates of the story according to the user's choices were done manually. In addition, the created stories sometimes lacked consistency, which reduced user satisfaction. It is necessary to provide a story creation system that can solve these problems and is more interactive and flexibly respond to user choices.

[1008] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1009] In this invention, the server includes a means for receiving selection information including information on a character, a situation, a genre, or a reader selected by a user, a means for generating an original story using a generative AI model, a means for presenting options regarding the progress of the story to the user in the middle of the story, a means for updating the progress of the story using the generative AI model based on the options selected by the user, and a means for displaying updated information on the story to the user. This allows the user to enjoy a story that is dynamically generated based on their own selection.

[1010] A "user" is an entity that uses the system to interactively create a story.

[1011] "Selected information" refers to information including the character, situation, genre, and reader selected by the user.

[1012] A "generative AI model" is an artificial intelligence technology that generates original stories based on user-selected information.

[1013] "Story progression" refers to the process and content of how the story unfolds.

[1014] "Choices" are multiple, selectable options for story progression presented to the user.

[1015] An "emotion analysis engine" is a technology that recognizes the user's emotions and adjusts the story progression based on those emotions.

[1016] "Natural language processing" is a language processing technique used when a generative AI model generates sentences using prompts.

[1017] "Updated information" refers to the content of the story that has been updated based on user selections and sentiment analysis.

[1018] The present invention relates to a system that allows users to interactively create stories, which consists of a user interface, a server process, a story display, and updating of the story progress based on user selections.

[1019] User Interface

[1020] A user accesses a story creation application using a terminal such as a smartphone or a PC. The application displays an interface for selecting information on characters, situations, genres, and readers. The user selects this information and transmits the selected information to the server via the terminal.

[1021] Server Processing

[1022] The server receives the selection information selected by the user. The server is built using a programming language such as Python or Java, and inputs the selection information into a generative AI model (e.g., ChatGPT). The generative AI model uses natural language processing technology to generate an original story based on the user's selection. The generated story is sent from the server to the terminal.

[1023] Presenting the story and giving options

[1024] The device displays the received story on the screen. As the story progresses, the user is presented with options to choose from during the story generated by the generative AI model. For example, options such as "Enter the cave," "Explore the forest," and "Talk to the villagers" are displayed. The user selects one of the options presented and sends the selection information to the server via the device.

[1025] Story progression and updates

[1026] The server receives the user's choice and generates the rest of the story using a generative AI model. The generative AI model generates the story as a prompt in the following format:

[1027] The main character is a brave adventurer, and you must create an adventure story set in medieval Europe. The actions the main character can choose in the next scene are to "enter the cave," "explore the forest," or "talk to the villagers." Generate a sequel for each option.

[1028] The new part of the story that is generated is again sent from the server to the terminal.

[1029] View story updates and give feedback

[1030] The device displays the updated story on the screen and provides feedback to the user, who can review the results of their choices and enjoy the progression of the story. This process of presenting choices and updating the story is repeated until the story is completed.

[1031] The present invention allows users to enjoy a story that is interactively generated based on their own selections, resulting in consistent stories and improved user satisfaction.

[1032] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1033] Step 1:

[1034] A user accesses a story creation application using a device such as a smartphone or a PC. The device displays a user interface and presents options for selecting information on characters, situations, genres, and readers. The user selects this information. The input is the information selected by the user, and the output is the state in which the selected information is complete.

[1035] Step 2:

[1036] The terminal receives the user's selection information and sends it to the server through an HTTP request. The input is the user's selection information, and the output is the selection information being sent to the server. This passes the selection information to the server.

[1037] Step 3:

[1038] The server sends a prompt to the generative AI model based on the received selection information. The input is the user's selection information, and the output is a prompt to the generative AI model. The server uses Python to generate a prompt like this:

[1039] "Create an adventure story for your hero, a brave adventurer, set in medieval Europe."

[1040] Step 4:

[1041] The generative AI model analyzes the prompt and generates an original part of the story based on the user's specifications. The input is the prompt, and the output is the generated story text. For example, the story generated by the generative AI model is "A brave adventurer arrives in a medieval village and sets out on an unknown adventure."

[1042] Step 5:

[1043] The server sends the generated story to the terminal. The input is the generated story text and the output is the message sent to the terminal. The server converts the story text to JSON format and sends it as an HTTP response.

[1044] Step 6:

[1045] The terminal parses the received story and displays it on the screen. The input is the story text received from the server and the output is the display to the user interface. Once the story is displayed, the user can read it.

[1046] Step 7:

[1047] During the story, the device presents the user with options generated by the generative AI model. The input is part of the generated story, and the output is the options presented to the user. For example, options such as "Enter the cave," "Explore the forest," and "Talk to the villagers" are displayed.

[1048] Step 8:

[1049] The user selects one of the options presented, and the device again sends the selection information to the server. The input is the user's selection information, and the output is the transmission to the server. Once the selection information is passed to the server, the server again uses the generative AI model to generate the next part of the story.

[1050] Step 9:

[1051] The server updates the story progression using a generative AI model based on the user's choices. The input is the user's selection information, and the output is the continuation of the generated story. If the user selects "Enter the cave," the generative AI model generates a story such as "The brave adventurer stepped into the dark cave."

[1052] Step 10:

[1053] The server sends the updated story to the terminal. The input is the generated story continuation, and the output is the message sent to the terminal. The server converts the story text to JSON format and sends it as an HTTP response.

[1054] Step 11:

[1055] The device displays the updated story on the screen and provides feedback to the user. The input is the updated story text received from the server, and the output is the display on the user interface. The user can confirm the results of their selection and enjoy the story progression again. This process of presenting options and updating the story is repeated until the story is completed.

[1056] (Application example 1)

[1057] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1058] Current interactive storytelling systems have difficulty incorporating educational content effectively, and users have limited options to maximize their learning. In addition, there are few examples of systems that dynamically customize the story progression based on user choices to provide educational value.

[1059] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1060] In this invention, the server includes a means for receiving selection information including information on a character, a situation, a genre, or a reader selected by a user, a means for generating an original story using a generative AI model based on the selection information, a means for presenting options regarding the progress of the story to the user during the story, a means for updating the progress of the story using the generative AI model based on the options selected by the user, a means for displaying update information on the story to the user, a means for selecting an educational theme, and a means for customizing the story based on the educational theme. This makes it possible to dynamically generate and update a story based on an educational theme selected by a user, providing an interactive and highly educational story experience.

[1061] A "character" is someone who appears in a story or is set as the protagonist.

[1062] A "situation" is the scene or condition in which a story or event occurs.

[1063] A "genre" is a category that classifies types and themes of stories.

[1064] "Reader information" refers to data and attribute information about the recipients of the story.

[1065] "Selection information" is data including information on the character, situation, genre, or reader selected by the user.

[1066] A "generative AI model" is an algorithm that uses artificial intelligence to generate data and text.

[1067] "Narrative progression" is the process by which a story's plot or storyline unfolds.

[1068] "Choices" are options that a user can choose from to affect the progression of the story.

[1069] "Updates" are story data that are generated or modified based on user selections.

[1070] "Educational Theme" refers to specific content or concepts related to learning or teaching.

[1071] "Customization" is the act of changing and adjusting a story or system according to the user's choices and needs.

[1072] The system that realizes this application example consists of five main parts: the user interface, server processing, story presentation and choice generation, story progress and update, and story display and feedback. Specifically, it takes the following form.

[1073] User Interface

[1074] The server provides an interface that allows users to access the story creation application using a terminal and select characters, situations, genres, and educational themes. The interface is built as a smartphone or tablet application and is developed using React Native.

[1075] Server Processing

[1076] The device sends the user's selections to the server, where requests are received and analyzed on the server side using Python and Flask. The server uses a generative AI model (OpenAI's ChatGPT) to generate an original story based on the selections. During the generation process, the server constructs prompts that match the characters, situations, and educational themes selected by the user, and inputs them into the AI ​​model.

[1077] Presenting a story and generating options

[1078] The device displays the original story and presents the user with options for progressing through the story. Options are displayed according to the situation. For example, in an ancient Egyptian setting, options include "Learn how to build pyramids" and "Explore Egyptian culture."

[1079] Story progression and updates

[1080] After the user selects an option, the selection information is sent to the server again. The server again uses the ChatGPT API to generate and update the story progression based on the selected option, and the prompt text reflects the user's selection.

[1081] Viewing and Feedback on Stories

[1082] The updated story is sent to the terminal, and the user can view the update information to check the progress of the story. If the user makes the right choice based on the story options, an educational learning effect can be obtained. Regarding the specific learning content, the story is appropriately customized according to the user's feedback.

[1083] For example, if you select an "Ancient Scholar" as the character, a "Medieval Castle" as the setting, and "Mathematics Basics" as the theme for a third-grade history lesson, the generative AI model will receive the following prompt as input and generate a story:

[1084] Input prompt example

[1085] "Generate the following story: Character: Ancient Scholar; Situation: Medieval Castle; Theme: Mathematics Basics. The Ancient Scholar teaches a group of children how to design a castle. Include the following options: a) how to calculate area b) how to measure height c) how much stone to use."

[1086] In this way, a system is realized that dynamically generates and updates stories based on conditions selected by the user, providing educational value.

[1087] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1088] Step 1:

[1089] The terminal provides an interface for users to access the story creation application and select characters, situations, genres, and educational themes. The user interface is built with React Native and receives user selection information. This input includes data such as the characters and situations selected by the user.

[1090] Step 2:

[1091] The device sends the user's selection information in JSON format to the server. The server uses Python and Flask to analyze the received selection information and constructs a prompt for the generative AI model (OpenAI's ChatGPT). The prompt specifically reflects the selection information.

[1092] Step 3:

[1093] The server uses a generative AI model to generate an original story based on the prompt sentence. The AI ​​model generates a series of text data, which becomes the initial part of the story. The prompt sentence is given as input, and the story text is obtained as output.

[1094] Step 4:

[1095] The server sends the generated story to the device, which then displays it to the user, inserting options into the appropriate places in the story so that the user can choose options for the progression of the story. The options are also displayed through an interface built with React Native.

[1096] Step 5:

[1097] The user selects one of the options presented and sends the selection information to the server again via the terminal. The server receives this selection information and again uses the generative AI model to generate the progression of the story based on it. Specifically, it constructs a prompt sentence again and inputs it into the AI ​​model. The prompt sentence reflects the user's selection.

[1098] Step 6:

[1099] The server sends the updated story parts to the terminal, and the terminal displays the updated story information to the user. The user checks the updated story and the story progresses interactively. This allows the story progress to change dynamically according to the user's selections.

[1100] Step 7:

[1101] This process is repeated until the end of the story. Each time the user selects an option at each stage of the story, the server updates the story based on the selection and displays the updated information on the terminal. Here, feedback content based on educational themes is also displayed to enhance the user's learning effect.

[1102] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[1103] In this case, the embodiment of the present invention further includes the following elements.

[1104] (Element 1: Incorporating an emotion engine)

[1105] This system is equipped with an emotion engine that can analyze information such as the user's facial expressions and voice and recognize the user's emotions.

[1106] (Element 2: Emotionally-driven story development)

[1107] Based on the user's emotions recognized by the emotion engine, the generative AI can adjust the story development. For example, if the user feels joy in the story, the emotion engine can add a happy development to the story.

[1108] (Element 3: Presenting options based on emotions)

[1109] Based on the user's emotions recognized by the emotion engine, the generative AI will adjust the options: for example, if the user feels sadness in the story, the generative AI can present comforting options.

[1110] (Element 4: Emotional Feedback)

[1111] The user's emotions may change as the story progresses or choices are presented. The emotion engine can detect changes in the user's emotions and reflect them in the development of the story or in adjusting choices.

[1112] By combining the above elements with the emotion engine, this system can recognize the user's emotions and adjust the story development and the presentation of options according to the emotions. This allows the user to become more emotionally immersed in the story and experience a more realistic feeling.

[1113] The process flow will be explained below.

[1114] Step 1: When a user accesses the story creation application, the device connects to the server.

[1115] Step 2: An interface for selecting information on the character, situation, genre, and reader is displayed on the terminal. The user selects this information and sends it to the server via the terminal.

[1116] Step 3: The server uses the generative AI to generate part of the story based on the received information. At the same time, the emotion engine recognizes the user's emotions. The generative AI and emotion engine work together to adjust the story development and options according to the user's emotions.

[1117] Step 4: The server sends the generated story and emotion-based options to the device.

[1118] Step 5: The device displays the received story and presents the user with options. At the same time, the emotion engine analyzes the user's emotions and detects changes in the user's emotions.

[1119] Step 6: The user selects one of the options presented and sends it to the server via the terminal. At the same time, the emotion engine detects changes in the user's emotions and sends them to the server.

[1120] Step 7: The server updates the story progression by coordinating the generative AI and emotion engine based on the received selection information and emotion information. The updated story and emotion-based choices are sent to the terminal.

[1121] Step 8: The device displays the received story update information and presents the user with options. At the same time, the emotion engine analyzes the user's emotions and detects changes in the user's emotions. Steps 6 to 7 are repeated.

[1122] This is the process flow of this system. The user selects information, and the device sends it to the server, generating a story, which then displays the story and presents options to the user. The user selects an option, and the device sends it to the server, updating the story, which then again displays the story and presents options to the user. As this process is repeated, the user's emotions are analyzed by the emotion engine and reflected in the development of the story and adjustments to the options.

[1123] Example 2

[1124] Next, a description will be given of Example 2. In the following description, the data processing device 12 is referred to as a "server" and the robot 414 is referred to as a "terminal."

[1125] Conventional interactive story generation systems have the problem that it is difficult to present story development and options that fully reflect the user's emotions. Therefore, there is a demand for a system that can adjust story development and options in real time according to the user's emotions.

[1126] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving selection information including information on a character, a situation, a genre, or a reader selected by a user, a means for generating an original story using a data generation model based on the selection information, a means for presenting options regarding the progress of the story to the user in the middle of the story, a means for updating the progress of the story using the data generation model based on the options selected by the user, and a means for acquiring the user's facial expression and voice and analyzing the user's emotions using an emotion analysis device in the updating means, a means for sending a prompt sentence to a generation AI model based on the user's emotions and generating the next story development, a means for displaying the generated story development and options to the user, and a means for detecting a change in the user's emotions again and continuously adjusting the story progress and options. This makes it possible to present a dynamic and real-time story development and options according to the user's emotions.

[1127] A "user" is an entity that uses this system to read a story and select options.

[1128] A "character" is someone who appears in a story or takes a leading role.

[1129] A "situation" refers to a scene in a story or a scene setting selected by the user.

[1130] "Genre" refers to the type or classification of a story, such as fantasy, suspense, romance, etc.

[1131] "Selected information" refers to information such as the character, situation, genre, etc. selected by the user.

[1132] A "data generation model" is a machine learning model that generates new data based on given input information.

[1133] A "narrative" refers to a textual sequence of events or a scenario that is presented to a user.

[1134] "Choices" refer to multiple options from which a user can choose regarding the progression of a story.

[1135] The "means for updating" is a function for moving the story progression to the next stage in response to a user's selection.

[1136] An "emotion analysis device" is a device or software that analyzes a user's facial expressions and voice data to determine their emotions.

[1137] A "generative AI model" is an artificial intelligence algorithm that generates new text based on a given prompt.

[1138] A "prompt sentence" is a sentence that is given to a generative AI model as instructions when it generates new text.

[1139] "Story development" refers to the progression or progress of a story.

[1140] "Presenting options" is the act of displaying options to the user for what to select next.

[1141] "Changes in the user's emotions" refers to changes in the user's state of mind due to the progression of the story or other factors.

[1142] The embodiment of this invention is an interactive story generation system that can analyze the user's emotions in real time and dynamically generate story development and options based on the analysis. This system uses the following main hardware and software:

[1143] Hardware

[1144] 1. Terminal:

[1145] Devices such as PCs, smartphones, tablets, etc. These devices must have a webcam and microphone.

[1146] 2. Server:

[1147] A high-performance computer that acts as the central nerve center of the entire system.

[1148] software

[1149] 1. Emotion analysis device:

[1150] Microsoft Azure Face API

[1151] Microsoft Azure Speech API

[1152] 2. Generative AI model:

[1153] OpenAI ChatGPT

[1154] Data processing and calculation flow

[1155] The server transmits facial expression and voice data acquired in real time from the user's device to the emotion analysis device, which analyzes the user's emotions. Based on the emotion data obtained as a result of the analysis, the server sends a prompt to the generative AI model to generate the next development of the story. This prompt is based on the user's emotions. For example, if the user is expressing the emotion of "joy," the following prompt is generated:

[1156] So you've detected a situation where the user is smiling. What's the next happy twist to add to the story?

[1157] Based on this prompt, the generative AI model generates the next story development and returns it to the server, which then displays the generated story development on the user's device.

[1158] Examples

[1159] Suppose the user starts reading a story. Suppose the story reaches a scene where a character gets happily married. The emotion analyzer (Microsoft Azure Face API) recognizes the user's facial expression as "happiness" and responds to the server with "emotion":"happiness". In response, the server generates the following prompt:

[1160] So you've detected a situation where the user is smiling. What's the next happy twist to add to the story?

[1161] The generative AI model (OpenAI ChatGPT) generates text such as "The character won the lottery" and returns it to the server. The server displays this to the user. In this way, the story unfolds dynamically according to the user's emotions.

[1162] This system allows users to become deeply immersed in the story, creating a more realistic experience. By continuously analyzing the user's emotions and adjusting the story development and choices based on the results, the system provides a story that is in tune with the user's interests and emotions.

[1163] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1164] Step 1: Initialize the system

[1165] 1. The server starts and loads the API credentials and models.

[1166] 2. The user opens a web browser on the terminal and accesses the system's URL.

[1167] 3. The server provides the user interface and the user is presented with a login screen.

[1168] Input: None

[1169] Output: The user interface is displayed in the terminal.

[1170] Specific behavior:

[1171] The server loads the interface module and generates the HTML and JavaScript to display the login screen.

[1172] Step 2: Recognizing User Emotions

[1173] 1. After logging in, the user allows access to the device's webcam and microphone.

[1174] 2. The device uses a camera and microphone to capture the user's facial expressions and voice in real time.

[1175] 3. The server sends the acquired image data to Microsoft Azure's Face API and analyzes emotions from facial expressions.

[1176] 4. The voice data is sent to Microsoft Azure's Speech API and emotion is analyzed from the voice tone.

[1177] Input: User login information, data from camera and microphone

[1178] Output: Emotion data based on facial expressions, emotion data based on voice tones

[1179] Specific behavior:

[1180] The server sends image data to the Face API and receives emotion tags such as "happiness" or "sadness." Similarly, it sends voice data to the Speech API and receives emotion evaluation results.

[1181] Step 3: Creating and developing a story

[1182] 1. The server determines the user's current emotion based on the emotion analysis results.

[1183] 2. The server generates a prompt based on the emotion (e.g., "Since the user is feeling happy, what happy development should we add next?").

[1184] 3. Send the prompt to the generative AI model to get the next story development.

[1185] 4. Display the generated narrative text to the user.

[1186] Input: Sentiment analysis results

[1187] Output: Story development text

[1188] Specific behavior:

[1189] The server processes the analysis results in the analysis module and generates a new prompt sentence. The generative AI model generates a new story paragraph based on this prompt and returns it. The server displays this text in the user interface.

[1190] Step 4: Present options based on emotions

[1191] 1. The server continuously monitors the user's emotional data.

[1192] 2. As the user reads the story, generate prompts that offer options related to the next development (e.g., "If the user is sad, how would you present a comforting scenario?").

[1193] 3. The prompt is sent to a generative AI model to generate appropriate options.

[1194] 4. Display the generated options to the user and let them select.

[1195] Input: Real-time emotion data

[1196] Output: The text of the choice

[1197] Specific behavior:

[1198] The server receives emotion data in real time and generates a prompt based on it. The generative AI model generates the next options based on this prompt and returns them. The server displays these options in the user interface.

[1199] Step 5: Emotional feedback and regulation

[1200] 1. As the story progresses, the server again collects the user's facial expressions and voice and performs emotion analysis.

[1201] 2. Based on the user's new emotional data, the server again sends the next story development and choices to the generative AI model for adjustment.

[1202] 3. Present the user again with the adjusted storyline and options.

[1203] Input: Recaptured facial expression data and voice data

[1204] Output: Newly adjusted story development and choices

[1205] Specific behavior:

[1206] The server regenerates the prompt based on the new analysis and sends it to the generative AI model, which generates new text and sends it back to the user, who sees the updated story.

[1207] By repeating the above steps, the system dynamically generates a story and provides options according to the user's emotions.

[1208] (Application example 2)

[1209] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1210] Conventional interactive story generation systems did not consider story development that reflected the user's emotions. As a result, it was difficult for the user to emotionally empathize with the content of the story, resulting in a lack of immersion. In response to this issue, the present invention aims to provide a deeper immersive and interactive reading experience by reflecting the user's emotions in real time and adjusting the story development and the presentation of options based on the user's emotions.

[1211] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1212] In this invention, the server includes: a means for receiving selection information including information on a character, a situation, a genre, and a reader selected by a user; A means for generating an original story using a prompt sentence that instructs the user to generate an original story based on the selected information and a generation AI; means for identifying the user's emotion using a camera and an emotion recognition model; means for presenting the user with a plurality of options regarding the progression of the story during the course of the story; A prompt sentence instructing to generate a continuation of the story based on any one of the options selected by the user and the identified user emotion, and a means for generating a continuation of the story using the generation AI; means for displaying the generated story to the user; a means for reading out the content of the story by a voice synthesis engine; Including, The process of presenting the options, generating the continuation of the story, and displaying and reading it to the user is repeated. This allows the story to develop in response to the user's emotions.

[1213] "User" refers to an individual who uses the system to interactively experience a story.

[1214] A "character" is a character or an animal or other entity that appears in a story.

[1215] "Situation" refers to the setting of the story, such as the location, circumstances, and background environment.

[1216] "Genre" refers to the classification or category to which a story belongs, such as fantasy, science fiction, or mystery.

[1217] "Reader" refers to the person who receives the story, i.e. the user or a third party.

[1218] "Selection information" refers to data including information about the character, situation, genre, or reader selected by the user.

[1219] "Generative AI" refers to algorithms or computer programs that use natural language processing to generate original stories based on given input information.

[1220] An "emotion recognition model" refers to an algorithm or computer program that analyzes a user's facial, voice, and other characteristics to identify their emotions.

[1221] "Story development" refers to the progression of the story or changes in the storyline, and is adjusted based on the user's selections and emotions.

[1222] "Choices" are multiple selectable options presented to the user during the course of the story.

[1223] The "updating means" refers to a method or device for dynamically updating the progress of the story based on the user's selection and emotion recognition results.

[1224] "Means for displaying" refers to a display device or interface that visually presents the progress of the story and selection information to the user.

[1225] A "speech synthesis engine" refers to an algorithm or software that converts text data into voice data and reads it aloud.

[1226] "Reading aloud" refers to the act of providing a story or options generated using a speech synthesis engine to the user as audio.

[1227] The present invention provides a system for accepting selection information including information on a character, a situation, a genre, or a reader selected by a user. Next, the configuration and operation of this system will be specifically described.

[1228] System Configuration

[1229] 1. User device:

[1230] Camera (to capture the user's face)

[1231] Display (to show the story and options)

[1232] A microphone (to receive audio input)

[1233] Speaker (to read the story aloud)

[1234] 2. Server:

[1235] Data generation model (algorithm for generating original stories)

[1236] Emotion recognition models (algorithms for identifying user emotions)

[1237] A natural language processing engine (to process the generated narrative in natural language)

[1238] A speech synthesis engine (to convert the generated text into audio data and read it aloud)

[1239] Program Processing

[1240] 1. Acceptance of Selection Information:

[1241] The user terminal transmits information about the character, situation, genre, or reader to the server. This selection information is used as setting information that forms the basis of the story.

[1242] 2. Narrative Generation:

[1243] The data generation model on the server generates a story using a prompt sentence that instructs the data generation model to generate an original story based on the selected information received, and the data generation model (generative AI). This data generation model utilizes a natural language processing engine to dynamically create an original story that the user does not know.

[1244] 3. Emotion recognition:

[1245] The user's face is captured by the camera on the user's device, and the emotion recognition model analyzes the facial expressions to determine the current emotion of the user.

[1246] 4. Story Development Adjustment:

[1247] The server adjusts the storyline when generating the next storyline based on the emotion identified by the emotion recognition model, for example, if the user is showing happy emotion, the storyline is set to take a happy turn.

[1248] 5. Present your options:

[1249] During the story, the server presents the user with multiple options regarding the next development of the story. The user selects one of the options to determine the progress of the story.

[1250] 6. Story Progression and Updates:

[1251] The server generates the continuation of the story using a prompt sentence that instructs the server to generate a continuation of the story based on any one of the options selected by the user and the identified user emotion, and the generation AI. The generated continuation of the story is displayed on the user terminal and is read aloud by a voice synthesis engine.

[1252] Examples

[1253] Suppose a user launches an application, selects the fantasy genre, chooses a "hero" as the character (protagonist), a "village" as the situation, and a "teenage male" as the reader. As the user reads the story, the camera captures the user's happy facial expression, and the emotion recognition model identifies the emotion "joy." In this case, the story develops in a happy way, with the hero defeating the dragon and being thanked by the villagers. Also, in the middle of the story, the options of "saving the village" and "searching for treasure" are presented, and if the user selects "searching for treasure," the story will progress based on that choice.

[1254] An example of an initial prompt for generating a story: "Please generate a story based on the genre "Fantasy", the character (protagonist) "Hero", and the situation "Village". The reader is a "teenage boy". Please generate a story that suits this reader."

[1255] An example of an initial prompt for generating a continuation of a story: "Generate a continuation of the story based on the option "Treasure Search." The reader's emotion is "Joy." Generate a continuation of the story that matches this reader's emotion."

[1256] As described above, the present invention makes it possible to reflect a user's emotions in real time and provide an interactive storytelling experience.

[1257] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1258] Step 1:

[1259] User input of selection information

[1260] The user selects information about the characters, situation, genre, and reader through the application on the terminal. This selection information is sent from the terminal to the server. The input data is received as a string of characters, which becomes the basis for the next story generation.

[1261] Step 2:

[1262] Beginning of story generation

[1263] The server uses a data generation model to generate an original story based on the selection information received in step 1. Specifically, it uses a natural language processing engine to output a story that matches the selection information as text data.

[1264] Step 3:

[1265] User Emotion Recognition

[1266] While the user reads the story, the device's camera captures the user's face. The facial image is sent to the server in real time, and the emotion recognition model analyzes the image. As a result of the analysis, the user's emotion is identified as "happy" or "sad," etc., and the emotion data is output.

[1267] Step 4:

[1268] Adjusting story development based on emotions

[1269] The server adjusts the development of the story by including the emotion data obtained in step 3 in the prompt sentence when generating the rest of the story. For example, if the user shows happy emotion, the server changes the development of the story by including a sentence such as "The reader is feeling happy" in the prompt sentence when generating the rest of the story.

[1270] Step 5:

[1271] Present options for story progression

[1272] During the story, the server presents the user with multiple options for the next development. For example, the server displays the question "Where should the hero go next?" along with options such as "Climb the mountain" or "Cross the river." The user selects an option from the options.

[1273] Step 6:

[1274] Choice-based story updates

[1275] The server receives the choice selected by the user and updates the story progression based on this. The data generation model is then used again to generate the next part of the story based on the choice. For example, if the user selects "Cross the river," a sentence such as "The hero mustered up his courage and decided to cross the river" is generated.

[1276] Step 7:

[1277] View and read stories

[1278] The updated story is displayed on the device's display and is read aloud by a speech synthesis engine. The speech synthesis engine converts the generated text data into speech data and reads it through the speaker. The input is the updated text data, and the output is speech data.

[1279] By repeating the above steps, an interactive storytelling experience based on the user's emotions and choices is realized.

[1280] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1281] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1282] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the robot 414.

[1283] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[1284] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1285] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1286] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1287] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[1288] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[1289] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[1290] Although the system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, the system according to the present disclosure is not necessarily implemented in a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program that runs on a personal computer, or an application that runs on a smartphone or the like. The method according to the present disclosure may be provided to a user in the form of SaaS (Software as a Service).

[1291] In the above embodiment, an example is given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to input data.

[1292] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1293] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[1294] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1295] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing the specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[1296] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[1297] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1298] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[1299] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the technology of the present disclosure.

[1300] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or standard was specifically and individually indicated to be incorporated by reference.

[1301] The following is further disclosed regarding the above embodiment.

[1302] (Claim 1)

[1303] An interactive story creation system, comprising: means for accepting selection information including character, situation, genre, or reader information selected by a user; means for generating an original story using a data generation model based on the selection information; A means for presenting the user with options regarding the progression of the story during the course of the story; means for updating a story progression using the data generation model based on choices selected by the user; and means for displaying story updates to the user, Repeatedly presenting the options, updating the story progress, and displaying to the user. system.

[1304] (Claim 2)

[1305] The system according to claim 1, characterized in that the data generation model performs natural language processing to generate sentences in generating a story.

[1306] (Claim 3)

[1307] 2. The system according to claim 1, wherein the updating means reflects the user's emotion, which is recognized using an emotion engine that recognizes the user's emotion, in the progress of the story.

[1308] (Claim 4)

[1309] 2. The system according to claim 1, wherein the presenting means adjusts the presented options based on the user's emotion recognized using an emotion engine that recognizes the user's emotion.

[1310] (Claim 5)

[1311] 2. The system of claim 1, wherein a user can change the outcome of a story by selecting a progression of the story.

[1312] (Claim 6)

[1313] 5. The system according to claim 3, wherein the emotion engine analyzes information such as the user's facial expression and voice, and recognizes the user's emotion.

[1314] "Example 1"

[1315] (Claim 1)

[1316] means for accepting selection information including character, situation, genre, or reader information selected by a user; A means for generating an original story using a generative AI model based on the selection information; A means for presenting the user with options regarding the progression of the story during the course of the story; means for updating a story progression using the generative AI model based on options selected by the user; and means for displaying story updates to the user, Repeatedly presenting the options, updating the story progress, and displaying to the user. system.

[1317] (Claim 2)

[1318] The system described in claim 1, characterized in that, in generating a story, the generative AI model performs natural language processing to generate sentences using prompt sentences.

[1319] (Claim 3)

[1320] 2. The system according to claim 1, wherein the updating means reflects the user's emotions, which are recognized using an emotion analysis engine that recognizes the user's emotions, in the progress of the story.

[1321] "Application example 1"

[1322] (Claim 1)

[1323] means for accepting selection information including character, situation, genre, or reader information selected by a user; A means for generating an original story using a generative AI model based on the selection information; A means for presenting the user with options regarding the progression of the story during the course of the story; means for updating a story progression using the generative AI model based on options selected by the user; means for displaying story updates to said user; a means for selecting educational topics; A means to customize stories based on educational themes; A system including:

[1324] (Claim 2)

[1325] The system according to claim 1, characterized in that, in generating a story, the generative AI model performs natural language processing to generate sentences.

[1326] (Claim 3)

[1327] The system of claim 1, wherein the updating means uses a generative AI model to present educational options based on the user's selection and reflect them in the progression of the story.

[1328] "Example 2 of combining emotion engines"

[1329] (Claim 1)

[1330] means for accepting selection information including character, situation, genre, or reader information selected by a user; means for generating an original story using a data generation model based on the selection information; means for presenting the user with story progression options mid-word; means for updating a story progression using the data generation model based on choices selected by the user; The updating means includes a means for acquiring a facial expression and a voice of a user and analyzing the emotion of the user using an emotion analysis device; A means for sending a prompt sentence to a generation AI model based on the user's emotion to generate a next story development; A means for displaying the generated story development and options to the user; means for detecting again changes in the user's emotions and continually adjusting the progress of the story and the choices; A system including:

[1331] (Claim 2)

[1332] 2. The system according to claim 1, further comprising a natural language processing unit for generating sentences using the data generation model.

[1333] (Claim 3)

[1334] 2. The system according to claim 1, wherein the emotion analysis device includes an image analysis device that analyzes a user's facial expression and a voice analysis device that analyzes a user's voice.

[1335] "Application example 2 when combining emotion engines"

[1336] (Claim 1)

[1337] means for accepting selection information including character, situation, genre, or reader information selected by a user; means for generating an original story using a data generation model based on the selection information; A means for identifying a user's emotion using a camera and an emotion recognition model; means for adjusting the development of the story based on the identified user's emotions; A means for presenting the user with options regarding the progression of the story during the course of the story; means for updating a story progression using the data generation model based on choices selected by the user; means for displaying story updates to said user; A means for reading the contents of the story aloud using a voice synthesis engine; Repeatedly presenting the options, updating the progress of the story, and displaying and reading to the user. system.

[1338] (Claim 2)

[1339] The system according to claim 1, wherein the data generation model performs natural language processing to generate sentences in generating a story, thereby realizing a development of the story in accordance with the user's emotions.

[1340] (Claim 3)

[1341] 2. The system according to claim 1, wherein the updating means reflects the user's emotion, recognized using an emotion recognition model that recognizes the user's emotion, in the progress of the story. [Explanation of symbols]

[1342] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for accepting selection information selected by a user, the selection information including character, situation, genre, and reader information; A means for generating an original story using a prompt sentence instructing the user to generate an original story based on the selected information and a generation AI; means for identifying the user's emotion using a camera and an emotion recognition model; means for presenting the user with a plurality of options regarding the progression of the story during the course of the story; A means for generating a continuation of the story using a prompt sentence instructing to generate a continuation of the story based on any one of the options selected by the user and the identified user emotion, and the generation AI; means for displaying the generated story to the user; Including, A system that repeatedly presents the options, generates the continuation of the story, and displays it to the user.

2. The device further includes a means for reading out the content of the story by a voice synthesis engine; The system of claim 1 , wherein the system repeatedly presents the options, generates a continuation of the story, and displays and reads the continuation to the user.

3. The system according to claim 1 , wherein the means for presenting options presents a plurality of options regarding the next development of the story as options regarding the progress of the story during the course of the story, and accepts selection of any one of the options from the user.

Images