System

Abstract

To provide a system.SOLUTION: There is provided a moving image for kids streaming application system having a generative AI built therein for determining the ending of a story or the behavior of a character, and the system includes: input means that inputs, to a server, input data including story data related to the details of the story and the character data of a child; option generation means that generates options, by the generative AI, on the basis of the input data; option list display means that displays an option list including the options on a terminal; transmission means that transmits proposals for selection from the options from the child, from the terminal to the server; story / moving image data generation means that generates story data or moving image data on the basis of the selection; and story / moving image data display means that displays the story data or the moving image data on the terminal.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional video streaming apps for children offer limited opportunities for children to actively engage with the story progression and character actions, and lack an environment that encourages creative thinking. [Means for solving the problem]

[0005] The present invention aims to provide a children's video streaming application system that incorporates generative AI to determine the ending of the story and the actions of the characters, allowing children to play an active role in the story and encouraging creative thinking. Specifically, the following measures are adopted:

[0006] - By providing a means for inputting story data from the server and data on the child's characteristics, it becomes possible to generate a story that takes into account the child's attributes and preferences.

[0007] By providing a means for generative AI to generate options based on input data, children are presented with options that allow them to be involved in the development of the story.

[0008] By providing a way to display a list of options on the device, children can suggest options and move the story forward through their own decisions.

[0009] By providing a means for receiving suggested choices from children and sending them to a server, it becomes possible to generate stories that reflect the children's opinions and choices.

[0010] By providing a means for generating story and video data based on the server's selection, stories and videos are displayed according to the options selected by the children.

[0011] These methods allow children to play an active role in the story and advance it based on their own choices, which can stimulate their creativity and thinking and provide them with a richer experience.

[0012] "Generative AI" is a natural language processing model that generates options for the characters' actions according to the content of the story and allows children to suggest options to progress the story. "Story data" refers to data related to the content of the story, such as the plot of the story and character information. "Children's characteristics data" refers to data that represents the children's attributes and preferences, such as their age, gender, and address. "Choice list" refers to a list of options related to the progress of the story, and is a list from which children can select the character's actions in the story. "Choice suggestions" refers to children proposing their own choices from the presented list of options. "Story generation" refers to generating the content of the story based on the input story data, children's characteristics data, and choice suggestions. "Video data" refers to video data according to the content of the story, and is generated according to the progress of the story and the actions of the characters. [Brief description of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] 4 is a sequence diagram showing a process flow of the data processing system according to the first embodiment. FIG. [Figure 12] 11 is a sequence diagram showing a process flow of the data processing system in application example 1. FIG. [Figure 13] FIG. 11 is a sequence diagram showing the flow of processing of the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 11 is a sequence diagram showing the flow of processing in the data processing system in application example 2 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. The processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit),

[0017] Examples include GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0018] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs voice in accordance with instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0029] It's a camera.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0036] An embodiment for implementing the present invention includes the following elements.

[0037] 1. Method of inputting data from the server:

[0038] The server receives the story data and the child's characteristic data.

[0039] Story data includes plot and character information for a story.

[0040] The child's characteristic data includes attribute information such as age, sex, and address.

[0041] 2. How choices are generated:

[0042] Based on the input data, the server generates options for the character's actions that correspond to the content of the story.

[0043] Appropriate options can be generated taking into account the child's characteristic data.

[0044] 3. How to present options:

[0045] The terminal displays the generated option list on the screen.

[0046] The child suggests a choice from a presented list of options.

[0047] 4. Means of sending your choice:

[0048] The terminal transmits the child's selection to the server.

[0049] 5. Narrative generation and video data display:

[0050] The server generates a story based on the selections received.

[0051] The server generates video data in accordance with the generated story.

[0052] The terminal displays the generated video data on a display.

[0053] In this way, the server and the device work together to control the progress of the story and the actions of the characters, allowing the child to actively participate.

[0054] As the story progresses and videos are displayed, you can experience the story through your own choices.

[0055] The process flow of each embodiment will be described below.

[0056] Step 1: Input data from the server

[0057] The server receives the story data and the child's characteristic data.

[0058] The story data includes information about the plot and characters of the story, and the child characteristic data includes attribute information such as age and gender.

[0059] Step 2: Generate options

[0060] The server generates options for the character's actions according to the content of the story, based on the received story data and the child's characteristic data.

[0061] Appropriate options can be generated taking into account the child's attribute information.

[0062] Step 3: Present the options

[0063] The terminal displays the generated option list on the screen.

[0064] The child suggests a choice from a presented list of options.

[0065] Step 4: Submit your selections

[0066] The terminal transmits the child's selection to the server.

[0067] Step 5: Generate narrative and display video data

[0068] The server generates a story based on the selections received.

[0069] The server generates video data in accordance with the generated story.

[0070] The terminal displays the generated video data on a display.

[0071] Step 6: Story progression and repetition

[0072] While the child enjoys the story presented, they are then asked to make suggestions for the next step.

[0073] The processes from step 2 to step 5 above are repeated as the story progresses.

[0074] Output Condition:

[0075] Server: Receives story data and child characteristic data, generates options, generates stories, and generates video data

[0076] Terminal: Display list of options, send selection, display video data

[0077] User (child): Suggestions for choices, story progression

[0078] Example 1

[0079] Next, a description will be given of Example 1. In the following description, the data processing device 12 is referred to as a "server" and the smart device 14 is referred to as a "terminal."

[0080] In conventional video streaming systems for children, it was difficult for users to interactively participate in the development of the story, and the story often progressed one-sidedly. Therefore, a system that allows children to make their own choices and experience the progression of the story is desired. There is also a demand for a system that can provide options according to the characteristics of each individual child, but this has not been fully realized in conventional technologies.

[0081] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0082] In this invention, the server includes a means for inputting story data and child characteristic data, a means for generating options based on the input data using a generative AI model (e.g. ChatGPT (registered trademark), BERT), and a means for generating story and video data. This allows options appropriate for the child's characteristics to be provided, and allows the user to experience the development of the story based on their own selection.

[0083] A "server" is a central management system that stores and processes data over a network.

[0084] "Story data" refers to data that includes information related to the progression of the story, such as the story plot and character information.

[0085] "Child characteristic data" refers to data including the child's attribute information, such as age, sex, and address.

[0086] A "generative AI model" is a model that uses artificial intelligence to analyze data and generate options, stories, etc.

[0087] The "option generation means" is a mechanism that generates options from which the user can choose based on input data.

[0088] A "choice list" is a collection of multiple options that are displayed on the screen for the user to select from.

[0089] A "terminal" is a device operated by a user, and is a device that displays options and transmits and receives data.

[0090] A "user selection suggestion means" is a mechanism that receives selection input from a user and transmits it to a server.

[0091] The "means for generating story and video data" is a mechanism for generating a new story based on the user's selection and creating video data corresponding to that story.

[0092] The "video data display means" is a mechanism for displaying the generated video data on the display of the terminal.

[0093] The "means for generating options taking into consideration characteristic data" is a mechanism for generating appropriate options based on the child's characteristic data.

[0094] Overall overview

[0095] The present invention relates to an interactive video streaming system for children, and provides a system in which users can participate in the progression of a story. This system dynamically generates story data and video data by making full use of servers, terminals, and generative AI models (e.g. ChatGPT (registered trademark), BERT), and provides users with an interactive experience.

[0096] server

[0097] The server is responsible for receiving story data and child characteristic data. For example, it receives data including the plot and character information of the story, and attribute information such as the child's age, gender, and address. The server also analyzes these data using a generative AI model (e.g., ChatGPT (registered trademark), BERT) and generates options based on the input data. Furthermore, the server generates a new story based on the user's selection and creates video data corresponding to the story.

[0098] Hardware / Software used:

[0099] Server equipment (high performance computers)

[0100] Generative AI models (e.g. ChatGPT, BERT)

[0101] Database (e.g. MySQL(registered trademark), PostgreSQL)

[0102] Terminal

[0103] The terminal is a device operated by the user, and its role is to display the list of options sent from the server on the screen. When the user selects an option, the information is sent to the server in real time. In addition, the generated video data is displayed to the user.

[0104] Hardware / Software used:

[0105] Tablets and PCs (user-operated devices)

[0106] Display (for video display)

[0107] Internet connection

[0108] User

[0109] Users (usually children) operate the device, choose the action they want from the options displayed, and experience the progression of the story as a result. This allows users to experience a unique story based on their own choices.

[0110] Examples of concrete examples and prompts

[0111] Example 1:

[0112] The server receives the plot data of "The Story of a Hero Defeating a Dragon" and the characteristic data of "a 7-year-old boy," and generates the options "attack," "run away," and "use an item." The device displays these options on the screen, and the user selects "attack," which is sent to the server. Based on this selection, the server generates a scene in which the hero attacks the dragon, and creates a corresponding animated video. The device displays this video.

[0113] Example 2:

[0114] The server receives the data "A story about a wizard choosing a spell" and the characteristic data "12-year-old girl," and generates the options "Fire magic," "Water magic," and "Wind magic." The device displays these options on the screen, and the user selects "Fire magic," which is sent to the server. Based on this selection, the server generates a scene in which the wizard casts a fire spell, and creates a corresponding animated video. The device displays this video.

[0115] Examples of prompt statements

[0116] Prompt statement example 1:

[0117] User: 7 year old boy

[0118] Story scene: The hero confronts the dragon

[0119] Choices: Attack, Run, Use an item

[0120] Output format: Animated video with the corresponding story of the scene's continuation

[0121] Prompt statement example 2:

[0122] User: 12 year old girl

[0123] Story scene: The wizard chooses a spell

[0124] Choices: Fire magic, Water magic, Wind magic

[0125] Output format: Animated video with the corresponding story of the scene's continuation

[0126] By feeding these prompts into a generative AI model (e.g. ChatGPT (registered trademark), BERT), the system can provide an interactive narrative experience like the one described above.

[0127] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0128] Step 1: Receiving data

[0129] The server is responsible for receiving story data and child characteristic data. The story data includes story plot and character information, and the child characteristic data includes attribute information such as age, gender, address, etc. The server obtains these data from an external database or a user device. It receives the story data and characteristic data as input and stores it internally.

[0130] Specific behavior:

[0131] The server retrieves plot data for "The Story of a Hero Slaying a Dragon."

[0132] The server receives the characteristic data of "7-year-old boy."

[0133] Input: Story data, child characteristics data

[0134] Output: Data stored inside the server

[0135] Step 2: Generate options

[0136] The server generates options using a generative AI model (e.g. ChatGPT (registered trademark), BERT) based on the received story data and the child's characteristics data. In this process, the server analyzes the input data and creates multiple options to present to the user.

[0137] Specific behavior:

[0138] The server recognizes this as a scene in which a hero confronts a dragon, and generates age-appropriate options such as "attack," "run away," or "use an item."

[0139] Input: Story data, child characteristics data

[0140] Output: Choice list (response)

[0141] Step 3: Displaying the options

[0142] The terminal has the role of displaying the option list sent from the server on the screen. The terminal receives the option list returned from the server and displays it in a format that is easy for the user to see.

[0143] Specific behavior:

[0144] The tablet device will display the options "attack," "run away," and "use item" on the screen.

[0145] Input: Choice list (response from server)

[0146] Output: The choices displayed on the screen.

[0147] Step 4: Sending user selections

[0148] The user selects one of the options displayed on the screen. The user's choice is sent to the server via the terminal. The terminal receives the user's input and forwards it to the server.

[0149] Specific behavior:

[0150] The child taps to select "attack," and the tablet sends the selection data to the server.

[0151] Input: User's choice

[0152] Output: Selection data sent to the server

[0153] Step 5: Generate a story

[0154] The server generates new story scenes based on the received choices, using generative AI models (e.g. ChatGPT, BERT) to create story progression content based on the choices. In this step, the storyline is updated based on the choices and the next scene is generated.

[0155] Specific behavior:

[0156] Based on the server's choice of "attack," a scene is generated in which the hero attacks the dragon.

[0157] Input: User selected data

[0158] Output: Next story scene

[0159] Step 6: Generate video data

[0160] The server generates video data based on the generated story. It uses generative AI models (e.g. ChatGPT, BERT) to create animated videos that correspond to the scenes in the story. In this dynamic process, visual data that matches the story is rendered.

[0161] Specific behavior:

[0162] The server creates animation data based on the "scene of the hero attacking the dragon" and encodes it in video format.

[0163] Input: Next story scene

[0164] Output: Video data

[0165] Step 7: View video data

[0166] The terminal displays the video data sent from the server to the user. The terminal receives the video data and plays it to visually present the progress of the story to the user.

[0167] Specific behavior:

[0168] The tablet device plays the animation video received from the server.

[0169] Input: Video data

[0170] Output: Video played on screen

[0171] Through the above processing steps, a child can interactively progress through the story and enjoy a unique story experience based on his or her own choices.

[0172] (Application example 1)

[0173] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0174] There is a demand for video streaming services for children to provide a more interactive and educational experience by dynamically changing the progression of the story according to the viewer's (child's) choices. However, current systems do not generate stories that correspond to the individual attributes of each child, nor do they present options, and the content is uniform. This makes it difficult to provide content that is appropriate for each child, resulting in low immersion and educational effectiveness.

[0175] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0176] In this invention, the server includes a means for inputting story data and child characteristic data, a means for generating options based on the input data using a generative AI model (e.g., ChatGPT (registered trademark), BERT), and a means for displaying a list of options on a terminal. This enables an interactive story progression based on the individual attributes of the child.

[0177] A "generative AI model" is an artificial intelligence model that generates dynamic narratives and options based on given data.

[0178] An "interactive story application system" is a system that provides an application in which the progress of a story changes according to the user's choices.

[0179] "Story data" is data necessary for the progression of the story, such as the story plot and character information.

[0180] "Child characteristic data" refers to data that includes a child's attribute information, such as age, gender, and address.

[0181] The “server” is a computer system that receives story data and child characteristic data, and generates options and stories based on a generative AI model (e.g., ChatGPT (registered trademark), BERT).

[0182] The "option generation means" is a mechanism for generating multiple options based on input data.

[0183] A "terminal" is a device that allows a user to view and select options.

[0184] The "means for receiving selection suggestions" is a mechanism for transmitting the options selected by the user to the server.

[0185] "Means for generating story and video data" refers to a mechanism for creating story progression and video data based on user selection.

[0186] "Video data display means" refers to a mechanism for showing the generated video data to the user.

[0187] This invention shows a specific method for realizing an interactive storytelling application system using generative AI models (e.g. ChatGPT (registered trademark), BERT). The system mainly consists of three elements: a server, a terminal, and a user.

[0188] server

[0189] The server receives story data and child characteristic data as input. For this, a web framework such as Django is used, and Python is used as the programming language. Based on this data, the server generates interactive options using a generative AI model (e.g. ChatGPT (registered trademark), BERT) and sends the option list to the terminal.

[0190] Story data refers to data necessary for the progression of the story, such as the story plot and character information.

[0191] The child's characteristic data is data including the child's attribute information such as age, sex, and address.

[0192] The server also receives the options selected by the user and generates new story progression and video data based on the options selected by the user. The generated video data is sent to the terminal.

[0193] Terminal

[0194] The terminal refers to a smartphone or tablet, and is built using a framework such as React Native. The terminal displays the list of options received from the server, sends the user's selection to the server, and displays new video data sent from the server.

[0195] User

[0196] The user is the child and makes his / her selection from a displayed list of options, the content of which is appropriately customized based on the child's characteristic data.

[0197] Examples

[0198] For example, if a user (child) selects "Character A climbs a tree," the server generates a new story progression based on this selection, creates video data of "the view that Character A sees from the top of the tree," and sends it to the terminal. On the other hand, if the user selects "Character B crosses the river," the story progresses to "Character B meets a new friend on the other side of the river," and video data for that is generated.

[0199] Examples of prompt statements

[0200] Generate interactive story choices based on the following criteria:

[0201] Child's demographic information: Age 7, gender male, address Tokyo

[0202] Story ID is 1

[0203] Please let me know what option is appropriate.

[0204] In this way, generative AI models (e.g. ChatGPT, BERT) are used to provide a personalized interactive storytelling experience for each child, enhancing educational and entertainment value.

[0205] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0206] Step 1:

[0207] The server receives the story data and the child's characteristics data. This data includes the story plot, character information, the child's age, gender, address, etc. Based on the input story data and child's characteristics data, it extracts relevant information from the database and formats it in the format required for subsequent processing. The output at this point is the formatted story data and child's characteristics data.

[0208] Step 2:

[0209] The server uses a generative AI model (e.g. ChatGPT (registered trademark), BERT) to generate options based on the input data. Specifically, the story data and the child's characteristics data are input into the AI ​​model, and a list of options is generated from the model. Data processing involves filtering so that options appropriate to the child's attributes are selected. The output is a list of options.

[0210] Step 3:

[0211] The server sends a list of options to the terminal. The sent list of options is displayed on the terminal. The user (child) sees this list and makes a selection. The output is the list of options displayed on the terminal.

[0212] Step 4:

[0213] The user selects one item from a list of options, and the device sends the selection to the server. The input is the user's selection, and the resulting concrete option is sent to the server. The output is the selection sent to the server.

[0214] Step 5:

[0215] The server generates story progression and new video data based on the user's selection. Based on the content of the choice, the server extracts the relevant story progression part from the story database and inputs it into a generative AI model (e.g. ChatGPT (registered trademark), BERT). Video data is generated based on the generated data. The output is the new story progression and video data.

[0216] Step 6:

[0217] The server sends the generated video data to the terminal. The terminal displays the received video data on the screen. The user can watch the progress of the new story. The input is the video data sent from the server, and the output is the video displayed on the terminal.

[0218] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0219] An embodiment for implementing the present invention includes the following elements.

[0220] 1. Method of inputting data from the server:

[0221] The server receives the story data, the child's characteristic data, and the user's emotion information.

[0222] The story data includes information about the plot and characters of the story, and the child characteristic data includes attribute information such as age and gender.

[0223] The user's emotional information is recognized from voice, facial expressions, etc. and processed by an emotion engine.

[0224] 2. How choices are generated:

[0225] The server generates options for the character's actions according to the content of the story based on the received story data, the child's characteristic data, and the user's emotion information.

[0226] The emotion engine can take into account the user's emotion information to generate appropriate options.

[0227] 3. How to present options:

[0228] The terminal displays the generated option list on the screen.

[0229] The child suggests a choice from a presented list of options.

[0230] 4. Means of sending your choice:

[0231] The terminal transmits the child's selection to the server.

[0232] 5. Narrative generation and video data display:

[0233] The server generates a story based on the selections received.

[0234] The server generates video data in accordance with the generated story.

[0235] The terminal displays the generated video data on a display.

[0236] 6. Emotion engine processing means:

[0237] The server receives the user's emotion information and processes it using an emotion engine.

[0238] The emotion engine recognizes the user's emotional information, such as voice and facial expressions, and adjusts the story progress and character behavior based on that information.

[0239] In this way, the server and the device work together to control the progression of the story and the actions of the characters, allowing the child to be actively involved. The child suggests options, the story progresses based on those options, and the videos are displayed, allowing the child to experience the story based on their own choices and emotions.

[0240] The process flow of each embodiment will be described below.

[0241] Step 1: Input data from the server

[0242] The server receives the story data, the child's behavior data, and the user's emotion information.

[0243] The story data includes information about the plot and characters of the story, and the child characteristic data includes attribute information such as age and gender.

[0244] The user's emotional information is recognized from voice, facial expressions, etc., and processed by the emotion engine. Step 2: Generation of options

[0245] The server generates options for the character's actions according to the content of the story based on the received story data, the child's characteristic data, and the user's emotion information.

[0246] The emotion engine can generate appropriate options by taking into account the user's emotion information. Step 3: Present options

[0247] The terminal displays the generated option list on the screen.

[0248] The child suggests a choice from a presented list of options.

[0249] Step 4: Submit your selections

[0250] The terminal transmits the child's selection to the server.

[0251] Step 5: Generate narrative and display video data

[0252] The server generates a story based on the selections received.

[0253] The server generates video data in accordance with the generated story.

[0254] The terminal displays the generated video data on a display.

[0255] Step 6: Emotion Engine Processing

[0256] The server receives the user's emotion information and processes it using an emotion engine.

[0257] The emotion engine recognizes the user's emotional information, such as voice and facial expressions, and adjusts the story progress and character behavior based on that information.

[0258] Through the above processing steps, the server and the device work together to control the progress of the story and the actions of the characters, allowing the child to actively participate. The child proposes options, the story progresses based on those options, and the videos are displayed, allowing the child to experience the story based on his or her own choices and emotions.

[0259] Example 2

[0260] Next, a description will be given of Example 2. In the following description, the data processing device 12 is referred to as a "server" and the smart device 14 is referred to as a "terminal."

[0261] In conventional interactive video systems for children, the progression of the story and the actions of the characters are prescribed, making it difficult to respond to the interests and emotions of individual children. In addition, there was a problem in that it was not possible to dynamically generate a story that incorporates changes in the user's emotions.

[0262] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an input means for story data, child attribute information, and user emotion information, a generation means for generating options based on the input data by a generative AI, and a processing means for an emotion engine that recognizes the user emotion information and adjusts the progress of the story and the behavior of the characters. This enables a personalized story progress based on the child attribute information and the user emotion.

[0263] "Story data" is digital data that includes information about the story plot and characters.

[0264] "Child attribute information" is data indicating the child's attributes such as age and sex.

[0265] "User emotion information" is data indicating the user's emotional state obtained from the user's voice, facial expression, etc.

[0266] The "input means" refers to a means by which the server receives story data, child attribute information, and user emotion information.

[0267] "Generative AI" is an artificial intelligence technology that generates story options or the story itself based on input data.

[0268] The "option generating means" is a means for generating options based on received story data, child attribute information, and user emotion information.

[0269] An "emotion engine" is software that recognizes the user's emotional information and adjusts the progress of the story and the behavior of characters.

[0270] "Video data" refers to animation and video data that is generated in accordance with the progression of the story.

[0271] A "Terminal" is a device (e.g., tablet, smartphone) that a User can interact with.

[0272] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0273] This invention relates to an interactive video system for children that incorporates generative AI to determine the ending of a story and the actions of characters. This system mainly operates in cooperation with a server and terminals.

[0274] The server accepts input of story data, child attribute information, and user emotion information. Story data includes the story plot and character information. Child attribute information includes information such as age and gender. User emotion information is obtained from voice and facial expressions, and processed by an emotion engine. Based on this input data, the server uses generative AI to generate appropriate options.

[0275] The generated options are sent to the terminal, which displays them on the screen. The user selects one of the options presented and sends it to the server via the terminal. The server generates a story based on the received selection and creates video data to match it. The video data is then sent back to the terminal and displayed to the user.

[0276] During this entire process, the server uses an emotion engine to recognize the user's emotional information in real time and adjust the story progression and character behavior, thus enabling a personalized interactive video experience.

[0277] As a concrete example, the server recognizes the story data of "Little Red Riding Hood," the attribute information of a 7-year-old girl, and the user's smile. After the emotion engine recognizes the user's smile, it generates an option during the story that reads, "Please select an animal that Little Red Riding Hood will encounter in the forest." This is displayed on the device's display, and the child selects "rabbit." Upon receiving this selection, the server generates a "scene in which Little Red Riding Hood talks to the rabbit," creates video data, and displays it on the device.

[0278] Examples of prompt statements

[0279] "Generate three options for animals that Little Red Riding Hood encounters in the forest for a 7-year-old girl to enjoy. If you detect a smile from the user, make it more fun."

[0280] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0281] Step 1: Data entry

[0282] The server inputs story data, the child's attribute information, and the user's emotion information. Specifically, as story data input, plot and character information for a story such as "Little Red Riding Hood" is obtained from a database. As the child's attribute information, data that the child is a "7-year-old girl" is received via sensors and cameras. As the user's emotion information, real-time voice and facial expression data is obtained from the camera and microphone, and this is processed by the emotion engine.

[0283] input:

[0284] Story data (e.g. the plot of "Little Red Riding Hood")

[0285] Child demographic information (e.g. 7 years old, girl)

[0286] User emotion information (e.g. smile detection)

[0287] output:

[0288] Integrated data passed to the server

[0289] Step 2: Generate options

[0290] Based on the received data, the server uses a generative AI model (e.g. ChatGPT (registered trademark), BERT) to generate appropriate options. A prompt is input to the generative AI model (e.g. ChatGPT (registered trademark), BERT), for example, "Generate three character actions that a 7-year-old girl would enjoy. If the user's smile is recognized, make it more enjoyable." The emotion engine takes into account the user's emotional information and generates appropriate options.

[0291] input:

[0292] Integrated data (story data, children's attribute information, user's emotion information)

[0293] output:

[0294] Generated choice list (e.g. 1. rabbit 2. squirrel 3. bird)

[0295] Step 3: Present the options

[0296] The device displays the generated list of options on the screen. Specifically, the tablet display shows, "Please choose an animal that Little Red Riding Hood will meet: 1. Rabbit 2. Squirrel 3. Small bird." The user, a child, taps the screen to make a selection.

[0297] input:

[0298] Generated Choice List

[0299] output:

[0300] Choice list displayed on the device

[0301] Step 4: Submit your selections

[0302] The terminal transmits the data selected by the child to the server. Specifically, when the user selects "rabbit," the selection information is transmitted to the server.

[0303] input:

[0304] Select a child (e.g. "rabbit")

[0305] output:

[0306] Selection data sent to the server

[0307] Step 5: Generate a story

[0308] The server generates a story based on the selections it receives: for example, a scene where Little Red Riding Hood meets and talks with the Rabbit, and creates video data to go with it. An emotion engine also helps at this stage, providing more personalized content.

[0309] input:

[0310] Selection data (e.g. "rabbit")

[0311] output:

[0312] Generated story (e.g. the scene where Little Red Riding Hood talks to the rabbit)

[0313] Step 6: Generate and display video data

[0314] The server generates video data in accordance with the progress of the story and sends it to the device. The device then displays the received video data on the display. Specifically, the tablet plays an animation of Little Red Riding Hood meeting and talking to the Rabbit.

[0315] input:

[0316] Generated story (e.g. the scene where Little Red Riding Hood talks to the rabbit)

[0317] output:

[0318] Generated video data, video playback on device

[0319] (Application example 2)

[0320] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0321] Traditional children's content lacked the ability to adjust the progress of the story and the actions of characters based on the individual attributes and emotions of the viewer, limiting the experience of viewers actively participating in the story. In addition, since only pre-fixed story developments could be selected, it was often unable to respond to the interests and concerns of the viewer. As a result, it was difficult to stimulate the viewer's motivation to learn and creativity.

[0322] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0323] In this invention, the server includes a means for inputting story data, child characteristic data, and user emotion information, a means for generating options based on the input data by a generative AI, and a means for adjusting the progress of the story and the behavior of the characters based on the user emotion information. This makes it possible to dynamically progress the story based on the individual attributes and emotions of the viewers.

[0324] "Story ending" and "character actions" refer to the final outcome or action of a character or plot in a scene or episode within a story.

[0325] "Generative AI" refers to artificial intelligence technology that automatically generates story development and character movements based on input data.

[0326] "Content delivery system" refers to the entire system for providing digital content to users, and in this context refers specifically to a system for delivering interactive stories for children.

[0327] "Story data" refers to data that includes plot and character information necessary for the progression of the story.

[0328] "Characteristic data" refers primarily to attribute data relating to a child's age and gender.

[0329] "Emotion information" is information that indicates the user's emotional state obtained from voice, facial expressions, etc.

[0330] "Options generation means" refers to a method or device for automatically generating options to be provided to a user.

[0331] An "option list" is a list of multiple options that a child can choose from as the story progresses.

[0332] "Means for receiving and transmitting suggestions" refers to a method or device for transmitting the child's selections to the server.

[0333] "Video data" refers to a sequence of image data intended for visual display and usually includes audio.

[0334] "Means for adjusting story progression" refers to a method or device for dynamically adjusting story progression or character behavior based on user input or emotional information.

[0335] "Display means" refers to a method or device for displaying the generated video or option list on a user terminal.

[0336] This invention is an interactive children's content distribution system in which a server and a terminal work in cooperation with each other, and includes the following elements:

[0337] Preparing the server and terminal

[0338] The server mainly receives story data, child characteristic data, and user emotion information. The server uses MongoDB (registered trademark) or Firebase (registered trademark) as a database, and utilizes Google (registered trademark) Cloud Vision API or IBM Watson (registered trademark) as an emotion recognition engine. The server is built using Node.js (registered trademark) or Python (Flask / Django framework).

[0339] The device is mainly intended to be a smartphone, and will be built using React Native (registered trademark) and Flutter (registered trademark) as the front-end platform. The device is equipped with a camera and microphone, and will capture the user's voice and facial expressions as emotional information.

[0340] Server Processing

[0341] The server receives the following data as input:

[0342] Narrative Data: information about the plot and characters of a story.

[0343] Child characteristics data: demographic information such as age and gender.

[0344] User emotion information: voice and facial expression data.

[0345] Based on this data, the generative AI generates options. The generated options are sent to the device. When the user's choice is sent from the device to the server, the server generates a story and video data based on the choice. The story and video are generated using libraries such as FFmpeg.

[0346] Terminal handling

[0347] The device displays a list of options received from the server to the user. The user selects from the options and the selection is sent to the server. The device also obtains the user's emotional information (voice and facial expressions) and sends it to the server in real time. This allows the server-side emotion engine to adjust the progress of the story and the character's actions.

[0348] Examples

[0349] For example, suppose a story about a "night adventure" is displayed. In this story, the child is presented with the choice of "preparing before leaving or not." If the child chooses "preparing," the story switches to a preparation scene. In addition, the device's camera and microphone capture the child's smile and tone of voice, and transmit this emotional information to the server. Depending on the child's smile, the server can change the setting of the story to a brighter, more enjoyable development.

[0350] Example prompt:

[0351] This is the story of a brave knight who goes on an adventure. The choices you make will determine how the adventure progresses. Let's start the adventure together!

[0352] In this way, the server and device work together to enable children to dynamically experience a story based on their own choices and emotions.

[0353] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0354] Step 1:

[0355] The server receives story data, child's characteristics data, and user's emotion information as input. This includes story plot and character information. Emotion information is voice data and facial expression data obtained from the terminal. These data are stored in MongoDB and Firebase.

[0356] Input: Story data, child's characteristics data, user's emotion information

[0357] Output: A record of the stored data

[0358] Specific behavior:

[0359] The server receives the data sent from the terminal via the API.

[0360] A process is executed to store the received data in a database.

[0361] Step 2:

[0362] The generative AI generates options based on the story data and the child's characteristics and emotional information. The generated options are used by the child to decide the progress of the story.

[0363] Input: Story data, child's characteristics data, user's emotion information

[0364] Output: Choice list

[0365] Specific behavior:

[0366] The generative AI on the server refers to the stored data and generates appropriate options.

[0367] The emotion engine analyzes emotional information and reflects it in generating options.

[0368] Step 3:

[0369] The device displays the list of options received from the server to the user. The user (child) selects the option they want from the presented options by tapping on it.

[0370] Input: Choice list

[0371] Output: User's choice

[0372] Specific behavior:

[0373] The front-end application displays the list of options in a GUI.

[0374] It recognizes the user's tapping actions and sends the selection to the server.

[0375] Step 4:

[0376] The server receives the user's selections from the terminal and generates the next story portion based on the selections, including video data corresponding to the selections.

[0377] Input: User's choice

[0378] Output: Generated story text, video data

[0379] Specific behavior:

[0380] The server receives the user's selections and updates the story data.

[0381] Video generation processing according to the progression of the story is carried out using FFmpeg.

[0382] Step 5:

[0383] The emotion engine analyzes the user's real-time emotion information and adapts the story progression and character behavior based on that information. Revised story and video data are generated.

[0384] Input: User emotion information

[0385] Output: Revised story text, video data

[0386] Specific behavior:

[0387] The server analyzes the voice and facial expression data transmitted in real time.

[0388] Based on the analysis results, the story's progression and character behavior are dynamically revised.

[0389] Step 6:

[0390] The terminal receives the generated video data from the server and displays it on the display, allowing the user to watch the progress of the new story.

[0391] Input: Generated story text, video data

[0392] Output: Narrative video shown on the display

[0393] Specific behavior:

[0394] The application on the terminal receives streaming video data from the server.

[0395] The received video is displayed on the user interface and played.

[0396] This allows users to interactively experience the story based on their choices and emotions.

[0397] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0398] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0399] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0400] [Second embodiment]

[0401] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0402] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0403] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0404] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0405] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0406] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0407] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0408] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0409] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0410] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0411] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0412] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0413] An embodiment for implementing the present invention includes the following elements.

[0414] 1. Method of inputting data from the server:

[0415] The server receives the story data and the child's characteristic data.

[0416] Story data includes plot and character information for a story.

[0417] The child's characteristic data includes attribute information such as age, sex, and address.

[0418] 2. How choices are generated:

[0419] Based on the input data, the server determines the actions of the characters according to the content of the story.

[0420] Generate options.

[0421] Appropriate options can be generated taking into account the child's characteristic data.

[0422] 3. How to present options:

[0423] The terminal displays the generated option list on the screen.

[0424] The child suggests a choice from a presented list of options.

[0425] 4. Means of sending your choice:

[0426] The terminal transmits the child's selection to the server.

[0427] 5. Narrative generation and video data display:

[0428] The server generates a story based on the selections received.

[0429] The server generates video data in accordance with the generated story.

[0430] The terminal displays the generated video data on a display.

[0431] In this way, the server and the device work together to control the progression of the story and the actions of the characters, allowing children to actively participate. Children can suggest options, the story progresses based on those options, and videos are displayed, allowing them to experience the story through their own choices.

[0432] The process flow will be explained below.

[0433] Step 1: Input data from the server

[0434] The server receives the story data and the child's characteristic data.

[0435] The story data includes information about the plot and characters of the story, and the child characteristic data includes attribute information such as age and gender.

[0436] Step 2: Generate options

[0437] The server generates options for the character's actions according to the content of the story, based on the received story data and the child's characteristic data.

[0438] Appropriate options can be generated taking into account the child's attribute information.

[0439] Step 3: Present the options

[0440] The terminal displays the generated option list on the screen.

[0441] The child suggests a choice from a presented list of options.

[0442] Step 4: Submit your selections

[0443] The terminal transmits the child's selection to the server.

[0444] Step 5: Generate narrative and display video data

[0445] The server generates a story based on the selections received.

[0446] The server generates video data in accordance with the generated story.

[0447] The terminal displays the generated video data on a display.

[0448] Step 6: Story progression and repetition

[0449] While the child enjoys the story presented, they are then asked to make suggestions for the next step.

[0450] The processes from step 2 to step 5 above are repeated as the story progresses.

[0451] Output Condition:

[0452] Server: Receives story data and child characteristic data, generates options, generates stories, and generates video data

[0453] Terminal: Display list of options, send selection, display video data

[0454] User (child): Suggestions for choices, story progression

[0455] Example 1

[0456] Next, a description will be given of Example 1. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0457] In conventional video streaming systems for children, it was difficult for users to interactively participate in the development of the story, and the story often progressed one-sidedly. Therefore, a system that allows children to make their own choices and experience the progression of the story is desired. There is also a demand for a system that can provide options according to the characteristics of each individual child, but this has not been fully realized in conventional technologies.

[0458] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0459] In this invention, the server includes a means for inputting story data and child characteristic data, a means for generating options based on the input data by a generative AI model, and a means for generating story and video data. This allows options suited to the child's characteristics to be provided, and enables the user to experience the development of the story based on their own selection.

[0460] A "server" is a central management system that stores and processes data over a network.

[0461] "Story data" refers to data that includes information related to the progression of the story, such as the story plot and character information.

[0462] "Child characteristic data" refers to data including the child's attribute information, such as age, sex, and address.

[0463] A "generative AI model" is a model that uses artificial intelligence to analyze data and generate options, stories, etc.

[0464] The "option generation means" is a mechanism that generates options from which the user can choose based on input data.

[0465] A "choice list" is a collection of multiple options that are displayed on the screen for the user to select from.

[0466] A "terminal" is a device operated by a user, and is a device that displays options and transmits and receives data.

[0467] A "user selection suggestion means" is a mechanism that receives selection input from a user and transmits it to a server.

[0468] The "means for generating story and video data" is a mechanism for generating a new story based on the user's selection and creating video data corresponding to that story.

[0469] The "video data display means" is a mechanism for displaying the generated video data on the display of the terminal.

[0470] The "means for generating options taking into consideration characteristic data" is a mechanism for generating appropriate options based on the child's characteristic data.

[0471] Overall overview

[0472] The present invention relates to an interactive video streaming system for children, and provides a system in which users can participate in the progression of a story. This system dynamically generates story data and video data by making full use of a server, a terminal, and a generative AI model, and provides an interactive experience for users.

[0473] server

[0474] The server receives story data and child characteristic data. For example, it receives data including the plot and character information of the story, and attribute information such as the child's age, gender, and address. The server also analyzes these data using a generative AI model and generates options based on the input data. The server then generates a new story based on the user's selection and creates video data corresponding to the story.

[0475] Hardware / Software used:

[0476] Server equipment (high performance computers)

[0477] Generative AI models (e.g. ChatGPT, BERT)

[0478] Database (e.g. MySQL, PostgreSQL)

[0479] Terminal

[0480] The terminal is a device operated by the user, and its role is to display the list of options sent from the server on the screen. When the user selects an option, the information is sent to the server in real time. In addition, the generated video data is displayed to the user.

[0481] Hardware / Software used:

[0482] Tablets and PCs (user-operated devices)

[0483] Display (for video display)

[0484] Internet connection

[0485] User

[0486] Users (usually children) operate the device, choose the action they want from the options displayed, and experience the progression of the story as a result. This allows users to experience a unique story based on their own choices.

[0487] Examples of concrete examples and prompts

[0488] Example 1:

[0489] The server receives the plot data of "The Story of a Hero Defeating a Dragon" and the characteristic data of "a 7-year-old boy," and generates the options "attack," "run away," and "use an item." The device displays these options on the screen, and the user selects "attack," which is sent to the server. Based on this selection, the server generates a scene in which the hero attacks the dragon, and creates a corresponding animated video. The device displays this video.

[0490] Example 2:

[0491] The server receives the data "A story about a wizard choosing a spell" and the characteristic data "12-year-old girl," and generates the options "Fire magic," "Water magic," and "Wind magic." The device displays these options on the screen, and the user selects "Fire magic," which is sent to the server. Based on this selection, the server generates a scene in which the wizard casts a fire spell, and creates a corresponding animated video. The device displays this video.

[0492] Examples of prompt statements

[0493] Prompt statement example 1:

[0494] User: 7 year old boy

[0495] Story scene: The hero confronts the dragon

[0496] Choices: Attack, Run, Use an item

[0497] Output format: Animated video with the corresponding story of the scene's continuation

[0498] Prompt statement example 2:

[0499] User: 12 year old girl

[0500] Story scene: The wizard chooses a spell

[0501] Choices: Fire magic, Water magic, Wind magic

[0502] Output format: Animated video with the corresponding story of the scene's continuation

[0503] By feeding these prompts into a generative AI model, the system can provide the interactive narrative experience described above.

[0504] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0505] Step 1: Receiving data

[0506] The server is responsible for receiving story data and child characteristic data. The story data includes story plot and character information, and the child characteristic data includes attribute information such as age, gender, address, etc. The server obtains these data from an external database or a user device. It receives the story data and characteristic data as input and stores it internally.

[0507] Specific behavior:

[0508] The server retrieves plot data for "The Story of a Hero Slaying a Dragon."

[0509] The server receives the characteristic data of "7-year-old boy."

[0510] Input: Story data, child characteristics data

[0511] Output: Data stored inside the server

[0512] Step 2: Generate options

[0513] The server uses a generative AI model to generate options based on the received story data and the child's characteristics data. In the process, the server analyzes the input data and creates multiple options to present to the user.

[0514] Specific behavior:

[0515] The server recognizes this as a scene in which a hero confronts a dragon, and generates age-appropriate options such as "attack," "run away," or "use an item."

[0516] Input: Story data, child characteristics data

[0517] Output: Choice list (response)

[0518] Step 3: Displaying the options

[0519] The terminal has the role of displaying the option list sent from the server on the screen. The terminal receives the option list returned from the server and displays it in a format that is easy for the user to see.

[0520] Specific behavior:

[0521] The tablet device will display the options "attack," "run away," and "use item" on the screen.

[0522] Input: Choice list (response from server)

[0523] Output: The choices displayed on the screen.

[0524] Step 4: Sending user selections

[0525] The user selects one of the options displayed on the screen. The user's choice is sent to the server via the terminal. The terminal receives the user's input and forwards it to the server.

[0526] Specific behavior:

[0527] The child taps to select "attack," and the tablet sends the selection data to the server.

[0528] Input: User's choice

[0529] Output: Selection data sent to the server

[0530] Step 5: Generate a story

[0531] The server generates new narrative scenes based on the received choices, using a generative AI model to create a narrative progression based on the choices. In this step, the storyline is updated based on the choices and the next scenes are generated.

[0532] Specific behavior:

[0533] Based on the server's choice of "attack," a scene is generated in which the hero attacks the dragon.

[0534] Input: User selected data

[0535] Output: Next story scene

[0536] Step 6: Generate video data

[0537] The server generates video data based on the generated story, and uses a generative AI model to create animated videos that correspond to the scenes in the story. This dynamic process renders visual data that matches the story.

[0538] Specific behavior:

[0539] The server creates animation data based on the "scene of the hero attacking the dragon" and encodes it in video format.

[0540] Input: Next story scene

[0541] Output: Video data

[0542] Step 7: View video data

[0543] The terminal displays the video data sent from the server to the user. The terminal receives the video data and plays it to visually present the progress of the story to the user.

[0544] Specific behavior:

[0545] The tablet device plays the animation video received from the server.

[0546] Input: Video data

[0547] Output: Video played on screen

[0548] Through the above processing steps, a child can interactively progress through the story and enjoy a unique story experience based on his or her own choices.

[0549] (Application example 1)

[0550] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0551] There is a demand for video streaming services for children to provide a more interactive and educational experience by dynamically changing the progression of the story according to the viewer's (child's) choices. However, current systems do not generate stories that correspond to the individual attributes of each child, nor do they present options, and the content is uniform. This makes it difficult to provide content that is appropriate for each child, resulting in low immersion and educational effectiveness.

[0552] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0553] In this invention, the server includes a means for inputting story data and child characteristic data, a means for generating options based on the input data by a generative AI model, and a means for displaying a list of options on a terminal, thereby enabling an interactive story progression based on the individual attributes of the child.

[0554] A "generative AI model" is an artificial intelligence model that generates dynamic narratives and options based on given data.

[0555] An "interactive story application system" is a system that provides an application in which the progress of a story changes according to the user's choices.

[0556] "Story data" is data necessary for the progression of the story, such as the story plot and character information.

[0557] "Child characteristic data" refers to data including child attribute information such as age, sex, and address.

[0558] The "server" is a computer system that receives story data and child characteristic data, and generates options and stories based on a generative AI model.

[0559] The "option generation means" is a mechanism for generating multiple options based on input data.

[0560] A "terminal" is a device that allows a user to view and select options.

[0561] The "means for receiving selection suggestions" is a mechanism for transmitting the options selected by the user to the server.

[0562] "Means for generating story and video data" refers to a mechanism for creating story progression and video data based on user selection.

[0563] "Video data display means" refers to a mechanism for showing the generated video data to the user.

[0564] This invention shows a concrete method for realizing an interactive storytelling application system using a generative AI model. The system mainly consists of three elements: a server, a terminal, and a user.

[0565] server

[0566] The server receives story data and child characteristics data as input. It uses a web framework such as Django and Python as the programming language. The server uses this data to generate interactive options using a generative AI model and sends the option list to the terminal.

[0567] Story data refers to data necessary for the progression of the story, such as the story plot and character information.

[0568] The child's characteristic data is data including the child's attribute information such as age, sex, and address.

[0569] The server also receives the options selected by the user and generates new story progression and video data based on the options selected by the user. The generated video data is sent to the terminal.

[0570] Terminal

[0571] The terminal refers to a smartphone or tablet, and is built using a framework such as React Native. The terminal displays the list of options received from the server, sends the user's selection to the server, and displays new video data sent from the server.

[0572] User

[0573] The user is the child and makes his / her selection from a displayed list of options, the content of which is appropriately customized based on the child's characteristic data.

[0574] Examples

[0575] For example, if a user (child) selects "Character A climbs a tree," the server generates a new story progression based on this selection, creates video data of "the view that Character A sees from the top of the tree," and sends it to the terminal. On the other hand, if the user selects "Character B crosses the river," the story progresses to "Character B meets a new friend on the other side of the river," and video data for that is generated.

[0576] Examples of prompt statements

[0577] Generate interactive story choices based on the following criteria:

[0578] Child's demographic information: Age 7, gender male, address Tokyo

[0579] Story ID is 1

[0580] Please let me know what option is appropriate.

[0581] In this way, generative AI models can be used to provide a personalized, interactive storytelling experience for each child, enhancing educational and entertainment value.

[0582] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0583] Step 1:

[0584] The server receives the story data and the child's characteristics data. This data includes the story plot, character information, the child's age, gender, address, etc. Based on the input story data and child's characteristics data, it extracts relevant information from the database and formats it in the format required for subsequent processing. The output at this point is the formatted story data and child's characteristics data.

[0585] Step 2:

[0586] The server uses a generative AI model to generate options based on the input data. Specifically, the story data and the child's characteristics data are input into the AI ​​model, and a list of options is generated from the model. Data processing involves filtering so that options appropriate to the child's attributes are selected. The output is a list of options.

[0587] Step 3:

[0588] The server sends a list of options to the terminal. The sent list of options is displayed on the terminal. The user (child) sees this list and makes a selection. The output is the list of options displayed on the terminal.

[0589] Step 4:

[0590] The user selects one item from a list of options, and the device sends the selection to the server. The input is the user's selection, and the resulting concrete option is sent to the server. The output is the selection sent to the server.

[0591] Step 5:

[0592] The server generates story progression and new video data based on the user's selection. Based on the content of the choice, the server extracts the relevant story progression from the story database and inputs it into the generative AI model. Video data is generated based on the generated data. The output is the new story progression and video data.

[0593] Step 6:

[0594] The server sends the generated video data to the terminal. The terminal displays the received video data on the screen. The user can watch the progress of the new story. The input is the video data sent from the server, and the output is the video displayed on the terminal.

[0595] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0596] An embodiment for implementing the present invention includes the following elements.

[0597] 1. Method of inputting data from the server:

[0598] The server receives the story data, the child's characteristic data, and the user's emotion information.

[0599] The story data includes information about the plot and characters of the story, and the child characteristic data includes attribute information such as age and gender.

[0600] The user's emotional information is recognized from voice, facial expressions, etc. and processed by an emotion engine.

[0601] 2. How choices are generated:

[0602] The server generates options for the character's actions according to the content of the story based on the received story data, the child's characteristic data, and the user's emotion information.

[0603] The emotion engine can take into account the user's emotion information to generate appropriate options.

[0604] 3. How to present options:

[0605] The terminal displays the generated option list on the screen.

[0606] The child suggests a choice from a presented list of options.

[0607] 4. Means of sending your choice:

[0608] The terminal transmits the child's selection to the server.

[0609] 5. Narrative generation and video data display:

[0610] The server generates a story based on the selections received.

[0611] The server generates video data in accordance with the generated story.

[0612] The terminal displays the generated video data on a display.

[0613] 6. Emotion engine processing means:

[0614] The server receives the user's emotion information and processes it using an emotion engine.

[0615] The emotion engine recognizes the user's emotional information, such as voice and facial expressions, and creates a story based on that information.

[0616] Adjust the progression and character behavior.

[0617] In this way, the server and the device work together to control the progression of the story and the actions of the characters, allowing the child to be actively involved. The child suggests options, the story progresses based on those options, and the videos are displayed, allowing the child to experience the story based on their own choices and emotions.

[0618] The process flow will be explained below.

[0619] Step 1: Input data from the server

[0620] The server receives the story data, the child's behavior data, and the user's emotion information.

[0621] The story data includes information about the plot and characters of the story, and the child characteristic data includes attribute information such as age and gender.

[0622] The user's emotional information is recognized from voice, facial expressions, etc., and processed by the emotion engine. Step 2: Generation of options

[0623] The server generates options for the character's actions according to the content of the story based on the received story data, the child's characteristic data, and the user's emotion information.

[0624] The emotion engine can generate appropriate options by taking into account the user's emotion information. Step 3: Present options

[0625] The terminal displays the generated option list on the screen.

[0626] The child suggests a choice from a presented list of options.

[0627] Step 4: Submit your selections

[0628] The terminal transmits the child's selection to the server.

[0629] Step 5: Generate narrative and display video data

[0630] The server generates a story based on the selections received.

[0631] The server generates video data in accordance with the generated story.

[0632] The terminal displays the generated video data on a display.

[0633] Step 6: Emotion Engine Processing

[0634] The server receives the user's emotion information and processes it using an emotion engine.

[0635] The emotion engine recognizes the user's emotional information, such as voice and facial expressions, and adjusts the story progress and character behavior based on that information.

[0636] Through the above processing steps, the server and the device work together to control the progress of the story and the actions of the characters, allowing the child to actively participate. The child proposes options, the story progresses based on those options, and the videos are displayed, allowing the child to experience the story based on his or her own choices and emotions.

[0637] Example 2

[0638] Next, a description will be given of Example 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0639] In conventional interactive video systems for children, the progression of the story and the actions of the characters are prescribed, making it difficult to respond to the interests and emotions of individual children. In addition, there was a problem in that it was not possible to dynamically generate a story that incorporates changes in the user's emotions.

[0640] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an input means for story data, child attribute information, and user emotion information, a generation means for generating options based on the input data by a generative AI, and a processing means for an emotion engine that recognizes the user emotion information and adjusts the progress of the story and the behavior of the characters. This enables a personalized story progress based on the child attribute information and the user emotion.

[0641] "Story data" is digital data that includes information about the story plot and characters.

[0642] "Child attribute information" is data indicating the child's attributes such as age and sex.

[0643] "User emotion information" is data indicating the user's emotional state obtained from the user's voice, facial expression, etc.

[0644] The "input means" refers to a means by which the server receives story data, child attribute information, and user emotion information.

[0645] "Generative AI" is an artificial intelligence technology that generates story options or the story itself based on input data.

[0646] The "option generating means" is a means for generating options based on received story data, child attribute information, and user emotion information.

[0647] An "emotion engine" is software that recognizes the user's emotional information and adjusts the progress of the story and the behavior of characters.

[0648] "Video data" refers to animation and video data that is generated in accordance with the progression of the story.

[0649] A "Terminal" is a device (e.g., tablet, smartphone) that a User can interact with.

[0650] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0651] This invention relates to an interactive video system for children that incorporates generative AI to determine the ending of a story and the actions of characters. This system mainly operates in cooperation with a server and terminals.

[0652] The server accepts input of story data, child attribute information, and user emotion information. Story data includes the story plot and character information. Child attribute information includes information such as age and gender. User emotion information is obtained from voice and facial expressions, and processed by an emotion engine. Based on this input data, the server uses generative AI to generate appropriate options.

[0653] The generated options are sent to the terminal, which displays them on the screen. The user selects one of the options presented and sends it to the server via the terminal. The server generates a story based on the received selection and creates video data to match it. The video data is then sent back to the terminal and displayed to the user.

[0654] During this entire process, the server uses an emotion engine to recognize the user's emotional information in real time and adjust the story progression and character behavior, thus enabling a personalized interactive video experience.

[0655] As a concrete example, the server recognizes the story data of "Little Red Riding Hood," the attribute information of a 7-year-old girl, and the user's smile. After the emotion engine recognizes the user's smile, it generates an option during the story that reads, "Please select an animal that Little Red Riding Hood will encounter in the forest." This is displayed on the device's display, and the child selects "rabbit." Upon receiving this selection, the server generates a "scene in which Little Red Riding Hood talks to the rabbit," creates video data, and displays it on the device.

[0656] Examples of prompt statements

[0657] "Generate three options for animals that Little Red Riding Hood encounters in the forest for a 7-year-old girl to enjoy. If you detect a smile from the user, make it more fun."

[0658] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0659] Step 1: Data entry

[0660] The server inputs story data, the child's attribute information, and the user's emotion information. Specifically, as story data input, plot and character information for a story such as "Little Red Riding Hood" is obtained from a database. As the child's attribute information, data that the child is a "7-year-old girl" is received via sensors and cameras. As the user's emotion information, real-time voice and facial expression data is obtained from the camera and microphone, and this is processed by the emotion engine.

[0661] input:

[0662] Story data (e.g. the plot of "Little Red Riding Hood")

[0663] Child demographic information (e.g. 7 years old, girl)

[0664] User emotion information (e.g. smile detection)

[0665] output:

[0666] Integrated data passed to the server

[0667] Step 2: Generate options

[0668] The server uses a generative AI model to generate appropriate options based on the received data. A prompt is input to the generative AI model, for example, "Generate three actions for a character that a 7-year-old girl can enjoy. If the user's smile is recognized, make it more enjoyable." The emotion engine takes into account the user's emotional information and generates appropriate options.

[0669] input:

[0670] Integrated data (story data, children's attributes, users' emotions)

[0671] output:

[0672] Generated choice list (e.g. 1. rabbit 2. squirrel 3. bird)

[0673] Step 3: Present the options

[0674] The device displays the generated list of options on the screen. Specifically, the tablet display shows, "Please choose an animal that Little Red Riding Hood will meet: 1. Rabbit 2. Squirrel 3. Small bird." The user, a child, taps the screen to make a selection.

[0675] input:

[0676] Generated Choice List

[0677] output:

[0678] Choice list displayed on the device

[0679] Step 4: Submit your selections

[0680] The terminal transmits the data selected by the child to the server. Specifically, when the user selects "rabbit," the selection information is transmitted to the server.

[0681] input:

[0682] Select a child (e.g. "rabbit")

[0683] output:

[0684] Selection data sent to the server

[0685] Step 5: Generate a story

[0686] The server generates a story based on the selections it receives: for example, a scene where Little Red Riding Hood meets and talks with the Rabbit, and creates video data to go with it. An emotion engine also helps at this stage, providing more personalized content.

[0687] input:

[0688] Selection data (e.g. "rabbit")

[0689] output:

[0690] Generated story (e.g. the scene where Little Red Riding Hood talks to the rabbit)

[0691] Step 6: Generate and display video data

[0692] The server generates video data in accordance with the progress of the story and sends it to the device. The device then displays the received video data on the display. Specifically, the tablet plays an animation of Little Red Riding Hood meeting and talking to the Rabbit.

[0693] input:

[0694] Generated story (e.g. the scene where Little Red Riding Hood talks to the rabbit)

[0695] output:

[0696] Generated video data, video playback on device

[0697] (Application example 2)

[0698] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0699] Traditional children's content lacked the ability to adjust the progress of the story and the actions of characters based on the individual attributes and emotions of the viewer, limiting the experience of viewers actively participating in the story. In addition, since only pre-fixed story developments could be selected, it was often unable to respond to the interests and concerns of the viewer. As a result, it was difficult to stimulate the viewer's motivation to learn and creativity.

[0700] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0701] In this invention, the server includes a means for inputting story data, child characteristic data, and user emotion information, a means for generating options based on the input data by a generative AI, and a means for adjusting the progress of the story and the behavior of the characters based on the user emotion information. This makes it possible to dynamically progress the story based on the individual attributes and emotions of the viewers.

[0702] "Story ending" and "character actions" refer to the final outcome or action of a character or plot in a scene or episode within a story.

[0703] "Generative AI" refers to artificial intelligence technology that automatically generates story development and character movements based on input data.

[0704] "Content delivery system" refers to the entire system for providing digital content to users, and in this context refers specifically to a system for delivering interactive stories for children.

[0705] "Story data" refers to data that includes plot and character information necessary for the progression of the story.

[0706] "Characteristic data" refers primarily to attribute data relating to a child's age and gender.

[0707] "Emotion information" is information that indicates the user's emotional state obtained from voice, facial expressions, etc.

[0708] "Options generation means" refers to a method or device for automatically generating options to be provided to a user.

[0709] An "option list" is a list of multiple options that a child can choose from as the story progresses.

[0710] "Means for receiving and transmitting suggestions" refers to a method or device for transmitting the child's selections to the server.

[0711] "Video data" refers to a sequence of image data intended for visual display and usually includes audio.

[0712] "Means for adjusting story progression" refers to a method or device for dynamically adjusting story progression or character behavior based on user input or emotional information.

[0713] "Display means" refers to a method or device for displaying the generated video or option list on a user terminal.

[0714] This invention is an interactive children's content distribution system in which a server and a terminal work in cooperation with each other, and includes the following elements:

[0715] Preparing the server and terminal

[0716] The server mainly receives story data, child characteristic data, and user emotion information. The server uses MongoDB or Firebase as a database, and Google Cloud Vision API or IBM Watson as an emotion recognition engine. The server is built using Node.js or Python (Flask / Django framework).

[0717] The device is mainly intended to be a smartphone, and will be built using React Native and Flutter as the front-end platform. The device is equipped with a camera and microphone, and will capture the user's voice and facial expressions as emotional information.

[0718] Server Processing

[0719] The server receives the following data as input:

[0720] Narrative Data: information about the plot and characters of a story.

[0721] Child characteristics data: demographic information such as age and gender.

[0722] User emotion information: voice and facial expression data.

[0723] Based on this data, the generative AI generates options. The generated options are sent to the device. When the user's choice is sent from the device to the server, the server generates a story and video data based on the choice. The story and video are generated using libraries such as FFmpeg.

[0724] Terminal handling

[0725] The device displays a list of options received from the server to the user. The user selects from the options and the selection is sent to the server. The device also obtains the user's emotional information (voice and facial expressions) and sends it to the server in real time. This allows the server-side emotion engine to adjust the progress of the story and the character's actions.

[0726] Examples

[0727] For example, suppose a story about a "night adventure" is displayed. In this story, the child is presented with the choice of "preparing before leaving or not." If the child chooses "preparing," the story switches to a preparation scene. In addition, the device's camera and microphone capture the child's smile and tone of voice, and transmit this emotional information to the server. Depending on the child's smile, the server can change the setting of the story to a brighter, more enjoyable development.

[0728] Example prompt:

[0729] This is the story of a brave knight who goes on an adventure. The choices you make will determine how the adventure progresses. Let's start the adventure together!

[0730] In this way, the server and device work together to enable children to dynamically experience a story based on their own choices and emotions.

[0731] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0732] Step 1:

[0733] The server receives story data, child's characteristics data, and user's emotion information as input. This includes story plot and character information. Emotion information is voice data and facial expression data obtained from the terminal. These data are stored in MongoDB and Firebase.

[0734] Input: Story data, child's characteristics data, user's emotion information

[0735] Output: A record of the stored data

[0736] Specific behavior:

[0737] The server receives the data sent from the terminal via the API.

[0738] A process is executed to store the received data in a database.

[0739] Step 2:

[0740] The generative AI generates options based on the story data and the child's characteristics and emotional information. The generated options are used by the child to decide the progress of the story.

[0741] Input: Story data, child's characteristics data, user's emotion information

[0742] Output: Choice list

[0743] Specific behavior:

[0744] The generative AI on the server refers to the stored data and generates appropriate options.

[0745] The emotion engine analyzes emotional information and reflects it in generating options.

[0746] Step 3:

[0747] The device displays the list of options received from the server to the user. The user (child) selects the option they want from the presented options by tapping on it.

[0748] Input: Choice list

[0749] Output: User's choice

[0750] Specific behavior:

[0751] The front-end application displays the list of options in a GUI.

[0752] It recognizes the user's tapping actions and sends the selection to the server.

[0753] Step 4:

[0754] The server receives the user's selections from the terminal and generates the next story portion based on the selections, including video data corresponding to the selections.

[0755] Input: User's choice

[0756] Output: Generated story text, video data

[0757] Specific behavior:

[0758] The server receives the user's selections and updates the story data.

[0759] Video generation processing according to the progression of the story is carried out using FFmpeg.

[0760] Step 5:

[0761] The emotion engine analyzes the user's real-time emotion information and adapts the story progression and character behavior based on that information. Revised story and video data are generated.

[0762] Input: User emotion information

[0763] Output: Revised story text, video data

[0764] Specific behavior:

[0765] The server analyzes the voice and facial expression data transmitted in real time.

[0766] Based on the analysis results, the story's progression and character behavior are dynamically revised.

[0767] Step 6:

[0768] The terminal receives the generated video data from the server and displays it on the display, allowing the user to watch the progress of the new story.

[0769] Input: Generated story text, video data

[0770] Output: Narrative video shown on the display

[0771] Specific behavior:

[0772] The application on the terminal receives streaming video data from the server.

[0773] The received video is displayed on the user interface and played.

[0774] This allows users to interactively experience the story based on their choices and emotions.

[0775] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0776] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0777] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0778] [Third embodiment]

[0779] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0780] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0781] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0782] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0783] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0784] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0785] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0786] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0787] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0788] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0789] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0790] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server", and the headset type terminal 314 will be referred to as the "terminal".

[0791] An embodiment for implementing the present invention includes the following elements.

[0792] 1. Method of inputting data from the server:

[0793] The server receives the story data and the child's characteristic data.

[0794] Story data includes plot and character information for a story.

[0795] The child's characteristic data includes attribute information such as age, sex, and address.

[0796] 2. How choices are generated:

[0797] Based on the input data, the server determines the actions of the characters according to the content of the story.

[0798] Generate options.

[0799] Appropriate options can be generated taking into account the child's characteristic data.

[0800] 3. How to present options:

[0801] The terminal displays the generated option list on the screen.

[0802] The child suggests a choice from a presented list of options.

[0803] 4. Means of sending your choice:

[0804] The terminal transmits the child's selection to the server.

[0805] 5. Narrative generation and video data display:

[0806] The server generates a story based on the selections received.

[0807] The server generates video data in accordance with the generated story.

[0808] The terminal displays the generated video data on a display.

[0809] In this way, the server and the device work together to control the progression of the story and the actions of the characters, allowing children to actively participate. Children can suggest options, the story progresses based on those options, and videos are displayed, allowing them to experience the story through their own choices.

[0810] The process flow will be explained below.

[0811] Step 1: Input data from the server

[0812] The server receives the story data and the child's characteristic data.

[0813] The story data includes information about the plot and characters of the story, and the child characteristic data includes attribute information such as age and gender.

[0814] Step 2: Generate options

[0815] The server generates options for the character's actions according to the content of the story, based on the received story data and the child's characteristic data.

[0816] Appropriate options can be generated taking into account the child's attribute information.

[0817] Step 3: Present the options

[0818] The terminal displays the generated option list on the screen.

[0819] The child suggests a choice from a presented list of options.

[0820] Step 4: Submit your selections

[0821] The terminal transmits the child's selection to the server.

[0822] Step 5: Generate narrative and display video data

[0823] The server generates a story based on the selections received.

[0824] The server generates video data in accordance with the generated story.

[0825] The terminal displays the generated video data on a display.

[0826] Step 6: Story progression and repetition

[0827] While the child enjoys the story presented, they are then asked to make suggestions for the next step.

[0828] The processes from step 2 to step 5 above are repeated as the story progresses.

[0829] Output Condition:

[0830] Server: Receives story data and child characteristic data, generates options, generates stories, and generates video data

[0831] Terminal: Display list of options, send selection, display video data

[0832] User (child): Suggestions for choices, story progression

[0833] Example 1

[0834] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[0835] In conventional video streaming systems for children, it was difficult for users to interactively participate in the development of the story, and the story often progressed one-sidedly. Therefore, a system that allows children to make their own choices and experience the progression of the story is desired. There is also a demand for a system that can provide options according to the characteristics of each individual child, but this has not been fully realized in conventional technologies.

[0836] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0837] In this invention, the server includes a means for inputting story data and child characteristic data, a means for generating options based on the input data by a generative AI model, and a means for generating story and video data. This allows options suited to the child's characteristics to be provided, and enables the user to experience the development of the story based on their own selection.

[0838] A "server" is a central management system that stores and processes data over a network.

[0839] "Story data" refers to data that includes information related to the progression of the story, such as the story plot and character information.

[0840] "Child characteristic data" refers to data including the child's attribute information, such as age, sex, and address.

[0841] A "generative AI model" is a model that uses artificial intelligence to analyze data and generate options, stories, etc.

[0842] The "option generation means" is a mechanism that generates options from which the user can choose based on input data.

[0843] A "choice list" is a collection of multiple options that are displayed on the screen for the user to select from.

[0844] A "terminal" is a device that is operated by a user and that displays options and transmits and receives data.

[0845] A "user selection suggestion means" is a mechanism that receives selection input from a user and transmits it to a server.

[0846] The "means for generating story and video data" is a mechanism for generating a new story based on the user's selection and creating video data corresponding to that story.

[0847] The "video data display means" is a mechanism for displaying the generated video data on the display of the terminal.

[0848] The "means for generating options taking into consideration characteristic data" is a mechanism for generating appropriate options based on the child's characteristic data.

[0849] Overall overview

[0850] The present invention relates to an interactive video streaming system for children, and provides a system in which users can participate in the progression of a story. This system dynamically generates story data and video data by making full use of a server, a terminal, and a generative AI model, and provides an interactive experience for users.

[0851] server

[0852] The server receives story data and child characteristic data. For example, it receives data including the plot and character information of the story, and attribute information such as the child's age, gender, and address. The server also analyzes these data using a generative AI model and generates options based on the input data. The server then generates a new story based on the user's selection and creates video data corresponding to the story.

[0853] Hardware / Software used:

[0854] Server equipment (high performance computers)

[0855] Generative AI models (e.g. ChatGPT, BERT)

[0856] Database (e.g. MySQL, PostgreSQL)

[0857] Terminal

[0858] The terminal is a device operated by the user, and its role is to display the list of options sent from the server on the screen. When the user selects an option, the information is sent to the server in real time. In addition, the generated video data is displayed to the user.

[0859] Hardware / Software used:

[0860] Tablets and PCs (user-operated devices)

[0861] Display (for video display)

[0862] Internet connection

[0863] User

[0864] Users (usually children) operate the device, choose the action they want from the options displayed, and experience the progression of the story as a result. This allows users to experience a unique story based on their own choices.

[0865] Examples of concrete examples and prompts

[0866] Example 1:

[0867] The server receives the plot data of "The Story of a Hero Defeating a Dragon" and the characteristic data of "a 7-year-old boy," and generates the options "attack," "run away," and "use an item." The device displays these options on the screen, and the user selects "attack," which is sent to the server. Based on this selection, the server generates a scene in which the hero attacks the dragon, and creates a corresponding animated video. The device displays this video.

[0868] Example 2:

[0869] The server receives the data "A story about a wizard choosing a spell" and the characteristic data "12-year-old girl," and generates the options "Fire magic," "Water magic," and "Wind magic." The device displays these options on the screen, and the user selects "Fire magic," which is sent to the server. Based on this selection, the server generates a scene in which the wizard casts a fire spell, and creates a corresponding animated video. The device displays this video.

[0870] Examples of prompt statements

[0871] Prompt statement example 1:

[0872] User: 7 year old boy

[0873] Story scene: The hero confronts the dragon

[0874] Choices: Attack, Run, Use an item

[0875] Output format: Animated video with the corresponding story of the scene's continuation

[0876] Prompt statement example 2:

[0877] User: 12 year old girl

[0878] Story scene: The wizard chooses a spell

[0879] Choices: Fire magic, Water magic, Wind magic

[0880] Output format: Animated video with the corresponding story of the scene's continuation

[0881] By feeding these prompts into a generative AI model, the system can provide the interactive narrative experience described above.

[0882] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0883] Step 1: Receiving data

[0884] The server is responsible for receiving story data and child characteristic data. The story data includes story plot and character information, and the child characteristic data includes attribute information such as age, gender, address, etc. The server obtains these data from an external database or a user device. It receives the story data and characteristic data as input and stores it internally.

[0885] Specific behavior:

[0886] The server retrieves plot data for "The Story of a Hero Slaying a Dragon."

[0887] The server receives the characteristic data of "7-year-old boy."

[0888] Input: Story data, child characteristics data

[0889] Output: Data stored inside the server

[0890] Step 2: Generate options

[0891] The server uses a generative AI model to generate options based on the received story data and the child's characteristics data. In the process, the server analyzes the input data and creates multiple options to present to the user.

[0892] Specific behavior:

[0893] The server recognizes this as a scene in which a hero confronts a dragon, and generates age-appropriate options such as "attack," "run away," or "use an item."

[0894] Input: Story data, child characteristics data

[0895] Output: Choice list (response)

[0896] Step 3: Displaying the options

[0897] The terminal has the role of displaying the option list sent from the server on the screen. The terminal receives the option list returned from the server and displays it in a format that is easy for the user to see.

[0898] Specific behavior:

[0899] The tablet device will display the options "attack," "run away," and "use item" on the screen.

[0900] Input: Choice list (response from server)

[0901] Output: The choices displayed on the screen.

[0902] Step 4: Sending user selections

[0903] The user selects one of the options displayed on the screen. The user's choice is sent to the server via the terminal. The terminal receives the user's input and forwards it to the server.

[0904] Specific behavior:

[0905] The child taps to select "attack," and the tablet sends the selection data to the server.

[0906] Input: User's choice

[0907] Output: Selection data sent to the server

[0908] Step 5: Generate a story

[0909] The server generates new narrative scenes based on the received choices, using a generative AI model to create a narrative progression based on the choices. In this step, the storyline is updated based on the choices and the next scenes are generated.

[0910] Specific behavior:

[0911] Based on the server's choice of "attack," a scene is generated in which the hero attacks the dragon.

[0912] Input: User selected data

[0913] Output: Next story scene

[0914] Step 6: Generate video data

[0915] The server generates video data based on the generated story, and uses a generative AI model to create animated videos that correspond to the scenes in the story. This dynamic process renders visual data that matches the story.

[0916] Specific behavior:

[0917] The server creates animation data based on the "scene of the hero attacking the dragon" and encodes it in video format.

[0918] Input: Next story scene

[0919] Output: Video data

[0920] Step 7: View video data

[0921] The terminal displays the video data sent from the server to the user. The terminal receives the video data and plays it to visually present the progress of the story to the user.

[0922] Specific behavior:

[0923] The tablet device plays the animation video received from the server.

[0924] Input: Video data

[0925] Output: Video played on screen

[0926] Through the above processing steps, a child can interactively progress through the story and enjoy a unique story experience based on his or her own choices.

[0927] (Application example 1)

[0928] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0929] There is a demand for video streaming services for children to provide a more interactive and educational experience by dynamically changing the progression of the story according to the viewer's (child's) choices. However, current systems do not generate stories that correspond to the individual attributes of each child, nor do they present options, and the content is uniform. This makes it difficult to provide content that is appropriate for each child, resulting in low immersion and educational effectiveness.

[0930] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0931] In this invention, the server includes a means for inputting story data and child characteristic data, a means for generating options based on the input data by a generative AI model, and a means for displaying a list of options on a terminal, thereby enabling an interactive story progression based on the individual attributes of the child.

[0932] A "generative AI model" is an artificial intelligence model that generates dynamic narratives and options based on given data.

[0933] An "interactive story application system" is a system that provides an application in which the progress of a story changes according to the user's choices.

[0934] "Story data" is data necessary for the progression of the story, such as the story plot and character information.

[0935] "Child characteristic data" refers to data including child attribute information such as age, sex, and address.

[0936] The "server" is a computer system that receives story data and child characteristic data, and generates options and stories based on a generative AI model.

[0937] The "option generation means" is a mechanism for generating multiple options based on input data.

[0938] A "terminal" is a device that allows a user to view and select options.

[0939] The "means for receiving selection suggestions" is a mechanism for transmitting the options selected by the user to the server.

[0940] "Means for generating story and video data" refers to a mechanism for creating story progression and video data based on user selection.

[0941] "Video data display means" refers to a mechanism for showing the generated video data to the user.

[0942] This invention shows a concrete method for realizing an interactive storytelling application system using a generative AI model. The system mainly consists of three elements: a server, a terminal, and a user.

[0943] server

[0944] The server receives story data and child characteristics data as input. It uses a web framework such as Django and Python as the programming language. The server uses this data to generate interactive options using a generative AI model and sends the option list to the terminal.

[0945] Story data refers to data necessary for the progression of the story, such as the story plot and character information.

[0946] The child's characteristic data is data including the child's attribute information such as age, sex, and address.

[0947] The server also receives the options selected by the user and generates new story progression and video data based on the options selected by the user. The generated video data is sent to the terminal.

[0948] Terminal

[0949] The terminal refers to a smartphone or tablet, and is built using a framework such as React Native. The terminal displays the list of options received from the server, sends the user's selection to the server, and displays new video data sent from the server.

[0950] User

[0951] The user is the child and makes his / her selection from a displayed list of options, the content of which is appropriately customized based on the child's characteristic data.

[0952] Examples

[0953] For example, if a user (child) selects "Character A climbs a tree," the server generates a new story progression based on this selection, creates video data of "the view that Character A sees from the top of the tree," and sends it to the terminal. On the other hand, if the user selects "Character B crosses the river," the story progresses to "Character B meets a new friend on the other side of the river," and video data for that is generated.

[0954] Examples of prompt statements

[0955] Generate interactive story choices based on the following criteria:

[0956] Child's demographic information: Age 7, gender male, address Tokyo

[0957] Story ID is 1

[0958] Please let me know what option is appropriate.

[0959] In this way, generative AI models can be used to provide a personalized, interactive storytelling experience for each child, enhancing educational and entertainment value.

[0960] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0961] Step 1:

[0962] The server receives the story data and the child's characteristics data. This data includes the story plot, character information, the child's age, gender, address, etc. Based on the input story data and child's characteristics data, it extracts relevant information from the database and formats it in the format required for subsequent processing. The output at this point is the formatted story data and child's characteristics data.

[0963] Step 2:

[0964] The server uses a generative AI model to generate options based on the input data. Specifically, the story data and the child's characteristics data are input into the AI ​​model, and a list of options is generated from the model. Data processing involves filtering so that options appropriate to the child's attributes are selected. The output is a list of options.

[0965] Step 3:

[0966] The server sends a list of options to the terminal. The sent list of options is displayed on the terminal. The user (child) sees this list and makes a selection. The output is the list of options displayed on the terminal.

[0967] Step 4:

[0968] The user selects one item from a list of options, and the device sends the selection to the server. The input is the user's selection, and the resulting concrete option is sent to the server. The output is the selection sent to the server.

[0969] Step 5:

[0970] The server generates story progression and new video data based on the user's selection. Based on the content of the choice, the server extracts the relevant story progression from the story database and inputs it into the generative AI model. Video data is generated based on the generated data. The output is the new story progression and video data.

[0971] Step 6:

[0972] The server sends the generated video data to the terminal. The terminal displays the received video data on the screen. The user can watch the progress of the new story. The input is the video data sent from the server, and the output is the video displayed on the terminal.

[0973] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0974] An embodiment for implementing the present invention includes the following elements.

[0975] 1. Method of inputting data from the server:

[0976] The server receives the story data, the child's characteristic data, and the user's emotion information.

[0977] The story data includes information about the plot and characters of the story, and the child characteristic data includes attribute information such as age and gender.

[0978] The user's emotional information is recognized from voice, facial expressions, etc. and processed by an emotion engine.

[0979] 2. How choices are generated:

[0980] The server generates options for the character's actions according to the content of the story based on the received story data, the child's characteristic data, and the user's emotion information.

[0981] The emotion engine can take into account the user's emotion information to generate appropriate options.

[0982] 3. How to present options:

[0983] The terminal displays the generated option list on the screen.

[0984] The child suggests a choice from a presented list of options.

[0985] 4. Means of sending your choice:

[0986] The terminal transmits the child's selection to the server.

[0987] 5. Narrative generation and video data display:

[0988] The server generates a story based on the selections received.

[0989] The server generates video data in accordance with the generated story.

[0990] The terminal displays the generated video data on a display.

[0991] 6. Emotion engine processing means:

[0992] The server receives the user's emotion information and processes it using an emotion engine.

[0993] The emotion engine recognizes the user's emotional information, such as voice and facial expressions, and creates a story based on that information.

[0994] Adjust the progression and character behavior.

[0995] In this way, the server and the device work together to control the progression of the story and the actions of the characters, allowing the child to be actively involved. The child suggests options, the story progresses based on those options, and the videos are displayed, allowing the child to experience the story based on their own choices and emotions.

[0996] The process flow will be explained below.

[0997] Step 1: Input data from the server

[0998] The server receives the story data, the child's behavior data, and the user's emotion information.

[0999] The story data includes information about the plot and characters of the story, and the child characteristic data includes attribute information such as age and gender.

[1000] The user's emotional information is recognized from voice, facial expressions, etc., and processed by the emotion engine. Step 2: Generation of options

[1001] The server generates options for the character's actions according to the content of the story based on the received story data, the child's characteristic data, and the user's emotion information.

[1002] The emotion engine can generate appropriate options by taking into account the user's emotion information. Step 3: Present options

[1003] The terminal displays the generated option list on the screen.

[1004] The child suggests a choice from a presented list of options.

[1005] Step 4: Submit your selections

[1006] The terminal transmits the child's selection to the server.

[1007] Step 5: Generate narrative and display video data

[1008] The server generates a story based on the selections received.

[1009] The server generates video data in accordance with the generated story.

[1010] The terminal displays the generated video data on a display.

[1011] Step 6: Emotion Engine Processing

[1012] The server receives the user's emotion information and processes it using an emotion engine.

[1013] The emotion engine recognizes the user's emotional information, such as voice and facial expressions, and adjusts the story progress and character behavior based on that information.

[1014] Through the above processing steps, the server and the device work together to control the progress of the story and the actions of the characters, allowing the child to actively participate. The child proposes options, the story progresses based on those options, and the videos are displayed, allowing the child to experience the story based on his or her own choices and emotions.

[1015] Example 2

[1016] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1017] In conventional interactive video systems for children, the progression of the story and the actions of the characters are prescribed, making it difficult to respond to the interests and emotions of individual children. In addition, there was a problem in that it was not possible to dynamically generate a story that incorporates changes in the user's emotions.

[1018] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an input means for story data, child attribute information, and user emotion information, a generation means for generating options based on the input data by a generative AI, and a processing means for an emotion engine that recognizes the user emotion information and adjusts the progress of the story and the behavior of the characters. This enables a personalized story progress based on the child attribute information and the user emotion.

[1019] "Story data" is digital data that includes information about the story plot and characters.

[1020] "Child attribute information" is data indicating the child's attributes such as age and sex.

[1021] "User emotion information" is data indicating the user's emotional state obtained from the user's voice, facial expression, etc.

[1022] The "input means" refers to a means by which the server receives story data, child attribute information, and user emotion information.

[1023] "Generative AI" is an artificial intelligence technology that generates story options or the story itself based on input data.

[1024] The "option generating means" is a means for generating options based on received story data, child attribute information, and user emotion information.

[1025] An "emotion engine" is software that recognizes the user's emotional information and adjusts the progress of the story and the behavior of characters.

[1026] "Video data" refers to animation and video data that is generated in accordance with the progression of the story.

[1027] A "Terminal" is a device (e.g., tablet, smartphone) that a User can interact with.

[1028] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1029] This invention relates to an interactive video system for children that incorporates generative AI to determine the ending of a story and the actions of characters. This system mainly operates in cooperation with a server and terminals.

[1030] The server accepts input of story data, child attribute information, and user emotion information. Story data includes the story plot and character information. Child attribute information includes information such as age and gender. User emotion information is obtained from voice and facial expressions, and processed by an emotion engine. Based on this input data, the server uses generative AI to generate appropriate options.

[1031] The generated options are sent to the terminal, which displays them on the screen. The user selects one of the options presented and sends it to the server via the terminal. The server generates a story based on the received selection and creates video data to match it. The video data is then sent back to the terminal and displayed to the user.

[1032] During this entire process, the server uses an emotion engine to recognize the user's emotional information in real time and adjust the story progression and character behavior, thus enabling a personalized interactive video experience.

[1033] As a concrete example, the server recognizes the story data of "Little Red Riding Hood," the attribute information of a 7-year-old girl, and the user's smile. After the emotion engine recognizes the user's smile, it generates an option during the story that reads, "Please select an animal that Little Red Riding Hood will encounter in the forest." This is displayed on the device's display, and the child selects "rabbit." Upon receiving this selection, the server generates a "scene in which Little Red Riding Hood talks to the rabbit," creates video data, and displays it on the device.

[1034] Examples of prompt statements

[1035] "Generate three options for animals that Little Red Riding Hood encounters in the forest for a 7-year-old girl to enjoy. If you detect a smile from the user, make it more fun."

[1036] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1037] Step 1: Data entry

[1038] The server inputs story data, the child's attribute information, and the user's emotion information. Specifically, as story data input, plot and character information for a story such as "Little Red Riding Hood" is obtained from a database. As the child's attribute information, data that the child is a "7-year-old girl" is received via sensors and cameras. As the user's emotion information, real-time voice and facial expression data is obtained from the camera and microphone, and this is processed by the emotion engine.

[1039] input:

[1040] Story data (e.g. the plot of "Little Red Riding Hood")

[1041] Child demographic information (e.g. 7 years old, girl)

[1042] User emotion information (e.g. smile detection)

[1043] output:

[1044] Integrated data passed to the server

[1045] Step 2: Generate options

[1046] The server uses a generative AI model to generate appropriate options based on the received data. A prompt is input to the generative AI model, for example, "Generate three actions for a character that a 7-year-old girl can enjoy. If the user's smile is recognized, make it more enjoyable." The emotion engine takes into account the user's emotional information and generates appropriate options.

[1047] input:

[1048] Integrated data (story data, children's attributes, users' emotions)

[1049] output:

[1050] Generated choice list (e.g. 1. rabbit 2. squirrel 3. bird)

[1051] Step 3: Present the options

[1052] The device displays the generated list of options on the screen. Specifically, the tablet display shows, "Please choose an animal that Little Red Riding Hood will meet: 1. Rabbit 2. Squirrel 3. Small bird." The user, a child, taps the screen to make a selection.

[1053] input:

[1054] Generated Choice List

[1055] output:

[1056] Choice list displayed on the device

[1057] Step 4: Submit your selections

[1058] The terminal transmits the data selected by the child to the server. Specifically, when the user selects "rabbit," the selection information is transmitted to the server.

[1059] input:

[1060] Select a child (e.g. "rabbit")

[1061] output:

[1062] Selection data sent to the server

[1063] Step 5: Generate a story

[1064] The server generates a story based on the selections it receives: for example, a scene where Little Red Riding Hood meets and talks with the Rabbit, and creates video data to go with it. An emotion engine also helps at this stage, providing more personalized content.

[1065] input:

[1066] Selection data (e.g. "rabbit")

[1067] output:

[1068] Generated story (e.g. the scene where Little Red Riding Hood talks to the rabbit)

[1069] Step 6: Generate and display video data

[1070] The server generates video data in accordance with the progress of the story and sends it to the device. The device then displays the received video data on the display. Specifically, the tablet plays an animation of Little Red Riding Hood meeting and talking to the Rabbit.

[1071] input:

[1072] Generated story (e.g. the scene where Little Red Riding Hood talks to the rabbit)

[1073] output:

[1074] Generated video data, video playback on device

[1075] (Application example 2)

[1076] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1077] Traditional children's content lacked the ability to adjust the progress of the story and the actions of characters based on the individual attributes and emotions of the viewer, limiting the experience of viewers actively participating in the story. In addition, since only pre-fixed story developments could be selected, it was often unable to respond to the interests and concerns of the viewer. As a result, it was difficult to stimulate the viewer's motivation to learn and creativity.

[1078] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1079] In this invention, the server includes a means for inputting story data, child characteristic data, and user emotion information, a means for generating options based on the input data by a generative AI, and a means for adjusting the progress of the story and the behavior of the characters based on the user emotion information. This makes it possible to dynamically progress the story based on the individual attributes and emotions of the viewers.

[1080] "Story ending" and "character actions" refer to the final outcome or action of a character or plot in a scene or episode within a story.

[1081] "Generative AI" refers to artificial intelligence technology that automatically generates story development and character movements based on input data.

[1082] "Content delivery system" refers to the entire system for providing digital content to users, and in this context refers specifically to a system for delivering interactive stories for children.

[1083] "Story data" refers to data that includes plot and character information necessary for the progression of the story.

[1084] "Characteristic data" refers primarily to attribute data relating to a child's age and gender.

[1085] "Emotion information" is information that indicates the user's emotional state obtained from voice, facial expressions, etc.

[1086] "Options generation means" refers to a method or device for automatically generating options to be provided to a user.

[1087] An "option list" is a list of multiple options that a child can choose from as the story progresses.

[1088] "Means for receiving and transmitting suggestions" refers to a method or device for transmitting the child's selections to the server.

[1089] "Video data" refers to a sequence of image data intended for visual display and usually includes audio.

[1090] "Means for adjusting story progression" refers to a method or device for dynamically adjusting story progression or character behavior based on user input or emotional information.

[1091] "Display means" refers to a method or device for displaying the generated video or option list on a user terminal.

[1092] This invention is an interactive children's content distribution system in which a server and a terminal work in cooperation with each other, and includes the following elements:

[1093] Preparing the server and terminal

[1094] The server mainly receives story data, child characteristic data, and user emotion information. The server uses MongoDB or Firebase as a database, and Google Cloud Vision API or IBM Watson as an emotion recognition engine. The server is built using Node.js or Python (Flask / Django framework).

[1095] The device is mainly intended to be a smartphone, and will be built using React Native and Flutter as the front-end platform. The device is equipped with a camera and microphone, and will capture the user's voice and facial expressions as emotional information.

[1096] Server Processing

[1097] The server receives the following data as input:

[1098] Narrative Data: information about the plot and characters of a story.

[1099] Child characteristics data: demographic information such as age and gender.

[1100] User emotion information: voice and facial expression data.

[1101] Based on this data, the generative AI generates options. The generated options are sent to the device. When the user's choice is sent from the device to the server, the server generates a story and video data based on the choice. The FFmpeg library and other libraries are used to generate the story and video.

[1102] Terminal handling

[1103] The device displays a list of options received from the server to the user. The user selects from the options and the selection is sent to the server. The device also obtains the user's emotional information (voice and facial expressions) and sends it to the server in real time. This allows the server-side emotion engine to adjust the progress of the story and the character's actions.

[1104] Examples

[1105] For example, suppose a story about a "night adventure" is displayed. In this story, the child is presented with the choice of "preparing before leaving or not." If the child chooses "preparing," the story switches to a preparation scene. In addition, the device's camera and microphone capture the child's smile and tone of voice, and transmit this emotional information to the server. Depending on the child's smile, the server can change the setting of the story to a brighter, more enjoyable development.

[1106] Example prompt:

[1107] This is the story of a brave knight who goes on an adventure. The choices you make will determine how the adventure progresses. Let's start the adventure together!

[1108] In this way, the server and device work together to enable children to dynamically experience a story based on their own choices and emotions.

[1109] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1110] Step 1:

[1111] The server receives story data, child's characteristics data, and user's emotion information as input. This includes story plot and character information. Emotion information is voice data and facial expression data obtained from the terminal. These data are stored in MongoDB and Firebase.

[1112] Input: Story data, child's characteristics data, user's emotion information

[1113] Output: A record of the stored data

[1114] Specific behavior:

[1115] The server receives the data sent from the terminal via the API.

[1116] A process is executed to store the received data in a database.

[1117] Step 2:

[1118] The generative AI generates options based on the story data and the child's characteristics and emotional information. The generated options are used by the child to decide the progress of the story.

[1119] Input: Story data, child's characteristics data, user's emotion information

[1120] Output: Choice list

[1121] Specific behavior:

[1122] The generative AI on the server refers to the stored data and generates appropriate options.

[1123] The emotion engine analyzes emotional information and reflects it in generating options.

[1124] Step 3:

[1125] The device displays the list of options received from the server to the user. The user (child) selects the option they want from the presented options by tapping on it.

[1126] Input: Choice list

[1127] Output: User's choice

[1128] Specific behavior:

[1129] The front-end application displays the list of options in a GUI.

[1130] It recognizes the user's tapping actions and sends the selection to the server.

[1131] Step 4:

[1132] The server receives the user's selections from the terminal and generates the next story portion based on the selections, including video data corresponding to the selections.

[1133] Input: User's choice

[1134] Output: Generated story text, video data

[1135] Specific behavior:

[1136] The server receives the user's selections and updates the story data.

[1137] Video generation processing according to the progression of the story is carried out using FFmpeg.

[1138] Step 5:

[1139] The emotion engine analyzes the user's real-time emotion information and adapts the story progression and character behavior based on that information. Revised story and video data are generated.

[1140] Input: User emotion information

[1141] Output: Revised story text, video data

[1142] Specific behavior:

[1143] The server analyzes the voice and facial expression data transmitted in real time.

[1144] Based on the analysis results, the story's progression and character behavior are dynamically revised.

[1145] Step 6:

[1146] The terminal receives the generated video data from the server and displays it on the display, allowing the user to watch the progress of the new story.

[1147] Input: Generated story text, video data

[1148] Output: Narrative video shown on the display

[1149] Specific behavior:

[1150] The application on the terminal receives streaming video data from the server.

[1151] The received video is displayed on the user interface and played.

[1152] This allows users to interactively experience the story based on their choices and emotions.

[1153] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1154] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1155] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1156] [Fourth embodiment]

[1157] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1158] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1159] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1160] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1161] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1162] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1163] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1164] The control target 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, legs, etc. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, legs, etc. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1165] Fig. 8 shows an example of main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1166] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1167] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1168] In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1169] Next, a description will be given of the specific processing by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1170] An embodiment for implementing the present invention includes the following elements.

[1171] 1. Method of inputting data from the server:

[1172] The server receives the story data and the child's characteristic data.

[1173] Story data includes plot and character information for a story.

[1174] The child's characteristic data includes attribute information such as age, sex, and address.

[1175] 2. How choices are generated:

[1176] Based on the input data, the server generates options for the character's actions that correspond to the content of the story.

[1177] Appropriate options can be generated taking into account the child's characteristic data.

[1178] 3. How to present options:

[1179] The terminal displays the generated option list on the screen.

[1180] The child suggests a choice from a presented list of options.

[1181] 4. Means of sending your choice:

[1182] The terminal transmits the child's selection to the server.

[1183] 5. Narrative generation and video data display:

[1184] The server generates a story based on the selections received.

[1185] The server generates video data in accordance with the generated story.

[1186] The terminal displays the generated video data on a display.

[1187] In this way, the server and the device work together to control the progression of the story and the actions of the characters, allowing children to actively participate. Children can suggest options, the story progresses based on those options, and videos are displayed, allowing them to experience the story through their own choices.

[1188] The process flow will be explained below.

[1189] Step 1: Input data from the server

[1190] The server receives the story data and the child's characteristic data.

[1191] The story data includes information about the plot and characters of the story, and the child characteristic data includes attribute information such as age and gender.

[1192] Step 2: Generate options

[1193] The server generates options for the character's actions according to the content of the story, based on the received story data and the child's characteristic data.

[1194] Appropriate options can be generated taking into account the child's attribute information.

[1195] Step 3: Present the options

[1196] The terminal displays the generated option list on the screen.

[1197] The child suggests a choice from a presented list of options.

[1198] Step 4: Submit your selections

[1199] The terminal transmits the child's selection to the server.

[1200] Step 5: Generate narrative and display video data

[1201] The server generates a story based on the selections received.

[1202] The server generates video data in accordance with the generated story.

[1203] The terminal displays the generated video data on a display.

[1204] Step 6: Story progression and repetition

[1205] While the child enjoys the story presented, they are then asked to make suggestions for the next step.

[1206] The processes from step 2 to step 5 above are repeated as the story progresses.

[1207] Output Condition:

[1208] Server: Receives story data and child characteristic data, generates options, generates stories, and generates video data

[1209] Terminal: Display list of options, send selection, display video data

[1210] User (child): Suggestions for choices, story progression

[1211] Example 1

[1212] Next, a description will be given of Example 1. In the following description, the data processing device 12 is referred to as a "server" and the robot 414 is referred to as a "terminal."

[1213] In conventional video streaming systems for children, it was difficult for users to interactively participate in the development of the story, and the story often progressed one-sidedly. Therefore, a system that allows children to make their own choices and experience the progression of the story is desired. There is also a demand for a system that can provide options according to the characteristics of each individual child, but this has not been fully realized in conventional technologies.

[1214] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1215] In this invention, the server includes a means for inputting story data and child characteristic data, a means for generating options based on the input data by a generative AI model, and a means for generating story and video data. This allows options suited to the child's characteristics to be provided, and enables the user to experience the development of the story based on their own selection.

[1216] A "server" is a central management system that stores and processes data over a network.

[1217] "Story data" refers to data that includes information related to the progression of the story, such as the story plot and character information.

[1218] "Child characteristic data" refers to data including the child's attribute information, such as age, sex, and address.

[1219] A "generative AI model" is a model that uses artificial intelligence to analyze data and generate options, stories, etc.

[1220] The "option generation means" is a mechanism that generates options from which the user can choose based on input data.

[1221] A "choice list" is a collection of multiple options that are displayed on the screen for the user to select from.

[1222] A "terminal" is a device that is operated by a user and that displays options and transmits and receives data.

[1223] A "user selection suggestion means" is a mechanism that receives selection input from a user and transmits it to a server.

[1224] The "means for generating story and video data" is a mechanism for generating a new story based on the user's selection and creating video data corresponding to that story.

[1225] The "video data display means" is a mechanism for displaying the generated video data on the display of the terminal.

[1226] The "means for generating options taking into consideration characteristic data" is a mechanism for generating appropriate options based on the child's characteristic data.

[1227] Overall overview

[1228] The present invention relates to an interactive video streaming system for children, and provides a system in which users can participate in the progression of a story. This system dynamically generates story data and video data by making full use of a server, a terminal, and a generative AI model, and provides an interactive experience for users.

[1229] server

[1230] The server receives story data and child characteristic data. For example, it receives data including the plot and character information of the story, and attribute information such as the child's age, gender, and address. The server also analyzes these data using a generative AI model and generates options based on the input data. The server then generates a new story based on the user's selection and creates video data corresponding to the story.

[1231] Hardware / Software used:

[1232] Server equipment (high performance computers)

[1233] Generative AI models (e.g. ChatGPT, BERT)

[1234] Database (e.g. MySQL, PostgreSQL)

[1235] Terminal

[1236] The terminal is a device operated by the user, and is responsible for displaying the list of options sent from the server on the screen. When the user selects an option, the information is sent to the server in real time. In addition, the generated video data is displayed to the user.

[1237] Hardware / Software used:

[1238] Tablets and PCs (user-operated devices)

[1239] Display (for video display)

[1240] Internet connection

[1241] User

[1242] Users (usually children) operate the device, choose the action they want from the options displayed, and experience the progress of the story as a result. This allows users to experience a unique story based on their own choices.

[1243] Examples of concrete examples and prompts

[1244] Example 1:

[1245] The server receives the plot data of "The Story of a Hero Defeating a Dragon" and the characteristic data of "a 7-year-old boy," and generates the options "attack," "run away," and "use an item." The device displays these options on the screen, and the user selects "attack," which is sent to the server. Based on this selection, the server generates a scene in which the hero attacks the dragon, and creates a corresponding animated video. The device displays this video.

[1246] Example 2:

[1247] The server receives the data "A story about a wizard choosing a spell" and the characteristic data "12-year-old girl," and generates the options "Fire magic," "Water magic," and "Wind magic." The device displays these options on the screen, and the user selects "Fire magic," which is sent to the server. Based on this selection, the server generates a scene in which the wizard casts a fire spell, and creates a corresponding animated video. The device displays this video.

[1248] Examples of prompt statements

[1249] Prompt statement example 1:

[1250] User: 7 year old boy

[1251] Story scene: The hero confronts the dragon

[1252] Choices: Attack, Run, Use an item

[1253] Output format: Animated video with the corresponding story of the scene's continuation

[1254] Prompt statement example 2:

[1255] User: 12 year old girl

[1256] Story scene: The wizard chooses a spell

[1257] Choices: Fire magic, Water magic, Wind magic

[1258] Output format: Animated video with the corresponding story of the scene's continuation

[1259] By feeding these prompts into a generative AI model, the system can provide the interactive narrative experience described above.

[1260] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1261] Step 1: Receiving data

[1262] The server is responsible for receiving story data and child characteristic data. The story data includes story plot and character information, and the child characteristic data includes attribute information such as age, gender, address, etc. The server obtains these data from an external database or a user device. It receives the story data and characteristic data as input and stores it internally.

[1263] Specific behavior:

[1264] The server retrieves plot data for "The Story of a Hero Slaying a Dragon."

[1265] The server receives the characteristic data of "7-year-old boy."

[1266] Input: Story data, child characteristics data

[1267] Output: Data stored inside the server

[1268] Step 2: Generate options

[1269] The server uses a generative AI model to generate options based on the received story data and the child's characteristics data. In the process, the server analyzes the input data and creates multiple options to present to the user.

[1270] Specific behavior:

[1271] The server recognizes this as a scene in which a hero confronts a dragon, and generates age-appropriate options such as "attack," "run away," or "use an item."

[1272] Input: Story data, child characteristics data

[1273] Output: Choice list (response)

[1274] Step 3: Displaying the options

[1275] The terminal has the role of displaying the option list sent from the server on the screen. The terminal receives the option list returned from the server and displays it in a format that is easy for the user to see.

[1276] Specific behavior:

[1277] The tablet device will display the options "attack," "run away," and "use item" on the screen.

[1278] Input: Choice list (response from server)

[1279] Output: The choices displayed on the screen.

[1280] Step 4: Sending user selections

[1281] The user selects one of the options displayed on the screen. The user's choice is sent to the server via the terminal. The terminal receives the user's input and forwards it to the server.

[1282] Specific behavior:

[1283] The child taps to select "attack," and the tablet sends the selection data to the server.

[1284] Input: User's choice

[1285] Output: Selection data sent to the server

[1286] Step 5: Generate a story

[1287] The server generates new narrative scenes based on the received choices, using a generative AI model to create a narrative progression based on the choices. In this step, the storyline is updated based on the choices and the next scenes are generated.

[1288] Specific behavior:

[1289] Based on the server's choice of "attack," a scene is generated in which the hero attacks the dragon.

[1290] Input: User selected data

[1291] Output: Next story scene

[1292] Step 6: Generate video data

[1293] The server generates video data based on the generated story, and uses a generative AI model to create animated videos that correspond to the scenes in the story. This dynamic process renders visual data that matches the story.

[1294] Specific behavior:

[1295] The server creates animation data based on the "scene of the hero attacking the dragon" and encodes it in video format.

[1296] Input: Next story scene

[1297] Output: Video data

[1298] Step 7: View video data

[1299] The terminal displays the video data sent from the server to the user. The terminal receives the video data and plays it to visually present the progress of the story to the user.

[1300] Specific behavior:

[1301] The tablet device plays the animation video received from the server.

[1302] Input: Video data

[1303] Output: Video played on screen

[1304] Through the above processing steps, a child can interactively progress through the story and enjoy a unique story experience based on his or her own choices.

[1305] (Application example 1)

[1306] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1307] There is a demand for video streaming services for children to provide a more interactive and educational experience by dynamically changing the progression of the story according to the viewer's (child's) choices. However, current systems do not generate stories that correspond to the individual attributes of each child, nor do they present options, and the content is uniform. This makes it difficult to provide content that is appropriate for each child, resulting in low immersion and educational effectiveness.

[1308] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1309] In this invention, the server includes a means for inputting story data and child characteristic data, a means for generating options based on the input data by a generative AI model, and a means for displaying a list of options on a terminal, thereby enabling an interactive story progression based on the individual attributes of the child.

[1310] A "generative AI model" is an artificial intelligence model that generates dynamic narratives and options based on given data.

[1311] An "interactive story application system" is a system that provides an application in which the progress of a story changes according to the user's choices.

[1312] "Story data" is data necessary for the progression of the story, such as the story plot and character information.

[1313] "Child characteristic data" refers to data including child attribute information such as age, sex, and address.

[1314] The "server" is a computer system that receives story data and child characteristic data, and generates options and stories based on a generative AI model.

[1315] The "option generation means" is a mechanism for generating multiple options based on input data.

[1316] A "terminal" is a device that allows a user to view and select options.

[1317] The "means for receiving selection suggestions" is a mechanism for transmitting the options selected by the user to the server.

[1318] "Means for generating story and video data" refers to a mechanism for creating story progression and video data based on user selection.

[1319] "Video data display means" refers to a mechanism for showing the generated video data to the user.

[1320] This invention shows a concrete method for realizing an interactive storytelling application system using a generative AI model. The system mainly consists of three elements: a server, a terminal, and a user.

[1321] server

[1322] The server receives story data and child characteristics data as input. It uses a web framework such as Django and Python as the programming language. The server uses this data to generate interactive options using a generative AI model and sends the option list to the terminal.

[1323] Story data refers to data necessary for the progression of the story, such as the story plot and character information.

[1324] The child's characteristic data is data including the child's attribute information such as age, sex, and address.

[1325] The server also receives the options selected by the user and generates new story progression and video data based on the options selected by the user. The generated video data is sent to the terminal.

[1326] Terminal

[1327] The terminal refers to a smartphone or tablet, and is built using a framework such as React Native. The terminal displays the list of options received from the server, sends the user's selection to the server, and displays new video data sent from the server.

[1328] User

[1329] The user is the child and makes his / her selection from a displayed list of options, the content of which is appropriately customized based on the child's characteristic data.

[1330] Examples

[1331] For example, if a user (child) selects "Character A climbs a tree," the server generates a new story progression based on this selection, creates video data of "the view that Character A sees from the top of the tree," and sends it to the terminal. On the other hand, if the user selects "Character B crosses the river," the story progresses to "Character B meets a new friend on the other side of the river," and video data for that is generated.

[1332] Examples of prompt statements

[1333] Generate interactive story choices based on the following criteria:

[1334] Child's demographic information: Age 7, gender male, address Tokyo

[1335] Story ID is 1

[1336] Please let me know what option is appropriate.

[1337] In this way, generative AI models can be used to provide a personalized, interactive storytelling experience for each child, enhancing educational and entertainment value.

[1338] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1339] Step 1:

[1340] The server receives the story data and the child's characteristics data. This data includes the story plot, character information, the child's age, gender, address, etc. Based on the input story data and child's characteristics data, it extracts relevant information from the database and formats it in the format required for subsequent processing. The output at this point is the formatted story data and child's characteristics data.

[1341] Step 2:

[1342] The server uses a generative AI model to generate options based on the input data. Specifically, the story data and the child's characteristics data are input into the AI ​​model, and a list of options is generated from the model. Data processing involves filtering so that options appropriate to the child's attributes are selected. The output is a list of options.

[1343] Step 3:

[1344] The server sends a list of options to the terminal. The sent list of options is displayed on the terminal. The user (child) sees this list and makes a selection. The output is the list of options displayed on the terminal.

[1345] Step 4:

[1346] The user selects one item from a list of options, and the device sends the selection to the server. The input is the user's selection, and the resulting concrete option is sent to the server. The output is the selection sent to the server.

[1347] Step 5:

[1348] The server generates story progression and new video data based on the user's selection. Based on the content of the choice, the server extracts the relevant story progression from the story database and inputs it into the generative AI model. Video data is generated based on the generated data. The output is the new story progression and video data.

[1349] Step 6:

[1350] The server sends the generated video data to the terminal. The terminal displays the received video data on the screen. The user can watch the progress of the new story. The input is the video data sent from the server, and the output is the video displayed on the terminal.

[1351] In addition, an emotion engine that estimates the emotion of the user may be further combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[1352] An embodiment for implementing the present invention includes the following elements.

[1353] 1. Method of inputting data from the server:

[1354] The server receives the story data, the child's characteristic data, and the user's emotion information.

[1355] The story data includes information about the plot and characters of the story, and the child characteristic data includes attribute information such as age and gender.

[1356] The user's emotional information is recognized from voice, facial expressions, etc. and processed by an emotion engine.

[1357] 2. How choices are generated:

[1358] The server generates options for the character's actions according to the content of the story based on the received story data, the child's characteristic data, and the user's emotion information.

[1359] The emotion engine can take into account the user's emotion information to generate appropriate options.

[1360] 3. How to present options:

[1361] The terminal displays the generated option list on the screen.

[1362] The child suggests a choice from a presented list of options.

[1363] 4. Means of sending your choice:

[1364] The terminal transmits the child's selection to the server.

[1365] 5. Narrative generation and video data display:

[1366] The server generates a story based on the selections received.

[1367] The server generates video data in accordance with the generated story.

[1368] The terminal displays the generated video data on a display.

[1369] 6. Emotion engine processing means:

[1370] The server receives the user's emotion information and processes it using an emotion engine.

[1371] The emotion engine recognizes the user's emotional information, such as voice and facial expressions, and creates a story based on that information.

[1372] Adjust the progression and character behavior.

[1373] In this way, the server and the device work together to control the progression of the story and the actions of the characters, allowing the child to be actively involved. The child suggests options, the story progresses based on those options, and the videos are displayed, allowing the child to experience the story based on their own choices and emotions.

[1374] The process flow will be explained below.

[1375] Step 1: Input data from the server

[1376] The server receives the story data, the child's behavior data, and the user's emotion information.

[1377] The story data includes information about the plot and characters of the story, and the child characteristic data includes attribute information such as age and gender.

[1378] The user's emotional information is recognized from voice, facial expressions, etc., and processed by the emotion engine. Step 2: Generation of options

[1379] The server generates options for the character's actions according to the content of the story based on the received story data, the child's characteristic data, and the user's emotion information.

[1380] The emotion engine can generate appropriate options by taking into account the user's emotion information. Step 3: Present options

[1381] The terminal displays the generated option list on the screen.

[1382] The child suggests a choice from a presented list of options.

[1383] Step 4: Submit your selections

[1384] The terminal transmits the child's selection to the server.

[1385] Step 5: Generate narrative and display video data

[1386] The server generates a story based on the selections received.

[1387] The server generates video data in accordance with the generated story.

[1388] The terminal displays the generated video data on a display.

[1389] Step 6: Emotion Engine Processing

[1390] The server receives the user's emotion information and processes it using an emotion engine.

[1391] The emotion engine recognizes the user's emotional information, such as voice and facial expressions, and adjusts the story progress and character behavior based on that information.

[1392] Through the above processing steps, the server and the device work together to control the progress of the story and the actions of the characters, allowing the child to actively participate. The child proposes options, the story progresses based on those options, and the videos are displayed, allowing the child to experience the story based on his or her own choices and emotions.

[1393] Example 2

[1394] Next, a description will be given of Example 2. In the following description, the data processing device 12 is referred to as a "server" and the robot 414 is referred to as a "terminal."

[1395] In conventional interactive video systems for children, the progression of the story and the actions of the characters are prescribed, making it difficult to respond to the interests and emotions of individual children. In addition, there was a problem in that it was not possible to dynamically generate a story that incorporates changes in the user's emotions.

[1396] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an input means for story data, child attribute information, and user emotion information, a generation means for generating options based on the input data by a generative AI, and a processing means for an emotion engine that recognizes the user emotion information and adjusts the progress of the story and the behavior of the characters. This enables a personalized story progress based on the child attribute information and the user emotion.

[1397] "Story data" is digital data that includes information about the story plot and characters.

[1398] "Child attribute information" is data indicating the child's attributes such as age and sex.

[1399] "User emotion information" is data indicating the user's emotional state obtained from the user's voice, facial expression, etc.

[1400] The "input means" refers to a means by which the server receives story data, child attribute information, and user emotion information.

[1401] "Generative AI" is an artificial intelligence technology that generates story options or the story itself based on input data.

[1402] The "option generating means" is a means for generating options based on received story data, child attribute information, and user emotion information.

[1403] An "emotion engine" is software that recognizes the user's emotional information and adjusts the progress of the story and the behavior of characters.

[1404] "Video data" refers to animation and video data that is generated in accordance with the progression of the story.

[1405] A "Terminal" is a device (e.g., tablet, smartphone) that a User can interact with.

[1406] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1407] This invention relates to an interactive video system for children that incorporates generative AI to determine the ending of a story and the actions of characters. This system mainly operates in cooperation with a server and terminals.

[1408] The server accepts input of story data, child attribute information, and user emotion information. Story data includes the story plot and character information. Child attribute information includes information such as age and gender. User emotion information is obtained from voice and facial expressions, and processed by an emotion engine. Based on this input data, the server uses generative AI to generate appropriate options.

[1409] The generated options are sent to the terminal, which displays them on the screen. The user selects one of the options presented and sends it to the server via the terminal. The server generates a story based on the received selection and creates video data to match it. The video data is then sent back to the terminal and displayed to the user.

[1410] During this entire process, the server uses an emotion engine to recognize the user's emotional information in real time and adjust the story progression and character behavior, thus enabling a personalized interactive video experience.

[1411] As a concrete example, the server recognizes the story data of "Little Red Riding Hood," the attribute information of a 7-year-old girl, and the user's smile. After the emotion engine recognizes the user's smile, it generates an option during the story that reads, "Please select an animal that Little Red Riding Hood will encounter in the forest." This is displayed on the device's display, and the child selects "rabbit." Upon receiving this selection, the server generates a "scene in which Little Red Riding Hood talks to the rabbit," creates video data, and displays it on the device.

[1412] Examples of prompt statements

[1413] "Generate three options for animals that Little Red Riding Hood encounters in the forest for a 7-year-old girl to enjoy. If you detect a smile from the user, make it more fun."

[1414] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1415] Step 1: Data entry

[1416] The server inputs story data, the child's attribute information, and the user's emotion information. Specifically, as story data input, plot and character information for a story such as "Little Red Riding Hood" is obtained from a database. As the child's attribute information, data that the child is a "7-year-old girl" is received via sensors and cameras. As the user's emotion information, real-time voice and facial expression data is obtained from the camera and microphone, and this is processed by the emotion engine.

[1417] input:

[1418] Story data (e.g. the plot of "Little Red Riding Hood")

[1419] Child demographic information (e.g. 7 years old, girl)

[1420] User emotion information (e.g. smile detection)

[1421] output:

[1422] Integrated data passed to the server

[1423] Step 2: Generate options

[1424] The server uses a generative AI model to generate appropriate options based on the received data. A prompt is input to the generative AI model, for example, "Generate three actions for a character that a 7-year-old girl can enjoy. If the user's smile is recognized, make it more enjoyable." The emotion engine takes into account the user's emotional information and generates appropriate options.

[1425] input:

[1426] Integrated data (story data, children's attribute information, user's emotion information)

[1427] output:

[1428] Generated choice list (e.g. 1. rabbit 2. squirrel 3. bird)

[1429] Step 3: Present the options

[1430] The device displays the generated list of options on the screen. Specifically, the tablet display shows, "Please choose an animal that Little Red Riding Hood will meet: 1. Rabbit 2. Squirrel 3. Small bird." The user, a child, taps the screen to make a selection.

[1431] input:

[1432] Generated Choice List

[1433] output:

[1434] Choice list displayed on the device

[1435] Step 4: Submit your selections

[1436] The terminal transmits the data selected by the child to the server. Specifically, when the user selects "rabbit," the selection information is transmitted to the server.

[1437] input:

[1438] Select a child (e.g. "rabbit")

[1439] output:

[1440] Selection data sent to the server

[1441] Step 5: Generate a story

[1442] The server generates a story based on the selections it receives: for example, a scene where Little Red Riding Hood meets and talks with the Rabbit, and creates video data to go with it. An emotion engine also helps at this stage, providing more personalized content.

[1443] input:

[1444] Selection data (e.g. "rabbit")

[1445] output:

[1446] Generated story (e.g. the scene where Little Red Riding Hood talks to the rabbit)

[1447] Step 6: Generate and display video data

[1448] The server generates video data in accordance with the progress of the story and sends it to the device. The device then displays the received video data on the display. Specifically, the tablet plays an animation of Little Red Riding Hood meeting and talking to the Rabbit.

[1449] input:

[1450] Generated story (e.g. the scene where Little Red Riding Hood talks to the rabbit)

[1451] output:

[1452] Generated video data, video playback on device

[1453] (Application example 2)

[1454] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1455] Traditional children's content lacked the ability to adjust the progress of the story and the actions of characters based on the individual attributes and emotions of the viewer, limiting the experience of viewers actively participating in the story. In addition, since only pre-fixed story developments could be selected, it was often unable to respond to the interests and concerns of the viewer. As a result, it was difficult to stimulate the viewer's motivation to learn and creativity.

[1456] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1457] In this invention, the server includes a means for inputting story data, child characteristic data, and user emotion information, a means for generating options based on the input data by a generative AI, and a means for adjusting the progress of the story and the behavior of the characters based on the user emotion information. This makes it possible to dynamically progress the story based on the individual attributes and emotions of the viewers.

[1458] "Story ending" and "character actions" refer to the final outcome or action of a character or plot in a scene or episode within a story.

[1459] "Generative AI" refers to artificial intelligence technology that automatically generates story development and character movements based on input data.

[1460] "Content delivery system" refers to the entire system for providing digital content to users, and in this context refers specifically to a system for delivering interactive stories for children.

[1461] "Story data" refers to data that includes plot and character information necessary for the progression of the story.

[1462] "Characteristic data" refers primarily to attribute data relating to a child's age and gender.

[1463] "Emotion information" is information that indicates the user's emotional state obtained from voice, facial expressions, etc.

[1464] "Options generation means" refers to a method or device for automatically generating options to be provided to a user.

[1465] An "option list" is a list of multiple options that a child can choose from as the story progresses.

[1466] "Means for receiving and transmitting suggestions" refers to a method or device for transmitting the child's selections to the server.

[1467] "Video data" refers to a sequence of image data intended for visual display and usually includes audio.

[1468] "Means for adjusting story progression" refers to a method or device for dynamically adjusting story progression or character behavior based on user input or emotional information.

[1469] "Display means" refers to a method or device for displaying the generated video or option list on a user terminal.

[1470] This invention is an interactive children's content distribution system in which a server and a terminal work in cooperation with each other, and includes the following elements:

[1471] Preparing the server and terminal

[1472] The server mainly receives story data, child characteristic data, and user emotion information. The server uses MongoDB or Firebase as a database, and Google Cloud Vision API or IBM Watson as an emotion recognition engine. The server is built using Node.js or Python (Flask / Django framework).

[1473] The device is mainly intended to be a smartphone, and will be built using React Native and Flutter as the front-end platform. The device is equipped with a camera and microphone, and will capture the user's voice and facial expressions as emotional information.

[1474] Server Processing

[1475] The server receives the following data as input:

[1476] Narrative Data: information about the plot and characters of a story.

[1477] Child characteristics data: demographic information such as age and gender.

[1478] User emotion information: voice and facial expression data.

[1479] Based on this data, the generative AI generates options. The generated options are sent to the device. When the user's choice is sent from the device to the server, the server generates a story and video data based on the choice. The FFmpeg library and other libraries are used to generate the story and video.

[1480] Terminal handling

[1481] The device displays a list of options received from the server to the user. The user selects from the options and the selection is sent to the server. The device also obtains the user's emotional information (voice and facial expressions) and sends it to the server in real time. This allows the server-side emotion engine to adjust the progress of the story and the character's actions.

[1482] Examples

[1483] For example, suppose a story about a "night adventure" is displayed. In this story, the child is presented with the choice of "preparing before leaving or not." If the child chooses "preparing," the story switches to a preparation scene. In addition, the device's camera and microphone capture the child's smile and tone of voice, and transmit this emotional information to the server. Depending on the child's smile, the server can change the setting of the story to a brighter, more enjoyable development.

[1484] Example prompt:

[1485] This is the story of a brave knight who goes on an adventure. The choices you make will determine how the adventure progresses. Let's start the adventure together!

[1486] In this way, the server and device work together to enable children to dynamically experience a story based on their own choices and emotions.

[1487] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1488] Step 1:

[1489] The server receives story data, child's characteristics data, and user's emotion information as input. This includes story plot and character information. Emotion information is voice data and facial expression data obtained from the terminal. These data are stored in MongoDB and Firebase.

[1490] Input: Story data, child's characteristics data, user's emotion information

[1491] Output: A record of the stored data

[1492] Specific behavior:

[1493] The server receives the data sent from the terminal via the API.

[1494] A process is executed to store the received data in a database.

[1495] Step 2:

[1496] The generative AI generates options based on the story data and the child's characteristics and emotional information. The generated options are used by the child to decide the progress of the story.

[1497] Input: Story data, child's characteristics data, user's emotion information

[1498] Output: Choice list

[1499] Specific behavior:

[1500] The generative AI on the server refers to the stored data and generates appropriate options.

[1501] The emotion engine analyzes emotional information and reflects it in generating options.

[1502] Step 3:

[1503] The device displays the list of options received from the server to the user. The user (child) selects the option they want from the presented options by tapping on it.

[1504] Input: Choice list

[1505] Output: User's choice

[1506] Specific behavior:

[1507] The front-end application displays the list of options in a GUI.

[1508] It recognizes the user's tapping actions and sends the selection to the server.

[1509] Step 4:

[1510] The server receives the user's selections from the terminal and generates the next story portion based on the selections, including video data corresponding to the selections.

[1511] Input: User's choice

[1512] Output: Generated story text, video data

[1513] Specific behavior:

[1514] The server receives the user's selections and updates the story data.

[1515] Video generation processing according to the progression of the story is carried out using FFmpeg.

[1516] Step 5:

[1517] The emotion engine analyzes the user's real-time emotion information and adapts the story progression and character behavior based on that information. Revised story and video data are generated.

[1518] Input: User emotion information

[1519] Output: Revised story text, video data

[1520] Specific behavior:

[1521] The server analyzes the voice and facial expression data transmitted in real time.

[1522] Based on the analysis results, the story's progression and character behavior are dynamically revised.

[1523] Step 6:

[1524] The terminal receives the generated video data from the server and displays it on the display, allowing the user to watch the progress of the new story.

[1525] Input: Generated story text, video data

[1526] Output: Narrative video shown on the display

[1527] Specific behavior:

[1528] The application on the terminal receives streaming video data from the server.

[1529] The received video is displayed on the user interface and played.

[1530] This allows users to interactively experience the story based on their choices and emotions.

[1531] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1532] The data generation model 58 is a so-called generative AI (Artificial Intelligence).

[1533] One example of a data generation model58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1534] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the robot 414.

[1535] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[1536] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1537] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1538] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1539] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. For robots, cars, motorcycles, etc., emotions can also be created based on various balances such as posture and battery level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated based on, for example, the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). In the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. In addition, in the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[1540] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[1541] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[1542] Although the system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, the system according to the present disclosure is not necessarily implemented in a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program that runs on a personal computer, or an application that runs on a smartphone or the like. The method according to the present disclosure may be provided to a user in the form of SaaS (Software as a Service).

[1543] In the above embodiment, an example is given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to input data.

[1544] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1545] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[1546] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1547] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing the specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[1548] The hardware resource that performs a particular process consists of one of these various processors.

[1549] Alternatively, it may be configured as a combination of two or more processors of the same type or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Furthermore, the hardware resource that executes a specific process may be a single processor.

[1550] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1551] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[1552] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the technology of the present disclosure.

[1553] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or standard was specifically and individually indicated to be incorporated by reference.

[1554] The following is further disclosed regarding the above embodiment.

[1555] [Claim 1] 1. A children's video streaming application system incorporating generative AI to determine story outcomes and character actions, the system including: A means of inputting story data and child characteristics data from the server A method for generating options based on input data using generative AI A means of displaying a list of options on a terminal A means for receiving selection suggestions from the child and sending them to the server A means of generating narrative and video data based on server selection A means for displaying video data on a terminal [Claim 2]

[1556] 2. The system according to claim 1, wherein the server includes a means for generating options taking into account characteristic data of the child.

[1557] [Claim 3]

[1558] 2. The system according to claim 1, wherein the server includes means for generating story and video data based on the story data and the child's characteristic data.

[1559] [Claim 4]

[1560] 2. The system of claim 1, further comprising an emotion engine for recognizing user emotions.

[1561] [Claim 5]

[1562] 2. The system according to claim 1, wherein the emotion engine recognizes the user's emotion information, such as voice and facial expressions, and adjusts the progress of the story and the behavior of the characters based on the information.

[1563] [Claim 6]

[1564] 2. The system according to claim 1, wherein the emotion engine generates options according to the user's emotions and reflects them in the option list.

[1565] "Example 1" (Claim 1) input means for inputting story data and child characteristic data from a server; A means for generating options based on input data by a generative AI model; A means for displaying a list of options on a terminal; means for receiving and transmitting selection suggestions from a user to said server; means for generating story and video data based on selections made by the server; A display means for displaying video data by the terminal; A system including:

[1566] (Claim 2) The system according to claim 1 , wherein the server generates the options taking into account the child's characteristic data.

[1567] (Claim 3) The system of claim 1 , wherein the server generates story and video data based on the story data and the child's characteristic data.

[1568] "Application example 1" (Claim 1)

[1569] An interactive narrative application system incorporating a generative AI model, comprising: input means for inputting story data and child characteristic data from a server; A means for generating options based on input data by a generative AI model; A means for displaying a list of options on a terminal; means for receiving and transmitting selection suggestions from a child to said server; means for generating story and video data based on selections made by the server; Display means for video data on the terminal A system including:

[1570] (Claim 2) The system according to claim 1, wherein the server includes a means for generating options taking into account the child's characteristic data.

[1571] (Claim 3) The system of claim 1, wherein the server includes means for generating story and video data based on the story data and the child's characteristic data.

[1572] "Example 2 of combining emotion engines" (Claim 1) An interactive video system for children that incorporates generative AI to determine the ending of a story and the actions of characters, an input means for inputting story data, attribute information of a child, and emotion information of a user from a server; A means for generating options based on input data using generative AI; A means for displaying a list of options on a terminal; means for receiving and transmitting selection suggestions from a child to said server; means for generating story and video data based on selections made by the server; A display means for displaying video data by the terminal; A processing means for an emotion engine that recognizes the user's emotion information and adjusts the story progression and character behavior A system including:

[1573] (Claim 2)

[1574] The system according to claim 1 , wherein the server generates options taking into account attribute information of the child.

[1575] (Claim 3)

[1576] 2. The system according to claim 1, wherein the server generates story and video data based on story data, the child's attribute information, and the user's emotion information.

[1577] "Application example 2 when combining emotion engines" (Claim 1) A children's content distribution system incorporating generative AI to determine the ending of a story and the actions of characters, an input means for inputting story data and child characteristic data from a server, and user emotion information; A means for generating options based on input data using generative AI; A means for displaying a list of options on a terminal; means for receiving and transmitting selection suggestions from a child to said server; means for generating story and video data based on selections made by the server; A means to adjust the progression of the story and the behavior of characters based on the user's emotional information, Display means for video data on the terminal A system including:

[1578] (Claim 2) The system according to claim 1, wherein the server includes a means for generating options taking into account the child's characteristic data.

[1579] (Claim 3) The system of claim 1, wherein the server includes means for generating story and video data based on the story data and the child's characteristic data. [Explanation of symbols]

[1580] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A children's video streaming application system incorporating generative AI to determine the ending of a story and the actions of characters, an input means for inputting input data including story data relating to the content of the story and data on the characteristics of the child into the server; an option generating means for generating options based on the input data by a generating AI; an option list display means for displaying an option list including the option in the terminal; a transmission means for transmitting a child's proposal for selection from the options from the terminal to the server; a narrative / video data generation means for generating narrative data or video data based on said selection; and A story / movie data display means for displaying the story data or the movie data on the terminal. A system including:

2. 2. The system according to claim 1, wherein said option generating means generates said options based on voice data of said child inputted via a voice input means.

3. 2. The system according to claim 1, wherein said story / movie data generating means generates said story data and said movie data based on said story data and said child characteristic data.

4. 10. The system of claim 1 further comprising an emotion engine for recognizing an emotion of a user.

Images