System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving a user's credentials and authenticating the user; means for receiving a request for a picture book; means for obtaining and sending fixed story information for the picture book to a device; means for receiving user interaction information and generating an ending based on the user interaction information; and means for sending the ending information to the device, wherein the ending information is generated by the AI means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] It is often the case that children demand the same picture book be read to them repeatedly, causing parents to tire of the repetition. This problem reduces the enjoyment of reading aloud between parents and children, and can lead to parents avoiding reading picture books. Furthermore, traditional picture books only allow children to enjoy a single, fixed story, which can make the reading experience monotonous. [Means for solving the problem]

[0005] This invention solves the above-mentioned problems by providing a system including: means for receiving user authentication information and authenticating the user; means for receiving a picture book data request and acquiring fixed story data for the picture book and transmitting it to a terminal; generation AI means for receiving user interaction data and generating an ending based on that data; and means for transmitting the generated ending data to the terminal. This system also collects touch position information during the reading process and provides an interactive element that changes the ending based on that information, allowing users to enjoy a different ending each time they read a story. This allows parents and children to continually enjoy a fresh reading experience.

[0006] "User authentication information" refers to information provided when a user is authenticated to access a system, and specifically includes a user name, password, and the like.

[0007] The "authentication means" is a system or module that receives a user's authentication information, verifies the user's identity based on that information, and authorizes access.

[0008] A "picture book data request" is a request sent from a terminal to a server to request data on a picture book that the user wants to read.

[0009] "Fixed story data for a picture book" is the basic part of the story, and is data for a picture book in which the parts other than the ending are fixed.

[0010] A "terminal" is a device that a user uses to access and interact with the system, such as a tablet or smartphone.

[0011] "Interaction data" is data related to actions such as touches and clicks that a user takes while reading a story.

[0012] A "generative AI means" is an artificial intelligence algorithm or module that generates new endings for a story based on the interaction data it receives.

[0013] "Ending data" is data containing the ending of a story that is automatically generated by a generating AI means.

[0014] An "interactive element" is an element that includes a function in which the user's actions, such as touching or clicking, on the story affect the progress or ending of the story. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] Overview of the invention

[0037] The present invention relates to an interactive digital picture book system that generates different endings based on specific conditions. The system comprises the following means.

[0038] 1. User authentication methods

[0039] When a user logs in using a device such as a tablet, they enter authentication information (user name and password).

[0040] The terminal sends this authentication information to the server, which then checks it against a database to authenticate the user.

[0041] If authentication is successful, the server generates an authentication token and sends it to the terminal, allowing the user to access the system.

[0042] 2. How to obtain picture book data

[0043] The user selects the picture book they want to read on the device. For example, consider the case where they select "Little Red Riding Hood."

[0044] The terminal sends a picture book data request to the server.

[0045] The server receives the request and retrieves the fixed story data for "Little Red Riding Hood" from the database.

[0046] The server sends the acquired story data to the terminal, which analyzes it and displays it on the screen.

[0047] 3. Means of generating endings

[0048] The user reads the story and interacts with each scene, for example touching the wolf in the scene where Little Red Riding Hood encounters the wolf.

[0049] The device sends touch location information to the server, which includes the location where the touch occurred and the scene context.

[0050] The server uses a generative AI means to generate a new ending based on the touch information.

[0051] The server transmits the generated ending data to the terminal.

[0052] 4. Ending Display Method

[0053] The terminal analyzes the received ending data and displays it on the screen.

[0054] Users can enjoy new endings.

[0055] Specific examples

[0056] As a concrete example, consider the case of using the picture book "Little Red Riding Hood."

[0057] User Authentication

[0058] The user enters login information (e.g., username "tanaka" and password "pass1234") into the tablet. The device sends this information to the server, which then authenticates it. If authentication is successful, the server returns an authentication token to the device.

[0059] Picture book data acquisition

[0060] User "Tanaka" selects to read "Little Red Riding Hood." The device sends the request to the server, which retrieves the story data for "Little Red Riding Hood" and sends it to the device. The device displays the data, and the user begins reading the story.

[0061] Ending Generation

[0062] During the story, the user "Tanaka" touches the wolf's nose when Little Red Riding Hood encounters the wolf. The device sends this touch information to the server. The server uses a generation AI method to generate a new ending, "Little Red Riding Hood becomes friends with the wolf," based on the input touch information. The server then sends this ending data to the device.

[0063] Displaying the ending

[0064] The device analyzes the received ending data and displays the new ending to the user "Tanaka." The user and their child can then enjoy the new ending.

[0065] In this way, the present invention makes the reading experience more diverse and interactive than conventional picture books, and makes parent-child reading time more fulfilling.

[0066] The processing flow will be explained below.

[0067] Program processing

[0068] User Authentication and Login

[0069] Step 1:

[0070] The user starts the application and goes to the login screen. The user enters their username and password and presses the "Login" button.

[0071] Step 2:

[0072] The terminal sends the username and password entered by the user to the server as an HTTP request.

[0073] Step 3:

[0074] The server checks the received username and password against the database and performs authentication. If authentication is successful, it generates an authentication token and returns it to the terminal. If authentication fails, it returns an error message to the terminal.

[0075] Acquiring picture book data

[0076] Step 4:

[0077] The user selects the picture book they want to read within the application, for example, "Little Red Riding Hood."

[0078] Step 5:

[0079] The terminal sends a request including information about the selected picture book to the server.

[0080] Step 6:

[0081] The server receives the request and retrieves the fixed story data for the corresponding picture book "Little Red Riding Hood" from the database.

[0082] Step 7:

[0083] The server transmits the acquired fixed story data to the terminal.

[0084] Step 8:

[0085] The device analyzes the received story data and displays it on the screen, allowing the user to begin reading the story.

[0086] Ending generation request

[0087] Step 9:

[0088] The user interacts with the story by touching and clicking, for example, by touching the wolf in the scene where Little Red Riding Hood encounters the wolf.

[0089] Step 10:

[0090] The terminal sends an ending generation request including the user's touch position and scene information to the server.

[0091] Automatic ending generation

[0092] Step 11:

[0093] The server receives the ending generation request and passes the touch position and scene information to the generation AI means.

[0094] Step 12:

[0095] The AI ​​generator generates a new ending based on the touch position and scene information, for example, an ending in which Little Red Riding Hood becomes friends with the wolf.

[0096] Step 13:

[0097] The server transmits the generated ending data to the terminal.

[0098] Displaying the ending

[0099] Step 14:

[0100] The terminal analyzes the received ending data and displays it on the screen.

[0101] Step 15:

[0102] Users can enjoy the story to the end while seeing new endings.

[0103] By following these steps, a different ending is generated each time, making the reading experience fresh.

[0104] Example 1

[0105] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0106] Digital picture books have become popular in recent years and are attracting attention as a form of education and entertainment for children. However, existing digital picture books have a fixed storyline and lack interactive elements that incorporate user interaction, which can lead to a one-sided reading experience that can easily become boring. Therefore, the present invention aims to provide a system that makes the reading experience more enjoyable and interactive by generating different endings based on user interaction.

[0107] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0108] In this invention, the server includes means for receiving user authentication information and authenticating the user, means for receiving a request for picture book data and acquiring fixed story data for the picture book and transmitting it to the terminal, generation AI means for receiving user interaction data and generating an ending based on that data, means for transmitting the generated ending data to the terminal, and means for analyzing the ending data displayed on the terminal and displaying it on the screen. This allows the user to interact with the picture book as they read it, resulting in a variety of endings and an interactive and engaging story-telling experience.

[0109] "User credentials" are data entered by a user to grant access to a system, typically including a username and password.

[0110] "Means for authenticating" refers to the processes and functions for receiving a user's authentication information, checking it against a database, and verifying the user's access rights.

[0111] A "request for picture book data" is an operation performed when a user wants to read a digital picture book and requests the contents of that picture book from the server.

[0112] "Fixed story data" refers to a set of pre-defined story structure and content data for a particular picture book.

[0113] "User interaction data" is information about operations and actions performed by the user while reading a picture book, and includes data such as touch position and timing.

[0114] "Generative AI means" refers to algorithms or processes that use artificial intelligence to generate new outputs based on specific input data.

[0115] "Ending data" is data that describes the content of the final part of the story, and is generated based on the user's interactions.

[0116] "Means for transmitting to the terminal" refers to the function by which the server processes data and sends the data over the network to the device used by the user.

[0117] "Means for displaying on the screen" refers to the process of analyzing the received data, outputting it in an appropriate format to the user interface, and displaying it on the terminal screen.

[0118] "Interactive elements" refer to a mechanism in which users directly interact with the system through operations and actions, and the results are reflected in real time.

[0119] "Specific scene information" is data about a specific scene in a story, and includes the unique characteristics and context of that scene.

[0120] This invention relates to an interactive digital picture book system that generates different endings based on specific conditions. This system uses a generative AI model to generate various endings based on the user's interactions while reading the digital picture book, providing the user with a new reading experience.

[0121] The system mainly consists of hardware including a server, terminals, and authentication database, and software that runs the generative AI model.

[0122] User authentication methods

[0123] The user enters login information (username, password) using a device such as a tablet. The device sends this authentication information to the server. The server receives the authentication information and authenticates the user by checking it against a database. If authentication is successful, the server generates an authentication token and sends it to the device. This token allows the user to access the system.

[0124] How to obtain picture book data

[0125] The user selects the picture book they want to read on their device. For example, if the user selects "Little Red Riding Hood," the device sends a picture book data request to the server. The server retrieves the fixed story data for "Little Red Riding Hood" from the database and sends that data to the device. The device analyzes the retrieved story data and displays it on the screen.

[0126] Means of generating endings

[0127] As the user reads the story, they interact with each scene. For example, if the user touches the wolf's nose in the scene where Little Red Riding Hood encounters the wolf, the device sends this touch position information and the scene context to the server. The server uses a generative AI model to generate a new ending based on this touch information. Specifically, the following prompt is input into the generative AI model: "Generate a story in which Little Red Riding Hood touches the wolf's nose and, as a result, becomes friends with the wolf."

[0128] The generated ending data is transmitted from the server to the terminal.

[0129] Ending display method

[0130] The device analyzes the received ending data and displays it on the screen, allowing the user to enjoy new endings. The process of generating endings is done in real time, and is designed to be instantly updated based on the user's interactions.

[0131] Specific examples

[0132] As a concrete example, consider the case where user "Tanaka" enters login information into a tablet device, is authenticated, and then selects "Little Red Riding Hood." The device receives story data from the server, and the content is displayed on the screen. During the story, user "Tanaka" touches the wolf's nose, and that information is sent to the server. The server uses a generative AI model to generate a new ending in which "Little Red Riding Hood becomes friends with the wolf" and sends it to the device. The device displays the new ending on the screen, and the user can enjoy the new conclusion to the story.

[0133] In this way, the present invention provides a system that generates a variety of endings based on user interactions, making the storytelling experience more interactive and enjoyable.

[0134] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0135] Step 1:

[0136] User enters login information

[0137] The user enters their username and password into the tablet device, which then sends this authentication information to the server.

[0138] Input: Username, Password

[0139] Output: Authentication information sent to the server

[0140] Step 2:

[0141] The server verifies the authentication information

[0142] The server compares the received authentication information with the database and determines the authentication result. If authentication is successful, it generates an authentication token and sends it to the terminal.

[0143] Input: Credentials

[0144] Data processing: Verification of authentication information, generation of authentication tokens

[0145] Output: Authentication token sent to the device

[0146] Step 3:

[0147] User selects a picture book

[0148] The user selects the picture book they want to read on their device, for example, "Little Red Riding Hood," and the device sends the request to the server.

[0149] Input: Picture book selection request

[0150] Output: The request sent to the server

[0151] Step 4:

[0152] The server retrieves the picture book data

[0153] The server retrieves the fixed story data of the relevant picture book from the database and sends the retrieved data to the terminal.

[0154] Input: Picture book selection request

[0155] Data processing: Acquiring story data

[0156] Output: Story data sent to device

[0157] Step 5:

[0158] The device displays the picture book data

[0159] The device analyzes the received picture book data and displays it on the screen, allowing the user to begin reading the story.

[0160] Input: Story data

[0161] Data processing: Data analysis

[0162] Output: Picture book data displayed on the screen

[0163] Step 6:

[0164] The user interacts

[0165] When the user interacts with the story (e.g., touching the wolf's nose) in a specific scene, the device sends the touch location information and the scene context to the server.

[0166] Input: Interaction (touch information)

[0167] Output: Touch information sent to the server

[0168] Step 7:

[0169] The server generates a new ending

[0170] The server uses a generative AI model to generate a new ending based on the touch information it receives.

[0171] Example prompt: "Generate a story in which Little Red Riding Hood touches the wolf's nose and becomes friends with him."

[0172] Input: Touch information, prompt text

[0173] Data processing: Generative AI model generates endings

[0174] Output: Ending data sent to the terminal

[0175] Step 8:

[0176] The device displays a new ending.

[0177] The device analyzes the received ending data and displays it on the screen, allowing the user to enjoy the new ending.

[0178] Input: Ending data

[0179] Data processing: Data analysis

[0180] Output: New ending shown on screen

[0181] (Application example 1)

[0182] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0183] Conventional picture books have a fixed storyline, meaning that once you've read it, you end up reading the same content again and again. This causes problems such as a lack of freshness for both children and parents when reading, and a lack of interactive elements. Also, because they only allow for a predetermined ending, it's difficult to provide real-time interaction or variety in the storyline based on user choices.

[0184] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0185] In this invention, the server includes means for receiving user authentication information and authenticating the user, means for receiving a picture book data request and acquiring fixed story data for the picture book and transmitting it to the information terminal, generation AI means for receiving user interaction data and generating an ending based on that data, means for transmitting the generated ending data to the information terminal, and means for analyzing and displaying the ending data, thereby making it possible to generate and display new endings in real time.

[0186] "User authentication information" refers to information required when a user accesses a system, such as a username and password.

[0187] A "request for picture book data" refers to the act of requesting data for a picture book that the user wants to read, or the request information.

[0188] "Fixed story data" refers to data of a predetermined series of stories in a particular picture book.

[0189] An "information terminal" is an electronic device used by a user, such as a smartphone, tablet, or computer.

[0190] "Interaction data" refers to information about operations and inputs performed by the user while reading a picture book, and includes, for example, touch position information.

[0191] "Generative AI means" refers to artificial intelligence technology used to generate new endings based on received data.

[0192] "Ending data" is data regarding the ending of a story created by a generative AI means.

[0193] The "means for analyzing and displaying" is a means having the function of understanding the received ending data and displaying it on an information terminal.

[0194] The present invention relates to an interactive digital picture book system that generates different endings based on specific conditions. The following is a detailed description showing how to specifically implement the present invention.

[0195] The server implements the invention using the following means.

[0196] 1. User Authentication

[0197] When a user logs in using a device such as a tablet or smartphone, they enter authentication information (username, password). The device sends this authentication information to the server, which then authenticates the user by checking it against a database. If authentication is successful, the server generates an authentication token and sends it to the device, allowing the user to access the system. The software used is Flask to perform the authentication process.

[0198] 2. Acquiring picture book data

[0199] The user selects a picture book they want to read on their device. For example, consider the case where "Little Red Riding Hood" is selected. The device sends a picture book data request to the server. The server receives the request and retrieves the fixed story data for "Little Red Riding Hood" from the database. The server then sends the retrieved story data to the device, which parses it and displays it on the screen. This process involves database operations and sending and receiving data in JSON format.

[0200] 3. Ending Generation

[0201] The user reads the story and interacts with each scene. For example, in the scene where Little Red Riding Hood encounters the wolf, the user touches the wolf's nose. The device sends touch location information to the server, which includes the location of the touch and the scene context. The server uses a generative AI method to generate a new ending based on the touch information. The generative AI model used is OpenAI's GPT-4. This model generates prompts based on specific scene information and the user's interaction data, and generates a new ending.

[0202] 4. Ending display

[0203] The server sends the generated ending data to the device. The device analyzes the received ending data and displays it on the screen. The user can enjoy the new ending. The device uses JavaScript and HTML to analyze the data and display it on the user interface.

[0204] Specific examples

[0205] If the user selects "Little Red Riding Hood" and touches the wolf's ear in the scene where Little Red Riding Hood encounters the wolf, this touch information is sent to the server. The server generates the following prompt sentence and passes it to the generation AI.

[0206] Prompt Sentence Examples

[0207] Generate an ending for the following scene in the story of Little Red Riding Hood. Scene: Little Red Riding Hood encounters the wolf. Touch location: Wolf's ear.

[0208] Based on this prompt, the AI ​​generates a new ending, "Little Red Riding Hood becomes friends with the wolf," and the server sends it to the device. The device displays this ending on the screen, allowing the user to enjoy the conclusion of the new story.

[0209] In this way, the present invention can provide a more diverse and interactive reading experience than conventional picture books, making parent-child reading time more fulfilling.

[0210] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0211] Step 1:

[0212] The user enters login information (e.g., username and password) into the terminal.

[0213] Input: Username and Password

[0214] How it works: The device sends these authentication information to the server, which checks it against a database and authenticates the user. If authentication is successful, the server generates an authentication token and returns it to the device.

[0215] Output: Authentication token

[0216] Step 2:

[0217] The user selects the picture book they want to read on the device.

[0218] Input: Picture book selection information (e.g., "Little Red Riding Hood")

[0219] Operation: The device sends a picture book data request to the server. The server receives the request, retrieves the fixed story data for the corresponding picture book from the database, and sends it to the device. The device analyzes the received story data and displays it on the screen.

[0220] Output: Fixed story data

[0221] Step 3:

[0222] The user reads the story and interacts with it (e.g., by touch) during the story.

[0223] Input: Touch location and scene context

[0224] How it works: The device sends touch location information and scene context to the server, which uses a generative AI model to generate a prompt based on that information.

[0225] Output: prompt statement

[0226] Step 4:

[0227] The server generates a new ending using a generative AI model.

[0228] Input: prompt statement

[0229] How it works: A generative AI model (e.g., OpenAI's GPT-4) generates an ending based on the prompt. The server retrieves the generated ending data and sends it to the device.

[0230] Output: New ending data

[0231] Step 5:

[0232] The device will analyze the new ending data and display it on the screen.

[0233] Input: New ending data

[0234] Behavior: The device parses the received ending data and converts it into a format suitable for the user interface. After parsing, the device displays the new ending on the screen.

[0235] Output: New ending displayed on screen

[0236] This allows users to enjoy new endings generated in real time based on specific interactions.

[0237] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0238] Overview of the invention

[0239] The present invention relates to a digital picture book system that automatically generates an ending based on user interaction. Furthermore, the system is characterized by incorporating an emotion engine that recognizes the user's emotions and reflecting them in the generation of the ending. The system is composed of the following means:

[0240] 1. User authentication methods

[0241] When a user logs in using a device such as a tablet, they enter authentication information (user name and password).

[0242] The terminal sends this authentication information to the server, which then checks it against a database to authenticate the user.

[0243] If authentication is successful, the server generates an authentication token and sends it to the terminal, allowing the user to access the system.

[0244] 2. How to obtain picture book data

[0245] The user selects the picture book they want to read on the device, for example, "Little Red Riding Hood."

[0246] The terminal sends a picture book data request to the server.

[0247] The server receives the request and retrieves the fixed story data for the corresponding picture book "Little Red Riding Hood" from the database.

[0248] The server sends the acquired story data to the terminal, which analyzes it and displays it on the screen.

[0249] 3. Means of generating endings

[0250] The user reads the story and interacts with each scene, for example touching the wolf in the scene where Little Red Riding Hood encounters the wolf.

[0251] The device transmits touch position information and scene information to the server.

[0252] The server uses a generative AI method to generate a new ending based on the touch information and scene information.

[0253] The server transmits the generated ending data to the terminal.

[0254] 4. Emotional Engine Tools

[0255] The emotion engine detects the user's emotions, for example by using a camera to analyze the user's facial expressions and recognize emotions.

[0256] It can also use a microphone to collect the user's voice and recognize emotions through voice analysis.

[0257] The emotion engine sends the recognized emotion data to the server.

[0258] 5. Ending Display Method

[0259] The server also reflects the emotional data in the ending generation and generates a new ending.

[0260] The device analyzes the received ending data and displays it on the screen, allowing the user to enjoy the story to the end while viewing the new ending.

[0261] Specific examples

[0262] As a concrete example, consider the case of using the picture book "Little Red Riding Hood."

[0263] User Authentication

[0264] The user enters login information (e.g., username "tanaka" and password "pass1234") into the tablet. The device sends this information to the server, which then authenticates it. If authentication is successful, the server returns an authentication token to the device.

[0265] Picture book data acquisition

[0266] User "Tanaka" selects to read "Little Red Riding Hood." The device sends the request to the server, which retrieves the story data for "Little Red Riding Hood" and sends it to the device. The device displays the data, and the user begins reading the story.

[0267] Ending Generation

[0268] During the story, the user "Tanaka" touches the wolf's nose at the scene where Little Red Riding Hood encounters the wolf. The device sends this touch position information and scene information to the server.

[0269] Emotion Engine

[0270] The moment the user touches the screen, their facial expression and voice are collected via a camera and microphone and analyzed by the emotion engine, which then recognizes the user's emotion as "surprise."

[0271] Displaying the ending

[0272] The server uses a generation AI to generate a new ending, "Little Red Riding Hood is surprised by the wolf's true identity but becomes friends with him," based on touch information, scene information, and emotional data of "surprise." The generated ending data is sent to the device, which analyzes it and displays it on the screen. The user can enjoy the story to the end while watching the new ending.

[0273] The processing flow will be explained below.

[0274] Detailed process steps for implementing the invention

[0275] User Authentication and Login

[0276] Step 1:

[0277] The user starts up the tablet and goes to the login screen. The user enters their username and password and presses the "Login" button.

[0278] Step 2:

[0279] The terminal sends the username and password entered by the user to the server as an HTTP request.

[0280] Step 3:

[0281] The server checks the received username and password against the database and performs authentication. If authentication is successful, it generates an authentication token and returns it to the terminal. If authentication fails, it returns an error message to the terminal.

[0282] Acquiring picture book data

[0283] Step 4:

[0284] The user selects the picture book they want to read within the application, for example, "Little Red Riding Hood."

[0285] Step 5:

[0286] The terminal sends a request including information about the selected picture book to the server.

[0287] Step 6:

[0288] The server receives the request and retrieves the fixed story data for the corresponding picture book "Little Red Riding Hood" from the database.

[0289] Step 7:

[0290] The server transmits the acquired fixed story data to the terminal.

[0291] Step 8:

[0292] The device analyzes the received story data and displays it on the screen, allowing the user to begin reading the story.

[0293] Ending generation request

[0294] Step 9:

[0295] The user interacts with the story by touching and clicking, for example, by touching the wolf in the scene where Little Red Riding Hood encounters the wolf.

[0296] Step 10:

[0297] The terminal transmits an ending generation request including the user's touch position information and scene information to the server.

[0298] Emotion recognition by emotion engine

[0299] Step 11:

[0300] At the moment the user touches the device, it uses the camera to capture the user's facial expressions and collects the user's voice through the microphone.

[0301] Step 12:

[0302] The device sends the collected facial expression data and voice data to the emotion engine.

[0303] Step 13:

[0304] The emotion engine uses face recognition algorithms and voice analysis algorithms to analyze the user's emotions, for example, if the user shows the emotion "surprise", it will get the result.

[0305] Step 14:

[0306] The emotion engine sends the analysis results to the server.

[0307] Automatic ending generation

[0308] Step 15:

[0309] The server receives the emotion data along with the ending generation request, and passes the touch position, scene information, and emotion data to the generation AI means.

[0310] Step 16:

[0311] The generative AI method generates new endings based on this data. For example, based on the emotion data of "surprise," it generates an ending that includes a scene in which Little Red Riding Hood is surprised by the wolf's unexpected true identity.

[0312] Step 17:

[0313] The server transmits the generated ending data to the terminal.

[0314] Displaying the ending

[0315] Step 18:

[0316] The terminal analyzes the received ending data and displays it on the screen.

[0317] Step 19:

[0318] Users can enjoy the story to the end while seeing new endings.

[0319] This allows the invention to provide an interactive ending based on the user's emotions, resulting in a more personalized reading experience.

[0320] Example 2

[0321] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0322] Conventional digital picture book systems have the problem of being unable to provide dynamic endings that reflect the user's interactions and emotions. As a result, they are unable to generate personalized endings that correspond to the user's individual experience, resulting in a lack of immersion.

[0323] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0324] In this invention, the server includes means for receiving user authentication information and authenticating the user, means for receiving a request for picture book data and acquiring fixed story data for the picture book and transmitting it to the terminal, generation AI means for receiving user interaction data and generating an ending based on that data, emotion engine means for collecting the user's facial expressions and voice and recognizing their emotions, means for generating a new ending using the recognized emotion data, and means for transmitting the generated ending data to the terminal. This makes it possible to generate a personalized ending based on the user's interaction and emotion data, providing a more immersive story experience.

[0325] "User authentication information" refers to information provided by a user when logging in to a system, and includes a username and password.

[0326] An "authentication token" is temporary data sent from the server to the terminal when the user has been successfully authenticated, and indicates the user's permission to access the system.

[0327] A "picture book data request" is request data sent from a terminal to a server when a user intends to read a particular picture book.

[0328] "Fixed story data" is data that includes the basic storyline and content of a digital picture book, and is predefined.

[0329] "Interaction data" is data generated when a user operates a digital picture book, and includes touch positions and event information.

[0330] "Generative AI means" refers to artificial intelligence technology that dynamically generates content based on user interaction data.

[0331] An "emotion engine" is a combination of software and hardware that analyzes and recognizes emotions from a user's facial expressions and voice.

[0332] "Emotion data" is data obtained as a result of analysis by the emotion engine, and represents the user's emotional state.

[0333] A "new ending" is the conclusion of a digital storybook that is dynamically generated based on the user's interaction and emotion data.

[0334] A "terminal" is a hardware device that a user uses to read a digital picture book, typically a tablet or smartphone.

[0335] MODE FOR CARRYING OUT THE INVENTION

[0336] Overview of the invention

[0337] This invention relates to a digital picture book system with an ending that is automatically generated based on user interaction. Furthermore, it is characterized by incorporating an emotion engine that recognizes the user's emotions and reflects them in the generation of the ending. The detailed configuration and operation of the system are described below.

[0338] Hardware and software used

[0339] Device: The device on which a user reads a digital picture book. Examples include tablets and smartphones.

[0340] Server: A central system that processes and stores data, handles authentication, data transmission, generative AI models, etc.

[0341] Generative AI model: AI technology for generating new endings based on interaction and emotion data.

[0342] Emotion engine: Software that recognizes emotions by analyzing the user's facial expressions and voice.

[0343] Detailed System Description

[0344] 1. User Authentication

[0345] A user enters login information into a terminal. The terminal sends the username and password to the server. The server authenticates the user against a database and generates an authentication token, which is sent to the terminal, allowing the user access to the system.

[0346] 2. Acquiring picture book data

[0347] The user selects the picture book they want to read on their device. For example, they select "Little Red Riding Hood." The device then sends the request to the server, which receives the request, retrieves the corresponding picture book data from the database, and sends it. The device then analyzes the data and displays it.

[0348] 3. Collecting interactions

[0349] The user reads the picture book and interacts with it at a specific scene. For example, the user touches the wolf's nose in the scene where Little Red Riding Hood encounters the wolf. The device records this touch information (touch position and scene information) and sends it to the server.

[0350] 4. Collecting Emotional Data

[0351] When a user interacts with the device, the device's camera captures the user's facial expressions and the microphone collects their voice. The emotion engine analyzes this data and recognizes the user's emotions. The recognized emotion data is then sent to the server.

[0352] 5. Ending Generation

[0353] The server sends prompts to the generative AI model based on the interaction data and emotion data, and the generative AI model generates a new ending. An example of this prompt is, "The user is reading the story of Little Red Riding Hood. In the scene where Little Red Riding Hood encounters the wolf, she touches the wolf's nose. At this time, the user has a surprised expression. Based on this information, please generate a new ending for the story."

[0354] 6. Ending display

[0355] The server then sends the generated ending data to the device, which then analyzes it and displays it on the screen, allowing the user to enjoy the story to the end while viewing the new ending.

[0356] Specific examples

[0357] As a concrete example, the following uses the picture book "Little Red Riding Hood."

[0358] The user enters their login information on the tablet, and the server authenticates them. After successful authentication, the user selects "Little Red Riding Hood," and the server sends the corresponding data. If, during the story, the user touches the wolf's nose and its expression shows surprise, the server sends a prompt to the generative AI model, generating a new ending that reflects the emotion of surprise. Ultimately, the user can enjoy an ending in which Little Red Riding Hood becomes friends with the wolf, despite being surprised by its true identity.

[0359] In this way, the present invention can provide a more immersive narrative experience by providing a personalized ending that reflects the user's interactions and emotions.

[0360] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0361] Step 1:

[0362] User Authentication

[0363] The user enters login information (username and password) into the terminal. The terminal sends this login information to the server. The server compares it with a database and performs authentication. It receives the authentication information as input and outputs the authentication result and authentication token. If authentication is successful, the server generates an authentication token and sends it to the terminal. This allows the user to access the system. A specific example of how this works is when a user enters the username "tanaka" and password "pass1234" on the tablet login screen, and the server authenticates the information.

[0364] Step 2:

[0365] Acquiring picture book data

[0366] The user selects the picture book they want to read on their device. For example, they select "Little Red Riding Hood." The device then sends a request for that picture book data to the server. The server receives this request as input, retrieves the corresponding picture book data from the database, and sends it to the device. The retrieved picture book data can then be displayed on the device as output. A specific example of how this works is when the user "Tanaka" selects "Little Red Riding Hood," and the server retrieves and sends the story data.

[0367] Step 3:

[0368] Interaction Collection

[0369] As the user reads the picture book, they interact with the device at certain scenes. For example, in the scene where Little Red Riding Hood encounters the wolf, the user touches the wolf's nose. The device sends this touch information and scene information as input to the server. The server receives this and processes the interaction data. A specific example of this behavior is the scene where the user "Tanaka" touches the wolf's nose and the device sends that information to the server.

[0370] Step 4:

[0371] Collecting Emotional Data

[0372] At the moment a user interacts, the device's camera captures the user's facial expression and the microphone collects their voice. The emotion engine receives this data as input, analyzes it, and recognizes the user's emotion. This recognized emotion data is sent to the server and used as output. A specific example is when the user "Tanaka" shows a surprised expression the moment he touches the wolf, and the device analyzes this with the emotion engine to generate emotion data for "surprise."

[0373] Step 5:

[0374] Ending Generation

[0375] The server sends the received interaction data and emotion data to the generative AI model as a prompt. The generative AI model receives this prompt as input and generates a new ending. The generated ending data is output and sent from the server to the device. A specific example of operation is when the server sends a prompt to the generative AI model saying, "The user is reading the story of Little Red Riding Hood. In the scene where Little Red Riding Hood encounters the wolf, he touches the wolf's nose. At this time, the user has a surprised expression. Based on this information, please generate a new ending for the story.", and a new ending is generated.

[0376] Step 6:

[0377] Displaying the ending

[0378] The device receives the new ending data from the server, analyzes it, and displays it on the screen. The user can enjoy the story while watching this new ending. Specifically, the device displays the ending in which Little Red Riding Hood becomes friends with the wolf despite being surprised by its true identity, and the user watches it.

[0379] (Application example 2)

[0380] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0381] Conventional digital picture book systems have limited ability to generate endings based on user interactions, resulting in a uniform user experience. In particular, they do not take user emotions into consideration, making it difficult to provide a personalized experience for each user. This limits the immersiveness and enjoyment of the story.

[0382] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0383] In this invention, the server includes means for receiving user authentication information and authenticating the user, means for receiving a request for picture book data and acquiring fixed story data for the picture book and transmitting it to the terminal, generation AI means for receiving user interaction data and generating an ending based on that data, an emotion engine for collecting, analyzing, and recognizing user emotion data, means for using a generation AI model to generate a custom experience based on the recognized emotion data and interaction data, and means for transmitting the generated ending data to the terminal. This makes it possible to generate a personalized ending based on the user's emotions and interactions, providing a deeper sense of immersion and enjoyment than ever before.

[0384] "User authentication information" refers to information required for a user to log in to a system, and typically refers to a username and password.

[0385] "Means of authentication" refers to a mechanism for verifying the identity of a user based on authentication information provided by the user.

[0386] "Picture book data request" refers to the act of a user requesting data of a picture book they want to read from a server.

[0387] "Fixed story data of a picture book" refers to predetermined story data of a picture book.

[0388] "Interaction Data" is data that describes how a user interacts with a system, including touch locations and options selected.

[0389] "Generative AI means" refers to artificial intelligence technology for generating new endings based on user interaction data.

[0390] An "emotion engine" is a system that detects and analyzes a user's emotions, typically using a camera and microphone to analyze facial expressions and voice.

[0391] "Emotion data" is data that indicates the user's emotional state, and is classified into categories such as joy, surprise, sadness, and anger.

[0392] "Custom experience" refers to an experience that is individually generated based on the user's emotional and interaction data.

[0393] "Ending data" refers to data about the ending of a story, created by a generative AI method.

[0394] The present invention is a digital picture book system that generates custom experiences and endings based on user interaction and emotional data. Detailed embodiments of the present invention will be described below.

[0395] overview

[0396] The present invention consists of the following means: user authentication, acquisition of picture book data, collection of interaction data, emotion recognition, ending generation using a generation AI means, and custom experience generation.

[0397] Components

[0398] 1. User authentication method

[0399] The user enters login information (e.g., username and password) into the terminal and sends the authentication information to the server.

[0400] The server verifies the user's identity based on the authentication information, and if authentication is successful, generates an authentication token and sends it to the terminal.

[0401] 2. How to obtain picture book data

[0402] The user selects the picture book they want to read on the terminal, and the terminal sends the request to the server.

[0403] The server retrieves the fixed story data of the relevant picture book from the database and transmits it to the terminal.

[0404] 3. Methods of collecting interaction data

[0405] As the user progresses through the story, touch location information is collected for specific scenes.

[0406] The terminal transmits this interaction data to the server.

[0407] 4. Emotion Engine

[0408] To analyze the user's facial expression, a facial image is acquired using the device's camera.

[0409] It also uses a microphone to collect the user's voice for voice analysis.

[0410] Based on the acquired data, the emotion engine recognizes the user's emotions and sends them to the server as emotion data.

[0411] 5. Generation AI means

[0412] Based on emotional and interaction data, generative AI methods generate new endings and custom experiences.

[0413] The server uses generative AI means to generate prompts based on the user's interaction data and emotion data, such as:

[0414] Example prompt: "Based on the user's emotion data: Surprise, interaction data: Product selection: Bag, Time: 2 PM, generate a custom shopping experience to further engage the user. For example, include features of the selected bag and special offers."

[0415] 6. Ending data transmission method

[0416] The server sends the generated ending data and custom experience data to the terminal.

[0417] The terminal analyzes the received data and displays it to the user.

[0418] Specific examples

[0419] For example, a user selects the picture book "Little Red Riding Hood" on a device and, as they read, touches the wolf in the scene where they encounter it. At this time, the device's camera captures the user's face, which is analyzed by the emotion engine and the user's emotion is recognized as "surprise." Based on the user's emotion data and interaction data, the server uses a generative AI method to generate an ending such as "Little Red Riding Hood is surprised by the wolf's true identity, but becomes friends with it." The generated ending data is sent to the device, allowing the user to enjoy the new ending.

[0420] As described above, the present invention makes it possible to provide a personalized story experience based on the user's emotions and interactions, thereby achieving a deeper sense of immersion and enjoyment.

[0421] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0422] Step 1:

[0423] The user enters login information (user name and password) into the terminal. The terminal sends this authentication information to the server. The server checks the received authentication information against a database, and if authentication is successful, generates an authentication token and sends it to the terminal. The input is the user's authentication information, and the output is the authentication token.

[0424] Step 2:

[0425] The user selects a picture book they want to read on their device. The device requests fixed story data for the picture book and sends this request to the server. The server retrieves the relevant picture book data from the database based on the received request and sends it to the device. The input is the picture book selection request, and the output is the fixed story data for the picture book.

[0426] Step 3:

[0427] As the user reads the story, they input touch position information for specific scenes. The device records this touch position information and sends it to the server. The input is the touch position information, and the output is the transmitted touch position information.

[0428] Step 4:

[0429] The device uses a camera to capture images of the user's face and simultaneously collects the user's voice through a microphone. This data is sent to an emotion engine to analyze the user's emotions. For example, the emotion "surprise" can be detected from facial expressions. The input is the user's face image and voice data, and the output is analyzed emotion data.

[0430] Step 5:

[0431] The server receives the touch location information obtained in step 3 and the emotional data analyzed in step 4. Based on this data, it uses generative AI means to generate a new ending or custom experience, using prompts such as the following:

[0432] Example prompt: "Based on the user's emotion data: Surprise, interaction data: Product selection: Bag, Time: 2 PM, generate a custom shopping experience to further engage the user. For example, include features of the selected bag and special offers."

[0433] The input is emotion data and interaction data, and the output is the generated ending data.

[0434] Step 6:

[0435] The server sends the ending data generated by the AI ​​generation means to the terminal. The terminal analyzes the received ending data and displays it to the user. The input is the generated ending data, and the output is the ending displayed to the user.

[0436] This allows users to enjoy new endings and provides a personalized story experience.

[0437] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0438] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0439] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0440] [Second embodiment]

[0441] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0442] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0443] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0444] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0445] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0446] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0447] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0448] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0449] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0450] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0451] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0452] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0453] Overview of the invention

[0454] The present invention relates to an interactive digital picture book system that generates different endings based on specific conditions. The system comprises the following means.

[0455] 1. User authentication methods

[0456] When a user logs in using a device such as a tablet, they enter authentication information (user name and password).

[0457] The terminal sends this authentication information to the server, which then checks it against a database to authenticate the user.

[0458] If authentication is successful, the server generates an authentication token and sends it to the terminal, allowing the user to access the system.

[0459] 2. How to obtain picture book data

[0460] The user selects the picture book they want to read on the device. For example, consider the case where they select "Little Red Riding Hood."

[0461] The terminal sends a picture book data request to the server.

[0462] The server receives the request and retrieves the fixed story data for "Little Red Riding Hood" from the database.

[0463] The server sends the acquired story data to the terminal, which analyzes it and displays it on the screen.

[0464] 3. Means of generating endings

[0465] The user reads the story and interacts with each scene, for example touching the wolf in the scene where Little Red Riding Hood encounters the wolf.

[0466] The device sends touch location information to the server, which includes the location where the touch occurred and the scene context.

[0467] The server uses a generative AI means to generate a new ending based on the touch information.

[0468] The server transmits the generated ending data to the terminal.

[0469] 4. Ending Display Method

[0470] The terminal analyzes the received ending data and displays it on the screen.

[0471] Users can enjoy new endings.

[0472] Specific examples

[0473] As a concrete example, consider the case of using the picture book "Little Red Riding Hood."

[0474] User Authentication

[0475] The user enters login information (e.g., username "tanaka" and password "pass1234") into the tablet. The device sends this information to the server, which then authenticates it. If authentication is successful, the server returns an authentication token to the device.

[0476] Picture book data acquisition

[0477] User "Tanaka" selects to read "Little Red Riding Hood." The device sends the request to the server, which retrieves the story data for "Little Red Riding Hood" and sends it to the device. The device displays the data, and the user begins reading the story.

[0478] Ending Generation

[0479] During the story, the user "Tanaka" touches the wolf's nose when Little Red Riding Hood encounters the wolf. The device sends this touch information to the server. The server uses a generation AI method to generate a new ending, "Little Red Riding Hood becomes friends with the wolf," based on the input touch information. The server then sends this ending data to the device.

[0480] Displaying the ending

[0481] The device analyzes the received ending data and displays the new ending to the user "Tanaka." The user and their child can then enjoy the new ending.

[0482] In this way, the present invention makes the reading experience more diverse and interactive than conventional picture books, and makes parent-child reading time more fulfilling.

[0483] The processing flow will be explained below.

[0484] Program processing

[0485] User Authentication and Login

[0486] Step 1:

[0487] The user starts the application and goes to the login screen. The user enters their username and password and presses the "Login" button.

[0488] Step 2:

[0489] The terminal sends the username and password entered by the user to the server as an HTTP request.

[0490] Step 3:

[0491] The server checks the received username and password against the database and performs authentication. If authentication is successful, it generates an authentication token and returns it to the terminal. If authentication fails, it returns an error message to the terminal.

[0492] Acquiring picture book data

[0493] Step 4:

[0494] The user selects the picture book they want to read within the application, for example, "Little Red Riding Hood."

[0495] Step 5:

[0496] The terminal sends a request including information about the selected picture book to the server.

[0497] Step 6:

[0498] The server receives the request and retrieves the fixed story data for the corresponding picture book "Little Red Riding Hood" from the database.

[0499] Step 7:

[0500] The server transmits the acquired fixed story data to the terminal.

[0501] Step 8:

[0502] The device analyzes the received story data and displays it on the screen, allowing the user to begin reading the story.

[0503] Ending generation request

[0504] Step 9:

[0505] The user interacts with the story by touching and clicking, for example, by touching the wolf in the scene where Little Red Riding Hood encounters the wolf.

[0506] Step 10:

[0507] The terminal sends an ending generation request including the user's touch position and scene information to the server.

[0508] Automatic ending generation

[0509] Step 11:

[0510] The server receives the ending generation request and passes the touch position and scene information to the generation AI means.

[0511] Step 12:

[0512] The AI ​​generator generates a new ending based on the touch position and scene information, for example, an ending in which Little Red Riding Hood becomes friends with the wolf.

[0513] Step 13:

[0514] The server transmits the generated ending data to the terminal.

[0515] Displaying the ending

[0516] Step 14:

[0517] The terminal analyzes the received ending data and displays it on the screen.

[0518] Step 15:

[0519] Users can enjoy the story to the end while seeing new endings.

[0520] By following these steps, a different ending is generated each time, making the reading experience fresh.

[0521] Example 1

[0522] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0523] Digital picture books have become popular in recent years and are attracting attention as a form of education and entertainment for children. However, existing digital picture books have a fixed storyline and lack interactive elements that incorporate user interaction, which can lead to a one-sided reading experience that can easily become boring. Therefore, the present invention aims to provide a system that makes the reading experience more enjoyable and interactive by generating different endings based on user interaction.

[0524] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0525] In this invention, the server includes means for receiving user authentication information and authenticating the user, means for receiving a request for picture book data and acquiring fixed story data for the picture book and transmitting it to the terminal, generation AI means for receiving user interaction data and generating an ending based on that data, means for transmitting the generated ending data to the terminal, and means for analyzing the ending data displayed on the terminal and displaying it on the screen. This allows the user to interact with the picture book as they read it, resulting in a variety of endings and an interactive and engaging story-telling experience.

[0526] "User credentials" are data entered by a user to grant access to a system, typically including a username and password.

[0527] "Means for authenticating" refers to the processes and functions for receiving a user's authentication information, checking it against a database, and verifying the user's access rights.

[0528] A "request for picture book data" is an operation performed when a user wants to read a digital picture book and requests the contents of that picture book from the server.

[0529] "Fixed story data" refers to a set of pre-defined story structure and content data for a particular picture book.

[0530] "User interaction data" is information about operations and actions performed by the user while reading a picture book, and includes data such as touch position and timing.

[0531] "Generative AI means" refers to algorithms or processes that use artificial intelligence to generate new outputs based on specific input data.

[0532] "Ending data" is data that describes the content of the final part of the story, and is generated based on the user's interactions.

[0533] "Means for transmitting to the terminal" refers to the function by which the server processes data and sends the data over the network to the device used by the user.

[0534] "Means for displaying on the screen" refers to the process of analyzing the received data, outputting it in an appropriate format to the user interface, and displaying it on the terminal screen.

[0535] "Interactive elements" refer to a mechanism in which users directly interact with the system through operations and actions, and the results are reflected in real time.

[0536] "Specific scene information" is data about a specific scene in a story, and includes the unique characteristics and context of that scene.

[0537] This invention relates to an interactive digital picture book system that generates different endings based on specific conditions. This system uses a generative AI model to generate various endings based on the user's interactions while reading the digital picture book, providing the user with a new reading experience.

[0538] The system mainly consists of hardware including a server, terminals, and authentication database, and software that runs the generative AI model.

[0539] User authentication methods

[0540] The user enters login information (username, password) using a device such as a tablet. The device sends this authentication information to the server. The server receives the authentication information and authenticates the user by checking it against a database. If authentication is successful, the server generates an authentication token and sends it to the device. This token allows the user to access the system.

[0541] How to obtain picture book data

[0542] The user selects the picture book they want to read on their device. For example, if the user selects "Little Red Riding Hood," the device sends a picture book data request to the server. The server retrieves the fixed story data for "Little Red Riding Hood" from the database and sends that data to the device. The device analyzes the retrieved story data and displays it on the screen.

[0543] Means of generating endings

[0544] As the user reads the story, they interact with each scene. For example, if the user touches the wolf's nose in the scene where Little Red Riding Hood encounters the wolf, the device sends this touch position information and the scene context to the server. The server uses a generative AI model to generate a new ending based on this touch information. Specifically, the following prompt is input into the generative AI model: "Generate a story in which Little Red Riding Hood touches the wolf's nose and, as a result, becomes friends with the wolf."

[0545] The generated ending data is transmitted from the server to the terminal.

[0546] Ending display method

[0547] The device analyzes the received ending data and displays it on the screen, allowing the user to enjoy new endings. The process of generating endings is done in real time, and is designed to be instantly updated based on the user's interactions.

[0548] Specific examples

[0549] As a concrete example, consider the case where user "Tanaka" enters login information into a tablet device, is authenticated, and then selects "Little Red Riding Hood." The device receives story data from the server, and the content is displayed on the screen. During the story, user "Tanaka" touches the wolf's nose, and that information is sent to the server. The server uses a generative AI model to generate a new ending in which "Little Red Riding Hood becomes friends with the wolf" and sends it to the device. The device displays the new ending on the screen, and the user can enjoy the new conclusion to the story.

[0550] In this way, the present invention provides a system that generates a variety of endings based on user interactions, making the storytelling experience more interactive and enjoyable.

[0551] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0552] Step 1:

[0553] User enters login information

[0554] The user enters their username and password into the tablet device, which then sends this authentication information to the server.

[0555] Input: Username, Password

[0556] Output: Authentication information sent to the server

[0557] Step 2:

[0558] The server verifies the authentication information

[0559] The server compares the received authentication information with the database and determines the authentication result. If authentication is successful, it generates an authentication token and sends it to the terminal.

[0560] Input: Credentials

[0561] Data processing: Verification of authentication information, generation of authentication tokens

[0562] Output: Authentication token sent to the device

[0563] Step 3:

[0564] User selects a picture book

[0565] The user selects the picture book they want to read on their device, for example, "Little Red Riding Hood," and the device sends the request to the server.

[0566] Input: Picture book selection request

[0567] Output: The request sent to the server

[0568] Step 4:

[0569] The server retrieves the picture book data

[0570] The server retrieves the fixed story data of the relevant picture book from the database and sends the retrieved data to the terminal.

[0571] Input: Picture book selection request

[0572] Data processing: Acquiring story data

[0573] Output: Story data sent to device

[0574] Step 5:

[0575] The device displays the picture book data

[0576] The device analyzes the received picture book data and displays it on the screen, allowing the user to begin reading the story.

[0577] Input: Story data

[0578] Data processing: Data analysis

[0579] Output: Picture book data displayed on the screen

[0580] Step 6:

[0581] The user interacts

[0582] When the user interacts with the story (e.g., touching the wolf's nose) in a specific scene, the device sends the touch location information and the scene context to the server.

[0583] Input: Interaction (touch information)

[0584] Output: Touch information sent to the server

[0585] Step 7:

[0586] The server generates a new ending

[0587] The server uses a generative AI model to generate a new ending based on the touch information it receives.

[0588] Example prompt: "Generate a story in which Little Red Riding Hood touches the wolf's nose and becomes friends with him."

[0589] Input: Touch information, prompt text

[0590] Data processing: Generative AI model generates endings

[0591] Output: Ending data sent to the terminal

[0592] Step 8:

[0593] The device displays a new ending.

[0594] The device analyzes the received ending data and displays it on the screen, allowing the user to enjoy the new ending.

[0595] Input: Ending data

[0596] Data processing: Data analysis

[0597] Output: New ending shown on screen

[0598] (Application example 1)

[0599] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0600] Conventional picture books have a fixed storyline, meaning that once you've read it, you end up reading the same content again and again. This causes problems such as a lack of freshness for both children and parents when reading, and a lack of interactive elements. Also, because they only allow for a predetermined ending, it's difficult to provide real-time interaction or variety in the storyline based on user choices.

[0601] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0602] In this invention, the server includes means for receiving user authentication information and authenticating the user, means for receiving a picture book data request and acquiring fixed story data for the picture book and transmitting it to the information terminal, generation AI means for receiving user interaction data and generating an ending based on that data, means for transmitting the generated ending data to the information terminal, and means for analyzing and displaying the ending data, thereby making it possible to generate and display new endings in real time.

[0603] "User authentication information" refers to information required when a user accesses a system, such as a username and password.

[0604] A "request for picture book data" refers to the act of requesting data for a picture book that the user wants to read, or the request information.

[0605] "Fixed story data" refers to data of a predetermined series of stories in a particular picture book.

[0606] An "information terminal" is an electronic device used by a user, such as a smartphone, tablet, or computer.

[0607] "Interaction data" refers to information about operations and inputs performed by the user while reading a picture book, and includes, for example, touch position information.

[0608] "Generative AI means" refers to artificial intelligence technology used to generate new endings based on received data.

[0609] "Ending data" is data regarding the ending of a story created by a generative AI means.

[0610] The "means for analyzing and displaying" is a means having the function of understanding the received ending data and displaying it on an information terminal.

[0611] The present invention relates to an interactive digital picture book system that generates different endings based on specific conditions. The following is a detailed description showing how to specifically implement the present invention.

[0612] The server implements the invention using the following means.

[0613] 1. User Authentication

[0614] When a user logs in using a device such as a tablet or smartphone, they enter authentication information (username, password). The device sends this authentication information to the server, which then authenticates the user by checking it against a database. If authentication is successful, the server generates an authentication token and sends it to the device, allowing the user to access the system. The software used is Flask to perform the authentication process.

[0615] 2. Acquiring picture book data

[0616] The user selects a picture book they want to read on their device. For example, consider the case where "Little Red Riding Hood" is selected. The device sends a picture book data request to the server. The server receives the request and retrieves the fixed story data for "Little Red Riding Hood" from the database. The server then sends the retrieved story data to the device, which parses it and displays it on the screen. This process involves database operations and sending and receiving data in JSON format.

[0617] 3. Ending Generation

[0618] The user reads the story and interacts with each scene. For example, in the scene where Little Red Riding Hood encounters the wolf, the user touches the wolf's nose. The device sends touch location information to the server, which includes the location of the touch and the scene context. The server uses a generative AI method to generate a new ending based on the touch information. The generative AI model used is OpenAI's GPT-4. This model generates prompts based on specific scene information and the user's interaction data, and generates a new ending.

[0619] 4. Ending display

[0620] The server sends the generated ending data to the device. The device analyzes the received ending data and displays it on the screen. The user can enjoy the new ending. The device uses JavaScript and HTML to analyze the data and display it on the user interface.

[0621] Specific examples

[0622] If the user selects "Little Red Riding Hood" and touches the wolf's ear in the scene where Little Red Riding Hood encounters the wolf, this touch information is sent to the server. The server generates the following prompt sentence and passes it to the generation AI.

[0623] Prompt Sentence Examples

[0624] Generate an ending for the following scene in the story of Little Red Riding Hood. Scene: Little Red Riding Hood encounters the wolf. Touch location: Wolf's ear.

[0625] Based on this prompt, the AI ​​generates a new ending, "Little Red Riding Hood becomes friends with the wolf," and the server sends it to the device. The device displays this ending on the screen, allowing the user to enjoy the conclusion of the new story.

[0626] In this way, the present invention can provide a more diverse and interactive reading experience than conventional picture books, making parent-child reading time more fulfilling.

[0627] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0628] Step 1:

[0629] The user enters login information (e.g., username and password) into the terminal.

[0630] Input: Username and Password

[0631] How it works: The device sends these authentication information to the server, which checks it against a database and authenticates the user. If authentication is successful, the server generates an authentication token and returns it to the device.

[0632] Output: Authentication token

[0633] Step 2:

[0634] The user selects the picture book they want to read on the device.

[0635] Input: Picture book selection information (e.g., "Little Red Riding Hood")

[0636] Operation: The device sends a picture book data request to the server. The server receives the request, retrieves the fixed story data for the corresponding picture book from the database, and sends it to the device. The device analyzes the received story data and displays it on the screen.

[0637] Output: Fixed story data

[0638] Step 3:

[0639] The user reads the story and interacts with it (e.g., by touch) during the story.

[0640] Input: Touch location and scene context

[0641] How it works: The device sends touch location information and scene context to the server, which uses a generative AI model to generate a prompt based on that information.

[0642] Output: prompt statement

[0643] Step 4:

[0644] The server generates a new ending using a generative AI model.

[0645] Input: prompt statement

[0646] How it works: A generative AI model (e.g., OpenAI's GPT-4) generates an ending based on the prompt. The server retrieves the generated ending data and sends it to the device.

[0647] Output: New ending data

[0648] Step 5:

[0649] The device will analyze the new ending data and display it on the screen.

[0650] Input: New ending data

[0651] Behavior: The device parses the received ending data and converts it into a format suitable for the user interface. After parsing, the device displays the new ending on the screen.

[0652] Output: New ending displayed on screen

[0653] This allows users to enjoy new endings generated in real time based on specific interactions.

[0654] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0655] Overview of the invention

[0656] The present invention relates to a digital picture book system that automatically generates an ending based on user interaction. Furthermore, the system is characterized by incorporating an emotion engine that recognizes the user's emotions and reflecting them in the generation of the ending. The system is composed of the following means:

[0657] 1. User authentication methods

[0658] When a user logs in using a device such as a tablet, they enter authentication information (user name and password).

[0659] The terminal sends this authentication information to the server, which then checks it against a database to authenticate the user.

[0660] If authentication is successful, the server generates an authentication token and sends it to the terminal, allowing the user to access the system.

[0661] 2. How to obtain picture book data

[0662] The user selects the picture book they want to read on the device, for example, "Little Red Riding Hood."

[0663] The terminal sends a picture book data request to the server.

[0664] The server receives the request and retrieves the fixed story data for the corresponding picture book "Little Red Riding Hood" from the database.

[0665] The server sends the acquired story data to the terminal, which analyzes it and displays it on the screen.

[0666] 3. Means of generating endings

[0667] The user reads the story and interacts with each scene, for example touching the wolf in the scene where Little Red Riding Hood encounters the wolf.

[0668] The device transmits touch position information and scene information to the server.

[0669] The server uses a generative AI method to generate a new ending based on the touch information and scene information.

[0670] The server transmits the generated ending data to the terminal.

[0671] 4. Emotional Engine Tools

[0672] The emotion engine detects the user's emotions, for example by using a camera to analyze the user's facial expressions and recognize emotions.

[0673] It can also use a microphone to collect the user's voice and recognize emotions through voice analysis.

[0674] The emotion engine sends the recognized emotion data to the server.

[0675] 5. Ending Display Method

[0676] The server also reflects the emotional data in the ending generation and generates a new ending.

[0677] The device analyzes the received ending data and displays it on the screen, allowing the user to enjoy the story to the end while viewing the new ending.

[0678] Specific examples

[0679] As a concrete example, consider the case of using the picture book "Little Red Riding Hood."

[0680] User Authentication

[0681] The user enters login information (e.g., username "tanaka" and password "pass1234") into the tablet. The device sends this information to the server, which then authenticates it. If authentication is successful, the server returns an authentication token to the device.

[0682] Picture book data acquisition

[0683] User "Tanaka" selects to read "Little Red Riding Hood." The device sends the request to the server, which retrieves the story data for "Little Red Riding Hood" and sends it to the device. The device displays the data, and the user begins reading the story.

[0684] Ending Generation

[0685] During the story, the user "Tanaka" touches the wolf's nose at the scene where Little Red Riding Hood encounters the wolf. The device sends this touch position information and scene information to the server.

[0686] Emotion Engine

[0687] The moment the user touches the screen, their facial expression and voice are collected via a camera and microphone and analyzed by the emotion engine, which then recognizes the user's emotion as "surprise."

[0688] Displaying the ending

[0689] The server uses a generation AI to generate a new ending, "Little Red Riding Hood is surprised by the wolf's true identity but becomes friends with him," based on touch information, scene information, and emotional data of "surprise." The generated ending data is sent to the device, which analyzes it and displays it on the screen. The user can enjoy the story to the end while watching the new ending.

[0690] The processing flow will be explained below.

[0691] Detailed process steps for implementing the invention

[0692] User Authentication and Login

[0693] Step 1:

[0694] The user starts up the tablet and goes to the login screen. The user enters their username and password and presses the "Login" button.

[0695] Step 2:

[0696] The terminal sends the username and password entered by the user to the server as an HTTP request.

[0697] Step 3:

[0698] The server checks the received username and password against the database and performs authentication. If authentication is successful, it generates an authentication token and returns it to the terminal. If authentication fails, it returns an error message to the terminal.

[0699] Acquiring picture book data

[0700] Step 4:

[0701] The user selects the picture book they want to read within the application, for example, "Little Red Riding Hood."

[0702] Step 5:

[0703] The terminal sends a request including information about the selected picture book to the server.

[0704] Step 6:

[0705] The server receives the request and retrieves the fixed story data for the corresponding picture book "Little Red Riding Hood" from the database.

[0706] Step 7:

[0707] The server transmits the acquired fixed story data to the terminal.

[0708] Step 8:

[0709] The device analyzes the received story data and displays it on the screen, allowing the user to begin reading the story.

[0710] Ending generation request

[0711] Step 9:

[0712] The user interacts with the story by touching and clicking, for example, by touching the wolf in the scene where Little Red Riding Hood encounters the wolf.

[0713] Step 10:

[0714] The terminal transmits an ending generation request including the user's touch position information and scene information to the server.

[0715] Emotion recognition by emotion engine

[0716] Step 11:

[0717] At the moment the user touches the device, it uses the camera to capture the user's facial expressions and collects the user's voice through the microphone.

[0718] Step 12:

[0719] The device sends the collected facial expression data and voice data to the emotion engine.

[0720] Step 13:

[0721] The emotion engine uses face recognition algorithms and voice analysis algorithms to analyze the user's emotions, for example, if the user shows the emotion "surprise", it will get the result.

[0722] Step 14:

[0723] The emotion engine sends the analysis results to the server.

[0724] Automatic ending generation

[0725] Step 15:

[0726] The server receives the emotion data along with the ending generation request, and passes the touch position, scene information, and emotion data to the generation AI means.

[0727] Step 16:

[0728] The generative AI method generates new endings based on this data. For example, based on the emotion data of "surprise," it generates an ending that includes a scene in which Little Red Riding Hood is surprised by the wolf's unexpected true identity.

[0729] Step 17:

[0730] The server transmits the generated ending data to the terminal.

[0731] Displaying the ending

[0732] Step 18:

[0733] The terminal analyzes the received ending data and displays it on the screen.

[0734] Step 19:

[0735] Users can enjoy the story to the end while seeing new endings.

[0736] This allows the invention to provide an interactive ending based on the user's emotions, resulting in a more personalized reading experience.

[0737] Example 2

[0738] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0739] Conventional digital picture book systems have the problem of being unable to provide dynamic endings that reflect the user's interactions and emotions. As a result, they are unable to generate personalized endings that correspond to the user's individual experience, resulting in a lack of immersion.

[0740] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0741] In this invention, the server includes means for receiving user authentication information and authenticating the user, means for receiving a request for picture book data and acquiring fixed story data for the picture book and transmitting it to the terminal, generation AI means for receiving user interaction data and generating an ending based on that data, emotion engine means for collecting the user's facial expressions and voice and recognizing their emotions, means for generating a new ending using the recognized emotion data, and means for transmitting the generated ending data to the terminal. This makes it possible to generate a personalized ending based on the user's interaction and emotion data, providing a more immersive story experience.

[0742] "User authentication information" refers to information provided by a user when logging in to a system, and includes a username and password.

[0743] An "authentication token" is temporary data sent from the server to the terminal when the user has been successfully authenticated, and indicates the user's permission to access the system.

[0744] A "picture book data request" is request data sent from a terminal to a server when a user intends to read a particular picture book.

[0745] "Fixed story data" is data that includes the basic storyline and content of a digital picture book, and is predefined.

[0746] "Interaction data" is data generated when a user operates a digital picture book, and includes touch positions and event information.

[0747] "Generative AI means" refers to artificial intelligence technology that dynamically generates content based on user interaction data.

[0748] An "emotion engine" is a combination of software and hardware that analyzes and recognizes emotions from a user's facial expressions and voice.

[0749] "Emotion data" is data obtained as a result of analysis by the emotion engine, and represents the user's emotional state.

[0750] A "new ending" is the conclusion of a digital storybook that is dynamically generated based on the user's interaction and emotion data.

[0751] A "terminal" is a hardware device that a user uses to read a digital picture book, typically a tablet or smartphone.

[0752] MODE FOR CARRYING OUT THE INVENTION

[0753] Overview of the invention

[0754] This invention relates to a digital picture book system with an ending that is automatically generated based on user interaction. Furthermore, it is characterized by incorporating an emotion engine that recognizes the user's emotions and reflects them in the generation of the ending. The detailed configuration and operation of the system are described below.

[0755] Hardware and software used

[0756] Device: The device on which a user reads a digital picture book. Examples include tablets and smartphones.

[0757] Server: A central system that processes and stores data, handles authentication, data transmission, generative AI models, etc.

[0758] Generative AI model: AI technology for generating new endings based on interaction and emotion data.

[0759] Emotion engine: Software that recognizes emotions by analyzing the user's facial expressions and voice.

[0760] Detailed System Description

[0761] 1. User Authentication

[0762] A user enters login information into a terminal. The terminal sends the username and password to the server. The server authenticates the user against a database and generates an authentication token, which is sent to the terminal, allowing the user access to the system.

[0763] 2. Acquiring picture book data

[0764] The user selects the picture book they want to read on their device. For example, they select "Little Red Riding Hood." The device then sends the request to the server, which receives the request, retrieves the corresponding picture book data from the database, and sends it. The device then analyzes the data and displays it.

[0765] 3. Collecting interactions

[0766] The user reads the picture book and interacts with it at a specific scene. For example, the user touches the wolf's nose in the scene where Little Red Riding Hood encounters the wolf. The device records this touch information (touch position and scene information) and sends it to the server.

[0767] 4. Collecting Emotional Data

[0768] When a user interacts with the device, the device's camera captures the user's facial expressions and the microphone collects their voice. The emotion engine analyzes this data and recognizes the user's emotions. The recognized emotion data is then sent to the server.

[0769] 5. Ending Generation

[0770] The server sends prompts to the generative AI model based on the interaction data and emotion data, and the generative AI model generates a new ending. An example of this prompt is, "The user is reading the story of Little Red Riding Hood. In the scene where Little Red Riding Hood encounters the wolf, she touches the wolf's nose. At this time, the user has a surprised expression. Based on this information, please generate a new ending for the story."

[0771] 6. Ending display

[0772] The server then sends the generated ending data to the device, which then analyzes it and displays it on the screen, allowing the user to enjoy the story to the end while viewing the new ending.

[0773] Specific examples

[0774] As a concrete example, the following uses the picture book "Little Red Riding Hood."

[0775] The user enters their login information on the tablet, and the server authenticates them. After successful authentication, the user selects "Little Red Riding Hood," and the server sends the corresponding data. If, during the story, the user touches the wolf's nose and its expression shows surprise, the server sends a prompt to the generative AI model, generating a new ending that reflects the emotion of surprise. Ultimately, the user can enjoy an ending in which Little Red Riding Hood becomes friends with the wolf, despite being surprised by its true identity.

[0776] In this way, the present invention can provide a more immersive narrative experience by providing a personalized ending that reflects the user's interactions and emotions.

[0777] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0778] Step 1:

[0779] User Authentication

[0780] The user enters login information (username and password) into the terminal. The terminal sends this login information to the server. The server compares it with a database and performs authentication. It receives the authentication information as input and outputs the authentication result and authentication token. If authentication is successful, the server generates an authentication token and sends it to the terminal. This allows the user to access the system. A specific example of how this works is when a user enters the username "tanaka" and password "pass1234" on the tablet login screen, and the server authenticates the information.

[0781] Step 2:

[0782] Acquiring picture book data

[0783] The user selects the picture book they want to read on their device. For example, they select "Little Red Riding Hood." The device then sends a request for that picture book data to the server. The server receives this request as input, retrieves the corresponding picture book data from the database, and sends it to the device. The retrieved picture book data can then be displayed on the device as output. A specific example of how this works is when the user "Tanaka" selects "Little Red Riding Hood," and the server retrieves and sends the story data.

[0784] Step 3:

[0785] Interaction Collection

[0786] As the user reads the picture book, they interact with the device at certain scenes. For example, in the scene where Little Red Riding Hood encounters the wolf, the user touches the wolf's nose. The device sends this touch information and scene information as input to the server. The server receives this and processes the interaction data. A specific example of this behavior is the scene where the user "Tanaka" touches the wolf's nose and the device sends that information to the server.

[0787] Step 4:

[0788] Collecting Emotional Data

[0789] At the moment a user interacts, the device's camera captures the user's facial expression and the microphone collects their voice. The emotion engine receives this data as input, analyzes it, and recognizes the user's emotion. This recognized emotion data is sent to the server and used as output. A specific example is when the user "Tanaka" shows a surprised expression the moment he touches the wolf, and the device analyzes this with the emotion engine to generate emotion data for "surprise."

[0790] Step 5:

[0791] Ending Generation

[0792] The server sends the received interaction data and emotion data to the generative AI model as a prompt. The generative AI model receives this prompt as input and generates a new ending. The generated ending data is output and sent from the server to the device. A specific example of operation is when the server sends a prompt to the generative AI model saying, "The user is reading the story of Little Red Riding Hood. In the scene where Little Red Riding Hood encounters the wolf, he touches the wolf's nose. At this time, the user has a surprised expression. Based on this information, please generate a new ending for the story.", and a new ending is generated.

[0793] Step 6:

[0794] Displaying the ending

[0795] The device receives the new ending data from the server, analyzes it, and displays it on the screen. The user can enjoy the story while watching this new ending. Specifically, the device displays the ending in which Little Red Riding Hood becomes friends with the wolf despite being surprised by its true identity, and the user watches it.

[0796] (Application example 2)

[0797] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0798] Conventional digital picture book systems have limited ability to generate endings based on user interactions, resulting in a uniform user experience. In particular, they do not take user emotions into consideration, making it difficult to provide a personalized experience for each user. This limits the immersiveness and enjoyment of the story.

[0799] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0800] In this invention, the server includes means for receiving user authentication information and authenticating the user, means for receiving a request for picture book data and acquiring fixed story data for the picture book and transmitting it to the terminal, generation AI means for receiving user interaction data and generating an ending based on that data, an emotion engine for collecting, analyzing, and recognizing user emotion data, means for using a generation AI model to generate a custom experience based on the recognized emotion data and interaction data, and means for transmitting the generated ending data to the terminal. This makes it possible to generate a personalized ending based on the user's emotions and interactions, providing a deeper sense of immersion and enjoyment than ever before.

[0801] "User authentication information" refers to information required for a user to log in to a system, and typically refers to a username and password.

[0802] "Means of authentication" refers to a mechanism for verifying the identity of a user based on authentication information provided by the user.

[0803] "Picture book data request" refers to the act of a user requesting data of a picture book they want to read from a server.

[0804] "Fixed story data of a picture book" refers to predetermined story data of a picture book.

[0805] "Interaction Data" is data that describes how a user interacts with a system, including touch locations and options selected.

[0806] "Generative AI means" refers to artificial intelligence technology for generating new endings based on user interaction data.

[0807] An "emotion engine" is a system that detects and analyzes a user's emotions, typically using a camera and microphone to analyze facial expressions and voice.

[0808] "Emotion data" is data that indicates the user's emotional state, and is classified into categories such as joy, surprise, sadness, and anger.

[0809] "Custom experience" refers to an experience that is individually generated based on the user's emotional and interaction data.

[0810] "Ending data" refers to data about the ending of a story, created by a generative AI method.

[0811] The present invention is a digital picture book system that generates custom experiences and endings based on user interaction and emotional data. Detailed embodiments of the present invention will be described below.

[0812] overview

[0813] The present invention consists of the following means: user authentication, acquisition of picture book data, collection of interaction data, emotion recognition, ending generation using a generation AI means, and custom experience generation.

[0814] Components

[0815] 1. User authentication method

[0816] The user enters login information (e.g., username and password) into the terminal and sends the authentication information to the server.

[0817] The server verifies the user's identity based on the authentication information, and if authentication is successful, generates an authentication token and sends it to the terminal.

[0818] 2. How to obtain picture book data

[0819] The user selects the picture book they want to read on the terminal, and the terminal sends the request to the server.

[0820] The server retrieves the fixed story data of the relevant picture book from the database and transmits it to the terminal.

[0821] 3. Methods of collecting interaction data

[0822] As the user progresses through the story, touch location information is collected for specific scenes.

[0823] The terminal transmits this interaction data to the server.

[0824] 4. Emotion Engine

[0825] To analyze the user's facial expression, a facial image is acquired using the device's camera.

[0826] It also uses a microphone to collect the user's voice for voice analysis.

[0827] Based on the acquired data, the emotion engine recognizes the user's emotions and sends them to the server as emotion data.

[0828] 5. Generation AI means

[0829] Based on emotional and interaction data, generative AI methods generate new endings and custom experiences.

[0830] The server uses generative AI means to generate prompts based on the user's interaction data and emotion data, such as:

[0831] Example prompt: "Based on the user's emotion data: Surprise, interaction data: Product selection: Bag, Time: 2 PM, generate a custom shopping experience to further engage the user. For example, include features of the selected bag and special offers."

[0832] 6. Ending data transmission method

[0833] The server sends the generated ending data and custom experience data to the terminal.

[0834] The terminal analyzes the received data and displays it to the user.

[0835] Specific examples

[0836] For example, a user selects the picture book "Little Red Riding Hood" on a device and, as they read, touches the wolf in the scene where they encounter it. At this time, the device's camera captures the user's face, which is analyzed by the emotion engine and the user's emotion is recognized as "surprise." Based on the user's emotion data and interaction data, the server uses a generative AI method to generate an ending such as "Little Red Riding Hood is surprised by the wolf's true identity, but becomes friends with it." The generated ending data is sent to the device, allowing the user to enjoy the new ending.

[0837] As described above, the present invention makes it possible to provide a personalized story experience based on the user's emotions and interactions, thereby achieving a deeper sense of immersion and enjoyment.

[0838] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0839] Step 1:

[0840] The user enters login information (user name and password) into the terminal. The terminal sends this authentication information to the server. The server checks the received authentication information against a database, and if authentication is successful, generates an authentication token and sends it to the terminal. The input is the user's authentication information, and the output is the authentication token.

[0841] Step 2:

[0842] The user selects a picture book they want to read on their device. The device requests fixed story data for the picture book and sends this request to the server. The server retrieves the relevant picture book data from the database based on the received request and sends it to the device. The input is the picture book selection request, and the output is the fixed story data for the picture book.

[0843] Step 3:

[0844] As the user reads the story, they input touch position information for specific scenes. The device records this touch position information and sends it to the server. The input is the touch position information, and the output is the transmitted touch position information.

[0845] Step 4:

[0846] The device uses a camera to capture images of the user's face and simultaneously collects the user's voice through a microphone. This data is sent to an emotion engine to analyze the user's emotions. For example, the emotion "surprise" can be detected from facial expressions. The input is the user's face image and voice data, and the output is analyzed emotion data.

[0847] Step 5:

[0848] The server receives the touch location information obtained in step 3 and the emotional data analyzed in step 4. Based on this data, it uses generative AI means to generate a new ending or custom experience, using prompts such as the following:

[0849] Example prompt: "Based on the user's emotion data: Surprise, interaction data: Product selection: Bag, Time: 2 PM, generate a custom shopping experience to further engage the user. For example, include features of the selected bag and special offers."

[0850] The input is emotion data and interaction data, and the output is the generated ending data.

[0851] Step 6:

[0852] The server sends the ending data generated by the AI ​​generation means to the terminal. The terminal analyzes the received ending data and displays it to the user. The input is the generated ending data, and the output is the ending displayed to the user.

[0853] This allows users to enjoy new endings and provides a personalized story experience.

[0854] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0855] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0856] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0857] [Third embodiment]

[0858] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0859] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0860] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0861] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0862] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0863] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0864] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0865] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0866] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0867] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0868] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0869] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0870] Overview of the invention

[0871] The present invention relates to an interactive digital picture book system that generates different endings based on specific conditions. The system comprises the following means.

[0872] 1. User authentication methods

[0873] When a user logs in using a device such as a tablet, they enter authentication information (user name and password).

[0874] The terminal sends this authentication information to the server, which then checks it against a database to authenticate the user.

[0875] If authentication is successful, the server generates an authentication token and sends it to the terminal, allowing the user to access the system.

[0876] 2. How to obtain picture book data

[0877] The user selects the picture book they want to read on the device. For example, consider the case where they select "Little Red Riding Hood."

[0878] The terminal sends a picture book data request to the server.

[0879] The server receives the request and retrieves the fixed story data for "Little Red Riding Hood" from the database.

[0880] The server sends the acquired story data to the terminal, which analyzes it and displays it on the screen.

[0881] 3. Means of generating endings

[0882] The user reads the story and interacts with each scene, for example touching the wolf in the scene where Little Red Riding Hood encounters the wolf.

[0883] The device sends touch location information to the server, which includes the location where the touch occurred and the scene context.

[0884] The server uses a generative AI means to generate a new ending based on the touch information.

[0885] The server transmits the generated ending data to the terminal.

[0886] 4. Ending Display Method

[0887] The terminal analyzes the received ending data and displays it on the screen.

[0888] Users can enjoy new endings.

[0889] Specific examples

[0890] As a concrete example, consider the case of using the picture book "Little Red Riding Hood."

[0891] User Authentication

[0892] The user enters login information (e.g., username "tanaka" and password "pass1234") into the tablet. The device sends this information to the server, which then authenticates it. If authentication is successful, the server returns an authentication token to the device.

[0893] Picture book data acquisition

[0894] User "Tanaka" selects to read "Little Red Riding Hood." The device sends the request to the server, which retrieves the story data for "Little Red Riding Hood" and sends it to the device. The device displays the data, and the user begins reading the story.

[0895] Ending Generation

[0896] During the story, the user "Tanaka" touches the wolf's nose when Little Red Riding Hood encounters the wolf. The device sends this touch information to the server. The server uses a generation AI method to generate a new ending, "Little Red Riding Hood becomes friends with the wolf," based on the input touch information. The server then sends this ending data to the device.

[0897] Displaying the ending

[0898] The device analyzes the received ending data and displays the new ending to the user "Tanaka." The user and their child can then enjoy the new ending.

[0899] In this way, the present invention makes the reading experience more diverse and interactive than conventional picture books, and makes parent-child reading time more fulfilling.

[0900] The processing flow will be explained below.

[0901] Program processing

[0902] User Authentication and Login

[0903] Step 1:

[0904] The user starts the application and goes to the login screen. The user enters their username and password and presses the "Login" button.

[0905] Step 2:

[0906] The terminal sends the username and password entered by the user to the server as an HTTP request.

[0907] Step 3:

[0908] The server checks the received username and password against the database and performs authentication. If authentication is successful, it generates an authentication token and returns it to the terminal. If authentication fails, it returns an error message to the terminal.

[0909] Acquiring picture book data

[0910] Step 4:

[0911] The user selects the picture book they want to read within the application, for example, "Little Red Riding Hood."

[0912] Step 5:

[0913] The terminal sends a request including information about the selected picture book to the server.

[0914] Step 6:

[0915] The server receives the request and retrieves the fixed story data for the corresponding picture book "Little Red Riding Hood" from the database.

[0916] Step 7:

[0917] The server transmits the acquired fixed story data to the terminal.

[0918] Step 8:

[0919] The device analyzes the received story data and displays it on the screen, allowing the user to begin reading the story.

[0920] Ending generation request

[0921] Step 9:

[0922] The user interacts with the story by touching and clicking, for example, by touching the wolf in the scene where Little Red Riding Hood encounters the wolf.

[0923] Step 10:

[0924] The terminal sends an ending generation request including the user's touch position and scene information to the server.

[0925] Automatic ending generation

[0926] Step 11:

[0927] The server receives the ending generation request and passes the touch position and scene information to the generation AI means.

[0928] Step 12:

[0929] The AI ​​generator generates a new ending based on the touch position and scene information, for example, an ending in which Little Red Riding Hood becomes friends with the wolf.

[0930] Step 13:

[0931] The server transmits the generated ending data to the terminal.

[0932] Displaying the ending

[0933] Step 14:

[0934] The terminal analyzes the received ending data and displays it on the screen.

[0935] Step 15:

[0936] Users can enjoy the story to the end while seeing new endings.

[0937] By following these steps, a different ending is generated each time, making the reading experience fresh.

[0938] Example 1

[0939] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0940] Digital picture books have become popular in recent years and are attracting attention as a form of education and entertainment for children. However, existing digital picture books have a fixed storyline and lack interactive elements that incorporate user interaction, which can lead to a one-sided reading experience that can easily become boring. Therefore, the present invention aims to provide a system that makes the reading experience more enjoyable and interactive by generating different endings based on user interaction.

[0941] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0942] In this invention, the server includes means for receiving user authentication information and authenticating the user, means for receiving a request for picture book data and acquiring fixed story data for the picture book and transmitting it to the terminal, generation AI means for receiving user interaction data and generating an ending based on that data, means for transmitting the generated ending data to the terminal, and means for analyzing the ending data displayed on the terminal and displaying it on the screen. This allows the user to interact with the picture book as they read it, resulting in a variety of endings and an interactive and engaging story-telling experience.

[0943] "User credentials" are data entered by a user to grant access to a system, typically including a username and password.

[0944] "Means for authenticating" refers to the processes and functions for receiving a user's authentication information, checking it against a database, and verifying the user's access rights.

[0945] A "request for picture book data" is an operation performed when a user wants to read a digital picture book and requests the contents of that picture book from the server.

[0946] "Fixed story data" refers to a set of pre-defined story structure and content data for a particular picture book.

[0947] "User interaction data" is information about operations and actions performed by the user while reading a picture book, and includes data such as touch position and timing.

[0948] "Generative AI means" refers to algorithms or processes that use artificial intelligence to generate new outputs based on specific input data.

[0949] "Ending data" is data that describes the content of the final part of the story, and is generated based on the user's interactions.

[0950] "Means for transmitting to the terminal" refers to the function by which the server processes data and sends the data over the network to the device used by the user.

[0951] "Means for displaying on the screen" refers to the process of analyzing the received data, outputting it in an appropriate format to the user interface, and displaying it on the terminal screen.

[0952] "Interactive elements" refer to a mechanism in which users directly interact with the system through operations and actions, and the results are reflected in real time.

[0953] "Specific scene information" is data about a specific scene in a story, and includes the unique characteristics and context of that scene.

[0954] This invention relates to an interactive digital picture book system that generates different endings based on specific conditions. This system uses a generative AI model to generate various endings based on the user's interactions while reading the digital picture book, providing the user with a new reading experience.

[0955] The system mainly consists of hardware including a server, terminals, and authentication database, and software that runs the generative AI model.

[0956] User authentication methods

[0957] The user enters login information (username, password) using a device such as a tablet. The device sends this authentication information to the server. The server receives the authentication information and authenticates the user by checking it against a database. If authentication is successful, the server generates an authentication token and sends it to the device. This token allows the user to access the system.

[0958] How to obtain picture book data

[0959] The user selects the picture book they want to read on their device. For example, if the user selects "Little Red Riding Hood," the device sends a picture book data request to the server. The server retrieves the fixed story data for "Little Red Riding Hood" from the database and sends that data to the device. The device analyzes the retrieved story data and displays it on the screen.

[0960] Means of generating endings

[0961] As the user reads the story, they interact with each scene. For example, if the user touches the wolf's nose in the scene where Little Red Riding Hood encounters the wolf, the device sends this touch position information and the scene context to the server. The server uses a generative AI model to generate a new ending based on this touch information. Specifically, the following prompt is input into the generative AI model: "Generate a story in which Little Red Riding Hood touches the wolf's nose and, as a result, becomes friends with the wolf."

[0962] The generated ending data is transmitted from the server to the terminal.

[0963] Ending display method

[0964] The device analyzes the received ending data and displays it on the screen, allowing the user to enjoy new endings. The process of generating endings is done in real time, and is designed to be instantly updated based on the user's interactions.

[0965] Specific examples

[0966] As a concrete example, consider the case where user "Tanaka" enters login information into a tablet device, is authenticated, and then selects "Little Red Riding Hood." The device receives story data from the server, and the content is displayed on the screen. During the story, user "Tanaka" touches the wolf's nose, and that information is sent to the server. The server uses a generative AI model to generate a new ending in which "Little Red Riding Hood becomes friends with the wolf" and sends it to the device. The device displays the new ending on the screen, and the user can enjoy the new conclusion to the story.

[0967] In this way, the present invention provides a system that generates a variety of endings based on user interactions, making the storytelling experience more interactive and enjoyable.

[0968] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0969] Step 1:

[0970] User enters login information

[0971] The user enters their username and password into the tablet device, which then sends this authentication information to the server.

[0972] Input: Username, Password

[0973] Output: Authentication information sent to the server

[0974] Step 2:

[0975] The server verifies the authentication information

[0976] The server compares the received authentication information with the database and determines the authentication result. If authentication is successful, it generates an authentication token and sends it to the terminal.

[0977] Input: Credentials

[0978] Data processing: Verification of authentication information, generation of authentication tokens

[0979] Output: Authentication token sent to the device

[0980] Step 3:

[0981] User selects a picture book

[0982] The user selects the picture book they want to read on their device, for example, "Little Red Riding Hood," and the device sends the request to the server.

[0983] Input: Picture book selection request

[0984] Output: The request sent to the server

[0985] Step 4:

[0986] The server retrieves the picture book data

[0987] The server retrieves the fixed story data of the relevant picture book from the database and sends the retrieved data to the terminal.

[0988] Input: Picture book selection request

[0989] Data processing: Acquiring story data

[0990] Output: Story data sent to device

[0991] Step 5:

[0992] The device displays the picture book data

[0993] The device analyzes the received picture book data and displays it on the screen, allowing the user to begin reading the story.

[0994] Input: Story data

[0995] Data processing: Data analysis

[0996] Output: Picture book data displayed on the screen

[0997] Step 6:

[0998] The user interacts

[0999] When the user interacts with the story (e.g., touching the wolf's nose) in a specific scene, the device sends the touch location information and the scene context to the server.

[1000] Input: Interaction (touch information)

[1001] Output: Touch information sent to the server

[1002] Step 7:

[1003] The server generates a new ending

[1004] The server uses a generative AI model to generate a new ending based on the touch information it receives.

[1005] Example prompt: "Generate a story in which Little Red Riding Hood touches the wolf's nose and becomes friends with him."

[1006] Input: Touch information, prompt text

[1007] Data processing: Generative AI model generates endings

[1008] Output: Ending data sent to the terminal

[1009] Step 8:

[1010] The device displays a new ending.

[1011] The device analyzes the received ending data and displays it on the screen, allowing the user to enjoy the new ending.

[1012] Input: Ending data

[1013] Data processing: Data analysis

[1014] Output: New ending shown on screen

[1015] (Application example 1)

[1016] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1017] Conventional picture books have a fixed storyline, meaning that once you've read it, you end up reading the same content again and again. This causes problems such as a lack of freshness for both children and parents when reading, and a lack of interactive elements. Also, because they only allow for a predetermined ending, it's difficult to provide real-time interaction or variety in the storyline based on user choices.

[1018] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1019] In this invention, the server includes means for receiving user authentication information and authenticating the user, means for receiving a picture book data request and acquiring fixed story data for the picture book and transmitting it to the information terminal, generation AI means for receiving user interaction data and generating an ending based on that data, means for transmitting the generated ending data to the information terminal, and means for analyzing and displaying the ending data, thereby making it possible to generate and display new endings in real time.

[1020] "User authentication information" refers to information required when a user accesses a system, such as a username and password.

[1021] A "request for picture book data" refers to the act of requesting data for a picture book that the user wants to read, or the request information.

[1022] "Fixed story data" refers to data of a predetermined series of stories in a particular picture book.

[1023] An "information terminal" is an electronic device used by a user, such as a smartphone, tablet, or computer.

[1024] "Interaction data" refers to information about operations and inputs performed by the user while reading a picture book, and includes, for example, touch position information.

[1025] "Generative AI means" refers to artificial intelligence technology used to generate new endings based on received data.

[1026] "Ending data" is data regarding the ending of a story created by a generative AI means.

[1027] The "means for analyzing and displaying" is a means having the function of understanding the received ending data and displaying it on an information terminal.

[1028] The present invention relates to an interactive digital picture book system that generates different endings based on specific conditions. The following is a detailed description showing how to specifically implement the present invention.

[1029] The server implements the invention using the following means.

[1030] 1. User Authentication

[1031] When a user logs in using a device such as a tablet or smartphone, they enter authentication information (username, password). The device sends this authentication information to the server, which then authenticates the user by checking it against a database. If authentication is successful, the server generates an authentication token and sends it to the device, allowing the user to access the system. The software used is Flask to perform the authentication process.

[1032] 2. Acquiring picture book data

[1033] The user selects a picture book they want to read on their device. For example, consider the case where "Little Red Riding Hood" is selected. The device sends a picture book data request to the server. The server receives the request and retrieves the fixed story data for "Little Red Riding Hood" from the database. The server then sends the retrieved story data to the device, which parses it and displays it on the screen. This process involves database operations and sending and receiving data in JSON format.

[1034] 3. Ending Generation

[1035] The user reads the story and interacts with each scene. For example, in the scene where Little Red Riding Hood encounters the wolf, the user touches the wolf's nose. The device sends touch location information to the server, which includes the location of the touch and the scene context. The server uses a generative AI method to generate a new ending based on the touch information. The generative AI model used is OpenAI's GPT-4. This model generates prompts based on specific scene information and the user's interaction data, and generates a new ending.

[1036] 4. Ending display

[1037] The server sends the generated ending data to the device. The device analyzes the received ending data and displays it on the screen. The user can enjoy the new ending. The device uses JavaScript and HTML to analyze the data and display it on the user interface.

[1038] Specific examples

[1039] If the user selects "Little Red Riding Hood" and touches the wolf's ear in the scene where Little Red Riding Hood encounters the wolf, this touch information is sent to the server. The server generates the following prompt sentence and passes it to the generation AI.

[1040] Prompt Sentence Examples

[1041] Generate an ending for the following scene in the story of Little Red Riding Hood. Scene: Little Red Riding Hood encounters the wolf. Touch location: Wolf's ear.

[1042] Based on this prompt, the AI ​​generates a new ending, "Little Red Riding Hood becomes friends with the wolf," and the server sends it to the device. The device displays this ending on the screen, allowing the user to enjoy the conclusion of the new story.

[1043] In this way, the present invention can provide a more diverse and interactive reading experience than conventional picture books, making parent-child reading time more fulfilling.

[1044] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1045] Step 1:

[1046] The user enters login information (e.g., username and password) into the terminal.

[1047] Input: Username and Password

[1048] How it works: The device sends these authentication information to the server, which checks it against a database and authenticates the user. If authentication is successful, the server generates an authentication token and returns it to the device.

[1049] Output: Authentication token

[1050] Step 2:

[1051] The user selects the picture book they want to read on the device.

[1052] Input: Picture book selection information (e.g., "Little Red Riding Hood")

[1053] Operation: The device sends a picture book data request to the server. The server receives the request, retrieves the fixed story data for the corresponding picture book from the database, and sends it to the device. The device analyzes the received story data and displays it on the screen.

[1054] Output: Fixed story data

[1055] Step 3:

[1056] The user reads the story and interacts with it (e.g., by touch) during the story.

[1057] Input: Touch location and scene context

[1058] How it works: The device sends touch location information and scene context to the server, which uses a generative AI model to generate a prompt based on that information.

[1059] Output: prompt statement

[1060] Step 4:

[1061] The server generates a new ending using a generative AI model.

[1062] Input: prompt statement

[1063] How it works: A generative AI model (e.g., OpenAI's GPT-4) generates an ending based on the prompt. The server retrieves the generated ending data and sends it to the device.

[1064] Output: New ending data

[1065] Step 5:

[1066] The device will analyze the new ending data and display it on the screen.

[1067] Input: New ending data

[1068] Behavior: The device parses the received ending data and converts it into a format suitable for the user interface. After parsing, the device displays the new ending on the screen.

[1069] Output: New ending displayed on screen

[1070] This allows users to enjoy new endings generated in real time based on specific interactions.

[1071] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1072] Overview of the invention

[1073] The present invention relates to a digital picture book system that automatically generates an ending based on user interaction. Furthermore, the system is characterized by incorporating an emotion engine that recognizes the user's emotions and reflecting them in the generation of the ending. The system is composed of the following means:

[1074] 1. User authentication methods

[1075] When a user logs in using a device such as a tablet, they enter authentication information (user name and password).

[1076] The terminal sends this authentication information to the server, which then checks it against a database to authenticate the user.

[1077] If authentication is successful, the server generates an authentication token and sends it to the terminal, allowing the user to access the system.

[1078] 2. How to obtain picture book data

[1079] The user selects the picture book they want to read on the device, for example, "Little Red Riding Hood."

[1080] The terminal sends a picture book data request to the server.

[1081] The server receives the request and retrieves the fixed story data for the corresponding picture book "Little Red Riding Hood" from the database.

[1082] The server sends the acquired story data to the terminal, which analyzes it and displays it on the screen.

[1083] 3. Means of generating endings

[1084] The user reads the story and interacts with each scene, for example touching the wolf in the scene where Little Red Riding Hood encounters the wolf.

[1085] The device transmits touch position information and scene information to the server.

[1086] The server uses a generative AI method to generate a new ending based on the touch information and scene information.

[1087] The server transmits the generated ending data to the terminal.

[1088] 4. Emotional Engine Tools

[1089] The emotion engine detects the user's emotions, for example by using a camera to analyze the user's facial expressions and recognize emotions.

[1090] It can also use a microphone to collect the user's voice and recognize emotions through voice analysis.

[1091] The emotion engine sends the recognized emotion data to the server.

[1092] 5. Ending Display Method

[1093] The server also reflects the emotional data in the ending generation and generates a new ending.

[1094] The device analyzes the received ending data and displays it on the screen, allowing the user to enjoy the story to the end while viewing the new ending.

[1095] Specific examples

[1096] As a concrete example, consider the case of using the picture book "Little Red Riding Hood."

[1097] User Authentication

[1098] The user enters login information (e.g., username "tanaka" and password "pass1234") into the tablet. The device sends this information to the server, which then authenticates it. If authentication is successful, the server returns an authentication token to the device.

[1099] Picture book data acquisition

[1100] User "Tanaka" selects to read "Little Red Riding Hood." The device sends the request to the server, which retrieves the story data for "Little Red Riding Hood" and sends it to the device. The device displays the data, and the user begins reading the story.

[1101] Ending Generation

[1102] During the story, the user "Tanaka" touches the wolf's nose at the scene where Little Red Riding Hood encounters the wolf. The device sends this touch position information and scene information to the server.

[1103] Emotion Engine

[1104] The moment the user touches the screen, their facial expression and voice are collected via a camera and microphone and analyzed by the emotion engine, which then recognizes the user's emotion as "surprise."

[1105] Displaying the ending

[1106] The server uses a generation AI to generate a new ending, "Little Red Riding Hood is surprised by the wolf's true identity but becomes friends with him," based on touch information, scene information, and emotional data of "surprise." The generated ending data is sent to the device, which analyzes it and displays it on the screen. The user can enjoy the story to the end while watching the new ending.

[1107] The processing flow will be explained below.

[1108] Detailed process steps for implementing the invention

[1109] User Authentication and Login

[1110] Step 1:

[1111] The user starts up the tablet and goes to the login screen. The user enters their username and password and presses the "Login" button.

[1112] Step 2:

[1113] The terminal sends the username and password entered by the user to the server as an HTTP request.

[1114] Step 3:

[1115] The server checks the received username and password against the database and performs authentication. If authentication is successful, it generates an authentication token and returns it to the terminal. If authentication fails, it returns an error message to the terminal.

[1116] Acquiring picture book data

[1117] Step 4:

[1118] The user selects the picture book they want to read within the application, for example, "Little Red Riding Hood."

[1119] Step 5:

[1120] The terminal sends a request including information about the selected picture book to the server.

[1121] Step 6:

[1122] The server receives the request and retrieves the fixed story data for the corresponding picture book "Little Red Riding Hood" from the database.

[1123] Step 7:

[1124] The server transmits the acquired fixed story data to the terminal.

[1125] Step 8:

[1126] The device analyzes the received story data and displays it on the screen, allowing the user to begin reading the story.

[1127] Ending generation request

[1128] Step 9:

[1129] The user interacts with the story by touching and clicking, for example, by touching the wolf in the scene where Little Red Riding Hood encounters the wolf.

[1130] Step 10:

[1131] The terminal transmits an ending generation request including the user's touch position information and scene information to the server.

[1132] Emotion recognition by emotion engine

[1133] Step 11:

[1134] At the moment the user touches the device, it uses the camera to capture the user's facial expressions and collects the user's voice through the microphone.

[1135] Step 12:

[1136] The device sends the collected facial expression data and voice data to the emotion engine.

[1137] Step 13:

[1138] The emotion engine uses face recognition algorithms and voice analysis algorithms to analyze the user's emotions, for example, if the user shows the emotion "surprise", it will get the result.

[1139] Step 14:

[1140] The emotion engine sends the analysis results to the server.

[1141] Automatic ending generation

[1142] Step 15:

[1143] The server receives the emotion data along with the ending generation request, and passes the touch position, scene information, and emotion data to the generation AI means.

[1144] Step 16:

[1145] The generative AI method generates new endings based on this data. For example, based on the emotion data of "surprise," it generates an ending that includes a scene in which Little Red Riding Hood is surprised by the wolf's unexpected true identity.

[1146] Step 17:

[1147] The server transmits the generated ending data to the terminal.

[1148] Displaying the ending

[1149] Step 18:

[1150] The terminal analyzes the received ending data and displays it on the screen.

[1151] Step 19:

[1152] Users can enjoy the story to the end while seeing new endings.

[1153] This allows the invention to provide an interactive ending based on the user's emotions, resulting in a more personalized reading experience.

[1154] Example 2

[1155] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1156] Conventional digital picture book systems have the problem of being unable to provide dynamic endings that reflect the user's interactions and emotions. As a result, they are unable to generate personalized endings that correspond to the user's individual experience, resulting in a lack of immersion.

[1157] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1158] In this invention, the server includes means for receiving user authentication information and authenticating the user, means for receiving a request for picture book data and acquiring fixed story data for the picture book and transmitting it to the terminal, generation AI means for receiving user interaction data and generating an ending based on that data, emotion engine means for collecting the user's facial expressions and voice and recognizing their emotions, means for generating a new ending using the recognized emotion data, and means for transmitting the generated ending data to the terminal. This makes it possible to generate a personalized ending based on the user's interaction and emotion data, providing a more immersive story experience.

[1159] "User authentication information" refers to information provided by a user when logging in to a system, and includes a username and password.

[1160] An "authentication token" is temporary data sent from the server to the terminal when the user has been successfully authenticated, and indicates the user's permission to access the system.

[1161] A "picture book data request" is request data sent from a terminal to a server when a user intends to read a particular picture book.

[1162] "Fixed story data" is data that includes the basic storyline and content of a digital picture book, and is predefined.

[1163] "Interaction data" is data generated when a user operates a digital picture book, and includes touch positions and event information.

[1164] "Generative AI means" refers to artificial intelligence technology that dynamically generates content based on user interaction data.

[1165] An "emotion engine" is a combination of software and hardware that analyzes and recognizes emotions from a user's facial expressions and voice.

[1166] "Emotion data" is data obtained as a result of analysis by the emotion engine, and represents the user's emotional state.

[1167] A "new ending" is the conclusion of a digital storybook that is dynamically generated based on the user's interaction and emotion data.

[1168] A "terminal" is a hardware device that a user uses to read a digital picture book, typically a tablet or smartphone.

[1169] MODE FOR CARRYING OUT THE INVENTION

[1170] Overview of the invention

[1171] This invention relates to a digital picture book system with an ending that is automatically generated based on user interaction. Furthermore, it is characterized by incorporating an emotion engine that recognizes the user's emotions and reflects them in the generation of the ending. The detailed configuration and operation of the system are described below.

[1172] Hardware and software used

[1173] Device: The device on which a user reads a digital picture book. Examples include tablets and smartphones.

[1174] Server: A central system that processes and stores data, handles authentication, data transmission, generative AI models, etc.

[1175] Generative AI model: AI technology for generating new endings based on interaction and emotion data.

[1176] Emotion engine: Software that recognizes emotions by analyzing the user's facial expressions and voice.

[1177] Detailed System Description

[1178] 1. User Authentication

[1179] A user enters login information into a terminal. The terminal sends the username and password to the server. The server authenticates the user against a database and generates an authentication token, which is sent to the terminal, allowing the user access to the system.

[1180] 2. Acquiring picture book data

[1181] The user selects the picture book they want to read on their device. For example, they select "Little Red Riding Hood." The device then sends the request to the server, which receives the request, retrieves the corresponding picture book data from the database, and sends it. The device then analyzes the data and displays it.

[1182] 3. Collecting interactions

[1183] The user reads the picture book and interacts with it at a specific scene. For example, the user touches the wolf's nose in the scene where Little Red Riding Hood encounters the wolf. The device records this touch information (touch position and scene information) and sends it to the server.

[1184] 4. Collecting Emotional Data

[1185] When a user interacts with the device, the device's camera captures the user's facial expressions and the microphone collects their voice. The emotion engine analyzes this data and recognizes the user's emotions. The recognized emotion data is then sent to the server.

[1186] 5. Ending Generation

[1187] The server sends prompts to the generative AI model based on the interaction data and emotion data, and the generative AI model generates a new ending. An example of this prompt is, "The user is reading the story of Little Red Riding Hood. In the scene where Little Red Riding Hood encounters the wolf, she touches the wolf's nose. At this time, the user has a surprised expression. Based on this information, please generate a new ending for the story."

[1188] 6. Ending display

[1189] The server then sends the generated ending data to the device, which then analyzes it and displays it on the screen, allowing the user to enjoy the story to the end while viewing the new ending.

[1190] Specific examples

[1191] As a concrete example, the following uses the picture book "Little Red Riding Hood."

[1192] The user enters their login information on the tablet, and the server authenticates them. After successful authentication, the user selects "Little Red Riding Hood," and the server sends the corresponding data. If, during the story, the user touches the wolf's nose and its expression shows surprise, the server sends a prompt to the generative AI model, generating a new ending that reflects the emotion of surprise. Ultimately, the user can enjoy an ending in which Little Red Riding Hood becomes friends with the wolf, despite being surprised by its true identity.

[1193] In this way, the present invention can provide a more immersive narrative experience by providing a personalized ending that reflects the user's interactions and emotions.

[1194] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1195] Step 1:

[1196] User Authentication

[1197] The user enters login information (username and password) into the terminal. The terminal sends this login information to the server. The server compares it with a database and performs authentication. It receives the authentication information as input and outputs the authentication result and authentication token. If authentication is successful, the server generates an authentication token and sends it to the terminal. This allows the user to access the system. A specific example of how this works is when a user enters the username "tanaka" and password "pass1234" on the tablet login screen, and the server authenticates the information.

[1198] Step 2:

[1199] Acquiring picture book data

[1200] The user selects the picture book they want to read on their device. For example, they select "Little Red Riding Hood." The device then sends a request for that picture book data to the server. The server receives this request as input, retrieves the corresponding picture book data from the database, and sends it to the device. The retrieved picture book data can then be displayed on the device as output. A specific example of how this works is when the user "Tanaka" selects "Little Red Riding Hood," and the server retrieves and sends the story data.

[1201] Step 3:

[1202] Interaction Collection

[1203] As the user reads the picture book, they interact with the device at certain scenes. For example, in the scene where Little Red Riding Hood encounters the wolf, the user touches the wolf's nose. The device sends this touch information and scene information as input to the server. The server receives this and processes the interaction data. A specific example of this behavior is the scene where the user "Tanaka" touches the wolf's nose and the device sends that information to the server.

[1204] Step 4:

[1205] Collecting Emotional Data

[1206] At the moment a user interacts, the device's camera captures the user's facial expression and the microphone collects their voice. The emotion engine receives this data as input, analyzes it, and recognizes the user's emotion. This recognized emotion data is sent to the server and used as output. A specific example is when the user "Tanaka" shows a surprised expression the moment he touches the wolf, and the device analyzes this with the emotion engine to generate emotion data for "surprise."

[1207] Step 5:

[1208] Ending Generation

[1209] The server sends the received interaction data and emotion data to the generative AI model as a prompt. The generative AI model receives this prompt as input and generates a new ending. The generated ending data is output and sent from the server to the device. A specific example of operation is when the server sends a prompt to the generative AI model saying, "The user is reading the story of Little Red Riding Hood. In the scene where Little Red Riding Hood encounters the wolf, he touches the wolf's nose. At this time, the user has a surprised expression. Based on this information, please generate a new ending for the story.", and a new ending is generated.

[1210] Step 6:

[1211] Displaying the ending

[1212] The device receives the new ending data from the server, analyzes it, and displays it on the screen. The user can enjoy the story while watching this new ending. Specifically, the device displays the ending in which Little Red Riding Hood becomes friends with the wolf despite being surprised by its true identity, and the user watches it.

[1213] (Application example 2)

[1214] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1215] Conventional digital picture book systems have limited ability to generate endings based on user interactions, resulting in a uniform user experience. In particular, they do not take user emotions into consideration, making it difficult to provide a personalized experience for each user. This limits the immersiveness and enjoyment of the story.

[1216] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1217] In this invention, the server includes means for receiving user authentication information and authenticating the user, means for receiving a request for picture book data and acquiring fixed story data for the picture book and transmitting it to the terminal, generation AI means for receiving user interaction data and generating an ending based on that data, an emotion engine for collecting, analyzing, and recognizing user emotion data, means for using a generation AI model to generate a custom experience based on the recognized emotion data and interaction data, and means for transmitting the generated ending data to the terminal. This makes it possible to generate a personalized ending based on the user's emotions and interactions, providing a deeper sense of immersion and enjoyment than ever before.

[1218] "User authentication information" refers to information required for a user to log in to a system, and typically refers to a username and password.

[1219] "Means of authentication" refers to a mechanism for verifying the identity of a user based on authentication information provided by the user.

[1220] "Picture book data request" refers to the act of a user requesting data of a picture book they want to read from a server.

[1221] "Fixed story data of a picture book" refers to predetermined story data of a picture book.

[1222] "Interaction Data" is data that describes how a user interacts with a system, including touch locations and options selected.

[1223] "Generative AI means" refers to artificial intelligence technology for generating new endings based on user interaction data.

[1224] An "emotion engine" is a system that detects and analyzes a user's emotions, typically using a camera and microphone to analyze facial expressions and voice.

[1225] "Emotion data" is data that indicates the user's emotional state, and is classified into categories such as joy, surprise, sadness, and anger.

[1226] "Custom experience" refers to an experience that is individually generated based on the user's emotional and interaction data.

[1227] "Ending data" refers to data about the ending of a story, created by a generative AI method.

[1228] The present invention is a digital picture book system that generates custom experiences and endings based on user interaction and emotional data. Detailed embodiments of the present invention will be described below.

[1229] overview

[1230] The present invention consists of the following means: user authentication, acquisition of picture book data, collection of interaction data, emotion recognition, ending generation using a generation AI means, and custom experience generation.

[1231] Components

[1232] 1. User authentication method

[1233] The user enters login information (e.g., username and password) into the terminal and sends the authentication information to the server.

[1234] The server verifies the user's identity based on the authentication information, and if authentication is successful, generates an authentication token and sends it to the terminal.

[1235] 2. How to obtain picture book data

[1236] The user selects the picture book they want to read on the terminal, and the terminal sends the request to the server.

[1237] The server retrieves the fixed story data of the relevant picture book from the database and transmits it to the terminal.

[1238] 3. Methods of collecting interaction data

[1239] As the user progresses through the story, touch location information is collected for specific scenes.

[1240] The terminal transmits this interaction data to the server.

[1241] 4. Emotion Engine

[1242] To analyze the user's facial expression, a facial image is acquired using the device's camera.

[1243] It also uses a microphone to collect the user's voice for voice analysis.

[1244] Based on the acquired data, the emotion engine recognizes the user's emotions and sends them to the server as emotion data.

[1245] 5. Generation AI means

[1246] Based on emotional and interaction data, generative AI methods generate new endings and custom experiences.

[1247] The server uses generative AI means to generate prompts based on the user's interaction data and emotion data, such as:

[1248] Example prompt: "Based on the user's emotion data: Surprise, interaction data: Product selection: Bag, Time: 2 PM, generate a custom shopping experience to further engage the user. For example, include features of the selected bag and special offers."

[1249] 6. Ending data transmission method

[1250] The server sends the generated ending data and custom experience data to the terminal.

[1251] The terminal analyzes the received data and displays it to the user.

[1252] Specific examples

[1253] For example, a user selects the picture book "Little Red Riding Hood" on a device and, as they read, touches the wolf in the scene where they encounter it. At this time, the device's camera captures the user's face, which is analyzed by the emotion engine and the user's emotion is recognized as "surprise." Based on the user's emotion data and interaction data, the server uses a generative AI method to generate an ending such as "Little Red Riding Hood is surprised by the wolf's true identity, but becomes friends with it." The generated ending data is sent to the device, allowing the user to enjoy the new ending.

[1254] As described above, the present invention makes it possible to provide a personalized story experience based on the user's emotions and interactions, thereby achieving a deeper sense of immersion and enjoyment.

[1255] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1256] Step 1:

[1257] The user enters login information (user name and password) into the terminal. The terminal sends this authentication information to the server. The server checks the received authentication information against a database, and if authentication is successful, generates an authentication token and sends it to the terminal. The input is the user's authentication information, and the output is the authentication token.

[1258] Step 2:

[1259] The user selects a picture book they want to read on their device. The device requests fixed story data for the picture book and sends this request to the server. The server retrieves the relevant picture book data from the database based on the received request and sends it to the device. The input is the picture book selection request, and the output is the fixed story data for the picture book.

[1260] Step 3:

[1261] As the user reads the story, they input touch position information for specific scenes. The device records this touch position information and sends it to the server. The input is the touch position information, and the output is the transmitted touch position information.

[1262] Step 4:

[1263] The device uses a camera to capture images of the user's face and simultaneously collects the user's voice through a microphone. This data is sent to an emotion engine to analyze the user's emotions. For example, the emotion "surprise" can be detected from facial expressions. The input is the user's face image and voice data, and the output is analyzed emotion data.

[1264] Step 5:

[1265] The server receives the touch location information obtained in step 3 and the emotional data analyzed in step 4. Based on this data, it uses generative AI means to generate a new ending or custom experience, using prompts such as the following:

[1266] Example prompt: "Based on the user's emotion data: Surprise, interaction data: Product selection: Bag, Time: 2 PM, generate a custom shopping experience to further engage the user. For example, include features of the selected bag and special offers."

[1267] The input is emotion data and interaction data, and the output is the generated ending data.

[1268] Step 6:

[1269] The server sends the ending data generated by the AI ​​generation means to the terminal. The terminal analyzes the received ending data and displays it to the user. The input is the generated ending data, and the output is the ending displayed to the user.

[1270] This allows users to enjoy new endings and provides a personalized story experience.

[1271] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1272] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1273] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1274] [Fourth embodiment]

[1275] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1276] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1277] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1278] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1279] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1280] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1281] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1282] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1283] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1284] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1285] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1286] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1287] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1288] Overview of the invention

[1289] The present invention relates to an interactive digital picture book system that generates different endings based on specific conditions. The system comprises the following means.

[1290] 1. User authentication methods

[1291] When a user logs in using a device such as a tablet, they enter authentication information (user name and password).

[1292] The terminal sends this authentication information to the server, which then checks it against a database to authenticate the user.

[1293] If authentication is successful, the server generates an authentication token and sends it to the terminal, allowing the user to access the system.

[1294] 2. How to obtain picture book data

[1295] The user selects the picture book they want to read on the device. For example, consider the case where they select "Little Red Riding Hood."

[1296] The terminal sends a picture book data request to the server.

[1297] The server receives the request and retrieves the fixed story data for "Little Red Riding Hood" from the database.

[1298] The server sends the acquired story data to the terminal, which analyzes it and displays it on the screen.

[1299] 3. Means of generating endings

[1300] The user reads the story and interacts with each scene, for example touching the wolf in the scene where Little Red Riding Hood encounters the wolf.

[1301] The device sends touch location information to the server, which includes the location where the touch occurred and the scene context.

[1302] The server uses a generative AI means to generate a new ending based on the touch information.

[1303] The server transmits the generated ending data to the terminal.

[1304] 4. Ending Display Method

[1305] The terminal analyzes the received ending data and displays it on the screen.

[1306] Users can enjoy new endings.

[1307] Specific examples

[1308] As a concrete example, consider the case of using the picture book "Little Red Riding Hood."

[1309] User Authentication

[1310] The user enters login information (e.g., username "tanaka" and password "pass1234") into the tablet. The device sends this information to the server, which then authenticates it. If authentication is successful, the server returns an authentication token to the device.

[1311] Picture book data acquisition

[1312] User "Tanaka" selects to read "Little Red Riding Hood." The device sends the request to the server, which retrieves the story data for "Little Red Riding Hood" and sends it to the device. The device displays the data, and the user begins reading the story.

[1313] Ending Generation

[1314] During the story, the user "Tanaka" touches the wolf's nose when Little Red Riding Hood encounters the wolf. The device sends this touch information to the server. The server uses a generation AI method to generate a new ending, "Little Red Riding Hood becomes friends with the wolf," based on the input touch information. The server then sends this ending data to the device.

[1315] Displaying the ending

[1316] The device analyzes the received ending data and displays the new ending to the user "Tanaka." The user and their child can then enjoy the new ending.

[1317] In this way, the present invention makes the reading experience more diverse and interactive than conventional picture books, and makes parent-child reading time more fulfilling.

[1318] The processing flow will be explained below.

[1319] Program processing

[1320] User Authentication and Login

[1321] Step 1:

[1322] The user starts the application and goes to the login screen. The user enters their username and password and presses the "Login" button.

[1323] Step 2:

[1324] The terminal sends the username and password entered by the user to the server as an HTTP request.

[1325] Step 3:

[1326] The server checks the received username and password against the database and performs authentication. If authentication is successful, it generates an authentication token and returns it to the terminal. If authentication fails, it returns an error message to the terminal.

[1327] Acquiring picture book data

[1328] Step 4:

[1329] The user selects the picture book they want to read within the application, for example, "Little Red Riding Hood."

[1330] Step 5:

[1331] The terminal sends a request including information about the selected picture book to the server.

[1332] Step 6:

[1333] The server receives the request and retrieves the fixed story data for the corresponding picture book "Little Red Riding Hood" from the database.

[1334] Step 7:

[1335] The server transmits the acquired fixed story data to the terminal.

[1336] Step 8:

[1337] The device analyzes the received story data and displays it on the screen, allowing the user to begin reading the story.

[1338] Ending generation request

[1339] Step 9:

[1340] The user interacts with the story by touching and clicking, for example, by touching the wolf in the scene where Little Red Riding Hood encounters the wolf.

[1341] Step 10:

[1342] The terminal sends an ending generation request including the user's touch position and scene information to the server.

[1343] Automatic ending generation

[1344] Step 11:

[1345] The server receives the ending generation request and passes the touch position and scene information to the generation AI means.

[1346] Step 12:

[1347] The AI ​​generator generates a new ending based on the touch position and scene information, for example, an ending in which Little Red Riding Hood becomes friends with the wolf.

[1348] Step 13:

[1349] The server transmits the generated ending data to the terminal.

[1350] Displaying the ending

[1351] Step 14:

[1352] The terminal analyzes the received ending data and displays it on the screen.

[1353] Step 15:

[1354] Users can enjoy the story to the end while seeing new endings.

[1355] By following these steps, a different ending is generated each time, making the reading experience fresh.

[1356] Example 1

[1357] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1358] Digital picture books have become popular in recent years and are attracting attention as a form of education and entertainment for children. However, existing digital picture books have a fixed storyline and lack interactive elements that incorporate user interaction, which can lead to a one-sided reading experience that can easily become boring. Therefore, the present invention aims to provide a system that makes the reading experience more enjoyable and interactive by generating different endings based on user interaction.

[1359] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1360] In this invention, the server includes means for receiving user authentication information and authenticating the user, means for receiving a request for picture book data and acquiring fixed story data for the picture book and transmitting it to the terminal, generation AI means for receiving user interaction data and generating an ending based on that data, means for transmitting the generated ending data to the terminal, and means for analyzing the ending data displayed on the terminal and displaying it on the screen. This allows the user to interact with the picture book as they read it, resulting in a variety of endings and an interactive and engaging story-telling experience.

[1361] "User credentials" are data entered by a user to grant access to a system, typically including a username and password.

[1362] "Means for authenticating" refers to the processes and functions for receiving a user's authentication information, checking it against a database, and verifying the user's access rights.

[1363] A "request for picture book data" is an operation performed when a user wants to read a digital picture book and requests the contents of that picture book from the server.

[1364] "Fixed story data" refers to a set of pre-defined story structure and content data for a particular picture book.

[1365] "User interaction data" is information about operations and actions performed by the user while reading a picture book, and includes data such as touch position and timing.

[1366] "Generative AI means" refers to algorithms or processes that use artificial intelligence to generate new outputs based on specific input data.

[1367] "Ending data" is data that describes the content of the final part of the story, and is generated based on the user's interactions.

[1368] "Means for transmitting to the terminal" refers to the function by which the server processes data and sends the data over the network to the device used by the user.

[1369] "Means for displaying on the screen" refers to the process of analyzing the received data, outputting it in an appropriate format to the user interface, and displaying it on the terminal screen.

[1370] "Interactive elements" refer to a mechanism in which users directly interact with the system through operations and actions, and the results are reflected in real time.

[1371] "Specific scene information" is data about a specific scene in a story, and includes the unique characteristics and context of that scene.

[1372] This invention relates to an interactive digital picture book system that generates different endings based on specific conditions. This system uses a generative AI model to generate various endings based on the user's interactions while reading the digital picture book, providing the user with a new reading experience.

[1373] The system mainly consists of hardware including a server, terminals, and authentication database, and software that runs the generative AI model.

[1374] User authentication methods

[1375] The user enters login information (username, password) using a device such as a tablet. The device sends this authentication information to the server. The server receives the authentication information and authenticates the user by checking it against a database. If authentication is successful, the server generates an authentication token and sends it to the device. This token allows the user to access the system.

[1376] How to obtain picture book data

[1377] The user selects the picture book they want to read on their device. For example, if the user selects "Little Red Riding Hood," the device sends a picture book data request to the server. The server retrieves the fixed story data for "Little Red Riding Hood" from the database and sends that data to the device. The device analyzes the retrieved story data and displays it on the screen.

[1378] Means of generating endings

[1379] As the user reads the story, they interact with each scene. For example, if the user touches the wolf's nose in the scene where Little Red Riding Hood encounters the wolf, the device sends this touch position information and the scene context to the server. The server uses a generative AI model to generate a new ending based on this touch information. Specifically, the following prompt is input into the generative AI model: "Generate a story in which Little Red Riding Hood touches the wolf's nose and, as a result, becomes friends with the wolf."

[1380] The generated ending data is transmitted from the server to the terminal.

[1381] Ending display method

[1382] The device analyzes the received ending data and displays it on the screen, allowing the user to enjoy new endings. The process of generating endings is done in real time, and is designed to be instantly updated based on the user's interactions.

[1383] Specific examples

[1384] As a concrete example, consider the case where user "Tanaka" enters login information into a tablet device, is authenticated, and then selects "Little Red Riding Hood." The device receives story data from the server, and the content is displayed on the screen. During the story, user "Tanaka" touches the wolf's nose, and that information is sent to the server. The server uses a generative AI model to generate a new ending in which "Little Red Riding Hood becomes friends with the wolf" and sends it to the device. The device displays the new ending on the screen, and the user can enjoy the new conclusion to the story.

[1385] In this way, the present invention provides a system that generates a variety of endings based on user interactions, making the storytelling experience more interactive and enjoyable.

[1386] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1387] Step 1:

[1388] User enters login information

[1389] The user enters their username and password into the tablet device, which then sends this authentication information to the server.

[1390] Input: Username, Password

[1391] Output: Authentication information sent to the server

[1392] Step 2:

[1393] The server verifies the authentication information

[1394] The server compares the received authentication information with the database and determines the authentication result. If authentication is successful, it generates an authentication token and sends it to the terminal.

[1395] Input: Credentials

[1396] Data processing: Verification of authentication information, generation of authentication tokens

[1397] Output: Authentication token sent to the device

[1398] Step 3:

[1399] User selects a picture book

[1400] The user selects the picture book they want to read on their device, for example, "Little Red Riding Hood," and the device sends the request to the server.

[1401] Input: Picture book selection request

[1402] Output: The request sent to the server

[1403] Step 4:

[1404] The server retrieves the picture book data

[1405] The server retrieves the fixed story data of the relevant picture book from the database and sends the retrieved data to the terminal.

[1406] Input: Picture book selection request

[1407] Data processing: Acquiring story data

[1408] Output: Story data sent to device

[1409] Step 5:

[1410] The device displays the picture book data

[1411] The device analyzes the received picture book data and displays it on the screen, allowing the user to begin reading the story.

[1412] Input: Story data

[1413] Data processing: Data analysis

[1414] Output: Picture book data displayed on the screen

[1415] Step 6:

[1416] The user interacts

[1417] When the user interacts with the story (e.g., touching the wolf's nose) in a specific scene, the device sends the touch location information and the scene context to the server.

[1418] Input: Interaction (touch information)

[1419] Output: Touch information sent to the server

[1420] Step 7:

[1421] The server generates a new ending

[1422] The server uses a generative AI model to generate a new ending based on the touch information it receives.

[1423] Example prompt: "Generate a story in which Little Red Riding Hood touches the wolf's nose and becomes friends with him."

[1424] Input: Touch information, prompt text

[1425] Data processing: Generative AI model generates endings

[1426] Output: Ending data sent to the terminal

[1427] Step 8:

[1428] The device displays a new ending.

[1429] The device analyzes the received ending data and displays it on the screen, allowing the user to enjoy the new ending.

[1430] Input: Ending data

[1431] Data processing: Data analysis

[1432] Output: New ending shown on screen

[1433] (Application example 1)

[1434] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1435] Conventional picture books have a fixed storyline, meaning that once you've read it, you end up reading the same content again and again. This causes problems such as a lack of freshness for both children and parents when reading, and a lack of interactive elements. Also, because they only allow for a predetermined ending, it's difficult to provide real-time interaction or variety in the storyline based on user choices.

[1436] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1437] In this invention, the server includes means for receiving user authentication information and authenticating the user, means for receiving a picture book data request and acquiring fixed story data for the picture book and transmitting it to the information terminal, generation AI means for receiving user interaction data and generating an ending based on that data, means for transmitting the generated ending data to the information terminal, and means for analyzing and displaying the ending data, thereby making it possible to generate and display new endings in real time.

[1438] "User authentication information" refers to information required when a user accesses a system, such as a username and password.

[1439] A "request for picture book data" refers to the act of requesting data for a picture book that the user wants to read, or the request information.

[1440] "Fixed story data" refers to data of a predetermined series of stories in a particular picture book.

[1441] An "information terminal" is an electronic device used by a user, such as a smartphone, tablet, or computer.

[1442] "Interaction data" refers to information about operations and inputs performed by the user while reading a picture book, and includes, for example, touch position information.

[1443] "Generative AI means" refers to artificial intelligence technology used to generate new endings based on received data.

[1444] "Ending data" is data regarding the ending of a story created by a generative AI means.

[1445] The "means for analyzing and displaying" is a means having the function of understanding the received ending data and displaying it on an information terminal.

[1446] The present invention relates to an interactive digital picture book system that generates different endings based on specific conditions. The following is a detailed description showing how to specifically implement the present invention.

[1447] The server implements the invention using the following means.

[1448] 1. User Authentication

[1449] When a user logs in using a device such as a tablet or smartphone, they enter authentication information (username, password). The device sends this authentication information to the server, which then authenticates the user by checking it against a database. If authentication is successful, the server generates an authentication token and sends it to the device, allowing the user to access the system. The software used is Flask to perform the authentication process.

[1450] 2. Acquiring picture book data

[1451] The user selects a picture book they want to read on their device. For example, consider the case where "Little Red Riding Hood" is selected. The device sends a picture book data request to the server. The server receives the request and retrieves the fixed story data for "Little Red Riding Hood" from the database. The server then sends the retrieved story data to the device, which parses it and displays it on the screen. This process involves database operations and sending and receiving data in JSON format.

[1452] 3. Ending Generation

[1453] The user reads the story and interacts with each scene. For example, in the scene where Little Red Riding Hood encounters the wolf, the user touches the wolf's nose. The device sends touch location information to the server, which includes the location of the touch and the scene context. The server uses a generative AI method to generate a new ending based on the touch information. The generative AI model used is OpenAI's GPT-4. This model generates prompts based on specific scene information and the user's interaction data, and generates a new ending.

[1454] 4. Ending display

[1455] The server sends the generated ending data to the device. The device analyzes the received ending data and displays it on the screen. The user can enjoy the new ending. The device uses JavaScript and HTML to analyze the data and display it on the user interface.

[1456] Specific examples

[1457] If the user selects "Little Red Riding Hood" and touches the wolf's ear in the scene where Little Red Riding Hood encounters the wolf, this touch information is sent to the server. The server generates the following prompt sentence and passes it to the generation AI.

[1458] Prompt Sentence Examples

[1459] Generate an ending for the following scene in the story of Little Red Riding Hood. Scene: Little Red Riding Hood encounters the wolf. Touch location: Wolf's ear.

[1460] Based on this prompt, the AI ​​generates a new ending, "Little Red Riding Hood becomes friends with the wolf," and the server sends it to the device. The device displays this ending on the screen, allowing the user to enjoy the conclusion of the new story.

[1461] In this way, the present invention can provide a more diverse and interactive reading experience than conventional picture books, making parent-child reading time more fulfilling.

[1462] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1463] Step 1:

[1464] The user enters login information (e.g., username and password) into the terminal.

[1465] Input: Username and Password

[1466] How it works: The device sends these authentication information to the server, which checks it against a database and authenticates the user. If authentication is successful, the server generates an authentication token and returns it to the device.

[1467] Output: Authentication token

[1468] Step 2:

[1469] The user selects the picture book they want to read on the device.

[1470] Input: Picture book selection information (e.g., "Little Red Riding Hood")

[1471] Operation: The device sends a picture book data request to the server. The server receives the request, retrieves the fixed story data for the corresponding picture book from the database, and sends it to the device. The device analyzes the received story data and displays it on the screen.

[1472] Output: Fixed story data

[1473] Step 3:

[1474] The user reads the story and interacts with it (e.g., by touch) during the story.

[1475] Input: Touch location and scene context

[1476] How it works: The device sends touch location information and scene context to the server, which uses a generative AI model to generate a prompt based on that information.

[1477] Output: prompt statement

[1478] Step 4:

[1479] The server generates a new ending using a generative AI model.

[1480] Input: prompt statement

[1481] How it works: A generative AI model (e.g., OpenAI's GPT-4) generates an ending based on the prompt. The server retrieves the generated ending data and sends it to the device.

[1482] Output: New ending data

[1483] Step 5:

[1484] The device will analyze the new ending data and display it on the screen.

[1485] Input: New ending data

[1486] Behavior: The device parses the received ending data and converts it into a format suitable for the user interface. After parsing, the device displays the new ending on the screen.

[1487] Output: New ending displayed on screen

[1488] This allows users to enjoy new endings generated in real time based on specific interactions.

[1489] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1490] Overview of the invention

[1491] The present invention relates to a digital picture book system that automatically generates an ending based on user interaction. Furthermore, the system is characterized by incorporating an emotion engine that recognizes the user's emotions and reflecting them in the generation of the ending. The system is composed of the following means:

[1492] 1. User authentication methods

[1493] When a user logs in using a device such as a tablet, they enter authentication information (user name and password).

[1494] The terminal sends this authentication information to the server, which then checks it against a database to authenticate the user.

[1495] If authentication is successful, the server generates an authentication token and sends it to the terminal, allowing the user to access the system.

[1496] 2. How to obtain picture book data

[1497] The user selects the picture book they want to read on the device, for example, "Little Red Riding Hood."

[1498] The terminal sends a picture book data request to the server.

[1499] The server receives the request and retrieves the fixed story data for the corresponding picture book "Little Red Riding Hood" from the database.

[1500] The server sends the acquired story data to the terminal, which analyzes it and displays it on the screen.

[1501] 3. Means of generating endings

[1502] The user reads the story and interacts with each scene, for example touching the wolf in the scene where Little Red Riding Hood encounters the wolf.

[1503] The device transmits touch position information and scene information to the server.

[1504] The server uses a generative AI method to generate a new ending based on the touch information and scene information.

[1505] The server transmits the generated ending data to the terminal.

[1506] 4. Emotional Engine Tools

[1507] The emotion engine detects the user's emotions, for example by using a camera to analyze the user's facial expressions and recognize emotions.

[1508] It can also use a microphone to collect the user's voice and recognize emotions through voice analysis.

[1509] The emotion engine sends the recognized emotion data to the server.

[1510] 5. Ending Display Method

[1511] The server also reflects the emotional data in the ending generation and generates a new ending.

[1512] The device analyzes the received ending data and displays it on the screen, allowing the user to enjoy the story to the end while viewing the new ending.

[1513] Specific examples

[1514] As a concrete example, consider the case of using the picture book "Little Red Riding Hood."

[1515] User Authentication

[1516] The user enters login information (e.g., username "tanaka" and password "pass1234") into the tablet. The device sends this information to the server, which then authenticates it. If authentication is successful, the server returns an authentication token to the device.

[1517] Picture book data acquisition

[1518] User "Tanaka" selects to read "Little Red Riding Hood." The device sends the request to the server, which retrieves the story data for "Little Red Riding Hood" and sends it to the device. The device displays the data, and the user begins reading the story.

[1519] Ending Generation

[1520] During the story, the user "Tanaka" touches the wolf's nose at the scene where Little Red Riding Hood encounters the wolf. The device sends this touch position information and scene information to the server.

[1521] Emotion Engine

[1522] The moment the user touches the screen, their facial expression and voice are collected via a camera and microphone and analyzed by the emotion engine, which then recognizes the user's emotion as "surprise."

[1523] Displaying the ending

[1524] The server uses a generation AI to generate a new ending, "Little Red Riding Hood is surprised by the wolf's true identity but becomes friends with him," based on touch information, scene information, and emotional data of "surprise." The generated ending data is sent to the device, which analyzes it and displays it on the screen. The user can enjoy the story to the end while watching the new ending.

[1525] The processing flow will be explained below.

[1526] Detailed process steps for implementing the invention

[1527] User Authentication and Login

[1528] Step 1:

[1529] The user starts up the tablet and goes to the login screen. The user enters their username and password and presses the "Login" button.

[1530] Step 2:

[1531] The terminal sends the username and password entered by the user to the server as an HTTP request.

[1532] Step 3:

[1533] The server checks the received username and password against the database and performs authentication. If authentication is successful, it generates an authentication token and returns it to the terminal. If authentication fails, it returns an error message to the terminal.

[1534] Acquiring picture book data

[1535] Step 4:

[1536] The user selects the picture book they want to read within the application, for example, "Little Red Riding Hood."

[1537] Step 5:

[1538] The terminal sends a request including information about the selected picture book to the server.

[1539] Step 6:

[1540] The server receives the request and retrieves the fixed story data for the corresponding picture book "Little Red Riding Hood" from the database.

[1541] Step 7:

[1542] The server transmits the acquired fixed story data to the terminal.

[1543] Step 8:

[1544] The device analyzes the received story data and displays it on the screen, allowing the user to begin reading the story.

[1545] Ending generation request

[1546] Step 9:

[1547] The user interacts with the story by touching and clicking, for example, by touching the wolf in the scene where Little Red Riding Hood encounters the wolf.

[1548] Step 10:

[1549] The terminal transmits an ending generation request including the user's touch position information and scene information to the server.

[1550] Emotion recognition by emotion engine

[1551] Step 11:

[1552] At the moment the user touches the device, it uses the camera to capture the user's facial expressions and collects the user's voice through the microphone.

[1553] Step 12:

[1554] The device sends the collected facial expression data and voice data to the emotion engine.

[1555] Step 13:

[1556] The emotion engine uses face recognition algorithms and voice analysis algorithms to analyze the user's emotions, for example, if the user shows the emotion "surprise", it will get the result.

[1557] Step 14:

[1558] The emotion engine sends the analysis results to the server.

[1559] Automatic ending generation

[1560] Step 15:

[1561] The server receives the emotion data along with the ending generation request, and passes the touch position, scene information, and emotion data to the generation AI means.

[1562] Step 16:

[1563] The generative AI method generates new endings based on this data. For example, based on the emotion data of "surprise," it generates an ending that includes a scene in which Little Red Riding Hood is surprised by the wolf's unexpected true identity.

[1564] Step 17:

[1565] The server transmits the generated ending data to the terminal.

[1566] Displaying the ending

[1567] Step 18:

[1568] The terminal analyzes the received ending data and displays it on the screen.

[1569] Step 19:

[1570] Users can enjoy the story to the end while seeing new endings.

[1571] This allows the invention to provide an interactive ending based on the user's emotions, resulting in a more personalized reading experience.

[1572] Example 2

[1573] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1574] Conventional digital picture book systems have the problem of being unable to provide dynamic endings that reflect the user's interactions and emotions. As a result, they are unable to generate personalized endings that correspond to the user's individual experience, resulting in a lack of immersion.

[1575] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1576] In this invention, the server includes means for receiving user authentication information and authenticating the user, means for receiving a request for picture book data and acquiring fixed story data for the picture book and transmitting it to the terminal, generation AI means for receiving user interaction data and generating an ending based on that data, emotion engine means for collecting the user's facial expressions and voice and recognizing their emotions, means for generating a new ending using the recognized emotion data, and means for transmitting the generated ending data to the terminal. This makes it possible to generate a personalized ending based on the user's interaction and emotion data, providing a more immersive story experience.

[1577] "User authentication information" refers to information provided by a user when logging in to a system, and includes a username and password.

[1578] An "authentication token" is temporary data sent from the server to the terminal when the user has been successfully authenticated, and indicates the user's permission to access the system.

[1579] A "picture book data request" is request data sent from a terminal to a server when a user intends to read a particular picture book.

[1580] "Fixed story data" is data that includes the basic storyline and content of a digital picture book, and is predefined.

[1581] "Interaction data" is data generated when a user operates a digital picture book, and includes touch positions and event information.

[1582] "Generative AI means" refers to artificial intelligence technology that dynamically generates content based on user interaction data.

[1583] An "emotion engine" is a combination of software and hardware that analyzes and recognizes emotions from a user's facial expressions and voice.

[1584] "Emotion data" is data obtained as a result of analysis by the emotion engine, and represents the user's emotional state.

[1585] A "new ending" is the conclusion of a digital storybook that is dynamically generated based on the user's interaction and emotion data.

[1586] A "terminal" is a hardware device that a user uses to read a digital picture book, typically a tablet or smartphone.

[1587] MODE FOR CARRYING OUT THE INVENTION

[1588] Overview of the invention

[1589] This invention relates to a digital picture book system with an ending that is automatically generated based on user interaction. Furthermore, it is characterized by incorporating an emotion engine that recognizes the user's emotions and reflects them in the generation of the ending. The detailed configuration and operation of the system are described below.

[1590] Hardware and software used

[1591] Device: The device on which a user reads a digital picture book. Examples include tablets and smartphones.

[1592] Server: A central system that processes and stores data, handles authentication, data transmission, generative AI models, etc.

[1593] Generative AI model: AI technology for generating new endings based on interaction and emotion data.

[1594] Emotion engine: Software that recognizes emotions by analyzing the user's facial expressions and voice.

[1595] Detailed System Description

[1596] 1. User Authentication

[1597] A user enters login information into a terminal. The terminal sends the username and password to the server. The server authenticates the user against a database and generates an authentication token, which is sent to the terminal, allowing the user access to the system.

[1598] 2. Acquiring picture book data

[1599] The user selects the picture book they want to read on their device. For example, they select "Little Red Riding Hood." The device then sends the request to the server, which receives the request, retrieves the corresponding picture book data from the database, and sends it. The device then analyzes the data and displays it.

[1600] 3. Collecting interactions

[1601] The user reads the picture book and interacts with it at a specific scene. For example, the user touches the wolf's nose in the scene where Little Red Riding Hood encounters the wolf. The device records this touch information (touch position and scene information) and sends it to the server.

[1602] 4. Collecting Emotional Data

[1603] When a user interacts with the device, the device's camera captures the user's facial expressions and the microphone collects their voice. The emotion engine analyzes this data and recognizes the user's emotions. The recognized emotion data is then sent to the server.

[1604] 5. Ending Generation

[1605] The server sends prompts to the generative AI model based on the interaction data and emotion data, and the generative AI model generates a new ending. An example of this prompt is, "The user is reading the story of Little Red Riding Hood. In the scene where Little Red Riding Hood encounters the wolf, she touches the wolf's nose. At this time, the user has a surprised expression. Based on this information, please generate a new ending for the story."

[1606] 6. Ending display

[1607] The server then sends the generated ending data to the device, which then analyzes it and displays it on the screen, allowing the user to enjoy the story to the end while viewing the new ending.

[1608] Specific examples

[1609] As a concrete example, the following uses the picture book "Little Red Riding Hood."

[1610] The user enters their login information on the tablet, and the server authenticates them. After successful authentication, the user selects "Little Red Riding Hood," and the server sends the corresponding data. If, during the story, the user touches the wolf's nose and its expression shows surprise, the server sends a prompt to the generative AI model, generating a new ending that reflects the emotion of surprise. Ultimately, the user can enjoy an ending in which Little Red Riding Hood becomes friends with the wolf, despite being surprised by its true identity.

[1611] In this way, the present invention can provide a more immersive narrative experience by providing a personalized ending that reflects the user's interactions and emotions.

[1612] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1613] Step 1:

[1614] User Authentication

[1615] The user enters login information (username and password) into the terminal. The terminal sends this login information to the server. The server compares it with a database and performs authentication. It receives the authentication information as input and outputs the authentication result and authentication token. If authentication is successful, the server generates an authentication token and sends it to the terminal. This allows the user to access the system. A specific example of how this works is when a user enters the username "tanaka" and password "pass1234" on the tablet login screen, and the server authenticates the information.

[1616] Step 2:

[1617] Acquiring picture book data

[1618] The user selects the picture book they want to read on their device. For example, they select "Little Red Riding Hood." The device then sends a request for that picture book data to the server. The server receives this request as input, retrieves the corresponding picture book data from the database, and sends it to the device. The retrieved picture book data can then be displayed on the device as output. A specific example of how this works is when the user "Tanaka" selects "Little Red Riding Hood," and the server retrieves and sends the story data.

[1619] Step 3:

[1620] Interaction Collection

[1621] As the user reads the picture book, they interact with the device at certain scenes. For example, in the scene where Little Red Riding Hood encounters the wolf, the user touches the wolf's nose. The device sends this touch information and scene information as input to the server. The server receives this and processes the interaction data. A specific example of this behavior is the scene where the user "Tanaka" touches the wolf's nose and the device sends that information to the server.

[1622] Step 4:

[1623] Collecting Emotional Data

[1624] At the moment a user interacts, the device's camera captures the user's facial expression and the microphone collects their voice. The emotion engine receives this data as input, analyzes it, and recognizes the user's emotion. This recognized emotion data is sent to the server and used as output. A specific example is when the user "Tanaka" shows a surprised expression the moment he touches the wolf, and the device analyzes this with the emotion engine to generate emotion data for "surprise."

[1625] Step 5:

[1626] Ending Generation

[1627] The server sends the received interaction data and emotion data to the generative AI model as a prompt. The generative AI model receives this prompt as input and generates a new ending. The generated ending data is output and sent from the server to the device. A specific example of operation is when the server sends a prompt to the generative AI model saying, "The user is reading the story of Little Red Riding Hood. In the scene where Little Red Riding Hood encounters the wolf, he touches the wolf's nose. At this time, the user has a surprised expression. Based on this information, please generate a new ending for the story.", and a new ending is generated.

[1628] Step 6:

[1629] Displaying the ending

[1630] The device receives the new ending data from the server, analyzes it, and displays it on the screen. The user can enjoy the story while watching this new ending. Specifically, the device displays the ending in which Little Red Riding Hood becomes friends with the wolf despite being surprised by its true identity, and the user watches it.

[1631] (Application example 2)

[1632] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1633] Conventional digital picture book systems have limited ability to generate endings based on user interactions, resulting in a uniform user experience. In particular, they do not take user emotions into consideration, making it difficult to provide a personalized experience for each user. This limits the immersiveness and enjoyment of the story.

[1634] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1635] In this invention, the server includes means for receiving user authentication information and authenticating the user, means for receiving a request for picture book data and acquiring fixed story data for the picture book and transmitting it to the terminal, generation AI means for receiving user interaction data and generating an ending based on that data, an emotion engine for collecting, analyzing, and recognizing user emotion data, means for using a generation AI model to generate a custom experience based on the recognized emotion data and interaction data, and means for transmitting the generated ending data to the terminal. This makes it possible to generate a personalized ending based on the user's emotions and interactions, providing a deeper sense of immersion and enjoyment than ever before.

[1636] "User authentication information" refers to information required for a user to log in to a system, and typically refers to a username and password.

[1637] "Means of authentication" refers to a mechanism for verifying the identity of a user based on authentication information provided by the user.

[1638] "Picture book data request" refers to the act of a user requesting data of a picture book they want to read from a server.

[1639] "Fixed story data of a picture book" refers to predetermined story data of a picture book.

[1640] "Interaction Data" is data that describes how a user interacts with a system, including touch locations and options selected.

[1641] "Generative AI means" refers to artificial intelligence technology for generating new endings based on user interaction data.

[1642] An "emotion engine" is a system that detects and analyzes a user's emotions, typically using a camera and microphone to analyze facial expressions and voice.

[1643] "Emotion data" is data that indicates the user's emotional state, and is classified into categories such as joy, surprise, sadness, and anger.

[1644] "Custom experience" refers to an experience that is individually generated based on the user's emotional and interaction data.

[1645] "Ending data" refers to data about the ending of a story, created by a generative AI method.

[1646] The present invention is a digital picture book system that generates custom experiences and endings based on user interaction and emotional data. Detailed embodiments of the present invention will be described below.

[1647] overview

[1648] The present invention consists of the following means: user authentication, acquisition of picture book data, collection of interaction data, emotion recognition, ending generation using a generation AI means, and custom experience generation.

[1649] Components

[1650] 1. User authentication method

[1651] The user enters login information (e.g., username and password) into the terminal and sends the authentication information to the server.

[1652] The server verifies the user's identity based on the authentication information, and if authentication is successful, generates an authentication token and sends it to the terminal.

[1653] 2. How to obtain picture book data

[1654] The user selects the picture book they want to read on the terminal, and the terminal sends the request to the server.

[1655] The server retrieves the fixed story data of the relevant picture book from the database and transmits it to the terminal.

[1656] 3. Methods of collecting interaction data

[1657] As the user progresses through the story, touch location information is collected for specific scenes.

[1658] The terminal transmits this interaction data to the server.

[1659] 4. Emotion Engine

[1660] To analyze the user's facial expression, a facial image is acquired using the device's camera.

[1661] It also uses a microphone to collect the user's voice for voice analysis.

[1662] Based on the acquired data, the emotion engine recognizes the user's emotions and sends them to the server as emotion data.

[1663] 5. Generation AI means

[1664] Based on emotional and interaction data, generative AI methods generate new endings and custom experiences.

[1665] The server uses generative AI means to generate prompts based on the user's interaction data and emotion data, such as:

[1666] Example prompt: "Based on the user's emotion data: Surprise, interaction data: Product selection: Bag, Time: 2 PM, generate a custom shopping experience to further engage the user. For example, include features of the selected bag and special offers."

[1667] 6. Ending data transmission method

[1668] The server sends the generated ending data and custom experience data to the terminal.

[1669] The terminal analyzes the received data and displays it to the user.

[1670] Specific examples

[1671] For example, a user selects the picture book "Little Red Riding Hood" on a device and, as they read, touches the wolf in the scene where they encounter it. At this time, the device's camera captures the user's face, which is analyzed by the emotion engine and the user's emotion is recognized as "surprise." Based on the user's emotion data and interaction data, the server uses a generative AI method to generate an ending such as "Little Red Riding Hood is surprised by the wolf's true identity, but becomes friends with it." The generated ending data is sent to the device, allowing the user to enjoy the new ending.

[1672] As described above, the present invention makes it possible to provide a personalized story experience based on the user's emotions and interactions, thereby achieving a deeper sense of immersion and enjoyment.

[1673] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1674] Step 1:

[1675] The user enters login information (user name and password) into the terminal. The terminal sends this authentication information to the server. The server checks the received authentication information against a database, and if authentication is successful, generates an authentication token and sends it to the terminal. The input is the user's authentication information, and the output is the authentication token.

[1676] Step 2:

[1677] The user selects a picture book they want to read on their device. The device requests fixed story data for the picture book and sends this request to the server. The server retrieves the relevant picture book data from the database based on the received request and sends it to the device. The input is the picture book selection request, and the output is the fixed story data for the picture book.

[1678] Step 3:

[1679] As the user reads the story, they input touch position information for specific scenes. The device records this touch position information and sends it to the server. The input is the touch position information, and the output is the transmitted touch position information.

[1680] Step 4:

[1681] The device uses a camera to capture images of the user's face and simultaneously collects the user's voice through a microphone. This data is sent to an emotion engine to analyze the user's emotions. For example, the emotion "surprise" can be detected from facial expressions. The input is the user's face image and voice data, and the output is analyzed emotion data.

[1682] Step 5:

[1683] The server receives the touch location information obtained in step 3 and the emotional data analyzed in step 4. Based on this data, it uses generative AI means to generate a new ending or custom experience, using prompts such as the following:

[1684] Example prompt: "Based on the user's emotion data: Surprise, interaction data: Product selection: Bag, Time: 2 PM, generate a custom shopping experience to further engage the user. For example, include features of the selected bag and special offers."

[1685] The input is emotion data and interaction data, and the output is the generated ending data.

[1686] Step 6:

[1687] The server sends the ending data generated by the AI ​​generation means to the terminal. The terminal analyzes the received ending data and displays it to the user. The input is the generated ending data, and the output is the ending displayed to the user.

[1688] This allows users to enjoy new endings and provides a personalized story experience.

[1689] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1690] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1691] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1692] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1693] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1694] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1695] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1696] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1697] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1698] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1699] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1700] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1701] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1702] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1703] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1704] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1705] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1706] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1707] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1708] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1709] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1710] The following is further disclosed regarding the above embodiment.

[1711] (Claim 1)

[1712] means for receiving user authentication information and authenticating the user;

[1713] means for receiving a request for picture book data, acquiring fixed story data of the picture book, and transmitting the acquired data to the terminal;

[1714] a generating AI means for receiving user interaction data and generating an ending based on the data;

[1715] means for transmitting the generated ending data to a terminal;

[1716] A system including:

[1717] (Claim 2)

[1718] The system according to claim 1 provides an interactive element that collects touch position information as the picture book is read and changes the ending based on that information.

[1719] (Claim 3)

[1720] 2. The system according to claim 1, wherein the system generates an ending corresponding to specific scene information.

[1721] "Example 1"

[1722] (Claim 1)

[1723] means for receiving user authentication information and authenticating the user;

[1724] means for receiving a request for picture book data, acquiring fixed story data of the picture book, and transmitting the acquired data to the terminal;

[1725] a generating AI means for receiving user interaction data and generating an ending based on the data;

[1726] means for transmitting the generated ending data to a terminal;

[1727] A means for analyzing the ending data displayed on the terminal and displaying it on the screen;

[1728] A system including:

[1729] (Claim 2)

[1730] The system according to claim 1 provides an interactive element that collects touch position information as the picture book is read and changes the ending based on that information.

[1731] (Claim 3)

[1732] 2. The system according to claim 1, wherein the system generates an ending corresponding to specific scene information.

[1733] "Application Example 1"

[1734] (Claim 1)

[1735] means for receiving user authentication information and authenticating the user;

[1736] means for receiving a request for picture book data, acquiring fixed story data of the picture book, and transmitting the acquired data to an information terminal;

[1737] a generating AI means for receiving user interaction data and generating an ending based on the data;

[1738] means for transmitting the generated ending data to an information terminal;

[1739] A means for analyzing and displaying ending data;

[1740] A system including:

[1741] (Claim 2)

[1742] The system according to claim 1 provides an interactive element that collects touch position information as the picture book is read and changes the ending based on that information.

[1743] (Claim 3)

[1744] The system of claim 1 uses a generative AI model to generate an ending that corresponds to specific scene information when generating an ending.

[1745] "Example 2: Combining Emotion Engines"

[1746] (Claim 1)

[1747] means for receiving user authentication information and authenticating the user;

[1748] means for receiving a request for picture book data, acquiring fixed story data of the picture book, and transmitting the acquired data to the terminal;

[1749] a generating AI means for receiving user interaction data and generating an ending based on the data;

[1750] an emotion engine means for collecting user's facial expressions and voice to recognize emotions;

[1751] a means for generating a new ending using the recognized emotion data; and

[1752] means for transmitting the generated ending data to a terminal;

[1753] A system including:

[1754] (Claim 2)

[1755] The system according to claim 1 provides an interactive element that collects touch position information as the picture book is read and changes the ending based on that information.

[1756] (Claim 3)

[1757] 2. The system according to claim 1, wherein the ending is generated based on specific scene information, user interaction data, and user emotion data.

[1758] "Application example 2 when combining emotion engines"

[1759] (Claim 1)

[1760] means for receiving user authentication information and authenticating the user;

[1761] means for receiving a request for picture book data, acquiring fixed story data of the picture book, and transmitting the acquired data to the terminal;

[1762] a generating AI means for receiving user interaction data and generating an ending based on the data;

[1763] An emotion engine that collects, analyzes, and recognizes user emotion data;

[1764] a means for using a generative AI model to generate custom experiences based on recognized emotion data and interaction data;

[1765] means for transmitting the generated ending data to a terminal;

[1766] A system including:

[1767] (Claim 2)

[1768] The system according to claim 1 provides an interactive element that collects touch position information as the picture book is read and changes the ending based on that information.

[1769] (Claim 3)

[1770] 2. The system of claim 1, wherein the system generates a new ending based on the recognized emotion data and specific scene information. [Explanation of symbols]

[1771] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving user authentication information and authenticating the user; means for receiving a request for picture book data, acquiring fixed story data of the picture book, and transmitting the acquired data to the terminal; a generating AI means for receiving user interaction data and generating an ending based on the data; means for transmitting the generated ending data to a terminal; A system including:

2. The system according to claim 1, wherein touch position information is collected during the reading of the picture book, and an interactive element is provided that changes the ending based on the collected touch position information.

3. 2. The system according to claim 1, wherein the ending is generated in correspondence with specific scene information.

Images