system

Abstract

Provide a system. [Solution] A means of providing digital picture books to young children, means for displaying the digital picture book data received from the server; means for turning pages in response to a user's operation; means for playing audio files; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional paper picture books and teaching materials have difficulty stimulating the concentration and creativity of young children, and are unable to provide efficient support when there are areas in the learning process where understanding is lacking. Furthermore, there are also insufficient means to alleviate the sense of loneliness when children feel distressed or lonely and have no one nearby to talk to. A new system was needed that utilizes digital technology to solve these issues and promote children's emotional and cognitive growth. [Means for solving the problem]

[0005] The present invention provides a system including means for providing digital picture books to young children, means for displaying digital picture book data received from a server, means for turning pages in response to user operations, and means for playing audio files. The system also provides support functions including means for children or students to input questions, means for sending the questions to a server, means for the server to analyze the questions and generate appropriate answers, means for the server to send the generated answers to the user, and means for the user to input additional questions. The system also provides emotional support functions including means for the server to receive messages based on the user's emotions, means for analyzing the received messages, means for generating response messages according to the emotions, means for sending the response messages to the user, and means for the user to input additional messages and continue the dialogue. This system enhances children's creativity and learning efficiency and provides emotional support.

[0006] A "digital picture book" is picture book-style content that can be viewed on a digital device and includes digital media such as images, text, and audio.

[0007] A "server" is a computer system that provides data and services over a network.

[0008] A "terminal" is a device such as a computer, tablet, or smartphone that can be directly operated by a user.

[0009] "Users" refer to people who use the system, and in this invention, this refers primarily to young children, pupils and students.

[0010] An "operation" refers to an action such as tapping, swiping, or typing that a user performs on a device.

[0011] "Page turning means" refers to an interaction method that allows a user to go forward or backward through pages of a digital picture book.

[0012] "Audio File" means a digital file for storing and playing audio data.

[0013] "Means for inputting questions" refers to an interface that allows users to input doubts or questions to the system.

[0014] "Analysis" refers to information processing technology that processes the data received by the server and understands its meaning and intent.

[0015] "Means for generating appropriate answers" refers to the process by which the server generates appropriate information or answers to the user's questions.

[0016] "Emotion-based messages" are messages that analyze the text or voice entered by the user and respond to their emotions.

[0017] The "means for generating a response message" is the process by which the server analyzes the input from the user and creates an appropriate message based on the results.

[0018] A "means for continuing a conversation" refers to the interface and protocol that allows a user and a server to continue a series of exchanges. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] This invention is a system that provides digital picture books to young children, helps children and students to deeply understand the content they are learning, and provides emotional support when they feel worried or lonely.

[0041] Providing digital picture books

[0042] server

[0043] 1. Maintain a database of digital picture books.

[0044] 2. When a request is received from the user (child), the corresponding digital picture book data is obtained and sent to the terminal.

[0045] Terminal

[0046] 1. Based on the digital picture book data received from the server, it is displayed to the user in an interactive interface.

[0047] 2. Turn pages in response to user taps and swipes.

[0048] 3. Play the audio file based on the user's instructions.

[0049] User

[0050] 1. Select the picture book you want to read from the options on the screen.

[0051] 2. Enjoy the content by tapping or swiping the screen to turn the pages.

[0052] 3. Tap to play audio descriptions related to characters and scenes.

[0053] Learning support features

[0054] server

[0055] 1. Receive a question from the user and analyze its content.

[0056] 2. Retrieve appropriate learning materials from the database and generate answers.

[0057] 3. Send the generated answer to the device.

[0058] Terminal

[0059] 1. The question entered by the user is sent to the server.

[0060] 2. Receive the response from the server and display it to the user.

[0061] 3. If the user enters additional questions, the additional questions are sent to the server again.

[0062] User

[0063] 1. Enter questions about content you didn't understand in class or at cram school.

[0064] 2. Check the answer displayed on the device.

[0065] 3. Enter additional questions as needed until you are satisfied.

[0066] Emotional support

[0067] server

[0068] 1. Receive a message from the user and analyze it using a sentiment analysis engine.

[0069] 2. Generate an appropriate response message that corresponds to the user's emotions.

[0070] 3. Send the generated response message to the terminal.

[0071] Terminal

[0072] 1. The user's concerns and questions are sent to the server.

[0073] 2. Receive the response message from the server and display it to the user.

[0074] 3. If the user enters a message again, the message is sent to the server again.

[0075] User

[0076] 1. Enter what is bothering you or making you feel lonely.

[0077] 2. Check the response message displayed on the device.

[0078] 3. Enter an additional message to continue the conversation, if desired.

[0079] For example, if a user inputs "I'm lonely because I haven't made any friends at school recently," the server will generate and send a response message such as "That's sad. But there are opportunities to make new friends. Do you have any favorite games or activities?" By repeating this kind of dialogue between the user and the server, the system helps to lighten the user's mood, even if only a little.

[0080] In this way, the educational and emotional support system that makes full use of digital technology aims to promote children's overall development, conveying the joy of picture books, deepening their understanding of learning, and alleviating feelings of loneliness.

[0081] The processing flow will be explained below.

[0082] Providing digital picture books

[0083] Server Processing

[0084] Step 1:

[0085] The server receives a request from a user.

[0086] Step 2:

[0087] The server retrieves the requested digital picture book data from the database.

[0088] Step 3:

[0089] The server transmits the acquired data to the terminal.

[0090] Terminal handling

[0091] Step 1:

[0092] The terminal receives the digital picture book data transmitted from the server.

[0093] Step 2:

[0094] The terminal analyzes the received data and generates an interactive interface.

[0095] Step 3:

[0096] The device detects the user's taps and swipes and performs page-turning processing accordingly.

[0097] Step 4:

[0098] The terminal plays the audio file in response to a user instruction.

[0099] User Actions

[0100] Step 1:

[0101] The user selects the picture book they want to read from the options on the screen.

[0102] Step 2:

[0103] Users tap or swipe the screen to turn pages.

[0104] Step 3:

[0105] The user can play audio guides by tapping on characters or scenes.

[0106] Learning support features

[0107] Server Processing

[0108] Step 1:

[0109] The server receives a question sent by a user.

[0110] Step 2:

[0111] The server analyzes the received question using an NLP (natural language processing) engine.

[0112] Step 3:

[0113] The server retrieves appropriate learning materials from a database and generates answers.

[0114] Step 4:

[0115] The server sends the generated response to the terminal.

[0116] Terminal handling

[0117] Step 1:

[0118] The terminal transmits the question entered by the user to the server.

[0119] Step 2:

[0120] The terminal receives the response from the server.

[0121] Step 3:

[0122] The terminal displays the received response to the user.

[0123] Step 4:

[0124] If the user inputs more questions, the terminal transmits the additional questions to the server again.

[0125] User Actions

[0126] Step 1:

[0127] The user inputs questions about content that he or she did not understand in class or at a cram school.

[0128] Step 2:

[0129] The user checks the answer displayed on the terminal.

[0130] Step 3:

[0131] The user continues to enter additional questions until satisfied.

[0132] Emotional support

[0133] Server Processing

[0134] Step 1:

[0135] The server receives the consultation content sent from the user.

[0136] Step 2:

[0137] The server analyzes the received message using an emotion analysis engine.

[0138] Step 3:

[0139] The server generates an appropriate response message according to the user's emotions.

[0140] Step 4:

[0141] The server sends the generated response message to the terminal.

[0142] Terminal handling

[0143] Step 1:

[0144] The terminal transmits the worries and consultation details input by the user to the server.

[0145] Step 2:

[0146] The terminal receives the response message from the server.

[0147] Step 3:

[0148] The terminal displays the received response message to the user.

[0149] Step 4:

[0150] If the user inputs a message again, the terminal transmits the additional message to the server again.

[0151] User Actions

[0152] Step 1:

[0153] The user inputs what is bothering them or making them feel lonely.

[0154] Step 2:

[0155] The user checks the response message displayed on the terminal.

[0156] Step 3:

[0157] The user may optionally enter additional messages to continue the dialogue.

[0158] Example 1

[0159] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0160] Although today's children are familiar with digital devices, there is a lack of effective digital picture books and learning support systems. Furthermore, it is difficult to provide appropriate support when children need emotional support. This invention proposes a system that provides digital picture books, learning support, and emotional support in an integrated manner, thereby solving these issues.

[0161] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0162] In this invention, the server includes means for providing digital picture books to young children, means for displaying digital picture book data received from the server on a terminal, means for turning pages in response to tapping or swiping operations by the user, and means for playing audio files in response to user instructions, thereby enabling children to learn interactively while enjoying the digital picture books.

[0163] In addition, the learning support function is enhanced by adding a means for children and students to input questions, a means for sending the input questions to a server, a means for the server to analyze the questions and generate appropriate answers, a means for sending the generated answers to the user, and a means for the user to input further questions.

[0164] Furthermore, the server includes means for receiving a message based on the user's emotions, means for analyzing the received message using an emotion analysis engine, means for generating a response message according to the user's emotions, means for sending the generated response message to the user, and means for the user to input another message and continue the dialogue, thereby making it possible to provide emotional support.

[0165] In this way, the present invention can provide an integrated system that comprehensively provides education and emotional support for children.

[0166] "Young children" generally refers to children from toddlers to early elementary school age.

[0167] A "digital picture book" is a picture book displayed on a computer or mobile device and is an electronic book that includes multimedia elements such as text, images, and audio.

[0168] A "server" refers to a computer system that has the function of receiving requests from users, processing data, and sending it to terminals.

[0169] "Database" refers to a data storage system for centrally managing and storing digital picture book data and learning materials.

[0170] An "interactive interface" refers to a user interface in which the content on the screen responds to user operations, enabling two-way interaction.

[0171] "Tap" refers to the act of lightly tapping with a fingertip on a touchscreen device.

[0172] "Swipe" refers to the action of sliding your finger across a touchscreen device.

[0173] "Audio file" refers to a file that stores audio data in digital format.

[0174] A "question" refers to text that a user enters about something they don't understand or about information they want to confirm.

[0175] A "natural language processing engine" refers to a computer program that analyzes text data and understands its meaning and content.

[0176] A "generative AI model" refers to a trained algorithm that uses artificial intelligence to generate text and voice data.

[0177] An "emotion analysis engine" refers to technology for identifying and analyzing user emotions from text data.

[0178] A "response message" refers to a message generated as a reply or response to a message received from a user.

[0179] "Means for continuing a dialogue" refers to a method for continuing to exchange messages between a user and a system.

[0180] This invention is a system that provides digital picture books to young children, supports them in understanding the learning content deeply, and also provides emotional support. A specific implementation method is shown below.

[0181] Providing digital picture books

[0182] server

[0183] The server maintains a database of digital picture books. This database stores image data, text data, audio files, etc. of picture books. When a request from a user (child) reaches the server via a terminal, the server searches the database for the relevant picture book data and retrieves it. The retrieved data is then converted into an appropriate format and sent to the terminal.

[0184] Terminal

[0185] The device displays the picture book data received from the server in an interactive interface. The received data is decoded, and the image data is displayed in an image viewer, and the text data is displayed in a text viewer. The user can turn pages or play character audio guides by tapping or swiping the screen.

[0186] User

[0187] The user selects the picture book they want to read from the list displayed on the device screen and taps or swipes the screen to view the contents. For example, if the user selects the picture book "Little Red Riding Hood," the server retrieves Little Red Riding Hood's data from the database and sends it to the device. Each time the user taps, the page turns, and an audio guide is played if necessary.

[0188] Learning support features

[0189] server

[0190] The server receives questions from users and analyzes them using a natural language processing engine. Based on the analyzed questions, it retrieves appropriate learning materials from a database and generates the optimal answer using a generative AI model. The generated answer is then sent to the device.

[0191] Terminal

[0192] The device sends the question entered by the user to the server, and the answer received from the server is displayed on the interface. For example, if a user enters "I don't know how to calculate the area of ​​a triangle," the server will generate an answer such as "The area of ​​a triangle can be calculated by multiplying the length of the base by the height and dividing by 2." The user can enter additional information until they are satisfied.

[0193] User

[0194] Users can enter questions into a text field on the device, see the answers displayed, and optionally enter additional questions to seek further answers.

[0195] Emotional support

[0196] server

[0197] The server receives emotion-based messages from users, analyzes them using an emotion analysis engine, and generates an appropriate response message using a generative AI model based on the analysis results, which is then sent to the device.

[0198] Terminal

[0199] The device sends the worries and concerns entered by the user to the server and displays the response message received from the server. For example, if the user enters "I'm lonely because I haven't made any friends at school recently," the server will generate and send a response message such as "That's sad. But there are opportunities to make new friends. Do you have any favorite games or activities?"

[0200] User

[0201] The user inputs their concerns or worries into the terminal, checks the response message that is displayed, and continues the dialogue by inputting the message again if necessary.

[0202] This invention provides an environment where children can grow and have fun through learning using digital devices by providing digital picture books, learning support, and emotional support in an integrated manner.

[0203] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0204] Providing digital picture books

[0205] Step 1:

[0206] server

[0207] The server maintains a database of digital picture books. The database stores image data, text data, audio files, etc. of picture books. When the server starts up, it reads this data and prepares it. Input: All data in the database. Output: Prepared digital picture book data.

[0208] Step 2:

[0209] server

[0210] The server receives a digital picture book request from the user (child). The request includes the user ID and the title or ID of the picture book they want to read. Input: Request from the user. Output: Identification information of the requested picture book.

[0211] Step 3:

[0212] server

[0213] The server searches and retrieves the corresponding digital picture book data from the database based on the identification information. Data is extracted efficiently using SQL queries. Input: Identification information. Output: Retrieved picture book data.

[0214] Step 4:

[0215] server

[0216] The server converts the acquired picture book data into an appropriate format such as JSON and sends it to the terminal. This is where the data is encoded. Input: Acquired picture book data. Output: Formatted picture book data.

[0217] Step 5:

[0218] Terminal

[0219] The device receives the digital picture book data sent from the server. The received data is encoded in JSON format or similar. Input: Data sent from the server. Output: Received encoded data.

[0220] Step 6:

[0221] Terminal

[0222] The terminal decodes the data it receives and converts it into a format that can be displayed in the interface. Image data is displayed in an image viewer, text in a text viewer. Input: The encoded data. Output: The data that can be displayed in the interface.

[0223] Step 7:

[0224] User

[0225] The user selects the book they want to read from the list of digital books displayed on the device screen and confirms the request. They then enjoy the content by turning the pages using on-screen operations (tapping and swiping). Input: Device operations (tapping and swiping). Output: Display of pages and playback of audio guide.

[0226] Learning support features

[0227] Step 1:

[0228] User

[0229] The user types a question into a text field on the device. The input is related to a learning topic that they do not understand. Input: Learning question. Output: The entered question text.

[0230] Step 2:

[0231] Terminal

[0232] The terminal sends the question entered by the user to the server. Input: Question from the user. Output: Question text sent to the server.

[0233] Step 3:

[0234] server

[0235] The server receives the question and analyzes the question using a natural language processing engine. Input: Received question text. Output: Analyzed question content.

[0236] Step 4:

[0237] server

[0238] The server retrieves appropriate learning materials from the database based on the analyzed question. It then uses a generative AI model to generate the optimal answer. Input: Analyzed question. Output: Generated answer text.

[0239] Step 5:

[0240] server

[0241] The server sends the generated answer to the terminal. The answer is encoded in JSON format or similar. Input: Generated answer text. Output: Encoded answer data.

[0242] Step 6:

[0243] Terminal

[0244] The terminal receives the response sent from the server and decodes it into a format that can be displayed. Input: Response data from the server. Output: Displayable response text.

[0245] Step 7:

[0246] User

[0247] The user checks the answer displayed on the terminal, and if they are not satisfied, they can enter an additional question. Input: Answer text, additional question. Output: New question text.

[0248] Emotional support

[0249] Step 1:

[0250] User

[0251] The user inputs the problem or problem content into the text field of the terminal. Input: Problem or problem content. Output: Input text message.

[0252] Step 2:

[0253] Terminal

[0254] The terminal sends the message entered by the user to the server. Input: Text message from the user. Output: Message sent to the server.

[0255] Step 3:

[0256] server

[0257] The server analyzes the received message with a sentiment analysis engine to identify the user's sentiment. Input: Received text message. Output: Identified sentiment data.

[0258] Step 4:

[0259] server

[0260] The server generates an appropriate response message using a generative AI model based on the identified emotion data. Input: Identified emotion data. Output: Generated response message.

[0261] Step 5:

[0262] server

[0263] The server sends the generated response message to the terminal. The response message is sent in text format. Input: Generated response message. Output: Response message sent to the terminal.

[0264] Step 6:

[0265] Terminal

[0266] The terminal displays the response message received from the server to the user. Input: Received response message. Output: Displayed response message.

[0267] Step 7:

[0268] User

[0269] The user checks the response message displayed on the terminal and continues the dialogue by inputting another message if necessary. Input: Input of a message on the terminal. Output: A new message to the server.

[0270] In this system, various data exchanges and processing are carried out between the server, terminals, and users to effectively realize the functions of providing digital picture books, learning support, and emotional support.

[0271] (Application example 1)

[0272] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0273] To support the learning and emotional development of young children, there is a need for systems that provide a consistent approach to reading picture books, learning support, and emotional support. However, no existing systems offer these multiple functions in an integrated manner; most are specialized for individual purposes. Therefore, there is a need for a comprehensive system that can maintain children's interest, deepen their understanding of learning, and provide emotional support in educational settings such as cram schools.

[0274] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0275] In this invention, the server includes means for providing digital picture books to young children, means for displaying digital picture book data received from the server, means for turning pages in response to user operations, means for playing audio files, means for children at cram schools to input questions using smartphones and send them to the server, means for displaying answers received from the server, and means for analyzing the user's emotions and generating and sending appropriate response messages. This makes it possible to provide appropriate answers to learning questions and emotional support while maintaining children's interest through the provision of digital picture books.

[0276] "Young children" refers primarily to young people from preschool to early elementary school age, an age group that is particularly interested in digital picture books and learning support.

[0277] A "digital picture book" is a picture book that is provided in digital format rather than on paper and can be viewed on an electronic device.

[0278] A "server" is a computer system that centrally manages digital picture book data, learning answers, emotion analysis messages, etc., and communicates with user terminals.

[0279] "User operation" refers to input actions that a user performs on the system through a terminal, such as tapping or swiping on the screen.

[0280] The "means for turning pages" is a mechanism that provides a function for turning pages of a digital picture book forward or backward in response to user operations.

[0281] "Audio file" refers to audio recording data, which is used to provide audio readings and explanations of digital picture books.

[0282] A "cram school" is an educational facility where children can receive learning support outside of class.

[0283] "Means for inputting questions" refers to the function that allows users to input questions about their studies as text information into the terminal.

[0284] "Sentiment analysis" refers to a technology that analyzes text data from a user and identifies the emotions the user is feeling.

[0285] A "response message" is a message that is generated based on the results of emotion analysis and sent to the user.

[0286] "Display means" refers to the function for displaying information such as digital picture book data, answers, and response messages sent from the server on the screen of the user's terminal.

[0287] This invention is a system that allows young children to learn while having fun with digital picture books and also receive emotional support. This system consists of a server, a terminal operated by the user, and the user. The configuration and operation of the system are described in detail below.

[0288] Providing digital picture books

[0289] server

[0290] The server maintains a database of digital picture books and is responsible for transmitting the corresponding digital picture book data when requested by a user (young child). The server is equipped with functions for efficiently managing and distributing the metadata and content of digital picture books.

[0291] Terminal

[0292] The device operated by the user (smartphone or tablet) receives the digital picture book data from the server and displays it in an interactive interface, allowing the user to turn pages by tapping or swiping the screen and play audio guides related to characters and scenes.

[0293] User

[0294] Users select the picture book they want to read from the options on the screen, advance through the pages through operations, and enjoy the contents of the picture book using audio guidance.

[0295] Learning support features

[0296] server

[0297] The server receives a question from the user, analyzes the question, retrieves appropriate learning materials from a database, and generates an answer, which is then sent to the terminal.

[0298] Terminal

[0299] The terminal transmits the questions entered by the user to the server, receives and displays the answers from the server, and if the user enters additional questions, the additional questions can be transmitted to the server again.

[0300] User

[0301] Users can enter questions about content they did not understand in class or at a cram school into the device, check the answers provided, and enter additional questions as needed.

[0302] Emotional support

[0303] server

[0304] The server receives messages from users, analyzes their emotions using AI, and generates an appropriate response message based on the analysis results, which is then sent to the device.

[0305] Terminal

[0306] The terminal inputs the user's worries or feelings of loneliness and sends them to the server. It receives and displays a response message from the server, and if the user inputs another message, it sends that message to the server again.

[0307] User

[0308] The user inputs their concerns or feelings of loneliness into the terminal, confirms the response message provided, and inputs an additional message to continue the dialogue if necessary.

[0309] Hardware and software used

[0310] Hardware used

[0311] Smartphones and tablets: Used by users as portable devices.

[0312] Server: A central management system that manages the digital picture book database and AI responses.

[0313] Software used

[0314] Flutter (registered trademark) / React Native: Used to build and operate the UI of smartphone apps.

[0315] requests (Python library): A library for enabling communication between the server and the terminal.

[0316] Generative AI model: AI technology used for sentiment analysis and response message generation.

[0317] Examples and prompts

[0318] As a specific example, if the user selects the digital picture book "Forest Stories," the terminal operates as follows.

[0319] 1. The user selects the "Forest Tales" digital picture book.

[0320] 2. The terminal sends a request to the server, receives the digital picture book data from the server, and displays it.

[0321] 3. When the user taps to turn the page, the appropriate page is displayed and the associated audio plays.

[0322] Example prompt sentence:

[0323] Get and display the page data for the digital picture book "Forest Stories." URL: https: / / example.com / digital_books / 001

[0324] In this way, the present invention provides a comprehensive system that provides relevant answers to educational questions and emotional support while retaining the interest of young children.

[0325] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0326] Step 1:

[0327] The user uses the terminal to make a digital picture book request.

[0328] Input: A request for a digital picture book based on the user's selection

[0329] Output: Request sent to server

[0330] How it works: The user selects a picture book, such as "Forest Stories," from the interface on the device, and the information is sent to the server.

[0331] Step 2:

[0332] The server receives the request, acquires the corresponding digital picture book data, and transmits it.

[0333] Input: Digital picture book request from user device

[0334] Output: Sending digital picture book data

[0335] Operation: The server retrieves the requested picture book data from the digital picture book database and sends it to the user's terminal.

[0336] Step 3:

[0337] The terminal displays the digital picture book data received from the server.

[0338] Input: Digital picture book data from the server

[0339] Output: Display a digital picture book on the screen

[0340] Operation: The device displays the received digital picture book data on an interactive interface, allowing the user to turn pages and play audio.

[0341] Step 4:

[0342] The user inputs a question related to the study and sends it to the server via the terminal.

[0343] Input: User's question

[0344] Output: Sending the query data to the server

[0345] How it works: The user enters a study question into the terminal, and the data is sent to the server.

[0346] Step 5:

[0347] The server analyzes the received question, retrieves appropriate learning materials from a database, and generates an answer.

[0348] Input: Question data from the user

[0349] Output: Generated training answers

[0350] How it works: The server analyzes the received question, retrieves relevant learning material from the database, and generates an appropriate answer.

[0351] Step 6:

[0352] The terminal displays the answer received from the server to the user.

[0353] Input: Learning response data from the server

[0354] Output: Display the answer to the user

[0355] Operation: The terminal displays the answer received from the server to the user, who then confirms the content.

[0356] Step 7:

[0357] If the user enters more questions, the questions are sent to the server again.

[0358] Input: User's additional question

[0359] Output: Sending additional questions to the server

[0360] How it works: The user enters a follow-up question into the terminal, and the data is sent back to the server.

[0361] Step 8:

[0362] The user inputs the emotional worries and consultation contents and transmits them to the server.

[0363] Input: User's sentiment message

[0364] Output: Sending emotion message to the server

[0365] How it works: The user enters a message about their worries or loneliness into the device, and the data is sent to the server.

[0366] Step 9:

[0367] The server analyzes the sentiment of the received message and generates an appropriate response message.

[0368] Input: Emotional message from the user

[0369] Output: The generated response message

[0370] How it works: The server analyzes the received message using an AI model and generates an appropriate emotional response message.

[0371] Step 10:

[0372] The terminal displays the response message received from the server to the user.

[0373] Input: Response message from the server

[0374] Output: Display a response message to the user

[0375] Operation: The terminal displays the received response message to the user, who confirms it.

[0376] Step 11:

[0377] The user inputs a message again to continue the dialogue and sends it to the server.

[0378] Input: User's additional emotion message

[0379] Output: Send additional emotion message to server

[0380] Operation: The user enters a message into the terminal to continue the dialogue, and the data is sent again to the server.

[0381] In this way, the present invention provides a system that supports the user's learning and also provides emotional support at the same time.

[0382] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0383] This invention is a system that provides digital picture books to young children, supports children and students in gaining a deeper understanding of the learning content, and provides emotional support when they feel distressed or lonely. By combining a specific emotion engine, the system recognizes the user's emotions and provides appropriate responses accordingly.

[0384] Providing digital picture books

[0385] server

[0386] 1. Maintain a database of digital picture books.

[0387] 2. When a request is received from the user (child), the corresponding digital picture book data is obtained and sent to the terminal.

[0388] Terminal

[0389] 1. Based on the digital picture book data received from the server, it is displayed to the user in an interactive interface.

[0390] 2. Detect user taps and swipes and perform page-turning processing accordingly.

[0391] 3. Play the audio file according to the user's instructions.

[0392] User

[0393] 1. Select the picture book you want to read from the options on the screen.

[0394] 2. Enjoy the content by tapping or swiping the screen to turn the pages.

[0395] 3. Tap to play audio descriptions related to characters and scenes.

[0396] Learning support features

[0397] server

[0398] 1. Receive a question from the user and analyze its content.

[0399] 2. Retrieve appropriate learning materials from the database and generate answers.

[0400] 3. Send the generated answer to the device.

[0401] Terminal

[0402] 1. The question entered by the user is sent to the server.

[0403] 2. Receive the response from the server and display it to the user.

[0404] 3. If the user enters additional questions, the additional questions are sent to the server again.

[0405] User

[0406] 1. Enter questions about content you didn't understand in class or at cram school.

[0407] 2. Check the answer displayed on the device.

[0408] 3. Enter additional questions as needed until you are satisfied.

[0409] Emotional Support and Emotion Engine

[0410] server

[0411] 1. Receive the consultation content sent by the user.

[0412] 2. The message is analyzed using a sentiment analysis engine to identify the user's sentiment.

[0413] 3. Generate an appropriate response message that corresponds to the user's emotions.

[0414] 4. Send the generated response message to the terminal.

[0415] 5. Accumulates user's past emotional data to improve the accuracy of future response messages.

[0416] Terminal

[0417] 1. The user's concerns and questions are sent to the server.

[0418] 2. Receive the response message from the server and display it to the user.

[0419] 3. If the user enters a message again, the message is sent to the server again.

[0420] User

[0421] 1. Enter what is bothering you or making you feel lonely.

[0422] 2. Check the response message displayed on the device.

[0423] 3. If necessary, enter an additional message to continue the conversation.

[0424] For example, if a user types, "I feel lonely because I haven't made any friends recently," the server's emotion analysis engine will identify the user's emotion as "loneliness." The server will then generate a response message such as, "That's tough. But you have a chance to make new friends. Maybe you should try looking for someone who has a common hobby with you," and send it to the device. If the user receives this message and continues the dialogue, it can alleviate the user's feelings of loneliness and worries.

[0425] The introduction of this system can improve children's creativity, learning efficiency, and emotional support, promoting their overall development.

[0426] The processing flow will be explained below.

[0427] Providing digital picture books

[0428] Server Processing

[0429] Step 1:

[0430] The server receives a request from a user.

[0431] Step 2:

[0432] The server retrieves the requested digital picture book data from the database.

[0433] Step 3:

[0434] The server transmits the acquired data to the terminal.

[0435] Terminal handling

[0436] Step 1:

[0437] The terminal receives the digital picture book data transmitted from the server.

[0438] Step 2:

[0439] The terminal analyzes the received data and generates an interactive interface.

[0440] Step 3:

[0441] The device detects the user's taps and swipes and performs page-turning processing accordingly.

[0442] Step 4:

[0443] The terminal plays the audio file in response to a user instruction.

[0444] User Actions

[0445] Step 1:

[0446] The user selects the picture book they want to read from the options on the screen.

[0447] Step 2:

[0448] Users tap or swipe the screen to turn pages.

[0449] Step 3:

[0450] The user can play audio guides by tapping on characters or scenes.

[0451] Learning support features

[0452] Server Processing

[0453] Step 1:

[0454] The server receives a question sent by a user.

[0455] Step 2:

[0456] The server analyzes the received question using an NLP (natural language processing) engine.

[0457] Step 3:

[0458] The server retrieves appropriate learning materials from a database and generates answers.

[0459] Step 4:

[0460] The server sends the generated response to the terminal.

[0461] Terminal handling

[0462] Step 1:

[0463] The terminal transmits the question entered by the user to the server.

[0464] Step 2:

[0465] The terminal receives the response from the server.

[0466] Step 3:

[0467] The terminal displays the received response to the user.

[0468] Step 4:

[0469] If the user inputs more questions, the terminal transmits the additional questions to the server again.

[0470] User Actions

[0471] Step 1:

[0472] The user inputs questions about content that he or she did not understand in class or at a cram school.

[0473] Step 2:

[0474] The user checks the answer displayed on the terminal.

[0475] Step 3:

[0476] The user continues to enter additional questions until satisfied.

[0477] Emotional Support and Emotion Engine

[0478] Server Processing

[0479] Step 1:

[0480] The server receives the consultation content sent from the user.

[0481] Step 2:

[0482] The server analyzes the received message using an emotion analysis engine to identify the user's emotion.

[0483] Step 3:

[0484] The server generates an appropriate response message according to the user's emotions based on the results of the emotion analysis.

[0485] Step 4:

[0486] The server sends the generated response message to the terminal.

[0487] Step 5:

[0488] The server accumulates the user's past emotional data and improves the accuracy of future response messages.

[0489] Terminal handling

[0490] Step 1:

[0491] The terminal transmits the worries and consultation details input by the user to the server.

[0492] Step 2:

[0493] The terminal receives the response message from the server.

[0494] Step 3:

[0495] The terminal displays the received response message to the user.

[0496] Step 4:

[0497] If the user inputs a message again, the terminal transmits the additional message to the server again.

[0498] User Actions

[0499] Step 1:

[0500] The user inputs what is bothering them or making them feel lonely.

[0501] Step 2:

[0502] The user checks the response message displayed on the terminal.

[0503] Step 3:

[0504] The user may optionally enter additional messages to continue the dialogue.

[0505] Example 2

[0506] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0507] In today's educational environment, there is a need for digital picture books that allow young children to deepen their knowledge while having fun learning, and for methods that encourage students to ask questions about the content they are learning and deepen their understanding. It is also important to provide appropriate emotional support when children feel distressed or lonely. There is a lack of systems that can meet these needs and promote children's holistic development.

[0508] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0509] In this invention, the server includes an information transmission device means for storing data, a display device means for displaying data received from the information transmission device, a device means for switching pages in response to operations, and a device means for playing audio data, which enables provision of digital picture books, understanding of learning content, and emotional support.

[0510] "Data" means electronic records of specific information or content.

[0511] An "information transmitting device" refers to a device that holds data and transmits it to a terminal or display device.

[0512] A "display device" is a device for visually displaying received data to a user.

[0513] An "operation" refers to an input action that a user performs using a terminal or a display device.

[0514] A "page-switching device" is a device that has the function of turning pages of a digital picture book or the like in response to a user's operation.

[0515] "Audio data" means electronic recordings of audio information.

[0516] A "device that plays back audio data" is a device that has the function of receiving and playing back audio data.

[0517] A "question input device" is a device that allows a user to input their own questions or doubts.

[0518] The "device for additional input" refers to a device that allows the user to input information again.

[0519] An "analyzing device" is a device that has the function of analyzing an input question or message and generating an appropriate answer or response.

[0520] An "emotion-based message" is a message whose content reflects the user's emotional state.

[0521] A "response message" is a response generated based on the analyzed content.

[0522] The "display device" refers to a device that displays the generated answer or response message in an easy-to-see manner for the user.

[0523] The present invention is a system that provides digital picture books to young children, supports children and students in gaining a deeper understanding of the learning content, and provides emotional support when they feel distressed or lonely. The following describes specific embodiments of the invention.

[0524] Providing digital picture books

[0525] server

[0526] The server maintains a database of digital picture books. When a request is received from a user (child), the server retrieves the corresponding digital picture book data from the database and sends it to the terminal. For this reason, the server has high-performance database software (e.g., MySQL (registered trademark)) and data transmission functions.

[0527] Terminal

[0528] The terminal receives the digital picture book data from the server and displays it to the user using an interactive interface. It is typically a tablet or smartphone, and is equipped with an LCD display or touchscreen. The terminal detects the user's taps and swipes and processes the pages accordingly. It also plays audio data in response to user instructions. This is achieved using a built-in speaker or earphones.

[0529] User

[0530] When reading a digital picture book, users select the book they want from a list on the screen. They turn the pages by tapping or swiping, and when they tap on parts related to characters or scenes, audio guides and sound effects are played.

[0531] Learning support features

[0532] server

[0533] The server receives a question from the user and analyzes it using a natural language processing engine (e.g., Google® Natural Language API). It retrieves appropriate learning materials from a database and generates an answer. The server is equipped with data analysis and transmission functions to send the generated answer to the device.

[0534] Terminal

[0535] The terminal sends the question entered by the user to the server, receives the answer from the server and displays it to the user, and if the user enters another question, sends the additional question back to the server and displays related additional learning materials.

[0536] User

[0537] Users can enter questions about things they didn't understand in class or at a cram school. They can check the answers and explanations displayed on the device to deepen their understanding. If necessary, they can enter further questions and repeat the process until they are satisfied.

[0538] Emotional Support and Emotion Engine

[0539] server

[0540] The server receives the consultation content sent by the user. It analyzes the message using an emotion analysis engine (e.g., IBM Watson (registered trademark) Tone Analyzer) to identify the user's emotion. It generates a response message according to the user's emotion and sends it to the device. It also accumulates data on the user's past emotions to improve accuracy.

[0541] Terminal

[0542] The terminal sends the worries or consultation details entered by the user to the server. It receives a response message from the server and displays it to the user. If the user wants to enter a message again, it can send that message to the server again.

[0543] User

[0544] The user inputs what is bothering them or what is making them feel lonely. They check the response message displayed on the terminal and, if necessary, input an additional message to continue the conversation.

[0545] Specific examples and examples of input to generative AI models

[0546] For example, if a user types, "I'm lonely because I haven't made any friends recently," the server will use its emotion analysis engine to identify the user's emotion as "loneliness." It will then generate a response message such as, "That's tough. But you have a chance to make new friends. Maybe you should try looking for someone who has common hobbies with you," and send it to the device. By receiving this message and continuing the dialogue, the user can ease their feelings of loneliness and distress.

[0547] Example of input prompt for generative AI model

[0548] 1. "I'm lonely and can't make friends. Please tell me what to do."

[0549] 2. "I'm having trouble understanding some of the things I'm studying at school. Can you explain?"

[0550] 3. "The picture book I want to read right now is one with lots of animals in it."

[0551] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0552] Providing digital picture books

[0553] Step 1:

[0554] The server maintains a database of digital picture books, which stores data on a plurality of digital picture books.

[0555] Input: A digital picture book request from a user

[0556] Data processing: Analyze user requests and identify the corresponding digital picture book data

[0557] Output: Identified digital picture book data

[0558] Specific operation: The server receives the HTTP request, executes an SQL query based on it, and retrieves the required data from the database.

[0559] Step 2:

[0560] The server transmits the corresponding digital picture book data to the terminal.

[0561] Input: Identified digital picture book data

[0562] Data processing: Digital picture book data is converted into packets according to communication protocols (e.g. HTTP, HTTPS).

[0563] Output: Data packets sent to the device

[0564] Specific operation: The server sends the digital picture book data encoded using HTTPS to the terminal.

[0565] Step 3:

[0566] The terminal displays the digital picture book data received from the server to the user on an interface.

[0567] Input: Digital picture book data received from the server

[0568] Data processing: Analyzes received data and converts it into a display format

[0569] Output: A digital picture book displayed in a form visible to the user

[0570] Specific operation: The device analyzes the images and text data of the digital picture book and displays them on the screen.

[0571] Step 4:

[0572] The device detects the user's taps and swipes and performs page-turning processing accordingly.

[0573] Input: User taps and swipes

[0574] Data processing: Analyzes the operation data and converts the data on the next page into a display format

[0575] Output: Next page displayed

[0576] Specific operation: The gesture engine analyzes touch operations and executes page transition animations.

[0577] Step 5:

[0578] The terminal plays the audio file in response to a user instruction.

[0579] Input: User's voice instructions (tap operation, etc.)

[0580] Data processing: Read and play audio files based on instructions

[0581] Output: Audio is played

[0582] Specific behavior: The device loads the audio file from storage and plays it using the media player.

[0583] Learning support features

[0584] Step 1:

[0585] The server receives a query from the user.

[0586] Input: The question the user types into the terminal

[0587] Data processing: Transfer received questions to the server

[0588] Output: Query data to the server

[0589] Specific operation: Sends the question data to the server as an HTTP POST request.

[0590] Step 2:

[0591] The server analyzes the received question using a natural language processing engine.

[0592] Input: Received question data

[0593] Data processing: Analysis using a natural language processing engine

[0594] Output: Analysis results

[0595] Specific operation: The server uses the Google Natural Language API to analyze the question data and extract keywords and intent.

[0596] Step 3:

[0597] The server retrieves appropriate learning materials from a database and generates answers.

[0598] Input: Analysis results of natural language processing

[0599] Data processing: Run database queries to identify appropriate learning materials and generate answers.

[0600] Output: Generated response data

[0601] Specific operation: Executes an SQL query to retrieve teaching material data and generates answer sentences using a template.

[0602] Step 4:

[0603] The server sends the generated response to the terminal.

[0604] Input: Generated response data

[0605] Data processing: Converts response data into packets according to the communication protocol

[0606] Output: Data packets sent to the device

[0607] Specific operation: Sends response data to the terminal via an HTTPS request.

[0608] Step 5:

[0609] The terminal displays the response received from the server to the user.

[0610] Input: Response data received from the server

[0611] Data processing: Converting response data into a display format

[0612] Output: The answer that is displayed to the user

[0613] Specific operation: The device renders and displays the received text data on the screen.

[0614] Step 6:

[0615] If the user has further questions, they enter them.

[0616] Input: User's additional question

[0617] Data processing: Send additional questions to the server

[0618] Output: Additional question data to the server

[0619] Specific operation: Sends the additional question to the server again as an HTTP POST request.

[0620] Emotional Support and Emotion Engine

[0621] Step 1:

[0622] The server receives the consultation content from the user.

[0623] Input: The consultation content entered by the user into the terminal

[0624] Data processing: Transfer received consultation details to the server

[0625] Output: Consultation content data to the server

[0626] Specific operation: Consultation content data is sent to the server as an HTTP POST request.

[0627] Step 2:

[0628] The server analyzes the consultation content using an emotion analysis engine and identifies the user's emotions.

[0629] Input: Received consultation content data

[0630] Data processing: Analysis using a sentiment analysis engine

[0631] Output: Emotion analysis results

[0632] Specific operation: Uses IBM Watson Tone Analyzer to analyze text data and extract emotions.

[0633] Step 3:

[0634] The server generates a response message according to the user's emotion.

[0635] Input: Sentiment analysis results

[0636] Data processing: Generate appropriate response messages

[0637] Output: Response message data

[0638] Specific operation: Generate a response message from a template based on the analysis results.

[0639] Step 4:

[0640] The server sends the generated response message to the terminal.

[0641] Input: Response message data

[0642] Data processing: Converts message data into packets according to the communication protocol

[0643] Output: Data packets sent to the device

[0644] Specific operation: Sends response message data to the terminal via an HTTPS request.

[0645] Step 5:

[0646] The terminal displays the response message received from the server to the user.

[0647] Input: Response message data received from the server

[0648] Data processing: Convert response message data into a display format

[0649] Output: The response message that is displayed to the user

[0650] Specific operation: The device renders and displays the received text data on the screen.

[0651] Step 6:

[0652] The user may optionally enter additional messages to continue the dialogue.

[0653] Input: User's additional message

[0654] Data processing: Send additional messages to the server

[0655] Output: Additional message data to the server

[0656] Specific behavior: Sends the additional message to the server again as an HTTP POST request.

[0657] (Application example 2)

[0658] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0659] Although conventional digital picture books are available for young children, they are limited in terms of emotional support, and do not provide sufficient emotional support to children. Furthermore, in terms of learning support, the interaction required to obtain appropriate answers to questions is limited. This has led to issues such as not being able to fully improve children's creativity and learning efficiency.

[0660] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for providing a digital picture book, means for identifying the user's emotion using an emotion analysis engine, and means for generating a response message corresponding to the user's emotion. This allows children to enjoy the digital picture book, become emotionally stable through emotional support, and obtain appropriate answers to their questions, thereby improving learning efficiency.

[0661] A "digital picture book" is a picture book in electronic format that can be accessed by young children via computers or mobile devices.

[0662] A "server" is a computer system that stores and processes digital picture book data and user requests, and provides the necessary information.

[0663] An "emotion analysis engine" is a system that has algorithms and functions for analyzing messages from users and identifying their emotions.

[0664] A "response message" is a message generated based on the user's emotional state, intended to provide emotional support and appropriate responses.

[0665] A "question" is an inquiry that a user (child or student) inputs to the server regarding learning content, etc.

[0666] An "answer" is appropriate information or explanation that the server generates after analyzing the user's question.

[0667] "Interaction" refers to the process of operations and inputs that a user makes in order to interact with a system, and includes, in particular, actions such as turning pages, entering questions, and sending emotional messages.

[0668] An "audio file" is digital audio data used when reading picture books or playing audio guides.

[0669] A "prompt sentence" is an input sentence that allows a user to communicate with a system, and is text that includes specific instructions or questions.

[0670] A "generative AI model" is an artificial intelligence algorithm used to realize functions such as reading digital picture books, learning support, and emotion analysis.

[0671] MODE FOR CARRYING OUT THE INVENTION

[0672] This system provides digital picture books to young children, helps them to understand the content they are learning deeply, and provides emotional support through emotional support. The system is structured through interactions between a server, terminals, and users.

[0673] Providing digital picture books

[0674] server

[0675] The server maintains a database of digital picture books. Upon receiving a request from a user, the server retrieves the corresponding digital picture book data and transmits it to the terminal.

[0676] Terminal

[0677] The device displays the digital picture book data received from the server to the user in an interactive interface, detects the user's taps and swipes, turns the pages accordingly, and plays audio files in response to the user's instructions.

[0678] User

[0679] Users can select the picture book they want to read from the options on the screen, tap or swipe the screen to turn the pages and enjoy the story. Tap on a character or scene to play an audio guide.

[0680] Learning support features

[0681] server

[0682] The server receives a question from the user, analyzes its content, retrieves appropriate learning materials from a database, generates an answer, and sends the answer to the device.

[0683] Terminal

[0684] The terminal sends the question entered by the user to the server, receives the answer from the server and displays it to the user, and if the user enters another question, sends the additional question to the server again.

[0685] User

[0686] Users can enter questions about content they did not understand in class or at a cram school, and check the answers displayed on the device. They can enter additional questions as needed until they are satisfied.

[0687] Emotional Support and Sentiment Analysis Engine

[0688] server

[0689] The server receives the consultation content sent by the user. It analyzes the message using an emotion analysis engine to identify the user's emotion. It generates an appropriate response message corresponding to the user's emotion and sends it to the device. The server also accumulates the user's past emotion data to improve the accuracy of future response messages.

[0690] Terminal

[0691] The terminal sends the worries or consultation details entered by the user to the server. It receives a response message from the server and displays it to the user. If the user wants to enter another message, it sends that message to the server again.

[0692] User

[0693] The user inputs what is bothering them or what is making them feel lonely, and checks the response message displayed on the terminal. If necessary, the user can input an additional message to continue the conversation.

[0694] System specific example

[0695] For example, if a user types, "I feel lonely because I haven't made any friends recently," the system's emotion analysis engine will identify the user's emotion as "loneliness." The server will then generate a response message such as, "That's sad. But you have a chance to make new friends. Maybe you should try looking for people who have common hobbies," and send it to the device.

[0696] Prompt Sentence Examples

[0697] Digital Picture Book ID: 12345

[0698] Question: "What is water made of?"

[0699] Emotional support: "I haven't been able to make any friends lately and I feel lonely."

[0700] This system uses an emotion analysis engine and generative AI model to provide quick and appropriate responses to users' emotions and questions. Users can input prompts and use functions such as digital picture book viewing, learning support, and emotional support in response. This is expected to improve children's creativity and learning efficiency.

[0701] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0702] Step 1:

[0703] The user selects a digital picture book

[0704] The user selects the digital picture book they want to read from the options displayed on the device screen. The ID of the selected picture book is entered into the device. Based on this input, the device sends a request for the picture book data to the server.

[0705] Step 2:

[0706] The server sends the digital picture book data

[0707] When the server receives the request, it retrieves the digital picture book data from the database and sends the retrieved picture book data to the terminal as output.

[0708] Step 3:

[0709] The device displays the digital picture book.

[0710] The terminal generates an interactive user interface based on the digital picture book data received from the server, and displays it so that the user can manually turn pages and play audio.

[0711] Step 4:

[0712] User enters study question

[0713] The user inputs a question related to the learning content into the terminal, which then transmits the input question to the server.

[0714] Step 5:

[0715] The server analyzes the question and generates the answer

[0716] The server analyzes the question sent by the user, retrieves appropriate learning materials from the database, and generates an answer, which is then sent to the terminal as output.

[0717] Step 6:

[0718] The device displays the answer

[0719] The terminal displays the answer received from the server to the user, who can then confirm the displayed answer and enter a follow-up question if necessary.

[0720] Step 7:

[0721] The user inputs emotional consultation content.

[0722] The user inputs emotional concerns such as worries and loneliness into the terminal, and the inputted concerns are sent to the server by the terminal.

[0723] Step 8:

[0724] The server analyzes the emotions and generates a response message.

[0725] The server analyzes the consultation content sent by the user using an emotion analysis engine to identify the emotion. Based on the identified emotion, it generates an appropriate response message. The generated response message is sent as output to the terminal.

[0726] Step 9:

[0727] The terminal displays a response message to the user.

[0728] The terminal displays the response message received from the server to the user. The user confirms the displayed response message and continues the dialogue by inputting a message again if necessary.

[0729] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0730] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0731] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0732] [Second embodiment]

[0733] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0734] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0735] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0736] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0737] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0738] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0739] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0740] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0741] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0742] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0743] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0744] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0745] This invention is a system that provides digital picture books to young children, helps children and students to deeply understand the content they are learning, and provides emotional support when they feel worried or lonely.

[0746] Providing digital picture books

[0747] server

[0748] 1. Maintain a database of digital picture books.

[0749] 2. When a request is received from the user (child), the corresponding digital picture book data is obtained and sent to the terminal.

[0750] Terminal

[0751] 1. Based on the digital picture book data received from the server, it is displayed to the user in an interactive interface.

[0752] 2. Turn pages in response to user taps and swipes.

[0753] 3. Play the audio file based on the user's instructions.

[0754] User

[0755] 1. Select the picture book you want to read from the options on the screen.

[0756] 2. Enjoy the content by tapping or swiping the screen to turn the pages.

[0757] 3. Tap to play audio descriptions related to characters and scenes.

[0758] Learning support features

[0759] server

[0760] 1. Receive a question from the user and analyze its content.

[0761] 2. Retrieve appropriate learning materials from the database and generate answers.

[0762] 3. Send the generated answer to the device.

[0763] Terminal

[0764] 1. The question entered by the user is sent to the server.

[0765] 2. Receive the response from the server and display it to the user.

[0766] 3. If the user enters additional questions, the additional questions are sent to the server again.

[0767] User

[0768] 1. Enter questions about content you didn't understand in class or at cram school.

[0769] 2. Check the answer displayed on the device.

[0770] 3. Enter additional questions as needed until you are satisfied.

[0771] Emotional support

[0772] server

[0773] 1. Receive a message from the user and analyze it using a sentiment analysis engine.

[0774] 2. Generate an appropriate response message that corresponds to the user's emotions.

[0775] 3. Send the generated response message to the terminal.

[0776] Terminal

[0777] 1. The user's concerns and questions are sent to the server.

[0778] 2. Receive the response message from the server and display it to the user.

[0779] 3. If the user enters a message again, the message is sent to the server again.

[0780] User

[0781] 1. Enter what is bothering you or making you feel lonely.

[0782] 2. Check the response message displayed on the device.

[0783] 3. Enter an additional message to continue the conversation, if desired.

[0784] For example, if a user inputs "I'm lonely because I haven't made any friends at school recently," the server will generate and send a response message such as "That's sad. But there are opportunities to make new friends. Do you have any favorite games or activities?" By repeating this kind of dialogue between the user and the server, the system helps to lighten the user's mood, even if only a little.

[0785] In this way, the educational and emotional support system that makes full use of digital technology aims to promote children's overall development, conveying the joy of picture books, deepening their understanding of learning, and alleviating feelings of loneliness.

[0786] The processing flow will be explained below.

[0787] Providing digital picture books

[0788] Server Processing

[0789] Step 1:

[0790] The server receives a request from a user.

[0791] Step 2:

[0792] The server retrieves the requested digital picture book data from the database.

[0793] Step 3:

[0794] The server transmits the acquired data to the terminal.

[0795] Terminal handling

[0796] Step 1:

[0797] The terminal receives the digital picture book data transmitted from the server.

[0798] Step 2:

[0799] The terminal analyzes the received data and generates an interactive interface.

[0800] Step 3:

[0801] The device detects the user's taps and swipes and performs page-turning processing accordingly.

[0802] Step 4:

[0803] The terminal plays the audio file in response to a user instruction.

[0804] User Actions

[0805] Step 1:

[0806] The user selects the picture book they want to read from the options on the screen.

[0807] Step 2:

[0808] Users tap or swipe the screen to turn pages.

[0809] Step 3:

[0810] The user can play audio guides by tapping on characters or scenes.

[0811] Learning support features

[0812] Server Processing

[0813] Step 1:

[0814] The server receives a question sent by a user.

[0815] Step 2:

[0816] The server analyzes the received question using an NLP (natural language processing) engine.

[0817] Step 3:

[0818] The server retrieves appropriate learning materials from a database and generates answers.

[0819] Step 4:

[0820] The server sends the generated response to the terminal.

[0821] Terminal handling

[0822] Step 1:

[0823] The terminal transmits the question entered by the user to the server.

[0824] Step 2:

[0825] The terminal receives the response from the server.

[0826] Step 3:

[0827] The terminal displays the received response to the user.

[0828] Step 4:

[0829] If the user inputs more questions, the terminal transmits the additional questions to the server again.

[0830] User Actions

[0831] Step 1:

[0832] The user inputs questions about content that he or she did not understand in class or at a cram school.

[0833] Step 2:

[0834] The user checks the answer displayed on the terminal.

[0835] Step 3:

[0836] The user continues to enter additional questions until satisfied.

[0837] Emotional support

[0838] Server Processing

[0839] Step 1:

[0840] The server receives the consultation content sent from the user.

[0841] Step 2:

[0842] The server analyzes the received message using an emotion analysis engine.

[0843] Step 3:

[0844] The server generates an appropriate response message according to the user's emotions.

[0845] Step 4:

[0846] The server sends the generated response message to the terminal.

[0847] Terminal handling

[0848] Step 1:

[0849] The terminal transmits the worries and consultation details input by the user to the server.

[0850] Step 2:

[0851] The terminal receives the response message from the server.

[0852] Step 3:

[0853] The terminal displays the received response message to the user.

[0854] Step 4:

[0855] If the user inputs a message again, the terminal transmits the additional message to the server again.

[0856] User Actions

[0857] Step 1:

[0858] The user inputs what is bothering them or making them feel lonely.

[0859] Step 2:

[0860] The user checks the response message displayed on the terminal.

[0861] Step 3:

[0862] The user may optionally enter additional messages to continue the dialogue.

[0863] Example 1

[0864] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0865] Although today's children are familiar with digital devices, there is a lack of effective digital picture books and learning support systems. Furthermore, it is difficult to provide appropriate support when children need emotional support. This invention proposes a system that provides digital picture books, learning support, and emotional support in an integrated manner, thereby solving these issues.

[0866] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0867] In this invention, the server includes means for providing digital picture books to young children, means for displaying digital picture book data received from the server on a terminal, means for turning pages in response to tapping or swiping operations by the user, and means for playing audio files in response to user instructions, thereby enabling children to learn interactively while enjoying the digital picture books.

[0868] In addition, the learning support function is enhanced by adding a means for children and students to input questions, a means for sending the input questions to a server, a means for the server to analyze the questions and generate appropriate answers, a means for sending the generated answers to the user, and a means for the user to input further questions.

[0869] Furthermore, the server includes means for receiving a message based on the user's emotions, means for analyzing the received message using an emotion analysis engine, means for generating a response message according to the user's emotions, means for sending the generated response message to the user, and means for the user to input another message and continue the dialogue, thereby making it possible to provide emotional support.

[0870] In this way, the present invention can provide an integrated system that comprehensively provides education and emotional support for children.

[0871] "Young children" generally refers to children from toddlers to early elementary school age.

[0872] A "digital picture book" is a picture book displayed on a computer or mobile device and is an electronic book that includes multimedia elements such as text, images, and audio.

[0873] A "server" refers to a computer system that has the function of receiving requests from users, processing data, and sending it to terminals.

[0874] "Database" refers to a data storage system for centrally managing and storing digital picture book data and learning materials.

[0875] An "interactive interface" refers to a user interface in which the content on the screen responds to user operations, enabling two-way interaction.

[0876] "Tap" refers to the act of lightly tapping with a fingertip on a touchscreen device.

[0877] "Swipe" refers to the action of sliding your finger across a touchscreen device.

[0878] "Audio file" refers to a file that stores audio data in digital format.

[0879] A "question" refers to text that a user enters about something they don't understand or about information they want to confirm.

[0880] A "natural language processing engine" refers to a computer program that analyzes text data and understands its meaning and content.

[0881] A "generative AI model" refers to a trained algorithm that uses artificial intelligence to generate text and voice data.

[0882] An "emotion analysis engine" refers to technology for identifying and analyzing user emotions from text data.

[0883] A "response message" refers to a message generated as a reply or response to a message received from a user.

[0884] "Means for continuing a dialogue" refers to a method for continuing to exchange messages between a user and a system.

[0885] This invention is a system that provides digital picture books to young children, supports them in understanding the learning content deeply, and also provides emotional support. A specific implementation method is shown below.

[0886] Providing digital picture books

[0887] server

[0888] The server maintains a database of digital picture books. This database stores image data, text data, audio files, etc. of picture books. When a request from a user (child) reaches the server via a terminal, the server searches the database for the relevant picture book data and retrieves it. The retrieved data is then converted into an appropriate format and sent to the terminal.

[0889] Terminal

[0890] The device displays the picture book data received from the server in an interactive interface. The received data is decoded, and the image data is displayed in an image viewer, and the text data is displayed in a text viewer. The user can turn pages or play character audio guides by tapping or swiping the screen.

[0891] User

[0892] The user selects the picture book they want to read from the list displayed on the device screen and taps or swipes the screen to view the contents. For example, if the user selects the picture book "Little Red Riding Hood," the server retrieves Little Red Riding Hood's data from the database and sends it to the device. Each time the user taps, the page turns, and an audio guide is played if necessary.

[0893] Learning support features

[0894] server

[0895] The server receives questions from users and analyzes them using a natural language processing engine. Based on the analyzed questions, it retrieves appropriate learning materials from a database and generates the optimal answer using a generative AI model. The generated answer is then sent to the device.

[0896] Terminal

[0897] The device sends the question entered by the user to the server, and the answer received from the server is displayed on the interface. For example, if a user enters "I don't know how to calculate the area of ​​a triangle," the server will generate an answer such as "The area of ​​a triangle can be calculated by multiplying the length of the base by the height and dividing by 2." The user can enter additional information until they are satisfied.

[0898] User

[0899] Users can enter questions into a text field on the device, see the answers displayed, and optionally enter additional questions to seek further answers.

[0900] Emotional support

[0901] server

[0902] The server receives emotion-based messages from users, analyzes them using an emotion analysis engine, and generates an appropriate response message using a generative AI model based on the analysis results, which is then sent to the device.

[0903] Terminal

[0904] The device sends the worries and concerns entered by the user to the server and displays the response message received from the server. For example, if the user enters "I'm lonely because I haven't made any friends at school recently," the server will generate and send a response message such as "That's sad. But there are opportunities to make new friends. Do you have any favorite games or activities?"

[0905] User

[0906] The user inputs their concerns or worries into the terminal, checks the response message that is displayed, and continues the dialogue by inputting the message again if necessary.

[0907] This invention provides an environment where children can grow and have fun through learning using digital devices by providing digital picture books, learning support, and emotional support in an integrated manner.

[0908] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0909] Providing digital picture books

[0910] Step 1:

[0911] server

[0912] The server maintains a database of digital picture books. The database stores image data, text data, audio files, etc. of picture books. When the server starts up, it reads this data and prepares it. Input: All data in the database. Output: Prepared digital picture book data.

[0913] Step 2:

[0914] server

[0915] The server receives a digital picture book request from the user (child). The request includes the user ID and the title or ID of the picture book they want to read. Input: Request from the user. Output: Identification information of the requested picture book.

[0916] Step 3:

[0917] server

[0918] The server searches and retrieves the corresponding digital picture book data from the database based on the identification information. Data is extracted efficiently using SQL queries. Input: Identification information. Output: Retrieved picture book data.

[0919] Step 4:

[0920] server

[0921] The server converts the acquired picture book data into an appropriate format such as JSON and sends it to the terminal. This is where the data is encoded. Input: Acquired picture book data. Output: Formatted picture book data.

[0922] Step 5:

[0923] Terminal

[0924] The device receives the digital picture book data sent from the server. The received data is encoded in JSON format or similar. Input: Data sent from the server. Output: Received encoded data.

[0925] Step 6:

[0926] Terminal

[0927] The terminal decodes the data it receives and converts it into a format that can be displayed in the interface. Image data is displayed in an image viewer, text in a text viewer. Input: The encoded data. Output: The data that can be displayed in the interface.

[0928] Step 7:

[0929] User

[0930] The user selects the book they want to read from the list of digital books displayed on the device screen and confirms the request. They then enjoy the content by turning the pages using on-screen operations (tapping and swiping). Input: Device operations (tapping and swiping). Output: Display of pages and playback of audio guide.

[0931] Learning support features

[0932] Step 1:

[0933] User

[0934] The user types a question into a text field on the device. The input is related to a learning topic that they do not understand. Input: Learning question. Output: The entered question text.

[0935] Step 2:

[0936] Terminal

[0937] The terminal sends the question entered by the user to the server. Input: Question from the user. Output: Question text sent to the server.

[0938] Step 3:

[0939] server

[0940] The server receives the question and analyzes the question using a natural language processing engine. Input: Received question text. Output: Analyzed question content.

[0941] Step 4:

[0942] server

[0943] The server retrieves appropriate learning materials from the database based on the analyzed question. It then uses a generative AI model to generate the optimal answer. Input: Analyzed question. Output: Generated answer text.

[0944] Step 5:

[0945] server

[0946] The server sends the generated answer to the terminal. The answer is encoded in JSON format or similar. Input: Generated answer text. Output: Encoded answer data.

[0947] Step 6:

[0948] Terminal

[0949] The terminal receives the response sent from the server and decodes it into a format that can be displayed. Input: Response data from the server. Output: Displayable response text.

[0950] Step 7:

[0951] User

[0952] The user checks the answer displayed on the terminal, and if they are not satisfied, they can enter an additional question. Input: Answer text, additional question. Output: New question text.

[0953] Emotional support

[0954] Step 1:

[0955] User

[0956] The user inputs the problem or problem content into the text field of the terminal. Input: Problem or problem content. Output: Input text message.

[0957] Step 2:

[0958] Terminal

[0959] The terminal sends the message entered by the user to the server. Input: Text message from the user. Output: Message sent to the server.

[0960] Step 3:

[0961] server

[0962] The server analyzes the received message with a sentiment analysis engine to identify the user's sentiment. Input: Received text message. Output: Identified sentiment data.

[0963] Step 4:

[0964] server

[0965] The server generates an appropriate response message using a generative AI model based on the identified emotion data. Input: Identified emotion data. Output: Generated response message.

[0966] Step 5:

[0967] server

[0968] The server sends the generated response message to the terminal. The response message is sent in text format. Input: Generated response message. Output: Response message sent to the terminal.

[0969] Step 6:

[0970] Terminal

[0971] The terminal displays the response message received from the server to the user. Input: Received response message. Output: Displayed response message.

[0972] Step 7:

[0973] User

[0974] The user checks the response message displayed on the terminal and continues the dialogue by inputting another message if necessary. Input: Input of a message on the terminal. Output: A new message to the server.

[0975] In this system, various data exchanges and processing are carried out between the server, terminals, and users to effectively realize the functions of providing digital picture books, learning support, and emotional support.

[0976] (Application example 1)

[0977] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0978] To support the learning and emotional development of young children, there is a need for systems that provide a consistent approach to reading picture books, learning support, and emotional support. However, no existing systems offer these multiple functions in an integrated manner; most are specialized for individual purposes. Therefore, there is a need for a comprehensive system that can maintain children's interest, deepen their understanding of learning, and provide emotional support in educational settings such as cram schools.

[0979] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0980] In this invention, the server includes means for providing digital picture books to young children, means for displaying digital picture book data received from the server, means for turning pages in response to user operations, means for playing audio files, means for children at cram schools to input questions using smartphones and send them to the server, means for displaying answers received from the server, and means for analyzing the user's emotions and generating and sending appropriate response messages. This makes it possible to provide appropriate answers to learning questions and emotional support while maintaining children's interest through the provision of digital picture books.

[0981] "Young children" refers primarily to young people from preschool to early elementary school age, an age group that is particularly interested in digital picture books and learning support.

[0982] A "digital picture book" is a picture book that is provided in digital format rather than on paper and can be viewed on an electronic device.

[0983] A "server" is a computer system that centrally manages digital picture book data, learning answers, emotion analysis messages, etc., and communicates with user terminals.

[0984] "User operation" refers to input actions that a user performs on the system through a terminal, such as tapping or swiping on the screen.

[0985] The "means for turning pages" is a mechanism that provides a function for turning pages of a digital picture book forward or backward in response to user operations.

[0986] "Audio file" refers to audio recording data, which is used to provide audio readings and explanations of digital picture books.

[0987] A "cram school" is an educational facility where children can receive learning support outside of class.

[0988] "Means for inputting questions" refers to the function that allows users to input questions about their studies as text information into the terminal.

[0989] "Sentiment analysis" refers to a technology that analyzes text data from a user and identifies the emotions the user is feeling.

[0990] A "response message" is a message that is generated based on the results of emotion analysis and sent to the user.

[0991] "Display means" refers to the function for displaying information such as digital picture book data, answers, and response messages sent from the server on the screen of the user's terminal.

[0992] This invention is a system that allows young children to learn while having fun with digital picture books and also receive emotional support. This system consists of a server, a terminal operated by the user, and the user. The configuration and operation of the system are described in detail below.

[0993] Providing digital picture books

[0994] server

[0995] The server maintains a database of digital picture books and is responsible for transmitting the corresponding digital picture book data when requested by a user (young child). The server is equipped with functions for efficiently managing and distributing the metadata and content of digital picture books.

[0996] Terminal

[0997] The device operated by the user (smartphone or tablet) receives the digital picture book data from the server and displays it in an interactive interface, allowing the user to turn pages by tapping or swiping the screen and play audio guides related to characters and scenes.

[0998] User

[0999] Users select the picture book they want to read from the options on the screen, advance through the pages through operations, and enjoy the contents of the picture book using audio guidance.

[1000] Learning support features

[1001] server

[1002] The server receives a question from the user, analyzes the question, retrieves appropriate learning materials from a database, and generates an answer, which is then sent to the terminal.

[1003] Terminal

[1004] The terminal transmits the questions entered by the user to the server, receives and displays the answers from the server, and if the user enters additional questions, the additional questions can be transmitted to the server again.

[1005] User

[1006] Users can enter questions about content they did not understand in class or at a cram school into the device, check the answers provided, and enter additional questions as needed.

[1007] Emotional support

[1008] server

[1009] The server receives messages from users, analyzes their emotions using AI, and generates an appropriate response message based on the analysis results, which is then sent to the device.

[1010] Terminal

[1011] The terminal inputs the user's worries or feelings of loneliness and sends them to the server. It receives and displays a response message from the server, and if the user inputs another message, it sends that message to the server again.

[1012] User

[1013] The user inputs their concerns or feelings of loneliness into the terminal, confirms the response message provided, and inputs an additional message to continue the dialogue if necessary.

[1014] Hardware and software used

[1015] Hardware used

[1016] Smartphones and tablets: Used by users as portable devices.

[1017] Server: A central management system that manages the digital picture book database and AI responses.

[1018] Software used

[1019] Flutter / React Native: Used to build and operate the UI of smartphone apps.

[1020] requests (Python library): A library for enabling communication between the server and the terminal.

[1021] Generative AI model: AI technology used for sentiment analysis and response message generation.

[1022] Examples and prompts

[1023] As a specific example, if the user selects the digital picture book "Forest Stories," the terminal operates as follows.

[1024] 1. The user selects the "Forest Tales" digital picture book.

[1025] 2. The terminal sends a request to the server, receives the digital picture book data from the server, and displays it.

[1026] 3. When the user taps to turn the page, the appropriate page is displayed and the associated audio plays.

[1027] Example prompt sentence:

[1028] Get and display the page data for the digital picture book "Forest Stories." URL: https: / / example.com / digital_books / 001

[1029] In this way, the present invention provides a comprehensive system that provides relevant answers to educational questions and emotional support while retaining the interest of young children.

[1030] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1031] Step 1:

[1032] The user uses the terminal to make a digital picture book request.

[1033] Input: A request for a digital picture book based on the user's selection

[1034] Output: Request sent to server

[1035] How it works: The user selects a picture book, such as "Forest Stories," from the interface on the device, and the information is sent to the server.

[1036] Step 2:

[1037] The server receives the request, acquires the corresponding digital picture book data, and transmits it.

[1038] Input: Digital picture book request from user device

[1039] Output: Sending digital picture book data

[1040] Operation: The server retrieves the requested picture book data from the digital picture book database and sends it to the user's terminal.

[1041] Step 3:

[1042] The terminal displays the digital picture book data received from the server.

[1043] Input: Digital picture book data from the server

[1044] Output: Display a digital picture book on the screen

[1045] Operation: The device displays the received digital picture book data on an interactive interface, allowing the user to turn pages and play audio.

[1046] Step 4:

[1047] The user inputs a question related to the study and sends it to the server via the terminal.

[1048] Input: User's question

[1049] Output: Sending the query data to the server

[1050] How it works: The user enters a study question into the terminal, and the data is sent to the server.

[1051] Step 5:

[1052] The server analyzes the received question, retrieves appropriate learning materials from a database, and generates an answer.

[1053] Input: Question data from the user

[1054] Output: Generated training answers

[1055] How it works: The server analyzes the received question, retrieves relevant learning material from the database, and generates an appropriate answer.

[1056] Step 6:

[1057] The terminal displays the answer received from the server to the user.

[1058] Input: Learning response data from the server

[1059] Output: Display the answer to the user

[1060] Operation: The terminal displays the answer received from the server to the user, who then confirms the content.

[1061] Step 7:

[1062] If the user enters more questions, the questions are sent to the server again.

[1063] Input: User's additional question

[1064] Output: Sending additional questions to the server

[1065] How it works: The user enters a follow-up question into the terminal, and the data is sent back to the server.

[1066] Step 8:

[1067] The user inputs the emotional worries and consultation contents and transmits them to the server.

[1068] Input: User's sentiment message

[1069] Output: Sending emotion message to the server

[1070] How it works: The user enters a message about their worries or loneliness into the device, and the data is sent to the server.

[1071] Step 9:

[1072] The server analyzes the sentiment of the received message and generates an appropriate response message.

[1073] Input: Emotional message from the user

[1074] Output: The generated response message

[1075] How it works: The server analyzes the received message using an AI model and generates an appropriate emotional response message.

[1076] Step 10:

[1077] The terminal displays the response message received from the server to the user.

[1078] Input: Response message from the server

[1079] Output: Display a response message to the user

[1080] Operation: The terminal displays the received response message to the user, who confirms it.

[1081] Step 11:

[1082] The user inputs a message again to continue the dialogue and sends it to the server.

[1083] Input: User's additional emotion message

[1084] Output: Send additional emotion message to server

[1085] Operation: The user enters a message into the terminal to continue the dialogue, and the data is sent again to the server.

[1086] In this way, the present invention provides a system that supports the user's learning and also provides emotional support at the same time.

[1087] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1088] This invention is a system that provides digital picture books to young children, supports children and students in gaining a deeper understanding of the learning content, and provides emotional support when they feel distressed or lonely. By combining a specific emotion engine, the system recognizes the user's emotions and provides appropriate responses accordingly.

[1089] Providing digital picture books

[1090] server

[1091] 1. Maintain a database of digital picture books.

[1092] 2. When a request is received from the user (child), the corresponding digital picture book data is obtained and sent to the terminal.

[1093] Terminal

[1094] 1. Based on the digital picture book data received from the server, it is displayed to the user in an interactive interface.

[1095] 2. Detect user taps and swipes and perform page-turning processing accordingly.

[1096] 3. Play the audio file according to the user's instructions.

[1097] User

[1098] 1. Select the picture book you want to read from the options on the screen.

[1099] 2. Enjoy the content by tapping or swiping the screen to turn the pages.

[1100] 3. Tap to play audio descriptions related to characters and scenes.

[1101] Learning support features

[1102] server

[1103] 1. Receive a question from the user and analyze its content.

[1104] 2. Retrieve appropriate learning materials from the database and generate answers.

[1105] 3. Send the generated answer to the device.

[1106] Terminal

[1107] 1. The question entered by the user is sent to the server.

[1108] 2. Receive the response from the server and display it to the user.

[1109] 3. If the user enters additional questions, the additional questions are sent to the server again.

[1110] User

[1111] 1. Enter questions about content you didn't understand in class or at cram school.

[1112] 2. Check the answer displayed on the device.

[1113] 3. Enter additional questions as needed until you are satisfied.

[1114] Emotional Support and Emotion Engine

[1115] server

[1116] 1. Receive the consultation content sent by the user.

[1117] 2. The message is analyzed using a sentiment analysis engine to identify the user's sentiment.

[1118] 3. Generate an appropriate response message that corresponds to the user's emotions.

[1119] 4. Send the generated response message to the terminal.

[1120] 5. Accumulates user's past emotional data to improve the accuracy of future response messages.

[1121] Terminal

[1122] 1. The user's concerns and questions are sent to the server.

[1123] 2. Receive the response message from the server and display it to the user.

[1124] 3. If the user enters a message again, the message is sent to the server again.

[1125] User

[1126] 1. Enter what is bothering you or making you feel lonely.

[1127] 2. Check the response message displayed on the device.

[1128] 3. If necessary, enter an additional message to continue the conversation.

[1129] For example, if a user types, "I feel lonely because I haven't made any friends recently," the server's emotion analysis engine will identify the user's emotion as "loneliness." The server will then generate a response message such as, "That's tough. But you have a chance to make new friends. Maybe you should try looking for someone who has a common hobby with you," and send it to the device. If the user receives this message and continues the dialogue, it can alleviate the user's feelings of loneliness and worries.

[1130] The introduction of this system can improve children's creativity, learning efficiency, and emotional support, promoting their overall development.

[1131] The processing flow will be explained below.

[1132] Providing digital picture books

[1133] Server Processing

[1134] Step 1:

[1135] The server receives a request from a user.

[1136] Step 2:

[1137] The server retrieves the requested digital picture book data from the database.

[1138] Step 3:

[1139] The server transmits the acquired data to the terminal.

[1140] Terminal handling

[1141] Step 1:

[1142] The terminal receives the digital picture book data transmitted from the server.

[1143] Step 2:

[1144] The terminal analyzes the received data and generates an interactive interface.

[1145] Step 3:

[1146] The device detects the user's taps and swipes and performs page-turning processing accordingly.

[1147] Step 4:

[1148] The terminal plays the audio file in response to a user instruction.

[1149] User Actions

[1150] Step 1:

[1151] The user selects the picture book they want to read from the options on the screen.

[1152] Step 2:

[1153] Users tap or swipe the screen to turn pages.

[1154] Step 3:

[1155] The user can play audio guides by tapping on characters or scenes.

[1156] Learning support features

[1157] Server Processing

[1158] Step 1:

[1159] The server receives a question sent by a user.

[1160] Step 2:

[1161] The server analyzes the received question using an NLP (natural language processing) engine.

[1162] Step 3:

[1163] The server retrieves appropriate learning materials from a database and generates answers.

[1164] Step 4:

[1165] The server sends the generated response to the terminal.

[1166] Terminal handling

[1167] Step 1:

[1168] The terminal transmits the question entered by the user to the server.

[1169] Step 2:

[1170] The terminal receives the response from the server.

[1171] Step 3:

[1172] The terminal displays the received response to the user.

[1173] Step 4:

[1174] If the user inputs more questions, the terminal transmits the additional questions to the server again.

[1175] User Actions

[1176] Step 1:

[1177] The user inputs questions about content that he or she did not understand in class or at a cram school.

[1178] Step 2:

[1179] The user checks the answer displayed on the terminal.

[1180] Step 3:

[1181] The user continues to enter additional questions until satisfied.

[1182] Emotional Support and Emotion Engine

[1183] Server Processing

[1184] Step 1:

[1185] The server receives the consultation content sent from the user.

[1186] Step 2:

[1187] The server analyzes the received message using an emotion analysis engine to identify the user's emotion.

[1188] Step 3:

[1189] The server generates an appropriate response message according to the user's emotions based on the results of the emotion analysis.

[1190] Step 4:

[1191] The server sends the generated response message to the terminal.

[1192] Step 5:

[1193] The server accumulates the user's past emotional data and improves the accuracy of future response messages.

[1194] Terminal handling

[1195] Step 1:

[1196] The terminal transmits the worries and consultation details input by the user to the server.

[1197] Step 2:

[1198] The terminal receives the response message from the server.

[1199] Step 3:

[1200] The terminal displays the received response message to the user.

[1201] Step 4:

[1202] If the user inputs a message again, the terminal transmits the additional message to the server again.

[1203] User Actions

[1204] Step 1:

[1205] The user inputs what is bothering them or making them feel lonely.

[1206] Step 2:

[1207] The user checks the response message displayed on the terminal.

[1208] Step 3:

[1209] The user may optionally enter additional messages to continue the dialogue.

[1210] Example 2

[1211] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1212] In today's educational environment, there is a need for digital picture books that allow young children to deepen their knowledge while having fun learning, and for methods that encourage students to ask questions about the content they are learning and deepen their understanding. It is also important to provide appropriate emotional support when children feel distressed or lonely. There is a lack of systems that can meet these needs and promote children's holistic development.

[1213] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1214] In this invention, the server includes an information transmission device means for storing data, a display device means for displaying data received from the information transmission device, a device means for switching pages in response to operations, and a device means for playing audio data, which enables provision of digital picture books, understanding of learning content, and emotional support.

[1215] "Data" means electronic records of specific information or content.

[1216] An "information transmitting device" refers to a device that holds data and transmits it to a terminal or display device.

[1217] A "display device" is a device for visually displaying received data to a user.

[1218] An "operation" refers to an input action that a user performs using a terminal or a display device.

[1219] A "page-switching device" is a device that has the function of turning pages of a digital picture book or the like in response to a user's operation.

[1220] "Audio data" means electronic recordings of audio information.

[1221] A "device that plays back audio data" is a device that has the function of receiving and playing back audio data.

[1222] A "question input device" is a device that allows a user to input their own questions or doubts.

[1223] The "device for additional input" refers to a device that allows the user to input information again.

[1224] An "analyzing device" is a device that has the function of analyzing an input question or message and generating an appropriate answer or response.

[1225] An "emotion-based message" is a message whose content reflects the user's emotional state.

[1226] A "response message" is a response generated based on the analyzed content.

[1227] The "display device" refers to a device that displays the generated answer or response message in an easy-to-see manner for the user.

[1228] The present invention is a system that provides digital picture books to young children, supports children and students in gaining a deeper understanding of the learning content, and provides emotional support when they feel distressed or lonely. The following describes specific embodiments of the invention.

[1229] Providing digital picture books

[1230] server

[1231] The server maintains a database of digital picture books. When it receives a request from a user (child), it retrieves the corresponding digital picture book data from the database and sends it to the device. For this reason, the server has high-performance database software (e.g., MySQL) and data transmission functions.

[1232] Terminal

[1233] The terminal receives the digital picture book data from the server and displays it to the user using an interactive interface. It is typically a tablet or smartphone, and is equipped with an LCD display or touchscreen. The terminal detects the user's taps and swipes and processes the pages accordingly. It also plays audio data in response to user instructions. This is achieved using a built-in speaker or earphones.

[1234] User

[1235] When reading a digital picture book, users select the book they want from a list on the screen. They turn the pages by tapping or swiping, and when they tap on parts related to characters or scenes, audio guides and sound effects are played.

[1236] Learning support features

[1237] server

[1238] The server receives a question from the user and analyzes it using a natural language processing engine (e.g., Google Natural Language API). It retrieves appropriate learning materials from a database and generates an answer. The server is equipped with data analysis and transmission functions to send the generated answer to the device.

[1239] Terminal

[1240] The terminal sends the question entered by the user to the server, receives the answer from the server and displays it to the user, and if the user enters another question, sends the additional question back to the server and displays related additional learning materials.

[1241] User

[1242] Users can enter questions about things they didn't understand in class or at a cram school. They can check the answers and explanations displayed on the device to deepen their understanding. If necessary, they can enter further questions and repeat the process until they are satisfied.

[1243] Emotional Support and Emotion Engine

[1244] server

[1245] The server receives the consultation content sent by the user. It analyzes the message using an emotion analysis engine (e.g., IBM Watson Tone Analyzer) to identify the user's emotion. It generates a response message based on the user's emotion and sends it to the device. It also accumulates data on the user's past emotions to improve accuracy.

[1246] Terminal

[1247] The terminal sends the worries or consultation details entered by the user to the server. It receives a response message from the server and displays it to the user. If the user wants to enter a message again, it can send that message to the server again.

[1248] User

[1249] The user inputs what is bothering them or what is making them feel lonely. They check the response message displayed on the terminal and, if necessary, input an additional message to continue the conversation.

[1250] Specific examples and examples of input to generative AI models

[1251] For example, if a user types, "I'm lonely because I haven't made any friends recently," the server will use its emotion analysis engine to identify the user's emotion as "loneliness." It will then generate a response message such as, "That's tough. But you have a chance to make new friends. Maybe you should try looking for someone who has common hobbies with you," and send it to the device. By receiving this message and continuing the dialogue, the user can ease their feelings of loneliness and distress.

[1252] Example of input prompt for generative AI model

[1253] 1. "I'm lonely and can't make friends. Please tell me what to do."

[1254] 2. "I'm having trouble understanding some of the things I'm studying at school. Can you explain?"

[1255] 3. "The picture book I want to read right now is one with lots of animals in it."

[1256] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1257] Providing digital picture books

[1258] Step 1:

[1259] The server maintains a database of digital picture books, which stores data on a plurality of digital picture books.

[1260] Input: A digital picture book request from a user

[1261] Data processing: Analyze user requests and identify the corresponding digital picture book data

[1262] Output: Identified digital picture book data

[1263] Specific operation: The server receives the HTTP request, executes an SQL query based on it, and retrieves the required data from the database.

[1264] Step 2:

[1265] The server transmits the corresponding digital picture book data to the terminal.

[1266] Input: Identified digital picture book data

[1267] Data processing: Digital picture book data is converted into packets according to communication protocols (e.g. HTTP, HTTPS).

[1268] Output: Data packets sent to the device

[1269] Specific operation: The server sends the digital picture book data encoded using HTTPS to the terminal.

[1270] Step 3:

[1271] The terminal displays the digital picture book data received from the server to the user on an interface.

[1272] Input: Digital picture book data received from the server

[1273] Data processing: Analyzes received data and converts it into a display format

[1274] Output: A digital picture book displayed in a form visible to the user

[1275] Specific operation: The device analyzes the images and text data of the digital picture book and displays them on the screen.

[1276] Step 4:

[1277] The device detects the user's taps and swipes and performs page-turning processing accordingly.

[1278] Input: User taps and swipes

[1279] Data processing: Analyzes the operation data and converts the data on the next page into a display format

[1280] Output: Next page displayed

[1281] Specific operation: The gesture engine analyzes touch operations and executes page transition animations.

[1282] Step 5:

[1283] The terminal plays the audio file in response to a user instruction.

[1284] Input: User's voice instructions (tap operation, etc.)

[1285] Data processing: Read and play audio files based on instructions

[1286] Output: Audio is played

[1287] Specific behavior: The device loads the audio file from storage and plays it using the media player.

[1288] Learning support features

[1289] Step 1:

[1290] The server receives a query from the user.

[1291] Input: The question the user types into the terminal

[1292] Data processing: Transfer received questions to the server

[1293] Output: Query data to the server

[1294] Specific operation: Sends the question data to the server as an HTTP POST request.

[1295] Step 2:

[1296] The server analyzes the received question using a natural language processing engine.

[1297] Input: Received question data

[1298] Data processing: Analysis using a natural language processing engine

[1299] Output: Analysis results

[1300] Specific operation: The server uses the Google Natural Language API to analyze the question data and extract keywords and intent.

[1301] Step 3:

[1302] The server retrieves appropriate learning materials from a database and generates answers.

[1303] Input: Analysis results of natural language processing

[1304] Data processing: Run database queries to identify appropriate learning materials and generate answers.

[1305] Output: Generated response data

[1306] Specific operation: Executes an SQL query to retrieve teaching material data and generates answer sentences using a template.

[1307] Step 4:

[1308] The server sends the generated response to the terminal.

[1309] Input: Generated response data

[1310] Data processing: Converts response data into packets according to the communication protocol

[1311] Output: Data packets sent to the device

[1312] Specific operation: Sends response data to the terminal via an HTTPS request.

[1313] Step 5:

[1314] The terminal displays the response received from the server to the user.

[1315] Input: Response data received from the server

[1316] Data processing: Converting response data into a display format

[1317] Output: The answer that is displayed to the user

[1318] Specific operation: The device renders and displays the received text data on the screen.

[1319] Step 6:

[1320] If the user has further questions, they enter them.

[1321] Input: User's additional question

[1322] Data processing: Send additional questions to the server

[1323] Output: Additional question data to the server

[1324] Specific operation: Sends the additional question to the server again as an HTTP POST request.

[1325] Emotional Support and Emotion Engine

[1326] Step 1:

[1327] The server receives the consultation content from the user.

[1328] Input: The consultation content entered by the user into the terminal

[1329] Data processing: Transfer received consultation details to the server

[1330] Output: Consultation content data to the server

[1331] Specific operation: Consultation content data is sent to the server as an HTTP POST request.

[1332] Step 2:

[1333] The server analyzes the consultation content using an emotion analysis engine and identifies the user's emotions.

[1334] Input: Received consultation content data

[1335] Data processing: Analysis using a sentiment analysis engine

[1336] Output: Emotion analysis results

[1337] Specific operation: Uses IBM Watson Tone Analyzer to analyze text data and extract emotions.

[1338] Step 3:

[1339] The server generates a response message according to the user's emotion.

[1340] Input: Sentiment analysis results

[1341] Data processing: Generate appropriate response messages

[1342] Output: Response message data

[1343] Specific operation: Generate a response message from a template based on the analysis results.

[1344] Step 4:

[1345] The server sends the generated response message to the terminal.

[1346] Input: Response message data

[1347] Data processing: Converts message data into packets according to the communication protocol

[1348] Output: Data packets sent to the device

[1349] Specific operation: Sends response message data to the terminal via an HTTPS request.

[1350] Step 5:

[1351] The terminal displays the response message received from the server to the user.

[1352] Input: Response message data received from the server

[1353] Data processing: Convert response message data into a display format

[1354] Output: The response message that is displayed to the user

[1355] Specific operation: The device renders and displays the received text data on the screen.

[1356] Step 6:

[1357] The user may optionally enter additional messages to continue the dialogue.

[1358] Input: User's additional message

[1359] Data processing: Send additional messages to the server

[1360] Output: Additional message data to the server

[1361] Specific behavior: Sends the additional message to the server again as an HTTP POST request.

[1362] (Application example 2)

[1363] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1364] Although conventional digital picture books are available for young children, they are limited in terms of emotional support, and do not provide sufficient emotional support to children. Furthermore, in terms of learning support, the interaction required to obtain appropriate answers to questions is limited. This has led to issues such as not being able to fully improve children's creativity and learning efficiency.

[1365] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for providing a digital picture book, means for identifying the user's emotion using an emotion analysis engine, and means for generating a response message corresponding to the user's emotion. This allows children to enjoy the digital picture book, become emotionally stable through emotional support, and obtain appropriate answers to their questions, thereby improving learning efficiency.

[1366] A "digital picture book" is a picture book in electronic format that can be accessed by young children via computers or mobile devices.

[1367] A "server" is a computer system that stores and processes digital picture book data and user requests, and provides the necessary information.

[1368] An "emotion analysis engine" is a system that has algorithms and functions for analyzing messages from users and identifying their emotions.

[1369] A "response message" is a message generated based on the user's emotional state, intended to provide emotional support and appropriate responses.

[1370] A "question" is an inquiry that a user (child or student) inputs to the server regarding learning content, etc.

[1371] An "answer" is appropriate information or explanation that the server generates after analyzing the user's question.

[1372] "Interaction" refers to the process of operations and inputs that a user makes in order to interact with a system, and includes, in particular, actions such as turning pages, entering questions, and sending emotional messages.

[1373] An "audio file" is digital audio data used when reading picture books or playing audio guides.

[1374] A "prompt sentence" is an input sentence that allows a user to communicate with a system, and is text that includes specific instructions or questions.

[1375] A "generative AI model" is an artificial intelligence algorithm used to realize functions such as reading digital picture books, learning support, and emotion analysis.

[1376] MODE FOR CARRYING OUT THE INVENTION

[1377] This system provides digital picture books to young children, helps them to understand the content they are learning deeply, and provides emotional support through emotional support. The system is structured through interactions between a server, terminals, and users.

[1378] Providing digital picture books

[1379] server

[1380] The server maintains a database of digital picture books. Upon receiving a request from a user, the server retrieves the corresponding digital picture book data and transmits it to the terminal.

[1381] Terminal

[1382] The device displays the digital picture book data received from the server to the user in an interactive interface, detects the user's taps and swipes, turns the pages accordingly, and plays audio files in response to the user's instructions.

[1383] User

[1384] Users can select the picture book they want to read from the options on the screen, tap or swipe the screen to turn the pages and enjoy the story. Tap on a character or scene to play an audio guide.

[1385] Learning support features

[1386] server

[1387] The server receives a question from the user, analyzes its content, retrieves appropriate learning materials from a database, generates an answer, and sends the answer to the device.

[1388] Terminal

[1389] The terminal sends the question entered by the user to the server, receives the answer from the server and displays it to the user, and if the user enters another question, sends the additional question to the server again.

[1390] User

[1391] Users can enter questions about content they did not understand in class or at a cram school, and check the answers displayed on the device. They can enter additional questions as needed until they are satisfied.

[1392] Emotional Support and Sentiment Analysis Engine

[1393] server

[1394] The server receives the consultation content sent by the user. It analyzes the message using an emotion analysis engine to identify the user's emotion. It generates an appropriate response message corresponding to the user's emotion and sends it to the device. The server also accumulates the user's past emotion data to improve the accuracy of future response messages.

[1395] Terminal

[1396] The terminal sends the worries or consultation details entered by the user to the server. It receives a response message from the server and displays it to the user. If the user wants to enter another message, it sends that message to the server again.

[1397] User

[1398] The user inputs what is bothering them or what is making them feel lonely, and checks the response message displayed on the terminal. If necessary, the user can input an additional message to continue the conversation.

[1399] System specific example

[1400] For example, if a user types, "I feel lonely because I haven't made any friends recently," the system's emotion analysis engine will identify the user's emotion as "loneliness." The server will then generate a response message such as, "That's sad. But you have a chance to make new friends. Maybe you should try looking for people who have common hobbies," and send it to the device.

[1401] Prompt Sentence Examples

[1402] Digital Picture Book ID: 12345

[1403] Question: "What is water made of?"

[1404] Emotional support: "I haven't been able to make any friends lately and I feel lonely."

[1405] This system uses an emotion analysis engine and generative AI model to provide quick and appropriate responses to users' emotions and questions. Users can input prompts and use functions such as digital picture book viewing, learning support, and emotional support in response. This is expected to improve children's creativity and learning efficiency.

[1406] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1407] Step 1:

[1408] The user selects a digital picture book

[1409] The user selects the digital picture book they want to read from the options displayed on the device screen. The ID of the selected picture book is entered into the device. Based on this input, the device sends a request for the picture book data to the server.

[1410] Step 2:

[1411] The server sends the digital picture book data

[1412] When the server receives the request, it retrieves the digital picture book data from the database and sends the retrieved picture book data to the terminal as output.

[1413] Step 3:

[1414] The device displays the digital picture book.

[1415] The terminal generates an interactive user interface based on the digital picture book data received from the server, and displays it so that the user can manually turn pages and play audio.

[1416] Step 4:

[1417] User enters study question

[1418] The user inputs a question related to the learning content into the terminal, which then transmits the input question to the server.

[1419] Step 5:

[1420] The server analyzes the question and generates the answer

[1421] The server analyzes the question sent by the user, retrieves appropriate learning materials from the database, and generates an answer, which is then sent to the terminal as output.

[1422] Step 6:

[1423] The device displays the answer

[1424] The terminal displays the answer received from the server to the user, who can then confirm the displayed answer and enter a follow-up question if necessary.

[1425] Step 7:

[1426] The user inputs emotional consultation content.

[1427] The user inputs emotional concerns such as worries and loneliness into the terminal, and the inputted concerns are sent to the server by the terminal.

[1428] Step 8:

[1429] The server analyzes the emotions and generates a response message.

[1430] The server analyzes the consultation content sent by the user using an emotion analysis engine to identify the emotion. Based on the identified emotion, it generates an appropriate response message. The generated response message is sent as output to the terminal.

[1431] Step 9:

[1432] The terminal displays a response message to the user.

[1433] The terminal displays the response message received from the server to the user. The user confirms the displayed response message and continues the dialogue by inputting a message again if necessary.

[1434] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1435] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1436] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1437] [Third embodiment]

[1438] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1439] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1440] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1441] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1442] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1443] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1444] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1445] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1446] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1447] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1448] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1449] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1450] This invention is a system that provides digital picture books to young children, helps children and students to deeply understand the content they are learning, and provides emotional support when they feel worried or lonely.

[1451] Providing digital picture books

[1452] server

[1453] 1. Maintain a database of digital picture books.

[1454] 2. When a request is received from the user (child), the corresponding digital picture book data is obtained and sent to the terminal.

[1455] Terminal

[1456] 1. Based on the digital picture book data received from the server, it is displayed to the user in an interactive interface.

[1457] 2. Turn pages in response to user taps and swipes.

[1458] 3. Play the audio file based on the user's instructions.

[1459] User

[1460] 1. Select the picture book you want to read from the options on the screen.

[1461] 2. Enjoy the content by tapping or swiping the screen to turn the pages.

[1462] 3. Tap to play audio descriptions related to characters and scenes.

[1463] Learning support features

[1464] server

[1465] 1. Receive a question from the user and analyze its content.

[1466] 2. Retrieve appropriate learning materials from the database and generate answers.

[1467] 3. Send the generated answer to the device.

[1468] Terminal

[1469] 1. The question entered by the user is sent to the server.

[1470] 2. Receive the response from the server and display it to the user.

[1471] 3. If the user enters additional questions, the additional questions are sent to the server again.

[1472] User

[1473] 1. Enter questions about content you didn't understand in class or at cram school.

[1474] 2. Check the answer displayed on the device.

[1475] 3. Enter additional questions as needed until you are satisfied.

[1476] Emotional support

[1477] server

[1478] 1. Receive a message from the user and analyze it using a sentiment analysis engine.

[1479] 2. Generate an appropriate response message that corresponds to the user's emotions.

[1480] 3. Send the generated response message to the terminal.

[1481] Terminal

[1482] 1. The user's concerns and questions are sent to the server.

[1483] 2. Receive the response message from the server and display it to the user.

[1484] 3. If the user enters a message again, the message is sent to the server again.

[1485] User

[1486] 1. Enter what is bothering you or making you feel lonely.

[1487] 2. Check the response message displayed on the device.

[1488] 3. Enter an additional message to continue the conversation, if desired.

[1489] For example, if a user inputs "I'm lonely because I haven't made any friends at school recently," the server will generate and send a response message such as "That's sad. But there are opportunities to make new friends. Do you have any favorite games or activities?" By repeating this kind of dialogue between the user and the server, the system helps to lighten the user's mood, even if only a little.

[1490] In this way, the educational and emotional support system that makes full use of digital technology aims to promote children's overall development, conveying the joy of picture books, deepening their understanding of learning, and alleviating feelings of loneliness.

[1491] The processing flow will be explained below.

[1492] Providing digital picture books

[1493] Server Processing

[1494] Step 1:

[1495] The server receives a request from a user.

[1496] Step 2:

[1497] The server retrieves the requested digital picture book data from the database.

[1498] Step 3:

[1499] The server transmits the acquired data to the terminal.

[1500] Terminal handling

[1501] Step 1:

[1502] The terminal receives the digital picture book data transmitted from the server.

[1503] Step 2:

[1504] The terminal analyzes the received data and generates an interactive interface.

[1505] Step 3:

[1506] The device detects the user's taps and swipes and performs page-turning processing accordingly.

[1507] Step 4:

[1508] The terminal plays the audio file in response to a user instruction.

[1509] User Actions

[1510] Step 1:

[1511] The user selects the picture book they want to read from the options on the screen.

[1512] Step 2:

[1513] Users tap or swipe the screen to turn pages.

[1514] Step 3:

[1515] The user can play audio guides by tapping on characters or scenes.

[1516] Learning support features

[1517] Server Processing

[1518] Step 1:

[1519] The server receives a question sent by a user.

[1520] Step 2:

[1521] The server analyzes the received question using an NLP (natural language processing) engine.

[1522] Step 3:

[1523] The server retrieves appropriate learning materials from a database and generates answers.

[1524] Step 4:

[1525] The server sends the generated response to the terminal.

[1526] Terminal handling

[1527] Step 1:

[1528] The terminal transmits the question entered by the user to the server.

[1529] Step 2:

[1530] The terminal receives the response from the server.

[1531] Step 3:

[1532] The terminal displays the received response to the user.

[1533] Step 4:

[1534] If the user inputs more questions, the terminal transmits the additional questions to the server again.

[1535] User Actions

[1536] Step 1:

[1537] The user inputs questions about content that he or she did not understand in class or at a cram school.

[1538] Step 2:

[1539] The user checks the answer displayed on the terminal.

[1540] Step 3:

[1541] The user continues to enter additional questions until satisfied.

[1542] Emotional support

[1543] Server Processing

[1544] Step 1:

[1545] The server receives the consultation content sent from the user.

[1546] Step 2:

[1547] The server analyzes the received message using an emotion analysis engine.

[1548] Step 3:

[1549] The server generates an appropriate response message according to the user's emotions.

[1550] Step 4:

[1551] The server sends the generated response message to the terminal.

[1552] Terminal handling

[1553] Step 1:

[1554] The terminal transmits the worries and consultation details input by the user to the server.

[1555] Step 2:

[1556] The terminal receives the response message from the server.

[1557] Step 3:

[1558] The terminal displays the received response message to the user.

[1559] Step 4:

[1560] If the user inputs a message again, the terminal transmits the additional message to the server again.

[1561] User Actions

[1562] Step 1:

[1563] The user inputs what is bothering them or making them feel lonely.

[1564] Step 2:

[1565] The user checks the response message displayed on the terminal.

[1566] Step 3:

[1567] The user may optionally enter additional messages to continue the dialogue.

[1568] Example 1

[1569] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1570] Although today's children are familiar with digital devices, there is a lack of effective digital picture books and learning support systems. Furthermore, it is difficult to provide appropriate support when children need emotional support. This invention proposes a system that provides digital picture books, learning support, and emotional support in an integrated manner, thereby solving these issues.

[1571] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1572] In this invention, the server includes means for providing digital picture books to young children, means for displaying digital picture book data received from the server on a terminal, means for turning pages in response to tapping or swiping operations by the user, and means for playing audio files in response to user instructions, thereby enabling children to learn interactively while enjoying the digital picture books.

[1573] In addition, the learning support function is enhanced by adding a means for children and students to input questions, a means for sending the input questions to a server, a means for the server to analyze the questions and generate appropriate answers, a means for sending the generated answers to the user, and a means for the user to input further questions.

[1574] Furthermore, the server includes means for receiving a message based on the user's emotions, means for analyzing the received message using an emotion analysis engine, means for generating a response message according to the user's emotions, means for sending the generated response message to the user, and means for the user to input another message and continue the dialogue, thereby making it possible to provide emotional support.

[1575] In this way, the present invention can provide an integrated system that comprehensively provides education and emotional support for children.

[1576] "Young children" generally refers to children from toddlers to early elementary school age.

[1577] A "digital picture book" is a picture book displayed on a computer or mobile device and is an electronic book that includes multimedia elements such as text, images, and audio.

[1578] A "server" refers to a computer system that has the function of receiving requests from users, processing data, and sending it to terminals.

[1579] "Database" refers to a data storage system for centrally managing and storing digital picture book data and learning materials.

[1580] An "interactive interface" refers to a user interface in which the content on the screen responds to user operations, enabling two-way interaction.

[1581] "Tap" refers to the act of lightly tapping with a fingertip on a touchscreen device.

[1582] "Swipe" refers to the action of sliding your finger across a touchscreen device.

[1583] "Audio file" refers to a file that stores audio data in digital format.

[1584] A "question" refers to text that a user enters about something they don't understand or about information they want to confirm.

[1585] A "natural language processing engine" refers to a computer program that analyzes text data and understands its meaning and content.

[1586] A "generative AI model" refers to a trained algorithm that uses artificial intelligence to generate text and voice data.

[1587] An "emotion analysis engine" refers to technology for identifying and analyzing user emotions from text data.

[1588] A "response message" refers to a message generated as a reply or response to a message received from a user.

[1589] "Means for continuing a dialogue" refers to a method for continuing to exchange messages between a user and a system.

[1590] This invention is a system that provides digital picture books to young children, supports them in understanding the learning content deeply, and also provides emotional support. A specific implementation method is shown below.

[1591] Providing digital picture books

[1592] server

[1593] The server maintains a database of digital picture books. This database stores image data, text data, audio files, etc. of picture books. When a request from a user (child) reaches the server via a terminal, the server searches the database for the relevant picture book data and retrieves it. The retrieved data is then converted into an appropriate format and sent to the terminal.

[1594] Terminal

[1595] The device displays the picture book data received from the server in an interactive interface. The received data is decoded, and the image data is displayed in an image viewer, and the text data is displayed in a text viewer. The user can turn pages or play character audio guides by tapping or swiping the screen.

[1596] User

[1597] The user selects the picture book they want to read from the list displayed on the device screen and taps or swipes the screen to view the contents. For example, if the user selects the picture book "Little Red Riding Hood," the server retrieves Little Red Riding Hood's data from the database and sends it to the device. Each time the user taps, the page turns, and an audio guide is played if necessary.

[1598] Learning support features

[1599] server

[1600] The server receives questions from users and analyzes them using a natural language processing engine. Based on the analyzed questions, it retrieves appropriate learning materials from a database and generates the optimal answer using a generative AI model. The generated answer is then sent to the device.

[1601] Terminal

[1602] The device sends the question entered by the user to the server, and the answer received from the server is displayed on the interface. For example, if a user enters "I don't know how to calculate the area of ​​a triangle," the server will generate an answer such as "The area of ​​a triangle can be calculated by multiplying the length of the base by the height and dividing by 2." The user can enter additional information until they are satisfied.

[1603] User

[1604] Users can enter questions into a text field on the device, see the answers displayed, and optionally enter additional questions to seek further answers.

[1605] Emotional support

[1606] server

[1607] The server receives emotion-based messages from users, analyzes them using an emotion analysis engine, and generates an appropriate response message using a generative AI model based on the analysis results, which is then sent to the device.

[1608] Terminal

[1609] The device sends the worries and concerns entered by the user to the server and displays the response message received from the server. For example, if the user enters "I'm lonely because I haven't made any friends at school recently," the server will generate and send a response message such as "That's sad. But there are opportunities to make new friends. Do you have any favorite games or activities?"

[1610] User

[1611] The user inputs their concerns or worries into the terminal, checks the response message that is displayed, and continues the dialogue by inputting the message again if necessary.

[1612] This invention provides an environment where children can grow and have fun through learning using digital devices by providing digital picture books, learning support, and emotional support in an integrated manner.

[1613] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1614] Providing digital picture books

[1615] Step 1:

[1616] server

[1617] The server maintains a database of digital picture books. The database stores image data, text data, audio files, etc. of picture books. When the server starts up, it reads this data and prepares it. Input: All data in the database. Output: Prepared digital picture book data.

[1618] Step 2:

[1619] server

[1620] The server receives a digital picture book request from the user (child). The request includes the user ID and the title or ID of the picture book they want to read. Input: Request from the user. Output: Identification information of the requested picture book.

[1621] Step 3:

[1622] server

[1623] The server searches and retrieves the corresponding digital picture book data from the database based on the identification information. Data is extracted efficiently using SQL queries. Input: Identification information. Output: Retrieved picture book data.

[1624] Step 4:

[1625] server

[1626] The server converts the acquired picture book data into an appropriate format such as JSON and sends it to the terminal. This is where the data is encoded. Input: Acquired picture book data. Output: Formatted picture book data.

[1627] Step 5:

[1628] Terminal

[1629] The device receives the digital picture book data sent from the server. The received data is encoded in JSON format or similar. Input: Data sent from the server. Output: Received encoded data.

[1630] Step 6:

[1631] Terminal

[1632] The terminal decodes the data it receives and converts it into a format that can be displayed in the interface. Image data is displayed in an image viewer, text in a text viewer. Input: The encoded data. Output: The data that can be displayed in the interface.

[1633] Step 7:

[1634] User

[1635] The user selects the book they want to read from the list of digital books displayed on the device screen and confirms the request. They then enjoy the content by turning the pages using on-screen operations (tapping and swiping). Input: Device operations (tapping and swiping). Output: Display of pages and playback of audio guide.

[1636] Learning support features

[1637] Step 1:

[1638] User

[1639] The user types a question into a text field on the device. The input is related to a learning topic that they do not understand. Input: Learning question. Output: The entered question text.

[1640] Step 2:

[1641] Terminal

[1642] The terminal sends the question entered by the user to the server. Input: Question from the user. Output: Question text sent to the server.

[1643] Step 3:

[1644] server

[1645] The server receives the question and analyzes the question using a natural language processing engine. Input: Received question text. Output: Analyzed question content.

[1646] Step 4:

[1647] server

[1648] The server retrieves appropriate learning materials from the database based on the analyzed question. It then uses a generative AI model to generate the optimal answer. Input: Analyzed question. Output: Generated answer text.

[1649] Step 5:

[1650] server

[1651] The server sends the generated answer to the terminal. The answer is encoded in JSON format or similar. Input: Generated answer text. Output: Encoded answer data.

[1652] Step 6:

[1653] Terminal

[1654] The terminal receives the response sent from the server and decodes it into a format that can be displayed. Input: Response data from the server. Output: Displayable response text.

[1655] Step 7:

[1656] User

[1657] The user checks the answer displayed on the terminal, and if they are not satisfied, they can enter an additional question. Input: Answer text, additional question. Output: New question text.

[1658] Emotional support

[1659] Step 1:

[1660] User

[1661] The user inputs the problem or problem content into the text field of the terminal. Input: Problem or problem content. Output: Input text message.

[1662] Step 2:

[1663] Terminal

[1664] The terminal sends the message entered by the user to the server. Input: Text message from the user. Output: Message sent to the server.

[1665] Step 3:

[1666] server

[1667] The server analyzes the received message with a sentiment analysis engine to identify the user's sentiment. Input: Received text message. Output: Identified sentiment data.

[1668] Step 4:

[1669] server

[1670] The server generates an appropriate response message using a generative AI model based on the identified emotion data. Input: Identified emotion data. Output: Generated response message.

[1671] Step 5:

[1672] server

[1673] The server sends the generated response message to the terminal. The response message is sent in text format. Input: Generated response message. Output: Response message sent to the terminal.

[1674] Step 6:

[1675] Terminal

[1676] The terminal displays the response message received from the server to the user. Input: Received response message. Output: Displayed response message.

[1677] Step 7:

[1678] User

[1679] The user checks the response message displayed on the terminal and continues the dialogue by inputting another message if necessary. Input: Input of a message on the terminal. Output: A new message to the server.

[1680] In this system, various data exchanges and processing are carried out between the server, terminals, and users to effectively realize the functions of providing digital picture books, learning support, and emotional support.

[1681] (Application example 1)

[1682] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1683] To support the learning and emotional development of young children, there is a need for systems that provide a consistent approach to reading picture books, learning support, and emotional support. However, no existing systems offer these multiple functions in an integrated manner; most are specialized for individual purposes. Therefore, there is a need for a comprehensive system that can maintain children's interest, deepen their understanding of learning, and provide emotional support in educational settings such as cram schools.

[1684] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1685] In this invention, the server includes means for providing digital picture books to young children, means for displaying digital picture book data received from the server, means for turning pages in response to user operations, means for playing audio files, means for children at cram schools to input questions using smartphones and send them to the server, means for displaying answers received from the server, and means for analyzing the user's emotions and generating and sending appropriate response messages. This makes it possible to provide appropriate answers to learning questions and emotional support while maintaining children's interest through the provision of digital picture books.

[1686] "Young children" refers primarily to young people from preschool to early elementary school age, an age group that is particularly interested in digital picture books and learning support.

[1687] A "digital picture book" is a picture book that is provided in digital format rather than on paper and can be viewed on an electronic device.

[1688] A "server" is a computer system that centrally manages digital picture book data, learning answers, emotion analysis messages, etc., and communicates with user terminals.

[1689] "User operation" refers to input actions that a user performs on the system through a terminal, such as tapping or swiping on the screen.

[1690] The "means for turning pages" is a mechanism that provides a function for turning pages of a digital picture book forward or backward in response to user operations.

[1691] "Audio file" refers to audio recording data, which is used to provide audio readings and explanations of digital picture books.

[1692] A "cram school" is an educational facility where children can receive learning support outside of class.

[1693] "Means for inputting questions" refers to the function that allows users to input questions about their studies as text information into the terminal.

[1694] "Sentiment analysis" refers to a technology that analyzes text data from a user and identifies the emotions the user is feeling.

[1695] A "response message" is a message that is generated based on the results of emotion analysis and sent to the user.

[1696] "Display means" refers to the function for displaying information such as digital picture book data, answers, and response messages sent from the server on the screen of the user's terminal.

[1697] This invention is a system that allows young children to learn while having fun with digital picture books and also receive emotional support. This system consists of a server, a terminal operated by the user, and the user. The configuration and operation of the system are described in detail below.

[1698] Providing digital picture books

[1699] server

[1700] The server maintains a database of digital picture books and is responsible for transmitting the corresponding digital picture book data when requested by a user (young child). The server is equipped with functions for efficiently managing and distributing the metadata and content of digital picture books.

[1701] Terminal

[1702] The device operated by the user (smartphone or tablet) receives the digital picture book data from the server and displays it in an interactive interface, allowing the user to turn pages by tapping or swiping the screen and play audio guides related to characters and scenes.

[1703] User

[1704] Users select the picture book they want to read from the options on the screen, advance through the pages through operations, and enjoy the contents of the picture book using audio guidance.

[1705] Learning support features

[1706] server

[1707] The server receives a question from the user, analyzes the question, retrieves appropriate learning materials from a database, and generates an answer, which is then sent to the terminal.

[1708] Terminal

[1709] The terminal transmits the questions entered by the user to the server, receives and displays the answers from the server, and if the user enters additional questions, the additional questions can be transmitted to the server again.

[1710] User

[1711] Users can enter questions about content they did not understand in class or at a cram school into the device, check the answers provided, and enter additional questions as needed.

[1712] Emotional support

[1713] server

[1714] The server receives messages from users, analyzes their emotions using AI, and generates an appropriate response message based on the analysis results, which is then sent to the device.

[1715] Terminal

[1716] The terminal inputs the user's worries or feelings of loneliness and sends them to the server. It receives and displays a response message from the server, and if the user inputs another message, it sends that message to the server again.

[1717] User

[1718] The user inputs their concerns or feelings of loneliness into the terminal, confirms the response message provided, and inputs an additional message to continue the dialogue if necessary.

[1719] Hardware and software used

[1720] Hardware used

[1721] Smartphones and tablets: Used by users as portable devices.

[1722] Server: A central management system that manages the digital picture book database and AI responses.

[1723] Software used

[1724] Flutter / React Native: Used to build and operate the UI of smartphone apps.

[1725] requests (Python library): A library for enabling communication between the server and the terminal.

[1726] Generative AI model: AI technology used for sentiment analysis and response message generation.

[1727] Examples and prompts

[1728] As a specific example, if the user selects the digital picture book "Forest Stories," the terminal operates as follows.

[1729] 1. The user selects the "Forest Tales" digital picture book.

[1730] 2. The terminal sends a request to the server, receives the digital picture book data from the server, and displays it.

[1731] 3. When the user taps to turn the page, the appropriate page is displayed and the associated audio plays.

[1732] Example prompt sentence:

[1733] Get and display the page data for the digital picture book "Forest Stories." URL: https: / / example.com / digital_books / 001

[1734] In this way, the present invention provides a comprehensive system that provides relevant answers to educational questions and emotional support while retaining the interest of young children.

[1735] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1736] Step 1:

[1737] The user uses the terminal to make a digital picture book request.

[1738] Input: A request for a digital picture book based on the user's selection

[1739] Output: Request sent to server

[1740] How it works: The user selects a picture book, such as "Forest Stories," from the interface on the device, and the information is sent to the server.

[1741] Step 2:

[1742] The server receives the request, acquires the corresponding digital picture book data, and transmits it.

[1743] Input: Digital picture book request from user device

[1744] Output: Sending digital picture book data

[1745] Operation: The server retrieves the requested picture book data from the digital picture book database and sends it to the user's terminal.

[1746] Step 3:

[1747] The terminal displays the digital picture book data received from the server.

[1748] Input: Digital picture book data from the server

[1749] Output: Display a digital picture book on the screen

[1750] Operation: The device displays the received digital picture book data on an interactive interface, allowing the user to turn pages and play audio.

[1751] Step 4:

[1752] The user inputs a question related to the study and sends it to the server via the terminal.

[1753] Input: User's question

[1754] Output: Sending the query data to the server

[1755] How it works: The user enters a study question into the terminal, and the data is sent to the server.

[1756] Step 5:

[1757] The server analyzes the received question, retrieves appropriate learning materials from a database, and generates an answer.

[1758] Input: Question data from the user

[1759] Output: Generated training answers

[1760] How it works: The server analyzes the received question, retrieves relevant learning material from the database, and generates an appropriate answer.

[1761] Step 6:

[1762] The terminal displays the answer received from the server to the user.

[1763] Input: Learning response data from the server

[1764] Output: Display the answer to the user

[1765] Operation: The terminal displays the answer received from the server to the user, who then confirms the content.

[1766] Step 7:

[1767] If the user enters more questions, the questions are sent to the server again.

[1768] Input: User's additional question

[1769] Output: Sending additional questions to the server

[1770] How it works: The user enters a follow-up question into the terminal, and the data is sent back to the server.

[1771] Step 8:

[1772] The user inputs the emotional worries and consultation contents and transmits them to the server.

[1773] Input: User's sentiment message

[1774] Output: Sending emotion message to the server

[1775] How it works: The user enters a message about their worries or loneliness into the device, and the data is sent to the server.

[1776] Step 9:

[1777] The server analyzes the sentiment of the received message and generates an appropriate response message.

[1778] Input: Emotional message from the user

[1779] Output: The generated response message

[1780] How it works: The server analyzes the received message using an AI model and generates an appropriate emotional response message.

[1781] Step 10:

[1782] The terminal displays the response message received from the server to the user.

[1783] Input: Response message from the server

[1784] Output: Display a response message to the user

[1785] Operation: The terminal displays the received response message to the user, who confirms it.

[1786] Step 11:

[1787] The user inputs a message again to continue the dialogue and sends it to the server.

[1788] Input: User's additional emotion message

[1789] Output: Send additional emotion message to server

[1790] Operation: The user enters a message into the terminal to continue the dialogue, and the data is sent again to the server.

[1791] In this way, the present invention provides a system that supports the user's learning and also provides emotional support at the same time.

[1792] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1793] This invention is a system that provides digital picture books to young children, supports children and students in gaining a deeper understanding of the learning content, and provides emotional support when they feel distressed or lonely. By combining a specific emotion engine, the system recognizes the user's emotions and provides appropriate responses accordingly.

[1794] Providing digital picture books

[1795] server

[1796] 1. Maintain a database of digital picture books.

[1797] 2. When a request is received from the user (child), the corresponding digital picture book data is obtained and sent to the terminal.

[1798] Terminal

[1799] 1. Based on the digital picture book data received from the server, it is displayed to the user in an interactive interface.

[1800] 2. Detect user taps and swipes and perform page-turning processing accordingly.

[1801] 3. Play the audio file according to the user's instructions.

[1802] User

[1803] 1. Select the picture book you want to read from the options on the screen.

[1804] 2. Enjoy the content by tapping or swiping the screen to turn the pages.

[1805] 3. Tap to play audio descriptions related to characters and scenes.

[1806] Learning support features

[1807] server

[1808] 1. Receive a question from the user and analyze its content.

[1809] 2. Retrieve appropriate learning materials from the database and generate answers.

[1810] 3. Send the generated answer to the device.

[1811] Terminal

[1812] 1. The question entered by the user is sent to the server.

[1813] 2. Receive the response from the server and display it to the user.

[1814] 3. If the user enters additional questions, the additional questions are sent to the server again.

[1815] User

[1816] 1. Enter questions about content you didn't understand in class or at cram school.

[1817] 2. Check the answer displayed on the device.

[1818] 3. Enter additional questions as needed until you are satisfied.

[1819] Emotional Support and Emotion Engine

[1820] server

[1821] 1. Receive the consultation content sent by the user.

[1822] 2. The message is analyzed using a sentiment analysis engine to identify the user's sentiment.

[1823] 3. Generate an appropriate response message that corresponds to the user's emotions.

[1824] 4. Send the generated response message to the terminal.

[1825] 5. Accumulates user's past emotional data to improve the accuracy of future response messages.

[1826] Terminal

[1827] 1. The user's concerns and questions are sent to the server.

[1828] 2. Receive the response message from the server and display it to the user.

[1829] 3. If the user enters a message again, the message is sent to the server again.

[1830] User

[1831] 1. Enter what is bothering you or making you feel lonely.

[1832] 2. Check the response message displayed on the device.

[1833] 3. If necessary, enter an additional message to continue the conversation.

[1834] For example, if a user types, "I feel lonely because I haven't made any friends recently," the server's emotion analysis engine will identify the user's emotion as "loneliness." The server will then generate a response message such as, "That's tough. But you have a chance to make new friends. Maybe you should try looking for someone who has a common hobby with you," and send it to the device. If the user receives this message and continues the dialogue, it can alleviate the user's feelings of loneliness and worries.

[1835] The introduction of this system can improve children's creativity, learning efficiency, and emotional support, promoting their overall development.

[1836] The processing flow will be explained below.

[1837] Providing digital picture books

[1838] Server Processing

[1839] Step 1:

[1840] The server receives a request from a user.

[1841] Step 2:

[1842] The server retrieves the requested digital picture book data from the database.

[1843] Step 3:

[1844] The server transmits the acquired data to the terminal.

[1845] Terminal handling

[1846] Step 1:

[1847] The terminal receives the digital picture book data transmitted from the server.

[1848] Step 2:

[1849] The terminal analyzes the received data and generates an interactive interface.

[1850] Step 3:

[1851] The device detects the user's taps and swipes and performs page-turning processing accordingly.

[1852] Step 4:

[1853] The terminal plays the audio file in response to a user instruction.

[1854] User Actions

[1855] Step 1:

[1856] The user selects the picture book they want to read from the options on the screen.

[1857] Step 2:

[1858] Users tap or swipe the screen to turn pages.

[1859] Step 3:

[1860] The user can play audio guides by tapping on characters or scenes.

[1861] Learning support features

[1862] Server Processing

[1863] Step 1:

[1864] The server receives a question sent by a user.

[1865] Step 2:

[1866] The server analyzes the received question using an NLP (natural language processing) engine.

[1867] Step 3:

[1868] The server retrieves appropriate learning materials from a database and generates answers.

[1869] Step 4:

[1870] The server sends the generated response to the terminal.

[1871] Terminal handling

[1872] Step 1:

[1873] The terminal transmits the question entered by the user to the server.

[1874] Step 2:

[1875] The terminal receives the response from the server.

[1876] Step 3:

[1877] The terminal displays the received response to the user.

[1878] Step 4:

[1879] If the user inputs more questions, the terminal transmits the additional questions to the server again.

[1880] User Actions

[1881] Step 1:

[1882] The user inputs questions about content that he or she did not understand in class or at a cram school.

[1883] Step 2:

[1884] The user checks the answer displayed on the terminal.

[1885] Step 3:

[1886] The user continues to enter additional questions until satisfied.

[1887] Emotional Support and Emotion Engine

[1888] Server Processing

[1889] Step 1:

[1890] The server receives the consultation content sent from the user.

[1891] Step 2:

[1892] The server analyzes the received message using an emotion analysis engine to identify the user's emotion.

[1893] Step 3:

[1894] The server generates an appropriate response message according to the user's emotions based on the results of the emotion analysis.

[1895] Step 4:

[1896] The server sends the generated response message to the terminal.

[1897] Step 5:

[1898] The server accumulates the user's past emotional data and improves the accuracy of future response messages.

[1899] Terminal handling

[1900] Step 1:

[1901] The terminal transmits the worries and consultation details input by the user to the server.

[1902] Step 2:

[1903] The terminal receives the response message from the server.

[1904] Step 3:

[1905] The terminal displays the received response message to the user.

[1906] Step 4:

[1907] If the user inputs a message again, the terminal transmits the additional message to the server again.

[1908] User Actions

[1909] Step 1:

[1910] The user inputs what is bothering them or making them feel lonely.

[1911] Step 2:

[1912] The user checks the response message displayed on the terminal.

[1913] Step 3:

[1914] The user may optionally enter additional messages to continue the dialogue.

[1915] Example 2

[1916] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1917] In today's educational environment, there is a need for digital picture books that allow young children to deepen their knowledge while having fun learning, and for methods that encourage students to ask questions about the content they are learning and deepen their understanding. It is also important to provide appropriate emotional support when children feel distressed or lonely. There is a lack of systems that can meet these needs and promote children's holistic development.

[1918] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1919] In this invention, the server includes an information transmission device means for storing data, a display device means for displaying data received from the information transmission device, a device means for switching pages in response to operations, and a device means for playing audio data, which enables provision of digital picture books, understanding of learning content, and emotional support.

[1920] "Data" means electronic records of specific information or content.

[1921] An "information transmitting device" refers to a device that holds data and transmits it to a terminal or display device.

[1922] A "display device" is a device for visually displaying received data to a user.

[1923] An "operation" refers to an input action that a user performs using a terminal or a display device.

[1924] A "page-switching device" is a device that has the function of turning pages of a digital picture book or the like in response to a user's operation.

[1925] "Audio data" means electronic recordings of audio information.

[1926] A "device that plays back audio data" is a device that has the function of receiving and playing back audio data.

[1927] A "question input device" is a device that allows a user to input their own questions or doubts.

[1928] The "device for additional input" refers to a device that allows the user to input information again.

[1929] An "analyzing device" is a device that has the function of analyzing an input question or message and generating an appropriate answer or response.

[1930] An "emotion-based message" is a message whose content reflects the user's emotional state.

[1931] A "response message" is a response generated based on the analyzed content.

[1932] The "display device" refers to a device that displays the generated answer or response message in an easy-to-see manner for the user.

[1933] The present invention is a system that provides digital picture books to young children, supports children and students in gaining a deeper understanding of the learning content, and provides emotional support when they feel distressed or lonely. The following describes specific embodiments of the invention.

[1934] Providing digital picture books

[1935] server

[1936] The server maintains a database of digital picture books. When it receives a request from a user (child), it retrieves the corresponding digital picture book data from the database and sends it to the device. For this reason, the server has high-performance database software (e.g., MySQL) and data transmission functions.

[1937] Terminal

[1938] The terminal receives the digital picture book data from the server and displays it to the user using an interactive interface. It is typically a tablet or smartphone, and is equipped with an LCD display or touchscreen. The terminal detects the user's taps and swipes and processes the pages accordingly. It also plays audio data in response to user instructions. This is achieved using a built-in speaker or earphones.

[1939] User

[1940] When reading a digital picture book, users select the book they want from a list on the screen. They turn the pages by tapping or swiping, and when they tap on parts related to characters or scenes, audio guides and sound effects are played.

[1941] Learning support features

[1942] server

[1943] The server receives a question from the user and analyzes it using a natural language processing engine (e.g., Google Natural Language API). It retrieves appropriate learning materials from a database and generates an answer. The server is equipped with data analysis and transmission functions to send the generated answer to the device.

[1944] Terminal

[1945] The terminal sends the question entered by the user to the server, receives the answer from the server and displays it to the user, and if the user enters another question, sends the additional question back to the server and displays related additional learning materials.

[1946] User

[1947] Users can enter questions about things they didn't understand in class or at a cram school. They can check the answers and explanations displayed on the device to deepen their understanding. If necessary, they can enter further questions and repeat the process until they are satisfied.

[1948] Emotional Support and Emotion Engine

[1949] server

[1950] The server receives the consultation content sent by the user. It analyzes the message using an emotion analysis engine (e.g., IBM Watson Tone Analyzer) to identify the user's emotion. It generates a response message based on the user's emotion and sends it to the device. It also accumulates data on the user's past emotions to improve accuracy.

[1951] Terminal

[1952] The terminal sends the worries or consultation details entered by the user to the server. It receives a response message from the server and displays it to the user. If the user wants to enter a message again, it can send that message to the server again.

[1953] User

[1954] The user inputs what is bothering them or what is making them feel lonely. They check the response message displayed on the terminal and, if necessary, input an additional message to continue the conversation.

[1955] Specific examples and examples of input to generative AI models

[1956] For example, if a user types, "I'm lonely because I haven't made any friends recently," the server will use its emotion analysis engine to identify the user's emotion as "loneliness." It will then generate a response message such as, "That's tough. But you have a chance to make new friends. Maybe you should try looking for someone who has common hobbies with you," and send it to the device. By receiving this message and continuing the dialogue, the user can ease their feelings of loneliness and distress.

[1957] Example of input prompt for generative AI model

[1958] 1. "I'm lonely and can't make friends. Please tell me what to do."

[1959] 2. "I'm having trouble understanding some of the things I'm studying at school. Can you explain?"

[1960] 3. "The picture book I want to read right now is one with lots of animals in it."

[1961] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1962] Providing digital picture books

[1963] Step 1:

[1964] The server maintains a database of digital picture books, which stores data on a plurality of digital picture books.

[1965] Input: A digital picture book request from a user

[1966] Data processing: Analyze user requests and identify the corresponding digital picture book data

[1967] Output: Identified digital picture book data

[1968] Specific operation: The server receives the HTTP request, executes an SQL query based on it, and retrieves the required data from the database.

[1969] Step 2:

[1970] The server transmits the corresponding digital picture book data to the terminal.

[1971] Input: Identified digital picture book data

[1972] Data processing: Digital picture book data is converted into packets according to communication protocols (e.g. HTTP, HTTPS).

[1973] Output: Data packets sent to the device

[1974] Specific operation: The server sends the digital picture book data encoded using HTTPS to the terminal.

[1975] Step 3:

[1976] The terminal displays the digital picture book data received from the server to the user on an interface.

[1977] Input: Digital picture book data received from the server

[1978] Data processing: Analyzes received data and converts it into a display format

[1979] Output: A digital picture book displayed in a form visible to the user

[1980] Specific operation: The device analyzes the images and text data of the digital picture book and displays them on the screen.

[1981] Step 4:

[1982] The device detects the user's taps and swipes and performs page-turning processing accordingly.

[1983] Input: User taps and swipes

[1984] Data processing: Analyzes the operation data and converts the data on the next page into a display format

[1985] Output: Next page displayed

[1986] Specific operation: The gesture engine analyzes touch operations and executes page transition animations.

[1987] Step 5:

[1988] The terminal plays the audio file in response to a user instruction.

[1989] Input: User's voice instructions (tap operation, etc.)

[1990] Data processing: Read and play audio files based on instructions

[1991] Output: Audio is played

[1992] Specific behavior: The device loads the audio file from storage and plays it using the media player.

[1993] Learning support features

[1994] Step 1:

[1995] The server receives a query from the user.

[1996] Input: The question the user types into the terminal

[1997] Data processing: Transfer received questions to the server

[1998] Output: Query data to the server

[1999] Specific operation: Sends the question data to the server as an HTTP POST request.

[2000] Step 2:

[2001] The server analyzes the received question using a natural language processing engine.

[2002] Input: Received question data

[2003] Data processing: Analysis using a natural language processing engine

[2004] Output: Analysis results

[2005] Specific operation: The server uses the Google Natural Language API to analyze the question data and extract keywords and intent.

[2006] Step 3:

[2007] The server retrieves appropriate learning materials from a database and generates answers.

[2008] Input: Analysis results of natural language processing

[2009] Data processing: Run database queries to identify appropriate learning materials and generate answers.

[2010] Output: Generated response data

[2011] Specific operation: Executes an SQL query to retrieve teaching material data and generates answer sentences using a template.

[2012] Step 4:

[2013] The server sends the generated response to the terminal.

[2014] Input: Generated response data

[2015] Data processing: Converts response data into packets according to the communication protocol

[2016] Output: Data packets sent to the device

[2017] Specific operation: Sends response data to the terminal via an HTTPS request.

[2018] Step 5:

[2019] The terminal displays the response received from the server to the user.

[2020] Input: Response data received from the server

[2021] Data processing: Converting response data into a display format

[2022] Output: The answer that is displayed to the user

[2023] Specific operation: The device renders and displays the received text data on the screen.

[2024] Step 6:

[2025] If the user has further questions, they enter them.

[2026] Input: User's additional question

[2027] Data processing: Send additional questions to the server

[2028] Output: Additional question data to the server

[2029] Specific operation: Sends the additional question to the server again as an HTTP POST request.

[2030] Emotional Support and Emotion Engine

[2031] Step 1:

[2032] The server receives the consultation content from the user.

[2033] Input: The consultation content entered by the user into the terminal

[2034] Data processing: Transfer received consultation details to the server

[2035] Output: Consultation content data to the server

[2036] Specific operation: Consultation content data is sent to the server as an HTTP POST request.

[2037] Step 2:

[2038] The server analyzes the consultation content using an emotion analysis engine and identifies the user's emotions.

[2039] Input: Received consultation content data

[2040] Data processing: Analysis using a sentiment analysis engine

[2041] Output: Emotion analysis results

[2042] Specific operation: Uses IBM Watson Tone Analyzer to analyze text data and extract emotions.

[2043] Step 3:

[2044] The server generates a response message according to the user's emotion.

[2045] Input: Sentiment analysis results

[2046] Data processing: Generate appropriate response messages

[2047] Output: Response message data

[2048] Specific operation: Generate a response message from a template based on the analysis results.

[2049] Step 4:

[2050] The server sends the generated response message to the terminal.

[2051] Input: Response message data

[2052] Data processing: Converts message data into packets according to the communication protocol

[2053] Output: Data packets sent to the device

[2054] Specific operation: Sends response message data to the terminal via an HTTPS request.

[2055] Step 5:

[2056] The terminal displays the response message received from the server to the user.

[2057] Input: Response message data received from the server

[2058] Data processing: Convert response message data into a display format

[2059] Output: The response message that is displayed to the user

[2060] Specific operation: The device renders and displays the received text data on the screen.

[2061] Step 6:

[2062] The user may optionally enter additional messages to continue the dialogue.

[2063] Input: User's additional message

[2064] Data processing: Send additional messages to the server

[2065] Output: Additional message data to the server

[2066] Specific behavior: Sends the additional message to the server again as an HTTP POST request.

[2067] (Application example 2)

[2068] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2069] Although conventional digital picture books are available for young children, they are limited in terms of emotional support, and do not provide sufficient emotional support to children. Furthermore, in terms of learning support, the interaction required to obtain appropriate answers to questions is limited. This has led to issues such as not being able to fully improve children's creativity and learning efficiency.

[2070] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for providing a digital picture book, means for identifying the user's emotion using an emotion analysis engine, and means for generating a response message corresponding to the user's emotion. This allows children to enjoy the digital picture book, become emotionally stable through emotional support, and obtain appropriate answers to their questions, thereby improving learning efficiency.

[2071] A "digital picture book" is a picture book in electronic format that can be accessed by young children via computers or mobile devices.

[2072] A "server" is a computer system that stores and processes digital picture book data and user requests, and provides the necessary information.

[2073] An "emotion analysis engine" is a system that has algorithms and functions for analyzing messages from users and identifying their emotions.

[2074] A "response message" is a message generated based on the user's emotional state, intended to provide emotional support and appropriate responses.

[2075] A "question" is an inquiry that a user (child or student) inputs to the server regarding learning content, etc.

[2076] An "answer" is appropriate information or explanation that the server generates after analyzing the user's question.

[2077] "Interaction" refers to the process of operations and inputs that a user makes in order to interact with a system, and includes, in particular, actions such as turning pages, entering questions, and sending emotional messages.

[2078] An "audio file" is digital audio data used when reading picture books or playing audio guides.

[2079] A "prompt sentence" is an input sentence that allows a user to communicate with a system, and is text that includes specific instructions or questions.

[2080] A "generative AI model" is an artificial intelligence algorithm used to realize functions such as reading digital picture books, learning support, and emotion analysis.

[2081] MODE FOR CARRYING OUT THE INVENTION

[2082] This system provides digital picture books to young children, helps them to understand the content they are learning deeply, and provides emotional support through emotional support. The system is structured through interactions between a server, terminals, and users.

[2083] Providing digital picture books

[2084] server

[2085] The server maintains a database of digital picture books. Upon receiving a request from a user, the server retrieves the corresponding digital picture book data and transmits it to the terminal.

[2086] Terminal

[2087] The device displays the digital picture book data received from the server to the user in an interactive interface, detects the user's taps and swipes, turns the pages accordingly, and plays audio files in response to the user's instructions.

[2088] User

[2089] Users can select the picture book they want to read from the options on the screen, tap or swipe the screen to turn the pages and enjoy the story. Tap on a character or scene to play an audio guide.

[2090] Learning support features

[2091] server

[2092] The server receives a question from the user, analyzes its content, retrieves appropriate learning materials from a database, generates an answer, and sends the answer to the device.

[2093] Terminal

[2094] The terminal sends the question entered by the user to the server, receives the answer from the server and displays it to the user, and if the user enters another question, sends the additional question to the server again.

[2095] User

[2096] Users can enter questions about content they did not understand in class or at a cram school, and check the answers displayed on the device. They can enter additional questions as needed until they are satisfied.

[2097] Emotional Support and Sentiment Analysis Engine

[2098] server

[2099] The server receives the consultation content sent by the user. It analyzes the message using an emotion analysis engine to identify the user's emotion. It generates an appropriate response message corresponding to the user's emotion and sends it to the device. The server also accumulates the user's past emotion data to improve the accuracy of future response messages.

[2100] Terminal

[2101] The terminal sends the worries or consultation details entered by the user to the server. It receives a response message from the server and displays it to the user. If the user wants to enter another message, it sends that message to the server again.

[2102] User

[2103] The user inputs what is bothering them or what is making them feel lonely, and checks the response message displayed on the terminal. If necessary, the user can input an additional message to continue the conversation.

[2104] System specific example

[2105] For example, if a user types, "I feel lonely because I haven't made any friends recently," the system's emotion analysis engine will identify the user's emotion as "loneliness." The server will then generate a response message such as, "That's sad. But you have a chance to make new friends. Maybe you should try looking for people who have common hobbies," and send it to the device.

[2106] Prompt Sentence Examples

[2107] Digital Picture Book ID: 12345

[2108] Question: "What is water made of?"

[2109] Emotional support: "I haven't been able to make any friends lately and I feel lonely."

[2110] This system uses an emotion analysis engine and generative AI model to provide quick and appropriate responses to users' emotions and questions. Users can input prompts and use functions such as digital picture book viewing, learning support, and emotional support in response. This is expected to improve children's creativity and learning efficiency.

[2111] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2112] Step 1:

[2113] The user selects a digital picture book

[2114] The user selects the digital picture book they want to read from the options displayed on the device screen. The ID of the selected picture book is entered into the device. Based on this input, the device sends a request for the picture book data to the server.

[2115] Step 2:

[2116] The server sends the digital picture book data

[2117] When the server receives the request, it retrieves the digital picture book data from the database and sends the retrieved picture book data to the terminal as output.

[2118] Step 3:

[2119] The device displays the digital picture book.

[2120] The terminal generates an interactive user interface based on the digital picture book data received from the server, and displays it so that the user can manually turn pages and play audio.

[2121] Step 4:

[2122] User enters study question

[2123] The user inputs a question related to the learning content into the terminal, which then transmits the input question to the server.

[2124] Step 5:

[2125] The server analyzes the question and generates the answer

[2126] The server analyzes the question sent by the user, retrieves appropriate learning materials from the database, and generates an answer, which is then sent to the terminal as output.

[2127] Step 6:

[2128] The device displays the answer

[2129] The terminal displays the answer received from the server to the user, who can then confirm the displayed answer and enter a follow-up question if necessary.

[2130] Step 7:

[2131] The user inputs emotional consultation content.

[2132] The user inputs emotional concerns such as worries and loneliness into the terminal, and the inputted concerns are sent to the server by the terminal.

[2133] Step 8:

[2134] The server analyzes the emotions and generates a response message.

[2135] The server analyzes the consultation content sent by the user using an emotion analysis engine to identify the emotion. Based on the identified emotion, it generates an appropriate response message. The generated response message is sent as output to the terminal.

[2136] Step 9:

[2137] The terminal displays a response message to the user.

[2138] The terminal displays the response message received from the server to the user. The user confirms the displayed response message and continues the dialogue by inputting a message again if necessary.

[2139] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2140] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2141] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2142] [Fourth embodiment]

[2143] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2144] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2145] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2146] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2147] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2148] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2149] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2150] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2151] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2152] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2153] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2154] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2155] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2156] This invention is a system that provides digital picture books to young children, helps children and students to deeply understand the content they are learning, and provides emotional support when they feel worried or lonely.

[2157] Providing digital picture books

[2158] server

[2159] 1. Maintain a database of digital picture books.

[2160] 2. When a request is received from the user (child), the corresponding digital picture book data is obtained and sent to the terminal.

[2161] Terminal

[2162] 1. Based on the digital picture book data received from the server, it is displayed to the user in an interactive interface.

[2163] 2. Turn pages in response to user taps and swipes.

[2164] 3. Play the audio file based on the user's instructions.

[2165] User

[2166] 1. Select the picture book you want to read from the options on the screen.

[2167] 2. Enjoy the content by tapping or swiping the screen to turn the pages.

[2168] 3. Tap to play audio descriptions related to characters and scenes.

[2169] Learning support features

[2170] server

[2171] 1. Receive a question from the user and analyze its content.

[2172] 2. Retrieve appropriate learning materials from the database and generate answers.

[2173] 3. Send the generated answer to the device.

[2174] Terminal

[2175] 1. The question entered by the user is sent to the server.

[2176] 2. Receive the response from the server and display it to the user.

[2177] 3. If the user enters additional questions, the additional questions are sent to the server again.

[2178] User

[2179] 1. Enter questions about content you didn't understand in class or at cram school.

[2180] 2. Check the answer displayed on the device.

[2181] 3. Enter additional questions as needed until you are satisfied.

[2182] Emotional support

[2183] server

[2184] 1. Receive a message from the user and analyze it using a sentiment analysis engine.

[2185] 2. Generate an appropriate response message that corresponds to the user's emotions.

[2186] 3. Send the generated response message to the terminal.

[2187] Terminal

[2188] 1. The user's concerns and questions are sent to the server.

[2189] 2. Receive the response message from the server and display it to the user.

[2190] 3. If the user enters a message again, the message is sent to the server again.

[2191] User

[2192] 1. Enter what is bothering you or making you feel lonely.

[2193] 2. Check the response message displayed on the device.

[2194] 3. Enter an additional message to continue the conversation, if desired.

[2195] For example, if a user inputs "I'm lonely because I haven't made any friends at school recently," the server will generate and send a response message such as "That's sad. But there are opportunities to make new friends. Do you have any favorite games or activities?" By repeating this kind of dialogue between the user and the server, the system helps to lighten the user's mood, even if only a little.

[2196] In this way, the educational and emotional support system that makes full use of digital technology aims to promote children's overall development, conveying the joy of picture books, deepening their understanding of learning, and alleviating feelings of loneliness.

[2197] The processing flow will be explained below.

[2198] Providing digital picture books

[2199] Server Processing

[2200] Step 1:

[2201] The server receives a request from a user.

[2202] Step 2:

[2203] The server retrieves the requested digital picture book data from the database.

[2204] Step 3:

[2205] The server transmits the acquired data to the terminal.

[2206] Terminal handling

[2207] Step 1:

[2208] The terminal receives the digital picture book data transmitted from the server.

[2209] Step 2:

[2210] The terminal analyzes the received data and generates an interactive interface.

[2211] Step 3:

[2212] The device detects the user's taps and swipes and performs page-turning processing accordingly.

[2213] Step 4:

[2214] The terminal plays the audio file in response to a user instruction.

[2215] User Actions

[2216] Step 1:

[2217] The user selects the picture book they want to read from the options on the screen.

[2218] Step 2:

[2219] Users tap or swipe the screen to turn pages.

[2220] Step 3:

[2221] The user can play audio guides by tapping on characters or scenes.

[2222] Learning support features

[2223] Server Processing

[2224] Step 1:

[2225] The server receives a question sent by a user.

[2226] Step 2:

[2227] The server analyzes the received question using an NLP (natural language processing) engine.

[2228] Step 3:

[2229] The server retrieves appropriate learning materials from a database and generates answers.

[2230] Step 4:

[2231] The server sends the generated response to the terminal.

[2232] Terminal handling

[2233] Step 1:

[2234] The terminal transmits the question entered by the user to the server.

[2235] Step 2:

[2236] The terminal receives the response from the server.

[2237] Step 3:

[2238] The terminal displays the received response to the user.

[2239] Step 4:

[2240] If the user inputs more questions, the terminal transmits the additional questions to the server again.

[2241] User Actions

[2242] Step 1:

[2243] The user inputs questions about content that he or she did not understand in class or at a cram school.

[2244] Step 2:

[2245] The user checks the answer displayed on the terminal.

[2246] Step 3:

[2247] The user continues to enter additional questions until satisfied.

[2248] Emotional support

[2249] Server Processing

[2250] Step 1:

[2251] The server receives the consultation content sent from the user.

[2252] Step 2:

[2253] The server analyzes the received message using an emotion analysis engine.

[2254] Step 3:

[2255] The server generates an appropriate response message according to the user's emotions.

[2256] Step 4:

[2257] The server sends the generated response message to the terminal.

[2258] Terminal handling

[2259] Step 1:

[2260] The terminal transmits the worries and consultation details input by the user to the server.

[2261] Step 2:

[2262] The terminal receives the response message from the server.

[2263] Step 3:

[2264] The terminal displays the received response message to the user.

[2265] Step 4:

[2266] If the user inputs a message again, the terminal transmits the additional message to the server again.

[2267] User Actions

[2268] Step 1:

[2269] The user inputs what is bothering them or making them feel lonely.

[2270] Step 2:

[2271] The user checks the response message displayed on the terminal.

[2272] Step 3:

[2273] The user may optionally enter additional messages to continue the dialogue.

[2274] Example 1

[2275] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2276] Although today's children are familiar with digital devices, there is a lack of effective digital picture books and learning support systems. Furthermore, it is difficult to provide appropriate support when children need emotional support. This invention proposes a system that provides digital picture books, learning support, and emotional support in an integrated manner, thereby solving these issues.

[2277] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2278] In this invention, the server includes means for providing digital picture books to young children, means for displaying digital picture book data received from the server on a terminal, means for turning pages in response to tapping or swiping operations by the user, and means for playing audio files in response to user instructions, thereby enabling children to learn interactively while enjoying the digital picture books.

[2279] In addition, the learning support function is enhanced by adding a means for children and students to input questions, a means for sending the input questions to a server, a means for the server to analyze the questions and generate appropriate answers, a means for sending the generated answers to the user, and a means for the user to input further questions.

[2280] Furthermore, the server includes means for receiving a message based on the user's emotions, means for analyzing the received message using an emotion analysis engine, means for generating a response message according to the user's emotions, means for sending the generated response message to the user, and means for the user to input another message and continue the dialogue, thereby making it possible to provide emotional support.

[2281] In this way, the present invention can provide an integrated system that comprehensively provides education and emotional support for children.

[2282] "Young children" generally refers to children from toddlers to early elementary school age.

[2283] A "digital picture book" is a picture book displayed on a computer or mobile device and is an electronic book that includes multimedia elements such as text, images, and audio.

[2284] A "server" refers to a computer system that has the function of receiving requests from users, processing data, and sending it to terminals.

[2285] "Database" refers to a data storage system for centrally managing and storing digital picture book data and learning materials.

[2286] An "interactive interface" refers to a user interface in which the content on the screen responds to user operations, enabling two-way interaction.

[2287] "Tap" refers to the act of lightly tapping with a fingertip on a touchscreen device.

[2288] "Swipe" refers to the action of sliding your finger across a touchscreen device.

[2289] "Audio file" refers to a file that stores audio data in digital format.

[2290] A "question" refers to text that a user enters about something they don't understand or about information they want to confirm.

[2291] A "natural language processing engine" refers to a computer program that analyzes text data and understands its meaning and content.

[2292] A "generative AI model" refers to a trained algorithm that uses artificial intelligence to generate text and voice data.

[2293] An "emotion analysis engine" refers to technology for identifying and analyzing user emotions from text data.

[2294] A "response message" refers to a message generated as a reply or response to a message received from a user.

[2295] "Means for continuing a dialogue" refers to a method for continuing to exchange messages between a user and a system.

[2296] This invention is a system that provides digital picture books to young children, supports them in understanding the learning content deeply, and also provides emotional support. A specific implementation method is shown below.

[2297] Providing digital picture books

[2298] server

[2299] The server maintains a database of digital picture books. This database stores image data, text data, audio files, etc. of picture books. When a request from a user (child) reaches the server via a terminal, the server searches the database for the relevant picture book data and retrieves it. The retrieved data is then converted into an appropriate format and sent to the terminal.

[2300] Terminal

[2301] The device displays the picture book data received from the server in an interactive interface. The received data is decoded, and the image data is displayed in an image viewer, and the text data is displayed in a text viewer. The user can turn pages or play character audio guides by tapping or swiping the screen.

[2302] User

[2303] The user selects the picture book they want to read from the list displayed on the device screen and taps or swipes the screen to view the contents. For example, if the user selects the picture book "Little Red Riding Hood," the server retrieves Little Red Riding Hood's data from the database and sends it to the device. Each time the user taps, the page turns, and an audio guide is played if necessary.

[2304] Learning support features

[2305] server

[2306] The server receives questions from users and analyzes them using a natural language processing engine. Based on the analyzed questions, it retrieves appropriate learning materials from a database and generates the optimal answer using a generative AI model. The generated answer is then sent to the device.

[2307] Terminal

[2308] The device sends the question entered by the user to the server, and the answer received from the server is displayed on the interface. For example, if a user enters "I don't know how to calculate the area of ​​a triangle," the server will generate an answer such as "The area of ​​a triangle can be calculated by multiplying the length of the base by the height and dividing by 2." The user can enter additional information until they are satisfied.

[2309] User

[2310] Users can enter questions into a text field on the device, see the answers displayed, and optionally enter additional questions to seek further answers.

[2311] Emotional support

[2312] server

[2313] The server receives emotion-based messages from users, analyzes them using an emotion analysis engine, and generates an appropriate response message using a generative AI model based on the analysis results, which is then sent to the device.

[2314] Terminal

[2315] The device sends the worries and concerns entered by the user to the server and displays the response message received from the server. For example, if the user enters "I'm lonely because I haven't made any friends at school recently," the server will generate and send a response message such as "That's sad. But there are opportunities to make new friends. Do you have any favorite games or activities?"

[2316] User

[2317] The user inputs their concerns or worries into the terminal, checks the response message that is displayed, and continues the dialogue by inputting the message again if necessary.

[2318] This invention provides an environment where children can grow and have fun through learning using digital devices by providing digital picture books, learning support, and emotional support in an integrated manner.

[2319] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2320] Providing digital picture books

[2321] Step 1:

[2322] server

[2323] The server maintains a database of digital picture books. The database stores image data, text data, audio files, etc. of picture books. When the server starts up, it reads this data and prepares it. Input: All data in the database. Output: Prepared digital picture book data.

[2324] Step 2:

[2325] server

[2326] The server receives a digital picture book request from the user (child). The request includes the user ID and the title or ID of the picture book they want to read. Input: Request from the user. Output: Identification information of the requested picture book.

[2327] Step 3:

[2328] server

[2329] The server searches and retrieves the corresponding digital picture book data from the database based on the identification information. Data is extracted efficiently using SQL queries. Input: Identification information. Output: Retrieved picture book data.

[2330] Step 4:

[2331] server

[2332] The server converts the acquired picture book data into an appropriate format such as JSON and sends it to the terminal. This is where the data is encoded. Input: Acquired picture book data. Output: Formatted picture book data.

[2333] Step 5:

[2334] Terminal

[2335] The device receives the digital picture book data sent from the server. The received data is encoded in JSON format or similar. Input: Data sent from the server. Output: Received encoded data.

[2336] Step 6:

[2337] Terminal

[2338] The terminal decodes the data it receives and converts it into a format that can be displayed in the interface. Image data is displayed in an image viewer, text in a text viewer. Input: The encoded data. Output: The data that can be displayed in the interface.

[2339] Step 7:

[2340] User

[2341] The user selects the book they want to read from the list of digital books displayed on the device screen and confirms the request. They then enjoy the content by turning the pages using on-screen operations (tapping and swiping). Input: Device operations (tapping and swiping). Output: Display of pages and playback of audio guide.

[2342] Learning support features

[2343] Step 1:

[2344] User

[2345] The user types a question into a text field on the device. The input is related to a learning topic that they do not understand. Input: Learning question. Output: The entered question text.

[2346] Step 2:

[2347] Terminal

[2348] The terminal sends the question entered by the user to the server. Input: Question from the user. Output: Question text sent to the server.

[2349] Step 3:

[2350] server

[2351] The server receives the question and analyzes the question using a natural language processing engine. Input: Received question text. Output: Analyzed question content.

[2352] Step 4:

[2353] server

[2354] The server retrieves appropriate learning materials from the database based on the analyzed question. It then uses a generative AI model to generate the optimal answer. Input: Analyzed question. Output: Generated answer text.

[2355] Step 5:

[2356] server

[2357] The server sends the generated answer to the terminal. The answer is encoded in JSON format or similar. Input: Generated answer text. Output: Encoded answer data.

[2358] Step 6:

[2359] Terminal

[2360] The terminal receives the response sent from the server and decodes it into a format that can be displayed. Input: Response data from the server. Output: Displayable response text.

[2361] Step 7:

[2362] User

[2363] The user checks the answer displayed on the terminal, and if they are not satisfied, they can enter an additional question. Input: Answer text, additional question. Output: New question text.

[2364] Emotional support

[2365] Step 1:

[2366] User

[2367] The user inputs the problem or problem content into the text field of the terminal. Input: Problem or problem content. Output: Input text message.

[2368] Step 2:

[2369] Terminal

[2370] The terminal sends the message entered by the user to the server. Input: Text message from the user. Output: Message sent to the server.

[2371] Step 3:

[2372] server

[2373] The server analyzes the received message with a sentiment analysis engine to identify the user's sentiment. Input: Received text message. Output: Identified sentiment data.

[2374] Step 4:

[2375] server

[2376] The server generates an appropriate response message using a generative AI model based on the identified emotion data. Input: Identified emotion data. Output: Generated response message.

[2377] Step 5:

[2378] server

[2379] The server sends the generated response message to the terminal. The response message is sent in text format. Input: Generated response message. Output: Response message sent to the terminal.

[2380] Step 6:

[2381] Terminal

[2382] The terminal displays the response message received from the server to the user. Input: Received response message. Output: Displayed response message.

[2383] Step 7:

[2384] User

[2385] The user checks the response message displayed on the terminal and continues the dialogue by inputting another message if necessary. Input: Input of a message on the terminal. Output: A new message to the server.

[2386] In this system, various data exchanges and processing are carried out between the server, terminals, and users to effectively realize the functions of providing digital picture books, learning support, and emotional support.

[2387] (Application example 1)

[2388] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2389] To support the learning and emotional development of young children, there is a need for systems that provide a consistent approach to reading picture books, learning support, and emotional support. However, no existing systems offer these multiple functions in an integrated manner; most are specialized for individual purposes. Therefore, there is a need for a comprehensive system that can maintain children's interest, deepen their understanding of learning, and provide emotional support in educational settings such as cram schools.

[2390] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2391] In this invention, the server includes means for providing digital picture books to young children, means for displaying digital picture book data received from the server, means for turning pages in response to user operations, means for playing audio files, means for children at cram schools to input questions using smartphones and send them to the server, means for displaying answers received from the server, and means for analyzing the user's emotions and generating and sending appropriate response messages. This makes it possible to provide appropriate answers to learning questions and emotional support while maintaining children's interest through the provision of digital picture books.

[2392] "Young children" refers primarily to young people from preschool to early elementary school age, an age group that is particularly interested in digital picture books and learning support.

[2393] A "digital picture book" is a picture book that is provided in digital format rather than on paper and can be viewed on an electronic device.

[2394] A "server" is a computer system that centrally manages digital picture book data, learning answers, emotion analysis messages, etc., and communicates with user terminals.

[2395] "User operation" refers to input actions that a user performs on the system through a terminal, such as tapping or swiping on the screen.

[2396] The "means for turning pages" is a mechanism that provides a function for turning pages of a digital picture book forward or backward in response to user operations.

[2397] "Audio file" refers to audio recording data, which is used to provide audio readings and explanations of digital picture books.

[2398] A "cram school" is an educational facility where children can receive learning support outside of class.

[2399] "Means for inputting questions" refers to the function that allows users to input questions about their studies as text information into the terminal.

[2400] "Sentiment analysis" refers to a technology that analyzes text data from a user and identifies the emotions the user is feeling.

[2401] A "response message" is a message that is generated based on the results of emotion analysis and sent to the user.

[2402] "Display means" refers to the function for displaying information such as digital picture book data, answers, and response messages sent from the server on the screen of the user's terminal.

[2403] This invention is a system that allows young children to learn while having fun with digital picture books and also receive emotional support. This system consists of a server, a terminal operated by the user, and the user. The configuration and operation of the system are described in detail below.

[2404] Providing digital picture books

[2405] server

[2406] The server maintains a database of digital picture books and is responsible for transmitting the corresponding digital picture book data when requested by a user (young child). The server is equipped with functions for efficiently managing and distributing the metadata and content of digital picture books.

[2407] Terminal

[2408] The device operated by the user (smartphone or tablet) receives the digital picture book data from the server and displays it in an interactive interface, allowing the user to turn pages by tapping or swiping the screen and play audio guides related to characters and scenes.

[2409] User

[2410] Users select the picture book they want to read from the options on the screen, advance through the pages through operations, and enjoy the contents of the picture book using audio guidance.

[2411] Learning support features

[2412] server

[2413] The server receives a question from the user, analyzes the question, retrieves appropriate learning materials from a database, and generates an answer, which is then sent to the terminal.

[2414] Terminal

[2415] The terminal transmits the questions entered by the user to the server, receives and displays the answers from the server, and if the user enters additional questions, the additional questions can be transmitted to the server again.

[2416] User

[2417] Users can enter questions about content they did not understand in class or at a cram school into the device, check the answers provided, and enter additional questions as needed.

[2418] Emotional support

[2419] server

[2420] The server receives messages from users, analyzes their emotions using AI, and generates an appropriate response message based on the analysis results, which is then sent to the device.

[2421] Terminal

[2422] The terminal inputs the user's worries or feelings of loneliness and sends them to the server. It receives and displays a response message from the server, and if the user inputs another message, it sends that message to the server again.

[2423] User

[2424] The user inputs their concerns or feelings of loneliness into the terminal, confirms the response message provided, and inputs an additional message to continue the dialogue if necessary.

[2425] Hardware and software used

[2426] Hardware used

[2427] Smartphones and tablets: Used by users as portable devices.

[2428] Server: A central management system that manages the digital picture book database and AI responses.

[2429] Software used

[2430] Flutter / React Native: Used to build and operate the UI of smartphone apps.

[2431] requests (Python library): A library for enabling communication between the server and the terminal.

[2432] Generative AI model: AI technology used for sentiment analysis and response message generation.

[2433] Examples and prompts

[2434] As a specific example, if the user selects the digital picture book "Forest Stories," the terminal operates as follows.

[2435] 1. The user selects the "Forest Tales" digital picture book.

[2436] 2. The terminal sends a request to the server, receives the digital picture book data from the server, and displays it.

[2437] 3. When the user taps to turn the page, the appropriate page is displayed and the associated audio plays.

[2438] Example prompt sentence:

[2439] Get and display the page data for the digital picture book "Forest Stories." URL: https: / / example.com / digital_books / 001

[2440] In this way, the present invention provides a comprehensive system that provides relevant answers to educational questions and emotional support while retaining the interest of young children.

[2441] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2442] Step 1:

[2443] The user uses the terminal to make a digital picture book request.

[2444] Input: A request for a digital picture book based on the user's selection

[2445] Output: Request sent to server

[2446] How it works: The user selects a picture book, such as "Forest Stories," from the interface on the device, and the information is sent to the server.

[2447] Step 2:

[2448] The server receives the request, acquires the corresponding digital picture book data, and transmits it.

[2449] Input: Digital picture book request from user device

[2450] Output: Sending digital picture book data

[2451] Operation: The server retrieves the requested picture book data from the digital picture book database and sends it to the user's terminal.

[2452] Step 3:

[2453] The terminal displays the digital picture book data received from the server.

[2454] Input: Digital picture book data from the server

[2455] Output: Display a digital picture book on the screen

[2456] Operation: The device displays the received digital picture book data on an interactive interface, allowing the user to turn pages and play audio.

[2457] Step 4:

[2458] The user inputs a question related to the study and sends it to the server via the terminal.

[2459] Input: User's question

[2460] Output: Sending the query data to the server

[2461] How it works: The user enters a study question into the terminal, and the data is sent to the server.

[2462] Step 5:

[2463] The server analyzes the received question, retrieves appropriate learning materials from a database, and generates an answer.

[2464] Input: Question data from the user

[2465] Output: Generated training answers

[2466] How it works: The server analyzes the received question, retrieves relevant learning material from the database, and generates an appropriate answer.

[2467] Step 6:

[2468] The terminal displays the answer received from the server to the user.

[2469] Input: Learning response data from the server

[2470] Output: Display the answer to the user

[2471] Operation: The terminal displays the answer received from the server to the user, who then confirms the content.

[2472] Step 7:

[2473] If the user enters more questions, the questions are sent to the server again.

[2474] Input: User's additional question

[2475] Output: Sending additional questions to the server

[2476] How it works: The user enters a follow-up question into the terminal, and the data is sent back to the server.

[2477] Step 8:

[2478] The user inputs the emotional worries and consultation contents and transmits them to the server.

[2479] Input: User's sentiment message

[2480] Output: Sending emotion message to the server

[2481] How it works: The user enters a message about their worries or loneliness into the device, and the data is sent to the server.

[2482] Step 9:

[2483] The server analyzes the sentiment of the received message and generates an appropriate response message.

[2484] Input: Emotional message from the user

[2485] Output: The generated response message

[2486] How it works: The server analyzes the received message using an AI model and generates an appropriate emotional response message.

[2487] Step 10:

[2488] The terminal displays the response message received from the server to the user.

[2489] Input: Response message from the server

[2490] Output: Display a response message to the user

[2491] Operation: The terminal displays the received response message to the user, who confirms it.

[2492] Step 11:

[2493] The user inputs a message again to continue the dialogue and sends it to the server.

[2494] Input: User's additional emotion message

[2495] Output: Send additional emotion message to server

[2496] Operation: The user enters a message into the terminal to continue the dialogue, and the data is sent again to the server.

[2497] In this way, the present invention provides a system that supports the user's learning and also provides emotional support at the same time.

[2498] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2499] This invention is a system that provides digital picture books to young children, supports children and students in gaining a deeper understanding of the learning content, and provides emotional support when they feel distressed or lonely. By combining a specific emotion engine, the system recognizes the user's emotions and provides appropriate responses accordingly.

[2500] Providing digital picture books

[2501] server

[2502] 1. Maintain a database of digital picture books.

[2503] 2. When a request is received from the user (child), the corresponding digital picture book data is obtained and sent to the terminal.

[2504] Terminal

[2505] 1. Based on the digital picture book data received from the server, it is displayed to the user in an interactive interface.

[2506] 2. Detect user taps and swipes and perform page-turning processing accordingly.

[2507] 3. Play the audio file according to the user's instructions.

[2508] User

[2509] 1. Select the picture book you want to read from the options on the screen.

[2510] 2. Enjoy the content by tapping or swiping the screen to turn the pages.

[2511] 3. Tap to play audio descriptions related to characters and scenes.

[2512] Learning support features

[2513] server

[2514] 1. Receive a question from the user and analyze its content.

[2515] 2. Retrieve appropriate learning materials from the database and generate answers.

[2516] 3. Send the generated answer to the device.

[2517] Terminal

[2518] 1. The question entered by the user is sent to the server.

[2519] 2. Receive the response from the server and display it to the user.

[2520] 3. If the user enters additional questions, the additional questions are sent to the server again.

[2521] User

[2522] 1. Enter questions about content you didn't understand in class or at cram school.

[2523] 2. Check the answer displayed on the device.

[2524] 3. Enter additional questions as needed until you are satisfied.

[2525] Emotional Support and Emotion Engine

[2526] server

[2527] 1. Receive the consultation content sent by the user.

[2528] 2. The message is analyzed using a sentiment analysis engine to identify the user's sentiment.

[2529] 3. Generate an appropriate response message that corresponds to the user's emotions.

[2530] 4. Send the generated response message to the terminal.

[2531] 5. Accumulates user's past emotional data to improve the accuracy of future response messages.

[2532] Terminal

[2533] 1. The user's concerns and questions are sent to the server.

[2534] 2. Receive the response message from the server and display it to the user.

[2535] 3. If the user enters a message again, the message is sent to the server again.

[2536] User

[2537] 1. Enter what is bothering you or making you feel lonely.

[2538] 2. Check the response message displayed on the device.

[2539] 3. If necessary, enter an additional message to continue the conversation.

[2540] For example, if a user types, "I feel lonely because I haven't made any friends recently," the server's emotion analysis engine will identify the user's emotion as "loneliness." The server will then generate a response message such as, "That's tough. But you have a chance to make new friends. Maybe you should try looking for someone who has a common hobby with you," and send it to the device. If the user receives this message and continues the dialogue, it can alleviate the user's feelings of loneliness and worries.

[2541] The introduction of this system can improve children's creativity, learning efficiency, and emotional support, promoting their overall development.

[2542] The processing flow will be explained below.

[2543] Providing digital picture books

[2544] Server Processing

[2545] Step 1:

[2546] The server receives a request from a user.

[2547] Step 2:

[2548] The server retrieves the requested digital picture book data from the database.

[2549] Step 3:

[2550] The server transmits the acquired data to the terminal.

[2551] Terminal handling

[2552] Step 1:

[2553] The terminal receives the digital picture book data transmitted from the server.

[2554] Step 2:

[2555] The terminal analyzes the received data and generates an interactive interface.

[2556] Step 3:

[2557] The device detects the user's taps and swipes and performs page-turning processing accordingly.

[2558] Step 4:

[2559] The terminal plays the audio file in response to a user instruction.

[2560] User Actions

[2561] Step 1:

[2562] The user selects the picture book they want to read from the options on the screen.

[2563] Step 2:

[2564] Users tap or swipe the screen to turn pages.

[2565] Step 3:

[2566] The user can play audio guides by tapping on characters or scenes.

[2567] Learning support features

[2568] Server Processing

[2569] Step 1:

[2570] The server receives a question sent by a user.

[2571] Step 2:

[2572] The server analyzes the received question using an NLP (natural language processing) engine.

[2573] Step 3:

[2574] The server retrieves appropriate learning materials from a database and generates answers.

[2575] Step 4:

[2576] The server sends the generated response to the terminal.

[2577] Terminal handling

[2578] Step 1:

[2579] The terminal transmits the question entered by the user to the server.

[2580] Step 2:

[2581] The terminal receives the response from the server.

[2582] Step 3:

[2583] The terminal displays the received response to the user.

[2584] Step 4:

[2585] If the user inputs more questions, the terminal transmits the additional questions to the server again.

[2586] User Actions

[2587] Step 1:

[2588] The user inputs questions about content that he or she did not understand in class or at a cram school.

[2589] Step 2:

[2590] The user checks the answer displayed on the terminal.

[2591] Step 3:

[2592] The user continues to enter additional questions until satisfied.

[2593] Emotional Support and Emotion Engine

[2594] Server Processing

[2595] Step 1:

[2596] The server receives the consultation content sent from the user.

[2597] Step 2:

[2598] The server analyzes the received message using an emotion analysis engine to identify the user's emotion.

[2599] Step 3:

[2600] The server generates an appropriate response message according to the user's emotions based on the results of the emotion analysis.

[2601] Step 4:

[2602] The server sends the generated response message to the terminal.

[2603] Step 5:

[2604] The server accumulates the user's past emotional data and improves the accuracy of future response messages.

[2605] Terminal handling

[2606] Step 1:

[2607] The terminal transmits the worries and consultation details input by the user to the server.

[2608] Step 2:

[2609] The terminal receives the response message from the server.

[2610] Step 3:

[2611] The terminal displays the received response message to the user.

[2612] Step 4:

[2613] If the user inputs a message again, the terminal transmits the additional message to the server again.

[2614] User Actions

[2615] Step 1:

[2616] The user inputs what is bothering them or making them feel lonely.

[2617] Step 2:

[2618] The user checks the response message displayed on the terminal.

[2619] Step 3:

[2620] The user may optionally enter additional messages to continue the dialogue.

[2621] Example 2

[2622] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2623] In today's educational environment, there is a need for digital picture books that allow young children to deepen their knowledge while having fun learning, and for methods that encourage students to ask questions about the content they are learning and deepen their understanding. It is also important to provide appropriate emotional support when children feel distressed or lonely. There is a lack of systems that can meet these needs and promote children's holistic development.

[2624] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2625] In this invention, the server includes an information transmission device means for storing data, a display device means for displaying data received from the information transmission device, a device means for switching pages in response to operations, and a device means for playing audio data, which enables provision of digital picture books, understanding of learning content, and emotional support.

[2626] "Data" means electronic records of specific information or content.

[2627] An "information transmitting device" refers to a device that holds data and transmits it to a terminal or display device.

[2628] A "display device" is a device for visually displaying received data to a user.

[2629] An "operation" refers to an input action that a user performs using a terminal or a display device.

[2630] A "page-switching device" is a device that has the function of turning pages of a digital picture book or the like in response to a user's operation.

[2631] "Audio data" means electronic recordings of audio information.

[2632] A "device that plays back audio data" is a device that has the function of receiving and playing back audio data.

[2633] A "question input device" is a device that allows a user to input their own questions or doubts.

[2634] The "device for additional input" refers to a device that allows the user to input information again.

[2635] An "analyzing device" is a device that has the function of analyzing an input question or message and generating an appropriate answer or response.

[2636] An "emotion-based message" is a message whose content reflects the user's emotional state.

[2637] A "response message" is a response generated based on the analyzed content.

[2638] The "display device" refers to a device that displays the generated answer or response message in an easy-to-see manner for the user.

[2639] The present invention is a system that provides digital picture books to young children, supports children and students in gaining a deeper understanding of the learning content, and provides emotional support when they feel distressed or lonely. The following describes specific embodiments of the invention.

[2640] Providing digital picture books

[2641] server

[2642] The server maintains a database of digital picture books. When it receives a request from a user (child), it retrieves the corresponding digital picture book data from the database and sends it to the device. For this reason, the server has high-performance database software (e.g., MySQL) and data transmission functions.

[2643] Terminal

[2644] The terminal receives the digital picture book data from the server and displays it to the user using an interactive interface. It is typically a tablet or smartphone, and is equipped with an LCD display or touchscreen. The terminal detects the user's taps and swipes and processes the pages accordingly. It also plays audio data in response to user instructions. This is achieved using a built-in speaker or earphones.

[2645] User

[2646] When reading a digital picture book, users select the book they want from a list on the screen. They turn the pages by tapping or swiping, and when they tap on parts related to characters or scenes, audio guides and sound effects are played.

[2647] Learning support features

[2648] server

[2649] The server receives a question from the user and analyzes it using a natural language processing engine (e.g., Google Natural Language API). It retrieves appropriate learning materials from a database and generates an answer. The server is equipped with data analysis and transmission functions to send the generated answer to the device.

[2650] Terminal

[2651] The terminal sends the question entered by the user to the server, receives the answer from the server and displays it to the user, and if the user enters another question, sends the additional question back to the server and displays related additional learning materials.

[2652] User

[2653] Users can enter questions about things they didn't understand in class or at a cram school. They can check the answers and explanations displayed on the device to deepen their understanding. If necessary, they can enter further questions and repeat the process until they are satisfied.

[2654] Emotional Support and Emotion Engine

[2655] server

[2656] The server receives the consultation content sent by the user. It analyzes the message using an emotion analysis engine (e.g., IBM Watson Tone Analyzer) to identify the user's emotion. It generates a response message based on the user's emotion and sends it to the device. It also accumulates data on the user's past emotions to improve accuracy.

[2657] Terminal

[2658] The terminal sends the worries or consultation details entered by the user to the server. It receives a response message from the server and displays it to the user. If the user wants to enter a message again, it can send that message to the server again.

[2659] User

[2660] The user inputs what is bothering them or what is making them feel lonely. They check the response message displayed on the terminal and, if necessary, input an additional message to continue the conversation.

[2661] Specific examples and examples of input to generative AI models

[2662] For example, if a user types, "I'm lonely because I haven't made any friends recently," the server will use its emotion analysis engine to identify the user's emotion as "loneliness." It will then generate a response message such as, "That's tough. But you have a chance to make new friends. Maybe you should try looking for someone who has common hobbies with you," and send it to the device. By receiving this message and continuing the dialogue, the user can ease their feelings of loneliness and distress.

[2663] Example of input prompt for generative AI model

[2664] 1. "I'm lonely and can't make friends. Please tell me what to do."

[2665] 2. "I'm having trouble understanding some of the things I'm studying at school. Can you explain?"

[2666] 3. "The picture book I want to read right now is one with lots of animals in it."

[2667] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2668] Providing digital picture books

[2669] Step 1:

[2670] The server maintains a database of digital picture books, which stores data on a plurality of digital picture books.

[2671] Input: A digital picture book request from a user

[2672] Data processing: Analyze user requests and identify the corresponding digital picture book data

[2673] Output: Identified digital picture book data

[2674] Specific operation: The server receives the HTTP request, executes an SQL query based on it, and retrieves the required data from the database.

[2675] Step 2:

[2676] The server transmits the corresponding digital picture book data to the terminal.

[2677] Input: Identified digital picture book data

[2678] Data processing: Digital picture book data is converted into packets according to communication protocols (e.g. HTTP, HTTPS).

[2679] Output: Data packets sent to the device

[2680] Specific operation: The server sends the digital picture book data encoded using HTTPS to the terminal.

[2681] Step 3:

[2682] The terminal displays the digital picture book data received from the server to the user on an interface.

[2683] Input: Digital picture book data received from the server

[2684] Data processing: Analyzes received data and converts it into a display format

[2685] Output: A digital picture book displayed in a form visible to the user

[2686] Specific operation: The device analyzes the images and text data of the digital picture book and displays them on the screen.

[2687] Step 4:

[2688] The device detects the user's taps and swipes and performs page-turning processing accordingly.

[2689] Input: User taps and swipes

[2690] Data processing: Analyzes the operation data and converts the data on the next page into a display format

[2691] Output: Next page displayed

[2692] Specific operation: The gesture engine analyzes touch operations and executes page transition animations.

[2693] Step 5:

[2694] The terminal plays the audio file in response to a user instruction.

[2695] Input: User's voice instructions (tap operation, etc.)

[2696] Data processing: Read and play audio files based on instructions

[2697] Output: Audio is played

[2698] Specific behavior: The device loads the audio file from storage and plays it using the media player.

[2699] Learning support features

[2700] Step 1:

[2701] The server receives a query from the user.

[2702] Input: The question the user types into the terminal

[2703] Data processing: Transfer received questions to the server

[2704] Output: Query data to the server

[2705] Specific operation: Sends the question data to the server as an HTTP POST request.

[2706] Step 2:

[2707] The server analyzes the received question using a natural language processing engine.

[2708] Input: Received question data

[2709] Data processing: Analysis using a natural language processing engine

[2710] Output: Analysis results

[2711] Specific operation: The server uses the Google Natural Language API to analyze the question data and extract keywords and intent.

[2712] Step 3:

[2713] The server retrieves appropriate learning materials from a database and generates answers.

[2714] Input: Analysis results of natural language processing

[2715] Data processing: Run database queries to identify appropriate learning materials and generate answers.

[2716] Output: Generated response data

[2717] Specific operation: Executes an SQL query to retrieve teaching material data and generates answer sentences using a template.

[2718] Step 4:

[2719] The server sends the generated response to the terminal.

[2720] Input: Generated response data

[2721] Data processing: Converts response data into packets according to the communication protocol

[2722] Output: Data packets sent to the device

[2723] Specific operation: Sends response data to the terminal via an HTTPS request.

[2724] Step 5:

[2725] The terminal displays the response received from the server to the user.

[2726] Input: Response data received from the server

[2727] Data processing: Converting response data into a display format

[2728] Output: The answer that is displayed to the user

[2729] Specific operation: The device renders and displays the received text data on the screen.

[2730] Step 6:

[2731] If the user has further questions, they enter them.

[2732] Input: User's additional question

[2733] Data processing: Send additional questions to the server

[2734] Output: Additional question data to the server

[2735] Specific operation: Sends the additional question to the server again as an HTTP POST request.

[2736] Emotional Support and Emotion Engine

[2737] Step 1:

[2738] The server receives the consultation content from the user.

[2739] Input: The consultation content entered by the user into the terminal

[2740] Data processing: Transfer received consultation details to the server

[2741] Output: Consultation content data to the server

[2742] Specific operation: Consultation content data is sent to the server as an HTTP POST request.

[2743] Step 2:

[2744] The server analyzes the consultation content using an emotion analysis engine and identifies the user's emotions.

[2745] Input: Received consultation content data

[2746] Data processing: Analysis using a sentiment analysis engine

[2747] Output: Emotion analysis results

[2748] Specific operation: Uses IBM Watson Tone Analyzer to analyze text data and extract emotions.

[2749] Step 3:

[2750] The server generates a response message according to the user's emotion.

[2751] Input: Sentiment analysis results

[2752] Data processing: Generate appropriate response messages

[2753] Output: Response message data

[2754] Specific operation: Generate a response message from a template based on the analysis results.

[2755] Step 4:

[2756] The server sends the generated response message to the terminal.

[2757] Input: Response message data

[2758] Data processing: Converts message data into packets according to the communication protocol

[2759] Output: Data packets sent to the device

[2760] Specific operation: Sends response message data to the terminal via an HTTPS request.

[2761] Step 5:

[2762] The terminal displays the response message received from the server to the user.

[2763] Input: Response message data received from the server

[2764] Data processing: Convert response message data into a display format

[2765] Output: The response message that is displayed to the user

[2766] Specific operation: The device renders and displays the received text data on the screen.

[2767] Step 6:

[2768] The user may optionally enter additional messages to continue the dialogue.

[2769] Input: User's additional message

[2770] Data processing: Send additional messages to the server

[2771] Output: Additional message data to the server

[2772] Specific behavior: Sends the additional message to the server again as an HTTP POST request.

[2773] (Application example 2)

[2774] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2775] Although conventional digital picture books are available for young children, they are limited in terms of emotional support, and do not provide sufficient emotional support to children. Furthermore, in terms of learning support, the interaction required to obtain appropriate answers to questions is limited. This has led to issues such as not being able to fully improve children's creativity and learning efficiency.

[2776] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for providing a digital picture book, means for identifying the user's emotion using an emotion analysis engine, and means for generating a response message corresponding to the user's emotion. This allows children to enjoy the digital picture book, become emotionally stable through emotional support, and obtain appropriate answers to their questions, thereby improving learning efficiency.

[2777] A "digital picture book" is a picture book in electronic format that can be accessed by young children via computers or mobile devices.

[2778] A "server" is a computer system that stores and processes digital picture book data and user requests, and provides the necessary information.

[2779] An "emotion analysis engine" is a system that has algorithms and functions for analyzing messages from users and identifying their emotions.

[2780] A "response message" is a message generated based on the user's emotional state, intended to provide emotional support and appropriate responses.

[2781] A "question" is an inquiry that a user (child or student) inputs to the server regarding learning content, etc.

[2782] An "answer" is appropriate information or explanation that the server generates after analyzing the user's question.

[2783] "Interaction" refers to the process of operations and inputs that a user makes in order to interact with a system, and includes, in particular, actions such as turning pages, entering questions, and sending emotional messages.

[2784] An "audio file" is digital audio data used when reading picture books or playing audio guides.

[2785] A "prompt sentence" is an input sentence that allows a user to communicate with a system, and is text that includes specific instructions or questions.

[2786] A "generative AI model" is an artificial intelligence algorithm used to realize functions such as reading digital picture books, learning support, and emotion analysis.

[2787] MODE FOR CARRYING OUT THE INVENTION

[2788] This system provides digital picture books to young children, helps them to understand the content they are learning deeply, and provides emotional support through emotional support. The system is structured through interactions between a server, terminals, and users.

[2789] Providing digital picture books

[2790] server

[2791] The server maintains a database of digital picture books. Upon receiving a request from a user, the server retrieves the corresponding digital picture book data and transmits it to the terminal.

[2792] Terminal

[2793] The device displays the digital picture book data received from the server to the user in an interactive interface, detects the user's taps and swipes, turns the pages accordingly, and plays audio files in response to the user's instructions.

[2794] User

[2795] Users can select the picture book they want to read from the options on the screen, tap or swipe the screen to turn the pages and enjoy the story. Tap on a character or scene to play an audio guide.

[2796] Learning support features

[2797] server

[2798] The server receives a question from the user, analyzes its content, retrieves appropriate learning materials from a database, generates an answer, and sends the answer to the device.

[2799] Terminal

[2800] The terminal sends the question entered by the user to the server, receives the answer from the server and displays it to the user, and if the user enters another question, sends the additional question to the server again.

[2801] User

[2802] Users can enter questions about content they did not understand in class or at a cram school, and check the answers displayed on the device. They can enter additional questions as needed until they are satisfied.

[2803] Emotional Support and Sentiment Analysis Engine

[2804] server

[2805] The server receives the consultation content sent by the user. It analyzes the message using an emotion analysis engine to identify the user's emotion. It generates an appropriate response message corresponding to the user's emotion and sends it to the device. The server also accumulates the user's past emotion data to improve the accuracy of future response messages.

[2806] Terminal

[2807] The terminal sends the worries or consultation details entered by the user to the server. It receives a response message from the server and displays it to the user. If the user wants to enter another message, it sends that message to the server again.

[2808] User

[2809] The user inputs what is bothering them or what is making them feel lonely, and checks the response message displayed on the terminal. If necessary, the user can input an additional message to continue the conversation.

[2810] System specific example

[2811] For example, if a user types, "I feel lonely because I haven't made any friends recently," the system's emotion analysis engine will identify the user's emotion as "loneliness." The server will then generate a response message such as, "That's sad. But you have a chance to make new friends. Maybe you should try looking for people who have common hobbies," and send it to the device.

[2812] Prompt Sentence Examples

[2813] Digital Picture Book ID: 12345

[2814] Question: "What is water made of?"

[2815] Emotional support: "I haven't been able to make any friends lately and I feel lonely."

[2816] This system uses an emotion analysis engine and generative AI model to provide quick and appropriate responses to users' emotions and questions. Users can input prompts and use functions such as digital picture book viewing, learning support, and emotional support in response. This is expected to improve children's creativity and learning efficiency.

[2817] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2818] Step 1:

[2819] The user selects a digital picture book

[2820] The user selects the digital picture book they want to read from the options displayed on the device screen. The ID of the selected picture book is entered into the device. Based on this input, the device sends a request for the picture book data to the server.

[2821] Step 2:

[2822] The server sends the digital picture book data

[2823] When the server receives the request, it retrieves the digital picture book data from the database and sends the retrieved picture book data to the terminal as output.

[2824] Step 3:

[2825] The device displays the digital picture book.

[2826] The terminal generates an interactive user interface based on the digital picture book data received from the server, and displays it so that the user can manually turn pages and play audio.

[2827] Step 4:

[2828] User enters study question

[2829] The user inputs a question related to the learning content into the terminal, which then transmits the input question to the server.

[2830] Step 5:

[2831] The server analyzes the question and generates the answer

[2832] The server analyzes the question sent by the user, retrieves appropriate learning materials from the database, and generates an answer, which is then sent to the terminal as output.

[2833] Step 6:

[2834] The device displays the answer

[2835] The terminal displays the answer received from the server to the user, who can then confirm the displayed answer and enter a follow-up question if necessary.

[2836] Step 7:

[2837] The user inputs emotional consultation content.

[2838] The user inputs emotional concerns such as worries and loneliness into the terminal, and the inputted concerns are sent to the server by the terminal.

[2839] Step 8:

[2840] The server analyzes the emotions and generates a response message.

[2841] The server analyzes the consultation content sent by the user using an emotion analysis engine to identify the emotion. Based on the identified emotion, it generates an appropriate response message. The generated response message is sent as output to the terminal.

[2842] Step 9:

[2843] The terminal displays a response message to the user.

[2844] The terminal displays the response message received from the server to the user. The user confirms the displayed response message and continues the dialogue by inputting a message again if necessary.

[2845] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2846] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2847] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2848] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2849] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arran...

Claims

1. A means of providing digital picture books to young children, means for displaying the digital picture book data received from the server; means for turning pages in response to a user's operation; means for playing audio files; A system including:

2. A means for students to input questions; means for sending a query to a server; a means by which the server analyzes the question and generates an appropriate answer; means for transmitting the generated answer to the user; 10. The system of claim 1, further comprising means for the user to input a question.

3. a means for the server to receive an emotion-based message from the user; means for analyzing received messages; means for generating a response message according to the emotion; means for sending a response message to the user; 2. The system of claim 1, further comprising means for the user to input a message again and continue the dialogue.

Images