System

Abstract

A system is provided.SOLUTION: This system includes a means for analyzing the contents of an electronic book and extracting important information, a means for automatically generating a quiz on the basis of the extracted information, a means for notifying a user of the generated quiz at prescribed timing, and a means for collecting and evaluating the answer of the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Many people have the problem of forgetting the contents of books they read in a short period of time, preventing them from fully achieving their learning goals. Specifically, data shows that people generally forget 56% of what they learned after one hour, and 79% after one month. This problem reduces the effectiveness of reading and learning, making it difficult to consolidate the information. Furthermore, traditional rereading methods lack an active learning approach and do not solidify the information in memory sufficiently. Therefore, more effective review methods are needed. [Means for solving the problem]

[0005] The present invention solves the above problems by the following means. Specifically, a system is provided that includes a means for analyzing the contents of an e-book and extracting important information, a means for automatically generating quizzes based on the extracted information, a means for notifying the user of the generated quizzes at a predetermined time, and a means for collecting and evaluating the user's answers. This system allows the user to periodically review important information in the form of quizzes, promoting memory consolidation through an active learning process. Furthermore, by setting a quiz notification schedule and notifying the user of quizzes at a predetermined time, the timing of review can be optimized, resulting in more effective memory consolidation. Furthermore, by providing a means for evaluating the answers entered by the user to the quizzes and notifying the user of the evaluation results, the system provides feedback on the user's level of understanding and improves motivation for continuous learning.

[0006] "Means for analyzing the contents of e-books and extracting important information" refers to a function that analyzes the text data of e-books and automatically extracts key points and important concepts necessary for learning and memorization.

[0007] "Means for automatically generating quizzes based on extracted information" refers to a function that automatically creates quiz-style questions that users can answer from important information obtained through analysis.

[0008] The "means for notifying the user of the generated quiz at a predetermined timing" refers to a function for notifying the user's terminal of the generated quiz based on a preset schedule.

[0009] "Means for collecting and evaluating user answers" refers to a function for collecting answers to quizzes given by users and evaluating their accuracy and level of understanding.

[0010] The "means for setting a quiz notification schedule" refers to a function for determining the optimal timing for notifying the user of a quiz and setting a notification schedule based on that timing.

[0011] The "means for notifying the user of the evaluation result" refers to a function for evaluating the answer result after the user answers the quiz and notifying the user of the evaluation feedback. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0013] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0016] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0017] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0018] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0020] [First embodiment]

[0021] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0022] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0025] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0028] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0032] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0033] This invention is a system that analyzes the contents of e-books, extracts important information, automatically generates quizzes, and allows readers to review the content. This system allows readers to retain the contents of books they have read for a long period of time. Specifically, the following process is carried out between the server, the terminal, and the user.

[0034] Server Processing

[0035] 1. Content analysis of e-books

[0036] The server receives the data of the e-book that the user has finished reading. It then analyzes the content of the e-book and extracts important information and key points. In this process, it uses natural language processing technology and text analysis algorithms to extract the essence of the text necessary for learning.

[0037] 2. Quiz Generation

[0038] The server automatically generates quizzes based on the extracted key points. For example, it creates specific questions such as "What are the basic principles of human-centered design?" and sets the correct answers. The generated quizzes are stored in a database.

[0039] 3. Set a notification schedule

[0040] The server sets a schedule for notifying users of quizzes, for example, setting it to notify users at specific times, such as after 2 days, 14 days, or 60 days, encouraging regular review and promoting knowledge retention.

[0041] Terminal handling

[0042] 1. Receive quiz notifications

[0043] The terminal receives the quiz notification sent from the server, which includes the generated quiz and the notification schedule.

[0044] 2. View the quiz

[0045] The device displays the received quiz notification to the user in a simple and easy-to-answer interface, such as a push notification on a smartphone or an in-app notification.

[0046] 3. Collecting user responses

[0047] The terminal allows the user to enter answers to the quiz and transmits the entered answers to the server.

[0048] User Action

[0049] 1. Check notifications

[0050] The user checks the notification from the device and becomes aware that there is a quiz. For example, when a notification arrives on the device two days later, the user checks it and answers the quiz.

[0051] 2. Answer the quiz

[0052] Users can interactively input answers to the displayed quiz questions. The input method is simple and intuitive, allowing answers to be entered by typing on the keyboard or by tapping.

[0053] Specific examples

[0054] For example, if a user finishes reading a book called "Design Thinking," the server analyzes the book's contents and extracts important key points, such as:

[0055] Human-Centered Design

[0056] Iterative Prototyping

[0057] The server then generates a quiz based on these key points:

[0058] "What are the basic principles of human-centered design?"

[0059] "What is the importance of iterative prototyping?"

[0060] The quiz will be sent to the user's device two days later. The user checks the notification and answers the quiz. The user's answers are sent to the server via the device, and the server evaluates the answers and notifies the user of the results. Through this process, the user can review the important content of the book and retain it in their long-term memory.

[0061] In this way, the system provides an environment in which readers can periodically review the contents of the book, thereby improving learning effectiveness.

[0062] The processing flow will be explained below.

[0063] Step 1:

[0064] When the user finishes reading an e-book, he or she presses a button on the device to record the end of reading, which notifies the system of the end of book reading event.

[0065] Step 2:

[0066] The device sends information about the e-book that the user has finished reading to the server. The sent information includes the user ID and the e-book ID.

[0067] Step 3:

[0068] Based on the received information, the server retrieves the contents of the e-book and begins analyzing it, using natural language processing technology to extract important key points and learning essences from the text.

[0069] Step 4:

[0070] The server automatically generates quizzes based on information extracted from the analysis results. The quizzes are created in the form of specific questions based on important key points. For example, a question might be generated such as, "What are the basic principles of human-centered design?"

[0071] Step 5:

[0072] The server saves the generated quiz in a database, along with the user ID, quiz content, and correct answer information.

[0073] Step 6:

[0074] The server sets a notification schedule for the quiz. The notification timing can be set to a specific schedule, such as 2 days, 14 days, or 60 days later.

[0075] Step 7:

[0076] When the set time comes, the server sends a quiz notification to the terminal, which includes the generated quiz and an interface for answering the question.

[0077] Step 8:

[0078] The terminal displays the received quiz notification to the user, who then checks the notification and opens an interface for answering the quiz.

[0079] Step 9:

[0080] The user inputs answers to quiz questions through the device interface, for example, "What are the basic principles of human-centered design?"

[0081] Step 10:

[0082] The terminal sends the user's answer to the server. The sent data includes the user ID, quiz ID, and the user's answer.

[0083] Step 11:

[0084] The server evaluates the received user's answer and checks whether the answer is correct by comparing it with the correct answer information.

[0085] Step 12:

[0086] The server sends the evaluation results to the device, including feedback such as "Correct" if the user's answer is correct, or "Incorrect" if the answer is incorrect.

[0087] Step 13:

[0088] The device notifies the user of the evaluation results received from the server, allowing the user to check whether their answers are correct and receive feedback to improve their understanding.

[0089] Example 1

[0090] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0091] With conventional methods for viewing electronic publications, it is difficult to periodically review the content once it has been read and retain it in your memory for a long period of time. In particular, it is difficult to extract important information and key points and provide an environment for effective learning.

[0092] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0093] In this invention, the server includes means for analyzing the contents of electronic publications and extracting important information, means for automatically generating questions based on the extracted information, and means for notifying the user of the generated questions at a predetermined timing, thereby enabling the user to periodically review important information in electronic publications and retain it in their memory for a long period of time.

[0094] An "electronic publication" is a book, information, or document provided in electronic form.

[0095] "Content analysis" is the process of analyzing text data and extracting important information or key points.

[0096] "Key information" refers to knowledge or facts contained in a text that are particularly important for learning and understanding.

[0097] "Automatic generation" is a process in which a machine or program independently creates questions or poses questions with minimal human intervention.

[0098] "Questions" are questions or quizzes that users answer to review the content they have learned and check their level of understanding.

[0099] The term "predetermined timing" refers to a specific time or period that has been set in advance.

[0100] "User" means an individual or organization that uses this system.

[0101] "Notification" is the act of informing users of information or issues and encouraging them to receive them.

[0102] An "answer" is an answer or response provided by a user to a question.

[0103] "Evaluation" is the process of analyzing the collected responses and determining their accuracy and validity.

[0104] A "server" is a computer system or network device for processing, storing, and serving information.

[0105] This invention is a system that analyzes the contents of electronic publications, extracts important information, and automatically generates questions. The system aims to encourage users to periodically review what they have read and retain it in their memory for a long period of time by fulfilling the roles of the server, terminal, and user.

[0106] Server processing format

[0107] Hardware and Software Use

[0108] The server has a network connection to receive digital publication data from users. The received data is temporarily stored on the server's storage (e.g., HDD or SSD). The server has installed natural language processing libraries (e.g., SpaCy, NLTK, Transformers, etc.) for content analysis.

[0109] Data analysis

[0110] The server analyzes the text of electronic publications using natural language processing techniques, tokenizing the text and extracting important keywords and phrases using TF-IDF and word embedding techniques, thereby extracting the essence necessary for learning.

[0111] Automatic question generation

[0112] Based on the extracted key points, a generative AI model (e.g., GPT-3 or T5 model) can be used to generate specific questions and their answers. For example, it is possible to generate a question such as, "What are the fundamental principles of human-centered design?" and its corresponding answer.

[0113] Save quizzes and schedule notifications

[0114] The generated questions are saved in a database (e.g. MySQL, PostgreSQL). At that time, the server sets a schedule for notifying users of the quiz. For example, it can set the quiz to be notified after 2 days, 14 days, or 60 days. This schedule information is also saved in the database.

[0115] Submit a quiz

[0116] Based on the notification schedule, quizzes are sent to devices at the set date and time using methods such as push notifications and email notifications.

[0117] Terminal processing format

[0118] Receive and view quiz notifications

[0119] The device receives quiz notifications sent from the server. The notifications include the generated quiz questions and answer options. The device displays the received quiz via push notification or within the app. The display format provides a simple and intuitive user interface.

[0120] Collecting and sending user responses

[0121] The device provides an input form and buttons for users to answer the quiz, temporarily stores the answers entered by the user, and sends the answer data to the server for appropriate association.

[0122] User processing format

[0123] Check notifications and answer quizzes

[0124] The user confirms the quiz notification on their device and knows that a quiz has arrived. They enter their answers to the quiz displayed on their device and press the submit button to send the answers to the server. Through this process, the user can review the important content of the book and retain it in their long-term memory.

[0125] Examples of concrete examples and prompts

[0126] For example, if a user finishes reading a book called "Design Thinking," the server analyzes the book's contents and extracts important key points, such as:

[0127] Human-Centered Design

[0128] Iterative Prototyping

[0129] The server then generates a problem based on these keypoints, such as:

[0130] "What are the basic principles of human-centered design?"

[0131] "What is the importance of iterative prototyping?"

[0132] The quiz will be sent to the user's device two days later. The user checks the notification and answers the quiz. The user's answers are sent to the server via the device, and the server evaluates the answers and notifies the user of the results. This allows the user to retain important information from the electronic publication in their long-term memory.

[0133] Example prompts for generative AI models:

[0134] "Generate one quiz each about the book 'Design Thinking', the 'Basic Principles of Human-Centered Design' and the 'Importance of Iterative Prototyping'."

[0135] In this way, the system provides an environment in which readers can periodically review the contents of the book, thereby improving learning effectiveness.

[0136] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0137] Server Processing Steps

[0138] Step 1: Receiving electronic publication data

[0139] Specific actions

[0140] The server receives electronic publication files (e.g., PDF, ePub) sent by users.

[0141] Input: Electronic publication file

[0142] Output: A file temporarily stored in the server's storage

[0143] The server stores the received file in storage and records the file path.

[0144] Step 2: Content analysis

[0145] Specific actions

[0146] The server reads the text of the electronic publication stored in the storage and tokenizes the text using a natural language processing library (e.g., SpaCy, NLTK).

[0147] Input: Text data of electronic publications

[0148] Output: Tokenized text, important keywords and phrases

[0149] Extract key points necessary for learning from tokenized text using TF-IDF and word embeddings.

[0150] Step 3: Automatic question generation

[0151] Specific actions

[0152] Based on the extracted keypoints, a prompt sentence is input into a generative AI model (e.g., GPT-3) to generate a question sentence.

[0153] Input: Keypoint data, prompt

[0154] Output: Generated question statement

[0155] Use prompts to generate questions such as, "What are the basic principles of human-centered design?"

[0156] Step 4: Save the quiz to the database

[0157] Specific actions

[0158] The generated questions and their answers are saved in a database (e.g. MySQL).

[0159] Input: Generated question and answer

[0160] Output: Quiz data stored in a database

[0161] The database stores quiz questions, answers, user IDs, notification schedules, etc.

[0162] Step 5: Set up the quiz notification schedule

[0163] Specific actions

[0164] The server sets a schedule for notifying the generated quiz (e.g., after 2 days, 14 days, or 60 days).

[0165] Input: Quiz data, notification schedule

[0166] Output: Quiz data with schedule information added

[0167] The schedule is recorded in the database and quizzes are set to be notified periodically.

[0168] Step 6: Submit your quiz

[0169] Specific actions

[0170] The server sends the quiz to the terminal based on the set notification schedule.

[0171] Input: Schedule information, quiz data

[0172] Output: Quiz notification to device

[0173] Notifications can be sent using push notifications or email notifications.

[0174] Terminal processing steps

[0175] Step 1: Receive quiz notifications

[0176] Specific actions

[0177] The terminal receives the quiz notification sent from the server.

[0178] Input:QuizNotification

[0179] Output: Notification received alert

[0180] The terminal displays an alert to inform the user of the received notification.

[0181] Step 2: View the quiz

[0182] Specific actions

[0183] The terminal displays the received quiz content to the user.

[0184] Input: Quiz notification data

[0185] Output: The quiz displayed in the user interface

[0186] The quizzes are displayed using a simple UI, for example via push notifications or in-app notifications.

[0187] Step 3: Collect user responses

[0188] Specific actions

[0189] The terminal provides an interface for the user to input answers to the quiz.

[0190] Input: User's answer

[0191] Output: Temporarily stored user response data

[0192] The answers entered by the user are temporarily stored for transmission to the server.

[0193] Step 4: Submit your response data

[0194] Specific actions

[0195] The terminal transmits the user's response data to the server.

[0196] Input: User response data

[0197] Output: Response data sent to the server

[0198] When sent to the server, the user ID and quiz ID are also sent and associated appropriately.

[0199] User processing steps

[0200] Step 1: Check notifications

[0201] Specific actions

[0202] The user checks the quiz notification that arrives on the device.

[0203] Input:QuizNotification

[0204] Output: User's perceived behavior

[0205] Tap the notification to open the app and go to the quiz screen.

[0206] Step 2: Take the quiz

[0207] Specific actions

[0208] The user inputs answers to the displayed quiz questions.

[0209] Input: Quiz question

[0210] Output: User's answer

[0211] To submit your answer, enter text into the input form or select an option to confirm your answer.

[0212] (Application example 1)

[0213] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0214] It is difficult to efficiently learn the contents of e-books and retain them in your memory for a long period of time. In addition, there is a lack of review methods available after normal reading, and there is no system to regularly review what you have read, so it is difficult to expect the contents to be retained in your memory.

[0215] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0216] In this invention, the server includes means for [analyzing the contents of the e-book and extracting important information], means for [automatically generating quizzes based on the extracted information], means for [notifying the user of the generated quizzes at predetermined times], means for [regularly scheduling the timing of quiz notifications], means for [collecting and evaluating the user's answers], and means for [feeding back the evaluation results to the user]. This allows the contents of the e-book to be efficiently reviewed and the learned content to be retained in the memory for a long period of time.

[0217] "Means for analyzing the contents of e-books and extracting important information" refers to a function that analyzes the text data of e-books using natural language processing technology and text analysis algorithms, and automatically extracts key points and important information necessary for learning.

[0218] "Means for automatically generating quizzes based on extracted information" refers to a function that automatically generates specific question-style quizzes to confirm the learning content from extracted key points and important information.

[0219] The "means for notifying the user of the generated quiz at a predetermined timing" is a function for notifying the user of the generated quiz on the user's terminal based on a preset schedule.

[0220] The "means for periodically scheduling the timing of quiz notifications" is a function for setting and managing the schedule for quiz notifications at specific intervals so that users can review efficiently.

[0221] The "means for collecting and evaluating user answers" is a function for collecting answers entered by users to quizzes and evaluating whether the answers are correct or not based on the collected answers.

[0222] The "means for feeding back the evaluation results to the user" is a function for notifying the user of the evaluation results and providing feedback.

[0223] This invention provides a system that analyzes the contents of e-books, extracts important information, automatically generates quizzes, and periodically notifies users. This invention allows users to retain the contents of books they have finished reading for a long period of time. The system of this invention performs the following processing between a server, a terminal, and a user.

[0224] Server Processing

[0225] The server has multiple functions and performs the following processes:

[0226] 1. Content analysis of e-books

[0227] The server receives data of the electronic book that the user has finished reading.

[0228] Next, the content of the e-book is analyzed to extract important information and key points, using natural language processing techniques and text analysis algorithms, specifically the Hugging Face transformers library and pipeline.

[0229] Example: Analyze the contents of the e-book "Design Thinking" and extract key points such as "human-centered design" and "iterative prototyping."

[0230] 2. Quiz Generation

[0231] The server automatically generates a quiz based on the extracted key points. For example, it creates a specific question such as "What are the basic principles of human-centered design?" and sets the correct answer.

[0232] This allows the user to efficiently review what they have learned.

[0233] 3. Set a notification schedule

[0234] The server sets the schedule for notifying users of the quiz. For example, it sets the quiz to be notified after 2 days, 14 days, 60 days, etc. This schedule is managed using the scheduler function of apscheduler.

[0235] Terminal handling

[0236] The terminal is in charge of interfacing with the user and performs the following processes.

[0237] 1. Receive quiz notifications

[0238] The terminal receives the quiz notification sent from the server, which includes the generated quiz and the notification schedule.

[0239] Example: A user's smartphone will be notified of a quiz asking, "What are the basic principles of human-centered design?"

[0240] 2. View the quiz

[0241] The device displays the received quiz notification to the user. The display format provides a simple interface that makes it easy to answer. For example, it uses the push notification function of a smartphone or the display function of a head-mounted display.

[0242] Example: A quiz will appear on the notification screen of your smartphone, and you can enter answers by tapping.

[0243] 3. Collecting user responses

[0244] The terminal allows the user to enter answers to the quiz and transmits the entered answers to the server.

[0245] User Action

[0246] The user performs the following process through his / her own terminal.

[0247] 1. Check notifications

[0248] The user checks the notification from the device and becomes aware that there is a quiz. For example, when a notification arrives on the device two days later, the user checks it and answers the quiz.

[0249] 2. Answer the quiz

[0250] Users can interactively input answers to the displayed quiz questions. The input method is simple and intuitive, allowing answers to be entered by typing on the keyboard or by tapping.

[0251] Specific examples

[0252] For example, if a user finishes reading a book called "Design Thinking," the server analyzes the book's contents and extracts key points such as:

[0253] Human-Centered Design

[0254] Iterative Prototyping

[0255] The server then generates a quiz based on these key points:

[0256] "What are the basic principles of human-centered design?"

[0257] "What is the importance of iterative prototyping?"

[0258] The quiz will be sent to the user's device two days later. The user checks the notification and answers the quiz. The user's answers are sent to the server via the device, and the server evaluates the answers and notifies the user of the results.

[0259] Prompt Sentence Examples

[0260] Prompt sentence to input to the generative AI model:

[0261] "Please summarize the following: {eBook text}"

[0262] In this way, the system provides an environment in which readers can periodically review the contents of the book, thereby improving learning effectiveness.

[0263] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0264] Step 1: Receiving e-book data

[0265] The server receives the e-book data that the user has finished reading. As input, the e-book file uploaded by the user through the application is sent to the server. The server stores this e-book data in its internal storage for processing.

[0266] Step 2: Content analysis and important information extraction

[0267] The server analyzes the stored e-book data using natural language processing techniques and text analysis algorithms. The server generates a summary of the e-book based on the extracted key points and important information. This process uses Hugging Face's transformers library and pipeline. The output is a summary text, which becomes the input for the next process.

[0268] Step 3: Auto-generate a quiz

[0269] The server automatically generates quizzes based on the extracted key points and summary text. Specifically, the generative AI model receives a prompt, "Please summarize the following: {e-book text}," and converts it into a quiz format. For example, it generates a question such as, "What are the basic principles of human-centered design?" The output is multiple quizzes, which serve as input for the next process.

[0270] Step 4: Set up a notification schedule

[0271] The server sets the notification schedule for the generated quiz. It receives the quiz data as input and schedules the notification to occur at a specific time (for example, after 2 days, 14 days, or 60 days). This schedule is managed using the scheduler function of apscheduler. The scheduled notification task is set as the output.

[0272] Step 5: Receive quiz notifications

[0273] The terminal receives quiz notifications sent from the server based on a set schedule. As input, the server sends quiz notification data to the terminal. The terminal receives and stores this data.

[0274] Step 6: View the quiz

[0275] The device displays the received quiz notification to the user. This can be done via smartphone push notifications, in-app notifications, or head-mounted displays. The device references the quiz notification data as input, and displays the quiz to the user as output.

[0276] Step 7: User answers

[0277] The user inputs answers to the displayed quiz questions. The answers entered by the user using tap operations or keyboard input are saved on the device.

[0278] Step 8: User submits answer

[0279] The terminal receives the user's answer data as input and sends it to the server.

[0280] Step 9: Evaluate user responses

[0281] The server evaluates the received user answers. Specifically, it uses an internal evaluation algorithm to evaluate whether the answer is correct. It receives the user's answer data as input and generates an evaluation result as output.

[0282] Step 10: Feedback of evaluation results

[0283] The server feeds back the evaluation results to the user. It references the evaluation result data as input and generates a result notification as output, which it sends to the terminal. The terminal receives this notification and displays it to the user.

[0284] This allows the server, terminal, and user to work together, making it possible to efficiently review and memorize the contents of e-books.

[0285] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0286] This invention combines a system that analyzes the contents of e-books, extracts important information, and notifies users of automatically generated quizzes at predetermined times with an emotion engine that recognizes the user's emotions, in order to enhance user learning. This makes it possible to provide feedback according to the user's emotions and adjust the content and difficulty of the quizzes, further enhancing the effectiveness of learning.

[0287] Server Processing

[0288] 1. Content analysis of e-books

[0289] The server receives the data of the e-book that the user has finished reading, analyzes the contents of the e-book, and extracts important information and key points.

[0290] 2. Quiz Generation

[0291] The server automatically generates quizzes based on the extracted key points, asking about important information, and stores the generated quizzes in a database.

[0292] 3. Use of Emotion Engine

[0293] After generating the quiz, the server collects user emotional data and adjusts the content and difficulty of the generated quiz based on this data. For example, if the user is feeling stressed, the difficulty level can be lowered to reduce the user's psychological burden.

[0294] 4. Set up a notification schedule

[0295] The server sets a quiz notification schedule and sets the quiz to be notified at a specific timing.

[0296] Terminal handling

[0297] 1. Collecting Emotional Data

[0298] The device uses sensors such as a camera and microphone to collect emotional data from the user's facial expressions and tone of voice, and the collected emotional data is sent to a server.

[0299] 2. Receiving and viewing quiz notifications

[0300] The device receives quiz notifications sent from the server and displays them to the user in an intuitive interface, such as push notifications or in-app notifications.

[0301] 3. Collecting user responses and displaying evaluation results

[0302] The terminal collects the user's answers and sends them to the server, and after receiving the evaluation results from the server, displays the evaluation results to the user.

[0303] User Action

[0304] 1. Check and answer quiz notifications

[0305] The user checks the quiz displayed on the terminal and enters answers to the questions displayed. The user's answers are then sent from the terminal to the server.

[0306] 2. Receiving Feedback

[0307] Users receive evaluation results for their answers from their devices, which include not only whether their answers were correct or incorrect, but also feedback based on the user's individual emotional state.

[0308] Specific examples

[0309] For example, if a user finishes reading a book called "Design Thinking," the server analyzes the book's contents and extracts the following key points:

[0310] Human-Centered Design

[0311] Iterative Prototyping

[0312] The server then generates a quiz based on these key points, like this:

[0313] "What are the basic principles of human-centered design?"

[0314] "What is the importance of iterative prototyping?"

[0315] If the emotion engine analyzes the user's facial expressions using a camera and determines that the user is likely to be lacking concentration, the server adjusts the difficulty of the quiz and starts the user with easier questions.

[0316] The quiz will be notified to the user's device two days later. The user will check the notification and answer the quiz. The user's answers will be evaluated by the server, and the results will be sent back to the device. At this time, the server will take the user's emotional state into consideration and tailor the feedback appropriately to provide useful information to the user.

[0317] In this way, by adding emotion recognition to the e-book review system, the present invention provides a learning environment suited to the user's psychological state, further enhancing the learning effect.

[0318] The processing flow will be explained below.

[0319] Step 1:

[0320] When the user finishes reading the e-book, they press a button on the device to indicate they are finished reading, and the device notifies the server that they have finished reading.

[0321] Step 2:

[0322] The device sends information including the ID of the e-book that has been read and the user ID to the server. Based on the sent information, the server retrieves the data of the corresponding e-book.

[0323] Step 3:

[0324] The server begins analyzing the received data, using natural language processing techniques to extract important information and key points from the text. This process involves grammatical analysis, key word extraction, and contextual analysis.

[0325] Step 4:

[0326] The server automatically generates a quiz based on the extracted key points. The generated quiz includes specific questions and their corresponding correct answers. For example, a generated question might be, "What are the basic principles of human-centered design?"

[0327] Step 5:

[0328] The server stores the generated quiz in a database, along with the user ID and schedule information, in addition to the quiz content.

[0329] Step 6:

[0330] The server sets a notification schedule for the quiz, for example, to notify the user 2 days, 14 days, and 60 days after the user finishes reading. This notification schedule is stored in a database and is triggered based on a timestamp.

[0331] Step 7:

[0332] The emotion engine then begins to operate. The device uses the camera and microphone to analyze the user's facial expressions and voice, collecting emotional data. This data is then sent to the server in real time.

[0333] Step 8:

[0334] The server analyzes the received emotional data and recognizes the user's emotional state, for example, determining whether the user is feeling stressed or focused.

[0335] Step 9:

[0336] The server adjusts the content and difficulty of the generated quiz based on the user's emotional state. For example, if the user's stress level is high, the server may lower the difficulty of the quiz or reduce the number of questions.

[0337] Step 10:

[0338] When the set date and time arrives, the server sends a quiz notification to the terminal, which includes the adjusted quiz content and an answer interface.

[0339] Step 11:

[0340] The terminal receives the notification from the server and displays it to the user, who then checks the notification and accesses an interface for answering the quiz.

[0341] Step 12:

[0342] The user answers questions on the display screen of the device, for example, by inputting answers to the question, "What are the basic principles of human-centered design?"

[0343] Step 13:

[0344] The terminal transmits the user's answer to the server. The transmitted data includes the user's answer and the associated quiz ID.

[0345] Step 14:

[0346] The server evaluates the received answer and determines whether it is correct or not, and the evaluation result is classified as correct, incorrect, partially correct, etc.

[0347] Step 15:

[0348] The server generates feedback based on the evaluation results, particularly depending on the user's emotional state. For example, if the user shows signs of anxiety, it may include an encouraging message.

[0349] Step 16:

[0350] The server sends the generated feedback to the device, which then displays the received feedback to the user. The user can check the evaluation results of their answers and the feedback based on their emotions.

[0351] Through these steps, users can effectively review the contents of the e-book and be provided with a learning environment that takes their emotional state into consideration.

[0352] Example 2

[0353] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0354] Conventional e-books lacked feedback to improve users' learning effectiveness or learning adjustments based on the user's emotional state. This made it difficult to provide an effective learning environment that matched the user's level of concentration and understanding. Furthermore, the fixed difficulty and content of quizzes did not reduce the user's psychological burden, potentially leading to a decline in motivation to learn.

[0355] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for [analyzing the contents of the e-book and extracting important information], a means for [automatically generating quizzes based on the extracted information], and a means for [adjusting the content and difficulty of the generated quizzes based on the user's emotional data]. This makes it possible to improve the user's learning effect and provide optimal feedback according to the user's emotional state.

[0356] "Analyzing the contents of e-books and extracting important information" refers to the process of analyzing the text data of e-books that users have finished reading and extracting important points and keywords necessary for learning and memorization.

[0357] "Automatically generating quizzes based on extracted information" refers to the process of automatically creating question-style questions to be posed to users using a program based on important information obtained through the analysis.

[0358] "Adjusting the content and difficulty of the generated quiz based on the user's emotional data" refers to the process of analyzing the user's emotional data (such as facial expressions and tone of voice) and optimizing the difficulty of the quiz and the content of the questions according to the results of that analysis.

[0359] "Notifying the user of the generated quiz at a predetermined timing" refers to a process of notifying the user of the quiz at an appropriate timing according to a pre-set schedule or the user's learning situation.

[0360] "Collecting emotion data from the terminal and transmitting it to the server" refers to the process of collecting data related to the user's emotions using sensors such as a camera and microphone, and transferring that data to the server.

[0361] "Collecting and evaluating user answers" refers to the process of acquiring the answers given by users to the quiz and evaluating the accuracy and content of the answers.

[0362] This invention combines a system that analyzes the contents of e-books, extracts important information, and presents automatically generated quizzes at predetermined times to enhance user learning, with an emotion engine that recognizes the user's emotions. Implementing this system requires collaboration between a server, terminals, and users.

[0363] Server Processing

[0364] The server first receives the e-book data that the user has finished reading. The e-book data is uploaded in a format such as an EPUB file. Next, it uses a natural language processing engine such as Google Cloud Natural Language API or IBM Watson Natural Language Understanding to analyze the content of the e-book and extract important information and key points.

[0365] Based on the extracted keypoints, a generative AI model (e.g., OpenAI GPT-4) is used to automatically generate a quiz, which is then stored in a database (e.g., MySQL or MongoDB) for later use.

[0366] The server then collects the emotional data sent from the device and analyzes the user's emotional data using the Microsoft Azure Emotion API and Affectiva SDK. Based on the analysis results, the content and difficulty of the quiz can be adjusted. For example, if the user is feeling stressed, the difficulty of the quiz can be lowered to reduce the psychological burden.

[0367] Finally, set up a notification schedule for the quiz and notify users of the quiz at the appropriate time using scheduling software such as a Cron job or Amazon CloudWatch Events.

[0368] Terminal handling

[0369] The device collects the user's emotional data using sensors such as a camera and microphone. This can be done using the device's built-in camera or an external camera or microphone. The emotional data includes information on facial expressions and tone of voice, and the collected data is sent to a server.

[0370] The app receives quiz notifications sent from the server and displays them to the user. Notifications can be sent in the form of push notifications or in-app notifications. Push notifications are implemented using Firebase Cloud Messaging (FCM), and in-app notifications are implemented using mobile frameworks such as React Native or Swift.

[0371] The device collects the user's answers and sends them to the server. Once the answers are evaluated, the server sends the results back to the user. The evaluation results not only indicate whether the answer was correct or incorrect, but also provide feedback based on the user's emotional state.

[0372] User Action

[0373] The user checks the quiz displayed on the device and enters answers to the questions. The user's answers are sent from the device to the server, where they are evaluated. The evaluation results are returned to the device, and correct / incorrect answers and feedback are displayed. The feedback includes advice that takes into account the user's emotional state, further enhancing the learning effect.

[0374] Specific examples

[0375] For example, suppose a user has finished reading a book called "Design Thinking." The server analyzes the book's contents and extracts key points such as "human-centered design" and "iterative prototyping." The server then sends the following prompt to the generative AI model: "Based on the key points in the book "Design Thinking," please generate the following quiz. The key points are 'human-centered design' and 'iterative prototyping.'"

[0376] The generative AI model generates a quiz like this:

[0377] "What are the basic principles of human-centered design?"

[0378] "What is the importance of iterative prototyping?"

[0379] If the emotion engine analyzes the user's facial expressions using a camera and determines that the user is likely to be lacking concentration, the server adjusts the difficulty of the quiz, starting with easier questions. The quiz is notified to the user's device two days later. The user checks the notification and answers the quiz. The server evaluates the user's answers and sends the results back to the device. At this time, feedback is provided taking into account the user's emotional state.

[0380] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0381] Server Processing

[0382] Step 1: Content analysis of e-books

[0383] Input: The e-book data the user has finished reading (e.g., EPUB file)

[0384] How it works: The server receives e-book data uploaded by a user, in a format such as an EPUB file.

[0385] Data processing: The server analyzes the text data of the e-book using the Google Cloud Natural Language API, identifying nouns, verbs, adjectives, etc. and mapping their relationships.

[0386] Output: Extracted important information or key points.

[0387] Step 2: Generate the quiz

[0388] Input: Extracted keypoints

[0389] How it works: The server generates a prompt based on the extracted keypoints using a generative AI model (e.g., OpenAI GPT-4).

[0390] Data processing: The server sends prompts to the generative AI model to automatically generate quizzes.

[0391] Output: An automatically generated quiz, specifically using the prompt: "Generate the following quiz based on the key points from the book 'Design Thinking'. The key points are 'human-centered design' and 'iterative prototyping'."

[0392] Step 3: Analyze sentiment data and tailor the quiz

[0393] Input: Generated quiz, emotion data collected from the device

[0394] How it works: The server receives emotion data sent from the device, analyzes it using the Microsoft Azure Emotion API, and evaluates the user's emotional state (e.g., stress level or concentration level) based on the analysis results.

[0395] Data processing: The content and difficulty of the quiz are adjusted based on the analysis of emotional data. For example, if the user is feeling stressed, the difficulty of the quiz can be lowered to reduce the psychological burden.

[0396] Output: The adapted quiz.

[0397] Step 4: Schedule and run notifications

[0398] Inputs: Adjusted quiz, scheduling data

[0399] How it works: The server schedules quiz notifications using Cron jobs and Amazon CloudWatch Events.

[0400] Data processing: Notify quizzes at specified dates and times based on a set schedule.

[0401] Output: Quiz notification to user device

[0402] Terminal handling

[0403] Step 1: Collect and send emotion data

[0404] Input: User's facial expression, tone of voice

[0405] How it works: The device uses the camera and microphone to collect emotional data about the user, including information about facial expressions and tone of voice.

[0406] Data processing: Analyzing emotional data using facial recognition and voice analysis algorithms.

[0407] Output: Send the analyzed emotion data to the server.

[0408] Step 2: Receive and view quiz notifications

[0409] Input: Quiz notification from the server

[0410] Operation: The device receives a quiz notification sent from the server and displays the notification to the user.

[0411] Data processing: Display received notifications as push notifications or in-app notifications using services such as Firebase Cloud Messaging (FCM).

[0412] Output: The quiz notification displayed to the user

[0413] Step 3: Collect user responses and display evaluation results

[0414] Input: User's answer

[0415] Operation: The device collects the answers entered by the user and sends them to the server.

[0416] Data processing: Receive the response data in the form and send it to the server via an HTTP POST request.

[0417] Output: Receives the evaluation results from the server and displays them to the user.

[0418] User Action

[0419] Step 1: Check the quiz notification and answer

[0420] Input: Quiz notification from your device

[0421] How it works: The user reviews the quiz and enters answers to the questions displayed.

[0422] Output: User's answer typed into the terminal

[0423] Step 2: Receiving feedback

[0424] Input: Evaluation result from the server

[0425] How it works: The user checks the evaluation results displayed on the device. The evaluation results include not only correct or incorrect answers, but also feedback based on the user's emotional state.

[0426] Output: Evaluation results and feedback displayed to the user

[0427] In this way, the roles of the server, terminal, and user are clearly separated and a detailed processing flow is described.

[0428] (Application example 2)

[0429] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0430] Conventional e-book learning systems did not provide feedback or adjust the difficulty of quizzes based on the user's emotional state. This could lead to stress and reduced concentration, resulting in ineffective learning. Furthermore, the notification function to prompt users to review at the appropriate time was insufficient.

[0431] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0432] In this invention, the server includes means for analyzing the content of the electronic medium and extracting important information, means for automatically generating quizzes based on the extracted information, means for notifying the user of the generated quizzes at a predetermined timing, means for analyzing and collecting the user's emotional state, means for adjusting the content and difficulty of the quiz according to the user's emotional state, and means for collecting and evaluating the user's answers, thereby enabling effective feedback and adjustment of the difficulty of the quiz according to the user's emotional state.

[0433] "Electronic media" is a collection of information stored or distributed in electronic form.

[0434] "Analysis" means analyzing data in detail to understand its structure and meaning.

[0435] "Significant information" is data that is particularly valuable for a particular purpose or context and is necessary for understanding and decision-making.

[0436] A "quiz" is a set of questions designed to gauge a user's knowledge and understanding.

[0437] "Automatic generation" means that something is created automatically by an algorithm or program, without human intervention.

[0438] "Notifying" means notifying a user of specific information at a specific time.

[0439] "Emotional state" refers to the user's psychological state, including stress, concentration, fatigue, and the like.

[0440] "Adjust" means to change or optimize according to conditions or circumstances.

[0441] "Collect" means to gather specific data.

[0442] To "evaluate" means to judge data or results based on specific criteria.

[0443] This invention combines a system that analyzes the contents of electronic media, extracts important information, and notifies users of automatically generated quizzes at the appropriate time with an emotion engine that recognizes the user's emotional state. This makes it possible to provide feedback according to the user's emotions and adjust the content and difficulty of the quiz, thereby improving learning effectiveness.

[0444] Server Processing

[0445] 1. Content analysis of electronic media:

[0446] The server receives the data from the electronic media that the user has finished reading and analyzes its content. Natural language processing (NLP) is used for the analysis to extract important information and key points. Specifically, Python and NLTK (Natural Language Toolkit) are used.

[0447] 2. Generate the quiz:

[0448] The server automatically generates quizzes that ask about important information based on the extracted key points. The generated quizzes are stored in a database. A generative AI model is used to generate the quizzes.

[0449] 3. Use of Emotion Engine:

[0450] The server analyzes the user's emotional state. The user's emotional state is evaluated based on data collected using the smartphone's camera and microphone. Specifically, OpenCV and Google Cloud Vision API are used. The content and difficulty of the quiz are adjusted according to the user's emotional state.

[0451] 4. Set the notification schedule:

[0452] The server sets a quiz notification schedule and configures the quiz notification to occur at specific times, and this schedule is adjusted based on the user's learning pace and emotional state.

[0453] Terminal handling

[0454] 1. Collecting Emotional Data:

[0455] The device uses sensors such as a camera and microphone to collect emotional data from the user's facial expressions and tone of voice. The collected emotional data is sent to a server, which uses the Microsoft Azure Emotion API for specific emotional analysis.

[0456] 2. Receive and view quiz notifications:

[0457] The device receives the quiz notification sent from the server and displays it to the user. The display format provides an intuitive interface such as push notification or in-app notification.

[0458] 3. Collect user responses and display the evaluation results:

[0459] The terminal collects the user's answers and sends them to the server, and after receiving the evaluation results from the server, displays the evaluation results to the user.

[0460] User Action

[0461] 1. Check the quiz notification and answer:

[0462] The user checks the quiz displayed on the terminal and enters answers to the questions displayed. The user's answers are then sent from the terminal to the server.

[0463] 2. Receiving Feedback:

[0464] Users receive evaluation results for their answers from their devices, which include not only whether their answers were correct or incorrect, but also feedback based on the user's individual emotional state.

[0465] Specific examples

[0466] For example, if a user finishes reading a book called "Design Thinking," the server will analyze the book's contents and extract key points such as:

[0467] Human-Centered Design

[0468] Iterative Prototyping

[0469] The server then generates a quiz based on these key points, like this:

[0470] "What are the basic principles of human-centered design?"

[0471] "What is the importance of iterative prototyping?"

[0472] If the emotion engine analyzes the user's facial expressions using a camera and determines that the user is likely to be lacking concentration, the server adjusts the difficulty of the quiz and starts the user with easier questions.

[0473] The quiz will be notified to the user's device two days later. The user will check the notification and answer the quiz. The user's answers will be evaluated by the server, and the results will be sent back to the device. At this time, the server will take into account the user's emotional state and tailor the feedback appropriately to provide useful information to the user. The following prompt sentences can be used:

[0474] "Please explain the basic principles of human-centered design (e.g., designing based on user needs)."

[0475] In this way, by adding emotion recognition to an electronic review system, the present invention provides a learning environment suited to the user's psychological state, further enhancing learning effectiveness.

[0476] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0477] Step 1:

[0478] The server receives the data from the electronic media that the user has finished reading. The input is the text data from the electronic media, which is analyzed to extract important information and key points. Specifically, natural language processing (NLP) technology is used to tokenize the text data and extract important words and phrases. The extracted information is output as key points.

[0479] Step 2:

[0480] The server automatically generates a quiz based on the extracted keypoints. The input is the extracted keypoints, and based on these, a generative AI model is used to create quiz questions. Specifically, the generative AI model generates questions related to the keypoints and stores them in a database. The generated quiz is then output.

[0481] Step 3:

[0482] The server sets the quiz notification schedule. The input is the user's learning progress data and emotional state data, and the notification timing is calculated based on this. Specifically, the server takes into account the user's learning patterns and emotional state to set a schedule for quiz notifications at the optimal timing. The set notification schedule is output.

[0483] Step 4:

[0484] The device collects the user's emotional state. The input is sensor data from the camera and microphone, which is analyzed to determine the user's emotional state. Specifically, the device uses OpenCV and the Microsoft Azure Emotion API to analyze the user's facial expressions and tone of voice from the collected sensor data. The analyzed emotional state data is output.

[0485] Step 5:

[0486] The device receives the quiz notification sent from the server and displays it to the user. The input is the quiz question sent from the server, which is then displayed to the user. Specifically, the device uses push notifications or in-app notifications to intuitively inform the user of the quiz. The displayed quiz is then output.

[0487] Step 6:

[0488] The user answers a quiz displayed on the terminal. The input is the quiz question displayed on the terminal, and the user inputs the answer to that question. In concrete terms, the user inputs the answer to the question and sends it to the server via the terminal. The user's answer is then output.

[0489] Step 7:

[0490] The server collects and evaluates the user's answers. The input is the user's answer data, which is then evaluated. Specifically, the server compares the user's answers with the correct answer data and generates an evaluation result. The generated evaluation result is then output.

[0491] Step 8:

[0492] The server adjusts the feedback according to the user's emotional state and notifies the user of the evaluation results. The input is the evaluation results and the user's emotional state data, and the feedback is created based on this. Specifically, the server generates feedback according to the user's emotional state and sends it to the terminal. The adjusted feedback is then output.

[0493] Step 9:

[0494] The device displays the evaluation results and feedback received from the server to the user. The input is the evaluation results and feedback sent from the server, which are then displayed to the user. Specifically, the device uses a notification function or an in-app interface to visually provide the evaluation results and feedback to the user. The displayed evaluation results and feedback are then output.

[0495] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0496] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0497] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0498] [Second embodiment]

[0499] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0500] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0501] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0502] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0503] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0504] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0505] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0506] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0507] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0508] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0509] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0510] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0511] This invention is a system that analyzes the contents of e-books, extracts important information, automatically generates quizzes, and allows readers to review the content. This system allows readers to retain the contents of books they have read for a long period of time. Specifically, the following process is carried out between the server, the terminal, and the user.

[0512] Server Processing

[0513] 1. Content analysis of e-books

[0514] The server receives the data of the e-book that the user has finished reading. It then analyzes the content of the e-book and extracts important information and key points. In this process, it uses natural language processing technology and text analysis algorithms to extract the essence of the text necessary for learning.

[0515] 2. Quiz Generation

[0516] The server automatically generates quizzes based on the extracted key points. For example, it creates specific questions such as "What are the basic principles of human-centered design?" and sets the correct answers. The generated quizzes are stored in a database.

[0517] 3. Set a notification schedule

[0518] The server sets a schedule for notifying users of quizzes, for example, setting it to notify users at specific times, such as after 2 days, 14 days, or 60 days, encouraging regular review and promoting knowledge retention.

[0519] Terminal handling

[0520] 1. Receive quiz notifications

[0521] The terminal receives the quiz notification sent from the server, which includes the generated quiz and the notification schedule.

[0522] 2. View the quiz

[0523] The device displays the received quiz notification to the user in a simple and easy-to-answer interface, such as a push notification on a smartphone or an in-app notification.

[0524] 3. Collecting user responses

[0525] The terminal allows the user to enter answers to the quiz and transmits the entered answers to the server.

[0526] User Action

[0527] 1. Check notifications

[0528] The user checks the notification from the device and becomes aware that there is a quiz. For example, when a notification arrives on the device two days later, the user checks it and answers the quiz.

[0529] 2. Answer the quiz

[0530] Users can interactively input answers to the displayed quiz questions. The input method is simple and intuitive, allowing answers to be entered by typing on the keyboard or by tapping.

[0531] Specific examples

[0532] For example, if a user finishes reading a book called "Design Thinking," the server analyzes the book's contents and extracts important key points, such as:

[0533] Human-Centered Design

[0534] Iterative Prototyping

[0535] The server then generates a quiz based on these key points:

[0536] "What are the basic principles of human-centered design?"

[0537] "What is the importance of iterative prototyping?"

[0538] The quiz will be sent to the user's device two days later. The user checks the notification and answers the quiz. The user's answers are sent to the server via the device, and the server evaluates the answers and notifies the user of the results. Through this process, the user can review the important content of the book and retain it in their long-term memory.

[0539] In this way, the system provides an environment in which readers can periodically review the contents of the book, thereby improving learning effectiveness.

[0540] The processing flow will be explained below.

[0541] Step 1:

[0542] When the user finishes reading an e-book, he or she presses a button on the device to record the end of reading, which notifies the system of the end of book reading event.

[0543] Step 2:

[0544] The device sends information about the e-book that the user has finished reading to the server. The sent information includes the user ID and the e-book ID.

[0545] Step 3:

[0546] Based on the received information, the server retrieves the contents of the e-book and begins analyzing it, using natural language processing technology to extract important key points and learning essences from the text.

[0547] Step 4:

[0548] The server automatically generates quizzes based on information extracted from the analysis results. The quizzes are created in the form of specific questions based on important key points. For example, a question might be generated such as, "What are the basic principles of human-centered design?"

[0549] Step 5:

[0550] The server saves the generated quiz in a database, along with the user ID, quiz content, and correct answer information.

[0551] Step 6:

[0552] The server sets a notification schedule for the quiz. The notification timing can be set to a specific schedule, such as 2 days, 14 days, or 60 days later.

[0553] Step 7:

[0554] When the set time comes, the server sends a quiz notification to the terminal, which includes the generated quiz and an interface for answering the question.

[0555] Step 8:

[0556] The terminal displays the received quiz notification to the user, who then checks the notification and opens an interface for answering the quiz.

[0557] Step 9:

[0558] The user inputs answers to quiz questions through the device interface, for example, "What are the basic principles of human-centered design?"

[0559] Step 10:

[0560] The terminal sends the user's answer to the server. The sent data includes the user ID, quiz ID, and the user's answer.

[0561] Step 11:

[0562] The server evaluates the received user's answer and checks whether the answer is correct by comparing it with the correct answer information.

[0563] Step 12:

[0564] The server sends the evaluation results to the device, including feedback such as "Correct" if the user's answer is correct, or "Incorrect" if the answer is incorrect.

[0565] Step 13:

[0566] The device notifies the user of the evaluation results received from the server, allowing the user to check whether their answers are correct and receive feedback to improve their understanding.

[0567] Example 1

[0568] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0569] With conventional methods for viewing electronic publications, it is difficult to periodically review the content once it has been read and retain it in your memory for a long period of time. In particular, it is difficult to extract important information and key points and provide an environment for effective learning.

[0570] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0571] In this invention, the server includes means for analyzing the contents of electronic publications and extracting important information, means for automatically generating questions based on the extracted information, and means for notifying the user of the generated questions at a predetermined timing, thereby enabling the user to periodically review important information in electronic publications and retain it in their memory for a long period of time.

[0572] An "electronic publication" is a book, information, or document provided in electronic form.

[0573] "Content analysis" is the process of analyzing text data and extracting important information or key points.

[0574] "Key information" refers to knowledge or facts contained in a text that are particularly important for learning and understanding.

[0575] "Automatic generation" is a process in which a machine or program independently creates questions or poses questions with minimal human intervention.

[0576] "Questions" are questions or quizzes that users answer to review the content they have learned and check their level of understanding.

[0577] The term "predetermined timing" refers to a specific time or period that has been set in advance.

[0578] "User" means an individual or organization that uses this system.

[0579] "Notification" is the act of informing users of information or issues and encouraging them to receive them.

[0580] An "answer" is an answer or response provided by a user to a question.

[0581] "Evaluation" is the process of analyzing the collected responses and determining their accuracy and validity.

[0582] A "server" is a computer system or network device for processing, storing, and serving information.

[0583] This invention is a system that analyzes the contents of electronic publications, extracts important information, and automatically generates questions. The system aims to encourage users to periodically review what they have read and retain it in their memory for a long period of time by fulfilling the roles of the server, terminal, and user.

[0584] Server processing format

[0585] Hardware and Software Use

[0586] The server has a network connection to receive digital publication data from users. The received data is temporarily stored on the server's storage (e.g., HDD or SSD). The server has installed natural language processing libraries (e.g., SpaCy, NLTK, Transformers, etc.) for content analysis.

[0587] Data analysis

[0588] The server analyzes the text of electronic publications using natural language processing techniques, tokenizing the text and extracting important keywords and phrases using TF-IDF and word embedding techniques, thereby extracting the essence necessary for learning.

[0589] Automatic question generation

[0590] Based on the extracted key points, a generative AI model (e.g., GPT-3 or T5 model) can be used to generate specific questions and their answers. For example, it is possible to generate a question such as, "What are the fundamental principles of human-centered design?" and its corresponding answer.

[0591] Save quizzes and schedule notifications

[0592] The generated questions are saved in a database (e.g. MySQL, PostgreSQL). At that time, the server sets a schedule for notifying users of the quiz. For example, it can set the quiz to be notified after 2 days, 14 days, or 60 days. This schedule information is also saved in the database.

[0593] Submit a quiz

[0594] Based on the notification schedule, quizzes are sent to devices at the set date and time using methods such as push notifications and email notifications.

[0595] Terminal processing format

[0596] Receive and view quiz notifications

[0597] The device receives quiz notifications sent from the server. The notifications include the generated quiz questions and answer options. The device displays the received quiz via push notification or within the app. The display format provides a simple and intuitive user interface.

[0598] Collecting and sending user responses

[0599] The device provides an input form and buttons for users to answer the quiz, temporarily stores the answers entered by the user, and sends the answer data to the server for appropriate association.

[0600] User processing format

[0601] Check notifications and answer quizzes

[0602] The user confirms the quiz notification on their device and knows that a quiz has arrived. They enter their answers to the quiz displayed on their device and press the submit button to send the answers to the server. Through this process, the user can review the important content of the book and retain it in their long-term memory.

[0603] Examples of concrete examples and prompts

[0604] For example, if a user finishes reading a book called "Design Thinking," the server analyzes the book's contents and extracts important key points, such as:

[0605] Human-Centered Design

[0606] Iterative Prototyping

[0607] The server then generates a problem based on these keypoints, such as:

[0608] "What are the basic principles of human-centered design?"

[0609] "What is the importance of iterative prototyping?"

[0610] The quiz will be sent to the user's device two days later. The user checks the notification and answers the quiz. The user's answers are sent to the server via the device, and the server evaluates the answers and notifies the user of the results. This allows the user to retain important information from the electronic publication in their long-term memory.

[0611] Example prompts for generative AI models:

[0612] "Generate one quiz each about the book 'Design Thinking', the 'Basic Principles of Human-Centered Design' and the 'Importance of Iterative Prototyping'."

[0613] In this way, the system provides an environment in which readers can periodically review the contents of the book, thereby improving learning effectiveness.

[0614] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0615] Server Processing Steps

[0616] Step 1: Receiving electronic publication data

[0617] Specific actions

[0618] The server receives electronic publication files (e.g., PDF, ePub) sent by users.

[0619] Input: Electronic publication file

[0620] Output: A file temporarily stored in the server's storage

[0621] The server stores the received file in storage and records the file path.

[0622] Step 2: Content analysis

[0623] Specific actions

[0624] The server reads the text of the electronic publication stored in storage and tokenizes the text using a natural language processing library (e.g., SpaCy, NLTK).

[0625] Input: Text data of electronic publications

[0626] Output: Tokenized text, important keywords and phrases

[0627] Extract key points necessary for learning from tokenized text using TF-IDF and word embeddings.

[0628] Step 3: Automatic question generation

[0629] Specific actions

[0630] Based on the extracted keypoints, a prompt sentence is input into a generative AI model (e.g., GPT-3) to generate a question sentence.

[0631] Input: Keypoint data, prompt

[0632] Output: Generated question statement

[0633] Use prompts to generate questions such as, "What are the basic principles of human-centered design?"

[0634] Step 4: Save the quiz to the database

[0635] Specific actions

[0636] The generated questions and their answers are saved in a database (e.g. MySQL).

[0637] Input: Generated question and answer

[0638] Output: Quiz data stored in a database

[0639] The database stores quiz questions, answers, user IDs, notification schedules, etc.

[0640] Step 5: Set up the quiz notification schedule

[0641] Specific actions

[0642] The server sets a schedule for notifying the generated quiz (e.g., after 2 days, 14 days, or 60 days).

[0643] Input: Quiz data, notification schedule

[0644] Output: Quiz data with schedule information added

[0645] The schedule is recorded in the database and quizzes are set to be notified periodically.

[0646] Step 6: Submit your quiz

[0647] Specific actions

[0648] The server sends the quiz to the terminal based on the set notification schedule.

[0649] Input: Schedule information, quiz data

[0650] Output: Quiz notification to device

[0651] Notifications can be sent using push notifications or email notifications.

[0652] Terminal processing steps

[0653] Step 1: Receive quiz notifications

[0654] Specific actions

[0655] The terminal receives the quiz notification sent from the server.

[0656] Input:QuizNotification

[0657] Output: Notification received alert

[0658] The terminal displays an alert to inform the user of the received notification.

[0659] Step 2: View the quiz

[0660] Specific actions

[0661] The terminal displays the received quiz content to the user.

[0662] Input: Quiz notification data

[0663] Output: The quiz displayed in the user interface

[0664] The quizzes are displayed using a simple UI, for example via push notifications or in-app notifications.

[0665] Step 3: Collect user responses

[0666] Specific actions

[0667] The terminal provides an interface for the user to input answers to the quiz.

[0668] Input: User's answer

[0669] Output: Temporarily stored user response data

[0670] The answers entered by the user are temporarily stored for transmission to the server.

[0671] Step 4: Submit your response data

[0672] Specific actions

[0673] The terminal transmits the user's response data to the server.

[0674] Input: User response data

[0675] Output: Response data sent to the server

[0676] When sent to the server, the user ID and quiz ID are also sent and associated appropriately.

[0677] User processing steps

[0678] Step 1: Check notifications

[0679] Specific actions

[0680] The user checks the quiz notification that arrives on the device.

[0681] Input:QuizNotification

[0682] Output: User's perceived behavior

[0683] Tap the notification to open the app and go to the quiz screen.

[0684] Step 2: Take the quiz

[0685] Specific actions

[0686] The user inputs answers to the displayed quiz questions.

[0687] Input: Quiz question

[0688] Output: User's answer

[0689] To submit your answer, enter text into the input form or select an option to confirm your answer.

[0690] (Application example 1)

[0691] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0692] It is difficult to efficiently learn the contents of e-books and retain them in your memory for a long period of time. In addition, there is a lack of review methods available after normal reading, and there is no system to regularly review what you have read, so it is difficult to expect the contents to be retained in your memory.

[0693] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0694] In this invention, the server includes means for [analyzing the contents of the e-book and extracting important information], means for [automatically generating quizzes based on the extracted information], means for [notifying the user of the generated quizzes at predetermined times], means for [regularly scheduling the timing of quiz notifications], means for [collecting and evaluating the user's answers], and means for [feeding back the evaluation results to the user]. This allows the contents of the e-book to be efficiently reviewed and the learned content to be retained in the memory for a long period of time.

[0695] "Means for analyzing the contents of e-books and extracting important information" refers to a function that analyzes the text data of e-books using natural language processing technology and text analysis algorithms, and automatically extracts key points and important information necessary for learning.

[0696] "Means for automatically generating quizzes based on extracted information" refers to a function that automatically generates specific question-style quizzes to confirm the learning content from extracted key points and important information.

[0697] The "means for notifying the user of the generated quiz at a predetermined timing" is a function for notifying the user of the generated quiz on the user's terminal based on a preset schedule.

[0698] The "means for periodically scheduling the timing of quiz notifications" is a function for setting and managing the schedule for quiz notifications at specific intervals so that users can review efficiently.

[0699] The "means for collecting and evaluating user answers" is a function for collecting answers entered by users to quizzes and evaluating whether the answers are correct or not based on the collected answers.

[0700] The "means for feeding back the evaluation results to the user" is a function for notifying the user of the evaluation results and providing feedback.

[0701] This invention provides a system that analyzes the contents of e-books, extracts important information, automatically generates quizzes, and periodically notifies users. This invention allows users to retain the contents of books they have finished reading for a long period of time. The system of this invention performs the following processing between a server, a terminal, and a user.

[0702] Server Processing

[0703] The server has multiple functions and performs the following processes:

[0704] 1. Content analysis of e-books

[0705] The server receives data of the electronic book that the user has finished reading.

[0706] Next, the content of the e-book is analyzed to extract important information and key points, using natural language processing techniques and text analysis algorithms, specifically the Hugging Face transformers library and pipeline.

[0707] Example: Analyze the contents of the e-book "Design Thinking" and extract key points such as "human-centered design" and "iterative prototyping."

[0708] 2. Quiz Generation

[0709] The server automatically generates a quiz based on the extracted key points. For example, it creates a specific question such as "What are the basic principles of human-centered design?" and sets the correct answer.

[0710] This allows the user to efficiently review what they have learned.

[0711] 3. Set a notification schedule

[0712] The server sets the schedule for notifying users of the quiz. For example, it sets the quiz to be notified after 2 days, 14 days, 60 days, etc. This schedule is managed using the scheduler function of apscheduler.

[0713] Terminal handling

[0714] The terminal is in charge of interfacing with the user and performs the following processes.

[0715] 1. Receive quiz notifications

[0716] The terminal receives the quiz notification sent from the server, which includes the generated quiz and the notification schedule.

[0717] Example: A user's smartphone will be notified of a quiz asking, "What are the basic principles of human-centered design?"

[0718] 2. View the quiz

[0719] The device displays the received quiz notification to the user. The display format provides a simple interface that makes it easy to answer. For example, it uses the push notification function of a smartphone or the display function of a head-mounted display.

[0720] Example: A quiz will appear on the notification screen of your smartphone, and you can enter answers by tapping.

[0721] 3. Collecting user responses

[0722] The terminal allows the user to enter answers to the quiz and transmits the entered answers to the server.

[0723] User Action

[0724] The user performs the following process through his / her own terminal.

[0725] 1. Check notifications

[0726] The user checks the notification from the device and becomes aware that there is a quiz. For example, when a notification arrives on the device two days later, the user checks it and answers the quiz.

[0727] 2. Answer the quiz

[0728] Users can interactively input answers to the displayed quiz questions. The input method is simple and intuitive, allowing answers to be entered by typing on the keyboard or by tapping.

[0729] Specific examples

[0730] For example, if a user finishes reading a book called "Design Thinking," the server analyzes the book's contents and extracts key points such as:

[0731] Human-Centered Design

[0732] Iterative Prototyping

[0733] The server then generates a quiz based on these key points:

[0734] "What are the basic principles of human-centered design?"

[0735] "What is the importance of iterative prototyping?"

[0736] The quiz will be sent to the user's device two days later. The user checks the notification and answers the quiz. The user's answers are sent to the server via the device, and the server evaluates the answers and notifies the user of the results.

[0737] Prompt Sentence Examples

[0738] Prompt sentence to input to the generative AI model:

[0739] "Please summarize the following: {eBook text}"

[0740] In this way, the system provides an environment in which readers can periodically review the contents of the book, thereby improving learning effectiveness.

[0741] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0742] Step 1: Receiving e-book data

[0743] The server receives the e-book data that the user has finished reading. As input, the e-book file uploaded by the user through the application is sent to the server. The server stores this e-book data in its internal storage for processing.

[0744] Step 2: Content analysis and important information extraction

[0745] The server analyzes the stored e-book data using natural language processing techniques and text analysis algorithms. The server generates a summary of the e-book based on the extracted key points and important information. This process uses Hugging Face's transformers library and pipeline. The output is a summary text, which becomes the input for the next process.

[0746] Step 3: Auto-generate a quiz

[0747] The server automatically generates quizzes based on the extracted key points and summary text. Specifically, the generative AI model receives a prompt, "Please summarize the following: {e-book text}," and converts it into a quiz format. For example, it generates a question such as, "What are the basic principles of human-centered design?" The output is multiple quizzes, which serve as input for the next process.

[0748] Step 4: Set up a notification schedule

[0749] The server sets the notification schedule for the generated quiz. It receives the quiz data as input and schedules the notification to occur at a specific time (for example, after 2 days, 14 days, or 60 days). This schedule is managed using the scheduler function of apscheduler. The scheduled notification task is set as the output.

[0750] Step 5: Receive quiz notifications

[0751] The terminal receives quiz notifications sent from the server based on a set schedule. As input, the server sends quiz notification data to the terminal. The terminal receives and stores this data.

[0752] Step 6: View the quiz

[0753] The device displays the received quiz notification to the user. This can be done via smartphone push notifications, in-app notifications, or head-mounted displays. The device references the quiz notification data as input, and displays the quiz to the user as output.

[0754] Step 7: User answers

[0755] The user inputs answers to the displayed quiz questions. The answers entered by the user using tap operations or keyboard input are saved on the device.

[0756] Step 8: User submits answer

[0757] The terminal receives the user's answer data as input and sends it to the server.

[0758] Step 9: Evaluate user responses

[0759] The server evaluates the received user answers. Specifically, it uses an internal evaluation algorithm to evaluate whether the answer is correct. It receives the user's answer data as input and generates an evaluation result as output.

[0760] Step 10: Feedback of evaluation results

[0761] The server feeds back the evaluation results to the user. It references the evaluation result data as input and generates a result notification as output, which it sends to the terminal. The terminal receives this notification and displays it to the user.

[0762] This allows the server, terminal, and user to work together, making it possible to efficiently review and memorize the contents of e-books.

[0763] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0764] This invention combines a system that analyzes the contents of e-books, extracts important information, and notifies users of automatically generated quizzes at predetermined times with an emotion engine that recognizes the user's emotions, in order to enhance user learning. This makes it possible to provide feedback according to the user's emotions and adjust the content and difficulty of the quizzes, further enhancing the effectiveness of learning.

[0765] Server Processing

[0766] 1. Content analysis of e-books

[0767] The server receives the data of the e-book that the user has finished reading, analyzes the contents of the e-book, and extracts important information and key points.

[0768] 2. Quiz Generation

[0769] The server automatically generates quizzes based on the extracted key points, asking about important information, and stores the generated quizzes in a database.

[0770] 3. Use of Emotion Engine

[0771] After generating the quiz, the server collects user emotional data and adjusts the content and difficulty of the generated quiz based on this data. For example, if the user is feeling stressed, the difficulty level can be lowered to reduce the user's psychological burden.

[0772] 4. Set up a notification schedule

[0773] The server sets a quiz notification schedule and sets the quiz to be notified at a specific timing.

[0774] Terminal handling

[0775] 1. Collecting Emotional Data

[0776] The device uses sensors such as a camera and microphone to collect emotional data from the user's facial expressions and tone of voice, and the collected emotional data is sent to a server.

[0777] 2. Receiving and viewing quiz notifications

[0778] The device receives quiz notifications sent from the server and displays them to the user in an intuitive interface, such as push notifications or in-app notifications.

[0779] 3. Collecting user responses and displaying evaluation results

[0780] The terminal collects the user's answers and sends them to the server, and after receiving the evaluation results from the server, displays the evaluation results to the user.

[0781] User Action

[0782] 1. Check and answer quiz notifications

[0783] The user checks the quiz displayed on the terminal and enters answers to the questions displayed. The user's answers are then sent from the terminal to the server.

[0784] 2. Receiving Feedback

[0785] Users receive evaluation results for their answers from their devices, which include not only whether their answers were correct or incorrect, but also feedback based on the user's individual emotional state.

[0786] Specific examples

[0787] For example, if a user finishes reading a book called "Design Thinking," the server analyzes the book's contents and extracts the following key points:

[0788] Human-Centered Design

[0789] Iterative Prototyping

[0790] The server then generates a quiz based on these key points, like this:

[0791] "What are the basic principles of human-centered design?"

[0792] "What is the importance of iterative prototyping?"

[0793] If the emotion engine analyzes the user's facial expressions using a camera and determines that the user is likely to be lacking concentration, the server adjusts the difficulty of the quiz and starts the user with easier questions.

[0794] The quiz will be notified to the user's device two days later. The user will check the notification and answer the quiz. The user's answers will be evaluated by the server, and the results will be sent back to the device. At this time, the server will take the user's emotional state into consideration and tailor the feedback appropriately to provide useful information to the user.

[0795] In this way, by adding emotion recognition to the e-book review system, the present invention provides a learning environment suited to the user's psychological state, further enhancing the learning effect.

[0796] The processing flow will be explained below.

[0797] Step 1:

[0798] When the user finishes reading the e-book, they press a button on the device to indicate they are finished reading, and the device notifies the server that they have finished reading.

[0799] Step 2:

[0800] The device sends information including the ID of the e-book that has been read and the user ID to the server. Based on the sent information, the server retrieves the data of the corresponding e-book.

[0801] Step 3:

[0802] The server begins analyzing the received data, using natural language processing techniques to extract important information and key points from the text. This process involves grammatical analysis, key word extraction, and contextual analysis.

[0803] Step 4:

[0804] The server automatically generates a quiz based on the extracted key points. The generated quiz includes specific questions and their corresponding correct answers. For example, a generated question might be, "What are the basic principles of human-centered design?"

[0805] Step 5:

[0806] The server stores the generated quiz in a database, along with the user ID and schedule information, in addition to the quiz content.

[0807] Step 6:

[0808] The server sets a notification schedule for the quiz, for example, to notify the user 2 days, 14 days, and 60 days after the user finishes reading. This notification schedule is stored in a database and is triggered based on a timestamp.

[0809] Step 7:

[0810] The emotion engine begins to operate. The device uses the camera and microphone to analyze the user's facial expressions and voice, collecting emotional data. This data is sent to the server in real time.

[0811] Step 8:

[0812] The server analyzes the received emotional data and recognizes the user's emotional state, for example, determining whether the user is feeling stressed or focused.

[0813] Step 9:

[0814] The server adjusts the content and difficulty of the generated quiz based on the user's emotional state. For example, if the user's stress level is high, the server may lower the difficulty of the quiz or reduce the number of questions.

[0815] Step 10:

[0816] When the set date and time arrives, the server sends a quiz notification to the terminal, which includes the adjusted quiz content and an answer interface.

[0817] Step 11:

[0818] The terminal receives the notification from the server and displays it to the user, who then checks the notification and accesses an interface for answering the quiz.

[0819] Step 12:

[0820] The user answers questions on the display screen of the device, for example, by inputting answers to questions such as "What are the basic principles of human-centered design?"

[0821] Step 13:

[0822] The terminal transmits the user's answer to the server. The transmitted data includes the user's answer and the associated quiz ID.

[0823] Step 14:

[0824] The server evaluates the received answer and determines whether it is correct or not, and the evaluation result is classified as correct, incorrect, partially correct, etc.

[0825] Step 15:

[0826] The server generates feedback based on the evaluation results, particularly depending on the user's emotional state. For example, if the user shows signs of anxiety, it may include an encouraging message.

[0827] Step 16:

[0828] The server sends the generated feedback to the device, which then displays the received feedback to the user. The user can check the evaluation results of their answers and the feedback based on their emotions.

[0829] Through these steps, users can effectively review the contents of the e-book and be provided with a learning environment that takes their emotional state into consideration.

[0830] Example 2

[0831] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0832] Conventional e-books lacked feedback to improve users' learning effectiveness or learning adjustments based on the user's emotional state. This made it difficult to provide an effective learning environment that matched the user's level of concentration and understanding. Furthermore, the fixed difficulty and content of quizzes did not reduce the user's psychological burden, potentially leading to a decline in motivation to learn.

[0833] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for [analyzing the contents of the e-book and extracting important information], a means for [automatically generating quizzes based on the extracted information], and a means for [adjusting the content and difficulty of the generated quizzes based on the user's emotional data]. This makes it possible to improve the user's learning effect and provide optimal feedback according to the user's emotional state.

[0834] "Analyzing the contents of e-books and extracting important information" refers to the process of analyzing the text data of e-books that users have finished reading and extracting important points and keywords necessary for learning and memorization.

[0835] "Automatically generating quizzes based on extracted information" refers to the process of automatically creating question-style questions to be posed to users using a program based on important information obtained through the analysis.

[0836] "Adjusting the content and difficulty of the generated quiz based on the user's emotional data" refers to the process of analyzing the user's emotional data (such as facial expressions and tone of voice) and optimizing the difficulty of the quiz and the content of the questions according to the results of that analysis.

[0837] "Notifying the user of the generated quiz at a predetermined timing" refers to a process of notifying the user of the quiz at an appropriate timing according to a pre-set schedule or the user's learning situation.

[0838] "Collecting emotion data from the terminal and transmitting it to the server" refers to the process of collecting data related to the user's emotions using sensors such as a camera and microphone, and transferring that data to the server.

[0839] "Collecting and evaluating user answers" refers to the process of acquiring the answers given by users to the quiz and evaluating the accuracy and content of the answers.

[0840] This invention combines a system that analyzes the contents of e-books, extracts important information, and presents automatically generated quizzes at predetermined times to enhance user learning, with an emotion engine that recognizes the user's emotions. Implementing this system requires collaboration between a server, terminals, and users.

[0841] Server Processing

[0842] The server first receives the e-book data that the user has finished reading. The e-book data is uploaded in a format such as an EPUB file. Next, it uses a natural language processing engine such as Google Cloud Natural Language API or IBM Watson Natural Language Understanding to analyze the content of the e-book and extract important information and key points.

[0843] Based on the extracted keypoints, a generative AI model (e.g., OpenAI GPT-4) is used to automatically generate a quiz, which is then stored in a database (e.g., MySQL or MongoDB) for later use.

[0844] The server then collects the emotional data sent from the device and analyzes the user's emotional data using the Microsoft Azure Emotion API and Affectiva SDK. Based on the analysis results, the content and difficulty of the quiz can be adjusted. For example, if the user is feeling stressed, the difficulty of the quiz can be lowered to reduce the psychological burden.

[0845] Finally, set up a notification schedule for the quiz and notify users of the quiz at the appropriate time using scheduling software such as a Cron job or Amazon CloudWatch Events.

[0846] Terminal handling

[0847] The device collects the user's emotional data using sensors such as a camera and microphone. This can be done using the device's built-in camera or an external camera or microphone. The emotional data includes information on facial expressions and tone of voice, and the collected data is sent to a server.

[0848] The app receives quiz notifications sent from the server and displays them to the user. Notifications can be sent in the form of push notifications or in-app notifications. Push notifications are implemented using Firebase Cloud Messaging (FCM), and in-app notifications are implemented using mobile frameworks such as React Native or Swift.

[0849] The device collects the user's answers and sends them to the server. Once the answers are evaluated, the server sends the results back to the user. The evaluation results not only indicate whether the answer was correct or incorrect, but also provide feedback based on the user's emotional state.

[0850] User Action

[0851] The user checks the quiz displayed on the device and enters answers to the questions. The user's answers are sent from the device to the server, where they are evaluated. The evaluation results are returned to the device, and correct / incorrect answers and feedback are displayed. The feedback includes advice that takes into account the user's emotional state, further enhancing the learning effect.

[0852] Specific examples

[0853] For example, suppose a user has finished reading a book called "Design Thinking." The server analyzes the book's contents and extracts key points such as "human-centered design" and "iterative prototyping." The server then sends the following prompt to the generative AI model: "Based on the key points in the book "Design Thinking," please generate the following quiz. The key points are 'human-centered design' and 'iterative prototyping.'"

[0854] The generative AI model generates a quiz like this:

[0855] "What are the basic principles of human-centered design?"

[0856] "What is the importance of iterative prototyping?"

[0857] If the emotion engine analyzes the user's facial expressions using a camera and determines that the user is likely to be lacking concentration, the server adjusts the difficulty of the quiz, starting with easier questions. The quiz is notified to the user's device two days later. The user checks the notification and answers the quiz. The server evaluates the user's answers and sends the results back to the device. At this time, feedback is provided taking into account the user's emotional state.

[0858] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0859] Server Processing

[0860] Step 1: Content analysis of e-books

[0861] Input: The e-book data the user has finished reading (e.g., EPUB file)

[0862] How it works: The server receives e-book data uploaded by a user, in a format such as an EPUB file.

[0863] Data processing: The server analyzes the text data of the e-book using the Google Cloud Natural Language API, identifying nouns, verbs, adjectives, etc. and mapping their relationships.

[0864] Output: Extracted important information or key points.

[0865] Step 2: Generate the quiz

[0866] Input: Extracted keypoints

[0867] How it works: The server generates a prompt based on the extracted keypoints using a generative AI model (e.g., OpenAI GPT-4).

[0868] Data processing: The server sends prompts to the generative AI model to automatically generate quizzes.

[0869] Output: An automatically generated quiz, specifically using the prompt: "Generate the following quiz based on the key points from the book 'Design Thinking'. The key points are 'human-centered design' and 'iterative prototyping'."

[0870] Step 3: Analyze sentiment data and tailor the quiz

[0871] Input: Generated quiz, emotion data collected from the device

[0872] How it works: The server receives emotion data sent from the device, analyzes it using the Microsoft Azure Emotion API, and evaluates the user's emotional state (e.g., stress level or concentration level) based on the analysis results.

[0873] Data processing: The content and difficulty of the quiz are adjusted based on the analysis of emotional data. For example, if the user is feeling stressed, the difficulty of the quiz can be lowered to reduce the psychological burden.

[0874] Output: The adapted quiz.

[0875] Step 4: Schedule and run notifications

[0876] Inputs: Adjusted quiz, scheduling data

[0877] How it works: The server schedules quiz notifications using Cron jobs and Amazon CloudWatch Events.

[0878] Data processing: Notify quizzes at specified dates and times based on a set schedule.

[0879] Output: Quiz notification to user device

[0880] Terminal handling

[0881] Step 1: Collect and send emotion data

[0882] Input: User's facial expression, tone of voice

[0883] How it works: The device uses the camera and microphone to collect emotional data about the user, including information about facial expressions and tone of voice.

[0884] Data processing: Analyzing emotional data using facial recognition and voice analysis algorithms.

[0885] Output: Send the analyzed emotion data to the server.

[0886] Step 2: Receive and view quiz notifications

[0887] Input: Quiz notification from the server

[0888] Operation: The device receives a quiz notification sent from the server and displays the notification to the user.

[0889] Data processing: Display received notifications as push notifications or in-app notifications using services such as Firebase Cloud Messaging (FCM).

[0890] Output: The quiz notification displayed to the user

[0891] Step 3: Collect user responses and display evaluation results

[0892] Input: User's answer

[0893] Operation: The device collects the answers entered by the user and sends them to the server.

[0894] Data processing: Receive the response data in the form and send it to the server via an HTTP POST request.

[0895] Output: Receives the evaluation results from the server and displays them to the user.

[0896] User Action

[0897] Step 1: Check the quiz notification and answer

[0898] Input: Quiz notification from your device

[0899] How it works: The user reviews the quiz and enters answers to the questions displayed.

[0900] Output: User's answer typed into the terminal

[0901] Step 2: Receiving feedback

[0902] Input: Evaluation result from the server

[0903] How it works: The user checks the evaluation results displayed on the device. The evaluation results include not only correct or incorrect answers, but also feedback based on the user's emotional state.

[0904] Output: Evaluation results and feedback displayed to the user

[0905] In this way, the roles of the server, terminal, and user are clearly separated and a detailed processing flow is described.

[0906] (Application example 2)

[0907] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0908] Conventional e-book learning systems did not provide feedback or adjust the difficulty of quizzes based on the user's emotional state. This could lead to stress and reduced concentration, resulting in ineffective learning. Furthermore, the notification function to prompt users to review at the appropriate time was insufficient.

[0909] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0910] In this invention, the server includes means for analyzing the content of the electronic medium and extracting important information, means for automatically generating quizzes based on the extracted information, means for notifying the user of the generated quizzes at a predetermined timing, means for analyzing and collecting the user's emotional state, means for adjusting the content and difficulty of the quiz according to the user's emotional state, and means for collecting and evaluating the user's answers, thereby enabling effective feedback and adjustment of the difficulty of the quiz according to the user's emotional state.

[0911] "Electronic media" is a collection of information stored or distributed in electronic form.

[0912] "Analysis" means analyzing data in detail to understand its structure and meaning.

[0913] "Significant information" is data that is particularly valuable for a particular purpose or context and is necessary for understanding and decision-making.

[0914] A "quiz" is a set of questions designed to gauge a user's knowledge and understanding.

[0915] "Automatic generation" means that something is created automatically by an algorithm or program, without human intervention.

[0916] "Notifying" means notifying a user of specific information at a specific time.

[0917] "Emotional state" refers to the user's psychological state, including stress, concentration, fatigue, and the like.

[0918] "Adjust" means to change or optimize according to conditions or circumstances.

[0919] "Collect" means to gather specific data.

[0920] To "evaluate" means to judge data or results based on specific criteria.

[0921] This invention combines a system that analyzes the contents of electronic media, extracts important information, and notifies users of automatically generated quizzes at the appropriate time with an emotion engine that recognizes the user's emotional state. This makes it possible to provide feedback according to the user's emotions and adjust the content and difficulty of the quiz, thereby improving learning effectiveness.

[0922] Server Processing

[0923] 1. Content analysis of electronic media:

[0924] The server receives the data from the electronic media that the user has finished reading and analyzes its content. Natural language processing (NLP) is used for the analysis to extract important information and key points. Specifically, Python and NLTK (Natural Language Toolkit) are used.

[0925] 2. Generate the quiz:

[0926] The server automatically generates quizzes that ask about important information based on the extracted key points. The generated quizzes are stored in a database. A generative AI model is used to generate the quizzes.

[0927] 3. Use of Emotion Engine:

[0928] The server analyzes the user's emotional state. The user's emotional state is evaluated based on data collected using the smartphone's camera and microphone. Specifically, OpenCV and Google Cloud Vision API are used. The content and difficulty of the quiz are adjusted according to the user's emotional state.

[0929] 4. Set the notification schedule:

[0930] The server sets a quiz notification schedule and configures the quiz notification to occur at specific times, and this schedule is adjusted based on the user's learning pace and emotional state.

[0931] Terminal handling

[0932] 1. Collecting Emotional Data:

[0933] The device uses sensors such as a camera and microphone to collect emotional data from the user's facial expressions and tone of voice. The collected emotional data is sent to a server, which uses the Microsoft Azure Emotion API for specific emotional analysis.

[0934] 2. Receive and view quiz notifications:

[0935] The device receives the quiz notification sent from the server and displays it to the user. The display format provides an intuitive interface such as push notification or in-app notification.

[0936] 3. Collect user responses and display the evaluation results:

[0937] The terminal collects the user's answers and sends them to the server, and after receiving the evaluation results from the server, displays the evaluation results to the user.

[0938] User Action

[0939] 1. Check the quiz notification and answer:

[0940] The user checks the quiz displayed on the terminal and enters answers to the questions displayed. The user's answers are then sent from the terminal to the server.

[0941] 2. Receiving Feedback:

[0942] Users receive evaluation results for their answers from their devices, which include not only whether their answers were correct or incorrect, but also feedback based on the user's individual emotional state.

[0943] Specific examples

[0944] For example, if a user finishes reading a book called "Design Thinking," the server will analyze the book's contents and extract key points such as:

[0945] Human-Centered Design

[0946] Iterative Prototyping

[0947] The server then generates a quiz based on these key points, like this:

[0948] "What are the basic principles of human-centered design?"

[0949] "What is the importance of iterative prototyping?"

[0950] If the emotion engine analyzes the user's facial expressions using a camera and determines that the user is likely to be lacking concentration, the server adjusts the difficulty of the quiz and starts the user with easier questions.

[0951] The quiz will be notified to the user's device two days later. The user will check the notification and answer the quiz. The user's answers will be evaluated by the server, and the results will be sent back to the device. At this time, the server will take into account the user's emotional state and tailor the feedback appropriately to provide useful information to the user. The following prompt sentences can be used:

[0952] "Please explain the basic principles of human-centered design (e.g., designing based on user needs)."

[0953] In this way, by adding emotion recognition to an electronic review system, the present invention provides a learning environment suited to the user's psychological state, further enhancing learning effectiveness.

[0954] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0955] Step 1:

[0956] The server receives the data from the electronic media that the user has finished reading. The input is the text data from the electronic media, which is analyzed to extract important information and key points. Specifically, natural language processing (NLP) technology is used to tokenize the text data and extract important words and phrases. The extracted information is output as key points.

[0957] Step 2:

[0958] The server automatically generates a quiz based on the extracted keypoints. The input is the extracted keypoints, and based on these, a generative AI model is used to create quiz questions. Specifically, the generative AI model generates questions related to the keypoints and stores them in a database. The generated quiz is then output.

[0959] Step 3:

[0960] The server sets the quiz notification schedule. The input is the user's learning progress data and emotional state data, and the notification timing is calculated based on this. Specifically, the server takes into account the user's learning patterns and emotional state to set a schedule for quiz notifications at the optimal timing. The set notification schedule is output.

[0961] Step 4:

[0962] The device collects the user's emotional state. The input is sensor data from the camera and microphone, which is analyzed to determine the user's emotional state. Specifically, the device uses OpenCV and the Microsoft Azure Emotion API to analyze the user's facial expressions and tone of voice from the collected sensor data. The analyzed emotional state data is output.

[0963] Step 5:

[0964] The device receives the quiz notification sent from the server and displays it to the user. The input is the quiz question sent from the server, which is then displayed to the user. Specifically, the device uses push notifications or in-app notifications to intuitively inform the user of the quiz. The displayed quiz is then output.

[0965] Step 6:

[0966] The user answers a quiz displayed on the terminal. The input is the quiz question displayed on the terminal, and the user inputs the answer to that question. In concrete terms, the user inputs the answer to the question and sends it to the server via the terminal. The user's answer is then output.

[0967] Step 7:

[0968] The server collects and evaluates the user's answers. The input is the user's answer data, which is then evaluated. Specifically, the server compares the user's answers with the correct answer data and generates an evaluation result. The generated evaluation result is then output.

[0969] Step 8:

[0970] The server adjusts the feedback according to the user's emotional state and notifies the user of the evaluation results. The input is the evaluation results and the user's emotional state data, and the feedback is created based on this. Specifically, the server generates feedback according to the user's emotional state and sends it to the terminal. The adjusted feedback is then output.

[0971] Step 9:

[0972] The device displays the evaluation results and feedback received from the server to the user. The input is the evaluation results and feedback sent from the server, which are then displayed to the user. Specifically, the device uses a notification function or an in-app interface to visually provide the evaluation results and feedback to the user. The displayed evaluation results and feedback are then output.

[0973] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0974] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0975] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0976] [Third embodiment]

[0977] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0978] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0979] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0980] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0981] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0982] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0983] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0984] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0985] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0986] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0987] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0988] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0989] This invention is a system that analyzes the contents of e-books, extracts important information, automatically generates quizzes, and allows readers to review the content. This system allows readers to retain the contents of books they have read for a long period of time. Specifically, the following process is carried out between the server, the terminal, and the user.

[0990] Server Processing

[0991] 1. Content analysis of e-books

[0992] The server receives the data of the e-book that the user has finished reading. It then analyzes the content of the e-book and extracts important information and key points. In this process, it uses natural language processing technology and text analysis algorithms to extract the essence of the text necessary for learning.

[0993] 2. Quiz Generation

[0994] The server automatically generates quizzes based on the extracted key points. For example, it creates specific questions such as "What are the basic principles of human-centered design?" and sets the correct answers. The generated quizzes are stored in a database.

[0995] 3. Set a notification schedule

[0996] The server sets a schedule for notifying users of quizzes, for example, setting it to notify users at specific times, such as after 2 days, 14 days, or 60 days, encouraging regular review and promoting knowledge retention.

[0997] Terminal handling

[0998] 1. Receive quiz notifications

[0999] The terminal receives the quiz notification sent from the server, which includes the generated quiz and the notification schedule.

[1000] 2. View the quiz

[1001] The device displays the received quiz notification to the user in a simple and easy-to-answer interface, such as a push notification on a smartphone or an in-app notification.

[1002] 3. Collecting user responses

[1003] The terminal allows the user to enter answers to the quiz and transmits the entered answers to the server.

[1004] User Action

[1005] 1. Check notifications

[1006] The user checks the notification from the device and becomes aware that there is a quiz. For example, when a notification arrives on the device two days later, the user checks it and answers the quiz.

[1007] 2. Answer the quiz

[1008] Users can interactively input answers to the displayed quiz questions. The input method is simple and intuitive, allowing answers to be entered by typing on the keyboard or by tapping.

[1009] Specific examples

[1010] For example, if a user finishes reading a book called "Design Thinking," the server analyzes the book's contents and extracts important key points, such as:

[1011] Human-Centered Design

[1012] Iterative Prototyping

[1013] The server then generates a quiz based on these key points:

[1014] "What are the basic principles of human-centered design?"

[1015] "What is the importance of iterative prototyping?"

[1016] The quiz will be sent to the user's device two days later. The user checks the notification and answers the quiz. The user's answers are sent to the server via the device, and the server evaluates the answers and notifies the user of the results. Through this process, the user can review the important content of the book and retain it in their long-term memory.

[1017] In this way, the system provides an environment in which readers can periodically review the contents of the book, thereby improving learning effectiveness.

[1018] The processing flow will be explained below.

[1019] Step 1:

[1020] When the user finishes reading an e-book, he or she presses a button on the device to record the end of reading, which notifies the system of the end of book reading event.

[1021] Step 2:

[1022] The device sends information about the e-book that the user has finished reading to the server. The sent information includes the user ID and the e-book ID.

[1023] Step 3:

[1024] Based on the received information, the server retrieves the contents of the e-book and begins analyzing it, using natural language processing technology to extract important key points and learning essences from the text.

[1025] Step 4:

[1026] The server automatically generates quizzes based on information extracted from the analysis results. The quizzes are created in the form of specific questions based on important key points. For example, a question might be generated such as, "What are the basic principles of human-centered design?"

[1027] Step 5:

[1028] The server saves the generated quiz in a database, along with the user ID, quiz content, and correct answer information.

[1029] Step 6:

[1030] The server sets a notification schedule for the quiz. The notification timing can be set to a specific schedule, such as 2 days, 14 days, or 60 days later.

[1031] Step 7:

[1032] When the set time comes, the server sends a quiz notification to the terminal, which includes the generated quiz and an interface for answering the question.

[1033] Step 8:

[1034] The terminal displays the received quiz notification to the user, who then checks the notification and opens an interface for answering the quiz.

[1035] Step 9:

[1036] The user inputs answers to quiz questions through the device interface, for example, "What are the basic principles of human-centered design?"

[1037] Step 10:

[1038] The terminal sends the user's answer to the server. The sent data includes the user ID, quiz ID, and the user's answer.

[1039] Step 11:

[1040] The server evaluates the received user's answer and checks whether the answer is correct by comparing it with the correct answer information.

[1041] Step 12:

[1042] The server sends the evaluation results to the device, including feedback such as "Correct" if the user's answer is correct, or "Incorrect" if the answer is incorrect.

[1043] Step 13:

[1044] The device notifies the user of the evaluation results received from the server, allowing the user to check whether their answers are correct and receive feedback to improve their understanding.

[1045] Example 1

[1046] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1047] With conventional methods for viewing electronic publications, it is difficult to periodically review the content once it has been read and retain it in your memory for a long period of time. In particular, it is difficult to extract important information and key points and provide an environment for effective learning.

[1048] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1049] In this invention, the server includes means for analyzing the contents of electronic publications and extracting important information, means for automatically generating questions based on the extracted information, and means for notifying the user of the generated questions at a predetermined timing, thereby enabling the user to periodically review important information in electronic publications and retain it in their memory for a long period of time.

[1050] An "electronic publication" is a book, information, or document provided in electronic form.

[1051] "Content analysis" is the process of analyzing text data and extracting important information or key points.

[1052] "Key information" refers to knowledge or facts contained in a text that are particularly important for learning and understanding.

[1053] "Automatic generation" is a process in which a machine or program independently creates questions or poses questions with minimal human intervention.

[1054] "Questions" are questions or quizzes that users answer to review the content they have learned and check their level of understanding.

[1055] The term "predetermined timing" refers to a specific time or period that has been set in advance.

[1056] "User" means an individual or organization that uses this system.

[1057] "Notification" is the act of informing users of information or issues and encouraging them to receive them.

[1058] An "answer" is an answer or response provided by a user to a question.

[1059] "Evaluation" is the process of analyzing the collected responses and determining their accuracy and validity.

[1060] A "server" is a computer system or network device for processing, storing, and serving information.

[1061] This invention is a system that analyzes the contents of electronic publications, extracts important information, and automatically generates questions. The system aims to encourage users to periodically review what they have read and retain it in their memory for a long period of time by fulfilling the roles of the server, terminal, and user.

[1062] Server processing format

[1063] Hardware and Software Use

[1064] The server has a network connection to receive digital publication data from users. The received data is temporarily stored on the server's storage (e.g., HDD or SSD). The server has installed natural language processing libraries (e.g., SpaCy, NLTK, Transformers, etc.) for content analysis.

[1065] Data analysis

[1066] The server analyzes the text of electronic publications using natural language processing techniques, tokenizing the text and extracting important keywords and phrases using TF-IDF and word embedding techniques, thereby extracting the essence necessary for learning.

[1067] Automatic question generation

[1068] Based on the extracted key points, a generative AI model (e.g., GPT-3 or T5 model) can be used to generate specific questions and their answers. For example, it is possible to generate a question such as, "What are the fundamental principles of human-centered design?" and its corresponding answer.

[1069] Save quizzes and schedule notifications

[1070] The generated questions are saved in a database (e.g. MySQL, PostgreSQL). At that time, the server sets a schedule for notifying users of the quiz. For example, it can set the quiz to be notified after 2 days, 14 days, or 60 days. This schedule information is also saved in the database.

[1071] Submit a quiz

[1072] Based on the notification schedule, quizzes are sent to devices at the set date and time using methods such as push notifications and email notifications.

[1073] Terminal processing format

[1074] Receive and view quiz notifications

[1075] The device receives quiz notifications sent from the server. The notifications include the generated quiz questions and answer options. The device displays the received quiz via push notification or within the app. The display format provides a simple and intuitive user interface.

[1076] Collecting and sending user responses

[1077] The device provides an input form and buttons for users to answer the quiz, temporarily stores the answers entered by the user, and sends the answer data to the server for appropriate association.

[1078] User processing format

[1079] Check notifications and answer quizzes

[1080] The user confirms the quiz notification on their device and knows that a quiz has arrived. They enter their answers to the quiz displayed on their device and press the submit button to send the answers to the server. Through this process, the user can review the important content of the book and retain it in their long-term memory.

[1081] Examples of concrete examples and prompts

[1082] For example, if a user finishes reading a book called "Design Thinking," the server analyzes the book's contents and extracts important key points, such as:

[1083] Human-Centered Design

[1084] Iterative Prototyping

[1085] The server then generates a problem based on these keypoints, such as:

[1086] "What are the basic principles of human-centered design?"

[1087] "What is the importance of iterative prototyping?"

[1088] The quiz will be sent to the user's device two days later. The user checks the notification and answers the quiz. The user's answers are sent to the server via the device, and the server evaluates the answers and notifies the user of the results. This allows the user to retain important information from the electronic publication in their long-term memory.

[1089] Example prompts for generative AI models:

[1090] "Generate one quiz each about the book 'Design Thinking', the 'Basic Principles of Human-Centered Design' and the 'Importance of Iterative Prototyping'."

[1091] In this way, the system provides an environment in which readers can periodically review the contents of the book, thereby improving learning effectiveness.

[1092] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1093] Server Processing Steps

[1094] Step 1: Receiving electronic publication data

[1095] Specific actions

[1096] The server receives electronic publication files (e.g., PDF, ePub) sent by users.

[1097] Input: Electronic publication file

[1098] Output: A file temporarily stored in the server's storage

[1099] The server stores the received file in storage and records the file path.

[1100] Step 2: Content analysis

[1101] Specific actions

[1102] The server reads the text of the electronic publication stored in the storage and tokenizes the text using a natural language processing library (e.g., SpaCy, NLTK).

[1103] Input: Text data of electronic publications

[1104] Output: Tokenized text, important keywords and phrases

[1105] Extract key points necessary for learning from tokenized text using TF-IDF and word embeddings.

[1106] Step 3: Automatic question generation

[1107] Specific actions

[1108] Based on the extracted keypoints, a prompt sentence is input into a generative AI model (e.g., GPT-3) to generate a question sentence.

[1109] Input: Keypoint data, prompt

[1110] Output: Generated question statement

[1111] Use prompts to generate questions such as, "What are the basic principles of human-centered design?"

[1112] Step 4: Save the quiz to the database

[1113] Specific actions

[1114] The generated questions and their answers are saved in a database (e.g. MySQL).

[1115] Input: Generated question and answer

[1116] Output: Quiz data stored in a database

[1117] The database stores quiz questions, answers, user IDs, notification schedules, etc.

[1118] Step 5: Set up the quiz notification schedule

[1119] Specific actions

[1120] The server sets a schedule for notifying the generated quiz (e.g., after 2 days, 14 days, or 60 days).

[1121] Input: Quiz data, notification schedule

[1122] Output: Quiz data with schedule information added

[1123] The schedule is recorded in the database and quizzes are set to be notified periodically.

[1124] Step 6: Submit your quiz

[1125] Specific actions

[1126] The server sends the quiz to the terminal based on the set notification schedule.

[1127] Input: Schedule information, quiz data

[1128] Output: Quiz notification to device

[1129] Notifications can be sent using push notifications or email notifications.

[1130] Terminal processing steps

[1131] Step 1: Receive quiz notifications

[1132] Specific actions

[1133] The terminal receives the quiz notification sent from the server.

[1134] Input:QuizNotification

[1135] Output: Notification received alert

[1136] The terminal displays an alert to inform the user of the received notification.

[1137] Step 2: View the quiz

[1138] Specific actions

[1139] The terminal displays the received quiz content to the user.

[1140] Input: Quiz notification data

[1141] Output: The quiz displayed in the user interface

[1142] The quizzes are displayed using a simple UI, for example via push notifications or in-app notifications.

[1143] Step 3: Collect user responses

[1144] Specific actions

[1145] The terminal provides an interface for the user to input answers to the quiz.

[1146] Input: User's answer

[1147] Output: Temporarily stored user response data

[1148] The answers entered by the user are temporarily stored for transmission to the server.

[1149] Step 4: Submit your response data

[1150] Specific actions

[1151] The terminal transmits the user's response data to the server.

[1152] Input: User response data

[1153] Output: Response data sent to the server

[1154] When sent to the server, the user ID and quiz ID are also sent and associated appropriately.

[1155] User processing steps

[1156] Step 1: Check notifications

[1157] Specific actions

[1158] The user checks the quiz notification that arrives on the device.

[1159] Input:QuizNotification

[1160] Output: User's perceived behavior

[1161] Tap the notification to open the app and go to the quiz screen.

[1162] Step 2: Take the quiz

[1163] Specific actions

[1164] The user inputs answers to the displayed quiz questions.

[1165] Input: Quiz question

[1166] Output: User's answer

[1167] To submit your answer, enter text into the input form or select an option to confirm your answer.

[1168] (Application example 1)

[1169] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1170] It is difficult to efficiently learn the contents of e-books and retain them in your memory for a long period of time. In addition, there is a lack of review methods available after normal reading, and there is no system to regularly review what you have read, so it is difficult to expect the contents to be retained in your memory.

[1171] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1172] In this invention, the server includes means for [analyzing the contents of the e-book and extracting important information], means for [automatically generating quizzes based on the extracted information], means for [notifying the user of the generated quizzes at predetermined times], means for [regularly scheduling the timing of quiz notifications], means for [collecting and evaluating the user's answers], and means for [feeding back the evaluation results to the user]. This allows the contents of the e-book to be efficiently reviewed and the learned content to be retained in the memory for a long period of time.

[1173] "Means for analyzing the contents of e-books and extracting important information" refers to a function that analyzes the text data of e-books using natural language processing technology and text analysis algorithms, and automatically extracts key points and important information necessary for learning.

[1174] "Means for automatically generating quizzes based on extracted information" refers to a function that automatically generates specific question-style quizzes to confirm the learning content from extracted key points and important information.

[1175] The "means for notifying the user of the generated quiz at a predetermined timing" is a function for notifying the user of the generated quiz on the user's terminal based on a preset schedule.

[1176] The "means for periodically scheduling the timing of quiz notifications" is a function for setting and managing the schedule for quiz notifications at specific intervals so that users can review efficiently.

[1177] The "means for collecting and evaluating user answers" is a function for collecting answers entered by users to quizzes and evaluating whether the answers are correct or not based on the collected answers.

[1178] The "means for feeding back the evaluation results to the user" is a function for notifying the user of the evaluation results and providing feedback.

[1179] This invention provides a system that analyzes the contents of e-books, extracts important information, automatically generates quizzes, and periodically notifies users. This invention allows users to retain the contents of books they have finished reading for a long period of time. The system of this invention performs the following processing between a server, a terminal, and a user.

[1180] Server Processing

[1181] The server has multiple functions and performs the following processes:

[1182] 1. Content analysis of e-books

[1183] The server receives data of the electronic book that the user has finished reading.

[1184] Next, the content of the e-book is analyzed to extract important information and key points, using natural language processing techniques and text analysis algorithms, specifically the Hugging Face transformers library and pipeline.

[1185] Example: Analyze the contents of the e-book "Design Thinking" and extract key points such as "human-centered design" and "iterative prototyping."

[1186] 2. Quiz Generation

[1187] The server automatically generates a quiz based on the extracted key points. For example, it creates a specific question such as "What are the basic principles of human-centered design?" and sets the correct answer.

[1188] This allows the user to efficiently review what they have learned.

[1189] 3. Set a notification schedule

[1190] The server sets the schedule for notifying users of the quiz. For example, it sets the quiz to be notified after 2 days, 14 days, 60 days, etc. This schedule is managed using the scheduler function of apscheduler.

[1191] Terminal handling

[1192] The terminal is in charge of interfacing with the user and performs the following processes.

[1193] 1. Receive quiz notifications

[1194] The terminal receives the quiz notification sent from the server, which includes the generated quiz and the notification schedule.

[1195] Example: A user's smartphone will be notified of a quiz asking, "What are the basic principles of human-centered design?"

[1196] 2. View the quiz

[1197] The device displays the received quiz notification to the user. The display format provides a simple interface that makes it easy to answer. For example, it uses the push notification function of a smartphone or the display function of a head-mounted display.

[1198] Example: A quiz will appear on the notification screen of your smartphone, and you can enter answers by tapping.

[1199] 3. Collecting user responses

[1200] The terminal allows the user to enter answers to the quiz and transmits the entered answers to the server.

[1201] User Action

[1202] The user performs the following process through his / her own terminal.

[1203] 1. Check notifications

[1204] The user checks the notification from the device and becomes aware that there is a quiz. For example, when a notification arrives on the device two days later, the user checks it and answers the quiz.

[1205] 2. Answer the quiz

[1206] Users can interactively input answers to the displayed quiz questions. The input method is simple and intuitive, allowing answers to be entered by typing on the keyboard or by tapping.

[1207] Specific examples

[1208] For example, if a user finishes reading a book called "Design Thinking," the server analyzes the book's contents and extracts key points such as:

[1209] Human-Centered Design

[1210] Iterative Prototyping

[1211] The server then generates a quiz based on these key points:

[1212] "What are the basic principles of human-centered design?"

[1213] "What is the importance of iterative prototyping?"

[1214] The quiz will be sent to the user's device two days later. The user checks the notification and answers the quiz. The user's answers are sent to the server via the device, and the server evaluates the answers and notifies the user of the results.

[1215] Prompt Sentence Examples

[1216] Prompt sentence to input to the generative AI model:

[1217] "Please summarize the following: {eBook text}"

[1218] In this way, the system provides an environment in which readers can periodically review the contents of the book, thereby improving learning effectiveness.

[1219] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1220] Step 1: Receiving e-book data

[1221] The server receives the e-book data that the user has finished reading. As input, the e-book file uploaded by the user through the application is sent to the server. The server stores this e-book data in its internal storage for processing.

[1222] Step 2: Content analysis and important information extraction

[1223] The server analyzes the stored e-book data using natural language processing techniques and text analysis algorithms. The server generates a summary of the e-book based on the extracted key points and important information. This process uses Hugging Face's transformers library and pipeline. The output is a summary text, which becomes the input for the next process.

[1224] Step 3: Auto-generate a quiz

[1225] The server automatically generates quizzes based on the extracted key points and summary text. Specifically, the generative AI model receives a prompt, "Please summarize the following: {e-book text}," and converts it into a quiz format. For example, it generates a question such as, "What are the basic principles of human-centered design?" The output is multiple quizzes, which serve as input for the next process.

[1226] Step 4: Set up a notification schedule

[1227] The server sets the notification schedule for the generated quiz. It receives the quiz data as input and schedules the notification to occur at a specific time (for example, after 2 days, 14 days, or 60 days). This schedule is managed using the scheduler function of apscheduler. The scheduled notification task is set as the output.

[1228] Step 5: Receive quiz notifications

[1229] The terminal receives quiz notifications sent from the server based on a set schedule. As input, the server sends quiz notification data to the terminal. The terminal receives and stores this data.

[1230] Step 6: View the quiz

[1231] The device displays the received quiz notification to the user. This can be done via smartphone push notifications, in-app notifications, or head-mounted displays. The device references the quiz notification data as input, and displays the quiz to the user as output.

[1232] Step 7: User answers

[1233] The user inputs answers to the displayed quiz questions. The answers entered by the user using tap operations or keyboard input are saved on the device.

[1234] Step 8: User submits answer

[1235] The terminal receives the user's answer data as input and sends it to the server.

[1236] Step 9: Evaluate user responses

[1237] The server evaluates the received user answers. Specifically, it uses an internal evaluation algorithm to evaluate whether the answer is correct. It receives the user's answer data as input and generates an evaluation result as output.

[1238] Step 10: Feedback of evaluation results

[1239] The server feeds back the evaluation results to the user. It references the evaluation result data as input and generates a result notification as output, which it sends to the terminal. The terminal receives this notification and displays it to the user.

[1240] This allows the server, terminal, and user to work together, making it possible to efficiently review and memorize the contents of e-books.

[1241] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1242] This invention combines a system that analyzes the contents of e-books, extracts important information, and notifies users of automatically generated quizzes at predetermined times with an emotion engine that recognizes the user's emotions, in order to enhance user learning. This makes it possible to provide feedback according to the user's emotions and adjust the content and difficulty of the quizzes, further enhancing the effectiveness of learning.

[1243] Server Processing

[1244] 1. Content analysis of e-books

[1245] The server receives the data of the e-book that the user has finished reading, analyzes the contents of the e-book, and extracts important information and key points.

[1246] 2. Quiz Generation

[1247] The server automatically generates quizzes based on the extracted key points, asking about important information, and stores the generated quizzes in a database.

[1248] 3. Use of Emotion Engine

[1249] After generating the quiz, the server collects user emotional data and adjusts the content and difficulty of the generated quiz based on this data. For example, if the user is feeling stressed, the difficulty level can be lowered to reduce the user's psychological burden.

[1250] 4. Set up a notification schedule

[1251] The server sets a quiz notification schedule and sets the quiz to be notified at a specific timing.

[1252] Terminal handling

[1253] 1. Collecting Emotional Data

[1254] The device uses sensors such as a camera and microphone to collect emotional data from the user's facial expressions and tone of voice, and the collected emotional data is sent to a server.

[1255] 2. Receiving and viewing quiz notifications

[1256] The device receives quiz notifications sent from the server and displays them to the user in an intuitive interface, such as push notifications or in-app notifications.

[1257] 3. Collecting user responses and displaying evaluation results

[1258] The terminal collects the user's answers and sends them to the server, and after receiving the evaluation results from the server, displays the evaluation results to the user.

[1259] User Action

[1260] 1. Check and answer quiz notifications

[1261] The user checks the quiz displayed on the terminal and enters answers to the questions displayed. The user's answers are then sent from the terminal to the server.

[1262] 2. Receiving Feedback

[1263] Users receive evaluation results for their answers from their devices, which include not only whether their answers were correct or incorrect, but also feedback based on the user's individual emotional state.

[1264] Specific examples

[1265] For example, if a user finishes reading a book called "Design Thinking," the server analyzes the book's contents and extracts the following key points:

[1266] Human-Centered Design

[1267] Iterative Prototyping

[1268] The server then generates a quiz based on these key points, like this:

[1269] "What are the basic principles of human-centered design?"

[1270] "What is the importance of iterative prototyping?"

[1271] If the emotion engine analyzes the user's facial expressions using a camera and determines that the user is likely to be lacking concentration, the server adjusts the difficulty of the quiz and starts the user with easier questions.

[1272] The quiz will be notified to the user's device two days later. The user will check the notification and answer the quiz. The user's answers will be evaluated by the server, and the results will be sent back to the device. At this time, the server will take the user's emotional state into consideration and tailor the feedback appropriately to provide useful information to the user.

[1273] In this way, by adding emotion recognition to the e-book review system, the present invention provides a learning environment suited to the user's psychological state, further enhancing the learning effect.

[1274] The processing flow will be explained below.

[1275] Step 1:

[1276] When the user finishes reading the e-book, they press a button on the device to indicate they are finished reading, and the device notifies the server that they have finished reading.

[1277] Step 2:

[1278] The device sends information including the ID of the e-book that has been read and the user ID to the server. Based on the sent information, the server retrieves the data of the corresponding e-book.

[1279] Step 3:

[1280] The server begins analyzing the received data, using natural language processing techniques to extract important information and key points from the text. This process involves grammatical analysis, key word extraction, and contextual analysis.

[1281] Step 4:

[1282] The server automatically generates a quiz based on the extracted key points. The generated quiz includes specific questions and their corresponding correct answers. For example, a generated question might be, "What are the basic principles of human-centered design?"

[1283] Step 5:

[1284] The server stores the generated quiz in a database, along with the user ID and schedule information, in addition to the quiz content.

[1285] Step 6:

[1286] The server sets a notification schedule for the quiz, for example, to notify the user 2 days, 14 days, and 60 days after the user finishes reading. This notification schedule is stored in a database and is triggered based on a timestamp.

[1287] Step 7:

[1288] The emotion engine begins to operate. The device uses the camera and microphone to analyze the user's facial expressions and voice, collecting emotional data. This data is sent to the server in real time.

[1289] Step 8:

[1290] The server analyzes the received emotional data and recognizes the user's emotional state, for example, determining whether the user is feeling stressed or focused.

[1291] Step 9:

[1292] The server adjusts the content and difficulty of the generated quiz based on the user's emotional state. For example, if the user's stress level is high, the server may lower the difficulty of the quiz or reduce the number of questions.

[1293] Step 10:

[1294] When the set date and time arrives, the server sends a quiz notification to the terminal, which includes the adjusted quiz content and an answer interface.

[1295] Step 11:

[1296] The terminal receives the notification from the server and displays it to the user, who then checks the notification and accesses an interface for answering the quiz.

[1297] Step 12:

[1298] The user answers questions on the display screen of the device, for example, by inputting answers to questions such as "What are the basic principles of human-centered design?"

[1299] Step 13:

[1300] The terminal transmits the user's answer to the server. The transmitted data includes the user's answer and the associated quiz ID.

[1301] Step 14:

[1302] The server evaluates the received answer and determines whether it is correct or not, and the evaluation result is classified as correct, incorrect, partially correct, etc.

[1303] Step 15:

[1304] The server generates feedback based on the evaluation results, particularly depending on the user's emotional state. For example, if the user shows signs of anxiety, it may include an encouraging message.

[1305] Step 16:

[1306] The server sends the generated feedback to the device, which then displays the received feedback to the user. The user can check the evaluation results of their answers and the feedback based on their emotions.

[1307] Through these steps, users can effectively review the contents of the e-book and be provided with a learning environment that takes their emotional state into consideration.

[1308] Example 2

[1309] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1310] Conventional e-books lacked feedback to improve users' learning effectiveness or learning adjustments based on the user's emotional state. This made it difficult to provide an effective learning environment that matched the user's level of concentration and understanding. Furthermore, the fixed difficulty and content of quizzes did not reduce the user's psychological burden, potentially leading to a decline in motivation to learn.

[1311] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for [analyzing the contents of the e-book and extracting important information], a means for [automatically generating quizzes based on the extracted information], and a means for [adjusting the content and difficulty of the generated quizzes based on the user's emotional data]. This makes it possible to improve the user's learning effect and provide optimal feedback according to the user's emotional state.

[1312] "Analyzing the contents of e-books and extracting important information" refers to the process of analyzing the text data of e-books that users have finished reading and extracting important points and keywords necessary for learning and memorization.

[1313] "Automatically generating quizzes based on extracted information" refers to the process of automatically creating question-style questions to be posed to users using a program based on important information obtained through the analysis.

[1314] "Adjusting the content and difficulty of the generated quiz based on the user's emotional data" refers to the process of analyzing the user's emotional data (such as facial expressions and tone of voice) and optimizing the difficulty of the quiz and the content of the questions according to the results of that analysis.

[1315] "Notifying the user of the generated quiz at a predetermined timing" refers to a process of notifying the user of the quiz at an appropriate timing according to a pre-set schedule or the user's learning situation.

[1316] "Collecting emotion data from the terminal and transmitting it to the server" refers to the process of collecting data related to the user's emotions using sensors such as a camera and microphone, and transferring that data to the server.

[1317] "Collecting and evaluating user answers" refers to the process of acquiring the answers given by users to the quiz and evaluating the accuracy and content of the answers.

[1318] This invention combines a system that analyzes the contents of e-books, extracts important information, and presents automatically generated quizzes at predetermined times to enhance user learning, with an emotion engine that recognizes the user's emotions. Implementing this system requires collaboration between a server, terminals, and users.

[1319] Server Processing

[1320] The server first receives the e-book data that the user has finished reading. The e-book data is uploaded in a format such as an EPUB file. Next, it uses a natural language processing engine such as Google Cloud Natural Language API or IBM Watson Natural Language Understanding to analyze the content of the e-book and extract important information and key points.

[1321] Based on the extracted keypoints, a generative AI model (e.g., OpenAI GPT-4) is used to automatically generate a quiz, which is then stored in a database (e.g., MySQL or MongoDB) for later use.

[1322] The server then collects the emotional data sent from the device and analyzes the user's emotional data using the Microsoft Azure Emotion API and Affectiva SDK. Based on the analysis results, the content and difficulty of the quiz can be adjusted. For example, if the user is feeling stressed, the difficulty of the quiz can be lowered to reduce the psychological burden.

[1323] Finally, set up a notification schedule for the quiz and notify users of the quiz at the appropriate time using scheduling software such as a Cron job or Amazon CloudWatch Events.

[1324] Terminal handling

[1325] The device collects the user's emotional data using sensors such as a camera and microphone. This can be done using the device's built-in camera or an external camera or microphone. The emotional data includes information on facial expressions and tone of voice, and the collected data is sent to a server.

[1326] The app receives quiz notifications sent from the server and displays them to the user. Notifications can be sent in the form of push notifications or in-app notifications. Push notifications are implemented using Firebase Cloud Messaging (FCM), and in-app notifications are implemented using mobile frameworks such as React Native or Swift.

[1327] The device collects the user's answers and sends them to the server. Once the answers are evaluated, the server sends the results back to the user. The evaluation results not only indicate whether the answer was correct or incorrect, but also provide feedback based on the user's emotional state.

[1328] User Action

[1329] The user checks the quiz displayed on the device and enters answers to the questions. The user's answers are sent from the device to the server, where they are evaluated. The evaluation results are returned to the device, and correct / incorrect answers and feedback are displayed. The feedback includes advice that takes into account the user's emotional state, further enhancing the learning effect.

[1330] Specific examples

[1331] For example, suppose a user has finished reading a book called "Design Thinking." The server analyzes the book's contents and extracts key points such as "human-centered design" and "iterative prototyping." The server then sends the following prompt to the generative AI model: "Based on the key points in the book "Design Thinking," please generate the following quiz. The key points are 'human-centered design' and 'iterative prototyping.'"

[1332] The generative AI model generates a quiz like this:

[1333] "What are the basic principles of human-centered design?"

[1334] "What is the importance of iterative prototyping?"

[1335] If the emotion engine analyzes the user's facial expressions using a camera and determines that the user is likely to be lacking concentration, the server adjusts the difficulty of the quiz, starting with easier questions. The quiz is notified to the user's device two days later. The user checks the notification and answers the quiz. The server evaluates the user's answers and sends the results back to the device. At this time, feedback is provided taking into account the user's emotional state.

[1336] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1337] Server Processing

[1338] Step 1: Content analysis of e-books

[1339] Input: The e-book data the user has finished reading (e.g., EPUB file)

[1340] How it works: The server receives e-book data uploaded by a user, in a format such as an EPUB file.

[1341] Data processing: The server analyzes the text data of the e-book using the Google Cloud Natural Language API, identifying nouns, verbs, adjectives, etc. and mapping their relationships.

[1342] Output: Extracted important information or key points.

[1343] Step 2: Generate the quiz

[1344] Input: Extracted keypoints

[1345] How it works: The server generates a prompt based on the extracted keypoints using a generative AI model (e.g., OpenAI GPT-4).

[1346] Data processing: The server sends prompts to the generative AI model to automatically generate quizzes.

[1347] Output: An automatically generated quiz, specifically using the prompt: "Generate the following quiz based on the key points from the book 'Design Thinking'. The key points are 'human-centered design' and 'iterative prototyping'."

[1348] Step 3: Analyze sentiment data and tailor the quiz

[1349] Input: Generated quiz, emotion data collected from the device

[1350] How it works: The server receives emotion data sent from the device, analyzes it using the Microsoft Azure Emotion API, and evaluates the user's emotional state (e.g., stress level or concentration level) based on the analysis results.

[1351] Data processing: The content and difficulty of the quiz are adjusted based on the analysis of emotional data. For example, if the user is feeling stressed, the difficulty of the quiz can be lowered to reduce the psychological burden.

[1352] Output: The adapted quiz.

[1353] Step 4: Schedule and run notifications

[1354] Inputs: Adjusted quiz, scheduling data

[1355] How it works: The server schedules quiz notifications using Cron jobs and Amazon CloudWatch Events.

[1356] Data processing: Notify quizzes at specified dates and times based on a set schedule.

[1357] Output: Quiz notification to user device

[1358] Terminal handling

[1359] Step 1: Collect and send emotion data

[1360] Input: User's facial expression, tone of voice

[1361] How it works: The device uses the camera and microphone to collect emotional data about the user, including information about facial expressions and tone of voice.

[1362] Data processing: Analyzing emotional data using facial recognition and voice analysis algorithms.

[1363] Output: Send the analyzed emotion data to the server.

[1364] Step 2: Receive and view quiz notifications

[1365] Input: Quiz notification from the server

[1366] Operation: The device receives a quiz notification sent from the server and displays the notification to the user.

[1367] Data processing: Display received notifications as push notifications or in-app notifications using services such as Firebase Cloud Messaging (FCM).

[1368] Output: The quiz notification displayed to the user

[1369] Step 3: Collect user responses and display evaluation results

[1370] Input: User's answer

[1371] Operation: The device collects the answers entered by the user and sends them to the server.

[1372] Data processing: Receive the response data in the form and send it to the server via an HTTP POST request.

[1373] Output: Receives the evaluation results from the server and displays them to the user.

[1374] User Action

[1375] Step 1: Check the quiz notification and answer

[1376] Input: Quiz notification from your device

[1377] How it works: The user reviews the quiz and enters answers to the questions displayed.

[1378] Output: User's answer typed into the terminal

[1379] Step 2: Receiving feedback

[1380] Input: Evaluation result from the server

[1381] How it works: The user checks the evaluation results displayed on the device. The evaluation results include not only correct or incorrect answers, but also feedback based on the user's emotional state.

[1382] Output: Evaluation results and feedback displayed to the user

[1383] In this way, the roles of the server, terminal, and user are clearly separated and a detailed processing flow is described.

[1384] (Application example 2)

[1385] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1386] Conventional e-book learning systems did not provide feedback or adjust the difficulty of quizzes based on the user's emotional state. This could lead to stress and reduced concentration, resulting in ineffective learning. Furthermore, the notification function to prompt users to review at the appropriate time was insufficient.

[1387] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1388] In this invention, the server includes means for analyzing the content of the electronic medium and extracting important information, means for automatically generating quizzes based on the extracted information, means for notifying the user of the generated quizzes at a predetermined timing, means for analyzing and collecting the user's emotional state, means for adjusting the content and difficulty of the quiz according to the user's emotional state, and means for collecting and evaluating the user's answers, thereby enabling effective feedback and adjustment of the difficulty of the quiz according to the user's emotional state.

[1389] "Electronic media" is a collection of information stored or distributed in electronic form.

[1390] "Analysis" means analyzing data in detail to understand its structure and meaning.

[1391] "Significant information" is data that is particularly valuable for a particular purpose or context and is necessary for understanding and decision-making.

[1392] A "quiz" is a set of questions designed to gauge a user's knowledge and understanding.

[1393] "Automatic generation" means that something is created automatically by an algorithm or program, without human intervention.

[1394] "Notifying" means notifying a user of specific information at a specific time.

[1395] "Emotional state" refers to the user's psychological state, including stress, concentration, fatigue, and the like.

[1396] "Adjust" means to change or optimize according to conditions or circumstances.

[1397] "Collect" means to gather specific data.

[1398] To "evaluate" means to judge data or results based on specific criteria.

[1399] This invention combines a system that analyzes the contents of electronic media, extracts important information, and notifies users of automatically generated quizzes at the appropriate time with an emotion engine that recognizes the user's emotional state. This makes it possible to provide feedback according to the user's emotions and adjust the content and difficulty of the quiz, thereby improving learning effectiveness.

[1400] Server Processing

[1401] 1. Content analysis of electronic media:

[1402] The server receives the data from the electronic media that the user has finished reading and analyzes its content. Natural language processing (NLP) is used for the analysis to extract important information and key points. Specifically, Python and NLTK (Natural Language Toolkit) are used.

[1403] 2. Generate the quiz:

[1404] The server automatically generates quizzes that ask about important information based on the extracted key points. The generated quizzes are stored in a database. A generative AI model is used to generate the quizzes.

[1405] 3. Use of Emotion Engine:

[1406] The server analyzes the user's emotional state. The user's emotional state is evaluated based on data collected using the smartphone's camera and microphone. Specifically, OpenCV and Google Cloud Vision API are used. The content and difficulty of the quiz are adjusted according to the user's emotional state.

[1407] 4. Set the notification schedule:

[1408] The server sets a quiz notification schedule and configures the quiz notification to occur at specific times, and this schedule is adjusted based on the user's learning pace and emotional state.

[1409] Terminal handling

[1410] 1. Collecting Emotional Data:

[1411] The device uses sensors such as a camera and microphone to collect emotional data from the user's facial expressions and tone of voice. The collected emotional data is sent to a server, which uses the Microsoft Azure Emotion API for specific emotional analysis.

[1412] 2. Receive and view quiz notifications:

[1413] The device receives the quiz notification sent from the server and displays it to the user. The display format provides an intuitive interface such as push notification or in-app notification.

[1414] 3. Collect user responses and display the evaluation results:

[1415] The terminal collects the user's answers and sends them to the server, and after receiving the evaluation results from the server, displays the evaluation results to the user.

[1416] User Action

[1417] 1. Check the quiz notification and answer:

[1418] The user checks the quiz displayed on the terminal and enters answers to the questions displayed. The user's answers are then sent from the terminal to the server.

[1419] 2. Receiving Feedback:

[1420] Users receive evaluation results for their answers from their devices, which include not only whether their answers were correct or incorrect, but also feedback based on the user's individual emotional state.

[1421] Specific examples

[1422] For example, if a user finishes reading a book called "Design Thinking," the server will analyze the book's contents and extract key points such as:

[1423] Human-Centered Design

[1424] Iterative Prototyping

[1425] The server then generates a quiz based on these key points, like this:

[1426] "What are the basic principles of human-centered design?"

[1427] "What is the importance of iterative prototyping?"

[1428] If the emotion engine analyzes the user's facial expressions using a camera and determines that the user is likely to be lacking concentration, the server adjusts the difficulty of the quiz and starts the user with easier questions.

[1429] The quiz will be notified to the user's device two days later. The user will check the notification and answer the quiz. The user's answers will be evaluated by the server, and the results will be sent back to the device. At this time, the server will take into account the user's emotional state and tailor the feedback appropriately to provide useful information to the user. The following prompt sentences can be used:

[1430] "Please explain the basic principles of human-centered design (e.g., designing based on user needs)."

[1431] In this way, by adding emotion recognition to an electronic review system, the present invention provides a learning environment suited to the user's psychological state, further enhancing learning effectiveness.

[1432] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1433] Step 1:

[1434] The server receives the data from the electronic media that the user has finished reading. The input is the text data from the electronic media, which is analyzed to extract important information and key points. Specifically, natural language processing (NLP) technology is used to tokenize the text data and extract important words and phrases. The extracted information is output as key points.

[1435] Step 2:

[1436] The server automatically generates a quiz based on the extracted keypoints. The input is the extracted keypoints, and based on these, a generative AI model is used to create quiz questions. Specifically, the generative AI model generates questions related to the keypoints and stores them in a database. The generated quiz is then output.

[1437] Step 3:

[1438] The server sets the quiz notification schedule. The input is the user's learning progress data and emotional state data, and the notification timing is calculated based on this. Specifically, the server takes into account the user's learning patterns and emotional state to set a schedule for quiz notifications at the optimal timing. The set notification schedule is output.

[1439] Step 4:

[1440] The device collects the user's emotional state. The input is sensor data from the camera and microphone, which is analyzed to determine the user's emotional state. Specifically, the device uses OpenCV and the Microsoft Azure Emotion API to analyze the user's facial expressions and tone of voice from the collected sensor data. The analyzed emotional state data is output.

[1441] Step 5:

[1442] The device receives the quiz notification sent from the server and displays it to the user. The input is the quiz question sent from the server, which is then displayed to the user. Specifically, the device uses push notifications or in-app notifications to intuitively inform the user of the quiz. The displayed quiz is then output.

[1443] Step 6:

[1444] The user answers a quiz displayed on the terminal. The input is the quiz question displayed on the terminal, and the user inputs the answer to that question. In concrete terms, the user inputs the answer to the question and sends it to the server via the terminal. The user's answer is then output.

[1445] Step 7:

[1446] The server collects and evaluates the user's answers. The input is the user's answer data, which is then evaluated. Specifically, the server compares the user's answers with the correct answer data and generates an evaluation result. The generated evaluation result is then output.

[1447] Step 8:

[1448] The server adjusts the feedback according to the user's emotional state and notifies the user of the evaluation results. The input is the evaluation results and the user's emotional state data, and the feedback is created based on this. Specifically, the server generates feedback according to the user's emotional state and sends it to the terminal. The adjusted feedback is then output.

[1449] Step 9:

[1450] The device displays the evaluation results and feedback received from the server to the user. The input is the evaluation results and feedback sent from the server, which are then displayed to the user. Specifically, the device uses a notification function or an in-app interface to visually provide the evaluation results and feedback to the user. The displayed evaluation results and feedback are then output.

[1451] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1452] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1453] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1454] [Fourth embodiment]

[1455] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1456] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1457] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1458] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1459] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1460] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1461] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1462] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1463] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1464] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1465] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1466] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1467] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1468] This invention is a system that analyzes the contents of e-books, extracts important information, automatically generates quizzes, and allows readers to review the content. This system allows readers to retain the contents of books they have read for a long period of time. Specifically, the following process is carried out between the server, the terminal, and the user.

[1469] Server Processing

[1470] 1. Content analysis of e-books

[1471] The server receives the data of the e-book that the user has finished reading. It then analyzes the content of the e-book and extracts important information and key points. In this process, it uses natural language processing technology and text analysis algorithms to extract the essence of the text necessary for learning.

[1472] 2. Quiz Generation

[1473] The server automatically generates quizzes based on the extracted key points. For example, it creates specific questions such as "What are the basic principles of human-centered design?" and sets the correct answers. The generated quizzes are stored in a database.

[1474] 3. Set a notification schedule

[1475] The server sets a schedule for notifying users of quizzes, for example, setting it to notify users at specific times, such as after 2 days, 14 days, or 60 days, encouraging regular review and promoting knowledge retention.

[1476] Terminal handling

[1477] 1. Receive quiz notifications

[1478] The terminal receives the quiz notification sent from the server, which includes the generated quiz and the notification schedule.

[1479] 2. View the quiz

[1480] The device displays the received quiz notification to the user in a simple and easy-to-answer interface, such as a push notification on a smartphone or an in-app notification.

[1481] 3. Collecting user responses

[1482] The terminal allows the user to enter answers to the quiz and transmits the entered answers to the server.

[1483] User Action

[1484] 1. Check notifications

[1485] The user checks the notification from the device and becomes aware that there is a quiz. For example, when a notification arrives on the device two days later, the user checks it and answers the quiz.

[1486] 2. Answer the quiz

[1487] Users can interactively input answers to the displayed quiz questions. The input method is simple and intuitive, allowing answers to be entered by typing on the keyboard or by tapping.

[1488] Specific examples

[1489] For example, if a user finishes reading a book called "Design Thinking," the server analyzes the book's contents and extracts important key points, such as:

[1490] Human-Centered Design

[1491] Iterative Prototyping

[1492] The server then generates a quiz based on these key points:

[1493] "What are the basic principles of human-centered design?"

[1494] "What is the importance of iterative prototyping?"

[1495] The quiz will be sent to the user's device two days later. The user checks the notification and answers the quiz. The user's answers are sent to the server via the device, and the server evaluates the answers and notifies the user of the results. Through this process, the user can review the important content of the book and retain it in their long-term memory.

[1496] In this way, the system provides an environment in which readers can periodically review the contents of the book, thereby improving learning effectiveness.

[1497] The processing flow will be explained below.

[1498] Step 1:

[1499] When the user finishes reading an e-book, he or she presses a button on the device to record the end of reading, which notifies the system of the end of book reading event.

[1500] Step 2:

[1501] The device sends information about the e-book that the user has finished reading to the server. The sent information includes the user ID and the e-book ID.

[1502] Step 3:

[1503] Based on the received information, the server retrieves the contents of the e-book and begins analyzing it, using natural language processing technology to extract important key points and learning essences from the text.

[1504] Step 4:

[1505] The server automatically generates quizzes based on information extracted from the analysis results. The quizzes are created in the form of specific questions based on important key points. For example, a question might be generated such as, "What are the basic principles of human-centered design?"

[1506] Step 5:

[1507] The server saves the generated quiz in a database, along with the user ID, quiz content, and correct answer information.

[1508] Step 6:

[1509] The server sets a notification schedule for the quiz. The notification timing can be set to a specific schedule, such as 2 days, 14 days, or 60 days later.

[1510] Step 7:

[1511] When the set time comes, the server sends a quiz notification to the terminal, which includes the generated quiz and an interface for answering the question.

[1512] Step 8:

[1513] The terminal displays the received quiz notification to the user, who then checks the notification and opens an interface for answering the quiz.

[1514] Step 9:

[1515] The user inputs answers to quiz questions through the device interface, for example, "What are the basic principles of human-centered design?"

[1516] Step 10:

[1517] The terminal sends the user's answer to the server. The sent data includes the user ID, quiz ID, and the user's answer.

[1518] Step 11:

[1519] The server evaluates the received user's answer and checks whether the answer is correct by comparing it with the correct answer information.

[1520] Step 12:

[1521] The server sends the evaluation results to the device, including feedback such as "Correct" if the user's answer is correct, or "Incorrect" if the answer is incorrect.

[1522] Step 13:

[1523] The device notifies the user of the evaluation results received from the server, allowing the user to check whether their answers are correct and receive feedback to improve their understanding.

[1524] Example 1

[1525] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1526] With conventional methods for viewing electronic publications, it is difficult to periodically review the content once it has been read and retain it in your memory for a long period of time. In particular, it is difficult to extract important information and key points and provide an environment for effective learning.

[1527] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1528] In this invention, the server includes means for analyzing the contents of electronic publications and extracting important information, means for automatically generating questions based on the extracted information, and means for notifying the user of the generated questions at a predetermined timing, thereby enabling the user to periodically review important information in electronic publications and retain it in their memory for a long period of time.

[1529] An "electronic publication" is a book, information, or document provided in electronic form.

[1530] "Content analysis" is the process of analyzing text data and extracting important information or key points.

[1531] "Key information" refers to knowledge or facts contained in a text that are particularly important for learning and understanding.

[1532] "Automatic generation" is a process in which a machine or program independently creates questions or poses questions with minimal human intervention.

[1533] "Questions" are questions or quizzes that users answer to review the content they have learned and check their level of understanding.

[1534] The term "predetermined timing" refers to a specific time or period that has been set in advance.

[1535] "User" means an individual or organization that uses this system.

[1536] "Notification" is the act of informing users of information or issues and encouraging them to receive them.

[1537] An "answer" is an answer or response provided by a user to a question.

[1538] "Evaluation" is the process of analyzing the collected responses and determining their accuracy and validity.

[1539] A "server" is a computer system or network device for processing, storing, and serving information.

[1540] This invention is a system that analyzes the contents of electronic publications, extracts important information, and automatically generates questions. The system aims to encourage users to periodically review what they have read and retain it in their memory for a long period of time by fulfilling the roles of the server, terminal, and user.

[1541] Server processing format

[1542] Hardware and Software Use

[1543] The server has a network connection to receive digital publication data from users. The received data is temporarily stored on the server's storage (e.g., HDD or SSD). The server has installed natural language processing libraries (e.g., SpaCy, NLTK, Transformers, etc.) for content analysis.

[1544] Data analysis

[1545] The server analyzes the text of electronic publications using natural language processing techniques, tokenizing the text and extracting important keywords and phrases using TF-IDF and word embedding techniques, thereby extracting the essence necessary for learning.

[1546] Automatic question generation

[1547] Based on the extracted key points, a generative AI model (e.g., GPT-3 or T5 model) can be used to generate specific questions and their answers. For example, it is possible to generate a question such as, "What are the fundamental principles of human-centered design?" and its corresponding answer.

[1548] Save quizzes and schedule notifications

[1549] The generated questions are saved in a database (e.g. MySQL, PostgreSQL). At that time, the server sets a schedule for notifying users of the quiz. For example, it can set the quiz to be notified after 2 days, 14 days, or 60 days. This schedule information is also saved in the database.

[1550] Submit a quiz

[1551] Based on the notification schedule, quizzes are sent to devices at the set date and time using methods such as push notifications and email notifications.

[1552] Terminal processing format

[1553] Receive and view quiz notifications

[1554] The device receives quiz notifications sent from the server. The notifications include the generated quiz questions and answer options. The device displays the received quiz via push notification or within the app. The display format provides a simple and intuitive user interface.

[1555] Collecting and sending user responses

[1556] The device provides an input form and buttons for users to answer the quiz, temporarily stores the answers entered by the user, and sends the answer data to the server for appropriate association.

[1557] User processing format

[1558] Check notifications and answer quizzes

[1559] The user confirms the quiz notification on their device and knows that a quiz has arrived. They enter their answers to the quiz displayed on their device and press the submit button to send the answers to the server. Through this process, the user can review the important content of the book and retain it in their long-term memory.

[1560] Examples of concrete examples and prompts

[1561] For example, if a user finishes reading a book called "Design Thinking," the server analyzes the book's contents and extracts important key points, such as:

[1562] Human-Centered Design

[1563] Iterative Prototyping

[1564] The server then generates a problem based on these keypoints, such as:

[1565] "What are the basic principles of human-centered design?"

[1566] "What is the importance of iterative prototyping?"

[1567] The quiz will be sent to the user's device two days later. The user checks the notification and answers the quiz. The user's answers are sent to the server via the device, and the server evaluates the answers and notifies the user of the results. This allows the user to retain important information from the electronic publication in their long-term memory.

[1568] Example prompts for generative AI models:

[1569] "Generate one quiz each about the book 'Design Thinking', the 'Basic Principles of Human-Centered Design' and the 'Importance of Iterative Prototyping'."

[1570] In this way, the system provides an environment in which readers can periodically review the contents of the book, thereby improving learning effectiveness.

[1571] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1572] Server Processing Steps

[1573] Step 1: Receiving electronic publication data

[1574] Specific actions

[1575] The server receives electronic publication files (e.g., PDF, ePub) sent by users.

[1576] Input: Electronic publication file

[1577] Output: A file temporarily stored in the server's storage

[1578] The server stores the received file in storage and records the file path.

[1579] Step 2: Content analysis

[1580] Specific actions

[1581] The server reads the text of the electronic publication stored in the storage and tokenizes the text using a natural language processing library (e.g., SpaCy, NLTK).

[1582] Input: Text data of electronic publications

[1583] Output: Tokenized text, important keywords and phrases

[1584] Extract key points necessary for learning from tokenized text using TF-IDF and word embeddings.

[1585] Step 3: Automatic question generation

[1586] Specific actions

[1587] Based on the extracted keypoints, a prompt sentence is input into a generative AI model (e.g., GPT-3) to generate a question sentence.

[1588] Input: Keypoint data, prompt

[1589] Output: Generated question statement

[1590] Use prompts to generate questions such as, "What are the basic principles of human-centered design?"

[1591] Step 4: Save the quiz to the database

[1592] Specific actions

[1593] The generated questions and their answers are saved in a database (e.g. MySQL).

[1594] Input: Generated question and answer

[1595] Output: Quiz data stored in a database

[1596] The database stores quiz questions, answers, user IDs, notification schedules, etc.

[1597] Step 5: Set up the quiz notification schedule

[1598] Specific actions

[1599] The server sets a schedule for notifying the generated quiz (e.g., after 2 days, 14 days, or 60 days).

[1600] Input: Quiz data, notification schedule

[1601] Output: Quiz data with schedule information added

[1602] The schedule is recorded in the database and quizzes are set to be notified periodically.

[1603] Step 6: Submit your quiz

[1604] Specific actions

[1605] The server sends the quiz to the terminal based on the set notification schedule.

[1606] Input: Schedule information, quiz data

[1607] Output: Quiz notification to device

[1608] Notifications can be sent using push notifications or email notifications.

[1609] Terminal processing steps

[1610] Step 1: Receive quiz notifications

[1611] Specific actions

[1612] The terminal receives the quiz notification sent from the server.

[1613] Input:QuizNotification

[1614] Output: Notification received alert

[1615] The terminal displays an alert to inform the user of the received notification.

[1616] Step 2: View the quiz

[1617] Specific actions

[1618] The terminal displays the received quiz content to the user.

[1619] Input: Quiz notification data

[1620] Output: The quiz displayed in the user interface

[1621] The quizzes are displayed using a simple UI, for example via push notifications or in-app notifications.

[1622] Step 3: Collect user responses

[1623] Specific actions

[1624] The terminal provides an interface for the user to input answers to the quiz.

[1625] Input: User's answer

[1626] Output: Temporarily stored user response data

[1627] The answers entered by the user are temporarily stored for transmission to the server.

[1628] Step 4: Submit your response data

[1629] Specific actions

[1630] The terminal transmits the user's response data to the server.

[1631] Input: User response data

[1632] Output: Response data sent to the server

[1633] When sent to the server, the user ID and quiz ID are also sent and associated appropriately.

[1634] User processing steps

[1635] Step 1: Check notifications

[1636] Specific actions

[1637] The user checks the quiz notification that arrives on the device.

[1638] Input:QuizNotification

[1639] Output: User's perceived behavior

[1640] Tap the notification to open the app and go to the quiz screen.

[1641] Step 2: Take the quiz

[1642] Specific actions

[1643] The user inputs answers to the displayed quiz questions.

[1644] Input: Quiz question

[1645] Output: User's answer

[1646] To submit your answer, enter text into the input form or select an option to confirm your answer.

[1647] (Application example 1)

[1648] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1649] It is difficult to efficiently learn the contents of e-books and retain them in your memory for a long period of time. In addition, there is a lack of review methods available after normal reading, and there is no system to regularly review what you have read, so it is difficult to expect the contents to be retained in your memory.

[1650] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1651] In this invention, the server includes means for [analyzing the contents of the e-book and extracting important information], means for [automatically generating quizzes based on the extracted information], means for [notifying the user of the generated quizzes at predetermined times], means for [regularly scheduling the timing of quiz notifications], means for [collecting and evaluating the user's answers], and means for [feeding back the evaluation results to the user]. This allows the contents of the e-book to be efficiently reviewed and the learned content to be retained in the memory for a long period of time.

[1652] "Means for analyzing the contents of e-books and extracting important information" refers to a function that analyzes the text data of e-books using natural language processing technology and text analysis algorithms, and automatically extracts key points and important information necessary for learning.

[1653] "Means for automatically generating quizzes based on extracted information" refers to a function that automatically generates specific question-style quizzes to confirm the learning content from extracted key points and important information.

[1654] The "means for notifying the user of the generated quiz at a predetermined timing" is a function for notifying the user of the generated quiz on the user's terminal based on a preset schedule.

[1655] The "means for periodically scheduling the timing of quiz notifications" is a function for setting and managing the schedule for quiz notifications at specific intervals so that users can review efficiently.

[1656] The "means for collecting and evaluating user answers" is a function for collecting answers entered by users to quizzes and evaluating whether the answers are correct or not based on the collected answers.

[1657] The "means for feeding back the evaluation results to the user" is a function for notifying the user of the evaluation results and providing feedback.

[1658] This invention provides a system that analyzes the contents of e-books, extracts important information, automatically generates quizzes, and periodically notifies users. This invention allows users to retain the contents of books they have finished reading for a long period of time. The system of this invention performs the following processing between a server, a terminal, and a user.

[1659] Server Processing

[1660] The server has multiple functions and performs the following processes:

[1661] 1. Content analysis of e-books

[1662] The server receives data of the electronic book that the user has finished reading.

[1663] Next, the content of the e-book is analyzed to extract important information and key points, using natural language processing techniques and text analysis algorithms, specifically the Hugging Face transformers library and pipeline.

[1664] Example: Analyze the contents of the e-book "Design Thinking" and extract key points such as "human-centered design" and "iterative prototyping."

[1665] 2. Quiz Generation

[1666] The server automatically generates a quiz based on the extracted key points. For example, it creates a specific question such as "What are the basic principles of human-centered design?" and sets the correct answer.

[1667] This allows the user to efficiently review what they have learned.

[1668] 3. Set a notification schedule

[1669] The server sets the schedule for notifying users of the quiz. For example, it sets the quiz to be notified after 2 days, 14 days, 60 days, etc. This schedule is managed using the scheduler function of apscheduler.

[1670] Terminal handling

[1671] The terminal is in charge of interfacing with the user and performs the following processes.

[1672] 1. Receive quiz notifications

[1673] The terminal receives the quiz notification sent from the server, which includes the generated quiz and the notification schedule.

[1674] Example: A user's smartphone will be notified of a quiz asking, "What are the basic principles of human-centered design?"

[1675] 2. View the quiz

[1676] The device displays the received quiz notification to the user. The display format provides a simple interface that makes it easy to answer. For example, it uses the push notification function of a smartphone or the display function of a head-mounted display.

[1677] Example: A quiz will appear on the notification screen of your smartphone, and you can enter answers by tapping.

[1678] 3. Collecting user responses

[1679] The terminal allows the user to enter answers to the quiz and transmits the entered answers to the server.

[1680] User Action

[1681] The user performs the following process through his / her own terminal.

[1682] 1. Check notifications

[1683] The user checks the notification from the device and becomes aware that there is a quiz. For example, when a notification arrives on the device two days later, the user checks it and answers the quiz.

[1684] 2. Answer the quiz

[1685] Users can interactively input answers to the displayed quiz questions. The input method is simple and intuitive, allowing answers to be entered by typing on the keyboard or by tapping.

[1686] Specific examples

[1687] For example, if a user finishes reading a book called "Design Thinking," the server analyzes the book's contents and extracts key points such as:

[1688] Human-Centered Design

[1689] Iterative Prototyping

[1690] The server then generates a quiz based on these key points:

[1691] "What are the basic principles of human-centered design?"

[1692] "What is the importance of iterative prototyping?"

[1693] The quiz will be sent to the user's device two days later. The user checks the notification and answers the quiz. The user's answers are sent to the server via the device, and the server evaluates the answers and notifies the user of the results.

[1694] Prompt Sentence Examples

[1695] Prompt sentence to input to the generative AI model:

[1696] "Please summarize the following: {eBook text}"

[1697] In this way, the system provides an environment in which readers can periodically review the contents of the book, thereby improving learning effectiveness.

[1698] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1699] Step 1: Receiving e-book data

[1700] The server receives the e-book data that the user has finished reading. As input, the e-book file uploaded by the user through the application is sent to the server. The server stores this e-book data in its internal storage for processing.

[1701] Step 2: Content analysis and important information extraction

[1702] The server analyzes the stored e-book data using natural language processing techniques and text analysis algorithms. The server generates a summary of the e-book based on the extracted key points and important information. This process uses Hugging Face's transformers library and pipeline. The output is a summary text, which becomes the input for the next process.

[1703] Step 3: Auto-generate a quiz

[1704] The server automatically generates quizzes based on the extracted key points and summary text. Specifically, the generative AI model receives a prompt, "Please summarize the following: {e-book text}," and converts it into a quiz format. For example, it generates a question such as, "What are the basic principles of human-centered design?" The output is multiple quizzes, which serve as input for the next process.

[1705] Step 4: Set up a notification schedule

[1706] The server sets the notification schedule for the generated quiz. It receives the quiz data as input and schedules the notification to occur at a specific time (for example, after 2 days, 14 days, or 60 days). This schedule is managed using the scheduler function of apscheduler. The scheduled notification task is set as the output.

[1707] Step 5: Receive quiz notifications

[1708] The terminal receives quiz notifications sent from the server based on a set schedule. As input, the server sends quiz notification data to the terminal. The terminal receives and stores this data.

[1709] Step 6: View the quiz

[1710] The device displays the received quiz notification to the user. This can be done via smartphone push notifications, in-app notifications, or head-mounted displays. The device references the quiz notification data as input, and displays the quiz to the user as output.

[1711] Step 7: User answers

[1712] The user inputs answers to the displayed quiz questions. The answers entered by the user using tap operations or keyboard input are saved on the device.

[1713] Step 8: User submits answer

[1714] The terminal receives the user's answer data as input and sends it to the server.

[1715] Step 9: Evaluate user responses

[1716] The server evaluates the received user answers. Specifically, it uses an internal evaluation algorithm to evaluate whether the answer is correct. It receives the user's answer data as input and generates an evaluation result as output.

[1717] Step 10: Feedback of evaluation results

[1718] The server feeds back the evaluation results to the user. It references the evaluation result data as input and generates a result notification as output, which it sends to the terminal. The terminal receives this notification and displays it to the user.

[1719] This allows the server, terminal, and user to work together, making it possible to efficiently review and memorize the contents of e-books.

[1720] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1721] This invention combines a system that analyzes the contents of e-books, extracts important information, and notifies users of automatically generated quizzes at predetermined times with an emotion engine that recognizes the user's emotions, in order to enhance user learning. This makes it possible to provide feedback according to the user's emotions and adjust the content and difficulty of the quizzes, further enhancing the effectiveness of learning.

[1722] Server Processing

[1723] 1. Content analysis of e-books

[1724] The server receives the data of the e-book that the user has finished reading, analyzes the contents of the e-book, and extracts important information and key points.

[1725] 2. Quiz Generation

[1726] The server automatically generates quizzes based on the extracted key points, asking about important information, and stores the generated quizzes in a database.

[1727] 3. Use of Emotion Engine

[1728] After generating the quiz, the server collects user emotional data and adjusts the content and difficulty of the generated quiz based on this data. For example, if the user is feeling stressed, the difficulty level can be lowered to reduce the user's psychological burden.

[1729] 4. Set up a notification schedule

[1730] The server sets a quiz notification schedule and sets the quiz to be notified at a specific timing.

[1731] Terminal handling

[1732] 1. Collecting Emotional Data

[1733] The device uses sensors such as a camera and microphone to collect emotional data from the user's facial expressions and tone of voice, and the collected emotional data is sent to a server.

[1734] 2. Receiving and viewing quiz notifications

[1735] The device receives quiz notifications sent from the server and displays them to the user in an intuitive interface, such as push notifications or in-app notifications.

[1736] 3. Collecting user responses and displaying evaluation results

[1737] The terminal collects the user's answers and sends them to the server, and after receiving the evaluation results from the server, displays the evaluation results to the user.

[1738] User Action

[1739] 1. Check and answer quiz notifications

[1740] The user checks the quiz displayed on the terminal and enters answers to the questions displayed. The user's answers are then sent from the terminal to the server.

[1741] 2. Receiving Feedback

[1742] Users receive evaluation results for their answers from their devices, which include not only whether their answers were correct or incorrect, but also feedback based on the user's individual emotional state.

[1743] Specific examples

[1744] For example, if a user finishes reading a book called "Design Thinking," the server analyzes the book's contents and extracts the following key points:

[1745] Human-Centered Design

[1746] Iterative Prototyping

[1747] The server then generates a quiz based on these key points, like this:

[1748] "What are the basic principles of human-centered design?"

[1749] "What is the importance of iterative prototyping?"

[1750] If the emotion engine analyzes the user's facial expressions using a camera and determines that the user is likely to be lacking concentration, the server adjusts the difficulty of the quiz and starts the user with easier questions.

[1751] The quiz will be notified to the user's device two days later. The user will check the notification and answer the quiz. The user's answers will be evaluated by the server, and the results will be sent back to the device. At this time, the server will take the user's emotional state into consideration and tailor the feedback appropriately to provide useful information to the user.

[1752] In this way, by adding emotion recognition to the e-book review system, the present invention provides a learning environment suited to the user's psychological state, further enhancing the learning effect.

[1753] The processing flow will be explained below.

[1754] Step 1:

[1755] When the user finishes reading the e-book, they press a button on the device to indicate they are finished reading, and the device notifies the server that they have finished reading.

[1756] Step 2:

[1757] The device sends information including the ID of the e-book that has been read and the user ID to the server. Based on the sent information, the server retrieves the data of the corresponding e-book.

[1758] Step 3:

[1759] The server begins analyzing the received data, using natural language processing techniques to extract important information and key points from the text. This process involves grammatical analysis, key word extraction, and contextual analysis.

[1760] Step 4:

[1761] The server automatically generates a quiz based on the extracted key points. The generated quiz includes specific questions and their corresponding correct answers. For example, a generated question might be, "What are the basic principles of human-centered design?"

[1762] Step 5:

[1763] The server stores the generated quiz in a database, along with the user ID and schedule information, in addition to the quiz content.

[1764] Step 6:

[1765] The server sets a notification schedule for the quiz, for example, to notify the user 2 days, 14 days, and 60 days after the user finishes reading. This notification schedule is stored in a database and is triggered based on a timestamp.

[1766] Step 7:

[1767] The emotion engine then begins to operate. The device uses the camera and microphone to analyze the user's facial expressions and voice, collecting emotional data. This data is then sent to the server in real time.

[1768] Step 8:

[1769] The server analyzes the received emotional data and recognizes the user's emotional state, for example, determining whether the user is feeling stressed or focused.

[1770] Step 9:

[1771] The server adjusts the content and difficulty of the generated quiz based on the user's emotional state. For example, if the user's stress level is high, the server may lower the difficulty of the quiz or reduce the number of questions.

[1772] Step 10:

[1773] When the set date and time arrives, the server sends a quiz notification to the terminal, which includes the adjusted quiz content and an answer interface.

[1774] Step 11:

[1775] The terminal receives the notification from the server and displays it to the user, who then checks the notification and accesses an interface for answering the quiz.

[1776] Step 12:

[1777] The user answers questions on the display screen of the device, for example, by inputting answers to the question, "What are the basic principles of human-centered design?"

[1778] Step 13:

[1779] The terminal transmits the user's answer to the server. The transmitted data includes the user's answer and the associated quiz ID.

[1780] Step 14:

[1781] The server evaluates the received answer and determines whether it is correct or not, and the evaluation result is classified as correct, incorrect, partially correct, etc.

[1782] Step 15:

[1783] The server generates feedback based on the evaluation results, particularly depending on the user's emotional state. For example, if the user shows signs of anxiety, it may include an encouraging message.

[1784] Step 16:

[1785] The server sends the generated feedback to the device, which then displays the received feedback to the user. The user can check the evaluation results of their answers and the feedback based on their emotions.

[1786] Through these steps, users can effectively review the contents of the e-book and be provided with a learning environment that takes their emotional state into consideration.

[1787] Example 2

[1788] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1789] Conventional e-books lacked feedback to improve users' learning effectiveness or learning adjustments based on the user's emotional state. This made it difficult to provide an effective learning environment that matched the user's level of concentration and understanding. Furthermore, the fixed difficulty and content of quizzes did not reduce the user's psychological burden, potentially leading to a decline in motivation to learn.

[1790] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for [analyzing the contents of the e-book and extracting important information], a means for [automatically generating quizzes based on the extracted information], and a means for [adjusting the content and difficulty of the generated quizzes based on the user's emotional data]. This makes it possible to improve the user's learning effect and provide optimal feedback according to the user's emotional state.

[1791] "Analyzing the contents of e-books and extracting important information" refers to the process of analyzing the text data of e-books that users have finished reading and extracting important points and keywords necessary for learning and memorization.

[1792] "Automatically generating quizzes based on extracted information" refers to the process of automatically creating question-style questions to be posed to users using a program based on important information obtained through the analysis.

[1793] "Adjusting the content and difficulty of the generated quiz based on the user's emotional data" refers to the process of analyzing the user's emotional data (such as facial expressions and tone of voice) and optimizing the difficulty of the quiz and the content of the questions according to the results of that analysis.

[1794] "Notifying the user of the generated quiz at a predetermined timing" refers to a process of notifying the user of the quiz at an appropriate timing according to a pre-set schedule or the user's learning situation.

[1795] "Collecting emotion data from the terminal and transmitting it to the server" refers to the process of collecting data related to the user's emotions using sensors such as a camera and microphone, and transferring that data to the server.

[1796] "Collecting and evaluating user answers" refers to the process of acquiring the answers given by users to the quiz and evaluating the accuracy and content of the answers.

[1797] This invention combines a system that analyzes the contents of e-books, extracts important information, and presents automatically generated quizzes at predetermined times to enhance user learning, with an emotion engine that recognizes the user's emotions. Implementing this system requires collaboration between a server, terminals, and users.

[1798] Server Processing

[1799] The server first receives the e-book data that the user has finished reading. The e-book data is uploaded in a format such as an EPUB file. Next, it uses a natural language processing engine such as Google Cloud Natural Language API or IBM Watson Natural Language Understanding to analyze the content of the e-book and extract important information and key points.

[1800] Based on the extracted keypoints, a generative AI model (e.g., OpenAI GPT-4) is used to automatically generate a quiz, which is then stored in a database (e.g., MySQL or MongoDB) for later use.

[1801] The server then collects the emotional data sent from the device and analyzes the user's emotional data using the Microsoft Azure Emotion API and Affectiva SDK. Based on the analysis results, the content and difficulty of the quiz can be adjusted. For example, if the user is feeling stressed, the difficulty of the quiz can be lowered to reduce the psychological burden.

[1802] Finally, set up a notification schedule for the quiz and notify users of the quiz at the appropriate time using scheduling software such as a Cron job or Amazon CloudWatch Events.

[1803] Terminal handling

[1804] The device collects the user's emotional data using sensors such as a camera and microphone. This can be done using the device's built-in camera or an external camera or microphone. The emotional data includes information on facial expressions and tone of voice, and the collected data is sent to a server.

[1805] The app receives quiz notifications sent from the server and displays them to the user. Notifications can be sent in the form of push notifications or in-app notifications. Push notifications are implemented using Firebase Cloud Messaging (FCM), and in-app notifications are implemented using mobile frameworks such as React Native or Swift.

[1806] The device collects the user's answers and sends them to the server. Once the answers are evaluated, the server sends the results back to the user. The evaluation results not only indicate whether the answer was correct or incorrect, but also provide feedback based on the user's emotional state.

[1807] User Action

[1808] The user checks the quiz displayed on the device and enters answers to the questions. The user's answers are sent from the device to the server, where they are evaluated. The evaluation results are returned to the device, and correct / incorrect answers and feedback are displayed. The feedback includes advice that takes into account the user's emotional state, further enhancing the learning effect.

[1809] Specific examples

[1810] For example, suppose a user has finished reading a book called "Design Thinking." The server analyzes the book's contents and extracts key points such as "human-centered design" and "iterative prototyping." The server then sends the following prompt to the generative AI model: "Based on the key points in the book "Design Thinking," please generate the following quiz. The key points are 'human-centered design' and 'iterative prototyping.'"

[1811] The generative AI model generates a quiz like this:

[1812] "What are the basic principles of human-centered design?"

[1813] "What is the importance of iterative prototyping?"

[1814] If the emotion engine analyzes the user's facial expressions using a camera and determines that the user is likely to be lacking concentration, the server adjusts the difficulty of the quiz, starting with easier questions. The quiz is notified to the user's device two days later. The user checks the notification and answers the quiz. The server evaluates the user's answers and sends the results back to the device. At this time, feedback is provided taking into account the user's emotional state.

[1815] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1816] Server Processing

[1817] Step 1: Content analysis of e-books

[1818] Input: The e-book data the user has finished reading (e.g., EPUB file)

[1819] How it works: The server receives e-book data uploaded by a user, in a format such as an EPUB file.

[1820] Data processing: The server analyzes the text data of the e-book using the Google Cloud Natural Language API, identifying nouns, verbs, adjectives, etc. and mapping their relationships.

[1821] Output: Extracted important information or key points.

[1822] Step 2: Generate the quiz

[1823] Input: Extracted keypoints

[1824] How it works: The server generates a prompt based on the extracted keypoints using a generative AI model (e.g., OpenAI GPT-4).

[1825] Data processing: The server sends prompts to the generative AI model to automatically generate quizzes.

[1826] Output: An automatically generated quiz, specifically using the prompt: "Generate the following quiz based on the key points from the book 'Design Thinking'. The key points are 'human-centered design' and 'iterative prototyping'."

[1827] Step 3: Analyze sentiment data and tailor the quiz

[1828] Input: Generated quiz, emotion data collected from the device

[1829] How it works: The server receives emotion data sent from the device, analyzes it using the Microsoft Azure Emotion API, and evaluates the user's emotional state (e.g., stress level or concentration level) based on the analysis results.

[1830] Data processing: The content and difficulty of the quiz are adjusted based on the analysis of emotional data. For example, if the user is feeling stressed, the difficulty of the quiz can be lowered to reduce the psychological burden.

[1831] Output: The adapted quiz.

[1832] Step 4: Schedule and run notifications

[1833] Inputs: Adjusted quiz, scheduling data

[1834] How it works: The server schedules quiz notifications using Cron jobs and Amazon CloudWatch Events.

[1835] Data processing: Notify quizzes at specified dates and times based on a set schedule.

[1836] Output: Quiz notification to user device

[1837] Terminal handling

[1838] Step 1: Collect and send emotion data

[1839] Input: User's facial expression, tone of voice

[1840] How it works: The device uses the camera and microphone to collect emotional data about the user, including information about facial expressions and tone of voice.

[1841] Data processing: Analyzing emotional data using facial recognition and voice analysis algorithms.

[1842] Output: Send the analyzed emotion data to the server.

[1843] Step 2: Receive and view quiz notifications

[1844] Input: Quiz notification from the server

[1845] Operation: The device receives a quiz notification sent from the server and displays the notification to the user.

[1846] Data processing: Display received notifications as push notifications or in-app notifications using services such as Firebase Cloud Messaging (FCM).

[1847] Output: The quiz notification displayed to the user

[1848] Step 3: Collect user responses and display evaluation results

[1849] Input: User's answer

[1850] Operation: The device collects the answers entered by the user and sends them to the server.

[1851] Data processing: Receive the response data in the form and send it to the server via an HTTP POST request.

[1852] Output: Receives the evaluation results from the server and displays them to the user.

[1853] User Action

[1854] Step 1: Check the quiz notification and answer

[1855] Input: Quiz notification from your device

[1856] How it works: The user reviews the quiz and enters answers to the questions displayed.

[1857] Output: User's answer typed into the terminal

[1858] Step 2: Receiving feedback

[1859] Input: Evaluation result from the server

[1860] How it works: The user checks the evaluation results displayed on the device. The evaluation results include not only correct or incorrect answers, but also feedback based on the user's emotional state.

[1861] Output: Evaluation results and feedback displayed to the user

[1862] In this way, the roles of the server, terminal, and user are clearly separated and a detailed processing flow is described.

[1863] (Application example 2)

[1864] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1865] Conventional e-book learning systems did not provide feedback or adjust the difficulty of quizzes based on the user's emotional state. This could lead to stress and reduced concentration, resulting in ineffective learning. Furthermore, the notification function to prompt users to review at the appropriate time was insufficient.

[1866] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1867] In this invention, the server includes means for analyzing the content of the electronic medium and extracting important information, means for automatically generating quizzes based on the extracted information, means for notifying the user of the generated quizzes at a predetermined timing, means for analyzing and collecting the user's emotional state, means for adjusting the content and difficulty of the quiz according to the user's emotional state, and means for collecting and evaluating the user's answers, thereby enabling effective feedback and adjustment of the difficulty of the quiz according to the user's emotional state.

[1868] "Electronic media" is a collection of information stored or distributed in electronic form.

[1869] "Analysis" means analyzing data in detail to understand its structure and meaning.

[1870] "Significant information" is data that is particularly valuable for a particular purpose or context and is necessary for understanding and decision-making.

[1871] A "quiz" is a set of questions designed to gauge a user's knowledge and understanding.

[1872] "Automatic generation" means that something is created automatically by an algorithm or program, without human intervention.

[1873] "Notifying" means notifying a user of specific information at a specific time.

[1874] "Emotional state" refers to the user's psychological state, including stress, concentration, fatigue, and the like.

[1875] "Adjust" means to change or optimize according to conditions or circumstances.

[1876] "Collect" means to gather specific data.

[1877] To "evaluate" means to judge data or results based on specific criteria.

[1878] This invention combines a system that analyzes the contents of electronic media, extracts important information, and notifies users of automatically generated quizzes at the appropriate time with an emotion engine that recognizes the user's emotional state. This makes it possible to provide feedback according to the user's emotions and adjust the content and difficulty of the quiz, thereby improving learning effectiveness.

[1879] Server Processing

[1880] 1. Content analysis of electronic media:

[1881] The server receives the data from the electronic media that the user has finished reading and analyzes its content. Natural language processing (NLP) is used for the analysis to extract important information and key points. Specifically, Python and NLTK (Natural Language Toolkit) are used.

[1882] 2. Generate the quiz:

[1883] The server automatically generates quizzes that ask about important information based on the extracted key points. The generated quizzes are stored in a database. A generative AI model is used to generate the quizzes.

[1884] 3. Use of Emotion Engine:

[1885] The server analyzes the user's emotional state. The user's emotional state is evaluated based on data collected using the smartphone's camera and microphone. Specifically, OpenCV and Google Cloud Vision API are used. The content and difficulty of the quiz are adjusted according to the user's emotional state.

[1886] 4. Set the notification schedule:

[1887] The server sets a quiz notification schedule and configures the quiz notification to occur at specific times, and this schedule is adjusted based on the user's learning pace and emotional state.

[1888] Terminal handling

[1889] 1. Collecting Emotional Data:

[1890] The device uses sensors such as a camera and microphone to collect emotional data from the user's facial expressions and tone of voice. The collected emotional data is sent to a server, which uses the Microsoft Azure Emotion API for specific emotional analysis.

[1891] 2. Receive and view quiz notifications:

[1892] The device receives the quiz notification sent from the server and displays it to the user. The display format provides an intuitive interface such as push notification or in-app notification.

[1893] 3. Collect user responses and display the evaluation results:

[1894] The terminal collects the user's answers and sends them to the server, and after receiving the evaluation results from the server, displays the evaluation results to the user.

[1895] User Action

[1896] 1. Check the quiz notification and answer:

[1897] The user checks the quiz displayed on the terminal and enters answers to the questions displayed. The user's answers are then sent from the terminal to the server.

[1898] 2. Receiving Feedback:

[1899] Users receive evaluation results for their answers from their devices, which include not only whether their answers were correct or incorrect, but also feedback based on the user's individual emotional state.

[1900] Specific examples

[1901] For example, if a user finishes reading a book called "Design Thinking," the server will analyze the book's contents and extract key points such as:

[1902] Human-Centered Design

[1903] Iterative Prototyping

[1904] The server then generates a quiz based on these key points, like this:

[1905] "What are the basic principles of human-centered design?"

[1906] "What is the importance of iterative prototyping?"

[1907] If the emotion engine analyzes the user's facial expressions using a camera and determines that the user is likely to be lacking concentration, the server adjusts the difficulty of the quiz and starts the user with easier questions.

[1908] The quiz will be notified to the user's device two days later. The user will check the notification and answer the quiz. The user's answers will be evaluated by the server, and the results will be sent back to the device. At this time, the server will take into account the user's emotional state and tailor the feedback appropriately to provide useful information to the user. The following prompt sentences can be used:

[1909] "Please explain the basic principles of human-centered design (e.g., designing based on user needs)."

[1910] In this way, by adding emotion recognition to an electronic review system, the present invention provides a learning environment suited to the user's psychological state, further enhancing learning effectiveness.

[1911] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1912] Step 1:

[1913] The server receives the data from the electronic media that the user has finished reading. The input is the text data from the electronic media, which is analyzed to extract important information and key points. Specifically, natural language processing (NLP) technology is used to tokenize the text data and extract important words and phrases. The extracted information is output as key points.

[1914] Step 2:

[1915] The server automatically generates a quiz based on the extracted keypoints. The input is the extracted keypoints, and based on these, a generative AI model is used to create quiz questions. Specifically, the generative AI model generates questions related to the keypoints and stores them in a database. The generated quiz is then output.

[1916] Step 3:

[1917] The server sets the quiz notification schedule. The input is the user's learning progress data and emotional state data, and the notification timing is calculated based on this. Specifically, the server takes into account the user's learning patterns and emotional state to set a schedule for quiz notifications at the optimal timing. The set notification schedule is output.

[1918] Step 4:

[1919] The device collects the user's emotional state. The input is sensor data from the camera and microphone, which is analyzed to determine the user's emotional state. Specifically, the device uses OpenCV and the Microsoft Azure Emotion API to analyze the user's facial expressions and tone of voice from the collected sensor data. The analyzed emotional state data is output.

[1920] Step 5:

[1921] The device receives the quiz notification sent from the server and displays it to the user. The input is the quiz question sent from the server, which is then displayed to the user. Specifically, the device uses push notifications or in-app notifications to intuitively inform the user of the quiz. The displayed quiz is then output.

[1922] Step 6:

[1923] The user answers a quiz displayed on the terminal. The input is the quiz question displayed on the terminal, and the user inputs the answer to that question. In concrete terms, the user inputs the answer to the question and sends it to the server via the terminal. The user's answer is then output.

[1924] Step 7:

[1925] The server collects and evaluates the user's answers. The input is the user's answer data, which is then evaluated. Specifically, the server compares the user's answers with the correct answer data and generates an evaluation result. The generated evaluation result is then output.

[1926] Step 8:

[1927] The server adjusts the feedback according to the user's emotional state and notifies the user of the evaluation results. The input is the evaluation results and the user's emotional state data, and the feedback is created based on this. Specifically, the server generates feedback according to the user's emotional state and sends it to the terminal. The adjusted feedback is then output.

[1928] Step 9:

[1929] The device displays the evaluation results and feedback received from the server to the user. The input is the evaluation results and feedback sent from the server, which are then displayed to the user. Specifically, the device uses a notification function or an in-app interface to visually provide the evaluation results and feedback to the user. The displayed evaluation results and feedback are then output.

[1930] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1931] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1932] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1933] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1934] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1935] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1936] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1937] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1938] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1939] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1940] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1941] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1942] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1943] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1944] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1945] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1946] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1947] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1948] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1949] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1950] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1951] The following is further disclosed regarding the above embodiment.

[1952] (Claim 1)

[1953] A means to analyze the contents of e-books and extract important information;

[1954] A method for automatically generating quizzes based on the extracted information;

[1955] A means for notifying the user of the generated quiz at a predetermined timing;

[1956] and means for collecting and evaluating user responses.

[1957] (Claim 2)

[1958] The system according to claim 1, further comprising means for setting a quiz notification schedule and notifying the user of the quiz at a predetermined timing.

[1959] (Claim 3)

[1960] The system of claim 1, further comprising means for evaluating answers entered by a user to a quiz and notifying the user of the evaluation results.

[1961] "Example 1"

[1962] (Claim 1)

[1963] A means for analyzing the content of electronic publications and extracting important information;

[1964] A method for automatically generating questions based on extracted information;

[1965] A means for notifying the user of the generated problem at a predetermined timing;

[1966] and a means for collecting and evaluating user responses.

[1967] (Claim 2)

[1968] The system of claim 1 [sets a problem notification schedule and notifies the user of problems at predetermined times].

[1969] (Claim 3)

[1970] The system of claim 1 [evaluates the answers entered by the user to the questions and notifies the user of the evaluation results].

[1971] "Application Example 1"

[1972] (Claim 1)

[1973] A means to analyze the contents of e-books and extract important information;

[1974] A method for automatically generating quizzes based on the extracted information;

[1975] A means for notifying the user of the generated quiz at a predetermined timing;

[1976] A method to schedule quiz notifications periodically,

[1977] a means for collecting and evaluating user responses;

[1978] and a means for providing feedback of the evaluation results to the user.

[1979] (Claim 2)

[1980] The system according to claim 1, further comprising means for setting a quiz notification schedule and notifying the user of the quiz at a predetermined timing.

[1981] (Claim 3)

[1982] The system of claim 1, further comprising means for evaluating answers entered by a user to a quiz and notifying the user of the evaluation results.

[1983] "Example 2: Combining Emotion Engines"

[1984] (Claim 1)

[1985] A means to analyze the contents of e-books and extract important information;

[1986] A method for automatically generating quizzes based on the extracted information;

[1987] A means for adjusting the content and difficulty of the generated quiz based on the user's emotional data;

[1988] A means for notifying the user of the generated quiz at a predetermined timing;

[1989] A means for collecting emotion data from the device and transmitting it to a server;

[1990] and means for collecting and evaluating user responses.

[1991] (Claim 2)

[1992] The system of claim 1, wherein a quiz notification schedule is set and the user is notified of the quiz at a predetermined timing.

[1993] (Claim 3)

[1994] The system of claim 1 [evaluates answers entered by a user to a quiz and notifies the user of the evaluation results].

[1995] "Application example 2 when combining emotion engines"

[1996] (Claim 1)

[1997] A means for analyzing the contents of electronic media and extracting important information;

[1998] A method for automatically generating quizzes based on the extracted information;

[1999] A means for notifying the user of the generated quiz at a predetermined timing;

[2000] a means for analyzing and collecting the user's emotional state;

[2001] A means for adjusting the content and difficulty of the quiz according to the user's emotional state;

[2002] and means for collecting and evaluating user responses.

[2003] (Claim 2)

[2004] The system according to claim 1, further comprising means for setting a quiz notification schedule and notifying the user of the quiz at a predetermined timing.

[2005] (Claim 3)

[2006] The system of claim 1, further comprising means for evaluating answers entered by a user to a quiz and notifying the user of the evaluation results. [Explanation of symbols]

[2007] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of analyzing the contents of e-books and extracting important information; A means for automatically generating quizzes based on the extracted information; means for notifying the user of the generated quiz at a predetermined timing; and means for collecting and evaluating user responses.

2. 2. The system according to claim 1, further comprising means for setting a notification schedule for the quiz and notifying the user of the quiz at a predetermined timing.

3. 2. The system according to claim 1, further comprising means for evaluating answers entered by a user to a quiz and notifying the user of the evaluation result.

Images