system

Abstract

We provide the system. [Solution] Information processing device means for receiving inquiries from users and analyzing those inquiries, An information output device means for generating a response to the user based on the analysis results and providing the response, A management device means that monitors the user's learning progress and schedules notifications to support planned learning, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern society, there are many working adults who aim to obtain qualifications while working. However, it is difficult to eliminate insufficient understanding and unclear points in learning, and there is a tendency to hesitate to ask others even simple questions. Therefore, it is difficult to proceed with learning efficiently and continuously, and a system for solving this problem is required.

Means for Solving the Problems

[0005] This invention is a system that includes an information processing device for receiving and analyzing inquiries from users, an information output device for generating appropriate responses to users based on the analysis results, and a management device for monitoring the user's learning progress and scheduling notifications to support planned learning. This configuration allows users to easily ask questions and proceed with their learning with 24-hour support.

[0006] "Users" refers to individuals who use this system to obtain qualifications or engage in learning.

[0007] "Inquiry" refers to the content that users input and send to the system regarding points they don't understand or questions they have while learning.

[0008] An "information processing device" refers to a device that has the function of analyzing inquiries received from users and understanding their content.

[0009] "Analysis" refers to the classification and semantic understanding of information that an information processing device performs in order to understand the content of a query.

[0010] "Response" refers to information generated by an information processing device based on analysis results and used to communicate with the user.

[0011] An "information output device" refers to a device that has the function of providing users with a response generated based on the analysis results.

[0012] "Learning progress" refers to the degree of achievement and progress of a user's learning plan.

[0013] A "management device" refers to a device that monitors the user's learning progress and provides appropriate learning plans and notifications.

[0014] "Notifications" refer to information sent by the management device to the user to remind them of their learning plan or to help maintain their motivation.

[0015] The "knowledge database" refers to a database that stores relevant learning information and materials and is referenced by an information processing device when generating responses.

Brief Description of Drawings

[0016] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10] It shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when the emotion engine is combined. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiment for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention provides a system that enables users to obtain qualifications or achieve specific learning goals. This system consists of multiple components, including an information processing device, an information output device, and a management device.

[0038] The server receives inquiries from users and analyzes them using natural language processing techniques. For example, if a user asks a question about grammar, the server structurally interprets the question and identifies key keywords.

[0039] Based on the analyzed information, the server consults a knowledge database to search for information that will provide the appropriate answer. The database pre-stores grammatical explanations, example sentences, and related materials. The server then generates a response based on the retrieved data, formatted in a way that is easily understandable to the user.

[0040] The terminal receives responses sent from the server and displays them to the user. The responses are displayed in text format, but can also be supplemented with audio output or visual content. Users can use this information to resolve questions and advance their learning.

[0041] Furthermore, the management device has the function of monitoring the user's learning progress and sending reminders based on a pre-set learning plan. For example, it provides users with notifications prompting them to review once a week and encouraging messages according to their progress towards achieving their plan.

[0042] As a concrete example, a user with a question about relative pronouns in English enters the question, and the server categorizes the question under the theme of "relative pronouns." The server then uses its knowledge database to prepare a text containing a basic explanation of relative pronouns and usage examples, which is delivered to the user via their terminal. Based on this information, the user can then complete short practice exercises to deepen their understanding.

[0043] In this way, users can ask questions without worrying about what others think and receive information that is optimally tailored to their individual learning style. Furthermore, continuous learning support allows them to aim for planned goal achievement.

[0044] The following describes the processing flow.

[0045] Step 1:

[0046] The user uses their device to enter a learning-related question and presses the submit button. At this point, the question data is sent from the device to the server.

[0047] Step 2:

[0048] The server analyzes the question data received from the terminal. Using a natural language processing engine, it extracts keywords and themes from the input text and understands the intent of the question.

[0049] Step 3:

[0050] The server references a knowledge database based on the analysis results. It searches the database for information related to the extracted keywords (such as grammatical explanations and example sentences) and retrieves the necessary data.

[0051] Step 4:

[0052] The server generates a response based on the acquired data. It organizes the information in a way that is easy for the user to understand, and presents it in text and visual formats.

[0053] Step 5:

[0054] The server sends the generated response to the terminal. This response may include the answer to the question, as well as supplementary information and learning hints.

[0055] Step 6:

[0056] The terminal displays the response received from the server to the user. In addition to displaying text information on the screen, it also provides a function to read the information aloud as needed.

[0057] Step 7:

[0058] Users resolve their questions and continue learning through the responses provided. If further questions arise, they can return to Step 1 and ask new questions.

[0059] Step 8:

[0060] The management device automatically records the user's learning progress and generates reminders based on the set learning plan. The generated reminders are notified to the user via the terminal at the appropriate time.

[0061] (Example 1)

[0062] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0063] Conventional learning support systems struggle to effectively analyze individual user inquiries and provide timely, appropriate responses. Furthermore, they lack automated notification features to monitor users' learning progress in real time and support planned learning.

[0064] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0065] In this invention, the server includes device means having an information processing function that receives inquiries from users and analyzes said inquiries in natural language using a generative model; device means having an output function that refers to a knowledge database based on the analysis results, generates a response for the user, and provides that response; and device means having a management function that monitors learning progress and schedules reminders in order to support the user's planned learning. This makes it possible to provide appropriate information tailored to the individual learning needs of the user and to provide continuous support for learning.

[0066] A "user" is an individual or group that uses the system's functions for the purpose of obtaining qualifications or achieving learning goals.

[0067] A "generative model" is an algorithmic model that uses machine learning techniques to perform analysis such as natural language processing.

[0068] "Natural language" refers to the language that humans normally use in everyday life, and is the type of language that can be converted into a format that can be processed by a computer.

[0069] "Information processing function" refers to a function that analyzes user input and performs calculations and data manipulation to provide appropriate output.

[0070] A "knowledge database" is a data storage system in which specific information and knowledge are systematically stored and can be searched and referenced.

[0071] An "output function" refers to a function that provides users with a response based on the analysis results, via a display device, audio output device, or similar device.

[0072] The "management function" is a feature that tracks the user's learning progress and schedules reminders and notifications based on their learning plan.

[0073] A "reminder" is a notification message sent to a user at a pre-set time to prompt them to take action.

[0074] In an embodiment of this invention, the system mainly consists of an information processing device, an information output device, and a management device. Users use a terminal connected to this system to learn and ask questions.

[0075] The server is equipped with a natural language processing engine that includes a generative AI model, and receives inquiries from users. This allows the server to analyze the input natural language and extract key keywords. This analysis process includes, for example, grammatical and semantic analysis of text-based questions.

[0076] The server then refers to its internal knowledge database based on the analysis results. This database hierarchically stores various specialized content, allowing the server to quickly retrieve relevant information. For example, if a user asks, "What is a relative pronoun?", the server extracts relevant information from the knowledge database based on the keyword "relative pronoun."

[0077] Based on the extracted information, the server generates an appropriate response. This response is formatted in natural language for easy user understanding and provided to the user as text, audio, or visual content. The terminal receives this response and displays or outputs it to the user through its screen or speaker.

[0078] The management system monitors the user's learning progress and sends reminders and notifications according to a pre-set learning plan. This function supports users in systematically progressing with their studies and efficiently achieving their goals. For example, it helps reinforce learning by sending reminders to users to review the material once a week.

[0079] As a concrete example, a user might use the prompt, "Please tell me about relative pronouns in English." Upon receiving this prompt, the server analyzes the question, searches its knowledge database to provide appropriate information, and generates and provides an answer that is easy for the user to understand. In this way, users can easily obtain information tailored to their individual learning needs.

[0080] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0081] Step 1:

[0082] The user uses a terminal to input learning and research questions in text format. This input is sent to the terminal as a prompt. The terminal receives this input and prepares it for transmission to the server. The input data is processed while maintaining its natural language format.

[0083] Step 2:

[0084] The server receives prompt messages sent from the terminal. This input data is passed to the generating AI model, which then begins analysis using a natural language processing engine. Data processing includes grammatical analysis and keyword extraction. Specifically, the sentence structure is analyzed to identify important keywords, which are then used as preparation for subsequent database lookups. The output consists of the analyzed keywords and contextual information of the sentence.

[0085] Step 3:

[0086] The server references a knowledge database based on the analyzed keywords. This database search involves data calculations to quickly retrieve relevant information and materials. For example, if the keyword "relative pronoun" is extracted, related grammatical explanations and example sentences are retrieved from the database. The output is a list containing all of this necessary information.

[0087] Step 4:

[0088] The server generates a response based on information retrieved from the database. Using a generative AI model, it formats the information into natural language text that is easy for the user to understand, and sometimes supplements the output with visual or auditory elements. Specifically, the server processes this output data into a format that is easy to generate and prepares it for transmission to the terminal.

[0089] Step 5:

[0090] The terminal receives the response sent from the server. The received data is prepared for the user to receive and displayed as text on the terminal, and output as audio or visual content as needed. The output is in a format that the user can easily understand and use. As a result, the user can obtain answers to their inquiries and proceed with learning and question resolution.

[0091] Step 6:

[0092] The management device monitors the user's learning progress, generates reminders based on a pre-configured learning plan, and sends them to the terminal according to the schedule. Specifically, it collects learning status data and identifies when notifications are needed in accordance with the plan. This is aimed at maintaining user motivation and providing output that encourages regular review and other activities. Output includes timely notifications and feedback to support the user's learning activities.

[0093] (Application Example 1)

[0094] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0095] Traditional learning support systems have faced challenges in providing accurate responses to individual user inquiries and real-time visual feedback. Furthermore, they lacked sufficient means to instantly provide information related to specific objects. Therefore, these challenges need to be addressed to provide a more effective and intuitive learning experience.

[0096] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0097] In this invention, the server includes information processing means for receiving and analyzing inquiries from users, information output means for generating and providing responses to the user based on the analysis results, and management means for monitoring the user's learning progress and scheduling notifications to support planned learning. This allows users to quickly receive information about objects in their line of sight and enjoy a more natural and intuitive learning experience.

[0098] An "information processing device" is a device that receives inquiries from users, analyzes those inquiries, and generates appropriate responses based on the analysis results.

[0099] An "information output device" is a device that provides users with a response generated based on the results of the analysis.

[0100] A "management device" is a device that monitors the user's learning progress and schedules notifications to support planned learning.

[0101] An "eye-tracking device" is a device that tracks a learner's gaze and provides visual feedback in real time.

[0102] An "object recognition device" is a device that recognizes objects in real time and provides information about their general characteristics and related information.

[0103] According to this invention, the server receives a query from a user and analyzes it using natural language processing technology. The information processing device analyzes the query as text data and refers to a knowledge database to generate an appropriate response based on the analysis results. This database stores various information, including grammatical explanations, example sentences, and related materials. Based on the analysis results, the server generates a response in an easily understandable format for the user and provides it through an information output device.

[0104] The terminal is equipped with an eye-tracking device that tracks the user's gaze direction in real time and displays relevant information on the screen as needed. This allows the object recognition device to recognize objects in the user's line of sight and provide information related to those objects.

[0105] The server monitors learning progress through a management device and schedules notifications based on the user's learning plan. This helps users effectively progress in their learning and achieve their planned goals.

[0106] For example, if a user wants to know more about a grammatical point, the server can analyze the request, retrieve the appropriate information from an existing database, and present it to the user. The user can receive this information through a glasses-type display or audio.

[0107] An example of a prompt message based on a generative AI model is sending an instruction to the server such as, "Please provide basic information and related details about the object the user is looking at," which can then retrieve appropriate information.

[0108] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0109] Step 1:

[0110] The server receives inquiries from users. These inquiries are input as natural language data in text format. Upon receiving this data, the server analyzes the text data using natural language processing techniques. As a result of the analysis, the subject and keywords of the inquiry are extracted.

[0111] Step 2:

[0112] The server consults a knowledge database based on the subject and keywords analyzed in Step 1. During the database lookup, it searches for relevant information and collects appropriate answer candidates. The output includes answer candidates and related information.

[0113] Step 3:

[0114] The server generates a user-optimized response based on the collected answer candidates. Text generation technology is used to construct the response in a format easily understood by the user. The result of the response generation is output to the user as natural language text.

[0115] Step 4:

[0116] The terminal receives text responses sent from the server and presents them to the user visually or audibly. If an eye-tracking device is integrated into the terminal, interactive responses are enabled by displaying relevant information based on the user's gaze. Output is the provision of information to the user as a display or audio output.

[0117] Step 5:

[0118] The server uses a management device to monitor the user's learning progress and schedule notifications according to the learning plan. These notifications include, for example, reminders and recommendations for the next learning activity. Operation logs and progress data are analyzed to develop an appropriate notification plan. The output is provided periodically in the form of notifications to the user.

[0119] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0120] The system implemented by this invention can understand the user's emotions and provide an appropriate response based on those emotions. The system consists mainly of an information processing device, an emotion engine, an information output device, and a management device, all of which work together in coordination.

[0121] The server receives inquiries from users through terminals. Inquiries are entered in text format and analyzed by an emotion engine. As part of the analysis, a process is included to detect emotions from the user's text. For example, natural language processing and emotion analysis algorithms are used to recognize whether the user is expressing emotions such as "anxiety" or "confusion."

[0122] The server retrieves relevant information from a knowledge database based on the emotions recognized by the emotion engine and the content of the received query. This information is then organized in an appropriate format according to the user's situation and provided as a response.

[0123] The terminal displays a response to the user that is based on the emotions sent from the server. This response is tailored to the user's emotions and may include more polite language and encouraging comments. For example, if the user expresses anxiety about difficult grammar, the server will send a detailed explanation along with an encouraging message to boost their motivation.

[0124] Furthermore, the management device has the function of linking the user's emotions with their learning progress and scheduling notifications. When a user is experiencing stress during their studies, the management device sends hints to reduce stress and notifications to encourage refreshment to the device, supporting an effective learning environment.

[0125] For example, if a user sends an inquiry stating, "My recent test results were poor," the emotion engine detects emotions such as "disappointment" or "anxiety." Based on this, the server suggests ways to review the test and also provides encouraging messages such as, "Let's try harder next time; there's room for improvement." In this way, the system responds to the user's emotions, enabling more personalized learning support.

[0126] The following describes the processing flow.

[0127] Step 1:

[0128] Users access the system through their devices and submit learning-related questions and their current status as text input. During this process, expressions containing emotions may be naturally included.

[0129] Step 2:

[0130] The terminal sends the text entered by the user to the server. The data sent includes information such as the user ID and the time of input.

[0131] Step 3:

[0132] The server passes the received text data to the emotion engine. This engine uses natural language processing techniques to analyze the emotions contained in the text and recognize specific emotional states (e.g., "stress," "anxiety," "satisfaction," etc.).

[0133] Step 4:

[0134] The server combines the analysis results from the emotion engine with the user's question to generate a query to the knowledge database. This allows it to search for information related to the question.

[0135] Step 5:

[0136] The server generates a response to the user based on information retrieved from the database. If the user is in a specific emotional state, the response is appropriately adjusted, for example, by adding comments expressing "encouragement" or "empathy."

[0137] Step 6:

[0138] The server sends the generated response data to the terminal. The response includes a message tailored based on information obtained from the knowledge database and emotional responses.

[0139] Step 7:

[0140] The terminal displays the response received from the server to the user. Flexible language that responds to emotions is used to provide information that is easy for the user to understand and that is sensitive to their feelings.

[0141] Step 8:

[0142] The user reviews the displayed response and continues learning. If needed, they can enter the question again to receive further information and support.

[0143] Step 9:

[0144] The management system generates timely reminders based on the user's learning progress and emotional records. This provides regular notifications to support the user's learning plan and motivation.

[0145] (Example 2)

[0146] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0147] There is a need for a system that can understand the user's emotional state, provide personalized responses, and effectively support learning. Furthermore, a challenge is to reduce learning stress based on the user's emotions and provide an efficient learning environment.

[0148] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0149] In this invention, the server includes an information processing device means that receives an inquiry from a user, analyzes the inquiry using natural language processing technology, and detects the user's emotions; an information output device means that generates a personalized response for the user based on the emotion analysis results and provides the response; and a management device means that monitors the user's learning progress and emotional state and schedules notifications for planned learning support. This enables efficient learning support that takes the user's emotions into account and promotes stress reduction.

[0150] An "information processing device" is a device that receives inquiries from users, analyzes those inquiries using natural language processing technology, and detects the user's emotions.

[0151] "Natural language processing" is a technology that uses machines to analyze, understand, and generate responses to human language.

[0152] "Sentiment analysis" is the process of extracting and identifying a user's emotions from text data.

[0153] An "information output device" is a device that provides users with personalized responses generated based on the results of emotion analysis.

[0154] A "management device" is a device that monitors the user's learning progress and emotional state, and schedules notifications for planned learning support.

[0155] A "personalized response" is a response that is individually tailored to the user's emotional state and needs.

[0156] "Learning support" refers to the provision of various information and notifications aimed at assisting users' learning activities and improving their efficiency.

[0157] The system for implementing this invention is based on information processing technology that has the ability to understand user emotions and respond appropriately. The system mainly consists of a server, terminals, and management devices, each working in cooperation with the others.

[0158] The server receives user inquiries through the terminal. In this process, the user's inquiry, provided in text format, is processed first within the system. Specifically, the server analyzes the inquiry using natural language processing techniques and identifies the user's emotions using an emotion engine. Sentiment analysis algorithms play a crucial role in this process.

[0159] For example, if a user sends an inquiry such as, "I got a lower score than I expected on my recent exam," the server will detect emotions such as "disappointment" or "anxiety." Once an emotion is detected, the server will refer to its knowledge database to gather relevant learning support information and generate a response tailored to the user's emotions. This may include learning advice and motivational messages.

[0160] The generated response is sent to the terminal and displayed to the user. The terminal receives the message sent from the server and presents it to the user in an emotionally sensitive manner. As a result, the response is tailored to the user's feelings and has the effect of supporting learning.

[0161] Furthermore, the management system monitors the user's learning progress and emotional state, and schedules notifications to prompt refreshment as needed. This continuous support allows users to enjoy an effective learning environment.

[0162] An example of a prompt might be, "Analyze the sentiment from the text entered by the user and generate a personalized response." This prompt instructs the generative AI model to generate a response based on the user's sentiment.

[0163] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0164] Step 1:

[0165] Users enter their inquiries in text format through their terminals and send them to the system. The inquiries reflect the user's situation and questions and are subject to sentiment-based analysis. The entered text is sent to the server.

[0166] Step 2:

[0167] The server receives text sent by the user. The received text is analyzed using natural language processing techniques to determine its linguistic structure. This process extracts data on the grammar and meaning of the text. Based on this, an emotion engine is used to perform emotion analysis. An emotion analysis algorithm is applied to identify emotions such as "anxiety" and "joy." The analysis results output the user's emotional state.

[0168] Step 3:

[0169] The server searches a knowledge database based on the sentiment analysis results and retrieves appropriate information. It selects information relevant to the user's question and collects data to generate an emotion-appropriate response. This prepares an appropriate response that takes emotions into consideration.

[0170] Step 4:

[0171] The server uses a generative AI model to create personalized responses. It takes sentiment analysis results and information from a knowledge database as input. Based on this, it outputs customized messages that match the user's emotions. The generated responses include encouragement and advice and are optimized for the user's situation.

[0172] Step 5:

[0173] The device receives responses sent from the server and displays them to the user. The outputted personalized message is shown on the user's screen, and the user can see it in real time. Through this process, the user receives emotionally sensitive feedback.

[0174] Step 6:

[0175] The management system monitors and records the user's learning progress and emotional state. Based on this information, it schedules notifications as needed. Learning history and emotional data are used as input data. Output includes timely refresh notifications and learning hints, which are sent to the user via their device. This allows the user to enjoy an efficient and stress-free learning environment.

[0176] (Application Example 2)

[0177] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0178] With the advancement of modern information technology, presenting advertisements that respond immediately to consumer emotions has become a crucial challenge in the market. However, conventional advertising systems are limited to delivering ads based on general user attributes, making it difficult to present ads based on the real-time emotions of individual users. This invention aims to solve these problems and provide a more personalized advertising service.

[0179] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0180] In this invention, the server includes an information processing device means for receiving inquiries or posts from users and analyzing said inquiries or posts; an information output device means for generating and providing advertisements that are appropriate to the user's emotions based on the analysis results; and a management device means for monitoring the user's emotions and scheduling the display of emotion-based advertisements. This enables the real-time display of advertisements that take into account the user's emotional state.

[0181] An "information processing device" is a device that receives inquiries or posts from users and analyzes them.

[0182] "Analysis results" refer to the content of emotions and information obtained when an information processing device analyzes a query or post in natural language.

[0183] "Advertisements" are promotional information generated based on analysis results and provided to users through information output devices.

[0184] An "information output device" is a device that provides users with advertisements generated based on the analysis results.

[0185] A "management device" is a device that monitors users' emotions and schedules the display of emotion-based advertisements.

[0186] A "knowledge repository" is a collection of information referenced to generate advertisements, and is a database containing predetermined advertising information.

[0187] The system implementing this invention performs a series of processes to optimize advertisements based on sentiment. The server utilizes an information processing device to collect inquiries or posts from users. The information processing device analyzes the posts using a natural language processing engine and extracts the user's sentiment. Google® Cloud Natural Language API can be used for this analysis.

[0188] Next, the server identifies information related to ad generation based on the sentiment analysis results and retrieves this information using a knowledge repository. Ad delivery platforms such as the Google Ads API can be used for ad generation. The ad is then generated and displayed to the user's device at the appropriate time via an information output device.

[0189] Furthermore, the management system monitors users' real-time emotional data and schedules when advertisements tailored to each user's emotions should be displayed. This enables the display of advertisements in a way that is relevant to their emotions.

[0190] For example, if a user posts "I'm feeling very stressed today," advertisements for relaxation products and services will be displayed. In this way, the system dynamically adjusts advertisements and provides personalized content to each user.

[0191] Examples of prompts to input into a generative AI model:

[0192] "Perform sentiment analysis and extract the sentiment from the following text: '{user post}'. Generate an ad that matches this sentiment."

[0193] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0194] Step 1:

[0195] The server receives posted data from the user's smartphone. This posted data is, for example, text from social media or messaging apps. The input is text data, and the output is stored on the server in a formatted, parseable data format.

[0196] Step 2:

[0197] The server uses a natural language processing engine (e.g., Google Cloud Natural Language API) to analyze the received post data. The input is formatted text data, and the output is sentiment analysis results. These results include sentiment categories such as "joy," "anger," and "sadness." During the analysis, keywords are extracted from the text and analyzed based on a pre-trained sentiment model.

[0198] Step 3:

[0199] The server selects appropriate advertisements based on the results of sentiment analysis. The input is the result of sentiment analysis, and the output is data of the selected advertisements. Using a knowledge repository, it searches for advertisement information related to sentiment and retrieves appropriate advertisements through an advertising platform (e.g., Google Ads API). This process uses an algorithm that selects the best advertisement from multiple candidates based on sentiment category.

[0200] Step 4:

[0201] The server transmits selected advertising data to the terminal and displays the advertisement on the user's smartphone via an information output device. The input is advertising data, and the output is the specific advertising content displayed on the user's smartphone. On the terminal, the received advertising data is immediately displayed and presented with an appropriate layout within the application so that the user can easily see it.

[0202] Step 5:

[0203] The management system records user sentiment and ad viewing history, and uses this information to select future ads. Inputs are sentiment analysis results and ad viewing history, and output is storage in a history database. This process allows for the understanding of long-term user sentiment trends, which can be used to improve future ad display strategies. The management system continues to analyze and schedule ad displays in a timely and individualized manner until the next sentiment analysis is performed.

[0204] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0205] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0206] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0207] [Second Embodiment]

[0208] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0209] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0210] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0211] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0212] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0213] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0214] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0215] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0216] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0217] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0218] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0219] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0220] This invention provides a system that enables users to obtain qualifications or achieve specific learning goals. This system consists of multiple components, including an information processing device, an information output device, and a management device.

[0221] The server receives inquiries from users and analyzes them using natural language processing techniques. For example, if a user asks a question about grammar, the server structurally interprets the question and identifies key keywords.

[0222] Based on the analyzed information, the server consults a knowledge database to search for information that will provide the appropriate answer. The database pre-stores grammatical explanations, example sentences, and related materials. The server then generates a response based on the retrieved data, formatted in a way that is easily understandable to the user.

[0223] The terminal receives responses sent from the server and displays them to the user. The responses are displayed in text format, but can also be supplemented with audio output or visual content. Users can use this information to resolve questions and advance their learning.

[0224] Furthermore, the management device has the function of monitoring the user's learning progress and sending reminders based on a pre-set learning plan. For example, it provides users with notifications prompting them to review once a week and encouraging messages according to their progress towards achieving their plan.

[0225] As a concrete example, a user with a question about relative pronouns in English enters the question, and the server categorizes the question under the theme of "relative pronouns." The server then uses its knowledge database to prepare a text containing a basic explanation of relative pronouns and usage examples, which is delivered to the user via their terminal. Based on this information, the user can then complete short practice exercises to deepen their understanding.

[0226] In this way, users can ask questions without worrying about what others think and receive information that is optimally tailored to their individual learning style. Furthermore, continuous learning support allows them to aim for planned goal achievement.

[0227] The following describes the processing flow.

[0228] Step 1:

[0229] The user uses their device to enter a learning-related question and presses the submit button. At this point, the question data is sent from the device to the server.

[0230] Step 2:

[0231] The server analyzes the question data received from the terminal. Using a natural language processing engine, it extracts keywords and themes from the input text and understands the intent of the question.

[0232] Step 3:

[0233] The server references a knowledge database based on the analysis results. It searches the database for information related to the extracted keywords (such as grammatical explanations and example sentences) and retrieves the necessary data.

[0234] Step 4:

[0235] The server generates a response based on the acquired data. It organizes the information in a way that is easy for the user to understand, and presents it in text and visual formats.

[0236] Step 5:

[0237] The server sends the generated response to the terminal. This response may include the answer to the question, as well as supplementary information and learning hints.

[0238] Step 6:

[0239] The terminal displays the response received from the server to the user. In addition to displaying text information on the screen, it also provides a function to read the information aloud as needed.

[0240] Step 7:

[0241] Users resolve their questions and continue learning through the responses provided. If further questions arise, they can return to Step 1 and ask new questions.

[0242] Step 8:

[0243] The management device automatically records the user's learning progress and generates reminders based on the set learning plan. The generated reminders are notified to the user via the terminal at the appropriate time.

[0244] (Example 1)

[0245] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0246] Conventional learning support systems struggle to effectively analyze individual user inquiries and provide timely, appropriate responses. Furthermore, they lack automated notification features to monitor users' learning progress in real time and support planned learning.

[0247] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0248] In this invention, the server includes device means having an information processing function that receives inquiries from users and analyzes said inquiries in natural language using a generative model; device means having an output function that refers to a knowledge database based on the analysis results, generates a response for the user, and provides that response; and device means having a management function that monitors learning progress and schedules reminders in order to support the user's planned learning. This makes it possible to provide appropriate information tailored to the individual learning needs of the user and to provide continuous support for learning.

[0249] A "user" is an individual or group that uses the system's functions for the purpose of obtaining qualifications or achieving learning goals.

[0250] A "generative model" is an algorithmic model that uses machine learning techniques to perform analysis such as natural language processing.

[0251] "Natural language" refers to the language that humans normally use in everyday life, and is the type of language that can be converted into a format that can be processed by a computer.

[0252] "Information processing function" refers to a function that analyzes user input and performs calculations and data manipulation to provide appropriate output.

[0253] A "knowledge database" is a data storage system in which specific information and knowledge are systematically stored and can be searched and referenced.

[0254] An "output function" refers to a function that provides users with a response based on the analysis results, via a display device, audio output device, or similar device.

[0255] The "management function" is a feature that tracks the user's learning progress and schedules reminders and notifications based on their learning plan.

[0256] A "reminder" is a notification message sent to a user at a pre-set time to prompt them to take action.

[0257] In an embodiment of this invention, the system mainly consists of an information processing device, an information output device, and a management device. Users use a terminal connected to this system to learn and ask questions.

[0258] The server is equipped with a natural language processing engine that includes a generative AI model, and receives inquiries from users. This allows the server to analyze the input natural language and extract key keywords. This analysis process includes, for example, grammatical and semantic analysis of text-based questions.

[0259] The server then refers to its internal knowledge database based on the analysis results. This database hierarchically stores various specialized content, allowing the server to quickly retrieve relevant information. For example, if a user asks, "What is a relative pronoun?", the server extracts relevant information from the knowledge database based on the keyword "relative pronoun."

[0260] Based on the extracted information, the server generates an appropriate response. This response is formatted in natural language for easy user understanding and provided to the user as text, audio, or visual content. The terminal receives this response and displays or outputs it to the user through its screen or speaker.

[0261] The management system monitors the user's learning progress and sends reminders and notifications according to a pre-set learning plan. This function supports users in systematically progressing with their studies and efficiently achieving their goals. For example, it helps reinforce learning by sending reminders to users to review the material once a week.

[0262] As a concrete example, a user might use the prompt, "Please tell me about relative pronouns in English." Upon receiving this prompt, the server analyzes the question, searches its knowledge database to provide appropriate information, and generates and provides an answer that is easy for the user to understand. In this way, users can easily obtain information tailored to their individual learning needs.

[0263] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0264] Step 1:

[0265] The user uses a terminal to input learning and research questions in text format. This input is sent to the terminal as a prompt. The terminal receives this input and prepares it for transmission to the server. The input data is processed while maintaining its natural language format.

[0266] Step 2:

[0267] The server receives prompt messages sent from the terminal. This input data is passed to the generating AI model, which then begins analysis using a natural language processing engine. Data processing includes grammatical analysis and keyword extraction. Specifically, the sentence structure is analyzed to identify important keywords, which are then used as preparation for subsequent database lookups. The output consists of the analyzed keywords and contextual information of the sentence.

[0268] Step 3:

[0269] The server references a knowledge database based on the analyzed keywords. This database search involves data calculations to quickly retrieve relevant information and materials. For example, if the keyword "relative pronoun" is extracted, related grammatical explanations and example sentences are retrieved from the database. The output is a list containing all of this necessary information.

[0270] Step 4:

[0271] The server generates a response based on information retrieved from the database. Using a generative AI model, it formats the information into natural language text that is easy for the user to understand, and sometimes supplements the output with visual or auditory elements. Specifically, the server processes this output data into a format that is easy to generate and prepares it for transmission to the terminal.

[0272] Step 5:

[0273] The terminal receives the response sent from the server. The received data is prepared for the user to receive and displayed as text on the terminal, and output as audio or visual content as needed. The output is in a format that the user can easily understand and use. As a result, the user can obtain answers to their inquiries and proceed with learning and question resolution.

[0274] Step 6:

[0275] The management device monitors the user's learning progress, generates reminders based on a pre-configured learning plan, and sends them to the terminal according to the schedule. Specifically, it collects learning status data and identifies when notifications are needed in accordance with the plan. This is aimed at maintaining user motivation and providing output that encourages regular review and other activities. Output includes timely notifications and feedback to support the user's learning activities.

[0276] (Application Example 1)

[0277] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0278] Traditional learning support systems have faced challenges in providing accurate responses to individual user inquiries and real-time visual feedback. Furthermore, they lacked sufficient means to instantly provide information related to specific objects. Therefore, these challenges need to be addressed to provide a more effective and intuitive learning experience.

[0279] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0280] In this invention, the server includes information processing means for receiving and analyzing inquiries from users, information output means for generating and providing responses to the user based on the analysis results, and management means for monitoring the user's learning progress and scheduling notifications to support planned learning. This allows users to quickly receive information about objects in their line of sight and enjoy a more natural and intuitive learning experience.

[0281] An "information processing device" is a device that receives an inquiry from a user, analyzes the inquiry, and generates an appropriate response based on the analysis result.

[0282] An "information output device" is a device for providing a response generated based on the analyzed result to the user.

[0283] A "management device" is a device that monitors the learning progress of a user and schedules notifications for supporting planned learning.

[0284] A "gaze tracking device" is a device for tracking the gaze of a learner and providing visual feedback in real time.

[0285] An "object recognition device" is a device for recognizing an object in real time and providing information related to its outline.

[0286] According to this invention, the server receives an inquiry from a user and analyzes it using natural language processing technology. The information processing device analyzes the inquiry as text data and refers to a knowledge database to generate an appropriate response based on the analysis result. Various information is stored in this database, including grammar explanations, example sentences, related materials, etc. The server generates a response in an easy-to-understand format for the user based on the analysis result and provides it through the information output device.

[0287] The terminal is equipped with a gaze tracking device that tracks the user's gaze direction in real time and displays relevant information on the display as needed. As a result, the object recognition device can recognize the object in front of the user's gaze and provide information related to the object.

[0288] The server monitors the learning progress through the management device and schedules notifications based on the user's learning plan. Thereby, the user can effectively proceed with learning and is supported in achieving planned goals.

[0289] For example, if a user wants to know more about a grammatical point, the server can analyze the request, retrieve the appropriate information from an existing database, and present it to the user. The user can receive this information through a glasses-type display or audio.

[0290] An example of a prompt message based on a generative AI model is sending an instruction to the server such as, "Please provide basic information and related details about the object the user is looking at," which can then retrieve appropriate information.

[0291] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0292] Step 1:

[0293] The server receives inquiries from users. These inquiries are input as natural language data in text format. Upon receiving this data, the server analyzes the text data using natural language processing techniques. As a result of the analysis, the subject and keywords of the inquiry are extracted.

[0294] Step 2:

[0295] The server consults a knowledge database based on the subject and keywords analyzed in Step 1. During the database lookup, it searches for relevant information and collects appropriate answer candidates. The output includes answer candidates and related information.

[0296] Step 3:

[0297] The server generates a user-optimized response based on the collected answer candidates. Text generation technology is used to construct the response in a format easily understood by the user. The result of the response generation is output to the user as natural language text.

[0298] Step 4:

[0299] The terminal receives text responses sent from the server and presents them to the user visually or audibly. If an eye-tracking device is integrated into the terminal, interactive responses are enabled by displaying relevant information based on the user's gaze. Output is the provision of information to the user as a display or audio output.

[0300] Step 5:

[0301] The server uses a management device to monitor the user's learning progress and schedule notifications according to the learning plan. These notifications include, for example, reminders and recommendations for the next learning activity. Operation logs and progress data are analyzed to develop an appropriate notification plan. The output is provided periodically in the form of notifications to the user.

[0302] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0303] The system implemented by this invention can understand the user's emotions and provide an appropriate response based on those emotions. The system consists mainly of an information processing device, an emotion engine, an information output device, and a management device, all of which work together in coordination.

[0304] The server receives inquiries from users through terminals. Inquiries are entered in text format and analyzed by an emotion engine. As part of the analysis, a process is included to detect emotions from the user's text. For example, natural language processing and emotion analysis algorithms are used to recognize whether the user is expressing emotions such as "anxiety" or "confusion."

[0305] The server obtains relevant information by referring to the knowledge database based on the emotions recognized by the emotion engine and the received inquiry content. This information is compiled in an appropriate form according to the user's situation and provided as a response.

[0306] The terminal displays the response based on the emotion sent from the server to the user. This response is adjusted considering the user's emotion and may include more polite words or encouraging comments. For example, when the user shows anxiety about difficult grammar, the server sends a motivating message to boost motivation in addition to a detailed explanation.

[0307] Furthermore, the management device has a function of linking the user's emotion and learning progress and scheduling notifications. When the user feels stressed in learning, the management device sends hints for stress reduction or notifications to prompt refresh to the terminal to support an effective learning environment.

[0308] As a specific example, when the user sends an inquiry saying "My recent test results were bad", the emotion engine detects emotions such as "frustration" and "uneasiness". Based on this, the server proposes a method for reviewing the test and presents a motivating message such as "Let's work hard next time. There is room for improvement". In this way, the system responds to the user's emotion and realizes more personalized learning support.

[0309] The following describes the processing flow.

[0310] Step 1:

[0311] The user accesses the system through the terminal and sends questions about learning or the current situation as text input. At this time, expressions including emotions may be naturally input.

[0312] Step 2:

[0313] The terminal sends the text entered by the user to the server. The data sent includes information such as the user ID and the time of input.

[0314] Step 3:

[0315] The server passes the received text data to the emotion engine. This engine uses natural language processing techniques to analyze the emotions contained in the text and recognize specific emotional states (e.g., "stress," "anxiety," "satisfaction," etc.).

[0316] Step 4:

[0317] The server combines the analysis results from the emotion engine with the user's question to generate a query to the knowledge database. This allows it to search for information related to the question.

[0318] Step 5:

[0319] The server generates a response to the user based on information retrieved from the database. If the user is in a specific emotional state, the response is appropriately adjusted, for example, by adding comments expressing "encouragement" or "empathy."

[0320] Step 6:

[0321] The server sends the generated response data to the terminal. The response includes a message tailored based on information obtained from the knowledge database and emotional responses.

[0322] Step 7:

[0323] The terminal displays the response received from the server to the user. Flexible language that responds to emotions is used to provide information that is easy for the user to understand and that is sensitive to their feelings.

[0324] Step 8:

[0325] The user reviews the displayed response and continues learning. If needed, they can enter the question again to receive further information and support.

[0326] Step 9:

[0327] The management system generates timely reminders based on the user's learning progress and emotional records. This provides regular notifications to support the user's learning plan and motivation.

[0328] (Example 2)

[0329] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0330] There is a need for a system that can understand the user's emotional state, provide personalized responses, and effectively support learning. Furthermore, a challenge is to reduce learning stress based on the user's emotions and provide an efficient learning environment.

[0331] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0332] In this invention, the server includes an information processing device means that receives an inquiry from a user, analyzes the inquiry using natural language processing technology, and detects the user's emotions; an information output device means that generates a personalized response for the user based on the emotion analysis results and provides the response; and a management device means that monitors the user's learning progress and emotional state and schedules notifications for planned learning support. This enables efficient learning support that takes the user's emotions into account and promotes stress reduction.

[0333] An "information processing device" is a device that receives inquiries from users, analyzes those inquiries using natural language processing technology, and detects the user's emotions.

[0334] "Natural language processing" is a technology that uses machines to analyze, understand, and generate responses to human language.

[0335] "Sentiment analysis" is the process of extracting and identifying a user's emotions from text data.

[0336] An "information output device" is a device that provides users with personalized responses generated based on the results of emotion analysis.

[0337] A "management device" is a device that monitors the user's learning progress and emotional state, and schedules notifications for planned learning support.

[0338] A "personalized response" is a response that is individually tailored to the user's emotional state and needs.

[0339] "Learning support" refers to the provision of various information and notifications aimed at assisting users' learning activities and improving their efficiency.

[0340] The system for implementing this invention is based on information processing technology that has the ability to understand user emotions and respond appropriately. The system mainly consists of a server, terminals, and management devices, each working in cooperation with the others.

[0341] The server receives user inquiries through the terminal. In this process, the user's inquiry, provided in text format, is processed first within the system. Specifically, the server analyzes the inquiry using natural language processing techniques and identifies the user's emotions using an emotion engine. Sentiment analysis algorithms play a crucial role in this process.

[0342] For example, if a user sends an inquiry such as, "I got a lower score than I expected on my recent exam," the server will detect emotions such as "disappointment" or "anxiety." Once an emotion is detected, the server will refer to its knowledge database to gather relevant learning support information and generate a response tailored to the user's emotions. This may include learning advice and motivational messages.

[0343] The generated response is sent to the terminal and displayed to the user. The terminal receives the message sent from the server and presents it to the user in an emotionally sensitive manner. As a result, the response is tailored to the user's feelings and has the effect of supporting learning.

[0344] Furthermore, the management system monitors the user's learning progress and emotional state, and schedules notifications to prompt refreshment as needed. This continuous support allows users to enjoy an effective learning environment.

[0345] An example of a prompt might be, "Analyze the sentiment from the text entered by the user and generate a personalized response." This prompt instructs the generative AI model to generate a response based on the user's sentiment.

[0346] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0347] Step 1:

[0348] Users enter their inquiries in text format through their terminals and send them to the system. The inquiries reflect the user's situation and questions and are subject to sentiment-based analysis. The entered text is sent to the server.

[0349] Step 2:

[0350] The server receives text sent by the user. The received text is analyzed using natural language processing techniques to determine its linguistic structure. This process extracts data on the grammar and meaning of the text. Based on this, an emotion engine is used to perform emotion analysis. An emotion analysis algorithm is applied to identify emotions such as "anxiety" and "joy." The analysis results output the user's emotional state.

[0351] Step 3:

[0352] The server searches a knowledge database based on the sentiment analysis results and retrieves appropriate information. It selects information relevant to the user's question and collects data to generate an emotion-appropriate response. This prepares an appropriate response that takes emotions into consideration.

[0353] Step 4:

[0354] The server uses a generative AI model to create personalized responses. It takes sentiment analysis results and information from a knowledge database as input. Based on this, it outputs customized messages that match the user's emotions. The generated responses include encouragement and advice and are optimized for the user's situation.

[0355] Step 5:

[0356] The device receives responses sent from the server and displays them to the user. The outputted personalized message is shown on the user's screen, and the user can see it in real time. Through this process, the user receives emotionally sensitive feedback.

[0357] Step 6:

[0358] The management system monitors and records the user's learning progress and emotional state. Based on this information, it schedules notifications as needed. Learning history and emotional data are used as input data. Output includes timely refresh notifications and learning hints, which are sent to the user via their device. This allows the user to enjoy an efficient and stress-free learning environment.

[0359] (Application Example 2)

[0360] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0361] With the advancement of modern information technology, presenting advertisements that respond immediately to consumer emotions has become a crucial challenge in the market. However, conventional advertising systems are limited to delivering ads based on general user attributes, making it difficult to present ads based on the real-time emotions of individual users. This invention aims to solve these problems and provide a more personalized advertising service.

[0362] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0363] In this invention, the server includes an information processing device means for receiving inquiries or posts from users and analyzing said inquiries or posts; an information output device means for generating and providing advertisements that are appropriate to the user's emotions based on the analysis results; and a management device means for monitoring the user's emotions and scheduling the display of emotion-based advertisements. This enables the real-time display of advertisements that take into account the user's emotional state.

[0364] An "information processing device" is a device that receives inquiries or posts from users and analyzes them.

[0365] "Analysis results" refer to the content of emotions and information obtained when an information processing device analyzes a query or post in natural language.

[0366] "Advertisements" are promotional information generated based on analysis results and provided to users through information output devices.

[0367] An "information output device" is a device that provides users with advertisements generated based on the analysis results.

[0368] A "management device" is a device that monitors users' emotions and schedules the display of emotion-based advertisements.

[0369] A "knowledge repository" is a collection of information referenced to generate advertisements, and is a database containing predetermined advertising information.

[0370] The system implementing this invention performs a series of processes to optimize advertisements based on sentiment. The server utilizes an information processing device to collect inquiries or posts from users. The information processing device analyzes the posts using a natural language processing engine to extract the user's sentiment. The Google Cloud Natural Language API can be used for this analysis.

[0371] Next, the server identifies information related to ad generation based on the sentiment analysis results and retrieves this information using a knowledge repository. Ad delivery platforms such as the Google Ads API can be used for ad generation. The ad is then generated and displayed to the user's device at the appropriate time via an information output device.

[0372] Furthermore, the management system monitors users' real-time emotional data and schedules when advertisements tailored to each user's emotions should be displayed. This enables the display of advertisements in a way that is relevant to their emotions.

[0373] For example, if a user posts "I'm feeling very stressed today," advertisements for relaxation products and services will be displayed. In this way, the system dynamically adjusts advertisements and provides personalized content to each user.

[0374] Examples of prompts to input into a generative AI model:

[0375] "Perform sentiment analysis and extract the sentiment from the following text: '{user post}'. Generate an ad that matches this sentiment."

[0376] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0377] Step 1:

[0378] The server receives posted data from the user's smartphone. This posted data is, for example, text from social media or messaging apps. The input is text data, and the output is stored on the server in a formatted, parseable data format.

[0379] Step 2:

[0380] The server uses a natural language processing engine (e.g., Google Cloud Natural Language API) to analyze the received post data. The input is formatted text data, and the output is sentiment analysis results. These results include sentiment categories such as "joy," "anger," and "sadness." During the analysis, keywords are extracted from the text and analyzed based on a pre-trained sentiment model.

[0381] Step 3:

[0382] The server selects appropriate advertisements based on the results of sentiment analysis. The input is the result of sentiment analysis, and the output is data of the selected advertisements. Using a knowledge repository, it searches for advertisement information related to sentiment and retrieves appropriate advertisements through an advertising platform (e.g., Google Ads API). This process uses an algorithm that selects the best advertisement from multiple candidates based on sentiment category.

[0383] Step 4:

[0384] The server transmits selected advertising data to the terminal and displays the advertisement on the user's smartphone via an information output device. The input is advertising data, and the output is the specific advertising content displayed on the user's smartphone. On the terminal, the received advertising data is immediately displayed and presented with an appropriate layout within the application so that the user can easily see it.

[0385] Step 5:

[0386] The management system records user sentiment and ad viewing history, and uses this information to select future ads. Inputs are sentiment analysis results and ad viewing history, and output is storage in a history database. This process allows for the understanding of long-term user sentiment trends, which can be used to improve future ad display strategies. The management system continues to analyze and schedule ad displays in a timely and individualized manner until the next sentiment analysis is performed.

[0387] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0388] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0389] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0390] [Third Embodiment]

[0391] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0392] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0393] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0394] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0395] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0396] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0397] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0398] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0399] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0400] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0401] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0402] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0403] This invention provides a system that enables users to obtain qualifications or achieve specific learning goals. This system consists of multiple components, including an information processing device, an information output device, and a management device.

[0404] The server receives inquiries from users and analyzes them using natural language processing techniques. For example, if a user asks a question about grammar, the server structurally interprets the question and identifies key keywords.

[0405] Based on the analyzed information, the server consults a knowledge database to search for information that will provide the appropriate answer. The database pre-stores grammatical explanations, example sentences, and related materials. The server then generates a response based on the retrieved data, formatted in a way that is easily understandable to the user.

[0406] The terminal receives responses sent from the server and displays them to the user. The responses are displayed in text format, but can also be supplemented with audio output or visual content. Users can use this information to resolve questions and advance their learning.

[0407] Furthermore, the management device has the function of monitoring the user's learning progress and sending reminders based on a pre-set learning plan. For example, it provides users with notifications prompting them to review once a week and encouraging messages according to their progress towards achieving their plan.

[0408] As a concrete example, a user with a question about relative pronouns in English enters the question, and the server categorizes the question under the theme of "relative pronouns." The server then uses its knowledge database to prepare a text containing a basic explanation of relative pronouns and usage examples, which is delivered to the user via their terminal. Based on this information, the user can then complete short practice exercises to deepen their understanding.

[0409] In this way, users can ask questions without worrying about what others think and receive information that is optimally tailored to their individual learning style. Furthermore, continuous learning support allows them to aim for planned goal achievement.

[0410] The following describes the processing flow.

[0411] Step 1:

[0412] The user uses their device to enter a learning-related question and presses the submit button. At this point, the question data is sent from the device to the server.

[0413] Step 2:

[0414] The server analyzes the question data received from the terminal. Using a natural language processing engine, it extracts keywords and themes from the input text and understands the intent of the question.

[0415] Step 3:

[0416] The server references a knowledge database based on the analysis results. It searches the database for information related to the extracted keywords (such as grammatical explanations and example sentences) and retrieves the necessary data.

[0417] Step 4:

[0418] The server generates a response based on the acquired data. It organizes the information in a way that is easy for the user to understand, and presents it in text and visual formats.

[0419] Step 5:

[0420] The server sends the generated response to the terminal. This response may include the answer to the question, as well as supplementary information and learning hints.

[0421] Step 6:

[0422] The terminal displays the response received from the server to the user. In addition to displaying text information on the screen, it also provides a function to read the information aloud as needed.

[0423] Step 7:

[0424] Users resolve their questions and continue learning through the responses provided. If further questions arise, they can return to Step 1 and ask new questions.

[0425] Step 8:

[0426] The management device automatically records the user's learning progress and generates reminders based on the set learning plan. The generated reminders are notified to the user via the terminal at the appropriate time.

[0427] (Example 1)

[0428] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0429] Conventional learning support systems struggle to effectively analyze individual user inquiries and provide timely, appropriate responses. Furthermore, they lack automated notification features to monitor users' learning progress in real time and support planned learning.

[0430] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0431] In this invention, the server includes device means having an information processing function that receives inquiries from users and analyzes said inquiries in natural language using a generative model; device means having an output function that refers to a knowledge database based on the analysis results, generates a response for the user, and provides that response; and device means having a management function that monitors learning progress and schedules reminders in order to support the user's planned learning. This makes it possible to provide appropriate information tailored to the individual learning needs of the user and to provide continuous support for learning.

[0432] A "user" is an individual or group that uses the system's functions for the purpose of obtaining qualifications or achieving learning goals.

[0433] A "generative model" is an algorithmic model that uses machine learning techniques to perform analysis such as natural language processing.

[0434] "Natural language" refers to the language that humans normally use in everyday life, and is the type of language that can be converted into a format that can be processed by a computer.

[0435] "Information processing function" refers to a function that analyzes user input and performs calculations and data manipulation to provide appropriate output.

[0436] A "knowledge database" is a data storage system in which specific information and knowledge are systematically stored and can be searched and referenced.

[0437] An "output function" refers to a function that provides users with a response based on the analysis results, via a display device, audio output device, or similar device.

[0438] The "management function" is a feature that tracks the user's learning progress and schedules reminders and notifications based on their learning plan.

[0439] A "reminder" is a notification message sent to a user at a pre-set time to prompt them to take action.

[0440] In an embodiment of this invention, the system mainly consists of an information processing device, an information output device, and a management device. Users use a terminal connected to this system to learn and ask questions.

[0441] The server is equipped with a natural language processing engine that includes a generative AI model, and receives inquiries from users. This allows the server to analyze the input natural language and extract key keywords. This analysis process includes, for example, grammatical and semantic analysis of text-based questions.

[0442] The server then refers to its internal knowledge database based on the analysis results. This database hierarchically stores various specialized content, allowing the server to quickly retrieve relevant information. For example, if a user asks, "What is a relative pronoun?", the server extracts relevant information from the knowledge database based on the keyword "relative pronoun."

[0443] Based on the extracted information, the server generates an appropriate response. This response is formatted in natural language for easy user understanding and provided to the user as text, audio, or visual content. The terminal receives this response and displays or outputs it to the user through its screen or speaker.

[0444] The management system monitors the user's learning progress and sends reminders and notifications according to a pre-set learning plan. This function supports users in systematically progressing with their studies and efficiently achieving their goals. For example, it helps reinforce learning by sending reminders to users to review the material once a week.

[0445] As a concrete example, a user might use the prompt, "Please tell me about relative pronouns in English." Upon receiving this prompt, the server analyzes the question, searches its knowledge database to provide appropriate information, and generates and provides an answer that is easy for the user to understand. In this way, users can easily obtain information tailored to their individual learning needs.

[0446] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0447] Step 1:

[0448] The user uses a terminal to input learning and research questions in text format. This input is sent to the terminal as a prompt. The terminal receives this input and prepares it for transmission to the server. The input data is processed while maintaining its natural language format.

[0449] Step 2:

[0450] The server receives prompt messages sent from the terminal. This input data is passed to the generating AI model, which then begins analysis using a natural language processing engine. Data processing includes grammatical analysis and keyword extraction. Specifically, the sentence structure is analyzed to identify important keywords, which are then used as preparation for subsequent database lookups. The output consists of the analyzed keywords and contextual information of the sentence.

[0451] Step 3:

[0452] The server references a knowledge database based on the analyzed keywords. This database search involves data calculations to quickly retrieve relevant information and materials. For example, if the keyword "relative pronoun" is extracted, related grammatical explanations and example sentences are retrieved from the database. The output is a list containing all of this necessary information.

[0453] Step 4:

[0454] The server generates a response based on information retrieved from the database. Using a generative AI model, it formats the information into natural language text that is easy for the user to understand, and sometimes supplements the output with visual or auditory elements. Specifically, the server processes this output data into a format that is easy to generate and prepares it for transmission to the terminal.

[0455] Step 5:

[0456] The terminal receives the response sent from the server. The received data is prepared for the user to receive and displayed as text on the terminal, and output as audio or visual content as needed. The output is in a format that the user can easily understand and use. As a result, the user can obtain answers to their inquiries and proceed with learning and question resolution.

[0457] Step 6:

[0458] The management device monitors the user's learning progress, generates reminders based on a pre-configured learning plan, and sends them to the terminal according to the schedule. Specifically, it collects learning status data and identifies when notifications are needed in accordance with the plan. This is aimed at maintaining user motivation and providing output that encourages regular review and other activities. Output includes timely notifications and feedback to support the user's learning activities.

[0459] (Application Example 1)

[0460] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0461] Traditional learning support systems have faced challenges in providing accurate responses to individual user inquiries and real-time visual feedback. Furthermore, they lacked sufficient means to instantly provide information related to specific objects. Therefore, these challenges need to be addressed to provide a more effective and intuitive learning experience.

[0462] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0463] In this invention, the server includes information processing means for receiving and analyzing inquiries from users, information output means for generating and providing responses to the user based on the analysis results, and management means for monitoring the user's learning progress and scheduling notifications to support planned learning. This allows users to quickly receive information about objects in their line of sight and enjoy a more natural and intuitive learning experience.

[0464] An "information processing device" is a device that receives inquiries from users, analyzes those inquiries, and generates appropriate responses based on the analysis results.

[0465] An "information output device" is a device that provides users with a response generated based on the results of the analysis.

[0466] A "management device" is a device that monitors the user's learning progress and schedules notifications to support planned learning.

[0467] An "eye-tracking device" is a device that tracks a learner's gaze and provides visual feedback in real time.

[0468] An "object recognition device" is a device that recognizes objects in real time and provides information about their general characteristics and related information.

[0469] According to this invention, the server receives a query from a user and analyzes it using natural language processing technology. The information processing device analyzes the query as text data and refers to a knowledge database to generate an appropriate response based on the analysis results. This database stores various information, including grammatical explanations, example sentences, and related materials. Based on the analysis results, the server generates a response in an easily understandable format for the user and provides it through an information output device.

[0470] The terminal is equipped with an eye-tracking device that tracks the user's gaze direction in real time and displays relevant information on the screen as needed. This allows the object recognition device to recognize objects in the user's line of sight and provide information related to those objects.

[0471] The server monitors learning progress through a management device and schedules notifications based on the user's learning plan. This helps users effectively progress in their learning and achieve their planned goals.

[0472] For example, if a user wants to know more about a grammatical point, the server can analyze the request, retrieve the appropriate information from an existing database, and present it to the user. The user can receive this information through a glasses-type display or audio.

[0473] An example of a prompt message based on a generative AI model is sending an instruction to the server such as, "Please provide basic information and related details about the object the user is looking at," which can then retrieve appropriate information.

[0474] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0475] Step 1:

[0476] The server receives inquiries from users. These inquiries are input as natural language data in text format. Upon receiving this data, the server analyzes the text data using natural language processing techniques. As a result of the analysis, the subject and keywords of the inquiry are extracted.

[0477] Step 2:

[0478] The server consults a knowledge database based on the subject and keywords analyzed in Step 1. During the database lookup, it searches for relevant information and collects appropriate answer candidates. The output includes answer candidates and related information.

[0479] Step 3:

[0480] The server generates a user-optimized response based on the collected answer candidates. Text generation technology is used to construct the response in a format easily understood by the user. The result of the response generation is output to the user as natural language text.

[0481] Step 4:

[0482] The terminal receives text responses sent from the server and presents them to the user visually or audibly. If an eye-tracking device is integrated into the terminal, interactive responses are enabled by displaying relevant information based on the user's gaze. Output is the provision of information to the user as a display or audio output.

[0483] Step 5:

[0484] The server uses a management device to monitor the user's learning progress and schedule notifications according to the learning plan. These notifications include, for example, reminders and recommendations for the next learning activity. Operation logs and progress data are analyzed to develop an appropriate notification plan. The output is provided periodically in the form of notifications to the user.

[0485] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0486] The system implemented by this invention can understand the user's emotions and provide an appropriate response based on those emotions. The system consists mainly of an information processing device, an emotion engine, an information output device, and a management device, all of which work together in coordination.

[0487] The server receives inquiries from users through terminals. Inquiries are entered in text format and analyzed by an emotion engine. As part of the analysis, a process is included to detect emotions from the user's text. For example, natural language processing and emotion analysis algorithms are used to recognize whether the user is expressing emotions such as "anxiety" or "confusion."

[0488] The server retrieves relevant information from a knowledge database based on the emotions recognized by the emotion engine and the content of the received query. This information is then organized in an appropriate format according to the user's situation and provided as a response.

[0489] The terminal displays a response to the user that is based on the emotions sent from the server. This response is tailored to the user's emotions and may include more polite language and encouraging comments. For example, if the user expresses anxiety about difficult grammar, the server will send a detailed explanation along with an encouraging message to boost their motivation.

[0490] Furthermore, the management device has the function of linking the user's emotions with their learning progress and scheduling notifications. When a user is experiencing stress during their studies, the management device sends hints to reduce stress and notifications to encourage refreshment to the device, supporting an effective learning environment.

[0491] For example, if a user sends an inquiry stating, "My recent test results were poor," the emotion engine detects emotions such as "disappointment" or "anxiety." Based on this, the server suggests ways to review the test and also provides encouraging messages such as, "Let's try harder next time; there's room for improvement." In this way, the system responds to the user's emotions, enabling more personalized learning support.

[0492] The following describes the processing flow.

[0493] Step 1:

[0494] Users access the system through their devices and submit learning-related questions and their current status as text input. During this process, expressions containing emotions may be naturally included.

[0495] Step 2:

[0496] The terminal sends the text entered by the user to the server. The data sent includes information such as the user ID and the time of input.

[0497] Step 3:

[0498] The server passes the received text data to the emotion engine. This engine uses natural language processing techniques to analyze the emotions contained in the text and recognize specific emotional states (e.g., "stress," "anxiety," "satisfaction," etc.).

[0499] Step 4:

[0500] The server combines the analysis results from the emotion engine with the user's question to generate a query to the knowledge database. This allows it to search for information related to the question.

[0501] Step 5:

[0502] The server generates a response to the user based on information retrieved from the database. If the user is in a specific emotional state, the response is appropriately adjusted, for example, by adding comments expressing "encouragement" or "empathy."

[0503] Step 6:

[0504] The server sends the generated response data to the terminal. The response includes a message tailored based on information obtained from the knowledge database and emotional responses.

[0505] Step 7:

[0506] The terminal displays the response received from the server to the user. Flexible language that responds to emotions is used to provide information that is easy for the user to understand and that is sensitive to their feelings.

[0507] Step 8:

[0508] The user reviews the displayed response and continues learning. If needed, they can enter the question again to receive further information and support.

[0509] Step 9:

[0510] The management system generates timely reminders based on the user's learning progress and emotional records. This provides regular notifications to support the user's learning plan and motivation.

[0511] (Example 2)

[0512] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0513] There is a need for a system that can understand the user's emotional state, provide personalized responses, and effectively support learning. Furthermore, a challenge is to reduce learning stress based on the user's emotions and provide an efficient learning environment.

[0514] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0515] In this invention, the server includes an information processing device means that receives an inquiry from a user, analyzes the inquiry using natural language processing technology, and detects the user's emotions; an information output device means that generates a personalized response for the user based on the emotion analysis results and provides the response; and a management device means that monitors the user's learning progress and emotional state and schedules notifications for planned learning support. This enables efficient learning support that takes the user's emotions into account and promotes stress reduction.

[0516] An "information processing device" is a device that receives inquiries from users, analyzes those inquiries using natural language processing technology, and detects the user's emotions.

[0517] "Natural language processing" is a technology that uses machines to analyze, understand, and generate responses to human language.

[0518] "Sentiment analysis" is the process of extracting and identifying a user's emotions from text data.

[0519] An "information output device" is a device that provides users with personalized responses generated based on the results of emotion analysis.

[0520] A "management device" is a device that monitors the user's learning progress and emotional state, and schedules notifications for planned learning support.

[0521] A "personalized response" is a response that is individually tailored to the user's emotional state and needs.

[0522] "Learning support" refers to the provision of various information and notifications aimed at assisting users' learning activities and improving their efficiency.

[0523] The system for implementing this invention is based on information processing technology that has the ability to understand user emotions and respond appropriately. The system mainly consists of a server, terminals, and management devices, each working in cooperation with the others.

[0524] The server receives user inquiries through the terminal. In this process, the user's inquiry, provided in text format, is processed first within the system. Specifically, the server analyzes the inquiry using natural language processing techniques and identifies the user's emotions using an emotion engine. Sentiment analysis algorithms play a crucial role in this process.

[0525] For example, if a user sends an inquiry such as, "I got a lower score than I expected on my recent exam," the server will detect emotions such as "disappointment" or "anxiety." Once an emotion is detected, the server will refer to its knowledge database to gather relevant learning support information and generate a response tailored to the user's emotions. This may include learning advice and motivational messages.

[0526] The generated response is sent to the terminal and displayed to the user. The terminal receives the message sent from the server and presents it to the user in an emotionally sensitive manner. As a result, the response is tailored to the user's feelings and has the effect of supporting learning.

[0527] Furthermore, the management system monitors the user's learning progress and emotional state, and schedules notifications to prompt refreshment as needed. This continuous support allows users to enjoy an effective learning environment.

[0528] An example of a prompt might be, "Analyze the sentiment from the text entered by the user and generate a personalized response." This prompt instructs the generative AI model to generate a response based on the user's sentiment.

[0529] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0530] Step 1:

[0531] Users enter their inquiries in text format through their terminals and send them to the system. The inquiries reflect the user's situation and questions and are subject to sentiment-based analysis. The entered text is sent to the server.

[0532] Step 2:

[0533] The server receives text sent by the user. The received text is analyzed using natural language processing techniques to determine its linguistic structure. This process extracts data on the grammar and meaning of the text. Based on this, an emotion engine is used to perform emotion analysis. An emotion analysis algorithm is applied to identify emotions such as "anxiety" and "joy." The analysis results output the user's emotional state.

[0534] Step 3:

[0535] The server searches a knowledge database based on the sentiment analysis results and retrieves appropriate information. It selects information relevant to the user's question and collects data to generate an emotion-appropriate response. This prepares an appropriate response that takes emotions into consideration.

[0536] Step 4:

[0537] The server uses a generative AI model to create personalized responses. It takes sentiment analysis results and information from a knowledge database as input. Based on this, it outputs customized messages that match the user's emotions. The generated responses include encouragement and advice and are optimized for the user's situation.

[0538] Step 5:

[0539] The device receives responses sent from the server and displays them to the user. The outputted personalized message is shown on the user's screen, and the user can see it in real time. Through this process, the user receives emotionally sensitive feedback.

[0540] Step 6:

[0541] The management system monitors and records the user's learning progress and emotional state. Based on this information, it schedules notifications as needed. Learning history and emotional data are used as input data. Output includes timely refresh notifications and learning hints, which are sent to the user via their device. This allows the user to enjoy an efficient and stress-free learning environment.

[0542] (Application Example 2)

[0543] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0544] With the advancement of modern information technology, presenting advertisements that respond immediately to consumer emotions has become a crucial challenge in the market. However, conventional advertising systems are limited to delivering ads based on general user attributes, making it difficult to present ads based on the real-time emotions of individual users. This invention aims to solve these problems and provide a more personalized advertising service.

[0545] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0546] In this invention, the server includes an information processing device means for receiving inquiries or posts from users and analyzing said inquiries or posts; an information output device means for generating and providing advertisements that are appropriate to the user's emotions based on the analysis results; and a management device means for monitoring the user's emotions and scheduling the display of emotion-based advertisements. This enables the real-time display of advertisements that take into account the user's emotional state.

[0547] An "information processing device" is a device that receives inquiries or posts from users and analyzes them.

[0548] "Analysis results" refer to the content of emotions and information obtained when an information processing device analyzes a query or post in natural language.

[0549] "Advertisements" are promotional information generated based on analysis results and provided to users through information output devices.

[0550] An "information output device" is a device that provides users with advertisements generated based on the analysis results.

[0551] A "management device" is a device that monitors users' emotions and schedules the display of emotion-based advertisements.

[0552] A "knowledge repository" is a collection of information referenced to generate advertisements, and is a database containing predetermined advertising information.

[0553] The system implementing this invention performs a series of processes to optimize advertisements based on sentiment. The server utilizes an information processing device to collect inquiries or posts from users. The information processing device analyzes the posts using a natural language processing engine to extract the user's sentiment. The Google Cloud Natural Language API can be used for this analysis.

[0554] Next, the server identifies information related to ad generation based on the sentiment analysis results and retrieves this information using a knowledge repository. Ad delivery platforms such as the Google Ads API can be used for ad generation. The ad is then generated and displayed to the user's device at the appropriate time via an information output device.

[0555] Furthermore, the management system monitors users' real-time emotional data and schedules when advertisements tailored to each user's emotions should be displayed. This enables the display of advertisements in a way that is relevant to their emotions.

[0556] For example, if a user posts "I'm feeling very stressed today," advertisements for relaxation products and services will be displayed. In this way, the system dynamically adjusts advertisements and provides personalized content to each user.

[0557] Examples of prompts to input into a generative AI model:

[0558] "Perform sentiment analysis and extract the sentiment from the following text: '{user post}'. Generate an ad that matches this sentiment."

[0559] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0560] Step 1:

[0561] The server receives posted data from the user's smartphone. This posted data is, for example, text from social media or messaging apps. The input is text data, and the output is stored on the server in a formatted, parseable data format.

[0562] Step 2:

[0563] The server uses a natural language processing engine (e.g., Google Cloud Natural Language API) to analyze the received post data. The input is formatted text data, and the output is sentiment analysis results. These results include sentiment categories such as "joy," "anger," and "sadness." During the analysis, keywords are extracted from the text and analyzed based on a pre-trained sentiment model.

[0564] Step 3:

[0565] The server selects appropriate advertisements based on the results of sentiment analysis. The input is the result of sentiment analysis, and the output is data of the selected advertisements. Using a knowledge repository, it searches for advertisement information related to sentiment and retrieves appropriate advertisements through an advertising platform (e.g., Google Ads API). This process uses an algorithm that selects the best advertisement from multiple candidates based on sentiment category.

[0566] Step 4:

[0567] The server transmits selected advertising data to the terminal and displays the advertisement on the user's smartphone via an information output device. The input is advertising data, and the output is the specific advertising content displayed on the user's smartphone. On the terminal, the received advertising data is immediately displayed and presented with an appropriate layout within the application so that the user can easily see it.

[0568] Step 5:

[0569] The management system records user sentiment and ad viewing history, and uses this information to select future ads. Inputs are sentiment analysis results and ad viewing history, and output is storage in a history database. This process allows for the understanding of long-term user sentiment trends, which can be used to improve future ad display strategies. The management system continues to analyze and schedule ad displays in a timely and individualized manner until the next sentiment analysis is performed.

[0570] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0571] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0572] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0573] [Fourth Embodiment]

[0574] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0575] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0576] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0577] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0578] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0579] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0580] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0581] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0582] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0583] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0584] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0585] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0586] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0587] This invention provides a system that enables users to obtain qualifications or achieve specific learning goals. This system consists of multiple components, including an information processing device, an information output device, and a management device.

[0588] The server receives inquiries from users and analyzes them using natural language processing techniques. For example, if a user asks a question about grammar, the server structurally interprets the question and identifies key keywords.

[0589] Based on the analyzed information, the server consults a knowledge database to search for information that will provide the appropriate answer. The database pre-stores grammatical explanations, example sentences, and related materials. The server then generates a response based on the retrieved data, formatted in a way that is easily understandable to the user.

[0590] The terminal receives responses sent from the server and displays them to the user. The responses are displayed in text format, but can also be supplemented with audio output or visual content. Users can use this information to resolve questions and advance their learning.

[0591] Furthermore, the management device has the function of monitoring the user's learning progress and sending reminders based on a pre-set learning plan. For example, it provides users with notifications prompting them to review once a week and encouraging messages according to their progress towards achieving their plan.

[0592] As a concrete example, a user with a question about relative pronouns in English enters the question, and the server categorizes the question under the theme of "relative pronouns." The server then uses its knowledge database to prepare a text containing a basic explanation of relative pronouns and usage examples, which is delivered to the user via their terminal. Based on this information, the user can then complete short practice exercises to deepen their understanding.

[0593] In this way, users can ask questions without worrying about what others think and receive information that is optimally tailored to their individual learning style. Furthermore, continuous learning support allows them to aim for planned goal achievement.

[0594] The following describes the processing flow.

[0595] Step 1:

[0596] The user uses their device to enter a learning-related question and presses the submit button. At this point, the question data is sent from the device to the server.

[0597] Step 2:

[0598] The server analyzes the question data received from the terminal. Using a natural language processing engine, it extracts keywords and themes from the input text and understands the intent of the question.

[0599] Step 3:

[0600] The server references a knowledge database based on the analysis results. It searches the database for information related to the extracted keywords (such as grammatical explanations and example sentences) and retrieves the necessary data.

[0601] Step 4:

[0602] The server generates a response based on the acquired data. It organizes the information in a way that is easy for the user to understand, and presents it in text and visual formats.

[0603] Step 5:

[0604] The server sends the generated response to the terminal. This response may include the answer to the question, as well as supplementary information and learning hints.

[0605] Step 6:

[0606] The terminal displays the response received from the server to the user. In addition to displaying text information on the screen, it also provides a function to read the information aloud as needed.

[0607] Step 7:

[0608] Users resolve their questions and continue learning through the responses provided. If further questions arise, they can return to Step 1 and ask new questions.

[0609] Step 8:

[0610] The management device automatically records the user's learning progress and generates reminders based on the set learning plan. The generated reminders are notified to the user via the terminal at the appropriate time.

[0611] (Example 1)

[0612] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0613] Conventional learning support systems struggle to effectively analyze individual user inquiries and provide timely, appropriate responses. Furthermore, they lack automated notification features to monitor users' learning progress in real time and support planned learning.

[0614] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0615] In this invention, the server includes device means having an information processing function that receives inquiries from users and analyzes said inquiries in natural language using a generative model; device means having an output function that refers to a knowledge database based on the analysis results, generates a response for the user, and provides that response; and device means having a management function that monitors learning progress and schedules reminders in order to support the user's planned learning. This makes it possible to provide appropriate information tailored to the individual learning needs of the user and to provide continuous support for learning.

[0616] A "user" is an individual or group that uses the system's functions for the purpose of obtaining qualifications or achieving learning goals.

[0617] A "generative model" is an algorithmic model that uses machine learning techniques to perform analysis such as natural language processing.

[0618] "Natural language" refers to the language that humans normally use in everyday life, and is the type of language that can be converted into a format that can be processed by a computer.

[0619] "Information processing function" refers to a function that analyzes user input and performs calculations and data manipulation to provide appropriate output.

[0620] A "knowledge database" is a data storage system in which specific information and knowledge are systematically stored and can be searched and referenced.

[0621] An "output function" refers to a function that provides users with a response based on the analysis results, via a display device, audio output device, or similar device.

[0622] The "management function" is a feature that tracks the user's learning progress and schedules reminders and notifications based on their learning plan.

[0623] A "reminder" is a notification message sent to a user at a pre-set time to prompt them to take action.

[0624] In an embodiment of this invention, the system mainly consists of an information processing device, an information output device, and a management device. Users use a terminal connected to this system to learn and ask questions.

[0625] The server is equipped with a natural language processing engine that includes a generative AI model, and receives inquiries from users. This allows the server to analyze the input natural language and extract key keywords. This analysis process includes, for example, grammatical and semantic analysis of text-based questions.

[0626] The server then refers to its internal knowledge database based on the analysis results. This database hierarchically stores various specialized content, allowing the server to quickly retrieve relevant information. For example, if a user asks, "What is a relative pronoun?", the server extracts relevant information from the knowledge database based on the keyword "relative pronoun."

[0627] Based on the extracted information, the server generates an appropriate response. This response is formatted in natural language for easy user understanding and provided to the user as text, audio, or visual content. The terminal receives this response and displays or outputs it to the user through its screen or speaker.

[0628] The management system monitors the user's learning progress and sends reminders and notifications according to a pre-set learning plan. This function supports users in systematically progressing with their studies and efficiently achieving their goals. For example, it helps reinforce learning by sending reminders to users to review the material once a week.

[0629] As a concrete example, a user might use the prompt, "Please tell me about relative pronouns in English." Upon receiving this prompt, the server analyzes the question, searches its knowledge database to provide appropriate information, and generates and provides an answer that is easy for the user to understand. In this way, users can easily obtain information tailored to their individual learning needs.

[0630] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0631] Step 1:

[0632] The user uses a terminal to input learning and research questions in text format. This input is sent to the terminal as a prompt. The terminal receives this input and prepares it for transmission to the server. The input data is processed while maintaining its natural language format.

[0633] Step 2:

[0634] The server receives prompt messages sent from the terminal. This input data is passed to the generating AI model, which then begins analysis using a natural language processing engine. Data processing includes grammatical analysis and keyword extraction. Specifically, the sentence structure is analyzed to identify important keywords, which are then used as preparation for subsequent database lookups. The output consists of the analyzed keywords and contextual information of the sentence.

[0635] Step 3:

[0636] The server references a knowledge database based on the analyzed keywords. This database search involves data calculations to quickly retrieve relevant information and materials. For example, if the keyword "relative pronoun" is extracted, related grammatical explanations and example sentences are retrieved from the database. The output is a list containing all of this necessary information.

[0637] Step 4:

[0638] The server generates a response based on information retrieved from the database. Using a generative AI model, it formats the information into natural language text that is easy for the user to understand, and sometimes supplements the output with visual or auditory elements. Specifically, the server processes this output data into a format that is easy to generate and prepares it for transmission to the terminal.

[0639] Step 5:

[0640] The terminal receives the response sent from the server. The received data is prepared for the user to receive and displayed as text on the terminal, and output as audio or visual content as needed. The output is in a format that the user can easily understand and use. As a result, the user can obtain answers to their inquiries and proceed with learning and question resolution.

[0641] Step 6:

[0642] The management device monitors the user's learning progress, generates reminders based on a pre-configured learning plan, and sends them to the terminal according to the schedule. Specifically, it collects learning status data and identifies when notifications are needed in accordance with the plan. This is aimed at maintaining user motivation and providing output that encourages regular review and other activities. Output includes timely notifications and feedback to support the user's learning activities.

[0643] (Application Example 1)

[0644] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0645] Traditional learning support systems have faced challenges in providing accurate responses to individual user inquiries and real-time visual feedback. Furthermore, they lacked sufficient means to instantly provide information related to specific objects. Therefore, these challenges need to be addressed to provide a more effective and intuitive learning experience.

[0646] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0647] In this invention, the server includes information processing means for receiving and analyzing inquiries from users, information output means for generating and providing responses to the user based on the analysis results, and management means for monitoring the user's learning progress and scheduling notifications to support planned learning. This allows users to quickly receive information about objects in their line of sight and enjoy a more natural and intuitive learning experience.

[0648] An "information processing device" is a device that receives inquiries from users, analyzes those inquiries, and generates appropriate responses based on the analysis results.

[0649] An "information output device" is a device that provides users with a response generated based on the results of the analysis.

[0650] A "management device" is a device that monitors the user's learning progress and schedules notifications to support planned learning.

[0651] An "eye-tracking device" is a device that tracks a learner's gaze and provides visual feedback in real time.

[0652] An "object recognition device" is a device that recognizes objects in real time and provides information about their general characteristics and related information.

[0653] According to this invention, the server receives a query from a user and analyzes it using natural language processing technology. The information processing device analyzes the query as text data and refers to a knowledge database to generate an appropriate response based on the analysis results. This database stores various information, including grammatical explanations, example sentences, and related materials. Based on the analysis results, the server generates a response in an easily understandable format for the user and provides it through an information output device.

[0654] The terminal is equipped with an eye-tracking device that tracks the user's gaze direction in real time and displays relevant information on the screen as needed. This allows the object recognition device to recognize objects in the user's line of sight and provide information related to those objects.

[0655] The server monitors learning progress through a management device and schedules notifications based on the user's learning plan. This helps users effectively progress in their learning and achieve their planned goals.

[0656] For example, if a user wants to know more about a grammatical point, the server can analyze the request, retrieve the appropriate information from an existing database, and present it to the user. The user can receive this information through a glasses-type display or audio.

[0657] An example of a prompt message based on a generative AI model is sending an instruction to the server such as, "Please provide basic information and related details about the object the user is looking at," which can then retrieve appropriate information.

[0658] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0659] Step 1:

[0660] The server receives inquiries from users. These inquiries are input as natural language data in text format. Upon receiving this data, the server analyzes the text data using natural language processing techniques. As a result of the analysis, the subject and keywords of the inquiry are extracted.

[0661] Step 2:

[0662] The server consults a knowledge database based on the subject and keywords analyzed in Step 1. During the database lookup, it searches for relevant information and collects appropriate answer candidates. The output includes answer candidates and related information.

[0663] Step 3:

[0664] The server generates a user-optimized response based on the collected answer candidates. Text generation technology is used to construct the response in a format easily understood by the user. The result of the response generation is output to the user as natural language text.

[0665] Step 4:

[0666] The terminal receives text responses sent from the server and presents them to the user visually or audibly. If an eye-tracking device is integrated into the terminal, interactive responses are enabled by displaying relevant information based on the user's gaze. Output is the provision of information to the user as a display or audio output.

[0667] Step 5:

[0668] The server uses a management device to monitor the user's learning progress and schedule notifications according to the learning plan. These notifications include, for example, reminders and recommendations for the next learning activity. Operation logs and progress data are analyzed to develop an appropriate notification plan. The output is provided periodically in the form of notifications to the user.

[0669] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0670] The system implemented by this invention can understand the user's emotions and provide an appropriate response based on those emotions. The system consists mainly of an information processing device, an emotion engine, an information output device, and a management device, all of which work together in coordination.

[0671] The server receives inquiries from users through terminals. Inquiries are entered in text format and analyzed by an emotion engine. As part of the analysis, a process is included to detect emotions from the user's text. For example, natural language processing and emotion analysis algorithms are used to recognize whether the user is expressing emotions such as "anxiety" or "confusion."

[0672] The server retrieves relevant information from a knowledge database based on the emotions recognized by the emotion engine and the content of the received query. This information is then organized in an appropriate format according to the user's situation and provided as a response.

[0673] The terminal displays a response to the user that is based on the emotions sent from the server. This response is tailored to the user's emotions and may include more polite language and encouraging comments. For example, if the user expresses anxiety about difficult grammar, the server will send a detailed explanation along with an encouraging message to boost their motivation.

[0674] Furthermore, the management device has the function of linking the user's emotions with their learning progress and scheduling notifications. When a user is experiencing stress during their studies, the management device sends hints to reduce stress and notifications to encourage refreshment to the device, supporting an effective learning environment.

[0675] For example, if a user sends an inquiry stating, "My recent test results were poor," the emotion engine detects emotions such as "disappointment" or "anxiety." Based on this, the server suggests ways to review the test and also provides encouraging messages such as, "Let's try harder next time; there's room for improvement." In this way, the system responds to the user's emotions, enabling more personalized learning support.

[0676] The following describes the processing flow.

[0677] Step 1:

[0678] Users access the system through their devices and submit learning-related questions and their current status as text input. During this process, expressions containing emotions may be naturally included.

[0679] Step 2:

[0680] The terminal sends the text entered by the user to the server. The data sent includes information such as the user ID and the time of input.

[0681] Step 3:

[0682] The server passes the received text data to the emotion engine. This engine uses natural language processing techniques to analyze the emotions contained in the text and recognize specific emotional states (e.g., "stress," "anxiety," "satisfaction," etc.).

[0683] Step 4:

[0684] The server combines the analysis results from the emotion engine with the user's question to generate a query to the knowledge database. This allows it to search for information related to the question.

[0685] Step 5:

[0686] The server generates a response to the user based on information retrieved from the database. If the user is in a specific emotional state, the response is appropriately adjusted, for example, by adding comments expressing "encouragement" or "empathy."

[0687] Step 6:

[0688] The server sends the generated response data to the terminal. The response includes a message tailored based on information obtained from the knowledge database and emotional responses.

[0689] Step 7:

[0690] The terminal displays the response received from the server to the user. Flexible language that responds to emotions is used to provide information that is easy for the user to understand and that is sensitive to their feelings.

[0691] Step 8:

[0692] The user reviews the displayed response and continues learning. If needed, they can enter the question again to receive further information and support.

[0693] Step 9:

[0694] The management system generates timely reminders based on the user's learning progress and emotional records. This provides regular notifications to support the user's learning plan and motivation.

[0695] (Example 2)

[0696] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0697] There is a need for a system that can understand the user's emotional state, provide personalized responses, and effectively support learning. Furthermore, a challenge is to reduce learning stress based on the user's emotions and provide an efficient learning environment.

[0698] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0699] In this invention, the server includes an information processing device means that receives an inquiry from a user, analyzes the inquiry using natural language processing technology, and detects the user's emotions; an information output device means that generates a personalized response for the user based on the emotion analysis results and provides the response; and a management device means that monitors the user's learning progress and emotional state and schedules notifications for planned learning support. This enables efficient learning support that takes the user's emotions into account and promotes stress reduction.

[0700] An "information processing device" is a device that receives inquiries from users, analyzes those inquiries using natural language processing technology, and detects the user's emotions.

[0701] "Natural language processing" is a technology that uses machines to analyze, understand, and generate responses to human language.

[0702] "Sentiment analysis" is the process of extracting and identifying a user's emotions from text data.

[0703] An "information output device" is a device that provides users with personalized responses generated based on the results of emotion analysis.

[0704] A "management device" is a device that monitors the user's learning progress and emotional state, and schedules notifications for planned learning support.

[0705] A "personalized response" is a response that is individually tailored to the user's emotional state and needs.

[0706] "Learning support" refers to the provision of various information and notifications aimed at assisting users' learning activities and improving their efficiency.

[0707] The system for implementing this invention is based on information processing technology that has the ability to understand user emotions and respond appropriately. The system mainly consists of a server, terminals, and management devices, each working in cooperation with the others.

[0708] The server receives user inquiries through the terminal. In this process, the user's inquiry, provided in text format, is processed first within the system. Specifically, the server analyzes the inquiry using natural language processing techniques and identifies the user's emotions using an emotion engine. Sentiment analysis algorithms play a crucial role in this process.

[0709] For example, if a user sends an inquiry such as, "I got a lower score than I expected on my recent exam," the server will detect emotions such as "disappointment" or "anxiety." Once an emotion is detected, the server will refer to its knowledge database to gather relevant learning support information and generate a response tailored to the user's emotions. This may include learning advice and motivational messages.

[0710] The generated response is sent to the terminal and displayed to the user. The terminal receives the message sent from the server and presents it to the user in an emotionally sensitive manner. As a result, the response is tailored to the user's feelings and has the effect of supporting learning.

[0711] Furthermore, the management system monitors the user's learning progress and emotional state, and schedules notifications to prompt refreshment as needed. This continuous support allows users to enjoy an effective learning environment.

[0712] An example of a prompt might be, "Analyze the sentiment from the text entered by the user and generate a personalized response." This prompt instructs the generative AI model to generate a response based on the user's sentiment.

[0713] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0714] Step 1:

[0715] Users enter their inquiries in text format through their terminals and send them to the system. The inquiries reflect the user's situation and questions and are subject to sentiment-based analysis. The entered text is sent to the server.

[0716] Step 2:

[0717] The server receives text sent by the user. The received text is analyzed using natural language processing techniques to determine its linguistic structure. This process extracts data on the grammar and meaning of the text. Based on this, an emotion engine is used to perform emotion analysis. An emotion analysis algorithm is applied to identify emotions such as "anxiety" and "joy." The analysis results output the user's emotional state.

[0718] Step 3:

[0719] The server searches a knowledge database based on the sentiment analysis results and retrieves appropriate information. It selects information relevant to the user's question and collects data to generate an emotion-appropriate response. This prepares an appropriate response that takes emotions into consideration.

[0720] Step 4:

[0721] The server uses a generative AI model to create personalized responses. It takes sentiment analysis results and information from a knowledge database as input. Based on this, it outputs customized messages that match the user's emotions. The generated responses include encouragement and advice and are optimized for the user's situation.

[0722] Step 5:

[0723] The device receives responses sent from the server and displays them to the user. The outputted personalized message is shown on the user's screen, and the user can see it in real time. Through this process, the user receives emotionally sensitive feedback.

[0724] Step 6:

[0725] The management system monitors and records the user's learning progress and emotional state. Based on this information, it schedules notifications as needed. Learning history and emotional data are used as input data. Output includes timely refresh notifications and learning hints, which are sent to the user via their device. This allows the user to enjoy an efficient and stress-free learning environment.

[0726] (Application Example 2)

[0727] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0728] With the advancement of modern information technology, presenting advertisements that respond immediately to consumer emotions has become a crucial challenge in the market. However, conventional advertising systems are limited to delivering ads based on general user attributes, making it difficult to present ads based on the real-time emotions of individual users. This invention aims to solve these problems and provide a more personalized advertising service.

[0729] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0730] In this invention, the server includes an information processing device means for receiving inquiries or posts from users and analyzing said inquiries or posts; an information output device means for generating and providing advertisements that are appropriate to the user's emotions based on the analysis results; and a management device means for monitoring the user's emotions and scheduling the display of emotion-based advertisements. This enables the real-time display of advertisements that take into account the user's emotional state.

[0731] An "information processing device" is a device that receives inquiries or posts from users and analyzes them.

[0732] "Analysis results" refer to the content of emotions and information obtained when an information processing device analyzes a query or post in natural language.

[0733] "Advertisements" are promotional information generated based on analysis results and provided to users through information output devices.

[0734] An "information output device" is a device that provides users with advertisements generated based on the analysis results.

[0735] A "management device" is a device that monitors users' emotions and schedules the display of emotion-based advertisements.

[0736] A "knowledge repository" is a collection of information referenced to generate advertisements, and is a database containing predetermined advertising information.

[0737] The system implementing this invention performs a series of processes to optimize advertisements based on sentiment. The server utilizes an information processing device to collect inquiries or posts from users. The information processing device analyzes the posts using a natural language processing engine to extract the user's sentiment. The Google Cloud Natural Language API can be used for this analysis.

[0738] Next, the server identifies information related to ad generation based on the sentiment analysis results and retrieves this information using a knowledge repository. Ad delivery platforms such as the Google Ads API can be used for ad generation. The ad is then generated and displayed to the user's device at the appropriate time via an information output device.

[0739] Furthermore, the management system monitors users' real-time emotional data and schedules when advertisements tailored to each user's emotions should be displayed. This enables the display of advertisements in a way that is relevant to their emotions.

[0740] For example, if a user posts "I'm feeling very stressed today," advertisements for relaxation products and services will be displayed. In this way, the system dynamically adjusts advertisements and provides personalized content to each user.

[0741] Examples of prompts to input into a generative AI model:

[0742] "Perform sentiment analysis and extract the sentiment from the following text: '{user post}'. Generate an ad that matches this sentiment."

[0743] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0744] Step 1:

[0745] The server receives posted data from the user's smartphone. This posted data is, for example, text from social media or messaging apps. The input is text data, and the output is stored on the server in a formatted, parseable data format.

[0746] Step 2:

[0747] The server uses a natural language processing engine (e.g., Google Cloud Natural Language API) to analyze the received post data. The input is formatted text data, and the output is sentiment analysis results. These results include sentiment categories such as "joy," "anger," and "sadness." During the analysis, keywords are extracted from the text and analyzed based on a pre-trained sentiment model.

[0748] Step 3:

[0749] The server selects appropriate advertisements based on the results of sentiment analysis. The input is the result of sentiment analysis, and the output is data of the selected advertisements. Using a knowledge repository, it searches for advertisement information related to sentiment and retrieves appropriate advertisements through an advertising platform (e.g., Google Ads API). This process uses an algorithm that selects the best advertisement from multiple candidates based on sentiment category.

[0750] Step 4:

[0751] The server transmits selected advertising data to the terminal and displays the advertisement on the user's smartphone via an information output device. The input is advertising data, and the output is the specific advertising content displayed on the user's smartphone. On the terminal, the received advertising data is immediately displayed and presented with an appropriate layout within the application so that the user can easily see it.

[0752] Step 5:

[0753] The management system records user sentiment and ad viewing history, and uses this information to select future ads. Inputs are sentiment analysis results and ad viewing history, and output is storage in a history database. This process allows for the understanding of long-term user sentiment trends, which can be used to improve future ad display strategies. The management system continues to analyze and schedule ad displays in a timely and individualized manner until the next sentiment analysis is performed.

[0754] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0755] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0756] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0757] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0758] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0759] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0760] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0761] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0762] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0763] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0764] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0765] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0766] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0767] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0768] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0769] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0770] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0771] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0772] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0773] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0774] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0775] The following is further disclosed regarding the embodiments described above.

[0776] (Claim 1)

[0777] Information processing device means for receiving inquiries from users and analyzing those inquiries,

[0778] An information output device means for generating a response to the user based on the analysis results and providing the response,

[0779] A management device means that monitors the user's learning progress and schedules notifications to support planned learning,

[0780] A system that includes this.

[0781] (Claim 2)

[0782] The system according to claim 1, wherein the information processing device has a function to analyze inquiries received from a user in natural language.

[0783] (Claim 3)

[0784] The system according to claim 1, wherein in order to generate the response, it refers to a knowledge database and obtains predetermined information.

[0785] "Example 1"

[0786] (Claim 1)

[0787] A device having an information processing function that receives inquiries from users and analyzes those inquiries in natural language using a generative model,

[0788] A device having an output function that references a knowledge database based on analysis results, generates a response for the user, and provides that response.

[0789] To support users in their planned learning, the device has management functions to monitor learning progress and schedule reminders.

[0790] A system that includes this.

[0791] (Claim 2)

[0792] The system according to claim 1, wherein the information processing function performs natural language analysis using a generative model and extracts a predetermined answer from a knowledge database based on the analysis.

[0793] (Claim 3)

[0794] The system according to claim 1, wherein the system provides the user with an answer via visual display or audio output, and has a function to adjust the output according to the user's level of understanding.

[0795] "Application Example 1"

[0796] (Claim 1)

[0797] Information processing device means for receiving inquiries from users and analyzing those inquiries,

[0798] An information output device means for generating a response to the user based on the analysis results and providing the response,

[0799] A management device means that monitors the user's learning progress and schedules notifications to support planned learning,

[0800] An eye-tracking device means for tracking the learner's gaze and providing visual feedback in real time,

[0801] An object recognition device means for recognizing objects in real time and providing their overview and related information,

[0802] A system that includes this.

[0803] (Claim 2)

[0804] The system according to claim 1, wherein the information processing device has a function to analyze inquiries received from a user in natural language.

[0805] (Claim 3)

[0806] The system according to claim 1, wherein in order to generate the response, it refers to a knowledge database and obtains predetermined information.

[0807] "Example 2 of combining an emotion engine"

[0808] (Claim 1)

[0809] An information processing device that receives inquiries from users, analyzes those inquiries using natural language processing technology, and detects the user's emotions.

[0810] An information output device means for generating and providing a personalized response to the user based on the emotion analysis results,

[0811] A management device means that monitors the user's learning progress and emotional state and schedules notifications for planned learning support,

[0812] A system that includes this.

[0813] (Claim 2)

[0814] The system according to claim 1, wherein the information processing device has a function of collecting appropriate information and generating a response by considering the emotional state of the user and referring to knowledge data.

[0815] (Claim 3)

[0816] The system according to claim 1, wherein the management device has a function to send notifications to the user prompting them to refresh based on their emotions and learning progress.

[0817] "Application example 2 when combining with an emotional engine"

[0818] (Claim 1)

[0819] Information processing device means for receiving inquiries or posts from users and analyzing said inquiries or posts,

[0820] Based on the analysis results, an information output device means for generating and providing advertisements that correspond to the user's emotions,

[0821] A management device means that monitors the user's emotions and schedules the display of emotions-based advertisements,

[0822] A system that includes this.

[0823] (Claim 2)

[0824] The system according to claim 1, wherein the information processing device has a function to analyze inquiries or posts received from users in natural language and to detect emotions.

[0825] (Claim 3)

[0826] The system according to claim 1, which, in order to generate the advertisement, refers to a knowledge repository, obtains predetermined advertising information, and displays it based on sentiment. [Explanation of Symbols]

[0827] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Information processing device means for receiving inquiries from users and analyzing those inquiries, An information output device means for generating a response to the user based on the analysis results and providing the response, A management device means that monitors the user's learning progress and schedules notifications to support planned learning, A system that includes this.

2. The system according to claim 1, wherein the information processing device has a function to analyze inquiries received from a user in natural language.

3. The system according to claim 1, wherein in order to generate the response, it refers to a knowledge database and obtains predetermined information.

Images